IRC logs of #shogun for Monday, 2013-08-19

--- Log opened Mon Aug 19 00:00:21 2013
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has quit [Quit: Ex-Chat]00:56
shogun-buildbotbuild #439 of nightly_none is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_none/builds/43903:46
-!- foulwall` [~user@2001:da8:215:6901:99ac:8cb9:9c4e:83f2] has joined #shogun04:17
-!- foulwall` [~user@2001:da8:215:6901:99ac:8cb9:9c4e:83f2] has quit [Remote host closed the connection]04:23
-!- foulwall [~user@2001:da8:215:6901:99ac:8cb9:9c4e:83f2] has joined #shogun04:27
-!- foulwall [~user@2001:da8:215:6901:99ac:8cb9:9c4e:83f2] has quit [Remote host closed the connection]04:28
-!- lisitsyn [~lisitsyn@5-28-17-141.clients.tlt.100megabit.ru] has quit [Quit: Leaving.]05:33
-!- gsomix [~gsomix@88.200.247.36] has quit [Quit: Leaving]07:22
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun08:34
shogun-notifier-shogun: Soeren Sonnenburg :develop * e8f9ea7 / examples/undocumented/libshogun/classifier_svmlight_string_features_precomputed_kernel.cpp: https://github.com/shogun-toolbox/shogun/commit/e8f9ea73f5309f4927af26bf6b40c6ff6cfbb71008:34
shogun-notifier-shogun: ifdef svmlight example to not fail when svmlight is not available08:34
shogun-buildbotbuild #930 of rpm1 - libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/93008:44
shogun-notifier-shogun: Soeren Sonnenburg :develop * 64e4618 / CMakeLists.txt: https://github.com/shogun-toolbox/shogun/commit/64e46187b7c9a91dcf362d93dae8907dd6dacf3108:46
shogun-notifier-shogun: disable warning on cygwing08:46
shogun-buildbotbuild #440 of nightly_none is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_none/builds/44008:50
shogun-buildbotbuild #931 of rpm1 - libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/93108:53
shogun-buildbotbuild #1436 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1436  blamelist: Soeren Sonnenburg <sonne@debian.org>08:54
shogun-buildbotbuild #932 of rpm1 - libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/932  blamelist: Soeren Sonnenburg <sonne@debian.org>08:56
shogun-buildbotbuild #1467 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1467  blamelist: Soeren Sonnenburg <sonne@debian.org>08:58
shogun-buildbotbuild #933 of rpm1 - libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/93309:02
@wikingmornin'09:18
shogun-buildbotbuild #1437 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1437  blamelist: Soeren Sonnenburg <sonne@debian.org>09:20
shogun-buildbotbuild #1555 of deb3 - modular_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1555  blamelist: Soeren Sonnenburg <sonne@debian.org>09:21
shogun-buildbotbuild #1468 of bsd1 - libshogun is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1468  blamelist: Soeren Sonnenburg <sonne@debian.org>09:21
shogun-buildbotbuild #934 of rpm1 - libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/93409:26
shogun-buildbotbuild #1554 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/155409:29
shogun-notifier-shogun: Viktor Gal :develop * 13837fe / tests/unit/latent/LatentSVM_unittest.cc: https://github.com/shogun-toolbox/shogun/commit/13837fe32615d16c627e7c1be9105da73954e75509:36
shogun-notifier-shogun: Fix mocking warning in LatentSVM unit test09:36
-!- sijin [~smuxi@144.214.225.109] has quit [Ping timeout: 245 seconds]09:50
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]10:11
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun10:12
-!- lisitsyn [~lisitsin@mxs.kg.ru] has joined #shogun10:14
shogun-buildbotbuild #0 of osx1 - libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/010:23
shogun-buildbotbuild #1556 of deb3 - modular_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1556  blamelist: Viktor Gal <viktor.gal@maeth.com>10:25
shogun-buildbotbuild #1438 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1438  blamelist: Viktor Gal <viktor.gal@maeth.com>10:26
shogun-buildbotbuild #1469 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1469  blamelist: Viktor Gal <viktor.gal@maeth.com>10:28
shogun-buildbotbuild #1 of osx1 - libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/110:37
@wikingoh woah we have osx bot!!! sonney2k coool!11:03
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]11:06
-!- shogun-buildbot_ [~shogun-bu@7nn.de] has joined #shogun11:06
shogun-buildbot_build #1260 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1260  blamelist: Viktor Gal <viktor.gal@maeth.com>11:07
@wikingshogun-buildbot_: force build --branch=develop 'osx1 - libshogun'11:08
shogun-buildbot_The build has been queued, I'll give a shout when it starts11:08
shogun-buildbot_build #2 forced11:10
shogun-buildbot_I'll give a shout when the build finishes11:10
-!- shogun-buildbot_ [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]11:13
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun11:13
@wikingshogun-buildbot: force build --branch=develop 'rpm1 - libshogun'11:13
shogun-buildbotbuild forced [ETA 20m10s]11:13
shogun-buildbotI'll give a shout when the build finishes11:13
-!- thoralf [~thoralf@enki.zib.de] has joined #shogun11:23
thoralfHello everybody.11:23
shogun-buildbotHey! build rpm1 - libshogun #938 is complete: Success [build successful]11:28
shogun-buildbotBuild details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/93811:28
@wiking\o/11:30
@wikingmmm we've lost osx11:39
-!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Read error: Connection reset by peer]12:27
-!- lisitsyn [~lisitsin@mxs.kg.ru] has joined #shogun12:33
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]12:36
-!- HeikoS [~heiko@nat-167-65.internal.eduroam.ucl.ac.uk] has joined #shogun12:50
-!- mode/#shogun [+o HeikoS] by ChanServ12:50
@wikingHeikoS: back? :)12:51
@wikingbtw do we have a simple function for SGMatrix to set a column vector?12:51
@wikingi.e SGMatrix.set_column(index, SGVector)12:51
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has joined #shogun13:17
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has quit [Remote host closed the connection]13:19
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has joined #shogun13:20
@HeikoSwiking: yes, but kicked out my office so currently moving things13:25
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has quit [Remote host closed the connection]13:25
@HeikoSwiking: we should have full slicing and view support on matrices btw :) like numpy13:34
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has joined #shogun13:34
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has quit [Remote host closed the connection]13:36
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has joined #shogun13:37
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has joined #shogun13:41
-!- mode/#shogun [+o iglesiasg] by ChanServ13:42
lisitsynHeikoS: looks like somebody is reinventing eigen ;)13:47
@iglesiasglisitsyn, does it support slicing already?13:47
lisitsyniglesiasg: block13:47
@iglesiasglisitsyn, aah true13:48
@iglesiasgI have seen it in this cheatsheet for Matlab users13:48
thoralfwiking: Im sometimes getting "/usr/bin/ld: can not read symbols: File truncated" when building shogun.13:53
thoralfwiking: Seems that ld isn't finished when it tries to build the examples or tests.13:53
@wikingHeikoS: yes indeed13:54
thoralfwiking: Does not happen all the time, but when setting high number with -j.13:54
@wikingthoralf: mmmm i wonder... this should be handled by cmake13:54
@wikingthoralf: as i added a dependency to libshogun for building examples and unit tests13:54
@wikingbtw is Stack thread safe ? :)13:55
thoralfwiking: Strange.13:55
-!- gsomix [~gsomix@178.45.32.127] has joined #shogun13:56
@wikingthoralf: yeah i think this is something do with cmake unfortunately13:56
@wikingthoralf: it should wait until the building of libshogun is finished13:56
thoralfwiking: But it works when re-running make in the build dir, so the hotfix is easy. ;)13:57
thoralfwiking: What I did before: I changed a class, ran "make" in a clean build dir, but c++ found out there is a (trivial) syntax error.  I fixed it without deleting the build dir an reran make, then the ld error appeared.  Running make a second time fixed it.13:59
thoralfwiking: Maybe that helps?13:59
@wikingwhere should we implement these (or where should it be placed in the shogun struct): http://www.scholarpedia.org/article/Ensemble_learning#Ensemble_combination_rules14:00
@wikingHeikoS: lisitsyn iglesiasg ^14:00
@wikingthoralf: no idea :(14:00
@iglesiasgwiking, classifier?14:02
@iglesiasgwiking, LPBoost is there at least14:03
thoralfwiking: Oh, it apparently happened in the unit tests: make[1]: *** [tests/unit/CMakeFiles/shogun-unit-test.dir/all] Error 214:03
@wikingiglesiasg: well it'd really just a class that takes a set of labels or a matrix and runs the voting on it14:03
@wikingi mean it wouldn't be actually a classifier :)14:04
@wikingin the classical sense14:04
thoralfwiking: It failed shortly after printing that it now links: libgtest_main.a, libgmock_main.a, libgtest.a, libgmock.a (and some other tests)14:04
thoralfwiking: Some dependencies between them?14:04
@wikingthoralf: tests/unit/cmakelists.txt14:05
@wikingadd_dependencies(shogun-unit-test gmock gtest)14:05
@wikingtarget_link_libraries(shogun-unit-test shogun gmock gtest)14:05
@wikingafaik target_link_libraries does define a dependency14:05
@wikingbut try adding shogun to add_dependencies as well14:05
thoralfwiking: FYI: Digging in the make output I found that "Linking CXX executable shogun-unit-test" never returned.  And that make tried to link gmock/gtest *after* shogun-unit-test.14:17
thoralfwiking: libshogun has already been finished then.14:18
lisitsynwiking: ensemble! :)14:18
@wikinglisitsyn: okey! :) currently BaggingMachine is about to be added to machine... should we change that as well/ :P14:19
lisitsynbegging machine14:22
lisitsyn:D14:22
@wiking:>14:22
@wikingindeed14:22
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has quit [Ping timeout: 260 seconds]14:31
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has joined #shogun14:51
-!- foulwall [~user@2001:da8:215:6901:f1fc:f038:aa50:32bc] has quit [Ping timeout: 245 seconds]15:00
shogun-buildbotbuild #4 of osx1 - libshogun is complete: Failure [failed test shell]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/415:17
@wikingyey almost :)15:19
@wikingsonney2k: what is the osx version on that machine 10.8 or 10.7?15:19
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has quit [Quit: Ex-Chat]15:22
@HeikoSlisitsyn, wiking eigen does not do proper slicing and views15:32
@HeikoSits all copies and writing15:33
@HeikoSbut not views on the same memory like in numpy15:33
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun15:34
shogun-notifier-shogun: Roman Votyakov :develop * 9e829cb / src/shogun/machine/gp/LaplacianInferenceMethod.cpp: https://github.com/shogun-toolbox/shogun/commit/9e829cb4fcc95101a7e5168893dc34c3b3760bae15:34
shogun-notifier-shogun: compute covariance matrix more efficiently15:34
shogun-notifier-shogun: Heiko Strathmann :develop * bb39545 / src/shogun/machine/gp/LaplacianInferenceMethod.cpp: https://github.com/shogun-toolbox/shogun/commit/bb39545f4c1d93b1648c301ee0ca7e86ac14745c15:34
shogun-notifier-shogun: Merge pull request #1440 from votjakovr/feature/gp_refactoring15:34
shogun-notifier-shogun:15:34
shogun-notifier-shogun: Compute covariance matrix more efficiently15:34
lisitsynHeikoS: ?? block doesn't copy things15:35
lisitsynA.block(1,1,5,5).array() = 3;15:35
@HeikoSlisitsyn: yes that is fine15:35
@HeikoSbut A=B(3:4,5:7) doesnt work15:35
@HeikoSsuch that modifying A changes B15:35
lisitsynwhy?15:36
lisitsynA.noalign() = B.block(3,5,1,2);15:36
lisitsynsth like that15:36
@HeikoSbut in my case, A and B point to the same memory15:37
@HeikoSjust different elements15:37
@HeikoSlisitsyn: but doesnt really matter for now, and also eigen is a better place for this than shogun15:37
thoralfwiking: I built shogun with -DENABLE_TESTING=ON, but when calling "make test" I'm getting warnings for each example.15:38
thoralfCould not find executable ./shogun-github/build/examples/undocumented/libshogun/so_factorgraph15:39
thoralfAnd "Looked in the following places:" followed by a long list of directories15:39
thoralfwiking: My cmake call was: cmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_TESTING=ON -DTRACE_MEMORY_ALLOCS=OFF ..15:39
thoralfwiking: Oh.  I was typing "make shogun" instead of "make all" - give me a second, trying this first.15:41
shogun-buildbotbuild #1830 of deb1 - libshogun is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb1%20-%20libshogun/builds/1830  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>15:41
thoralfwiking: Sorry.  Works now. ;)15:42
thoralfHeikoS: I fixed one of your unit tests (among others): https://github.com/shogun-toolbox/shogun/pull/144315:58
@HeikoSthoralf: cool, thanks!16:00
@HeikoSjust waiting for travis to give ok16:00
thoralfHeikoS: My sanity checks are breaking tests for interfaces I didn't test locally.  Grep for "matrix features" in the build output.16:24
shogun-buildbotbuild #1557 of deb3 - modular_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1557  blamelist: Roman Votyakov <votjakovr@gmail.com>16:24
shogun-buildbotbuild #1439 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1439  blamelist: Roman Votyakov <votjakovr@gmail.com>16:25
shogun-buildbotbuild #1470 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1470  blamelist: Roman Votyakov <votjakovr@gmail.com>16:28
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun16:29
@sonney2kthoralf, the thing with the bias term and liblinear is that it used to use a w vector of size dim+116:32
thoralfsonney2k: I know.16:32
@sonney2kthoralf, when the bias is enabled - so one dim in w is b16:32
@sonney2kthoralf, however we changed all of liblinear to explicitly work with a separate b16:33
@sonney2kwell we == I16:33
thoralfsonney2k: Anyway, there is an off-by-one in streaming/sparse_features which only happens when learning without bias.16:33
thoralfsonney2k: When learning with bias, then the +1 saves you from segfaults. ;)16:33
@sonney2kwiking, 10.816:34
shogun-buildbotbuild #5 of osx1 - libshogun is complete: Failure [failed test shell]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/5  blamelist: Roman Votyakov <votjakovr@gmail.com>16:36
thoralfsonney2k: liblinear allocates weights with dim/dim+1 depending on the bias setting.  When computing dense_dot() with streaming/sparse_features returns, then sparse_features telling dimensionality is $n$, but the max feature index is also $n$, which leads dense_dot to read excess w's last element.16:37
@wikingsonney2k: ok i'll give u a link for the dmg16:37
thoralfwiking: I cannot run tests for python_modular on my machine.  It's always complaining about "ImportError: No module named shogun.XXX"16:54
thoralfwiking: I'm not supposed to "make install" first?16:54
@wikingthoralf: yes16:57
@wikingthoralf: as .py files are not populated unders <pythonlibpath>/shogun/...16:57
thoralfwiking: Can I set the install path to somewhere relative to build/? ;)16:58
@wikingthoralf: -DCMAKE_INSTALL_PREFIX16:59
@wikingand you put it where u set that whatever you want it to be...16:59
@wikingrtfm cmake16:59
-!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]17:05
shogun-buildbotbuild #1261 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1261  blamelist: Roman Votyakov <votjakovr@gmail.com>17:06
thoralfwiking: Thanks for command line switch and sort for not reading 16k lines of cmake manpage at once.  I think someone should update the READMEs - some shortcuts of how to use shogun.17:19
thoralfwiking: Otherwise you eventually will kill us... ;)17:19
thoralfs/sort/sorry/17:20
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has joined #shogun17:30
-!- mode/#shogun [+o iglesiasg] by ChanServ17:31
thoralfThe example python_modular/features_io_modular.py (line 25) breaks because something is wrong with examples/undocumented/python_modular/fm_train_sparsereal.bin, but I can't tell what exactly.17:31
@wikingthoralf: there's a cmake faq ;)17:31
thoralfIt's loaded from SparseRealFeatures(fm_train_real.dat) and then saved back to disk.17:31
thoralfwiking: Grr.17:34
thoralfwiking: It's really too much pain for just fixing a bug and contributing a patch.17:35
thoralfI know it's not your fault, but it's still too complicated. ;)17:35
@HeikoSthoralf: currently using virutalenv17:49
@HeikoSthats soo cool17:49
thoralfHeikoS: Yeah, it's great.17:49
@HeikoSthoralf: best thing is the freeze option17:49
@HeikoSso I can just install my stuff along the cluster17:49
thoralfHeikoS: What's that?17:49
thoralfOh.17:49
thoralfReally?17:50
@HeikoSyou can save the installed packages17:50
@HeikoSand then you get a text file which you can use on other machines to reproduce the env17:50
thoralfwiking: I need some assistance.  I'm trying to debug run a python_modular example and get a (c++) stacktrace of what happens there.17:51
thoralfwiking: Usually, I'd just run "gdb python script.py"17:51
thoralfwiking: I see there's something in the ctest file in build/examples/undocumented/python_modular, but I can't get it running17:52
thoralfHeikoS: I'm giving up, because I can't tell how to fix the python examples.  I'll send another patch without assertions soon.18:00
@HeikoSthoralf: ok18:00
@HeikoSthoralf: I know debugging python is a pain18:00
@HeikoSwhat you can do is to run it with valgrind then you at least get the shogun place where things fail18:00
thoralfwiking: It's running, but I don't see a backtrace because python exists normally.18:00
thoralfHeikoS: valgrind won't care about failing "REQUIRE"s ;)18:01
@HeikoSah18:01
@HeikoSsorry I though something else was wrong18:01
-!- thoralf [~thoralf@enki.zib.de] has quit [Quit: Konversation terminated!]18:17
-!- thoralf [~thoralf@enki.zib.de] has joined #shogun18:17
@HeikoSwiking: around?18:33
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]18:34
-!- foulwall [~user@2001:da8:215:c252:b1e8:dc7c:8bc4:a5da] has joined #shogun18:39
@wikingyes yes18:41
@HeikoSwiking: I want  to do the following18:42
@HeikoScluster here has many computers with different architectures18:42
@HeikoSand I want to use shogun python from all of them18:43
@HeikoSso now I do this virtualenv stuff18:43
thoralfSomething is really wrong with python_modular.18:43
@HeikoSand compile shogun against it18:43
thoralfIt just started 20000 threads.18:43
thoralf(And growing)18:43
@HeikoSwiking: how would you do that?18:43
@HeikoSwiking: I just wrote a script that installs python locally and then got clones shogun and compiles18:43
@HeikoSwiking: but there is a subtlety: all machines share the same FS, but machines might have different architectures18:44
@HeikoSso each job will have to do/update this local installation on start18:44
@HeikoSany idea how to best do this?18:44
@wikingHeikoS: ?18:44
@HeikoSeven worse, there might be multiple jobs on the same machine18:44
@wikingHeikoS: so they share the repo?18:44
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has quit [Quit: Leaving.]18:45
@HeikoSwiking: what?18:45
@wikingHeikoS: share fs -> share the cloned repo of shogun u use for building?18:45
@HeikoSwiking: yes but that is not good18:45
@HeikoSthere should be one thing per job18:45
thoralfHeikoS: -DCMAKE_INSTALL_PREFIX18:45
@HeikoSso every job starts with building shogun18:45
thoralfHeikoS: Solved my problem. :)18:45
thoralfHeikoS: Just install locally18:46
@HeikoSthoralf: so depending on the job id the prefix changes you mean?18:46
@wikingHeikoS: install it to a specific dir....18:46
thoralfHeikoS: Yes,18:46
thoralfHeikoS: Same problem here.18:46
@HeikoSthoralf: do you use this distributed stuff with shogun?18:46
thoralfHeikoS: Got harded with cmake, but is possible. ;)18:46
thoralfHeikoS: What do you mean?18:47
@HeikoSit would be cool to offer a script for this18:47
@wikingthoralf: harded?18:47
@HeikoSthoralf, wiking it would be good to have a script that locally installs shogun to a dir, including a local python install18:47
@HeikoSwith virtualenv and pip18:47
thoralfs/harded/harder/18:47
@HeikoSso that I just need to give a path and the rest is done automagically18:47
@wikingthoralf: how?18:48
@wikingthoralf: -DCMAKE_INSTALL_PREFIX18:48
@wikingi mean18:48
thoralfPYTHON=/usr/bin/python PYTHONPATH=xxx/python2.7/dist-packages:. LD_LIBRARY_PATH=xxx/lib:. python script18:48
@wikingHeikoS: -DCMAKE_INSTALL_PREFIX=18:48
@HeikoSwiking: ok, I will play a bit and share the script then18:49
@wikingHeikoS: that'll install u the shogun stuff whereever you want it to18:49
@HeikoSwiking: yeah but to use python there is more involved18:49
@HeikoSif I dont have root rights18:49
@HeikoSwhich is why I want to use virtualenv18:49
@wikingHeikoS: ?18:49
@wikingi dont get it18:49
@wikingthe whole shogun thing will be installed under18:49
@wiking<prefix>18:49
@HeikoSwiking: yes, but now imagine python is not installed18:50
@wiking<prefix>/lib <prefix>/include <prefix>/lib/python...18:50
@wikingHeikoS: install it to <prefix>18:50
@HeikoSwiking: exact, and now there are multiple architectures for the same FS18:50
@wikingHeikoS: and? u create different <prefix> for different arch18:50
@HeikoSwiking: yes18:51
@HeikoSthats what I want18:51
@HeikoSwith 1 click ;)18:51
@wikingHeikoS: what do u mean 1 click?18:51
@HeikoSwiking: a one liner18:51
@wikingHeikoS: create a script for it and that's all18:51
@HeikoSwiking: yes thats what I want, but not that experienced with this stuff18:51
@wikingi mean seriously it's not shogun's job to do this18:51
@HeikoSwiking: you are right18:51
@HeikoSbut still I want it, dont want to add it to shogun, just asking for advice18:52
@wikingah i see18:52
@wikingsorry then i misunderstood18:52
@HeikoSwiking: could you help me a bit with that?18:53
@HeikoSIll share a google doc for live editing18:53
@HeikoSsent18:53
@wikingHeikoS: better share a gist ;)18:53
@HeikoSwiking: yeah once its more mature18:54
@HeikoSbecause now I want to edit it a lot18:54
thoralfAnyone knows what this tester.py does?19:03
@HeikoSthoralf: yes19:03
@HeikoSit runs the integration tests19:03
@HeikoSwhich means that it runs all python examples, and compares their output to the stored file19:03
thoralfHeikoS: It starts 1000 threads per second... ;)19:03
@HeikoSthoralf: really? thats not nice :)19:04
thoralfHeikoS: My system is unresponsible for about 5-10 seconds.19:04
thoralfNo mouse movement.19:04
@HeikoSthoralf: oh man, thats not good19:04
@HeikoSany idea why it does that?19:04
thoralfNo:(19:04
@HeikoSthoralf: it didnt before (on my machine=)19:05
thoralflibshogun-evaluation_cross_validation_locked_comparison starts about 10000 threads at once (?)19:06
thoralflibshogun-modelselection_grid_search_kernel started 19000 threads19:08
@HeikoSthoralf: did someone parallelize the cross-validation maybe?19:08
thoralfSomething is wrong here.19:08
@HeikoSbecause these examples did not do that when I wrote them19:08
@HeikoSall 1 threaded19:08
thoralfI only have 8 cores on my workstation... I would expect that nobody tries to start more than 16 at once. ;)19:09
thoralfHeikoS: Btw. another PR which should be merged before 1443, because it fixes the tests: https://github.com/shogun-toolbox/shogun/pull/144419:10
@HeikoSthoralf: why is this needed?19:11
@HeikoSthe maximum?19:11
thoralfHeikoS: Because the overhead is bigger than running 1 thread per core :)19:11
@HeikoSok, waiting fo travis19:12
@HeikoSoh it did not trigger yet19:12
shogun-buildbotbuild #6 of osx1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/619:15
thoralfwiking: I happened again that cmake started building gtest/gmock while Linking CXX executable shogun-unit-test19:28
@wikingthoralf: /j #cmake19:29
@wikingthoralf: and start asking around19:29
thoralfwiking: Sorry, I can't start debugging something else yet.  I just you might be interested in bug reports related to cmake.19:31
@wikingthoralf: this kind of stuff i cannot help with as they are really cmake related bugs19:33
@wikingso you are better of talking with cmake developers19:33
thoralfwiking: I understand.  But in this case I won't do anything because this exceeds what I can do in my time.19:34
thoralfwiking: I'd really like to.19:34
-!- gsomix_ [~gsomix@178.45.77.251] has joined #shogun19:38
-!- foulwall [~user@2001:da8:215:c252:b1e8:dc7c:8bc4:a5da] has quit [Ping timeout: 245 seconds]19:38
@sonney2kwiking, so what is missing wrt cmake now - static interfaces, detection of ruby narray and well all the failing tests of course then we are back19:40
@sonney2kwiking, which of these things can you still do?19:40
-!- gsomix [~gsomix@178.45.32.127] has quit [Ping timeout: 240 seconds]19:40
@sonney2kwiking, ohh and any news on the video upload?19:42
-!- foulwall [~user@2001:da8:215:c252:b1e8:dc7c:8bc4:a5da] has joined #shogun19:45
@sonney2kwiking, any ideas about http://shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1261/steps/test/logs/stdio ?19:47
@wikingsonney2k: narray i'll check becuase that's really weird... but i'll do it19:47
@sonney2kHeikoS, looks like you broke the tests? http://shogun-toolbox.org/buildbot/builders/deb1%20-%20libshogun/builds/1830/steps/compile/logs/stdio19:48
@sonney2kHeikoS, MMDKernelSelectionMedian_unittest.cc19:48
@wikingsonney2k: for cygwin maybe a make install would help... but i'm not so sure, basically it just cannot find the shared lib, but i'll try setting an env variable or something19:48
@HeikoSsonney2k: I did not even commit anything19:49
@sonney2kwiking, Re cygwin didn't this work?19:49
@wikingsonney2k: i never seen cygwin running examples...19:49
@HeikoSwiking, sonney2k is there a way to statically link certain parts of shogun?19:50
@wikingsonney2k: http://shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/90019:50
@sonney2kCMakeFiles/shogun-unit-test.dir/statistics/MMDKernelSelectionMedian_unittest.cc.o: In function `CmpHelperEQ<double, double>':19:50
@sonney2k/usr/include/gtest/gtest.h:1328: undefined reference to `testing::internal::EqFailure(char const*, char const*, testing::internal::String const&, testing::internal::String const&, bool)'19:50
@sonney2kso it is w/ linking already19:50
@wikingsonney2k: this is like the old ./configure stuff... it never ran examples19:50
@iglesiasgtime for the weekly report!19:50
@HeikoSsonney2k: yes I saw that, but no idea whats the problem there19:50
@sonney2kwiking, seems like LD_LIBRARY_PATH can be set withing cygwin19:51
@sonney2kwiking, so that would fix it19:51
@sonney2kHeikoS, no19:51
@wikingsonney2k: i'll do that soonish19:51
@sonney2kwiking, and whats up with the workshop videos?19:52
@wikingsonney2k: b/w problem here19:52
@sonney2kwiking, so now what?19:52
@wikingsonney2k: i'll be on a good net this week19:52
@wikingi'll try to upload as mcuh as i can then19:53
@sonney2kwiking, ok19:53
@wikingbtw: any ideas how i can find fast unique values in an SGVector?19:53
@HeikoSwiking: best you can do is linear I guess19:53
@wikingHeikoS: and creating a histogram of an SGVector elements? i mean w/o using some funky std::map shit?19:54
@HeikoSwiking: why not using map?19:55
@wikingHeikoS: well i didn't want it19:55
@wikingbut seems i have to19:55
@HeikoSwiking: best way19:56
@HeikoSand almost no effort19:56
@HeikoShaha use boost ;)19:56
@wikingHeikoS: heheh it's going to be part of shogun19:56
@wiking;)19:56
@HeikoSor add a unique method to vector then its also easy19:56
foulwallsonney2k , I've checked applications/arts,  and got error "[ERROR] ALPHABET too small to contain all symbols in histogram", don't know why19:57
thoralfwiking: unique values?  What about hashing?19:58
@wikingthoralf: std::map will then do that for me ;P19:58
thoralfwiking: Sorry, I was lagging.  Python tests are still sucking :(19:59
thoralfsonney2k: Any idea?  I have 3 tests which are dying after generating tons of threads: integration-python_modular-tester, libshogun-modelselection_grid_search_kernel, python_modular-modelselection_grid_search_libsvr_modular20:00
@HeikoSwiking: what do I do if my installed swig version is 1.320:07
@HeikoSthoralf, sonney2k ^20:07
@HeikoSah sorry20:08
@HeikoSinstall swig2.0 does it20:08
@HeikoSswig points to 1.320:08
@sonney2kthoralf, yeah libsvm's parallelization might be slow20:08
thoralfsonney2k: But 20000 threads?20:08
@HeikoSwiking: Error at cmake/FindSuperLU.cmake:67 (include)20:09
@HeikoSinclude could not find load file20:09
@wikingHeikoS: it's there in the repo20:09
@wikingso i wonder how's that possible at all20:10
@sonney2kthoralf, sure kernel rows computed in parallel20:10
@HeikoSwiking: whats the name?20:10
@wikingcmake/FindSuperLU.cmake20:10
@HeikoSwiking: how to install it?20:11
@sonney2kwiking, do we have an option w/ cmake to specify a swig executable / python executable?20:11
@wikingHeikoS: ?20:11
@HeikoScmake outputs an error and dies20:11
@HeikoSwith this superlu message, what to do?20:12
@wikingHeikoS: dunno man... cmake/FindSuperLU.cmake is in the repository20:12
@wikingit's impossible it cannot find it20:12
@wikingsonney2k: yes20:12
@wikingsonney2k: one way is to set -DCMAKE_PREFIX_PATH20:13
@wikingwhere the python/swig resides u want20:13
@wikingother way is to set the exact path20:13
@wikingbut this are all standard cmake stuff20:13
@wikingi'm not the manual of cmake :(20:13
@sonney2kwiking, we really need a cmake documentation. you are repeating things too often :/20:13
@sonney2kwiking, no but this would only look for swig right? if my swig executable is named swig2.2 then what?20:14
@wikingcheck your FindSWIG.cmake20:14
@wikingsonney2k: the cmake documentation is pretty fucking great20:14
@wikingjust need to google it20:14
@wikingsonney2k: and yes i agree that we need cmake related info about shogun20:15
@wikingbut not about the things you are asking now20:15
@wikingbecause those are really cmake related FAQs20:15
thoralfsonney2k: Mind if I add a limit to simultaneous sg_parallel->get_num_threads(); threads? ;)20:15
@sonney2kthoralf, these are *not* parallel threads20:15
@sonney2kthoralf, just #cpu20:15
thoralfsonney2k: It looks like it's creating all threads at a time.20:16
@sonney2kfoulwall, I guess you used an example file that contains not just 'ACGT' characters20:17
@sonney2kthoralf, then it would be a bug - why do you think so?20:17
@HeikoSwiking: I found the problem: cmake_push_check_state() Unknown cmake command20:18
@HeikoScmake 2.8.520:18
thoralfsonney2k: It calls pthread_create in a loop.20:18
thoralfsonney2k: My fault.20:19
thoralfSorry.20:19
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has quit [Quit: Ex-Chat]20:19
foulwallsonney2k: I did  ` python arts.py data/ARTS.dat.bz2  ` in applications/arts , is it correct?20:19
thoralfsonney2k: Okay, it's no problem in there.20:19
thoralfsonney2k: But then it doesn't explain 20000 threads when running libshogun-modelselection_grid_search_kernel or python_modular-modelselection_grid_search_libsvr_modular20:20
@sonney2kfoulwall, no20:21
foulwallsonney2k: argh, do I need to extract ARTS.data.bz2?20:21
@sonney2kfoulwall, let me find a file for you20:21
@sonney2kfoulwall, no20:21
@sonney2ksec20:21
thoralfsonney2k: gtg, can wait until tomorrow.20:23
@sonney2kfoulwall, https://dl.dropboxusercontent.com/u/31480471/dna.fa20:23
@sonney2kthoralf, well does threads=1 help?20:24
foulwallok sonney2k , let me have a try20:24
@sonney2kthoralf, IIRC modsel is not done in parallel20:24
@sonney2kfoulwall, it will take ~600MB CPU20:24
@sonney2kfoulwall, and at least 5mins20:24
thoralfsonney2k: I'm just running "make tests", so it should work like everywhere else.20:25
@sonney2kfoulwall, did it work?20:28
foulwallsonney2k: finished , only got -1, -42 , how do I demostrate them on web?20:29
foulwallfinished in 134 secs:)20:30
@HeikoSwiking:  shogun needs cmake>2.8.520:33
@HeikoSthats a bit annoying20:33
@wikingHeikoS: life's cruel20:33
@sonney2kfoulwall, you have an old version of shogun installed then20:33
@wikingHeikoS: you can replace cmake_push_check_state20:33
@wikingin FindSuperLu20:34
foulwallthan I'll recompile the newest one20:34
@wikingand then it won't need that20:34
@HeikoSwiking: yeah, will try some other things20:34
foulwallsonney2k: and retry20:34
@HeikoSI need things to work automatically thats the problem20:34
@sonney2kfoulwall, you should have gotton -42 and some real valued numbers. so just plot the numbers in a curve20:34
@wikingHeikoS: well it's only that Find* stuff... so i really think that there's a way around.. look in other Find.cmake script20:34
foulwallok sonney2k20:34
@sonney2kfoulwall, the -42 is sth that you should maybe move to 0 in the curve20:35
@HeikoSah we should have a binary shogun release with static stuff this eats my nerves ;)20:36
foulwallsonney2k: ok20:36
@HeikoSbtw how is the .deb coming along?20:37
@sonney2kHeikoS, we don't even have cmake replacing all of configure so far so far far away20:43
@HeikoSwiking, sonney2k please see my latest push20:55
@HeikoSand let me know if oyu agree20:55
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun20:55
shogun-notifier-shogun: Heiko Strathmann :develop * fbddd15 / cmake/CMakePushCheckState.cmake: https://github.com/shogun-toolbox/shogun/commit/fbddd15d9bb6f1a4ff706d60b4a1c3d7c3f8db7e20:55
shogun-notifier-shogun: added push check state file to make superlu check work for cmake 2.8.420:55
@sonney2kHeikoS, why do we need that? It is already in cmake?20:57
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has quit [Ping timeout: 264 seconds]20:57
@HeikoSsonney2k: yes but not in the old version I have here on the institute cluster20:58
@HeikoSso adding it makes things work20:58
@sonney2kHeikoS, I would rather require a new version then20:58
@HeikoSsonney2k: this would deny cmake for me, but this mini fix makes it work so I thought it is easier this way20:58
@sonney2kHeikoS, no you could easily download http://www.cmake.org/files/v2.8/cmake-2.8.11.2-Linux-i386.tar.gz and be happy20:59
@HeikoSsonney2k: you mean install locally?21:00
@sonney2kHeikoS, sure.21:00
@sonney2kI did this on many buildbots21:00
@HeikoSok then, we can remove the thing again,21:00
@HeikoSI gotta go now though, will do it tomorrow21:00
@HeikoSsee you!21:01
@sonney2kHeikoS, do you know which version21:01
@sonney2kHeikoS, has that file?21:01
-!- lambday [67157e37@gateway/web/freenode/ip.103.21.126.55] has joined #shogun21:01
@sonney2klambday, hey. Looks like HeikoS is about to leave so be fast ...21:01
lambdaysonney2k: oh21:02
lambdayHeikoS: hi21:02
lambdayHeikoS: I will check out colpack (its the same one that Krylstat uses) for graph coloring thing21:02
lambdayand check out how our sparse matrix things fit into that21:03
lambdayHeikoS: but I am kinda worried about CG solvers....21:03
-!- foulwall [~user@2001:da8:215:c252:b1e8:dc7c:8bc4:a5da] has quit [Ping timeout: 264 seconds]21:03
lambdaywithout preconditioners, for some data they don't seem to converge21:03
lambdayHeikoS: but I will check on real data (the ozone matrix) and then will see... the thing is, I don't have a reference value to check it with (our CStatistics::log_det fails for that huge matrix)21:04
lambdayHeikoS: I will try if I can use Krylstat to generate that21:04
lambdaysonney2k: do we have logistic regression?21:05
@sonney2klambday, yes with liblinear21:05
lambdaysonney2k: alright.. great! one of my friends was asking about it, I said I'll check out... will tell him21:06
lambday:D21:06
@sonney2klambday, you have to use L2R_LR_DUAL or L1R_LR (or see LIBLINEAR_SOLVER_TYPE in Liblinear.h)21:06
lambdaysonney2k: okay.. thanks :)21:07
-!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has joined #shogun21:26
shogun-buildbotbuild #1831 of deb1 - libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb1%20-%20libshogun/builds/183121:46
shogun-buildbotbuild #1440 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1440  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>21:46
shogun-buildbotbuild #1471 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1471  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>21:50
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun21:56
shogun-buildbotbuild #7 of osx1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/7  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>21:58
-!- lambday [67157e37@gateway/web/freenode/ip.103.21.126.55] has quit [Ping timeout: 250 seconds]22:03
shogun-buildbotbuild #1558 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/155822:21
shogun-buildbotbuild #1262 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1262  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>22:27
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun22:39
@wikingHeikoS: yo23:13
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]23:55
--- Log closed Tue Aug 20 00:00:23 2013

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!