IRC logs of #shogun for Friday, 2013-09-27

--- Log opened Fri Sep 27 00:00:06 2013
--- Day changed Fri Sep 27 2013
wiking sonney2k ok then it's weird ... as CGradientModelSelection depends on NLOPT00:00
wikingsonney2k: but maybe it doesn't have the right version yet... as i needed to edit class_list.cpp.py00:00
wikingand i dont know when was that commit00:00
@sonney2kwiking, ok maybe00:02
@sonney2kshogun-buildbot, force build nightly_default00:02
shogun-buildbotThe build has been queued, I'll give a shout when it starts00:02
@sonney2kshogun-buildbot, force build --branch=develop nightly_default00:02
@sonney2k*sigh*00:02
shogun-buildbotThe build has been queued, I'll give a shout when it starts00:02
@HeikoSwiking: we want to drop NLOPT soon00:05
@HeikoSwiking: what was done to the unit-tests? some of them take so long when running make test now?00:06
wikingeh?00:06
wikingi havent touched utests00:06
@sonney2kwiking, this one looks good http://shogun-toolbox.org/static/notebook/bss_jade.html00:06
wikingsonney2k: noup00:07
wikingit's not00:07
@sonney2kwiking, shift reload00:07
wikingok getting there...00:07
wiking;P00:07
@HeikoSwiking: so make test basically takes ages to run the unit tests, make unit-tests doesnt, any ideas why that is?00:08
@sonney2khushell, for some reason your FGM still fails :/00:09
@sonney2kdon't know why00:09
@sonney2khttp://shogun-toolbox.org/static/notebook/FGM.html00:09
@sonney2kdata is there00:09
wikingHeikoS: ctest -R unit00:10
@sonney2khushell, what is this bmrm.p ? and bmrm_stats.p ?00:10
@sonney2khushell, ahh how long does training take?00:11
wikingsonney2k: it's libbmrm stuff00:11
@sonney2kanything above 20s will be killed00:11
@sonney2kthat might explain it...00:11
wikinghehehe00:12
wikingthat's for sure not enough00:12
@sonney2kwiking, what is?00:12
@sonney2klet me put 300s there00:12
wiking20s00:12
@sonney2kmore is not accepable00:12
wikingwhy not?00:12
wikingit's a noteok00:12
wikingnotebook note a test script00:13
@sonney2kwiking, so you want someone to watch how the notebook is constructed for > 5 mins?00:13
wikingno00:13
wikingbut this is generated nightly00:14
wikingso who the fuck cares if it takes more than 5 mins00:14
wikingif it's compiled once  you dont need no 5 mins to watch it00:14
wiking(see generated ipynb and html)00:14
@HeikoSwiking, sonney2k what about having a seperate test branch for long-term tests=?00:15
wikingHeikoS: not maintainable00:15
@HeikoSsome tests just take time to check accuracy or similar00:15
@sonney2kwiking, we run this nightly!00:15
@HeikoSso rather shorten them and make sure that things are accurate otherwise?00:15
wikingsonney2k: and nightly an extra 5 mins is a lot?00:15
@sonney2kHeikoS, yes makes sense - maybe we flag them _slow or so in the filename?00:15
@HeikoSsonney2k: and run only nightly or so00:16
@sonney2kHeikoS, yeah00:16
@sonney2kwiking, per example!00:16
@sonney2kthink 100 examples00:16
wikingsonney2k: and?00:16
@HeikoSwhat about having a limit of 5 mins for notebooks?00:16
wikingfirst: i really dont see us having 100 notebooks anywhere in the future00:16
@sonney2kwiking, > 8 hrs?!?00:16
@sonney2kwiking, I do00:17
wikingsecond you can easily run those notebooks parallel00:17
@sonney2knotebooks are fun to do so we will have plenty00:17
wikingso with 4 cores it's already just about 2 hours00:17
wiking;)00:17
@sonney2kwiking, not on this machine00:17
@sonney2kit is weak00:17
@HeikoScurrently we only have a few notebooks00:17
wikingsonney2k: yeah but those could be run anywhere00:17
@HeikoSso 5 mins should be fine right?00:17
@sonney2kwe then need to do this on fatbot and transfer stuff over00:17
wikingsonney2k: it's no problem00:18
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has joined #shogun00:18
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1183378100:18
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has left #shogun []00:18
wikingsonney2k: i told you the simple command for transfering dirs with buildbot00:18
wikingso really a 5 minutes running time for a notebook shouldn't be a problem00:19
wikingwe have like max 8 notebooks atm00:19
wikingand one of them takes longer than coupld of seconds to run00:19
wikingso having 1-2 notebooks that is over 3 minutes00:19
wikingis totally ok imho00:19
shogun-notifier-shogun: Heiko Strathmann :develop * 9bd3ef1 / examples/undocumented/python_modular/ (3 files): https://github.com/shogun-toolbox/shogun/commit/9bd3ef13f2c25a8cd274693f17bf35ca97a8b3c500:20
shogun-notifier-shogun: reduce number of x-validation repeats and number of parameter combinations in grid-search00:20
shogun-notifier-shogun: Heiko Strathmann :develop * 7f21c9b / examples/undocumented/python_modular/mathematics_logdet.py: https://github.com/shogun-toolbox/shogun/commit/7f21c9b0f748f8e273c8bd10ea9f21ef43c02c4600:20
shogun-notifier-shogun: decreased number of iterations to make test run faster00:20
shogun-notifier-shogun: Heiko Strathmann :develop * f607173 / examples/undocumented/python_modular/ (4 files): https://github.com/shogun-toolbox/shogun/commit/f607173050485950fc46ef6949d25053f3a1d70600:20
shogun-notifier-shogun: Merge pull request #1681 from karlnapf/develop00:20
shogun-notifier-shogun:00:20
shogun-notifier-shogun: speed ups for tests00:20
wikingHeikoS: mmm00:27
wiking/home/wiking/shogun/examples/undocumented/python_modular/modelselection_grid_search_krr_modular.py00:27
wikingstill running for more than a minute now00:27
wikingbut maybe it's my fault00:27
wikingjust a sec00:27
* sonney2k off00:28
shogun-notifier-shogun: Heiko Strathmann :develop * dc4786b / doc/ipython-notebooks/ (2 files): https://github.com/shogun-toolbox/shogun/commit/dc4786bf950d34a761eaeca9b32320311bdeb15a00:28
shogun-notifier-shogun: removed notebooks since there will be a more complete one soon00:28
wikingmmm00:28
@HeikoSwiking: maybe its just slow00:28
wikingHeikoS: used to be faster00:28
@HeikoSit should be like at least 10 times faster now00:28
@HeikoSsonney2k: remove the gp notebooks, bye!00:29
wikingit doesn't return00:29
@HeikoSsonney2k: removed00:29
wiking303/534 Test #300: integration-python_modular-tester-modelselection_grid_search_krr_modular ............................................................   Passed    1.08 sec00:30
wikingin the past00:30
shogun-buildbotbuild #1901 of deb3 - modular_interfaces is complete: Failure [failed test python modular]  Build details are at http://buildbot.shogun-toolbox.org/builders/deb3%20-%20modular_interfaces/builds/1901  blamelist: Viktor Gal <viktor.gal@maeth.com>00:30
shogun-buildbotbuild #537 forced00:30
shogun-buildbotI'll give a shout when the build finishes00:30
@HeikoSwiking: ?00:30
@HeikoSah00:30
wiking519/534 Test #763: python_modular-modelselection_grid_search_krr_modular ...............................................................................   Passed    0.61 sec00:30
wikingnow00:30
wikingit doesn't fucking return00:30
@HeikoSwiking: debug/release stuff00:30
wikingit's just running forever00:30
@HeikoSwiking: the unit tests also take *much* longer when running with ctest00:30
@HeikoScompare make unit-tests to make test times00:30
wikingHeikoS: ?!00:31
wikingwhaaat?00:31
wikingah man00:31
shogun-buildbotbuild #537 of nightly_default is complete: Failure [failed configure]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/53700:31
shogun-buildbotbuild #538 forced00:31
shogun-buildbotI'll give a shout when the build finishes00:31
wikingof course it does00:31
wikingbecuase it's completely differnet setup00:31
@HeikoSwell ok00:31
wikingshouldn't compare the two00:32
wikingwhen u do make unit-test00:32
wikingit basically runs ./tests/unit/shogun-unit-test00:32
wikingbut if u run ctest -R unit00:32
wikingthat'll run00:32
wiking./tests/unit/shogun-unit-test --gtest_filter=SOMETHING.*00:32
wikingand it'll run that as many <SOMETHING>s are00:33
wikingthis way you get a better idea which unit test actually failed + ctest invoked valgrind is more understandable as well00:34
@HeikoSwiking: I see, thanks for the explanation00:34
wikingbut this shit is oooold00:34
wikingat least a week old :P00:34
@HeikoShehe, I am in holidays ;)00:34
@HeikoScurrently writing the GP notebook, quite fun00:35
@HeikoSto create these notebooks00:35
@HeikoSsee what one can do with shogun00:35
-!- new_lido [~walid@41.218.173.234] has joined #shogun00:35
@HeikoSwiking: and teaches me one thing: when you think something works (compiled/tested/bla) then trying to use it on some real problem is a totally different thing. Bugs bugs bugs bugs00:36
shogun-buildbotbuild #44 of debian wheezy - memcheck is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/debian%20wheezy%20-%20memcheck/builds/4400:36
wikingHeikoS: :D00:36
@HeikoSwiking: btw how do you run the notebook server?00:37
@HeikoSwiking: with --pylab inline?00:37
@HeikoSor do I have to do this on my own?00:37
wikingi've put this into the profile conf00:40
-!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has quit [Read error: Connection reset by peer]00:40
wikingc.IPKernelApp.pylab = 'inline'00:41
shogun-buildbotbuild #53 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/53  blamelist: Viktor Gal <viktor.gal@maeth.com>00:43
@HeikoSwiking: what do you think about the idea of getting a sponsor to pay someone to do a GSoC like thing for us?00:49
-!- FSCV [~FSCV@50.7.50.60] has quit [Quit: Leaving]00:55
shogun-buildbotbuild #35 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/35  blamelist: Viktor Gal <viktor.gal@maeth.com>00:59
wikingHeikoS: yeah totally01:04
wikingHeikoS: i'm free ;) get a sponsor :P01:04
@HeikoSwiking: I mean imagine how many cool things we could do ;)01:04
wikingyeah i know01:05
@HeikoSwiking: the thing is it is quite some work to make this happen01:05
@HeikoSso its only intersting if we really want that01:05
@HeikoSinteresting01:05
@HeikoSneed some stuff to show01:05
wikingi mean after the release i hope i can start again doing some more interesting stuff and not this tidious pre-release cleanup shit01:05
@HeikoSotherwise no-one will bother01:05
@HeikoSwiking: I think the clean up stuff is quite important01:05
wikingyeah01:05
wikingbut it's really getting on my nerves01:06
@HeikoSunderstanable01:06
wikingi'm excited about 3.0 so i'm pushing it01:06
@HeikoSman01:06
@HeikoSunderstandable01:06
wikingbut i hope that i can shift my commits to more interesting stuff ;P01:06
wikingoh btw01:06
@HeikoSI am really excited about making shogun more accessible01:06
wikingi need to run something on a cluster01:06
@HeikoSah cool01:06
wikingi'll try your stuff01:07
wikingbut i think i'll have to write an opensource backend01:07
wikingif this works out01:07
wikingthen we have another very complex example for latent svm01:07
wiking;)01:07
wikinglatentSOSVM01:07
wikingbut that is certainly not an ipython notebook demo :P01:07
wikingas it really takes like a day to train the model :P01:08
@HeikoSoh wow01:08
@HeikoSwiking: yeah let me know whether this independent computation thing works for you01:08
@HeikoSwiking: for me it does it job nicely01:08
wikingHeikoS: we'll see01:09
@HeikoSwiking: but I heavily use pickle01:09
wikingHeikoS: i'm wondering about sponsors though01:09
@HeikoSand I did some nastly hacks01:09
wikingHeikoS: i rather imagine we start doing ML consulting :P01:09
@HeikoSwiking: yeah thats one thing i also had in mind01:09
@HeikoSwiking: open-source ML software + how-to use it properly01:10
shogun-buildbotbuild #38 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/38  blamelist: Viktor Gal <viktor.gal@maeth.com>01:17
-!- travis-ci [~travis-ci@ec2-54-224-190-134.compute-1.amazonaws.com] has joined #shogun01:18
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1183378101:18
-!- travis-ci [~travis-ci@ec2-54-224-190-134.compute-1.amazonaws.com] has left #shogun []01:18
wikingHeikoS: https://travis-ci.org/shogun-toolbox/shogun/jobs/1184289201:22
wikingmodelselection_grid_search_krr_modular01:22
shogun-buildbotbuild #538 of nightly_default is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/53801:22
wikinghas there been changes lately in model selection01:22
wikingi really wonder why it started to do this01:23
@HeikoSroman changed some things01:23
wikingit was working fine before01:23
@HeikoSfew days ago01:23
@HeikoSchecking01:23
@HeikoSwiking: https://travis-ci.org/shogun-toolbox/shogun/builds/1163129101:25
@HeikoSlast green travis01:25
@HeikoS219 Test #172: python_modular-modelselection_grid_search_krr_modular ......................   Passed    2.07 sec01:25
@HeikoSwiking: I think he broke it01:27
@HeikoSah it was his 76 files changed PR01:28
@iglesiasggood night guys01:38
@iglesiasgI've got an interview tom, so wish me luck! :)01:38
-!- iglesiasg [~iglesias@93.158.36.26] has quit [Quit: leaving]01:39
shogun-buildbotbuild #54 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/54  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>02:03
-!- travis-ci [~travis-ci@ec2-204-236-220-70.compute-1.amazonaws.com] has joined #shogun02:24
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1184288902:24
-!- travis-ci [~travis-ci@ec2-204-236-220-70.compute-1.amazonaws.com] has left #shogun []02:24
-!- dsockwel1 [~twgs@199.167.199.97] has joined #shogun02:32
shogun-notifier-shogun: Heiko Strathmann :gp_notebook * 23a9df9 / doc/ipython-notebooks/gaussian_process/gaussian_process_classification.ipynb: https://github.com/shogun-toolbox/shogun/commit/23a9df9d58a56da1ff8cab9cade802910d0c11f502:38
shogun-notifier-shogun: FITC sparse regression paragraph02:38
shogun-buildbotbuild #55 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/55  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>02:43
shogun-notifier-shogun: Heiko Strathmann :gp_notebook * 3049cbb / doc/ipython-notebooks/gaussian_process/ (2 files): https://github.com/shogun-toolbox/shogun/commit/3049cbb25cbdcb1890a2a8dafda826e68c5f659702:45
shogun-notifier-shogun: renamed file02:45
shogun-buildbotbuild #36 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/36  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>02:53
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has quit [Quit: Leaving.]02:58
shogun-buildbotbuild #39 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/39  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>03:08
shogun-buildbotbuild #468 of nightly_all is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_all/builds/46803:24
shogun-buildbotbuild #539 of nightly_default is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/53903:46
-!- sonne|osx_ [~sonne@f053045155.adsl.alicedsl.de] has joined #shogun03:49
-!- sonne|osx [~sonne@f053040133.adsl.alicedsl.de] has quit [Ping timeout: 240 seconds]03:51
-!- sonne|osx_ is now known as sonne|osx03:51
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has joined #shogun04:19
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1184318704:19
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has left #shogun []04:19
-!- travis-ci [~travis-ci@ec2-54-224-190-134.compute-1.amazonaws.com] has joined #shogun05:01
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1184614605:01
-!- travis-ci [~travis-ci@ec2-54-224-190-134.compute-1.amazonaws.com] has left #shogun []05:01
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]05:45
-!- foulwall [~zhengyang@114.255.40.22] has joined #shogun05:50
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has joined #shogun05:58
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1184626905:58
-!- travis-ci [~travis-ci@ec2-54-235-6-157.compute-1.amazonaws.com] has left #shogun []05:58
-!- sonne|osx [~sonne@f053045155.adsl.alicedsl.de] has quit [Quit: sonne|osx]06:49
-!- zxtx [~zv@c-50-133-182-155.hsd1.ma.comcast.net] has joined #shogun07:14
-!- gsomix [~gsomix@46.20.65.216] has quit [Ping timeout: 240 seconds]07:31
-!- gsomix [~gsomix@46.20.65.216] has joined #shogun07:44
gsomixgood morning08:07
-!- zxtx [~zv@c-50-133-182-155.hsd1.ma.comcast.net] has quit [Ping timeout: 256 seconds]08:28
wikingsonne|work: ok i've figured out the problem with some of the notebook generations09:23
wikingexport LD_LIBRARY_PATH="`pwd`/build_install/lib:$LD_LIBRARY_PATH"09:24
wikingexport PYTHONPATH="`pwd`/build_install/lib/python2.7/dist-packages:$PYTHONPATH"09:24
wikingbut in the nightly09:24
wiking-- Installing: /home/buildslave/nightly_default/build/build/build/lib/python2.7/dist-packages/shogun/__init__.py09:24
wikingso i'll change this now09:24
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]09:25
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun09:25
wikingshogun-toolbox: force build --branch=develop 'nightly_default'09:26
wikingshogun-buildbot: force build --branch=develop 'nightly_default'09:26
shogun-buildbotbuild #540 forced09:26
shogun-buildbotI'll give a shout when the build finishes09:26
sonne|workwiking: ahh ok09:43
sonne|workgsomix: moin moin09:44
-!- iglesiasg [~iglesias@93.158.36.26] has joined #shogun09:44
-!- mode/#shogun [+o iglesiasg] by ChanServ09:44
sonne|workfoulwall: hey foulwal!09:44
sonne|workfoulwall ^09:44
wikingsonne|work: hehe there's something seriously wrong with the cygwin bot09:48
wikingsonne|work: Elapsed18 hrs, 2 mins, 36 secs09:48
wiking18 hours :P09:48
sonne|workwiking: give me gdb and I fixed it09:48
sonne|workohh man09:48
wikingand that's compiling09:48
sonne|workthat is broken beyond repair09:48
wikingnow test09:48
wiking*not test09:48
wikingsonne|work: cant u install ccache on cygwin (once you have access to it)09:49
sonne|workwiking: it is installed!09:50
wikingah woah yes09:50
wikingand it even found it09:50
wikingmmm09:50
wikingi guess then maybe the cache is tained or something :P09:50
shogun-buildbotbuild #540 of nightly_default is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54009:57
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]10:00
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun10:00
wikingshogun-buildbot: force build --branch=develop 'nightly_default'10:00
shogun-buildbotbuild #541 forced10:00
shogun-buildbotI'll give a shout when the build finishes10:00
-!- iglesiasg [~iglesias@93.158.36.26] has quit [Quit: leaving]10:10
sonne|workwiking: but the notebook traceback looks cool now doesn't it :)10:21
wikingsonne|work: yeah :) but it still failing... hope this rebuild will fix it10:22
sonne|workwiking: what is this shogun crap10:32
sonne|workrunning /home/buildslave/nightly_default/build/doc/ipython-notebooks/statistics/mmd_two_sample_testing.ipynb10:32
sonne|workFailed to run cell:10:32
sonne|workfrom shogun.Features import RealFeatures10:32
sonne|workfrom shogun.Kernel import GaussianKernel, CustomKernel10:32
sonne|work...10:32
sonne|workthis notebook needs to use modshogun10:33
wikingmmm wtf10:33
wikingi dont understand this10:33
sonne|workwiking: can we please kill the shogun emulation10:33
wiking  PYTHONPATH=`pwd`/build/build/lib/python2.7/dist-packages:10:33
shogun-buildbotbuild #541 of nightly_default is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54110:33
wikingemulation?10:33
wikingah yeah no it's not the problem atm10:33
sonne|workwell it should be from modshogun import FOO10:33
sonne|worktrue10:33
sonne|workthe modshogun stuff fails too10:33
wikingImportError                               Traceback (most recent call last)<ipython-input-2-da8b46f88d7c> in <module>()10:33
wiking----> 1 from modshogun import StringCharFeatures, RAWBYTE, HashedDocDotFeatures, NGramTokenizer10:33
wikingImportError: No module named modshogun10:34
sonne|workwiking: btw you can test if it works by just running10:34
sonne|work/usr/local/bin/update_*10:34
wikingsonne|work: yeah i know10:34
sonne|workfrom the buildslave account10:34
wikingi'm just wondering why it does not fucking work10:34
sonne|workso no need to run the whole nightly_default10:34
sonne|workI only tested with deb3 python modular10:34
sonne|workseems like BB configs are different10:34
sonne|workfoulwall: do you need help or does it work now with the import?10:35
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]10:47
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun10:47
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has joined #shogun10:54
-!- mode/#shogun [+o HeikoS] by ChanServ10:54
-!- besser82 [~besser82@77-21-40-118-dynip.superkabel.de] has joined #shogun10:59
-!- besser82 [~besser82@77-21-40-118-dynip.superkabel.de] has quit [Changing host]10:59
-!- besser82 [~besser82@fedora/besser82] has joined #shogun10:59
wikinghushell: hy here?11:01
hushellwiking: hey11:01
wikinghushell: on your machine how long did it take to run FGM.ipynb11:01
wikingroughly11:02
hushellmore than 20 mins11:02
wikingoh i see :)11:02
wikingsonne|work: ^11:02
wiking:D11:02
wikingi'll have to adapt this to fatbot11:02
hushellBMRM took 547.777660847 seconds + SGD took 456.158573866 seconds11:03
hushellwiking: but I used pickle to save the models, so 2nd run shouldn't be costly11:04
hushellwiking: one thing I noticed: integration tests are slower, each of them takes more than 90s, there are how many such tests roughly?11:07
sonne|workhushell: what kind of machine did you use11:11
sonne|workwiking: that is a lot!11:11
sonne|workhushell: caching is not used on buildbots so...11:12
wikinghushell: that's fixed... although there's one problem still with gridsearch...11:12
wikingsonne|work: will fix it11:12
sonne|workwiking: fix what?11:12
wikingsonne|work: running time..11:13
sonne|workhow?11:13
wikingdont worry11:14
wikingwill fix it11:14
wikingconcentrate on other stuff11:14
hushellsonne|work: Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz x 4, Mem 6GB, Ubuntu 12.0411:15
sonne|workhushell: might be a bit faster then on fatbot11:16
hushellit's acceptable in my laptop, how long it takes on buildbots?11:17
sonne|workno idea11:19
sonne|workhushell: we had a time limit of 20s11:20
sonne|workhushell: now 300s11:20
sonne|workbut likely not sufficient11:20
wikingbbl11:20
hushellsonne|work: the new version takes longer :/11:20
wikinghushell: ?11:20
wikinghushell: you mean the integration test?11:21
wikinghushell: as there has been no change in SOSVM and stuff..11:21
hushellbecause now I am using two solvers, so double training time11:21
wikingah you mean SOSGD?11:21
sonne|workhushell: in the same notebook?11:21
wikinghushell: have u tested NCBM for CDualLibQPBMSOSVM11:21
wiking?11:21
hushellwiking: yeah, integration tests are slow, I run it for 1 hour, but only finished 100 tests11:22
wikinghushell: ignore integration tests11:22
wikinghushell: they just got partially fixed yesterday11:22
wikinghushell: and there's still one integration test that basically goes into infinte loop11:22
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun11:22
shogun-notifier-shogun: lambday :develop * 0486ff3 / tests/unit/mathematics/linalg/ (2 files): https://github.com/shogun-toolbox/shogun/commit/0486ff3032e4e17593d954b1399d1ae440a2052a11:22
shogun-notifier-shogun: reduced size and accuracy for speeding up unit-tests11:22
shogun-notifier-shogun: lambday :develop * 392bffc / src/shogun/mathematics/linalg/linsolver/IterativeShiftedLinearFamilySolver.h: https://github.com/shogun-toolbox/shogun/commit/392bffccf620665444abcd6a7d5ca99ad0f3211b11:22
shogun-notifier-shogun: minor documentation correction11:22
shogun-notifier-shogun: Soeren Sonnenburg :develop * 608b7e5 / / (3 files): https://github.com/shogun-toolbox/shogun/commit/608b7e54d6c8cc7b0630380087af4a575b6886dd11:22
shogun-notifier-shogun: Merge pull request #1683 from lambday/develop11:22
shogun-notifier-shogun:11:22
wikingso ignore that for now11:22
hushellyeah, I compared bmrm with sgd11:22
shogun-notifier-shogun: reduced size and accuracy for speeding up unit-tests11:22
wikinghushell: no11:22
wikinghushell: CDualLibQPBMSOSVM with NCBM11:22
hushellncbm is better than bmrm?11:23
wikinghushell: so when u construct CDualLibQPBMSOSVM11:23
sonne|workhushell: so which one wins?11:23
wikingthen before doing anything11:23
hushellSGD is winning, http://nbviewer.ipython.org/657947211:23
wikinghushell: CDualLibQPBMSOSVM.set_solver(NCBM)11:23
wikingplease try to run this way once11:24
hushellwiking: I haven't tested all of them, I see there are a couple of solvers11:24
hushellwiking: so which one is best in your experience?11:24
wikinghushell: ncbm is very different from the others11:25
wikinghushell: you could try P3BMRM but you need parameter tuning there..11:25
hushellwiking: okay, this seems necessary11:25
wikinghushell: but P3BMRM with right parameters usually is heaps better than simple BMRM11:25
wikinghushell: but just test NCBM with the same params11:25
wikingjust use NCBM as solver...11:25
wikingi'm really curious about it's performance11:26
sonne|workhushell: interesting - would not have expected that. I guess your iterations are rather cheap but not the optimization steps11:26
hushellwiking: I could run all of them and see what happened :)11:26
hushellI read the BMRM paper, only few differences in fact, but the NCBM seems quite different11:27
hushellsonne|work: yeah, SGD update w for each example, the cutting plane based methods have to reoptimize with all previous constraints, but which better, is a question11:29
-!- gsomix is now known as gsomix|classes11:29
sonne|workhushell: I would have expected that the number of iterations of BMRM is *much* less11:30
sonne|workhushell: with optimization time being higher11:30
hushellsonne|work: I didn't understand the convergence analysis part thoroughly, you will be interested in Table 1 of this paper : http://www.pletscher.org/papers/lacostejulien2013fwstruct.pdf11:32
sonne|workhushell: too bad we don't have this one ... it seems even much faster11:35
sonne|workbcfw-wavg11:35
hushellwiking: I'll you know the results of comparison experiment, maybe tomorrow night11:35
lisitsyn1hey there11:35
hushellsonne|work: this is what I am going to do this weekend :)11:35
-!- lisitsyn1 is now known as lisitsyn11:35
sonne|workhushell: cool!11:36
hushellsonne|work: it was included in gsoc plan, but I was too slow :(11:36
sonne|workhushell: well you did a lot11:36
sonne|workand post gsoc you can do as much as you want with no deadline or anything in the back11:37
sonne|workmuch more relaxed this wya11:37
sonne|workway11:37
hushellyeah, looking forward to finish all tasks :D11:38
hushellI'll try to send a PR every one or two weeks11:38
hushellgoing to bed now, talk you later11:39
sonne|workhushell: cu!11:40
lisitsynHeikoS: sonne|work: ipython keeps receiving enormous amounts of money :D11:47
lisitsynsonne|work: wiking: doodle please11:55
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has quit [Ping timeout: 248 seconds]11:57
sonne|worklisitsyn: which dodole?12:01
sonne|worklisitsyn: sure makes sense it is crazy sexy cool12:01
-!- new_lido [~walid@41.218.173.234] has quit [Quit: Leaving]12:03
lisitsynsonne|work: ahh you don't receive mails?12:10
lisitsynsonne|work: http://doodle.com/rgpvdife4s78bnkh this one12:10
shogun-buildbotbuild #56 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/56  blamelist: lambday <heavensdevil6909@gmail.com>12:49
shogun-buildbotbuild #37 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/37  blamelist: lambday <heavensdevil6909@gmail.com>, Soeren Sonnenburg <sonne@debian.org>13:00
shogun-buildbotbuild #40 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/40  blamelist: lambday <heavensdevil6909@gmail.com>, Soeren Sonnenburg <sonne@debian.org>13:15
sonne|worklisitsyn: as usual I am utterly confused about time zones and what this is about?13:20
lisitsynsonne|work: a) I set it to berlin but it should be shown b) it is dev meeting13:27
lisitsynb) cont'd - about release mainly13:27
sonne|workHeikoS: well we need to be a e.V. to get money and for sure not from M$ - we are OSS and not MSS13:36
besser82sonne|work: you need $ for what?  Maybe I can help....13:36
sonne|workbesser82: no idea - the others are talking about $13:37
sonne|workso fear gsoc money helped very much with paying rent for the servers though13:37
besser82sonne|work: if it's about servers I can helpout with f-i  :D13:37
besser82sonne|work: And who was talking about $? can't find any in history  :(13:38
sonne|workbesser82: heh well developers is what we need most imho though13:39
sonne|workone cannot have too many I would say13:39
besser82sonne|work: that's true  ;)13:39
besser82sonne|work: but if it's about servers / appliances and such, I can get some for you paid by redhat ;)13:40
sonne|workbesser82: well I would feel more comfortable to run the demos / ipython notebook vms on a different machine than the buildbot / webserver13:42
sonne|workbut IMHO doing it on the buildbot aka fatbot is sufficient (for now)13:42
besser82sonne|work: that's what HeikoS said as well ;)   For this I'm already in talks with Kevin Fenzi (f-i master mind) to setup a cloud-appliance for you ;)13:43
sonne|workfor all the rest the only thing I can imagine we need money for is to meet 1-2 times a year13:43
besser82sonne|work: I can look if I could do sth. there  ;)13:44
besser82sonne|work: Like a workshop on Flock e.g.13:45
besser82sonne|work: Flock visit is paid by redhat fully  ;)13:45
besser82sonne|work: So we I can arrange to get some room / time for ml-stuff centered around SHOGUN on those events13:46
sonne|workbesser82: anyway I will share the plane trip with HeikoS to SFO and back so we can discuss plenty13:47
sonne|workalrighty work time!13:48
shogun-buildbotbuild #57 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/57  blamelist: lambday <heavensdevil6909@gmail.com>, Soeren Sonnenburg <sonne@debian.org>13:58
besser82sonne|work: btw. don't want lobby or so.  Just proposing possiblities....14:01
sonne|workbesser82: we will need your cmake file ...14:01
besser82sonne|work: I'm on it  ;)14:02
lisitsynbesser82: I would be really happy if we get funding for some meetings14:10
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has quit [Ping timeout: 248 seconds]14:10
lisitsynit seems we lack it14:10
besser82lisitsyn:  Let me look what i can do  ;)14:10
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has joined #shogun14:11
-!- mode/#shogun [+o HeikoS] by ChanServ14:11
lisitsynbesser82: ah btw do you have any webserver to share tapkee website? ;)14:11
lisitsynof course I wouldn't mind to put fedora banner somewher14:11
lisitsyne*14:11
besser82lisitsyn: let me look if I can get some cloud-appliance  ;)14:12
besser82lisitsyn: powered by fedora-infra  ;)14:12
lisitsynbesser82: I am now on free heroku14:12
lisitsynwell I used to employ digital ocean but I decided to save some money cause heroku works as well14:12
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]14:22
-!- travis-ci [~travis-ci@ec2-54-234-173-106.compute-1.amazonaws.com] has joined #shogun14:54
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1185769614:54
-!- travis-ci [~travis-ci@ec2-54-234-173-106.compute-1.amazonaws.com] has left #shogun []14:54
-!- zxtx [~zv@c-50-133-182-155.hsd1.ma.comcast.net] has joined #shogun15:39
-!- FSCV [~FSCV@50.7.50.60] has joined #shogun15:47
-!- van51 [~van51@athedsl-409794.home.otenet.gr] has joined #shogun16:38
-!- zxtx [~zv@c-50-133-182-155.hsd1.ma.comcast.net] has quit [Ping timeout: 252 seconds]16:55
van51sonne|work: sonney2k: ping17:05
-!- gsomix|classes is now known as gsomix17:16
gsomixgood evening17:16
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has joined #shogun17:44
-!- sonne|osx [~sonne@f053045155.adsl.alicedsl.de] has joined #shogun17:44
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has left #shogun []17:51
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has joined #shogun18:02
-!- mode/#shogun [+o HeikoS] by ChanServ18:02
-!- HeikoS [~heiko@218.4.166.178.rev.vodafone.pt] has quit [Client Quit]18:04
-!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]18:08
-!- FSCV [~FSCV@50.7.50.60] has quit [Quit: This computer has gone to sleep]18:19
wikinghushell: ping?18:33
-!- foulwall [~zhengyang@114.255.40.22] has quit [Ping timeout: 246 seconds]18:38
wikingyoooohoooo18:40
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun18:53
shogun-notifier-shogun: Roman Votyakov :develop * 5c44d11 / src/shogun/regression/KernelRidgeRegression.cpp: https://github.com/shogun-toolbox/shogun/commit/5c44d11914bffdc6c2888bd04dcd822bb959956518:53
shogun-notifier-shogun: set epsilon 1e-4 by default for krr18:53
shogun-notifier-shogun: Heiko Strathmann :develop * a1c0f00 / src/shogun/regression/KernelRidgeRegression.cpp: https://github.com/shogun-toolbox/shogun/commit/a1c0f00fb0030a6988b492b657b18f9eb074725318:53
shogun-notifier-shogun: Merge pull request #1685 from votjakovr/feature/gp_refactoring18:53
shogun-notifier-shogun:18:53
shogun-notifier-shogun: Set epsilon=1e-4 by default for krr18:53
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has quit [Ping timeout: 248 seconds]19:38
@sonney2kvan51, pong19:39
@sonney2kwiking, any fixes wrt notebook generation?19:40
van51sonney2k: hey!19:41
van51sonney2k: I tried to re-run the experiments with the spectrum kernel using libsvm but I ran into some problems19:41
-!- lambday [67157f4c@gateway/web/freenode/ip.103.21.127.76] has joined #shogun19:42
shogun-buildbotbuild #1906 of deb3 - modular_interfaces is complete: Failure [failed test libshogun]  Build details are at http://buildbot.shogun-toolbox.org/builders/deb3%20-%20modular_interfaces/builds/1906  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>19:42
van51sonney2k: first of all, the preprocessor wouldn't work with a dataset of 50k19:42
van51sonney2k: so I switch it to on-the-fly preprocessing19:43
van51sonney2k: but I got a seg fault and valgrind reported it in radix_sort19:43
-!- votjakovr [~votjakovr@host-46-241-3-209.bbcustomer.zsttk.net] has joined #shogun19:44
wikingsonney2k: yeah i'm notebook now again19:45
wikingsonney2k: writing NotebookManager for ipython :)19:45
wikingand i'll reconfig the buildbot to be able to run notebooks on a remote bot as well19:46
wiking(i.e. on fatbot19:46
van51sonney2k: I'll also start to implement fast food19:46
wikinghehe19:52
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]19:53
wikinghttp://www.picloud.com/pricing/19:53
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun19:53
@sonney2kwiking, you forgot the reconfig the bot19:55
@sonney2kconfig itself is OK!19:55
@sonney2kvan51, it is too big indeed19:56
@sonney2kvan51, you need to use 'linadd' which is only available with svmlight19:56
wikingsonney2k: ?19:56
@sonney2kwiking, make reconfig19:57
wikingsonney2k: why?19:57
wikingwhat did i change last tim?19:57
@sonney2kwiking, now the configure did add the right install path19:57
@sonney2kwiking, the path but forgot to reconfig19:57
van51sonney2k: ok, w/ or w/o on-the-fly preprocessing?19:57
@sonney2kvan51, you have to use the CommUlongStringKenrel and the SortU* preprocessor19:58
@sonney2kand yes apply() before training19:58
van51sonney2k: yeah. I am using those19:58
wikingsonney2k: i would say lets provide notebook cloud, but with some kind fo a registartion method19:58
wikingsonney2k: let's say oauth2 and then u can login with facebook/googleacc/twitter?19:59
wikingok?19:59
@sonney2kwiking, no idea what you are talking about...19:59
wikingsonney2k: notebook server19:59
@sonney2kwiking, I am trying to get the .html generated and on the website19:59
van51sonney2k: thanks. I'll try again in a while19:59
@sonney2kwiking, ahh ok sure why not19:59
wikingsonney2k: and this way everybody has it's own home etc....19:59
@sonney2kvan51, could you please try with a very small data set?19:59
wikingand about sharing atm i really dont care :P20:00
@sonney2kwiking, and where is this run?20:00
wikingsonney2k: fatbot20:00
wikingfor the moment20:00
van51sonney2k: ok. give me a moment20:00
wikingno better place20:00
wikingif somebody gives us $$$ we move it to amazon ec220:00
wikingi can cut off the CPU time20:00
@sonney2kwiking, yeah well maybe besser82 can help out. if you intend to do it this big it might actually become popular20:00
wiking+ memory20:00
wikingand if somebody wants to have something extra20:01
wikingthen we'll think about how to fix20:01
@sonney2kvan51, please post the code somewhere for me to look at20:01
wikingafaik this will not be big for the first couple of months20:01
wikingso we are on the save side ;P20:01
van51sonney2k: let me convert it to use svmlight20:01
@sonney2kwiking, well it depends a lot on timing20:01
wikingand in the meanwhile we'll try to find a way how we can get some sponsors for hosting this somewhere else20:01
besser82wiking, sonney2k: what's about me?20:01
@sonney2kwiking, look when I do a blog post it will be on planet.debian.org which is read by *many* people20:01
@sonney2kwiking, if we have the workshop videos on youtube20:02
wikingsonney2k: we'll have it no worries20:02
wiking(videos)20:02
wikingbesser82: hosting ipython server :P20:02
@sonney2kwiking, and google also blogs about our gsoc experience20:02
@sonney2kand it is announced on G+ machine learning/big data etc etc20:02
wikingbesser82: probably we are going to have problems with resources20:02
@sonney2kit surely will get plenty of hits20:02
besser82wiking: I have already talked with HeikoS about that, I'm going to get some cloud appliance for it....20:03
wikingsonney2k: i'll throttle it :P20:03
@sonney2kwe are not weka but what we have with the notbooks etc it is cool20:03
@sonney2kif we appear in some newsletter at some point (like linux magazin last time)20:03
besser82wiking: Or possibly a few more  ;)20:03
@sonney2kor somebody writes a short ML intro20:03
wikingsonney2k: yeah true20:03
@sonney2kfor example for heise / CT20:04
wikingbut this is what we can do now20:04
besser82wiking, sonney2k:  You want publicity?20:04
wikingwe dont have money20:04
@sonney2kour server will be dead20:04
wiking-> no cloud20:04
besser82wiking, sonney2k:  How about Heise and linux magazine  ;)20:04
van51sonney2k: https://gist.github.com/van51/673256720:04
besser82wiking, sonney2k:  Have some connections there  ;)20:04
wikingif somebody gives me an access to a cloud where we pay after resource20:04
@sonney2kbesser82, well we try to make 3.0 to be a hgiht quality sexy and cool20:04
@sonney2kso this would be the time to do sth about it20:04
wikingthen we can rebill that for the users as well20:05
wikingbut then again20:05
@sonney2kbesser82, exactly heise / linux magazin would be the right people20:05
wikingit should be free for small demos20:05
besser82wiking, sonney2k:  And I know Thorsten Lemhuis  and Michael Schwendt  ;)20:05
wikingotherwise ppl will not use it20:05
@sonney2kbesser82, somebody already wrote already about shogun in linux magazin for our (crappy) 2.1 release20:05
wiking2.1 was coooooool20:05
wiking:D20:06
@sonney2k3.0 is really eyecandy if we can get it right20:06
wikingso i'll do this shit and setup some shaping20:06
@sonney2kbut it is a lot of work20:06
besser82wiking, sonney2k: I can arrange sth. short before relase  ;)20:06
wikingbesser82: when?20:06
@sonney2kthat is my only concern20:06
wikingbesser82: or what?20:06
wiking:)20:06
@sonney2kno idea we can live up to the standards we want to have20:06
wikingsonney2k: well we are getting there now20:06
besser82wiking, sonney2k: Publicity in iX and Linux Magazine20:06
wikingbesser82: we need cloud :)20:07
besser82wiking: that, too20:07
wikingok but when?20:07
@sonney2kbesser82, yeah that would be thousands of readers20:07
wikingi mean i know that i shouldn't be anxious like this20:07
@sonney2kbesser82, and I am afraid of heise DDOS20:07
wikingbut still i need to prepare shit for this notebook20:07
wikingserver20:07
besser82wiking: I'm currently dealing with f-i people to get some cloud-appliances20:07
wikingsonney2k: do we have $$?20:08
@sonney2kwiking, ok I will hack sth together for the website20:08
wikingsonney2k: you mean ref to the notebooks?20:08
besser82wiking, sonney2k: Heise DDOS?20:08
@sonney2kwiking, a bit yes20:08
wikingsonney2k: can we buy an ec2?20:08
@sonney2kwiking, we get $500 per student20:08
wikingor anything that is similar20:09
wiking?20:09
besser82wiking, sonney2k: you don't need ec220:09
@sonney2kbesser82, :)20:09
wikingbesser82: ok anything is cool for me20:09
wikingsonney2k: i'm fine by sponsoring a little bit of my own money20:09
besser82wiking, sonney2k: I'll give you c-as, just needs some days ;)20:09
wikingjust to get this started asap20:09
@sonney2kI have some contacts into amazon I could ask them ...20:09
wikingi mean anyhow this would be on demand shit20:10
@sonney2kbesser82, never heard of heise DDoS attacks tacking down everything?20:10
wikingso there's no real fee if it's not used20:10
besser82sonney2k: not yet :)20:10
wikingbesser82: is it possible to buy cloud service where u want to get some sponsored access for us?20:10
@sonney2kbesser82, any project website mentioned in 'heise newsticker' was usually not reachable directly afterwards20:11
wikingbesser82: as maybe the easiest way is that i signup/pay now a little bit to start up with the setup/tests and then later we might get a 'discount' or something20:11
besser82wiking: fedora-infra is completely free  ;)20:11
besser82wiking: just needs a few days...20:12
besser82wiking: ~ middle of next week  ;)20:12
besser82wiking: I can arrange ~3 c-a20:12
besser82wiking: and is hosted on own servers  ;)20:13
wikingbesser82: yeah i get that20:13
wikingi'm just concernt about the timeframe20:13
wikingbut no worries i have other bugs to fix20:14
wikingso i'll wait another 3 days20:14
wikingmaybe i'll check till then how to this oauth based ipython notebook server shit20:14
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun20:18
@sonney2kwiking, looks like it works http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/542/steps/notebooks/logs/stdio :)20:20
lambdayvotjakovr: could you please have a look at https://github.com/shogun-toolbox/shogun/issues/168420:22
wikingloool20:23
wikingWe are pleased to announce that the IPython project has received a $1.15M grant from the Alfred P. Sloan foundation, that20:23
votjakovrlambday: hi! thanks for the report i'll have a look20:23
shogun-buildbotbuild #58 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20static%20analysis/builds/58  blamelist: Roman Votyakov <votjakovr@gmail.com>20:23
lambdayvotjakovr: thanks :)20:23
wikingthis is crazy with ipython :)20:23
wikingsonney2k: LMNN timed out20:24
wikingsonney2k: 3 cells failed to complete 25 code cells from notebook20:24
wiking(lmnn)20:24
votjakovrlambday: am i right, that you just run ctest and get this output?20:25
lambdayvotjakovr: in debug20:25
wikinggp regression is failing as well20:25
wikingwho wrote this: http://shogun-toolbox.org/static/notebook/gp_regression.html20:25
wikingand this one is just not generated at all http://www.shogun-toolbox.org/static/notebook/FGM.html20:25
lambdaywiking: is there any time limit for the notebooks?20:26
votjakovrwiking: it seem like gp_regression notebook was removed20:26
wikingsonney2k: ^20:27
@sonney2kvotjakovr, yes heiko removed it20:27
@sonney2kvotjakovr, he said he has some BIIIIIIG notebook coming up20:28
votjakovrsonney2k: yeah i know :)20:28
votjakovrsonney2k: i'm working on some cool examples for it20:28
@sonney2klambday, yes there is a time limit20:28
@sonney2kbut I we can change it20:28
lambdaysonney2k: what is it? my notebook takes quite a lot of time20:29
@sonney2klooks like hushell yours and fernandos notebook require too much time20:29
@sonney2klambday, how much?20:29
lambdaysonney2k: have huge data... although reduced many things but still in my machine it takes around 20+ mins20:29
@sonney2klambday, we run this stuff nightly and there is a 1200s timeout with no output20:29
@sonney2klambday, your machine is core i7 right?20:30
lambdaysonney2k: that's bad for my notebook :(20:30
lambdaysonney2k: yep20:30
@sonney2klambday, IMHO this is too big20:30
@sonney2klambday, can you reduce it a little?20:30
lambdaysonney2k: if I try to reduce it, we'll have many warning messages in the html itself...20:30
@sonney2klambday, why?20:30
lambdaysonney2k: yeah me and heiko discussed a few things - first I am trying to steal your pthreaded sparse-matrix * vector code :D and use that whenever we have HAVE_PTHREAD defined20:31
lambdayas per our experience from the benchmark goes this would help quite a lot20:31
lambdaysonney2k: if this too doesn't help much... then we'll try reducing the iterations in the iterative solvers that we use... (which in turn prints warning Did not converge!)20:32
@sonney2klambday, but can't you reduce the data set or so?20:33
lambdaysonney2k: but another problem is also there... we compute direct log-det using CStatistics::log_det method which is, as expected, the monster20:33
@sonney2klambday, I mean use a subset of dims/examples20:33
lambdaysonney2k: I can... in fact I can remove a whole dataset20:33
@sonney2klambday, maybe it even runs out of memory on this poor machine20:33
@sonney2kit is not even corei7 yet20:33
@sonney2ksome old core2 duo20:34
lambdaysonney2k: how much memory do we have in 7nn?20:34
shogun-buildbotbuild #38 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20undefined%20behaviour%20analysis/builds/38  blamelist: Roman Votyakov <votjakovr@gmail.com>20:34
lambdayoh20:34
@sonney2kquadcore at least20:34
lambdaysonney2k: I will surely reduce that... in fact my notebook is huge!!20:34
* sonney2k reruns notebook generation20:35
@sonney2klambday, how much memory does it need?20:35
lambdaysonney2k: I can cut down a lot of things which I mainly used to show the power of our method on a really large dataset20:35
lambdaysonney2k: errr... didn't notice!20:35
@sonney2klambday, I would do it this way: write some constants at the beginning of the notebook indicating that you just take a subset and give a comment that this can be 1000 times bigger20:35
lambdaysonney2k: for direct log_det computation, it creates a cholesky factor with almost 150 mil entriesz20:35
lambday]nnz entries that it20:35
@sonney2klambday, this machine here has only 4gb20:36
lambdaysonney2k: :'(20:36
@sonney2klambday, for the buildbot, webserver, git etc etc20:36
lambdaysonney2k: I gotta remove the first large matrix completely!!20:36
lambdaylater parts are still affordable20:36
@sonney2kI guess that is why it fails20:37
@sonney2kbut as I said keep the code but take subsets and indicate that this is just limited to be quicker for a demo20:37
@sonney2kwhatever fernando's demo is doing it really eats CPU20:38
@sonney2k3 cpus busy20:38
lambdaysonney2k: I can comment out the CStatistics::log_det part (which is the main culprit) and say that for demo we have commented this out, and if you run this you'll get this value20:38
lambdaysonney2k: will try to do these things over this weekend20:39
@sonney2klambday, ok maybe show it heiko then :)20:40
@sonney2kvan51, I just read your code20:40
@sonney2kvan51, sorry that it took so long20:40
lambdaysonney2k: yeah.. he suggested that I redeuce the time to 10 mins max20:40
lambdaysonney2k: oh and do you think using const SGVector<float64_t>& vec=foo() instead of SGVector<float64_t> vec=foo() would help much??20:41
@sonney2kvan51, please use conv_feats->obtain_from_char(s_feats, 7-1, 7, 0, false);20:41
shogun-buildbotbuild #542 of nightly_default is complete: Failure [failed notebooks]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54220:41
lambdaysonney2k: the second just increases the refcount, right?20:43
lambdayso not much costly.. even when passing as params20:43
lambdayusing reference won't speed up a lot I guess20:43
@sonney2kvan51, apart from that the code looks good. Does it work on a small data set?20:44
@sonney2klambday, how often do you do the SGVector operation?20:45
@sonney2klambday, what it does is increase the refcount20:45
@sonney2kthis is done in a thread safe manner20:46
lambdaysonney2k: very very often20:46
@sonney2kthe rest is just copy of pointer and length20:46
@sonney2klambday, very very as in ?20:46
@sonney2k1e6 ?20:46
@sonney2k1e9 ?20:46
lambdaysonney2k: let me see.. 1000 iterations default, ~30 times... say 30k for one estimate20:47
lambdaysonney2k: at some ocassions, we do it 50000 iterations, ~30 times...20:47
lambdayanyway its just three added operations, won't cost much20:48
lambdaypassing around SGVector is done in a lot of places I guess20:48
@sonney2kgive me a single number please20:48
lambdayI didn't use reference, just simple SGVector20:48
shogun-buildbotbuild #41 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang34%20-%20thread%20analysis/builds/41  blamelist: Roman Votyakov <votjakovr@gmail.com>20:48
lambdaysonney2k: 30k to 5mil20:48
@sonney2k5e6 only?20:49
lambdayyeah20:49
@sonney2kthen this is nothing20:49
@sonney2kmaybe 100ms20:49
@sonney2kthat is not the bottleneck20:49
@sonney2kyou must be doing sth more expensive elsewhere20:50
lambdaysonney2k: alright.. yeah bottleneck is SGSparseMatrix*SGVector, which *has to be done* in pthreaded way whenever we can20:50
lambdaysonney2k: I will add that code soon20:51
@sonney2klambday, so this is the part that is done 5e6 times too right?20:51
lambdaysonney2k: yep20:51
@sonney2kwiking, I think LMNN is not converging in this example20:51
@sonney2kwiking, it is verry slow20:51
@sonney2kdid eat > 1 hour cpu time before being killed20:52
lambdaysonney2k: could you please let me know how much mine takes on that machine??20:53
@sonney2klambday, I will create some script that measures the time20:54
lambdaysonney2k: that would help a lot on which portions should I comment out.. thanks :)20:55
van51sonney2k: hey, sorry I was eating20:58
van51sonney2k: I used it on 500 examples and it seg'faulted: https://gist.github.com/van51/673352320:59
@sonney2kvan51, :(20:59
@sonney2kwe have no tests for anything for that old code21:00
@sonney2kvan51, it could be a big in SVMlight introduced due to some refactoring21:00
@sonney2kor the kernel21:00
van51sonney2k: I can look into the kernel21:00
@sonney2kvan51, please compile again in Debug mode and then get a valgrind trace21:01
@sonney2kthis is currently not very helpful21:01
van51sonney2k: ok, when I update it I will ping you21:01
@sonney2kvan51, regarding testing the kernel:21:02
@sonney2kvan51, you could just compute the kernel matrix21:03
@sonney2kvan51, and compare it to what you get with the hashed thing21:03
@sonney2kvan51, it is the same as if you used 7-grams21:03
@sonney2kvan51, in your hashedngramdocfeatures21:03
van51sonney2k: I think we had compared it kernel-wise21:04
@sonney2kvan51, if you can please do this in a test21:04
van51sonney2k: sure21:04
@sonney2kvan51, hmmhh well the the add_to_normal call in the kernel is exactly what add_to_dense vec is doing21:05
@sonney2knormal vector == the dense vector21:05
@sonney2kso you could test that too if it is that21:06
@sonney2kif it is svmlight I guess I have to figure out what it is :`-(21:06
van51sonney2k: haha, don't get sad before we get to that21:07
votjakovrlambday: could you please help me to reproduce the error? i've configured latest develop branch with cmake -DENABLE_TESTING=ON -DPythonModular=ON -DCMAKE_BUILD_TYPE=Debug and then run make and then ctest and it doesn't report this error21:17
lambdayvotjakovr: weird.. I did the same and it gave this error... wait I am trying again21:17
@sonney2kFGMs also time out21:17
@sonney2kalright so final evaluation emails are out21:18
@sonney2kcongrats to everyone who passed21:18
@sonney2k(actually all of you :P )21:18
lambdaysonney2k: yayyyy :D21:19
lambdaysonney2k: can I ask shogun buildbot to dance? :D21:20
@sonney2klambday, sure do it :)21:20
lambdayshogun-buildbot: dance21:21
shogun-buildbot<(^.^<)21:21
shogun-buildbot<(^.^)>21:21
shogun-buildbot(>^.^)>21:21
shogun-buildbot(7^.^)721:21
shogun-buildbot(>^.^<)21:21
lambdayhahaha :D21:21
@sonney2kheh21:21
@sonney2kvan51, keep me updated - I will be off now but read the logs21:22
* sonney2k off21:22
van51sonney2k: ok, I just updated the valgrind output btw21:22
van51sonney2k: https://gist.github.com/van51/673352321:22
@sonney2kvan51, at least you see the real location now21:23
van51sonney2k: yeah, I will look into it :)21:23
@sonney2kthanks21:23
@sonney2kok now for real21:23
* sonney2k Zzzzz..21:23
lambdayvotjakovr: I just did this again and it still gives the error... might be something wrong21:28
votjakovrlambday: hmm... ok, i'll try to find the bug21:33
lambdayvotjakovr: I did rebase this morning.. forgot to rebase this time... may be something you fixed in your PR this morning.. I am trying.. will let you know21:34
-!- FSCV [~FSCV@50.7.50.60] has joined #shogun21:34
-!- new_lido_ [~walid@41.218.175.70] has joined #shogun21:34
votjakovrlambday: ok, thanks21:34
lambdayvotjakovr: sorry roman still same error... :(21:39
lambdayif someone else can't reproduce this then something must be wrong with any of my local pkgs..21:40
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]21:43
-!- shogun-buildbot_ [~shogun-bu@7nn.de] has joined #shogun21:43
votjakovrlambday: heh, no problem, i'll try to review the code to find a possible bug... Btw travis also didn't catch it21:43
lambdayvotjakovr: possible that some of my local pkgs is doing something funny... ExactInferenceMethod only uses eigen3 right?21:44
votjakovrlambday: yeah21:45
lambdayweird!21:46
votjakovrindeed21:46
lambdayvotjakovr: let's ask someone else try this and if he too doesn't get any such things we can close this issue21:46
votjakovrlambday: ok :)21:49
-!- shogun-buildbot_ [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]21:53
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun21:53
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]21:53
wikingshogun-buildbot: force build --branch=develop 'nightly_default'21:53
shogun-buildbotbuild #543 forced21:53
shogun-buildbotI'll give a shout when the build finishes21:53
-!- lambday [67157f4c@gateway/web/freenode/ip.103.21.127.76] has quit [Ping timeout: 250 seconds]22:04
-!- lisitsyn [~lisitsyn@109-226-79-195.clients.tlt.100megabit.ru] has joined #shogun22:08
shogun-buildbotHey! build nightly_default #543 is complete: Exception [exception tutorial updatedoc notebooks upload]22:18
shogun-buildbotBuild details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54322:18
-!- votjakovr [~votjakovr@host-46-241-3-209.bbcustomer.zsttk.net] has quit [Quit: Good bye!]22:19
van51sonney2k: I believe the bug is in add_to_normal22:21
van51sonney2k: I called it externally (out of SVMLight) and it still caused a crush22:22
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]22:29
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun22:29
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]22:39
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun22:39
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Client Quit]22:42
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun22:42
wikingshogun-buildbot: force build --branch=develop 'nightly_default'22:43
shogun-buildbotbuild #544 forced22:43
shogun-buildbotI'll give a shout when the build finishes22:43
shogun-buildbotbuild #544 of nightly_default is complete: Failure [failed notebooks]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54422:53
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting]22:55
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun22:55
wikingshogun-buildbot: force build --branch=develop 'nightly_default'22:56
shogun-buildbotbuild #545 forced22:56
shogun-buildbotI'll give a shout when the build finishes22:56
shogun-buildbotHey! build nightly_default #545 is complete: Exception [exception updatedoc upload]23:01
shogun-buildbotBuild details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/54523:01
-!- FSCV [~FSCV@50.7.50.60] has quit [Quit: Leaving]23:38
-!- zxtx [~zv@c-50-133-182-155.hsd1.ma.comcast.net] has joined #shogun23:52
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 252 seconds]23:56
-!- van51 [~van51@athedsl-409794.home.otenet.gr] has quit [Ping timeout: 246 seconds]23:58
--- Log closed Sat Sep 28 00:00:18 2013

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!