IRC logs of #shogun for Wednesday, 2013-04-03

--- Log opened Wed Apr 03 00:00:18 2013
-!- FSCV [~FSCV@187.210.54.166] has quit [Read error: No route to host]00:13
-!- FSCV [~FSCV@187.210.54.166] has joined #shogun00:15
-!- FSCV [~FSCV@187.210.54.166] has quit [Quit: Leaving]00:23
-!- heiko [~heiko@027fc803.bb.sky.com] has quit [Quit: Leaving.]01:04
-!- heiko [~heiko@027fc803.bb.sky.com] has joined #shogun01:09
-!- heiko [~heiko@027fc803.bb.sky.com] has quit [Client Quit]01:10
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]01:23
-!- hoijui [~hoijui@dslb-088-075-046-171.pools.arcor-ip.net] has joined #shogun01:53
-!- hoijui [~hoijui@dslb-088-075-046-171.pools.arcor-ip.net] has quit [Client Quit]01:57
shogun-buildbot_build #342 of nightly_default is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/34204:20
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun06:10
shogun-notifier-shogun: Soeren Sonnenburg :master * 7d9539b / tests/integration/python_static/test_all.sh,tests/integration/blacklist: https://github.com/shogun-toolbox/shogun/commit/7d9539bb89c83ead12e7455601da71d74536876d06:10
shogun-notifier-shogun: disable broken HMM in tests for now06:10
shogun-buildbot_build #856 of deb2 - static_interfaces is complete: Failure [failed test python_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/856  blamelist: Soeren Sonnenburg <sonne@debian.org>06:24
shogun-buildbot_build #982 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/982  blamelist: Soeren Sonnenburg <sonne@debian.org>07:00
shogun-notifier-shogun: Soeren Sonnenburg :master * 2c1d6f0 / tests/integration/blacklist: https://github.com/shogun-toolbox/shogun/commit/2c1d6f022c8c1216bbb724df91ffa038e507378008:21
shogun-notifier-shogun: really blacklist HMM08:21
blackburnsonney2k: I don't remember, I have a feeling it is broken since 1.008:26
shogun-buildbot_build #857 of deb2 - static_interfaces is complete: Failure [failed test octave_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/857  blamelist: Soeren Sonnenburg <sonne@debian.org>08:27
@sonney2kblackburn, I hardly recall but IMHO for 1.0 we fixed all old tests (like we do now?)08:28
@sonney2kblackburn, ^ hurray08:28
@sonney2know we are at octave_static!08:28
blackburnsonney2k: heh08:28
@sonney2kblackburn, actually it is only failing w/ svmsgd and other linear methods that use sparse features08:30
@sonney2kb'cause python static doesn't support sparse features!08:30
@sonney2ksvmsgd & etc seem to be working but infact they just return08:30
shogun-notifier-shogun: Soeren Sonnenburg :master * 573b475 / tests/integration/ (4 files): https://github.com/shogun-toolbox/shogun/commit/573b475142a7e8a837eb36c34b9494ed8dc124b808:37
shogun-notifier-shogun: add blacklist support to all interfaces' integration tests08:37
@sonney2kblackburn, so only svmsgd stuff & svmlin stuff are broken08:39
@sonney2kand it seems we have a train time limit on this stuff08:39
@sonney2kof 0.5 secs...08:39
@sonney2kmight all be crap...08:39
@sonney2klets better compare directly with svmsgd08:39
shogun-buildbot_build #858 of deb2 - static_interfaces is complete: Failure [failed test octave_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/858  blamelist: Soeren Sonnenburg <sonne@debian.org>08:47
sonne|workblackburn: how about we blacklist all the failing stuff and add these as entry tasks?08:52
blackburnsonne|work: like  HMM? :D08:52
sonne|workI mean time is pressing a bit and it is not so difficult to compare the sgd executable directly with what shogun's sgd produces08:52
sonne|workok HMM is something I have to do08:52
sonne|workbut it is a matter of git bisect08:52
sonne|workit worked including tests at some point08:53
sonne|workso it not difficult to find out which commit broke things08:53
shogun-buildbot_build #983 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/983  blamelist: Soeren Sonnenburg <sonne@debian.org>09:02
shogun-notifier-shogun: Soeren Sonnenburg :master * e43606f / tests/integration/blacklist: https://github.com/shogun-toolbox/shogun/commit/e43606f9efc783cf4728d03ba4241f4224161a3509:02
shogun-notifier-shogun: blacklist failing SGD/SVMLin tests09:02
shogun-buildbot_build #984 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/984  blamelist: Soeren Sonnenburg <sonne@debian.org>09:17
shogun-buildbot_build #859 of deb2 - static_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/85909:28
shogun-notifier-shogun: Soeren Sonnenburg :master * a1a3884 / tests/integration/blacklist: https://github.com/shogun-toolbox/shogun/commit/a1a388408a0bff950856fa4d582448b79c23b4f309:37
shogun-notifier-shogun: blacklist Fisher/TOP kernels09:37
shogun-buildbot_build #985 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/985  blamelist: Soeren Sonnenburg <sonne@debian.org>09:46
shogun-notifier-shogun: Soeren Sonnenburg :master * 9b34a5a / tests/integration/ (2 files): https://github.com/shogun-toolbox/shogun/commit/9b34a5a16644ddd466d2e41e2d684bcdccbcbaec09:55
shogun-notifier-shogun: blacklist support for octave/python_modular integration tests09:55
blackburnsonne|work: http://tapkee-bootstrap.herokuapp.com/ bootstrap appears to be easy09:55
sonne|workblackburn: bootstrap?09:59
sonne|workwhat do you mean?09:59
sonne|workthe ML method?09:59
blackburnsonne|work: http://twitter.github.com/bootstrap/09:59
sonne|workok10:00
sonne|workit should be :D10:00
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has joined #shogun10:02
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has left #shogun []10:02
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has joined #shogun10:07
-!- heiko [~heiko@027fc803.bb.sky.com] has joined #shogun10:18
shogun-buildbot_build #986 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/986  blamelist: Soeren Sonnenburg <sonne@debian.org>10:20
-!- blackburn [~blackburn@46.20.65.51] has quit [Quit: Leaving.]10:22
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has joined #shogun10:26
escorciavHi, could someone help me with some doubts about latent svm code?10:27
sonne|workescorciav: wiking is your man - he wrote that stuff!10:34
escorciavThanks, I'm new with IRC. How can I contact to wiking?10:35
escorciavor wiking means that I need to read a Wiki10:36
escorciav???10:36
escorciavI click "query" on wiking user in the right tab, please let me know If I'm using bad the IRC chat10:41
escorciavThanks for your reply @sonne|work10:41
sonne|workescorciav: no he would respond if he was available...10:42
escorciavI have 3-5 questions, you recommend me to post all my question or wait for wiking request. I'm Sorry, if I'm bother you with my silly complains.10:47
-!- n4nd0 [~nando@n186-p150.kthopen.kth.se] has joined #shogun10:48
sonne|workescorciav: well post them - if we see wiking around we will make him talk to you or you write him an email (CC'ing the mailinglist)10:48
sonne|workn4nd0: gooood morning !10:48
n4nd0sonne|work: hey! good morning :)10:48
n4nd0I am finally back again!10:48
sonne|workno plane crashes or other incidents10:49
n4nd0sonne|work: no, not really10:49
sonne|work(except that we still have snow in berlin)10:49
n4nd0haha lot of snow here in Sweden too10:49
n4nd0I found none in Italy though :D10:49
escorciavThanks @sonne|work, I can wait 2-3 hour in IRC.10:50
n4nd0I have seen some commits with fixes for the kernels, the tests are fine now?10:50
sonne|workn4nd0: the kernels were actually more or less fine10:51
sonne|workn4nd0: what didn't work with oligokernel is that you wrote sth like OligoStringKernel(k=3, width=50)10:52
sonne|workand keyword arguments are not supported by seig10:52
sonne|workswig10:52
sonne|workbut somehow it silently ignored this!10:52
sonne|workand just used the default constructor10:52
n4nd0ouch, my bad. Sorry about that10:52
sonne|workno IMHO this is a bug...10:52
n4nd0I see there is only one HMM test failing now10:53
n4nd0but it seems it is already blacklisted10:54
shogun-buildbot_build #987 of deb3 - modular_interfaces is complete: Failure [failed test octave_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/987  blamelist: Soeren Sonnenburg <sonne@debian.org>10:54
-!- heiko [~heiko@027fc803.bb.sky.com] has left #shogun []10:55
-!- tom__ [2eda6d58@gateway/web/freenode/ip.46.218.109.88] has joined #shogun11:11
tom__hi all!11:11
n4nd0hey tom__11:11
tom__I have a question about CAlphabet::translate_from_single_order method. who'is interested ? :p11:12
sonne|workn4nd0: we now only have to fix the octave_modular stuff11:13
sonne|workthat should be rather easy11:13
sonne|worktom__: shoot!11:13
sonne|worktom__: I wrote that stuff but a decade back11:13
-!- tom__ [2eda6d58@gateway/web/freenode/ip.46.218.109.88] has quit [Ping timeout: 245 seconds]11:17
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has quit [Ping timeout: 245 seconds]11:17
n4nd0sonne|work: why do this cannot create instance errors appear in octave_modular?11:17
-!- blackburn [~lisitsin@mxs.kg.ru] has joined #shogun11:17
n4nd0Some names that have changed in the code and not updated in the tests?11:17
sonne|workn4nd0: one needs to check the scripts11:18
sonne|workn4nd0: we have an obsolete test suite11:18
n4nd0sonne|work: I am going to build for octave modular and see11:18
sonne|workwhich however is the only way to still do tests for static interfaces11:18
sonne|workand interfaces != python_modular11:18
sonne|workso when you go to shogun/tests/integration/octave_modular11:19
sonne|workyou will see some handcrafted .m scripts that run the tests11:19
sonne|workthese might need an update for all the changes we had with e.g. labels11:19
sonne|work$ grep Label *.m11:20
sonne|workclassifier.m:lab=Labels(classifier_labels);11:20
sonne|workregression.m:lab=Labels(regression_labels);11:20
sonne|workno wonder...11:20
n4nd0ok11:21
sonne|workn4nd0: would be cool if you have time to fix this11:29
sonne|workthe tests should work then11:29
n4nd0sonne|work: yes, let me take a look. I am building atm.11:29
sonne|workok11:29
sonne|workgood thing is that you don't have to recompile you only need to change these scripts11:29
sonne|workrecompile again I mean11:30
n4nd0it will be faster then11:30
blackburnescorciav: may be n4nd0 and I can help you somehow?11:34
blackburnsonne|work: n4nd0: have any of you tried -Og in gcc 4.8.0?11:35
n4nd0blackburn: mm no, let me check. Another level of optimization?11:35
blackburnn4nd0: yes debug - no optimization at all11:36
blackburnshould be fast11:36
blackburnbut probably just as fast as clang :D11:36
blackburnmonster gcc is damn slow11:36
blackburnsonney2k: I was thinking about swig problems and etc11:36
n4nd0hehe, the idea sounds nice in any case11:36
blackburnI consider one anti-pattern as a solution actually11:37
blackburnit is called 'string typization' you know11:37
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has left #shogun []11:37
blackburnget("width")11:37
sonne|workhaha11:37
sonne|workyou are arriving at the static interfaces :)11:37
blackburnsonne|work: no11:37
blackburnsonne|work: let me explain11:38
blackburnremember that idea I hade11:38
blackburnhad*11:38
blackburnsvm.parameter("width")11:38
blackburnetc11:38
blackburnand actually we *already* use it in modelselection11:38
blackburnthe only problem is that it won't work as I just realized :D11:38
blackburnsetters could work but not the getters11:39
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has joined #shogun11:39
-!- escorciav [ba574947@gateway/web/freenode/ip.186.87.73.71] has left #shogun []11:39
sonne|workblackburn: they could too if you pass stuff by reference11:39
blackburnwhat is declaration of such get?11:39
blackburn???& get(string name)11:39
sonne|workvoid getFoo(string name, type& value)11:40
blackburnhow would it look like in python?11:40
sonne|workgetFoo("width")11:40
blackburnand it would return type?11:40
sonne|worktype11:41
sonne|workyes11:41
blackburnhow does it dispatch the type?11:41
blackburnbut it won't work for java11:42
sonne|workactually it doesn't help at all11:43
blackburnsonne|work: I hate writing getters11:43
blackburn:D11:43
sonne|workfor getFoo(string name, type& value) to be wrapped the function with the type needs to be declared11:43
-!- lambday [0e8b614f@gateway/web/freenode/ip.14.139.97.79] has joined #shogun11:45
blackburnsonne|work: I have an idea how would it look like in C++ but not in other interfaces11:45
blackburnsonne|work: I'd like to review the API for shogun 3.011:51
blackburnsome ideas haunt me and I just have to realize them11:52
blackburnsonne|work: just like any other software initial API can appear bad but later you realize how to do that properly - I think that's time11:53
sonne|workI don't mind - git has branches you know :D11:54
blackburnsonne|work: reviewing API means discussion of API11:55
sonne|workso think of sth and then we can discuss11:55
sonne|workI don't currently know how to do it 'better'11:56
blackburnsonne|work: first is modelselection11:56
sonne|workagreed11:56
sonne|workbut I think we were all happy with what you proposed11:56
blackburnsonne|work: then we have to think about preprocessor chains11:56
blackburnall that append_preprocessor etc should be reviewed11:57
blackburnit should look easier11:57
blackburnnext is unite label+features11:57
sonne|workblackburn: but label & features -> unsupervised doesn't make sense11:59
sonne|workblackburn: better do the same with combined kernels/ features we do for preprocessors11:59
blackburnsonne|work: labels are optional11:59
-!- heiko [~heiko@nat-180-85.internal.eduroam.ucl.ac.uk] has joined #shogun12:16
blackburnsonne|work: http://divshot.github.com/geo-bootstrap/#forms12:21
n4nd0no idea what is going on with my compiler, I have tried three times to build shogun with octave-modular12:29
n4nd0c++: internal compiler error: Killed (program cc1plus)12:29
n4nd0Please submit a full bug report,12:29
n4nd0with preprocessed source if appropriate.12:29
blackburnn4nd0: you met out of memory probably12:34
n4nd0but the memory tracer didn't show that12:35
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]12:55
n4nd0I cannot believe it, I just can't build with octave_modular it seems12:56
-!- blackburn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]13:26
-!- blackburn [~lisitsin@mxs.kg.ru] has joined #shogun13:32
-!- heiko [~heiko@nat-180-85.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.]13:38
-!- heiko [~heiko@nat-180-85.internal.eduroam.ucl.ac.uk] has joined #shogun13:40
sonne|workn4nd0: try to compile with --disable-optimization13:42
heikosonne|work, blackburn any comments on the nu-svr extension in the PR?13:42
sonne|worknu svr is kind of useless IMHO13:43
sonne|workIIRC it was much slower doing the optimization13:43
sonne|workbecause it started with non-zero alphas13:43
sonne|workbut I might err...13:43
heikosonne|work: it is slower13:43
heikonoticed that13:43
heikobut people may want to have it anyway, like the guy on the mailing list?13:44
heikoso why not offer both? we do that for SVC also13:44
sonne|workheiko: if it doesn't break any of our tests why not13:44
sonne|workbtw, why do you have C end nu in your example?13:44
n4nd0sonne|work: I was compiling with disable-optimization already. It seems my system is not using swap memory although I think I did a partition for that. I have to check how to fix this.13:44
heikosonne|work: it doesnt, is there anything else I need to do except for passing this through to the LibSVR13:45
sonne|workn4nd0: how much memory do you have? for octave modular you will need 3-4GB13:45
n4nd04GB13:45
sonne|workn4nd0: then close all applications!13:45
sonne|workand compile again13:46
n4nd0I had just this terminal running irssi and a couple of tabs in the browser13:46
n4nd0I think I should better take a look and make use of the swap13:46
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has joined #shogun13:47
sonne|workheiko: I don't understand? why should libsvm need a C if you have a nu?13:47
sonne|workIIRC you need some size of the epsilon tube and the nu13:47
sonne|workerr no the tube got estimated too13:48
sonne|workso just nu right?13:48
sonne|workn4nd0: btw please twitter that we applied at GSoC13:49
sonne|workn4nd0: octave modular compiles on our buildbot (also only 4GB memory)13:49
sonne|workso it should work just fine13:49
n4nd0I guess the OS needs also some memory13:50
n4nd0the buildbot probably doesn't have a graphical OS while I am running one13:50
n4nd0it could make sense13:50
-!- sumit_ [73f91219@gateway/web/freenode/ip.115.249.18.25] has joined #shogun13:50
heikosonne|work: nu replaces the epsilon in svr13:51
heikobut there remains a C in the opt. problem13:51
heikoif I remember correctly13:51
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has quit [Ping timeout: 245 seconds]13:53
sonne|workno it replaces C & epsilon13:53
heikosonne|work: oh, I better do some reading then :)13:55
sonne|workI am not sure either13:55
sonne|workfor sure nu-svm no C13:56
heikosonne|work: I thinnk the C remais13:56
heikosvr is different13:56
heikoin svc I agree13:56
sonne|workahh ok so the nu just controls percentage of errors then13:56
heikosonne|work, yes13:57
heikoTraining ?-Support Vector Regression: Theory and Algorithms13:57
heikocontains a comparison13:57
blackburnheiko: I am not sure what to comment :)13:57
sonne|workso it has some mapping to epsilon13:57
heikoblackburn: I did not build the libsvm wrapper so I wasnt sure whether its just enough to pass the different solver type to libsvm13:58
heikosonne|work, blackburn, ok will merge then ...13:58
sonne|workok then13:58
sonne|workheiko: well if you compare with libsvm's nu-svr and results are the same then why not13:59
n4nd0sonne|work, heiko, blackburn : retweet guys13:59
heikosonne|work, compare? I am using libsvr13:59
heikoyou mean calling it directly?13:59
-!- ZeThomas [~thomaspr@2a02:2c40:100:a000:1c78:deb5:6522:b5ca] has joined #shogun14:00
heikoyes, I planned to do this, also for libsvr itself, so that we have a unit test that checks the output for a simple case14:00
blackburnn4nd0: may be tweet it from ShogunToolbox?14:00
n4nd0mm, that would require me to remember the pass :D14:01
n4nd0found it14:01
ZeThomashey blackburn, I'm still struggling with the implementation of my own kernel14:02
blackburnZeThomas: hey14:02
blackburnask me14:02
sonne|workZeThomas: or read the excellent tutorial ;)14:02
ZeThomasI recompiled with the appropriate flags set, and I think I kind of got it figured out14:02
ZeThomassonne|work, where's this tutorial?14:03
sonne|workhttp://www.shogun-toolbox.org/doc/en/current/developer_tutorial.html14:03
sonne|workhow to implement your own kernel14:03
blackburnsonne|work: no, directors14:03
sonne|workReverseLinearKernel14:03
blackburnZeThomas: that's for shogun internal C++ stuff14:03
n4nd0tweet done14:03
sonne|workdirectors ahh14:03
ZeThomasah ok, I got already scared by the amount of C on that page :)14:04
blackburnZeThomas: yeah that was the naming in 197614:04
blackburnZeThomas: ah btw I forgot to ask you - is your dataset big?14:04
ZeThomasnot yet, but I can be14:05
blackburnif your kernel takes significant time and kernel matrix can fit into the memory it is better to use CustomKernel still14:05
blackburnbut if you have >10000 training vectors directors is the only way I see14:06
ZeThomasblackburn: oh, so for small datasets CustomKernel is better?14:06
blackburnZeThomas: yeah sorry I didn't mention that14:07
ZeThomasblackburn: np, but I can still implement it through DirectorKernel right, for scaleability14:08
blackburnZeThomas: yes14:08
blackburnZeThomas: so have you got any problems with implementing it?14:08
ZeThomasok, so this is my basic set-up, don't know if it's any good: http://pastebin.com/P4771fNP14:08
blackburnZeThomas: looks legit14:10
ZeThomasso I then do kernel.init(trainset, trainset); svm = SVMLight(C, kernel, labels); svm.train()14:10
ZeThomasthis works14:10
ZeThomasbut if after, I do kernel.set_rhs(testset); rhs = kernel.get_rhs(); svm.apply(rhs)14:11
blackburnohh14:11
blackburnno need to do apply(rhs)14:12
blackburnI think that's enough to just svm.apply()14:12
ZeThomasok, because this throws an error: terminate called after throwing an instance of 'Swig::DirectorMethodException'14:12
lambdayheiko: hi14:13
heikolambday: hi!14:13
blackburnZeThomas: yeah that should not happen but directors are so experimental14:13
ZeThomasoh ok, it works with apply()14:14
lambdayheiko: I modified the code as blackburn suggested and committed.. could you please have a look?14:14
lambdayit works both for new and older eigen14:14
ZeThomasblackburn: cool, thanks a lot for the help!14:14
heikolambday: sure14:14
blackburnZeThomas: you are welcome14:14
ZeThomasblackburn: btw is there a huge performance increase when using CustomKernel for smaller datasets?14:15
lambdayheiko: for older eigen, I used SparseLLT, which does not use permutation and is a lot slower..14:15
heikolambday: okay, mmh14:15
blackburnZeThomas: yeah it may be significant if your kernel is slow14:16
heikoso can one do the permutation by hand in this case?14:16
lambdaybut if its >= 3.1.0, it uses simplicialLLT which performs amd14:16
lambdayand a lot faster14:16
lambdayheiko: that I haven't tried14:16
heikolambday: thats good!14:16
ZeThomasblackburn: so I should consider implementing it through that right now? train data is at a measly 1000 max14:17
heikolambday:  the whole point is to do the permutation, if this is not done, there is no way of computing the logdet for large matrices14:17
blackburnZeThomas: oh then I'd rather precompute everything14:17
lambdayheiko: for the unit test, I used a 1000x1000 sparse matrix, using simplicialLLT, it takes 112 ms in my machine, but using SparseLLT it takes around 7400 ms14:17
heikolambday: is it hard to compute the permutation and apply it by hand?14:17
heikolambday: that sounds reasonable14:17
heikoproblem is more the memory14:17
ZeThomasblackburn: so you mean compute the upper triangular, and populate that in my CustomKernel object?14:18
lambdayheiko: yes..14:18
-!- sumit_ [73f91219@gateway/web/freenode/ip.115.249.18.25] has quit [Ping timeout: 245 seconds]14:18
blackburnZeThomas: yes14:18
lambdayheiko: I need to check the older eigen sparse module..14:19
ZeThomasblackburn: and then the whole matrix of k(x,y) for x \in train and y \in test?14:19
heikolambday: the gsoc projects will probably need those permutations at some point anyway ...14:19
blackburnheiko: lambday: I use SimplicialLDLT in eigen >3.1.0 and SimplicialCholesky in eigen <3.1.014:19
heikolambday: there is a class for the permutation14:19
blackburnZeThomas: yes exactly14:19
blackburnZeThomas: two matrices14:19
lambdayheiko: yes... ordering14:19
ZeThomasblackburn: ok, I see... final question: it's ok though to overwrite the initial gram matrix by this second one, in the same object? or would you advise against that?14:20
heikolambday:  http://eigen.tuxfamily.org/dox-devel/classEigen_1_1AMDOrdering.html14:21
heikodoenst sound too hard, does it? :)14:21
blackburnZeThomas: I don't think you will be able to overwrite it14:22
heikolambday: I think it should be fine if you just put that in front of the sparse LLT cholesky14:22
ZeThomasblackburn: how do I set it then, in my kernel?14:22
lambdayheiko: yes.. simplicialLLT uses this class before computing.. but I am not sure if it was there in eigen <3.1.0..14:23
lambdayheiko: let me check14:23
heikolambday: ah now I see14:23
heikothis class might not have existed14:23
lambdayheiko: yes that's the problem14:23
heikoblackburn: can't we ask users for eigen3.1? :D14:23
lambdayheiko: eigen >= 3.1.0 uses ordering by default14:23
lambdayI checked the source14:23
blackburnZeThomas: you create CustomKernel from your matrix14:24
lambdayheiko: that would save a lot of pain :D14:24
heikolambday: ok good, so lets find out where the ordering came in14:24
blackburnZeThomas: you can just svm.set_kernel(CustomKernel(train_matrix))14:24
blackburnthen train and svm.set_kernel(CustomKernel(test_matrix))14:24
blackburnand then apply()14:24
ZeThomasblackburn: so a new kernel object then, and replace the kernel in the svm14:24
blackburnyes that would be simplest I think14:24
heikolambday, blackburn we could just output a runtime warning if this method is called without the permutation14:24
ZeThomasblackburn: ah, okay, i see14:25
lambdayheiko: umm... yes14:25
blackburnheiko: I don't know - it is still in 12.04 LTS14:25
heikoblackburn: so maybe just make it madatory?14:25
blackburn3.0.?14:25
blackburn3.0.x14:25
heikoah14:25
heikoso better not14:25
blackburnheiko: that's pretty hard requirement14:26
heikoblackburn: agree14:26
heikolambday: so pls have a look whether the ordering class is in 3.0.x14:26
blackburnheiko: what is the issue with ordering?14:26
lambdayheiko: okay let me check14:26
heikoif yes, include it in the code, if not, just put a warning "sparse Cholesky is computed without reordering - might take long or run out of memory. Install Eigen >3.1 in order to use reordering ..."14:27
heikosomething like that14:27
heikolambday: SG_SWARNING("CStatistics::log_det(): Blablablla")14:28
lambdayheiko: alright....14:28
heikolambday: code is great work btw!14:28
lambdayheiko: thanks man... you and blackburn are so helpful...14:29
heikoblackburn: just knows everything :D14:29
lambdayheiko: hey good news seems there is a Amd.h in my older eigen... not sure it its 3.0.x... let me check14:30
lambdayheiko: yes its there in 3.0.6.. I'm trying to use it then14:40
heikolambday: great! make sure that you also somehow test it to produce the same results as your other test14:41
lambdayheiko: alright... :)14:42
lambdayheiko: as blackburn said, we could have used SimplicialCholesky for eigen < 3.1.0 instead but we need the the access the diagonal.. in SimplicialCholesky I am not sure if we can do that... m_diag is protected and there is no get method14:51
heikolambday: what about the ordering class then?14:52
lambdayheiko: I just checked, the older version uses amd in simplicialCholesky class...14:53
heikook good14:53
lambdaywith permutation and all...14:53
heikoso if there is a way to access the diagonal we can use this one14:53
heikoshould be possible14:53
lambdayheiko: but SimplicialCholesky doesn't profide a matrixL sort of method that SimplicialLLT does...14:54
lambdayheiko: yes14:54
heikolambday: if it doesnt give its diagonal, maybe just steal the code14:55
heikoto permute etc14:55
lambdayheiko: ummm... steal the code? as in?14:56
heikolambday: just do the permutation by hand as they do it in eigen14:56
lambdaythere is another way..14:56
heikothat is?14:56
lambdayheiko: yes yes... that is the another way that I was saying :D14:57
heikook14:57
lambdayheiko: let me try...14:57
heikolambday: have a try14:57
heikoif it turns out to be too annoying, well go with the warning14:58
lambdayheiko: hmm...14:58
n4nd0sonne|work: it finally compiled15:00
sonne|workn4nd0: hurray15:01
sonne|workwhat did you do?15:01
n4nd0sonne|work: my OS with a couple of terminals and chromium with a few tabs runs with 0.7GB. During compilation the peak has been 3.7GB RAM and 1.3GB swap15:01
sonne|workyeah15:01
n4nd0isn't a bit too much?? :)15:02
-!- ZeThomas [~thomaspr@2a02:2c40:100:a000:1c78:deb5:6522:b5ca] has quit [Quit: Leaving]15:04
sonne|workn4nd0: yes it is15:09
sonne|workthis is due to swig & massive use of templates (in shogun & octave)15:09
n4nd0sonne|work: so no real way to cut it down if we want to keep templates?15:12
sonne|workn4nd0: well yes - if we could implement a splitcpp program15:15
sonne|workor enhance swig such that it splits this huge file into say 10 files15:16
sonne|workthen compile time would go down and so would memory usage15:16
n4nd0no idea what splitcpp is15:16
sonne|workn4nd0: a program that would split up any .cpp file into multiple small ones15:17
sonne|worksacrificing static functions15:17
n4nd0aham15:17
sonne|worksomething to be written15:17
n4nd0static functions or inline?15:17
n4nd0why would split in several files sacrifice static?15:17
n4nd0gtg now, see you later15:22
-!- n4nd0 [~nando@n186-p150.kthopen.kth.se] has quit [Quit: leaving]15:22
sonne|workhttp://www.lazycplusplus.com/download.html15:31
sonne|workmaybe that thing can help us...15:31
blackburnsonne|work: how?15:33
sonne|workhttp://www.hwaci.com/sw/mkhdr/15:33
blackburnsonne|work: I can't get what ist the help of generating header15:34
sonne|workblackburn: we need to split up the wrapper file to multiple files15:34
sonne|workso we need .h's for function / class definitions15:34
sonne|workand functions in different file15:35
sonne|works15:35
blackburnso divide .cpp15:35
blackburnand generate headers?15:35
sonne|workblackburn: if you look at the *wrap.cxx you will see that it contains a huge jump table at the end15:37
sonne|workall these entries are function pointers to some static (local) functions15:38
sonne|workso if we would make these functions non-static15:38
sonne|worksplit up the file somehow15:38
blackburnI see15:38
sonne|workand have one .cpp with that jump table15:38
sonne|workthat would include all the function *definitions*15:39
sonne|workwe would be fine15:39
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun15:49
shogun-notifier-shogun: Soeren Sonnenburg :master * 2e9367e / tests/integration/octave_modular/ (2 files): https://github.com/shogun-toolbox/shogun/commit/2e9367e561366376486135abf464f8d6a2155c8c15:49
shogun-notifier-shogun: use binary/regression labels in octave integration tests15:49
-!- blackburn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]16:11
-!- lambday [0e8b614f@gateway/web/freenode/ip.14.139.97.79] has quit [Ping timeout: 245 seconds]16:17
-!- ahcorde [5813e5d6@gateway/web/freenode/ip.88.19.229.214] has joined #shogun16:24
shogun-buildbot_build #988 of deb3 - modular_interfaces is complete: Failure [failed test octave_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/988  blamelist: Soeren Sonnenburg <sonne@debian.org>16:27
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has joined #shogun16:32
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has quit [Ping timeout: 256 seconds]16:37
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has quit [Quit: Page closed]16:42
-!- travis-ci [~travis-ci@ec2-50-19-59-96.compute-1.amazonaws.com] has joined #shogun16:48
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/601708216:48
-!- travis-ci [~travis-ci@ec2-50-19-59-96.compute-1.amazonaws.com] has left #shogun []16:48
-!- ZeThomas [~thomaspr@2a02:2c40:100:a000:1c78:deb5:6522:b5ca] has joined #shogun17:02
ZeThomasI'm getting this weird behaviour from shogun lately (2.1.0 - python-modular --enable-swig-directors)17:06
ZeThomasMost of the time it stops after a random number of iterations when I run my tests, without output, just "exit code 139"17:07
ZeThomasand this one time I then get this: http://pastebin.com/JsLAFTFY17:08
@sonney2kZeThomas, looks like you are getting random memory corruptions17:17
ZeThomassonney2k, how is this caused?17:18
@sonney2kno idea17:20
shogun-notifier-shogun: Soeren Sonnenburg :master * 8587385 / tests/integration/octave_modular/ (3 files): https://github.com/shogun-toolbox/shogun/commit/858738543784ee7e269f4cff00bf50df6a20800c17:20
shogun-notifier-shogun: a couple of fixes for the tests17:20
@sonney2khard to debug17:20
ZeThomasso there's nothing I can do about this?17:23
@sonney2kZeThomas, show us the code / a minimal example to reproduce this17:25
ZeThomasok, on it17:25
-!- ahcorde [5813e5d6@gateway/web/freenode/ip.88.19.229.214] has quit [Ping timeout: 245 seconds]17:26
ZeThomashere you go: http://pastebin.com/1GwtYEC717:29
ZeThomasoh, and test_learner_fraction basically calls learner.test([...]) with random shuffles of the data17:32
ZeThomasI'm running it now with a different learner, that uses features and the LinearKernel, and that seems to work fine17:33
ZeThomasso it's because of my CustomKernel?17:33
shogun-notifier-shogun: Soeren Sonnenburg :master * 395b4dc / tests/integration/octave_modular/ (5 files): https://github.com/shogun-toolbox/shogun/commit/395b4dce2b1dd6208e0c8c2a37f9ad29e052248617:36
shogun-notifier-shogun: fix further octave modular tests17:36
@sonney2kZeThomas, wait you cannot derive from CustomKernel17:37
@sonney2konly from DirectorKernels17:37
@sonney2kthere is no way of overloading shogun's custom kernel17:37
ZeThomasoh17:37
@sonney2kI wonder why swig didn't complain17:37
@sonney2kZeThomas, all of the core of shogun is in C++17:37
@sonney2kfor speed17:37
@sonney2kso we only allow very few classes named Director* to be overloaded from python17:38
ZeThomasok, I see17:38
@sonney2ksry, gtg17:38
ZeThomasso what is the best strategy then to do?17:38
-!- travis-ci [~travis-ci@ec2-50-19-59-96.compute-1.amazonaws.com] has joined #shogun17:47
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/601943217:47
-!- travis-ci [~travis-ci@ec2-50-19-59-96.compute-1.amazonaws.com] has left #shogun []17:47
-!- travis-ci [~travis-ci@ec2-107-22-45-48.compute-1.amazonaws.com] has joined #shogun17:57
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/601980017:57
-!- travis-ci [~travis-ci@ec2-107-22-45-48.compute-1.amazonaws.com] has left #shogun []17:57
-!- ZeThomas [~thomaspr@2a02:2c40:100:a000:1c78:deb5:6522:b5ca] has quit [Quit: Leaving]18:06
shogun-buildbot_build #989 of deb3 - modular_interfaces is complete: Failure [failed test octave_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/989  blamelist: Soeren Sonnenburg <sonne@debian.org>18:08
shogun-buildbot_build #990 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/990  blamelist: Soeren Sonnenburg <sonne@debian.org>18:13
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has joined #shogun19:40
-!- blackburn [~blackburn@83.234.54.216] has joined #shogun19:59
-!- sumit [73f91219@gateway/web/freenode/ip.115.249.18.25] has quit [Ping timeout: 245 seconds]20:34
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]20:36
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun20:55
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun21:00
shogun-notifier-shogun: Soeren Sonnenburg :master * 680c8c6 / / (5 files): https://github.com/shogun-toolbox/shogun/commit/680c8c6455b0832a03ec5d73db96b669feb2197621:00
shogun-notifier-shogun: fix remaining octave_modular tests and use same default values in21:00
shogun-notifier-shogun: LAKernel no matter what constructor is used21:00
@sonney2kn4nd0, cross you fingers21:00
n4nd0sonney2k: crossed, together with toes21:00
@sonney2kn4nd0, if that all works all that is left to do is switch to the new development model and put the workshop on the website front page and announce it properly21:08
@sonney2kthen finger & toes crossed if GSoC happens w/ us we can fully concentrate on that21:09
@sonney2kn4nd0, if you have time please try to get versioning into the shogun website...21:11
n4nd0sonney2k: versioning for the articles I understand21:20
n4nd0the thing of storing a history of the changes21:20
shogun-buildbot_build #696 of cyg1 - libshogun is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/696  blamelist: Soeren Sonnenburg <sonne@debian.org>21:22
shogun-notifier-shogun: Heiko Strathmann :master * 40cf905 / tests/unit/features/DataGenerators_unittest.cc: https://github.com/shogun-toolbox/shogun/commit/40cf905896f0df62fc8f20234fef0494b17d8f4021:38
shogun-notifier-shogun: lowered accuracy a bit since test failed locally21:38
shogun-notifier-shogun: Heiko Strathmann :master * 6cbec42 / src/shogun/classifier/svm/LibSVM.cpp,src/shogun/classifier/svm/LibSVM.h: https://github.com/shogun-toolbox/shogun/commit/6cbec422f3ec8876d746e6bb4ed71d7afca50be921:38
shogun-notifier-shogun: cleaner handling of solver type21:38
shogun-notifier-shogun: Heiko Strathmann :master * b5ca129 / src/shogun/regression/svr/LibSVR.cpp,src/shogun/regression/svr/LibSVR.h: https://github.com/shogun-toolbox/shogun/commit/b5ca129990ecc05f55795ef404a74911b6914dbb21:38
shogun-notifier-shogun: added the possibility to use nu-svr21:38
shogun-notifier-shogun: Heiko Strathmann :master * f5c2be0 / examples/undocumented/libshogun/ (2 files): https://github.com/shogun-toolbox/shogun/commit/f5c2be04631be209c7430acbb1454862bf55672321:38
shogun-notifier-shogun: added simple example for libsvr21:38
shogun-notifier-shogun: Heiko Strathmann :master * e422c66 / examples/undocumented/libshogun/regression_libsvr.cpp: https://github.com/shogun-toolbox/shogun/commit/e422c6660128c577cf0ad548c7e4d570040620a221:38
shogun-notifier-shogun: slightly midified example to usee NU-SVR21:38
shogun-notifier-shogun: Heiko Strathmann :master * 630d41e / src/shogun/regression/svr/LibSVR.h: https://github.com/shogun-toolbox/shogun/commit/630d41ec2c89ec80b0f716908e3dd6b94b001dca21:38
shogun-notifier-shogun: updated class documentation for libsvr21:38
shogun-notifier-shogun: Heiko Strathmann :master * c71a337 / src/shogun/regression/svr/LibSVR.cpp,src/shogun/regression/svr/LibSVR.h: https://github.com/shogun-toolbox/shogun/commit/c71a337d6d985d74110f0a6577fd81f53dbac7a221:38
shogun-notifier-shogun: minor documentation/copyright update21:38
shogun-notifier-shogun: Heiko Strathmann :master * c14e37a / src/NEWS: https://github.com/shogun-toolbox/shogun/commit/c14e37a3e273b43b491a64de5008f610c992a4c821:38
shogun-notifier-shogun: mention nu-svr in the news21:38
shogun-notifier-shogun: Heiko Strathmann :master * da3bb9c / src/shogun/regression/svr/LibSVR.cpp: https://github.com/shogun-toolbox/shogun/commit/da3bb9c0ef2b0eff637c8abb6875eeabd59fc4e721:38
shogun-notifier-shogun: fixed a copy/paste bug21:38
-!- heiko [~heiko@nat-180-85.internal.eduroam.ucl.ac.uk] has left #shogun []21:40
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 255 seconds]21:49
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun21:51
shogun-buildbot_build #991 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/99121:55
shogun-buildbot_build #697 of cyg1 - libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/69721:58
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving]22:13
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun22:27
-!- heiko [~heiko@027fc813.bb.sky.com] has joined #shogun23:12
shogun-notifier-shogun: Heiko Strathmann :master * ad6f2b6 / src/shogun/statistics/ (2 files): https://github.com/shogun-toolbox/shogun/commit/ad6f2b6946551183f22693e31eaeb9864da9338b23:19
shogun-notifier-shogun: removed python_modular compile warning23:19
shogun-notifier-shogun: Heiko Strathmann :master * 43e954f / src/shogun/statistics/ (2 files): https://github.com/shogun-toolbox/shogun/commit/43e954fca0cb15a2cc8858d756db0522c5da87b923:19
shogun-notifier-shogun: Merge pull request #963 from karlnapf/master23:19
shogun-notifier-shogun:23:19
shogun-notifier-shogun: fix a warning23:19
--- Log closed Thu Apr 04 00:00:19 2013

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!