IRC logs of #shogun for Monday, 2012-06-25

--- Log opened Mon Jun 25 00:00:07 2012
n4nd0blackburn: after comparing the results with svm-struct, I am more confident that the algo is ok :)00:24
blackburncool00:24
blackburnn4nd0: so coincides?00:45
n4nd0blackburn: the values of the accuracy are similar00:46
n4nd0it's not a very good test to rely on it probably00:46
blackburnaccuracies are bad to compare..00:46
n4nd0but but it feels better at least00:46
n4nd0yeah I know ... but to compare the w is not going to work out directly00:46
blackburnwhy?00:47
n4nd0because it is not exactly the same problem00:47
n4nd0and I have to look a bit more on their code where the differences are00:47
n4nd0and what similarity to expect00:47
blackburnheh00:48
n4nd0in short it is something like00:52
n4nd0I compare with svm-struct 1-slack, primal formulation00:52
n4nd0something like00:52
n4nd0min ||w||^2 + C/n * slack00:52
n4nd0we do something like00:52
n4nd0min ||w||^2 + C * sum_i slack_i00:53
n4nd0there's a theorem in the paper to say that the same w is solution of both problems when00:53
n4nd0slack = sum_i slack_i00:53
n4nd0what seems a bit obvious to tell the truth00:54
blackburnshould be different C only??00:54
n4nd0C is not a problem, I can change it00:55
n4nd0the problem is that in their formulation00:55
n4nd0slack is just one variable in the optimization vector00:55
n4nd0mine it is a group of variables00:55
n4nd0and probably sum_i slack_i (result of my problem) is not the same as slack (result of their problem)00:56
n4nd0do you see what I mean to say?00:56
blackburnhow can that be?00:56
blackburnit looks like both are just sums of slacks00:56
blackburnwhat is different?00:56
n4nd0the degrees of freedom you have to do the optimization I think00:57
blackburnmin ||w||^2 + C/n * slack00:58
blackburnwhat is slack here?00:58
n4nd0from the optimization point of view, I think it is just another variable00:59
n4nd0what it represents, I am not sure00:59
n4nd0I have not looked into it yet00:59
blackburnonly one?01:00
n4nd0yeah, one slack instead of one for each example01:00
blackburnsounds pretty crazy hehe01:01
n4nd0maybe :)01:01
blackburnhttp://svmlight.joachims.org/svm_struct.html01:01
blackburnthis one?01:01
n4nd01-slack formualtion is called01:03
n4nd0http://tfinley.net/research/joachims_etal_09a.pdf01:05
n4nd0page 901:05
blackburnwhoo01:06
blackburnn4nd0: cool01:08
blackburnn4nd0: do you know eigen library?01:08
n4nd0I have read something about it yes, why?01:09
blackburnwant to link it with shogun01:09
blackburneigen301:09
blackburnthis would make me able to drop off some crappy code like arpack wrapper01:10
-!- heiko [~heiko@host86-174-151-208.range86-174.btcentralplus.com] has joined #shogun01:38
CIA-18shogun: Sergey Lisitsyn master * r0587779 / (11 files in 5 dirs): Merge pull request #597 from pluskid/multiclass (+24 more commits...) - http://git.io/eRBWFw01:49
-!- heiko [~heiko@host86-174-151-208.range86-174.btcentralplus.com] has quit [Ping timeout: 264 seconds]01:52
shogun-buildbotbuild #1100 of libshogun is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/1100  blamelist: Chiyuan Zhang <pluskid@gmail.com>01:53
n4nd0good night guys01:53
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving]01:53
shogun-buildbotbuild #1101 of libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/110101:57
CIA-18shogun: Sergey Lisitsyn master * r1381692 / (10 files in 4 dirs): Added Multitask Logistic Regression - http://git.io/DSJA6g02:06
CIA-18shogun: Sergey Lisitsyn master * r43fe9aa / (10 files in 4 dirs): Merge branch 'slep' of git://github.com/lisitsyn/shogun - http://git.io/Qy5Q7Q02:06
-!- blackburn [~blackburn@31.28.43.76] has quit [Ping timeout: 252 seconds]02:26
-!- wiking [~wiking@78-23-189-112.access.telenet.be] has joined #shogun02:51
-!- wiking [~wiking@78-23-189-112.access.telenet.be] has quit [Changing host]02:51
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun02:51
-!- K0stIa [~kostia@alt2.hk.cvut.cz] has joined #shogun04:08
-!- K0stIa [~kostia@alt2.hk.cvut.cz] has left #shogun []04:08
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]08:12
-!- wiking [~wiking@b252h112.ugent.be] has joined #shogun08:46
-!- wiking [~wiking@b252h112.ugent.be] has quit [Changing host]08:46
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun08:46
-!- uricamic [~uricamic@2001:718:2:1634:2806:9057:c143:ab58] has joined #shogun08:49
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun10:04
-!- heiko [~heiko@host86-174-151-208.range86-174.btcentralplus.com] has joined #shogun10:22
-!- alexlovesdata [c25faea9@gateway/web/freenode/ip.194.95.174.169] has joined #shogun11:04
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 244 seconds]11:31
@sonney2kheiko, not to mix up stuff again tomorrow we meet at 11 UTC right?11:47
heikosonney2k, ehm :)11:48
heikodate correct, let me check the time again :)11:48
@sonney2kheh11:48
heikojep11:48
@sonney2kmail sent11:49
heikonice11:50
@sonney2kwiking, any news on your PR?11:50
wikingmeet?11:50
wikingsonney2k: there's a meeting tomorrow at 11 UTC?11:50
@sonney2kyes12:05
-!- contact2 [9320543b@gateway/web/freenode/ip.147.32.84.59] has joined #shogun12:13
-!- blackburn [~blackburn@31.28.43.76] has joined #shogun12:31
blackburntomorrow?12:33
blackburn25 is today?12:33
blackburnwtf12:33
blackburn:D12:33
-!- pluskid [~pluskid@111.120.22.158] has joined #shogun12:42
@sonney2kblackburn: argh please send a correction12:48
@sonney2kthx12:57
pluskidsonney2k: what are the steps to export a template class to python_modular?12:57
@sonney2kpluskid well include the .h files and then use ÷template12:59
pluskidsonney2k: any example?12:59
pluskidah, maybe I can look at CDenseFeatures12:59
@sonney2kyes13:00
@sonney2kI am typing on my mobile13:00
pluskidsonney2k: thanks!13:00
@sonney2kso it's a bit hard to write much13:00
pluskidyeah13:01
pluskidjust received the correction email of refined meeting date13:02
CIA-18shogun: Sergey Lisitsyn master * rf765117 / (4 files in 2 dirs): A bunch of fixes for tree part of SLEP - http://git.io/_bI8nA13:02
CIA-18shogun: Sergey Lisitsyn master * rcf577de / src/shogun/kernel/Kernel.cpp : Removed warning in kernel - http://git.io/z4mXfQ13:02
CIA-18shogun: Sergey Lisitsyn master * ra45e370 / src/shogun/transfer/multitask/Task.cpp : Rearranged initialization in CTask - http://git.io/g1fZLw13:07
-!- pluskid [~pluskid@111.120.22.158] has quit [Quit: Leaving]13:12
CIA-18shogun: Sergey Lisitsyn master * rd3db2a6 / src/shogun/classifier/svm/OnlineLibLinear.cpp : Removed warning in OnlineLibLinear - http://git.io/J_8WKA13:13
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking]13:15
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun13:22
-!- nickon [~noneedtok@dD5774105.access.telenet.be] has joined #shogun13:23
-!- heiko [~heiko@host86-174-151-208.range86-174.btcentralplus.com] has quit [Quit: Leaving.]13:41
-!- pluskid [~pluskid@1.204.127.18] has joined #shogun13:54
pluskidHi all14:10
pluskidhas anyone got compile error with the latest code?14:10
pluskidhttp://pastebin.com/n4BV9S0T14:10
blackburnsegmentation fault??14:10
pluskidyeah14:11
blackburnno I've never ever seen that14:11
pluskidupdate again and try re-compiling14:11
pluskidI also didn't see that before today14:11
-!- pluskid [~pluskid@1.204.127.18] has quit [Ping timeout: 255 seconds]14:21
-!- pluskid [~pluskid@202.130.113.141] has joined #shogun14:22
n4nd0pluskid: hi, is the error fixed after re-compilation?14:30
pluskidn4nd0: no, any idea of this?14:30
n4nd0pluskid: it can be because a couple of parts in Machine.i needs to have and ifdef there14:31
pluskidifdef what?14:32
n4nd0pluskid: USE_MOSEK14:32
pluskidhmm...14:32
n4nd0pluskid: check it in Structure.i14:32
n4nd0#ifdef USE_MOSEK ... #endif14:32
n4nd0pluskid: can you please try if it fixes it?14:32
pluskidok14:33
n4nd0my guess is that it should go around APPLY_STRUCTURED(CPrimalMosekSOSVM); and %rename(apply_generic) CPrimalMosekSOSVM::apply(CFeatures* data=NULL);14:34
pluskidstill segfault14:35
pluskidtrying to clean and re-make14:35
n4nd0ok14:35
n4nd0I'd say that make clean and make should make it work14:36
n4nd0since that code compiled in the buildbot14:36
n4nd0haha make should make it work lol14:36
pluskid:D14:36
pluskid:(14:38
pluskidstill segfault14:38
pluskidI'll be right back soon14:38
n4nd0ok14:39
n4nd0we can check later if it's a problem related to swig's version14:39
n4nd0blackburn: your Machine.i has also the lines ^ with CPrimalMosekSOSVM right?14:41
blackburnn4nd0: yes14:41
n4nd0blackburn: and no problems compiling?14:41
blackburnno segfaults for sure14:41
n4nd0blackburn: ok14:41
blackburndo you have segfaults too?14:41
n4nd0no no14:41
n4nd0but I have my configure prepared to use those files14:41
n4nd0PrimalMosekSOSVM.h and so14:42
n4nd0but probably you and pluskid don't have it14:42
blackburnyeah we have no mosek14:44
-!- heiko [d4550102@gateway/web/freenode/ip.212.85.1.2] has joined #shogun15:00
heikotest.15:06
blackburn.tset15:06
heikoyeh :) works15:07
heikoI am in a local library15:07
blackburnай эм ин э локал либрари15:07
heikocannot stay in my room any longer ;)15:07
blackburnканнот стай ин май рум ани лонгер15:07
heikounfortunately, only http proxy so no git straightforward15:07
blackburnheh15:08
n4nd0blackburn: are you teaching us some Russian? :)15:09
blackburnn4nd0: I just wrote what heiko wrote in transliteration :D15:10
n4nd0haha15:10
heiko;)15:10
n4nd0that would be cool15:10
n4nd0if Russian is just English with other characters :P15:10
blackburnйес ит кан би15:10
blackburnиф руссиан ис джаст инглиш виз озер карактерс15:11
blackburnn4nd0: btw about spain - it was the most boring match ever15:12
blackburnshame on you :D15:13
n4nd0blackburn: :O15:13
n4nd0shame on France15:13
n4nd0Spain did its game15:13
blackburnspa - por should be interesting15:13
blackburnn4nd0: have you seen eng - ita yesterday?15:14
pluskidn4nd0: do I have to re-run ./configure script?15:16
n4nd0blackburn: yeah15:16
n4nd0blackburn: I was woing to ask you about that :) Pirlo is a brave guy15:16
n4nd0pluskid: I don't think so15:16
pluskidhmm...15:17
blackburnn4nd0: really great match15:17
n4nd0blackburn: ??15:17
blackburn??15:17
blackburn:)15:17
n4nd0blackburn: Italy against a wall15:17
pluskidI'm using swig 2.0.7-1, what's your version?15:17
n4nd0blackburn: they should have made a new feature for the match statistics, shoots to the defenders...15:18
blackburnn4nd0: well but still interesting15:18
blackburna lot of shots15:18
n4nd0pluskid: SWIG Version 2.0.415:18
n4nd0pluskid: I don't know if that may be a problem15:18
pluskidhmm15:18
n4nd0blackburn: what is your swig version?15:18
n4nd0blackburn: do you know buildbot's one too?15:19
blackburnmine is 2.0.415:19
blackburnbuilbot has something similar I think15:19
n4nd0blackburn: I think that the match was ok, I had more fun because I was at home with my Italian friend :D15:19
blackburnpluskid: where did you get 2.0.7 :D15:19
blackburnah you have arch linux right15:19
n4nd0mmm I don't really know if the problem can be caused because of swig's version15:20
pluskidyeah15:20
pluskidI failed to compile after git rebase to the latest code15:20
pluskidmaybe I can try to find what's incompatible here by binary search in the git history15:20
n4nd0after doing make clean15:20
n4nd0?15:20
pluskidyeah, I've done make clean15:21
pluskidmaybe also try with clang-swig15:21
pluskid-.-bb15:21
pluskidah, no such thing "clang-swig"15:21
n4nd0what I don't understand is that the problem seems to be caused by the lines in Machine.i that use CPrimalMosekSOSVM15:21
n4nd0but if one does ifdef USE_MOSEK they shouldn't be executed15:22
pluskidbut I've wrapped with ifdef as you suggested15:22
n4nd0yeah, that's why15:22
n4nd0with ifdef, they shouldn't be executed by swig15:22
n4nd0because you are not compiling with such a USE_MOSEK, are you?15:23
pluskidno15:23
n4nd0mmm ok15:23
n4nd0so if ifdef didn't solve it, I'd remove it15:23
pluskidseems they are not being executed15:23
pluskidbecause I no longer get the warning %extend defined for an undeclared class shogun::CPrimalMosekSOSVM.15:24
pluskidbut still segfault15:24
n4nd0did you do make clean after adding them?15:24
n4nd0aaah15:24
pluskidI think I did15:24
pluskidbut I can try again15:24
n4nd0that the extend thing is just a warning15:24
n4nd0I thought it was the direct cause of the segfault15:24
n4nd0then probably even blackburn and the buildbot get that warning15:24
blackburnyes I get it15:25
pluskidanyway, no error message, just segfault seems weird15:25
blackburndebug it with valgrind :D15:25
pluskidmy gcc isn't compiled with debug information15:26
pluskidneither do swig :p15:26
n4nd0pluskid: give 5 min and I will rebase the latest code and compile15:26
pluskidn4nd0: thanks!15:26
n4nd0pluskid: maybe I join you in the segfault club15:26
pluskidhaha, welcome!15:26
blackburnfirst rule of segfault club15:28
n4nd0pluskid: for what interfaces is your shogun configured?15:33
pluskidn4nd0: only python_modular15:33
n4nd0pluskid: no segfault here15:39
pluskidn4nd0: :-/ , then all I can do is binary search now15:39
n4nd0pluskid: is your swig version stable?15:40
pluskidn4nd0: not sure, Arch has some bleeding edge packages...15:40
n4nd0aham15:40
pluskidI have habits to do system update everyday15:42
pluskidfirst I'll revert to an old version where I remember I can compile correctly to see if there's something wrong with my swig15:43
pluskidcompiled after checking out back to a very old version...15:52
pluskidJun. 0915:52
n4nd0oh, that's old15:54
n4nd0but it's good that you can compile now15:54
pluskidtrying some newer version15:57
pluskidcompiles on Jun. 2316:01
pluskidMerge pull request #599 from cwidmer/master16:01
pluskidfailed on Jun. 2316:04
pluskidMerge pull request #596 from iglesias/so16:04
pluskidn4nd0: seems something in the so is incompatible in my local box16:05
pluskidluckily there are only 7 commits between16:05
pluskidI'm afraid I have to try them later16:06
pluskidcurrently the thunder is toooooooooooo threatening here...16:06
pluskidnot sure it will do harm to my computer16:06
pluskidturn off my laptop for safety16:06
pluskid:D16:06
-!- pluskid [~pluskid@202.130.113.141] has quit [Quit: Leaving]16:06
n4nd0pluskid: it must be something related to swig16:07
n4nd0at least that is my guess16:07
n4nd0probably in the commit + so for modular interfaces + python example16:07
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving]16:12
-!- blackburn [~blackburn@31.28.43.76] has quit [Ping timeout: 245 seconds]16:12
-!- contact2 [9320543b@gateway/web/freenode/ip.147.32.84.59] has quit [Quit: Page closed]17:07
-!- uricamic [~uricamic@2001:718:2:1634:2806:9057:c143:ab58] has quit [Quit: Leaving.]17:25
-!- heiko [d4550102@gateway/web/freenode/ip.212.85.1.2] has quit [Quit: Page closed]17:29
-!- blackburn [b22d393f@gateway/web/freenode/ip.178.45.57.63] has joined #shogun17:46
-!- puffin444 [62e3926e@gateway/web/freenode/ip.98.227.146.110] has joined #shogun17:47
-!- gsomix [~gsomix@95.67.169.72] has joined #shogun18:11
gsomixexams are over!18:12
gsomixhi all18:12
puffin444That's wonderful gsomix!18:15
-!- blackburn [b22d393f@gateway/web/freenode/ip.178.45.57.63] has quit [Ping timeout: 245 seconds]18:17
-!- blackburn [~blackburn@31.28.43.76] has joined #shogun18:51
-!- heiko [~heiko@host86-180-159-168.range86-180.btcentralplus.com] has joined #shogun18:55
@sonney2kblackburn, what was pluskid's compile segfault about?18:57
@sonney2kblackburn, did gcc segfault?18:57
blackburnsonney2k: no idea he didn't solve that yet18:57
blackburnsonney2k: yeap18:57
@sonney2kblackburn, then his memory is RIP18:57
blackburnsonney2k: wait but with earlier versions it was ok18:58
@sonney2kguess he needs to buy new ram18:58
blackburnstrange18:58
@sonney2kblackburn, oder version of gcc?18:58
blackburnolder git18:58
blackburnrevision18:59
blackburnhahah18:59
blackburnforgot the word18:59
@sonney2kblackburn, then I still vote for RAM18:59
blackburnis it from your experience?18:59
@sonney2kyes18:59
blackburnok we ll see19:01
-!- nickon [~noneedtok@dD5774105.access.telenet.be] has quit [Ping timeout: 265 seconds]19:04
CIA-18shogun: Sergey Lisitsyn master * r9264c84 / (2 files in 2 dirs): Updated multitask logistic regression - http://git.io/3PjWjQ19:05
CIA-18shogun: Sergey Lisitsyn master * r6e86d28 / (2 files in 2 dirs): Merge branch 'slep' of git://github.com/lisitsyn/shogun - http://git.io/fkHZgg19:05
blackburnsonney2k: we all forgot about weekly reports19:09
blackburn:D19:09
blackburnwhoops something is wrong with my gitconfig19:10
@sonney2kblackburn, well then hurry up...!19:10
CIA-18shogun: Sergey Lisitsyn master * r845c6c5 / src/shogun/transfer/multitask/Task.h : Added dummy task doc - http://git.io/8KgrvQ19:12
blackburnargh19:13
CIA-18shogun: Sergey Lisitsyn master * r19a6e35 / src/shogun/lib/SGSparseMatrix.h : Removed useless nulling - http://git.io/p2HBFQ19:18
blackburnhooray19:18
-!- heiko [~heiko@host86-180-159-168.range86-180.btcentralplus.com] has quit [Ping timeout: 272 seconds]19:42
-!- heiko1 [~heiko@host86-183-74-41.range86-183.btcentralplus.com] has joined #shogun19:42
puffin444heiko1, is that you or an evil clone of heiko?19:43
-!- heiko2 [~heiko@host86-180-159-222.range86-180.btcentralplus.com] has joined #shogun19:46
heiko2puffin444, how are things?19:46
blackburntriple heiko19:47
puffin444things are going quite well. I am very close to making a pull request for the model selection stuff19:47
heiko2nice19:47
-!- heiko1 [~heiko@host86-183-74-41.range86-183.btcentralplus.com] has quit [Ping timeout: 272 seconds]19:47
heiko2the thing you stumbled over took me some hours already, hope to get that fixed soon19:48
heiko2there were two problems, one easy one (already fixed, have to submit patch though) and one harder one (concept problem)19:48
heiko2Are you currently using the get_combinations method?19:49
puffin444no19:49
puffin444I am using a new method I added, get_random_combination19:50
heiko2ok, then you should have no problems with that19:50
puffin444It just returns one random combination19:50
puffin444Yeah it all works19:50
heiko2why do you need that?19:50
puffin444I use a random combination to start the gradient search19:51
heiko2ah ok19:51
heiko2and that might be replaced by a grid-search later right?19:51
puffin444Yes. It might be combined somehow with grid search19:51
heiko2and how is the class structure for the modsel stuff now?19:53
puffin444There is now a superclass called MachineEvaluation. CrossValidation and GradientEvaluation inherit from this class19:54
heiko2so no changes from the stuff we agreed on last time we talked?19:54
puffin444Nope.19:54
puffin444The gradient selection model as it stands can be used with any machine as long as it has a corresponding differentiable function.19:55
heiko2Thats very cool19:55
heiko2I think this will make things so much easier in the future19:56
puffin444I think so. I was surprised actually by the small amount of work required to get hyperparameter learning up and running. This was the part of the project that worried me the most.19:56
heiko2The horrors are always in the details :)19:57
puffin444Yes :)19:57
heiko2Say, do you plan to do this automatic relevance determination stuff?19:58
puffin444Not sure what your're talking about actually.19:59
heiko2if your data has multiple dimensions, one could optimise sigma in every dimension seperately19:59
heiko2rather than using the same value for each20:00
puffin444Oh. I'm not sure about that. I'll see what Oliver thinks. The way its structured now though it wouldn't be too hard to add.20:00
heiko2let me find the page in the book where I saw that20:01
heiko2I like it because it has this nice feature-selection interpretation (dimensions that do not help are wiped out)20:01
puffin444I think it would just be a matter of adding/modifiying an inference method class and a likelihood model class that calculates a different gradient.20:02
puffin444Nothing in the model selection framework would have to be changed :)20:02
heiko2well basically in the model-selection chapter20:02
heiko2probably yes20:03
heiko2I mean yes youre right20:03
puffin444By the way the NLOPT and the optimizer in the GPML are giving me answers which are exactly the same20:05
heiko2yes I saw that, awesome :D20:06
puffin444Very reassuring :)20:06
heiko2indeed20:06
heiko2what problem did you try?20:06
-!- puffin444_ [62e3926e@gateway/web/freenode/ip.98.227.146.110] has joined #shogun20:07
puffin444_sorry about that20:08
puffin444_It was sort a simple case (with some variations in the training vectors)20:08
puffin444_Even with radical changes in individual training vectors I consistently got the same answers20:08
heiko2thats great then20:08
heiko2perhaps it would be good to test it on different kinds of problems (easy/hard, different scales etc)20:09
heiko2you could also assert for correct results in order to detect errors when the framework is changes (these sometimes arent found immediately)20:10
-!- puffin444 [62e3926e@gateway/web/freenode/ip.98.227.146.110] has quit [Ping timeout: 245 seconds]20:10
puffin444_what20:10
heiko2?20:10
puffin444_I guess the irc bot reports these things late20:10
puffin444_hah20:10
heiko2oh, yes :) what did you read?20:11
puffin444_== puffin444 [62e3926e@gateway/web/freenode/ip.98.227.146.110] has quit [Ping timeout: 245 seconds]20:11
puffin444_as I was logged in :)20:11
heiko2(19:08:53) heiko: thats great then20:11
heiko2(19:09:16) heiko: perhaps it would be good to test it on different kinds of problems (easy/hard, different scales etc)20:11
heiko2(19:10:09) heiko: you could also assert for correct results in order to detect errors when the framework is changes (these sometimes arent found immediately)20:11
puffin444_Yes I do want to test both the GP and the gradient search it on a variety of data sets,20:12
heiko2ok then, I will have a look at the modsel code once you commit it20:13
puffin444_Yeah there are some details I do have questions about.20:14
-!- alexlovesdata [c25faea9@gateway/web/freenode/ip.194.95.174.169] has quit [Ping timeout: 245 seconds]20:14
heiko2Now?20:14
puffin444_For example currently the generalize framework makes an evaluation and returns a general result which must be downcast.20:15
puffin444_Just a few small things.20:15
puffin444_about coding styles, decisions, etc.20:15
heiko2we usually solve these problem with enums, like in CKernel20:16
heiko2but ok20:16
heiko2let me know when you submit, I am really interested in the gradient stuff20:16
puffin444_Then it might be beneficial for an enum type system to be added for Evaluation Results. I'll ask a few of these questions with the pull request.20:17
puffin444_I definitely will. I just want to clean up a little of the code and check for memory bugs and then I will submit.20:17
heiko2alright20:17
puffin444_Sorry about some of the coding conventions. I'll make sure to clean that up.20:18
heiko2No worries :)20:19
puffin444_Oliver was going to join us, but I just got word he's stuck on a late train.20:20
heiko2ah ok20:20
heiko2send him my greetings, I share the pain with the deutsche bahn :)20:21
puffin444_Hah I'm sure its better than Amtrack20:22
heiko2hehe, train companies seem to be horrible everywhere ;)20:22
heiko2alright then, I have to leave soon, have a good evening20:24
heiko2ehm, is it evening for you?20:24
heiko2 :)20:24
heiko2dont know20:24
puffin444_It's early afternoon :).20:25
puffin444_Have a good evening!20:25
heiko2thanks :) take care, bye20:26
-!- os252 [55b3f5d9@gateway/web/freenode/ip.85.179.245.217] has joined #shogun20:32
os252puffin444: sorry; now online.20:32
puffin444_Hi oliver20:32
os252hi20:32
os252did you guys discuss things already?20:33
os252Sorry; I missed a connection and then had to wait for some time.20:33
blackburnpuffin444_: gsomix: guys we totally forgot about weekly reports - could you please?20:33
puffin444_We talked about the model selection framework.20:33
-!- heiko2 [~heiko@host86-180-159-222.range86-180.btcentralplus.com] has quit [Ping timeout: 272 seconds]20:33
puffin444_blackburn, will do.20:33
puffin444_We talked about the model selection framework.20:34
puffin444_It's done now. I just want to clean up a little bit of the code and look for memory leaks before I submit a pull request.20:34
puffin444_Hyperparameter learning is now an option for GPs, and the answer from NLOPT corresponds almost exactly to that of GPML.20:35
os252graet.20:36
os252answer = marginal likelihood and parameter settings?20:36
puffin444_Yes. I get an exact same answer for marginal likelihood. Parameter settings are the same for the most part. There is some variability with sigma but the sigma tends to be very small (1e-7) range.20:37
os252ok, that's most likely numerical.. nice.20:38
os252How did you end up handling the scale of the kernel?20:38
puffin444_I have not tackled that problem yet.20:39
os252so the scale is =1 ?20:39
puffin444_Yes as of now.20:39
os252wondering how you can then get the same results than gpml which learns the scale.20:39
os252or do you simulated from that kernel?20:40
puffin444_I am using the kernel which does not have a scale in GPML20:40
os252I see20:40
os252ok20:40
os252The scale is quite crucial.20:40
os252One can either introduce scaled kernels in shogun or need to rescale them in the gp classes.20:41
os252Not sure what is better.20:41
puffin444_I see. So can the scale be independent of kernel type? Within the GP classes?20:41
os252Introducing scaling in shogun kernels would seem cleanest but may break quite a few tings.20:41
os252it could20:41
os252just like the noise is added to the diagonal you can rescale the kernel matrix.20:41
os252relative scaling if you hav a sum of kernels is done by the combinator class.20:42
os252It's probably the simplest solution given the existing shogun code base20:42
puffin444_It might just be safer at this point to put it in the GP code. It should be very easy to do.20:43
os252yes20:43
os252sounds good.20:43
os252also: those trivial parameters I'd all put on log scale.20:43
os252If you have the choice (as for this one)20:43
os252You may also wonder whether it makes sense to implement the range restrictions in the kernel funciton, i.e. have a queriable interface where you automatically determine the legal range for each parameter.20:44
os252But that's rather nice to have.20:44
os252I think given where you are the priorities are probably :20:44
os252* finish the parameter learning machinery as you had planed20:44
os252* add scaling of kernels20:44
os252* add uncertainty to apply_regression20:44
os252* make a few demos to look at20:45
os252* caching (see my email)20:45
-!- gsomix [~gsomix@95.67.169.72] has quit [Remote host closed the connection]20:45
os252* get more kernels that have derivatives (the combinator is very useful to have sums of kernels)20:45
os252That's a suggestion. With these items, the core part of the project would be sort of done.20:46
puffin444_Hey oliver,  which week are we on for the GSoC?20:46
puffin444_Did we just finish up the 5th?20:47
os252I think so20:47
os252So still plenty of time :-)20:48
puffin444_Okay, so according to the original plan after this I move on the Sparse Approximation20:48
os252But you also had ambitious plans.20:48
puffin444_Yes, :)20:48
os252Yeah, you may want to rethink this a little. Once the basics work, which is now the case - most things get faster.20:49
-!- nickon [~noneedtok@dD5774105.access.telenet.be] has joined #shogun20:49
puffin444_I definitely first need to work on some these things you mentioned.20:49
os252A question is also how many of the kernels to make GP ready, including derivatives. That could also be a substantial time sink.20:49
puffin444_Yes.20:49
os252Perhaps worth consulting Soeren which priorities he'd suggest.20:50
os252If it takes now an extra week to get this all solid and the basics working that seems all fine. I can't imagine it'll take you more than 2 weeks to get most of this done.20:51
os252There is also plenty of scope to move some of these things to the sparse part then and get it done while implementing these things20:51
puffin444_Yes.20:51
os252(for example caching, which is sort of a speed issue -> sparse as well)20:51
puffin444_So in my progress report how about I choose these as my goals for this week:20:52
puffin444_1. Add Scaling of kernels20:53
puffin444_2. Allow apply_regression in GaussianProcessRegression to return variances as well as means20:53
puffin444_3. Add a few visual demos20:53
puffin444_4. Make the calcuations in GaussianProcessRegression more efficient.20:53
puffin444_(caching)20:53
puffin444_5. Add derivatives to some more kernels.20:53
os252I was hoping you stop here ;-)20:54
puffin444_yes20:54
puffin444_sorry :)20:54
os252Yes, sounds good. Ask for guidance regarding which kernels.20:54
puffin444_I will.20:54
@sonney2kpuffin444_, gaussian, poly, linear20:54
os252I definitely would add linear and the combinator (sum)20:54
@sonney2kthat's it20:54
puffin444_Gaussian is already there, Will add poly, linear, and combinator20:55
os252Thanks sonney2k. Definitely a good start. But let's add sum - which is the real advantage of GPs over SVMs ;-)20:55
os252puffin444: great.20:55
@sonney2ksum == combined kernel?20:55
puffin444_ok20:56
os252It's called combined in shogun, yes.20:56
@sonney2k \sum_i beta_i K_i(x,x') ?20:56
@sonney2kk20:56
@sonney2kyes then20:56
@sonney2kbut then each kernel already has a weight20:56
blackburnwhat is advantage over svm?20:56
@sonney2kthough only used in combined kernel20:56
os252It's a bit annoying to implement because you need to take care of derivatives and call the code of relevant kernel functions that are part of the combinator.20:56
@sonney2kos252, I don't think it is hard ...20:57
os252blackburn: well you can just handle large number of kernel parameters as you don't need to cross validate anything, i.e. can realistically lern the releative weights of many components.20:57
@sonney2kos252, we want some *fancy* example20:57
os252sonney2k, you mean nice demos?20:58
os252We should definitely have some nice regression demos with sum and product kernels; those are nice for a large class of regression problems.20:58
blackburnos252: sorry I am totally lame in GPs - so you don't have to do xval to get the best parameters values?20:59
puffin444_I'll put in some demos20:59
os252blackburn, right - yes. You still can (if you have few of them and just want to predict) but the standard is to optimize marignal likelihod,20:59
os252i.e. p(y | x, K, \theta) where \theta is the set of kernel parameters.20:59
blackburnI see20:59
@sonney2kI am also no GP person, os252 how is this optimized?21:00
os252So merely finding the most probable state of parameters is an easy (optimization) problem and hence you can realistically make use of far more complex kernels than when you need a grid...21:00
@sonney2kos252, but don't you need to have some priors for that then?21:00
os252You can put some on, but don't need to (given enough data).21:00
@sonney2kor how do you define the optimization problem?21:00
os252p(y| x,K,\theta) = N(y| 0, K(x,theta))21:01
os252K: choice of kernel ( complicated convoluted thing)21:01
os252x: inputs21:01
os252theta: parameter of K21:01
os252\hat{theta} = \argmax \ln N(y| 0, K(x,\theta))21:01
@sonney2kN == normal distribution I guess21:01
os252If you know the gradients of K w.r.t all \theta_i you can easily calculate gradients of \ln N(...)21:02
os252yes, N = normal21:02
os252And then stick the whole thing in a gradeint-based optimizer. It's not convex for almost any choice of K but local optima are generally pretty good.21:02
os252puffin444 is almost there. You can soo give it a go pretty soon ;-)21:03
blackburnare you going to use l-bfgs-b for that?21:04
@sonney2kos252, sorry for asking a stupid question , but you determine mean and covariance matrix for N right?21:04
@sonney2kis O the mean?21:04
@sonney2kan K() the cov?21:04
os252puffin444: NLOPT/ bfgsb decided? NLOPT has an lbfgsb implementation in it and much more... so I'd favor that.21:05
os252sonney2k.21:05
puffin444_NLOPT is in there right now.21:05
blackburnI did that for you :D21:05
puffin444_Works beautifully21:05
os252So mean can be non-zero. We'll have a few simple mean functions (most simply variable constant).21:05
puffin444_thanks blackburn.21:05
os252In practice actually zero-meaning data is good enough for most.21:06
os252The covaraince is constructed from the kernel.21:06
blackburnsonney2k: ah while you are there21:06
os252If you knot the parameters (\theta)21:06
blackburnsonney2k: have you seen eigen3?21:06
os252K_{i,j} = k(x_i,x_j) + \delta_{i==j} \sigma^221:06
os252or better21:07
os252K_{i,j} = k(x_i,x_j | \theta ) + \delta_{i==j} \sigma^221:07
@sonney2kos252, ok21:07
@sonney2kblackburn, yes21:07
os252all you do is now optimize the marginal likelihood w.r.t to \theta, \sigma^2 which in turn determine the covariance matrix and hence the likelihood of the data.21:07
@sonney2kblackburn, wanted to use it for shogun at some point - but it is not optimized for many CPU archs21:08
@sonney2kos252, the principle is clear now21:08
blackburnsonney2k: do you need many?21:08
os252sonney2k: Is it really still an issue? [Eigen]21:08
os252I love Eigen. Suddenly I can read my C++ maths code and can virtually copy and paste python -> C++.21:08
@sonney2kblackburn, not personally no21:09
blackburnI would like to have it here21:09
os252Since the last realease you can build Intel MKL (and by the next release any Blas/Lapack conform library) under neith it either..21:09
@sonney2kblackburn, so you want to drop atlas/lapack?21:09
os252In case you are worried about performance.21:09
blackburnsonney2k: well lapack has only a few of capabilities of eigen321:10
@sonney2kos252, from the benchmarks I've seen it is faster on modern archs sseN etc21:10
blackburnso if you are ok with it I would even use it for new code21:11
os252Yeah..21:11
os252Just saw Eigen 3.1 is out.21:11
os252So they will now definitely go for dual interfaciting.21:11
os252Either Eigen with blas interface or vice versa which is nice.21:11
blackburnos252: do you know whether they provide standart blas interface?21:12
os252I find it great to use; use it for virtually all my projects now.21:12
os252they do!21:12
blackburncblas_dgemm blabla21:12
os252I think they provide the full interface and use it for their benchmarks.21:12
blackburnthen I would like to drop atlas/lapack dependency21:12
blackburnsonney2k:21:12
blackburn^21:12
os252Yes, take a look at that. It will reduce compile headache a lot.21:13
os252THe sole downside (unfortunately not minor): Eigen is a header-library and hence definitely does not decrease compile time.21:13
blackburnos252: I thought you are more math person :)21:13
os252That may be an issue given the size of shogun. I am not sure how fast it is though if you are using the blas stuff (that should be compilable).21:13
@sonney2kwell if we can avoid using it in *headers* it won't make a difference21:14
blackburnos252: most time consuming are interfaces21:14
@sonney2kbecause interfaces are generated from headers only21:15
os252Well, like maths that still runs fastish (genetics -> GWAS -> terrabytes of data -> waiting ....21:15
blackburnsonney2k: I need your decision then :)21:15
os252I guess if you continue using shogun-internal matrices and arrays you can.21:15
os252(i.e. don't write public functions that take eigen arrays as arguments).21:16
blackburnwe don't really need to use shogun-internal matrices in solvers and etc21:16
blackburnusually it is double* here21:16
os252ok.. then all good.21:16
os252You can create Eigen Matrices from a double* etc. in functions.21:17
@sonney2kblackburn, err you want to do that before the end of gsoc?21:17
@sonney2kto me it sounds like something to do after21:17
blackburnsonney2k: yeah why not21:17
blackburnit would speed up my development21:17
@sonney2kwe only have 2 months before we want to release?!21:17
blackburnsonney2k: it changes nothing21:18
blackburnblas and lapack would still work21:18
@sonney2kI don't like a halfway solution ... can one emulate lapack/blas calls?21:19
blackburnsonney2k: interface is provided!21:19
blackburnalready21:19
blackburnjust as it is21:19
@sonney2kas in create a Matrix from double?21:19
@sonney2kdouble*, n, m21:19
blackburnsonney2k: we don't have to21:20
blackburnhalfway is ok here I think21:20
blackburnwe don't need to change everything right now21:20
os252yes... it has lapak/blas interfaces. and interfacting from double* is easy.21:20
os252You could just add it in and switch over for bits and pieces21:20
blackburnI would like to use it only in new code for now21:21
blackburnI am doing a lot of vectorized algos now21:21
@sonney2kblackburn, it is another huge depency just for a few lines of code21:21
blackburnsonney2k: we can remove atlas dependency21:21
@sonney2kos252, any example?21:21
blackburnsudo apt-get install eigen3 is not any difficult21:22
os252 Map<MatrixXf> mf(double *,rows,columns);21:22
os252:-)21:22
puffin444_Oliver, I will get to work on these things ASAP.21:23
blackburnsonney2k: check what I do in shogun/lib/slep/slep_mt_lsr.cpp - basically vector additions and etc21:23
os252puffin444, ok great!21:23
os252It's just a bunch of headers. YOu can even just copy them into shogun. There is no compiled piece of code in there.21:23
blackburnI would really like to use some thing like eigen3 here - it is slower to use for loops here (both to develop and compute)21:24
-!- puffin444_ [62e3926e@gateway/web/freenode/ip.98.227.146.110] has quit [Quit: Page closed]21:24
@sonney2kos252, fair enough21:28
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun21:29
blackburnsonney2k: there was an example how its api works21:32
blackburnthey showed that all the abstraction go away on compilation21:32
os252Here: http://eigen.tuxfamily.org/index.php?title=API_Showcase21:32
os252Here, I used it for sparse factor analysis, barely anymore code than the python predecessor https://github.com/PMBio/peer/blob/master/src/sparsefa.cpp21:33
blackburnos252: was numpy not enough fast?21:36
os252depends.21:36
os252For anything you can code up without for loops there is little difference (obviously).21:37
blackburnI mean for your project21:37
os252I mainly wanted it in C++ due to ease of interfacing with R.21:37
blackburnahh21:37
blackburnyeah main reason here too I think :D21:37
os252yeah - you won't bee numpy by a mile. I't spretty optimized by now.21:37
os252In particular for models like GPs where the dominant cost is inverting or decomposing a huge matrix...21:38
n4nd0did you know about this apache mahout guys? http://mahout.apache.org/21:38
@sonney2kblackburn, well I don't know how far you are with your project but you can certainly hack up configure support for eigen3, identify the cblas* / lapack functions we use and then we can just wrap them?21:43
blackburnsonney2k: why to wrap them?21:43
@sonney2kblackburn, how can we drop lapack/blas if we don't do that?21:45
blackburnsonney2k: eigen3 provides headers of cblas_* stuff21:45
@sonney2kblackburn, without using blas/lapack right?21:46
@sonney2kthen try to replace it...21:46
blackburnyes21:46
blackburnok I'll try21:46
@sonney2kshogun-buildbot, status21:47
shogun-buildbotcmdline_static: idle, last build 19h45m01s ago: failed test_121:47
shogun-buildbotcsharp_modular: idle, last build 165h50m41s ago: build successful21:47
shogun-buildbotjava_modular: idle, last build 165h35m17s ago: failed test_121:47
shogun-buildbotlibshogun: idle, last build 2h24m51s ago: failed configure21:47
shogun-buildbotlua_modular: idle, last build 165h43m09s ago: failed test_121:47
shogun-buildbotnightly_all: idle, last build 44m18s ago: failed configure21:47
shogun-buildbotnightly_default: idle, last build 44m04s ago: failed configure21:47
shogun-buildbotnightly_none: idle, last build 44m33s ago: failed configure21:47
shogun-buildbotoctave_modular: idle, last build 165h58m50s ago: failed test_121:47
shogun-buildbotoctave_static: idle, last build 19h39m52s ago: failed test_121:47
shogun-buildbotpython_modular: idle, last build 165h27m32s ago: failed test_121:47
shogun-buildbotpython_static: idle, last build 8h38m57s ago: failed configure21:47
shogun-buildbotr_modular: idle, last build 165h18m54s ago: build successful21:47
shogun-buildbotr_static: idle, last build 19h49m08s ago: failed configure21:47
@sonney2kbah21:47
@sonney2kblackburn, if it works all good21:49
blackburnsonney2k: then?21:52
-!- os252 [55b3f5d9@gateway/web/freenode/ip.85.179.245.217] has quit [Quit: Page closed]22:22
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has left #shogun []22:29
-!- nickon [~noneedtok@dD5774105.access.telenet.be] has quit [Quit: ( www.nnscript.com :: NoNameScript 4.22 :: www.esnation.com )]22:32
-!- ckwidmer [8ca3fe9d@gateway/web/freenode/ip.140.163.254.157] has joined #shogun22:36
-!- tejaswi [~tejaswidp@117.192.131.69] has joined #shogun22:51
-!- tejaswi [~tejaswidp@117.192.131.69] has quit [Read error: Connection timed out]23:07
--- Log closed Tue Jun 26 00:00:07 2012

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!