IRC logs of #shogun for Saturday, 2011-08-13

« Friday, 2011-08-12 Index Sunday, 2011-08-14 »
--- Log opened Sat Aug 13 00:00:18 2011
serialhexhey blackburn whats up?00:24
serialhexaaahhhh!!! im put :P00:24
blackburnserialhex: hi00:25
blackburntrying to remove java serialization stuff but shogun serialization should stay %)00:25
blackburnhow are you?00:25
-!- blackburn [~blackburn@188.122.238.50] has quit [Quit: Leaving.]00:32
-!- f-x [~user@117.192.192.210] has quit [Ping timeout: 260 seconds]04:25
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun05:51
-!- in3xes_ [~in3xes@180.149.49.227] has joined #shogun06:11
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]06:14
-!- in3xes__ [~in3xes@180.149.49.227] has joined #shogun06:16
-!- in3xes_ [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]06:20
-!- in3xes__ is now known as in3xes06:21
-!- in3xes [~in3xes@180.149.49.227] has quit [Client Quit]06:21
CIA-87shogun: Soeren Sonnenburg master * rc1d50d1 / src/shogun/base/SGObject.h : fix compile error when compiling without reference counting - http://bit.ly/rb88nP07:07
CIA-87shogun: Soeren Sonnenburg master * r52fbaf1 / (3 files in 3 dirs): kill remaining warnings for python_modular - http://bit.ly/q2G7wi07:28
-!- alesis-novik [~alesis@188.74.87.206] has joined #shogun07:30
-!- mrsrikanth [~mrsrikant@59.92.57.168] has joined #shogun08:26
-!- blackburn [~blackburn@188.122.238.50] has joined #shogun09:35
blackburnsonney2k: the new problem with java serialization is that some java stuff is still here12:15
blackburn...sr..org.shogun.RealFeatures.OL./......xr..org.shogun.DotFeatures;A7xJM.+...xr..org.shogun.Features..^E.._....xr..org.shogun.SGObject.77...i....xpz....<<_SHOGUN_SERIALIZABLE_ASCII_FILE12:16
blackburnand so on12:16
blackburnthat shit in the beginning should go, right?12:16
@sonney2kblackburn, you mean in the output of the serialzed file?12:53
@sonney2kblackburn, that is ok12:54
@sonney2kblackburn, it is the same when we python pickle12:54
@sonney2kblackburn, there is some $LANG specific header in tehre12:54
blackburnbut how to avoid this?13:01
blackburnI mean we can't read java output in python13:01
blackburnor&13:01
blackburnor?13:01
@sonney2kblackburn, why avoid this?13:06
@sonney2kthis is something java specific13:06
@sonney2kI mean we use java serialization for java object13:06
@sonney2kand python  serialziation for python objects13:07
blackburnyeah but if it is not there we could exchange objects13:07
blackburnbetween java,python and everything else13:07
@sonney2kblackburn, if we want to exchange objects we just use shogun's serialzation code13:07
@sonney2ksave_serializable()13:07
blackburnhmm yes13:07
@sonney2kload_serializable()13:07
blackburnsure13:07
blackburnokay, I'll try to get load to work now13:08
blackburnit fails now for some strange reaso13:08
@sonney2kthe $LANG native stuff is only a convenience for people working in this $LANG13:08
blackburnsonney2k: could you give me some advice about how to get distances between promoters?13:08
@sonney2kI hope I will have time tonight to fix the sparse bug13:08
blackburnor where to get comprehension of all this stuff13:09
@sonney2kblackburn, yes look at my diss there is a tss recognition chapter in there13:09
blackburnokay13:09
@sonney2kand then use e.g. 2 spectrum kernels + 1 weighteddegreepositionstringkernel13:09
blackburnhow to get distance from kernel?13:09
@sonney2kor when you want to use dotfeatures (explicit feature representatiosn) then wdfeatures / weightedspec features13:10
@sonney2kyou could use kerneldistance13:10
@sonney2kk13:10
@sonney2kgtg13:10
blackburnthanks13:10
CIA-87shogun: Sergey Lisitsyn master * r9350b71 / src/interfaces/modular/SGBase.i : Got java externalization to work - http://bit.ly/qkjZah13:38
-!- blackburn [~blackburn@188.122.238.50] has quit [Quit: Leaving.]13:39
-!- mrsrikanth [~mrsrikant@59.92.57.168] has quit [Quit: Leaving]14:21
-!- mrsrikanth [~mrsrikant@59.92.57.168] has joined #shogun14:24
-!- mrsrikanth [~mrsrikant@59.92.57.168] has quit [Quit: Leaving]14:34
-!- blackburn [~blackburn@188.122.238.50] has joined #shogun14:44
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun15:44
@sonney2kblackburn, hmmhh the buildbot didn't catch your commit...16:19
blackburnwhat you mean saing didn't catch?16:20
@sonney2kblackburn, it didn't trigger any build16:20
@sonney2kahh exceptions.OSError: [Errno 12] Cannot allocate memory16:21
@sonney2kthat's why16:21
blackburnsonney2k: what do you think if I rewrite PCA and KPCA and call it SimplePCA and SimpleKernelPCA16:22
blackburnwithout any init and fancy options16:22
@sonney2kwhy rewrite?16:22
blackburnit takes more if I try to get into this code16:23
-!- shogun-buildbot [~shogun-bu@87.118.92.43] has quit [Quit: buildmaster reconfigured: bot disconnecting]16:23
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun16:23
blackburnanyway, the main point is not to use init, what do you think about it?16:23
blackburni.e. project vectors in apply_to_feature_matrix16:24
blackburnlike LLE, mds and other dimreduction preprocs do16:24
@sonney2kblackburn, but why?16:24
@sonney2kone can use pca/kpca really for preprocessing16:25
blackburnhow can one use pca in the init/apply way?16:25
@sonney2kand e.g. the code for PCA has only a few lines...16:25
blackburnbtw PCA is broken now16:26
@sonney2kblackburn, well you can determine the subspace and then project all vectors to come into it16:26
@sonney2kwhat happened?16:26
blackburndon't know but it acts strange16:26
@sonney2k?16:27
blackburnit can't preprocess even flat grid16:27
@sonney2kthat is still to unspecific...16:28
@sonney2kwhat happens?16:28
blackburnresults aren't right for sure16:30
@sonney2kit was correct at some point ...16:31
@sonney2kblackburn, I am working on the sparse features issue now16:31
@sonney2kl8r16:31
blackburnsonney2k: http://dl.dropbox.com/u/10139213/share/pca.png16:31
blackburnand it becomes random sometimes16:32
@sonney2kthis looks like uninited memory16:32
@sonney2kvalgrind16:32
blackburnI guess something is wrong with eigenvalues16:33
@sonney2kit looks more like some init function or so is not called and you are getting random data16:34
@sonney2kblackburn, python features_string_sliding_window_modular.py17:18
@sonney2kit needs the dynamic array ...17:18
blackburnuh17:18
blackburnsorry didn't noticed that17:19
@sonney2kblackburn, I've partially fixed the sparse stuff - but really we need some new way of dealing with things17:19
@sonney2kblackburn, what I would love to have is the following17:20
@sonney2kwhen one calls get_feature_vector()17:20
@sonney2kone gets some SGVector17:20
@sonney2kand then calls vec.free_vector()17:20
@sonney2kthat's it17:20
@sonney2kno longer all the special stuff17:20
blackburn??17:21
@sonney2kwe need the same for sparse vectors and strings17:21
@sonney2kblackburn, did you use get_feature_vector()?17:21
blackburnI work with feature matrices17:21
blackburnwhat do you mean special stuff?17:22
-!- in3xes_ [~in3xes@180.149.49.227] has joined #shogun17:23
@sonney2kwell you always have to call free_feature_vector()17:24
@sonney2ksame for sparse/strings17:25
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]17:26
@sonney2kblackburn, clear what I mean?17:32
@sonney2kblackburn, btw python modelselection_grid_search_linear_modular.py17:32
@sonney2kdid you remove that too?17:32
blackburnyeah17:32
blackburnremove what?17:32
blackburnehm17:33
blackburnnope17:33
@sonney2kblackburn, ok then I should bug heiko about it17:33
blackburnDynArray is the only thing to restore?17:33
CIA-87shogun: Soeren Sonnenburg master * raecb8bf / (4 files in 3 dirs): fix crashers in sparse features and svmocas - http://bit.ly/qmpxeA17:35
@sonney2kDynIntArray so far only yes17:35
-!- in3xes_ is now known as in3xes17:37
@sonney2kshogun-buildbot, status all17:39
shogun-buildbotcmdline_static: idle, last build 10h24m09s ago: build successful17:39
shogun-buildbotcsharp_modular: idle, last build 10h14m30s ago: build successful17:39
shogun-buildbotjava_modular: idle, last build 10h16m05s ago: build successful17:39
shogun-buildbotlibshogun: building(compiling)17:39
shogun-buildbotlua_modular: idle, last build 10h14m26s ago: build successful17:39
shogun-buildbotnightly_all: idle, last build 20h38m13s ago: failed configure17:39
shogun-buildbotnightly_none: idle, last build 20h36m56s ago: failed compile17:39
shogun-buildbotoctave_modular: idle, last build 10h06m16s ago: build successful17:39
shogun-buildbotoctave_static: idle, last build 10h24m00s ago: build successful17:39
shogun-buildbotpython_modular: idle, last build 10h07m04s ago: build successful17:39
shogun-buildbotpython_static: idle, last build 10h24m07s ago: build successful17:39
shogun-buildbotr_modular: idle, last build 10h07m05s ago: build successful17:39
shogun-buildbotr_static: idle, last build 10h24m10s ago: build successful17:39
shogun-buildbotruby_modular: idle, last build 10h15m59s ago: build successful17:39
@sonney2khmmhh now it is building...17:39
@sonney2kblackburn, what did you do to him that he ignored you?17:39
blackburnsonney2k: I don't know, may be he doesn't like russians17:40
blackburn:D17:40
@sonney2kmust be memory issues17:40
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]18:06
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun18:09
blackburnsonney2k: kernel pca init is a little bit tricky, have you finished it?19:36
@sonney2kblackburn, kernel pca init works20:10
@sonney2kthe problem is that apply does not20:10
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]20:11
blackburnясно20:17
CIA-87shogun: Soeren Sonnenburg master * rad1314a / (2 files in 2 dirs): fix remaining warnings in libshogun - http://bit.ly/pYmTBW20:21
@sonney2kblackburn, look just 3 warnings http://shogun-toolbox.org:5347/waterfall20:22
@sonney2kblackburn, regarding kernelpca20:22
blackburn3 warnings left?!20:22
@sonney2kI ported what vojtech uses20:22
@sonney2kblackburn, in libshogun20:22
blackburnsonney2k: I'm looking at it now20:22
blackburncan't understand how to project/apply data20:22
@sonney2kblackburn, do you have the matlab file / mex file?20:23
blackburnit seems it involves different routine20:23
blackburnI have one from his statistical pr toolbox20:23
@sonney2kI did compare the result of the kpca components - they where the same20:23
@sonney2kblackburn, exactly that is what I am using20:23
@sonney2kerr was using20:23
blackburnsonney2k: so kpcarec should be used as mapping to new feature space?20:24
-!- f-x [~user@117.192.212.140] has joined #shogun20:33
blackburnsonney2k: so you killed out all the doxygen warnings?20:35
@sonney2k0 warnings http://shogun-toolbox.org:5347/waterfall!20:38
@sonney2kblackburn, no20:38
@sonney2kdoxygen ones are still there20:39
@sonney2kf-x, I hopefullly correctly fixed the vw warning https://github.com/shogun-toolbox/shogun/commit/ad1314a98b0c84ad32e19f0485cfc8c3c182a00320:39
f-xsonney2k: yeah, i checked it out.. looks nice20:40
@sonney2kblackburn, I don't remember the variable names - in fact I forgot everything about kpca (well almost ;)20:40
@sonney2kf-x, I think it was wrong before even ...20:40
@sonney2kthe char* x[total]20:40
blackburnsonney2k: not variable names, which .m routine makes a projection20:41
f-xsonney2k: wrong? in that it wouldn't work?20:41
@sonney2kf-x, that would be total char* pointers20:41
f-xoh - the allocation of the array?20:41
@sonney2kso not just one size_t variable20:41
@sonney2kblackburn, I only recall that it called some mex function to apply kpca20:41
blackburnokay20:42
f-xsonney2k: yes, it should have been char x[total]20:44
@sonney2kyeah or just size_t x :)20:44
f-xexactly :)20:44
f-xi'll put in the actual learning algorithm code soon, hopefully that won't produce any warnings20:45
f-xsonney2k: do we have the feature of loading a saved learner state?20:45
f-xand then continuing from that?20:45
@sonney2kf-x, not in general no20:45
@sonney2ksvmlight can do sth like this when alphas are set20:45
@sonney2kfor online learners / liblinear that would be relatively easy to add but it is not there yet20:46
f-xsonney2k: hmm i was wondering if there's a general method to do this, or if i should do it in the VW-specific way for now20:46
f-xok, so now i'll do it how they do it for VW20:46
@sonney2kyes20:47
f-xbut not compatible with other online learners20:47
@sonney2kblackburn, that are the remaining warnings for python_modular - ~1200 and pretty much all documentation http://shogun-toolbox.org:5347/builders/python_modular/builds/6/steps/compile/logs/warnings%20(1240)/text - search for preprocessor and you will find enough to fix :)20:49
@sonney2kf-x, and you search for vw in http://shogun-toolbox.org:5347/builders/python_modular/builds/6/steps/compile/logs/warnings%20(1240)/text for warnings20:50
@sonney2kto fix20:50
@sonney2kundocumented stuff20:50
f-xyeah, sure.. that's the next task20:53
blackburnsonney2k: serialization of libsvm in java fails21:00
blackburnwith memory corruption21:00
blackburnI guess not my fault21:00
-!- f-x [~user@117.192.212.140] has quit [Remote host closed the connection]21:00
@sonney2kblackburn, doesn't it fail for everything?21:00
blackburnno, serialization in java works for labels, features and kernels for sure21:01
CIA-87shogun: Sergey Lisitsyn master * r86d2c7f / src/interfaces/modular/Library.i : Restored Dynarray modular renames - http://bit.ly/pNTBJJ21:03
CIA-87shogun: Sergey Lisitsyn master * r0f67ca8 / src/interfaces/modular/SGBase.i : Codestyle fix for java externalization routines - http://bit.ly/nGysFr21:03
CIA-87shogun: Sergey Lisitsyn master * re57e521 / examples/undocumented/java_modular/serialization_svmlight_modular.java : Improved java svmlight externalization example - http://bit.ly/q7np0B21:03
blackburnsonney2k: you could check by yourself with examples/undocumented/java_modular/serialization_svmlight_modular.java21:03
blackburnshogun-buildbot: pull the trigger21:03
@sonney2kblackburn, that fails too here21:04
@sonney2kin pythoon modular21:05
@sonney2khmmhh21:05
blackburnboom21:05
@sonney2krebooted21:05
@sonney2kwell out of memory21:05
@sonney2kwe need fatter buildbots21:06
@sonney2ktoo bad that I couldn't plug in 16G ...21:06
blackburnsonney2k: I guess we should ignore moreee21:07
shogun-buildbotbuild #1 of nightly_none is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/nightly_none/builds/121:07
@sonney2khurray21:07
@sonney2kthe full one of course failed because I have no cplex here21:07
blackburnme too21:09
blackburnsonney2k: now we have 616 classes21:11
blackburnin interfaces21:11
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun21:12
blackburnsonney2k: what is Plif?21:12
@sonney2kblackburn, piecewise linear function21:12
@sonney2kneeded in structured output stuff21:12
blackburnoooh21:12
blackburnbad thing I can't see any class to remove21:13
-!- f-x [~user@117.192.212.140] has joined #shogun21:20
blackburnsonney2k: may be reducing number of different types could reduce memory usage21:26
blackburnseriously, who ever wants to use uint8_t21:27
@sonney2kblackburn, certainly21:27
@sonney2kI use it all the time for strings21:27
blackburnbut simplefeatures?21:27
@sonney2kand think of images21:27
blackburnso your point is we can't remove any?21:28
blackburnsonney2k: hmm found one way21:30
blackburnsonney2k: swig creates wrappers for "short", etc21:30
blackburnbool, clock_t21:31
blackburnI guess we can avoid this21:32
blackburndon't sure yet21:32
-!- f-x [~user@117.192.212.140] has quit [Remote host closed the connection]21:33
@sonney2kclock_t?21:33
blackburnyeah, it creates wrappers for clock_t too21:34
blackburnwithout renaming though21:34
blackburnhmm no21:36
blackburndamn it21:36
shogun-buildbotbuild #8 of octave_modular is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org:5347/builders/octave_modular/builds/8  blamelist: blackburn91@gmail.com21:45
CIA-87shogun: Sergey Lisitsyn master * rdbf2ab3 / (5 files): Got rid of Preprocessor doxygen warnings - http://bit.ly/pBq6Zj21:45
blackburnhehe21:46
-!- alesis-novik [~alesis@188.74.87.206] has quit [Quit: Leaving]21:47
blackburnshogun-buildbot: status21:53
shogun-buildbotcmdline_static: building(compiling) [ETA 15m35s]21:53
shogun-buildbotcsharp_modular: building(compiling) [ETA 5m00s]21:53
shogun-buildbotjava_modular: building(compiling) [ETA 5m22s]21:53
shogun-buildbotlibshogun: idle, last build 1m37s ago: build successful21:53
shogun-buildbotlua_modular: building(compiling) [ETA 5m02s]21:53
shogun-buildbotnightly_all: idle, last build 53m14s ago: failed configure21:53
shogun-buildbotnightly_none: idle, last build 46m10s ago: build successful21:53
shogun-buildbotoctave_modular: building(compiling) [ETA 15m05s]21:53
shogun-buildbotoctave_static: building(compiling) [ETA 8m41s]21:53
shogun-buildbotpython_modular: building(compiling) [ETA 2 seconds], (compiling) [ETA 11m26s]21:53
shogun-buildbotpython_static: building(compiling) [ETA 8m34s]21:53
shogun-buildbotr_modular: building(compiling) [ETA 9 seconds], (compiling) [ETA 9 seconds]21:53
shogun-buildbotr_static: building(compiling) [ETA 15m48s]21:53
shogun-buildbotruby_modular: building(compiling) [ETA 4m59s]21:53
-!- f-x [~user@117.192.212.140] has joined #shogun21:57
shogun-buildbotbuild #10 of python_modular is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/python_modular/builds/1022:27
blackburn122022:27
shogun-buildbotbuild #11 of octave_modular is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/octave_modular/builds/1122:29
CIA-87shogun: Sergey Lisitsyn master * rb1d1a6b / (src/shogun/base/Parameter.h src/shogun/base/Version.h): Fixed more warnings (Parameter,Version) - http://bit.ly/pGn1YX22:59
-!- alesis-novik [~alesis@cpat001.wlan.net.ed.ac.uk] has joined #shogun23:15
blackburn98223:26
-!- blackburn [~blackburn@188.122.238.50] has quit [Quit: Leaving.]23:33
CIA-87shogun: Shashwat Lal Das master * r3b4d080 / (4 files): Added dump_regressor to VwRegressor. - http://bit.ly/p8REQb23:42
CIA-87shogun: Shashwat Lal Das master * r94fd8ae / (2 files): Made VW version part of env. - http://bit.ly/oY5IM323:42
CIA-87shogun: Shashwat Lal Das master * r5d60b6a / src/shogun/classifier/vw/VwRegressor.cpp : Minor type change in VwRegressor. - http://bit.ly/nEr9R723:42
CIA-87shogun: Soeren Sonnenburg master * r60e3bad / (6 files in 2 dirs):23:42
CIA-87shogun: Merge pull request #293 from frx/streaming_vw23:42
CIA-87shogun: dump_regressor added to VwRegressor - http://bit.ly/nLN9uo23:42
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]23:45
--- Log closed Sun Aug 14 00:00:28 2011
« Friday, 2011-08-12 Index Sunday, 2011-08-14 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!