IRC logs of #shogun for Friday, 2011-07-01

--- Log opened Fri Jul 01 00:00:28 2011
--- Day changed Fri Jul 01 2011
@sonney2kblackburn, but you do the git push github or so from the data/ dir?00:00
blackburnI was on <no branch>00:00
f-xsonney2k: but there's a problem00:00
f-xwe'd always be returning a float* vector00:01
f-xis that the way it should be done?00:01
@sonney2kf-x, no00:01
f-xthat's one place where i'm ever so confused00:01
@sonney2kf-x, so it has to stay00:01
@sonney2kf-x, and the StreamingFileFromSparseFeatures needs to be there too00:02
f-xsonney2k: yes, to cater to <T>00:02
blackburnsonney2k: update data and try examples/undocumented/python_modular/graphical/ (if have time)00:02
@sonney2kf-x, so I think we have a plan right?00:03
f-xsonney2k: best to go over it once more00:03
@sonney2kf-x, I mean you do the minor things in the current pull request just now such that I can merge before I go to bed00:03
@sonney2kand then do these StreamingFileFromDotFeatures for SGD00:04
f-xsonney2k: and SGD works purely on StreamingDotFeatures?00:04
@sonney2kso convert it then to use this00:05
f-xand it doesn't derive from LinearMachine anymore, but from OnlineLinearMachine?00:05
f-x(which has a CStreamingDotFeatures as a member)00:06
@sonney2kf-x, yes00:06
@sonney2kI think later we can do the same with SGD-QN00:07
@sonney2kand add a streamingliblinear variant00:07
blackburnI become a vim jedi day by day :D00:07
@sonney2kthat should all be very easy once you have done this for SGD00:08
CIA-32shogun: Sergey Lisitsyn master * r3e88af5 / examples/undocumented/python_modular/graphical/ : Fix for swissroll graphical example -
f-xsonney2k: yeah.. should be..00:08
blackburnnow I can write my own GPS system: I did a triangulation in LMDS hehe00:09
@sonney2kf-x, anyway for midterm it is sufficient to have a working SGD with the online system00:09
@sonney2kblackburn, you scary weasel!00:10
f-xsonney2k: yeah.. so i'm pretty eager to get it done as soon as possible00:10
blackburnthe most surprising thing about LMDS is that it is simply Nystom approximation to eigenproblem00:10
@sonney2kf-x, you could then run some benchmarks on data sets from
f-xsonney2k: right.. and properly tweak the parser..00:11
@sonney2kwith the parallel parsing stuff this should beat them all :)00:11
f-xsonney2k: in theory, certainly :)00:11
@sonney2kf-x, then ask John to organize a new LSL challenge and win :D00:12
blackburnrun my example run my example00:12
f-xsonney2k: ooh now you're dreaming :D00:12
@sonney2kf-x, I am pretty sure there is no one except John with vw in that does the parsing / learning in parallel00:13
f-xsonney2k: hmm.. john has done some pretty impressive things00:14
f-xbut i'll need help from both of you to optimize the parser for shogun00:14
f-xheaven knows the stuff john's done to get his parser running so fast00:15
@sonney2kf-x, anyway for now we have a plan - so please fix the pull request now00:15
f-xsonney2k: working on it... need to revert all the Stream* changes I made.. 5 minutes00:16
blackburnI hope this summer one guy will port JL's cover-tree00:16
@sonney2kblackburn, I guess you will :)00:17
@sonney2kblackburn, btw did you git add data and commit and push?00:17
@sonney2kIIRC this is the way to require the new data set in the shogun src repository00:18
blackburnhm stop00:18
blackburnI did git add toy00:18
blackburnin shogun/data00:18
blackburnnot in shogun00:18
@sonney2kblackburn, try in shogun too00:19
blackburnwill it work?00:19
blackburnI have nothing to change right now :)00:19
@sonney2kblackburn, it should00:20
@sonney2kif you don't want to do it I can try00:20
blackburnabout cover tree: I found one guy wishing to make some fancy things - he will port it :)00:20
blackburnif not - I will00:20
@sonney2kblackburn, great :)00:20
blackburnsonney2k: try my example ;)00:21
@sonney2kblackburn, I have to go over Heiko's emails and others too00:21
@sonney2kI will when I have more time (next week I am back to work)00:21
blackburnI will be crying as your baby :D00:22
blackburnsonney2k: aren't you working now?00:22
@sonney2kyesterday the baby was crying until midnight and I was too 0xdeadbeef to do anything00:22
@sonney2kblackburn, I am enjoying my shogun spare time :D00:23
blackburni.e. what you mean back to work :)00:23
blackburnwhoa, I want to do some research00:24
f-xsonney2k: pull request sent from a new branch.. closed the other one since that branch was getting messy with the Stream* stuff I had committed on my machine00:30
@sonney2kf-x, we don't have a SG_PRINTF00:33
@sonney2kbut just SG_PRINT00:33
@sonney2kf-x, I guess you should try to compile first :)00:33
f-x'make' worked since examples don't compile by default in my conf00:34
f-xone minute00:34
@sonney2kf-x, and can we get rid of the CStreamingFileFromSimpleFeatures in the example?00:34
@sonney2kI mean that could be nicely hidden in CStreamingSimpleFeatures00:35
f-xsonney2k: yeah, but that will take a bit more time00:35
@sonney2kf-x, ok postpone that one00:35
f-xbut it'll surely be done00:35
f-xsonney2k: i think the SG_PRINT thing is done.. iirc it was only in the example00:36
@sonney2kf-x, I am not sure if we should set FP_DOT or not better FP_STREAMING_DOT00:36
@sonney2kf-x, that is something to keep in mind too00:37
f-xwe could have FP_STREAMING separately00:37
f-xtwo properties?00:37
f-xone check for FP_STREAMING directly reveals whether it is any Streaming feature or not00:37
@sonney2kIIRC FP_DOT is set when the current dotfeatures interface is satisfied (so one can cast to CDotFeatures)00:37
@sonney2kso FP_STREAMING makes sense but it needs to be FP_STREAMING_DOT too00:38
@sonney2kbecause these are not dotfeatures compatibel00:38
f-xsonney2k: ok. i made the FP_DOT change in the hope that SGD could be made compatible with CDotFeatures00:38
f-xbut it can't now00:38
@sonney2kf-x, well you can just put the FP_STREAMING_DOT in there and add that enum00:38
f-xsonney2k: ok. adding00:39
@sonney2kto this herelibshogun/features/FeatureTypes.h00:39
@sonney2kf-x, you still have the multiple SG_INFO's in there00:41
@sonney2kthese should be SG_ERROR() with multiline string00:41
f-xsonney2k: where? i thought i removed them00:42
f-xone minute00:42
@sonney2kapart from that I am fine with your patch :)00:42
f-xokay StreamingFile.cpp00:42
f-xsonney2k: done, hopefully00:44
@sonney2kf-x, and it compiles now?00:46
f-xsonney2k: one minute, shogun isn't compiling for me because i don't have arpack i think00:47
blackburnarpack is not essential00:47
blackburnf-x: is it failing without arpack??00:48
f-xblackburn: i don't know.. my g++ doesn't have the HAVE_ARPACK defined either00:48
f-xone minute00:48
blackburnwhat is the error?00:48
f-xlib/arpack.cpp: In function ‘void shogun::arpack_dsaupd(double*, int, int, char*, double*, double*, int&)’:00:48
f-xlib/arpack.cpp:71:25: error: ‘dsaupd_’ was not declared in this scope00:48
f-xlib/arpack.cpp:108:54: error: ‘dseupd_’ was not declared in this scope00:48
f-x 00:48
* sonney2k tries to compile w/o arpack support00:49
@sonney2kblackburn, I guess you want to fix this - right?00:49
f-xsonney2k: compiles for you?00:50
CIA-32shogun: Sergey Lisitsyn master * rdd517a0 / src/libshogun/lib/arpack.cpp : Fix for arpack.cpp -
@sonney2kf-x, now yes :)00:50
blackburnsonney2k: could you try?00:50
blackburnok now?00:51
@sonney2kf-x, please compile test00:58
f-xsonney2k: yeah, the example?00:58
@sonney2kshogun + the example or did you already?00:58
f-xno, i mean i'm checking the example now00:59
f-xfunny it used to compile before.. you're getting the problem there, right?00:59
@sonney2kf-x, I mean I cannot all the time check if everthing compiles otherwise I need to pay €€€ to pay for the energy consumption00:59
@sonney2kf-x, blackburn fixed it already00:59
@sonney2kjust git pull00:59
f-xsonney2k: did all that.. i'm compiling my own example after that01:00
f-xare you getting a compile error?01:01
f-xsonney2k: revert, don't merge the patch now.. i'll fix it now so it can be merged tomorrow01:02
f-xsorry if you're getting an error01:02
@sonney2kf-x, ok01:02
f-xseems to be because i changed the printf to SG_PRINT and somehow it doesn't see the io object01:03
blackburnhmm i got a similar error while trying to add SG_PRINT to wrapper01:03
f-xblackburn: how did you fix it?01:03
blackburnI didn't fix it - removed SG_PRINT :D01:04
blackburnactually it wasn't needed there01:04
blackburnif you will find - tell me too :)01:04
f-xit compiles when i replace SG_PRINT to SG_SPRINT01:04
f-xdo you know what the difference between the two is?01:05
blackburnwhat is the difference between SG_PRINT and SG_SPRINT?01:05
f-xSG_SPRINT uses the sg_io object01:06
f-xSG_PRINT uses the io object01:06
f-xas far as implementation is concerned01:06
f-xdunno why the erro01:06
@sonney2kwhat is the error?01:10
f-xsonney2k: using SG_PRINT01:10
f-xshows 'io not declared in this scope'01:10
@sonney2kf-x, use SG_SPRINT01:10
f-xswitched to SG_SPRINT instead01:10
@sonney2kf-x, so it compiles now?01:11
f-xsonney2k: the example compiles01:12
f-xwere you getting the problem in the example?01:12
@sonney2kf-x, I didn't check at all01:12
@sonney2kf-x, I just wanted to say that you should compile before doing pull requests :)01:12
f-xsonney2k: had compiled earlier, but not after the SG_PRINT change :(01:13
f-xanyway everything compiles now (fingers crossed)01:13
@sonney2kf-x, ok then I can merge now?01:14
f-xsonney2k: oh and saw your previous statement about using SG_PRINT only within an SGObject.. hadn't seen it carefully when you actually said it01:14
f-xsonney2k: yes01:14
CIA-32shogun: Shashwat Lal Das master * r1265221 / src/libshogun/lib/arpack.cpp : Merge branch 'master' of into streaming_1 -
CIA-32shogun: Shashwat Lal Das master * r0bfb38f / examples/undocumented/libshogun/streaming_from_simple.cpp : Fixed the example to use SG_SPRINT instead of SG_PRINT -
CIA-32shogun: Soeren Sonnenburg master * r23bb602 / (12 files in 3 dirs):01:15
CIA-32shogun: Merge pull request #160 from frx/streaming_101:15
CIA-32shogun: Streaming*File changes as per other pull request -
f-xsonney2k: now i'll do a git clone to really make sure01:16
@sonney2kf-x, just did - works01:26
@sonney2keven java_modular compiles :D01:27
f-xsonney2k: okay, but it needs a minor fix01:28
f-xfor the example to work properly01:28
f-xi'll make a request01:28
f-xsonney2k: it's temporary, but handles the delete[] issue for now01:30
f-xi used to delete[] the vector after release_example() was called01:30
f-xbut we need to switch over to free_feature_vector()01:30
@sonney2kf-x, not urgent I would say01:31
@sonney2kdo it properly and then send a pull request01:31
@sonney2kanyways - I will have to go to bed now01:31
f-xsonney2k: okay.. see you!01:31
f-x(btw don't try the example without merging this pull request - it will result in an invalid delete[] since an arbitrary vector from SimpleFeatures cannot be freed like that)01:32
-!- f-x [~user@] has left #shogun ["ERC Version 5.3 (IRC client for Emacs)"]01:35
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]01:40
-!- in3xes_ [~in3xes@] has joined #shogun11:05
-!- blackburn [~blackburn@] has joined #shogun11:48
CIA-32shogun: Shashwat Lal Das master * r490545f / (3 files in 2 dirs): Made some changes for the delete[] to be called optionally after release of example from the parser. -
CIA-32shogun: Soeren Sonnenburg master * r3821870 / (3 files in 2 dirs):11:54
CIA-32shogun: Merge pull request #161 from frx/streaming_111:54
CIA-32shogun: Made some changes for the delete[] to be called optionally after release -
CIA-32shogun: Soeren Sonnenburg master * r473aefb / data : require new version of data -
blackburnwhat's up with data?11:58
@sonney2kblackburn, I did what you should have done :)12:03
@sonney2kgit add data ; git commit -a ; + push12:03
blackburnwhat is you told me yesterday12:03
blackburnI thought you are telling me how to add file to data12:03
@sonney2kblackburn, no this is to say that the shogun src repo needs a certain data version12:04
blackburnunderstand now12:05
@sonney2kblackburn, dammed I am being greeted today with: SMART file - your harddrive will die soon backup all data in the next 24 hrs12:39
blackburnSMART file?12:44
blackburnLLE acts slower with arpack..12:47
CIA-32shogun: Sergey Lisitsyn master * r3725079 / src/libshogun/preprocessor/LocallyLinearEmbedding.cpp : Codestyle fixes for LocallyLinearEmbedding -
blackburnsonney2k: which parameter is bad?13:02
@sonney2kand copying with 70k/S from a 2 TB drive supports this claim damn well :(13:02
blackburnreallocated sector count?13:03
blackburnhow much it is?13:03
@sonney2k  5 Reallocated_Sector_Ct   0x0033   017   017   036    Pre-fail  Always   FAILING_NOW 341113:03
blackburnI see13:03
blackburnhad it two times actually13:04
blackburnon my notebooks :D13:04
blackburnin my experience better to *not* try to read file if it is not reading well13:05
@sonney2kcannot cancel it13:05
@sonney2kI will try to copy as much as possible13:05
@sonney2kluckily it is my backup drive13:06
blackburnis it an old drive?13:06
@sonney2kthough I prefer to have backups13:06
blackburnhmm why it is failing13:06
@sonney2kmaybe 1-2 years13:06
blackburnpretty strange13:06
@sonney2kI will order a new one and see if I can RMA the drive in the meantime13:06
blackburnsonney2k: yes you should be able to return it13:07
blackburnI've returned one :)13:07
@sonney2kI only buy drives with >5 years warrenty so yes13:07
@sonney2kthough the seagate tool complains13:08
blackburnoh yes, in case of such a big warranty - exactly13:08
@sonney2kwebsite complains that my serial is unknown13:08
-!- in3xes_ [~in3xes@] has quit [Ping timeout: 240 seconds]13:08
@sonney2know it is copying with 80M/s13:08
-!- f-x [~user@] has joined #shogun13:21
-!- in3xes_ [~in3xes@] has joined #shogun13:27
-!- in3xes_ [~in3xes@] has quit [Ping timeout: 240 seconds]14:02
-!- in3xes_ [~in3xes@] has joined #shogun14:15
-!- in3xes__ [~in3xes@] has joined #shogun14:19
-!- in3xes1 [~in3xes@] has joined #shogun14:19
-!- in3xes [~in3xes@] has quit [Ping timeout: 240 seconds]14:23
-!- in3xes_ [~in3xes@] has quit [Ping timeout: 276 seconds]14:24
-!- heiko [] has joined #shogun14:34
-!- in3xes__ [~in3xes@] has quit [Quit: Ooops..!!]14:38
CIA-32shogun: Heiko Strathmann master * rde52e68 / src/libshogun/features/StringFeatures.h : fixed a segfault bug as pointed out by Jonas Behr -
-!- f-x [~user@] has quit [Ping timeout: 260 seconds]15:40
-!- alesis-novik [~alesis@] has quit [Quit: I'll be Bach]15:49
-!- blackburn [~blackburn@] has quit [Quit: Leaving.]16:00
-!- f-x [~user@] has joined #shogun16:07
-!- heiko [] has quit [Quit: Leaving.]16:28
-!- f-x [~user@] has quit [Remote host closed the connection]17:04
-!- alesis-novik [] has joined #shogun17:23
-!- f-x [~user@] has joined #shogun17:47
-!- blackburn [~blackburn@] has joined #shogun18:05
-!- in3xes1 is now known as in3xes18:32
-!- heiko [] has joined #shogun19:15
-!- alesis-novik [] has quit [Read error: Operation timed out]19:25
-!- alesis-novik [] has joined #shogun19:27
alesis-noviksonney2k, are you around?19:40
blackburnalesis-novik: how are you? :)19:49
alesis-novikHey blackburn. Fairly fine :)19:49
alesis-novikblackburn, maybe you can help19:49
blackburnmay be19:49
blackburnwhat is the issue?19:50
alesis-novikI'm nearly done with moving all fields to be SG* stuff19:50
alesis-novikbut what about destructors? do I clean up (free_*) or not?19:50
blackburnyes, you probably should do19:51
blackburne.g. in ClassicMDS I did it this way:19:51
blackburneigenvectors are stored in SGVector with do_free true19:51
blackburnwhen get_eigenvectors is called it returns the same but with do_free - false19:52
blackburnand in destructor I just do free_vector19:52
alesis-novikBut doesn't that free the gotten eigenvectors?19:52
blackburnyou mean if eigenvectors gotten somewhere and then after this object will be deleted?19:53
blackburnbad way19:53
blackburnwell in most cases nobody deletes objects certainly in their scripts19:54
alesis-novikThat's my issue I guess. That not only we have to be very careful internally, but the end user will also have to be very careful not to delete something useful19:54
blackburnthe only way to avoid this is to copy matrices/vectors..19:55
alesis-novikbut we decided against copying19:55
blackburnwell I can suggest one other fancy way to do it19:56
alesis-novikWell, copy or use an SGObject-like thing (SG_REF UNREF19:56
blackburnyes, SG_REF or may be store some bool variable holding true if we gave some internal vector19:56
blackburnand just check it in destructor19:57
alesis-novikBut that has to be done on a more global scale. I guess for now I'll just free in the destructor19:58
alesis-novik(hopefully nothing will break :D)19:58
blackburnwell really19:58
blackburnnobody do some 'del' in python or so19:58
blackburnif someone does - he should be carefully :)19:58
alesis-novikBut if someone would to use the library from c++ :D19:59
blackburnfor the brave - they should know what they does20:00
alesis-novikI really like the elegant way ref unref does it in, but it would give quite an overhead overall I guess20:04
-!- heiko [] has quit [Ping timeout: 258 seconds]20:04
blackburnI think it would be ok if you just write some ATTENTION ACHTUNG it doesn't copy :D20:07
in3xesf-x, coding wearing a lungi? :D20:30
CIA-32shogun: Sergey Lisitsyn master * rffa80d7 / src/libshogun/classifier/GaussianNaiveBayes.h : Fixed doc and some more at gaussian naive bayes -
-!- in3xes_ [~in3xes@] has joined #shogun21:54
-!- alesis-novik [] has quit [Ping timeout: 264 seconds]21:55
-!- alesis-novik [] has joined #shogun21:56
CIA-32shogun: Sergey Lisitsyn master * ra879603 / (2 files): ClassicMDS: Added warning for inconsistent embeddings, doc fixes -
-!- serialhex [] has quit [Remote host closed the connection]22:46
CIA-32shogun: Soeren Sonnenburg master * rb8bcb1c / (25 files in 4 dirs): get all octave_modular examples back to work -
CIA-32shogun: Soeren Sonnenburg master * r7a576ae / (3 files in 2 dirs): Merge branch 'master' of -
@sonney2kf-x, around?23:02
-!- alesis-novik [] has quit [Ping timeout: 240 seconds]23:03
-!- alesis-novik [] has joined #shogun23:04
blackburnoctave examples wasn't working? hehe23:06
-!- alesis_ [] has joined #shogun23:12
-!- alesis-novik [] has quit [Read error: Connection reset by peer]23:13
@sonney2kblackburn, well octave is still dying on exit23:14
@sonney2kbut at least the examples work23:14
blackburnf-k octave :D23:15
@sonney2know for R23:15
blackburnoh I have to write +1 wrapper for ARPACK23:16
blackburnI've found that some damned shift-invert mode is good for LLE23:17
blackburnsonney2k: what do you think about clojure?23:20
blackburnI think I'll make some clojure+shogun things during autumn or so23:22
f-xsonney2k: i'm here now23:25
@sonney2kblackburn, I have no idea about it23:28
@sonney2kf-x, I was looking at your patch23:28
f-xsonney2k: i saw your comments now23:29
@sonney2kf-x, can't we directly use the right feature type in constructor?23:29
blackburnsonney2k: it is something of interest because 'clojure guys' usually use mahout23:29
f-xsonney2k: well it's the template issue23:29
f-xwe can do it that way, but these classes have to be templated too23:29
@sonney2kf-x, ok but now you just cast to which type?23:31
f-xsonney2k: the casting is done by the read functions23:31
@sonney2kI mean you still have the same problem at some later stage23:31
f-xsonney2k: yes.. i didn't template them because they all derive from StreamingFileFromFeatures23:32
f-xwhich has a CFeatures object23:32
f-xif we template them, we can remove this features object in the base class and replace it with proper features objects23:33
f-xin the derived classes23:33
@sonney2kf-x, I still don't understand how you can be sure you have the right feature object at hand23:35
f-xsonney2k: right now, there's no check for that.. only the casting in the read* functions23:35
f-xwhat do you suggest? template them?23:35
f-xthat would be consistent, and i actually wanted to ask you about this very issue23:36
@sonney2kf-x, what would be the problem?23:36
f-xsonney2k: no problem, just a few changes in the base class..23:37
@sonney2kI mean SimpleFeatures/StringFeatures/SparseFeatures etc are all templated23:37
f-xsonney2k: yes, consistency would be there.. i should have followed the same pattern..23:38
@sonney2kf-x, I currently don't see a problem with templating again23:42
@sonney2kI mean we won't derive from these classes that are exactly defined for these feature types23:42
f-xsonney2k: that's true.. so now i'll remove that generic CFeatures object from the base class23:45
f-xleave it upto the derived classes to have their own specific Features objects23:45
--- Log closed Sat Jul 02 00:00:38 2011

Generated by 2.10.0 by Marius Gedminas - find it at!