IRC logs of #shogun for Friday, 2011-07-29

--- Log opened Fri Jul 29 00:00:46 2011
@sonney2kblackburn, hmmhh I just checked out some hosting solutions for a potential build bot00:02
serialhexsonney2k: i'm working on getting shogun to compile correctly with different versions of ruby, then i will work on making sure the examples work.00:02
@sonney2k60 € / month for a decent quad core with 4G memory00:03
@sonney2kshould be sufficient I would say00:03
@sonney2k60 EUR00:03
serialhexthough right now i can't get shogun to work at _all_ with my system set-up, though i have narray & ruby (both 1.8 & 1.9 installed)00:03
@sonney2kserialhex, I meant - how do I even run an example?00:03
blackburnsonney2k: are you sure we need it?00:03
serialhexoh, it should be: ruby example.rb00:03
@sonney2kblackburn, I think it is better than running things here on my home machine00:03
blackburnwhich things?00:04
@sonney2kat least that would mean faster webserver  / other devels could login to build...00:04
@sonney2kblackburn, build service00:04
blackburnI don't understand.. what is build service?00:04
@sonney2kserialhex, yes but when I run it from the examples dir it cannot find modshogun - how do I specify the ruby include path to fix that?00:05
serialhexi think sploving got classifier_libsvm_minimal_modular.rb running fine... but idk about any others...00:05
@sonney2kblackburn, just building00:05
serialhexahh, ok i see00:05
@sonney2kserialhex, I think the others work to - I am just clueless how to run them00:05
blackburnwell for me it is ok to compile here..00:06
serialhexbtw: i *love* physical books... dead trees rock!00:07
@sonney2kblackburn, yes sure, but we need automated build tests00:09
@sonney2kblackburn, or did you know that static interfaces where all broken for some time?00:09
@sonney2kand blackburn we need to run this on osx / cygwin too00:11
blackburnhmm I see00:11
serialhexok sonney2k, this is the pain-in-the-ass-until-we-get-ruby_modular-stuff-installed way to do things: ruby -I../../../src/interfaces/ruby_modular example.rb #(that is a capital i)00:11
serialhexthat will prepend the ruby_modular dir to ruby's $LOAD_PATH and should allow you to run the example from the examples dir00:12
serialhexnow - off to eat00:12
@sonney2kthanks00:13
@sonney2k*yawn*00:42
@sonney2knite folks00:42
blackburnsee you00:42
-!- blackburn [~blackburn@109.226.104.206] has quit [Quit: Leaving.]01:43
-!- in3xes [~in3xes@180.149.49.230] has quit [Quit: Leaving]04:57
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun05:58
CIA-87shogun: Soeren Sonnenburg master * r107276d / examples/undocumented/ruby_modular/check.sh : add a ruby examples check script - https://github.com/shogun-toolbox/shogun/commit/107276dba7c6f4cb8713b6e81a887f6f8dbada1d07:43
CIA-87shogun: Soeren Sonnenburg master * r620037e / examples/undocumented/ruby_modular/check.sh : add single file test - https://github.com/shogun-toolbox/shogun/commit/620037ed381d27a99b8dc2ce304b85fe58a0159d07:43
CIA-87shogun: Soeren Sonnenburg master * r01b3fc3 / src/NEWS : Merge branch 'master' of github.com:shogun-toolbox/shogun - https://github.com/shogun-toolbox/shogun/commit/01b3fc30fe6a5a4926c0f750177efe8b7100610a07:43
-!- sploving1 [~sploving@124.16.139.134] has joined #shogun09:27
@sonney2ksploving1, how is it going?10:24
sploving1sonney2k, not work yet10:27
-!- heiko [~heiko@134.91.54.89] has joined #shogun10:40
@sonney2ksploving1, I have an idea - maybe you *for testing* just add a function set_foo(SGMatrix<float64_t> m) { printf("set\n"); } to Label.h10:41
@sonney2ksploving1, or add a constructor to some other class that takes a SGMatrix...10:41
@sonney2kmoin heiko10:41
@sonney2kheiko, you did allocations on stack - no chance the malloc trace stuff couldn detect that...10:42
heikomoin10:42
heikooh10:42
heikowhere did i do that?10:43
@sonney2kheiko, in your example10:43
heikooh10:43
heikoi see10:43
@sonney2kI 'fixed' that10:43
heikothats  why it showed allocated blocks10:43
heikodamn, could have thought about that10:44
@sonney2kheiko, yeah - I mean there was no SG_UNREF for certain objects so ...10:44
heikohuh, but valgrind said it was clean10:45
heikoi will have a look10:45
sploving1sonney2k, minimal_test.cs in c#dir , how to change it to use modshogun?10:46
CIA-87shogun: Soeren Sonnenburg master * r89bf991 / src/shogun/lib/DataType.h : introduce SGCachedVector prototype - https://github.com/shogun-toolbox/shogun/commit/89bf991a142d09d55b2e99e171e3e655e0f90e9010:46
@sonney2kheiko, btw ^10:46
@sonney2khave a look at my cached vector attempt10:46
@sonney2kmaybe it works for simplefeautres10:46
@sonney2kheiko, feel free to work on this further!10:46
heikook,10:46
sploving1how let it find modshogun, in c#?10:47
heikosonney2k, you only did change the grid searh linear example, right?10:47
@sonney2kyes and more10:47
@sonney2kyour string slen patch caused build failures all over...10:47
heikooh no :(10:47
@sonney2ksploving1, does it need to be modified even?10:48
heikosorry for that, swig typemaps?10:49
@sonney2kheiko, typemaps and static interfaces and examples10:49
-!- in3xes [~in3xes@180.149.49.227] has quit [Remote host closed the connection]10:49
sploving1sonney2k, yeap. I donot know how to let find the lib modshogun.so. I googled csharp libraray path, but no good results10:50
@sonney2ksploving1, just set the LD_LIBRARY_PATH IIRC10:51
sploving1then whant does the Library.init_shogun_with_defaults();10:51
sploving1GaussianKernel k = new GaussianKernel();10:51
sploving110:51
sploving1this is what you write in the minimal_test.cs10:52
@sonney2kthe same like in java10:52
sploving1you mean deal with c# exactly like java?10:52
sploving1javac path and java path?10:53
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun10:53
@sonney2ksploving1, ?10:53
@sonney2kof course not10:53
sploving1there is no c# expert:(10:55
sploving1here10:55
-!- blackburn [~blackburn@109.226.104.206] has joined #shogun11:08
-!- sploving1 [~sploving@124.16.139.134] has left #shogun []11:13
-!- [1]warpy [~warpy@bzq-79-181-25-215.red.bezeqint.net] has joined #shogun11:45
[1]warpyhey guys, anyone here ?11:48
@bettyboohug11:48
heikohi11:53
heikosonney2, just found this: int32_t *ClList = (int32_t*) calloc(XSize, sizeof(int32_t));11:53
heikoin KMeans, i guess this should be replaced by SG_CALLOC, right?11:53
@sonney2kheiko, yes12:12
-!- gsomix [~gsomix@88.200.214.229] has joined #shogun12:12
gsomixhi all12:12
@bettyboohiho12:12
@sonney2k[1]warpy, please ask here12:12
heikosonney2, just thought about distance machines could store their model. I think they just have to completely copy the feature data of their distance variable12:16
@sonney2kheiko, ?12:17
@sonney2kisn't it the same like kernelmachine?12:17
heikokernelmachine only has to store its SV12:18
@sonney2kdistance machine too or?12:18
heikokernel machine only has one variable: distance12:19
@sonney2kok KNN is an exception ...12:19
@sonney2kheiko, did you mean distance machine?12:19
* sonney2k is getting confused12:19
heikoyes12:19
heikoi was currently working on KNN12:19
heikoin apply, the method distances_lhs() is called which compute distances to ALL feature vectors12:20
heikoso all of them have to be copied12:20
@sonney2kheiko, yes for KNN it is true12:20
@sonney2kinfact KNN has no training procedure12:20
heikotrue12:20
heikommh12:21
heikoso store_model does not have to do anything12:21
@sonney2kwell it needs the training data when testing12:22
heikook then12:22
@sonney2kso I think it is correct to assume that it has to copy all training data12:22
heikoKMeans does not even have an apply method,12:22
heikoso no use in model selection anyway12:22
heikobecause it is called12:22
heikosame for hierachical12:23
@sonney2kheiko, well one could define an apply method in k-means that just assigns the nearest cluster center12:23
@sonney2ksame for other clustering methods12:23
heikook12:23
heikobut in both KMeans, no model storage is needed since radius and distances are stored in new arrays anyways12:24
heikoso DistanceMachine will just not do nothing in store_model_features and KNN will overwrite it and store all feature data12:25
@sonney2kheiko, makes sense... maybe more general would be to do distance machine in the same way as kernel machine.12:26
@sonney2kon the other hand we don't have such machines yet...12:26
heikoyes, and also there is no variable that saves indices of needed features12:27
@sonney2kso ok then just do it for KNN12:27
heikosonney2k, just had another thought about kmeans12:32
@sonney2kwhich is?12:32
heikoit saves centers for clusters in a variable12:32
heikowhy not store these in the underlying distance variable12:32
heikothen in apply, the distance_lhs method could be used12:32
heikoif i do this now, i have to compute distance to all cluster centers by hand12:33
-!- gsomix [~gsomix@88.200.214.229] has quit [Ping timeout: 252 seconds]12:33
@sonney2kheiko, you mean just create a new feature object with centers?12:34
CIA-87shogun: Heiko Strathmann master * rdba50f2 / (3 files):12:35
CIA-87shogun: -made search space smaller12:35
CIA-87shogun: -removed stuff from stacdk to heap to be able to trace memory blocks12:35
CIA-87shogun: -cleaned up comments - https://github.com/shogun-toolbox/shogun/commit/dba50f2be70b444ea53778284c06ad878bcff04712:35
CIA-87shogun: Soeren Sonnenburg master * r3ba5d3b / (3 files):12:35
CIA-87shogun: Merge pull request #246 from karlnapf/master12:35
CIA-87shogun: example corrections - https://github.com/shogun-toolbox/shogun/commit/3ba5d3b7fd505a444d5d8bdb6dd3595e5aa79aa212:35
heikosonney2k, yes like this12:41
heikowhat do you think?12:41
heikothen the code of DistanceMachine could be reused in apply12:43
@sonney2kheiko, yes i like it12:44
-!- gsomix [~gsomix@109.169.238.89] has joined #shogun12:46
-!- in3xes_ [~in3xes@180.149.49.227] has joined #shogun12:51
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]12:55
-!- in3xes__ [~in3xes@180.149.49.227] has joined #shogun13:00
-!- in3xes__ is now known as in3xes13:01
-!- in3xes_ [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]13:03
[1]warpyhey sonney2k, when you have 1600 vectors, each has around 2mb of data, what is the best way to run a learning process on it. is it to load it all into memory, or is there some kind of trick that you used in shogun ?13:21
@sonney2k[1]warpy, I would use a kernel machine and precompute the kernel matrix13:28
CIA-87shogun: Soeren Sonnenburg master * r2d44753 / (2 files in 2 dirs):13:30
CIA-87shogun: fix compile csharp errors13:30
CIA-87shogun: - made read_vector* variables protected such that there is no get_/set_13:30
CIA-87shogun: naming clash13:30
CIA-87shogun: - use same signature for get_num_vectors() in attributefeatures to fix warning - https://github.com/shogun-toolbox/shogun/commit/2d44753eb5506eb6415c5dfee71e082754cc83fb13:30
CIA-87shogun: Soeren Sonnenburg master * r035f3cb / examples/undocumented/csharp_modular/minimal_test.cs : fix minimal_test example to work with modshogun - https://github.com/shogun-toolbox/shogun/commit/035f3cbd443ae6eba756fb7cc609305ad90be90c13:30
CIA-87shogun: Soeren Sonnenburg master * r1510d31 / (3 files): Merge branch 'master' of github.com:shogun-toolbox/shogun - https://github.com/shogun-toolbox/shogun/commit/1510d315efcc4018dcbf47838a561ddc6a8d77cc13:30
@sonney2kheiko, in your machine train transition - did you make the train_machine function private/protected?13:31
heikoyes13:32
CIA-87shogun: Soeren Sonnenburg master * r37e9718 / (30 files in 3 dirs):13:32
CIA-87shogun: Merge pull request #247 from karlnapf/master13:32
CIA-87shogun: completely applied renaming train -> train_machine (was only done for KernelMachines before) - https://github.com/shogun-toolbox/shogun/commit/37e9718706668604cdda9d1f69a20ca0ee3a6ba013:32
@sonney2kyes13:32
heikoyes?13:32
@sonney2kyes!13:33
heiko:D13:33
@sonney2kheiko, I hope you are now always compiling with trace mallocs :)13:36
heikoyes i do13:36
heikowhy?13:36
heikobut my examples become very slow this way13:36
heikosince there are ~0.5mio allocs :)13:36
heikotherefore i made them smaller so that make-tests runs in a reasonable time13:36
[1]warpysonney2k, is this something that is done in shogun with a couple of commands already ?13:41
@sonney2kheiko, yeah your examples should be *really* small anyways - these will be run on some poor arm based machines too...13:42
heikotrue :)13:42
@sonney2k[1]warpy, if you can squeeze your examples into memory then yes otherwise you need to do the splitting / merging yourself13:44
CIA-87shogun: Soeren Sonnenburg master * r29575c9 / examples/undocumented/csharp_modular/check.sh :13:49
CIA-87shogun: add proper check.sh for csharp13:49
CIA-87shogun: syntax is13:49
CIA-87shogun:  ./check.sh minimal_test.cs13:49
CIA-87shogun: or just13:49
CIA-87shogun: ./check.sh - https://github.com/shogun-toolbox/shogun/commit/29575c97d1af503136bd48221cff86355e1f7a3013:49
heikosonney2k, just read through KMeans::train. the method sets the cluster centers as rhs anyways13:49
@sonney2k[1]warpy, btw I just tried - you don't need any org - whatever in csharp13:49
@sonney2kjust have a look at the minimal_test.cs example13:49
heikobut I dont understand why rhs is used for this13:49
heikoand not lhs13:50
@sonney2kheiko, isn't that symmetric?13:50
heikooh, that might be13:50
@sonney2kI mean after training it makes sense to change it but before - doesn't really matter13:50
heikowhat do you mean by this?13:51
heikooh, mmh I think this is because some classification is done13:52
heikosomething random is going on there13:52
heikois this to compute variances?13:52
heikothen rhs would make sense13:52
@sonney2k[1]warpy, and please get Load.cs to work13:53
@sonney2kI am closing your pull request for now13:53
@sonney2kLoad.cs should just read an array13:53
@sonney2kerr read an ascii file13:53
@sonney2kand then return arrays with numbers13:53
[1]warpysonney2k, what do you mean org ?13:56
@sonney2kall the using stuff13:56
@sonney2kall the org.shogun / org.jblas13:56
@sonney2klaodlibrary stuff13:56
@sonney2kshould all go13:56
[1]warpydid you pull it ?13:57
@sonney2klook at the minimal example13:57
@sonney2kI closed the request13:57
@sonney2kplease send updated examples13:57
[1]warpyokay13:57
[1]warpyi will remove them all13:57
[1]warpywhat about System.loadLibrary("modshogun");14:07
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 258 seconds]14:25
[1]warpysonney2k,14:26
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun14:45
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 276 seconds]14:51
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun15:02
@sonney2k[1]warpy, as I said above - not needed15:20
CIA-87shogun: Evgeniy Andreev master * raf57ba7 / (2 files): Arrays is replaced by SGVector in CGaussianNaiveBayes - https://github.com/shogun-toolbox/shogun/commit/af57ba7c3682fbbef3c80668bee31a7f3a583ce715:21
CIA-87shogun: Soeren Sonnenburg master * rccd86e9 / (2 files):15:21
CIA-87shogun: Merge pull request #248 from gsomix/master15:21
CIA-87shogun: Arrays are replaced by SGVector in CGaussianNaiveBayes - https://github.com/shogun-toolbox/shogun/commit/ccd86e93c704b6d1e53637039ab81310cdcaa84d15:21
-!- [1]warpy [~warpy@bzq-79-181-25-215.red.bezeqint.net] has quit [Quit: HydraIRC -> http://www.hydrairc.com <- Po-ta-to, boil em, mash em, stick em in a stew.]16:17
-!- heiko [~heiko@134.91.54.89] has left #shogun []16:48
-!- f-x [~user@117.192.206.212] has joined #shogun16:55
-!- Ziyuan [~Ziyuan@116.21.254.24] has joined #shogun17:24
ZiyuanI opened mathematics/Math.h and got a few errors, like sqrtf cannot be resolved17:47
ZiyuanMy "math.h"'s problem?17:48
@sonney2kZiyuan, you need POSIX.1-2001. system18:57
-!- genix [~gsomix@95.67.178.232] has joined #shogun18:57
-!- gsomix [~gsomix@109.169.238.89] has quit [Ping timeout: 260 seconds]19:00
-!- in3xes_ [~in3xes@180.149.49.227] has joined #shogun19:00
CIA-87shogun: Heiko Strathmann master * r63fada6 / (2 files):19:02
CIA-87shogun: added init method19:02
CIA-87shogun: made store_model_features do nothing - https://github.com/shogun-toolbox/shogun/commit/63fada6e196ece6f418e146d6b7ebd97c22783ef19:02
CIA-87shogun: Heiko Strathmann master * r660b180 / src/shogun/machine/DistanceMachine.cpp : store_model_features is now in header file - https://github.com/shogun-toolbox/shogun/commit/660b1805f3ba1476064b5390981dd327b952c9f919:02
CIA-87shogun: Heiko Strathmann master * rc94f370 / src/shogun/machine/DistanceMachine.cpp : added parameter include - https://github.com/shogun-toolbox/shogun/commit/c94f3705178f992d98b2f75e527933daa25d603519:02
CIA-87shogun: Soeren Sonnenburg master * r91831a1 / (2 files):19:02
CIA-87shogun: Merge pull request #249 from karlnapf/master19:02
CIA-87shogun: fix small issues - https://github.com/shogun-toolbox/shogun/commit/91831a1d5307c860f27910bc76127fe0c2e83b6b19:02
-!- in3xes [~in3xes@180.149.49.227] has quit [Ping timeout: 240 seconds]19:04
-!- in3xes_ is now known as in3xes19:07
-!- genix [~gsomix@95.67.178.232] has quit [Quit: Ухожу я от вас (xchat 2.4.5 или старше)]19:07
-!- f-x [~user@117.192.206.212] has quit [Remote host closed the connection]19:32
-!- blackburn [~blackburn@109.226.104.206] has quit [Quit: Leaving.]19:58
-!- blackburn [~blackburn@109.226.104.206] has joined #shogun20:04
-!- srikanth [~mrsrikant@59.92.80.200] has joined #shogun20:39
@sonney2kblackburn, when you do the java tests, could you please compile with trace mallocs and then check the output at the end?21:10
@sonney2kthere should be no leaks...21:10
blackburnyes, okay21:10
blackburnsonney2k: have you any machine with ACML?21:10
@sonney2kacml?21:11
blackburnyeah, amd core math library21:11
blackburnwe have various ifdefs in lapack about ACML21:11
blackburnsonney2k: I have stupidity assault, what is the clapack_* things in lapack.{h,cpp}?21:18
blackburnit is instead of ATLAS, right?21:19
@sonney2kblackburn, when atlas is not available we can still fall back to blas/lapack21:22
@sonney2kand acml/mkl etc all are compatible (with some variance in __ naming)21:23
blackburnaha I see21:23
-!- srikanth [~mrsrikant@59.92.80.200] has quit [Quit: Leaving]21:25
-!- Ziyuan [~Ziyuan@116.21.254.24] has quit [Ping timeout: 258 seconds]21:34
@sonney2kblackburn, uh oh leaks everywhere21:48
blackburnsonney2k: where?21:48
@sonney2kin the modular interfaces it seems ... I tried lua and python21:48
blackburnbad21:49
@sonney2kthe good news is that we can trace the things down easily now21:50
@sonney2kblackburn, some of them are false alarms though...21:58
blackburnwow I got 2x speedup for arpack haha22:03
@sonney2kcongrats!22:08
blackburnsome inverse -> LU system solving transition22:09
blackburnhaha 4x for LTSA22:13
blackburnsonney2k: LTSA took 34s for 5000 examples22:15
-!- in3xes [~in3xes@180.149.49.227] has quit [Quit: Leaving]22:55
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun22:57
blackburnsonney2k: is there any way to disable atlas? :)23:15
blackburn* removed atlas23:18
blackburn:D23:18
@sonney2kdisable-lapack23:20
blackburnnot, need to disable atlas23:21
CIA-87shogun: Sergey Lisitsyn master * r2c7f253 / (2 files): Added DPOTRS and DGETRS wrappers for LAPACK - https://github.com/shogun-toolbox/shogun/commit/2c7f2531f27cc259ae43468d9f12cc34c1990d2423:43
CIA-87shogun: Sergey Lisitsyn master * rf1167d6 / src/shogun/mathematics/arpack.cpp : Improved ARPACK performance by getting rid of matrix inverse - https://github.com/shogun-toolbox/shogun/commit/f1167d6343e7e947da8c92fd71deee3b3d32cbd823:43
--- Log closed Sat Jul 30 00:00:52 2011

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!