IRC logs of #shogun for Monday, 2011-09-12

« Sunday, 2011-09-11 Index Tuesday, 2011-09-13 »
--- Log opened Mon Sep 12 00:00:22 2011
-!- in3xes [~in3xes@180.149.49.227] has joined #shogun04:43
-!- in3xes [~in3xes@180.149.49.227] has quit [Remote host closed the connection]05:17
-!- sonne|work [~sonnenbu@194.78.35.195] has joined #shogun09:00
sonne|worksomeone remind we that we should reduce compile time and resources needed09:00
-!- sonne|work [~sonnenbu@194.78.35.195] has quit [Ping timeout: 258 seconds]16:07
-!- sonne|work [~sonnenbu@194.78.35.195] has joined #shogun16:22
-!- sonne|work [~sonnenbu@194.78.35.195] has left #shogun []16:25
-!- blackburn [~blackburn@188.168.2.14] has joined #shogun18:57
blackburnsonney2k: have you any sparse matrix experience?18:58
@sonney2kblackburn, ?19:05
@sonney2kblack hole experience too19:05
blackburnsonney2k: I have troubles with understanding sparse format in superlu19:06
blackburnmay be you can help me somehow?19:06
@sonney2kis it ccs or crs ?19:07
blackburnsonney2k: do you realize I don't know anything about sparse matrices?19:07
@sonney2khttp://en.wikipedia.org/wiki/Sparse_matrix19:08
blackburncolumn19:08
@sonney2kbasically you can either compress the column or the row index or both19:08
blackburnhttps://gist.github.com/121178819:09
@sonney2kso what is your question now?19:09
@sonney2kyou put in the number of non-zero values into nnz19:10
@sonney2kall the values into nzval19:10
@sonney2k(array that is nnz long)19:10
blackburnyeah it is simple19:10
blackburnbut two last ones19:10
@sonney2krowptr is nrow+1 entries19:11
@sonney2kand has indices into colind19:11
blackburnso col_ind stores indexes of non zeros19:11
blackburnbut what stores rowptr19:12
@sonney2ktelling where each rows starts19:12
blackburnoh19:12
@sonney2kseems like this is crs19:13
blackburnyeah sure19:13
blackburnsonney2k: I've found http://netlib.org/linalg/html_templates/node91.html#SECTION0093110000000000000019:14
blackburnnot it is more clear, thanks19:14
blackburnnow*19:14
@sonney2kyes19:14
@sonney2kblackburn, I tried to compile shogun at work today19:17
@sonney2kand it was sooo slow19:17
@sonney2kthis is really the big downside now :/19:18
blackburnsonney2k: no idea how to avoid this19:18
@sonney2kotherwise it is pretty handy when analysing data19:18
@sonney2kwe have to avoid inline if it is not really really required19:18
blackburndo you think it makes compilation so slow?19:20
@sonney2kblackburn, I think the biggest share have the templates and the typemaps19:29
@sonney2kblackburn, the only workaround I see is to reduce the typemap code by creating a number of local (static) functions that the typemap code is calling for everything19:30
@sonney2ksuch that there is basically only a function call in each typemap19:30
@sonney2kregarding templates I don't know what we could improve19:31
@sonney2kmabye get rid of CArray* and just use SGVector etc19:31
blackburnsonney2k: yeah Streaming* features with 20 or so variants slowed down compilation19:32
blackburnevery time we add something template-based it gives us ~20 classes more :)19:33
blackburnsonney2k: didn't understand about static19:33
@sonney2kblackburn, btw for IIRC swig 2.0.4 and above we can use -builtin to swig20:01
@sonney2kso it should be faster / no need for .py files20:02
@sonney2kblackburn, yeah great code size increased from 608255 lines to 51677120:05
blackburnINcreased?20:05
@sonney2kincreased from 516771 to 60825520:09
@sonney2kyes20:09
blackburn.cxx?20:09
@sonney2kyes20:09
blackburnso compilation time is increased too, right?20:09
@sonney2kwith -modern and -modernargs it went down a little to 60577620:09
@sonney2kI dont' know it feels faster but that can be idealistic thinking20:11
* sonney2k recompiles20:11
@sonney2k3 minutes20:14
@sonney2kfor python_modular w/o optimizations20:14
@sonney2k3 minutes 25 sec with the old thing (500kLoC) and 3 minutes with the new one (600kLoC)20:19
@sonney2kweird huh?20:19
blackburnsonney2k: very20:20
@sonney2kand the old one needs a 2MB big modshogun.py20:20
@sonney2kthe new one not20:20
@sonney2kblackburn, try yourself20:23
blackburnsonney2k: try what?20:23
@sonney2kjust add -builtin -modern -modernargs to SWIGFLAGS in configure20:23
@sonney2kand then measure compilation time20:23
blackburnsonney2k: I have old swig20:23
@sonney2kpffh20:24
blackburnI get lazy to install new one :)20:24
@sonney2kblackburn, shall I force you by making it mandatory?20:39
@sonney2kseriously, I would love to get rid of the modshogun.py20:39
blackburnsonney2k: why you want to check on different machine?20:40
@sonney2kI don't trust myself20:40
@sonney2kblackburn, I had a look at the generated wrapper code. it is really tough to reduce it at all20:41
blackburnsonney2k: installing swig 2.0.420:42
@sonney2kI guess we can only reduce it by supporting fewer types / or merging a few classes20:42
@sonney2klike the array / sgvector one20:42
blackburnsonney2k: what have you measured?21:19
@sonney2ktime make21:19
@sonney2knot the configure21:20
blackburnokay21:20
blackburn????! I forgot to measure with old swig21:20
--- Log closed Mon Sep 12 21:24:23 2011
--- Log opened Mon Sep 12 21:24:39 2011
-!- shogun-irclog [~shogun@7nn.de] has joined #shogun21:24
-!- Irssi: #shogun: Total of 5 nicks [0 ops, 0 halfops, 0 voices, 5 normal]21:24
-!- Irssi: Join to #shogun was synced in 7 secs21:24
-!- sonney2k [~shogun@7nn.de] has joined #shogun21:24
blackburntook forever21:32
sonney2kblackburn, what took forever?21:46
blackburnsonney2k: compilation21:46
blackburnsonney2k:21:46
blackburnreal14m32.816s21:46
blackburnuser16m25.566s21:46
blackburnsys0m33.222s21:46
sonney2khow can real be lower than user?!? ahh multi core21:46
sonney2kok and is this with -builtin -modern -modernargs21:46
sonney2kor without?21:46
blackburnit is w/o21:47
sonney2kthen do w/21:47
blackburnin progress21:47
sonney2kblackburn, what is you machine btw?21:47
sonney2kcore 2 duo?21:47
blackburnsonney2k: intel pentium T430021:47
sonney2kI see21:48
sonney2kblackburn, but you did disable optimizations did you?21:48
blackburnsonney2k: not21:48
blackburn:D21:48
sonney2k--disable-optimization --enable-debug ?21:48
sonney2kok then let me try w/ optimizations enabled21:48
sonney2kblackburn, btw https://plus.google.com/114765095157367281222/posts21:50
sonney2klooks like our blog post got reshared by chris dibona :-))21:50
blackburnwow21:50
sonney2khope that give the project some more attention21:51
blackburnMUCH improves our chances to dive into next gsoc21:51
blackburn8 shares more21:51
sonney2kpreprocessor/LocallyLinearEmbedding.cpp: In member function ‘shogun::SGMatrix<double> shogun::CLocallyLinearEmbedding::find_null_space(shogun::SGMatrix<double>, int, bool)’:21:51
sonney2kpreprocessor/LocallyLinearEmbedding.cpp:327: warning: ‘eigenvectors’ may be used uninitialized in this function21:51
sonney2kdo you have that too?21:52
sonney2ka new warning21:52
blackburnsonney2k: yes I will fix it soon21:52
blackburneveryone says we did a great job21:54
blackburnand only I know I fucked up being too lazy :D21:54
sonney2know it compiles much longer it seems ...21:55
sonney2kblackburn, that is because you are extremely critical ...21:55
blackburnnot so extremely21:56
sonney2khttp://www.youtube.com/watch?v=zfTmpKmLRjM&feature=youtu.be - fun21:56
blackburn:)21:59
blackburnsonney2k: are markov models used for classification?21:59
sonney2kblackburn, 7 minutes 48s22:04
blackburnreal15m15.059s22:04
blackburnuser16m42.483s22:04
blackburnsys0m37.406s22:04
sonney2kso it is slower22:05
sonney2kbah22:05
blackburnsonney2k: I guess the same, I was watching youtube, etc22:05
blackburn20s of difference doesn't really mean22:05
sonney2kyeah but 45 seconds?22:05
sonney2kargh forgot the time infront of the make22:13
sonney2kabout 8 minutes22:13
sonney2kso very similar22:14
sonney2kbut no longer a .py monster of 2mb hanging around22:14
blackburnyeah22:14
sonney2kprobably now all the documentation is in the .cxx file22:14
sonney2kI guess that explains why it grew by 100kLoC22:14
blackburnah22:15
blackburnI see22:15
sonney2kthis should indeed not really increase compile time22:18
sonney2kI guess it is worth to require swig 2.0.4 and python2.6 for the next version and enable that feature22:19
blackburnsure22:19
sonney2kalright22:24
sonney2kbed time - will train some GNB tomorrow btw :)22:24
sonney2kso I need to be prepared to fix your code ;-)22:24
blackburnsonney2k: uhh22:24
blackburnsonney2k: have you saved my python gnb code?22:25
sonney2kI don't have it22:25
sonney2kseems like you implemented it for N classes even...22:25
blackburnsonney2k: hmm a minute22:25
sonney2kI am really going to bed now - please email thanks!22:26
blackburnokay22:26
blackburnsee you22:26
-!- blackburn [~blackburn@188.168.2.14] has quit [Quit: Leaving.]23:13
--- Log closed Tue Sep 13 00:00:26 2011
« Sunday, 2011-09-11 Index Tuesday, 2011-09-13 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!