IRC logs of #shogun for Sunday, 2011-09-25

« Saturday, 2011-09-24 Index Monday, 2011-09-26 »
--- Log opened Sun Sep 25 00:00:25 2011
-!- blackburn [~blackburn@31.28.44.65] has quit [Quit: Leaving.]00:59
-!- sonne|work [~sonnenbu@194.78.35.195] has quit [Ping timeout: 260 seconds]06:00
-!- sonne|work [~sonnenbu@194.78.35.195] has joined #shogun06:14
-!- mrsrikanth [~mrsrikant@59.92.79.76] has joined #shogun07:11
-!- mrsrikanth [~mrsrikant@59.92.79.76] has quit [Quit: Leaving]08:35
-!- blackburn [~blackburn@31.28.44.65] has joined #shogun10:53
blackburnsonney2k: why we have no -msse enabled?11:36
blackburnor msse211:37
blackburnor msse311:37
blackburnsonney2k: I have enabled -msse -msse2 -msse311:53
blackburnbefore:11:53
blackburnSHOGUN Took 5.209960s11:53
blackburnafter:11:53
blackburnSHOGUN Took 3.441315s11:53
CIA-3shogun: Sergey Lisitsyn master * r3f81b07 / (src/configure src/shogun/preprocessor/KernelPCA.h): Added MSSE{1,2,3} flags to configure - http://git.io/LKTUKA11:57
blackburnoh11:59
blackburnforgot to commit KernelPCA separately :)11:59
blackburnsonney2k: arpack dsymv issue resolved!12:59
CIA-3shogun: Sergey Lisitsyn master * r134f863 / (4 files): Improved memory usage efficiency in DR preprocessor - http://git.io/mDfTTQ13:34
-!- mrsrikanth [~mrsrikant@59.92.84.60] has joined #shogun14:30
blackburnsonney2k: !16:24
blackburnsonney2k: what the fuck is in cpudetection?! it is not working anyhow..16:26
blackburnwhile sse is detected ok I have no -msse16:27
blackburnand 3dnow too16:27
blackburnk6? k7? :D16:29
CIA-3shogun: Sergey Lisitsyn master * rd6ac38d / src/configure : Fixed mistype at configure, changed flags - http://git.io/dmJMOw16:31
CIA-3shogun: Sergey Lisitsyn master * r2642355 / src/shogun/lib/DataType.h : Added [] for SGMatrix - http://git.io/xD20kw16:31
blackburnwhat is O9??16:59
CIA-3shogun: Sergey Lisitsyn master * r16e761a / (6 files in 2 dirs): Updated [] for SGMatrix and made it used by DR preprocessors - http://git.io/xvUnrw17:09
CIA-3shogun: Sergey Lisitsyn master * r75685b6 / src/shogun/preprocessor/LocallyLinearEmbedding.cpp : Removed junk from LLE - http://git.io/NB-MsA17:40
@sonney2kblackburn, re19:51
blackburnout for dinner :)19:51
@sonney2kcu19:51
@sonney2kblackburn, I am reverting you msse patch19:53
@sonney2kblackburn, what are you doing??! Look at 3f81b07f4716de1b5c40f801d81a62bf071b7732 - it says Added MSSE{1,2,3} flags to configure19:55
@sonney2kbut in fact it changes stuff in KernelPCA.h19:55
blackburnsonney2k: back19:56
blackburnsonney2k: why to revert it?19:56
@sonney2kwe are using march=native19:56
@sonney2kthat should enable every possible speedup19:56
blackburnit doesnt'19:57
blackburnsonney2k: with sse enabled thing became 1.5x faster19:57
blackburnit doesn't enable sse/sse2/sse319:57
blackburnI can say it for sure19:58
@sonney2kit does enable it19:58
@sonney2kI am really sure19:58
@sonney2kI am not sure if mfpmath sse is set though19:58
@sonney2kanyhow please don't do such commits19:59
blackburnsuch like what?19:59
@sonney2kI mean don't write you did 'X' and then do 'X' and 'Y'19:59
@sonney2kdo git commit configure19:59
@sonney2kand git commit KernelPCA.h19:59
@sonney2k*separately*19:59
blackburnsonney2k: if you read my message here you would know I did it by mistake20:00
blackburnokay I will check if removing sse won't change speed20:00
@sonney2kok20:00
@sonney2kblackburn, the gcc doc says mfpmath=sse is enabled on x86_64 by default20:01
blackburnsonney2k: I have x86_3220:01
blackburnsonney2k: I'm not sure sse is enabled with native20:02
@sonney2kblackburn, it is - just not for math stuff20:02
blackburnsonney2k: eh?20:02
blackburnsonney2k: what is O9?20:03
blackburnand why not O3?20:03
@sonney2kbtw http://ubuntuforums.org/showthread.php?t=1477356 suggests that mfpmath=both is even faster20:08
@sonney2kblackburn, what did you do that arpack works now on cheng's machine?20:09
blackburn../../shogun/preprocessor/Isomap.h:97: warning: inline function ‘int32_t shogun::CIsomap::get_k() const’ used but never defined20:09
blackburnany suggestion why I got that?20:09
blackburnsonney2k: I guess removing unnecessary #include <cblas.h> helped20:09
@sonney2kblackburn, ok then it must have been some bad interaction between arpack/atlas20:10
@sonney2kclbas20:10
blackburnsonney2k: arpack is not involved here20:10
blackburnthe error was related to my code20:10
@sonney2kblackburn, why not? it was arpack where cheng's stuff failed20:11
blackburnsonney2k: my arpack *wrapper*20:11
@sonney2kand?20:11
@sonney2kit includes arpack header or?20:11
blackburnheader?20:11
blackburnit is fortran 77 lib :)20:11
@sonney2kI don't understand then why it works now20:12
blackburnsonney2k: lapack.h includes cblas.h20:12
blackburnarpack.h was too20:12
blackburnthe only reason I can mind20:12
blackburnsonney2k: can you remind me how to impl inline function in .cpp?20:13
@sonney2khttp://ondioline.org/mail/cmov-a-bad-idea-on-out-of-order-cpus20:14
@sonney2kcmov should go20:14
blackburnokay20:14
blackburnsonney2k: there are a lot of cpu detection useless things, would you remove it?20:14
@sonney2kwhat is useless?20:14
blackburnsonney2k: I will check if sse is really enabled on my machine20:15
@sonney2kbtw -ffast-math might help a lot20:15
@sonney2k-msse is not needed20:15
blackburne.g. detection of amd k620:15
blackburn:D20:15
@sonney2kwell shogun exists for some time - so it will be tuned for old cpu's too20:15
@sonney2kbut fast-math might cause trouble - if we enable that we should check if the test suite still works20:15
blackburnThis option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications for math functions20:16
blackburnlets avoid it20:16
@sonney2kread on please20:16
@sonney2k           On Darwin systems, the math library never sets "errno".  There is20:16
@sonney2k           therefore no reason for the compiler to consider the possibility20:16
@sonney2k           that it might, and -fno-math-errno is the default.20:16
blackburnsonney2k: we can set on some things contained under -ffast-math20:17
blackburne.g. we can't enable finite math20:17
@sonney2khow about mfpmath=both?20:19
blackburnsonney2k: I will check in a min20:20
blackburnjust need to finish .h->.cpp moving20:20
@sonney2kmy take is we enable everything when the test suite still passes20:20
@sonney2kok20:20
@sonney2kregarding your err ./../shogun/preprocessor/Isomap.h:9 -  you probably have to remove inline ?20:21
blackburnsonney2k: already did20:21
blackburnsonney2k: so you didn't answered, why not O3?20:21
@sonney2kI just used the highest possible level20:22
@sonney2kfeel free to change it to O320:22
blackburnehm?20:22
blackburnsonney2k: some options enabled now is O3 level20:22
blackburnso may be just enable all of them would make things faster20:22
blackburnI feel myself speed maniac today hah20:23
@sonney2kI don't understand what you are saying20:23
blackburnnevermind20:23
blackburnsonney2k: have you made progress on python interface?20:24
@sonney2k?20:24
@sonney2kyou mean the array_interafce?20:25
blackburnyes20:25
@sonney2kI learned that one should implement the protocal buffer interface http://docs.python.org/dev/c-api/buffer.html#PyObject_GetBuffer20:26
@sonney2kand now I dont' know how to get a handle to the PyObject that swig creates20:26
blackburnbad20:27
@sonney2kto modify the PyTypeObject20:27
blackburnarray__ looked better :)20:27
blackburnsonney2k: there is some mentions of sse is not always being enabled with native20:29
@sonney2kblackburn, well no20:29
@sonney2kthe buffer protocol is much better20:30
@sonney2kit is default in python2.6 / 3.x20:30
@sonney2kand supported by more than numpy20:30
blackburnsonney2k: I removed sse, will check if it became slower20:31
-!- mrsrikanth [~mrsrikant@59.92.84.60] has quit [Quit: Leaving]20:32
blackburnsonney2k: okay believe you, sse is enabled20:34
blackburnsonney2k: predictive commoning speed down compilation really much..20:40
blackburnsonney2k: fpmath=both is slower21:31
@sonney2kseems like a lot depends on the actual algorithm you are trying to run21:34
blackburnsonney2k: I'm running LLE21:35
@sonney2kwhat is the difference btw?21:35
blackburnwill tell you in 5 min21:35
blackburnwhen it is compiled without fpmath21:36
blackburnsonney2k: is there any way to inline functions in .cpp?21:36
@sonney2k?21:36
@sonney2kit is done automatically21:36
blackburnit doesn't work when I inline functions21:36
blackburnah21:36
blackburnokay21:36
CIA-3shogun: Sergey Lisitsyn master * ra6426c4 / (6 files): Moved set/get code from .h to .cpp for DR preprocessors - http://git.io/PT7XcA21:37
blackburnsonney2k: can we use another options for swig, not ones we use for libshogun?21:39
@sonney2kcurrently not21:39
blackburnsonney2k: I guess we should - because we don't need to optimize math, etc21:40
blackburninterfaces would compile faster21:40
@sonney2kthat is true - if we don't have functions in .h files basically nothing needs to be that fast - -O2 should be sufficient21:41
blackburnsonney2k: I have moved all the .h code to .cpp for my preprocs21:41
@sonney2kk21:44
@sonney2kI guess even O0 or O1 could be OK21:44
blackburnagree21:44
blackburnsonney2k: do you think my dimreduction 'subtoolbox' have any chances to be published on JMLR mloss?21:45
blackburnI'm currently looking, these guys do more non-trivial things..21:46
@sonney2kif it is the best dimred oss toolbox available - sure21:46
blackburnsonney2k: if not?21:47
@sonney2kthey will complain21:47
blackburnsonney2k: what is the best toolbox?21:48
@sonney2kI don't know21:48
blackburnmy algos are the fastest ones.. that is the only thing I know21:48
blackburnsonney2k: http://jmlr.csail.mit.edu/papers/volume9/klanke08a/klanke08a.pdf it looks pretty 'easy' but accepted22:10
blackburnbut algo is non-trivial I guess22:10
@sonney2kyes non-trivial22:10
@sonney2k1 non-trivial algorithm is enough btw22:11
@sonney2kbtw that was the first submission we got :)22:11
blackburnI have only standard algos22:12
blackburnbut they are really fast :) the only advantage22:12
blackburnsonney2k: sse fpmath is a little faster..22:17
@sonney2kok so then add just this flag to COMP_OPTS22:18
CIA-3shogun: Sergey Lisitsyn master * rf634c62 / src/configure : Removed unnecessary sse flags - http://git.io/79Q8cg22:19
@sonney2kblackburn, btw we have already COMPFLAGS_SWIG22:21
blackburnsonney2k: what should be here? O1 only?22:23
@sonney2kgood question... I think we might need -fPIC on some archs too22:23
@sonney2kmaybe -fPIC -g -O22:24
@sonney2kbut -O should not be there when disable-optimization is enabled22:24
blackburnsonney2k: regarding mloss, can I treat a part of shogun as not-a-shogun? :)22:24
@sonney2kI have no idea how to do that practically22:25
blackburnsonney2k: I see, it needs some customization22:25
blackburnI guess I have zero chances22:25
blackburnsad22:25
@sonney2kI think there was some weka extension recently22:25
@sonney2knot that recently maybe22:25
@sonney2kbut check how they did it22:25
blackburnsonney2k: well there is no mention of NLDR in your mloss paper22:26
@sonney2kNLDR?22:26
@sonney2kahh22:26
blackburnnonlinear dim reduction22:26
blackburnsonney2k: that is the weka paper http://jmlr.csail.mit.edu/papers/volume11/bifet10a/bifet10a.pdf22:26
blackburnsonney2k: the one more reason to separate from preprocessors haha22:27
blackburnsonney2k: where can I place tutorial on dimensionality reduction?22:37
blackburnpython_modular examples?22:37
blackburnor applications?22:37
@sonney2khmmhh or doc/tutorial ?22:38
blackburnsonney2k: not doc, tutorial with python22:38
@sonney2kor new dir tutorial/python ?22:44
blackburnno idea, okay will do first22:45
@sonney2khttp://matt.eifelle.com/2008/11/04/exposing-an-array-interface-with-swig-for-a-cc-structure/22:54
@sonney2khah22:55
@sonney2kthat is how I can do ti22:55
@sonney2kit22:55
blackburnsonney2k: but not buffer interface?22:58
@sonney2kit solves the problem on how to access the pyobject22:59
blackburnI see23:00
-!- blackburn [~blackburn@31.28.44.65] has quit [Quit: Leaving.]23:52
--- Log closed Mon Sep 26 00:00:29 2011
« Saturday, 2011-09-24 Index Monday, 2011-09-26 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!