IRC logs of #shogun for Saturday, 2011-08-20

« Friday, 2011-08-19 Index Sunday, 2011-08-21 »
--- Log opened Sat Aug 20 00:00:06 2011
CIA-87shogun: Soeren Sonnenburg master * r57c1c84 / examples/undocumented/java_modular/check.sh : improve java's check.sh - http://git.io/nKDWSA07:20
CIA-87shogun: Soeren Sonnenburg master * r7d3b613 / examples/undocumented/csharp_modular/check.sh : improve mono tests - http://git.io/1AVYCw07:22
shogun-buildbotbuild #75 of r_modular is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/r_modular/builds/7510:09
-!- blackburn [~blackburn@188.122.238.50] has joined #shogun11:40
CIA-87shogun: Sergey Lisitsyn master * r2ca6531 / .gitignore : Updated gitignore - http://git.io/G53bhQ12:06
CIA-87shogun: Sergey Lisitsyn master * r569d85e / (19 files in 9 dirs): Yet another warnings removal - http://git.io/btXRNA12:46
-!- blackburn [~blackburn@188.122.238.50] has quit [Quit: Leaving.]14:32
shogun-buildbotbuild #76 of r_modular is complete: Failure [failed test_1]  Build details are at http://www.shogun-toolbox.org:5347/builders/r_modular/builds/76  blamelist: blackburn91@gmail.com17:17
shogun-buildbotbuild #75 of python_modular is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/python_modular/builds/7517:30
shogun-buildbotbuild #77 of r_modular is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org:5347/builders/r_modular/builds/7717:30
-!- alesis-novik [~alesis@188.74.87.206] has joined #shogun18:04
alesis-novikHey sonney2k , where's the warning log again (or undocumented stuff log)18:10
-!- blackburn [~blackburn@188.122.238.50] has joined #shogun18:19
@sonney2khttp://shogun-toolbox.org:5347/waterfall18:56
@sonney2kalesis-novik, e.g http://shogun-toolbox.org:5347/builders/python_modular/builds/74/steps/compile/logs/warnings%20%2853%2918:56
-!- blackburn [~blackburn@188.122.238.50] has quit [Quit: Leaving.]18:59
-!- f-x [~user@117.192.222.10] has joined #shogun19:21
-!- alesis-novik [~alesis@188.74.87.206] has quit [Quit: Leaving]19:22
-!- blackburn [~blackburn@188.122.238.50] has joined #shogun19:51
blackburnsonney2k: what's up with buildbots?19:52
@sonney2kblackburn, ?19:52
blackburnit seems interfaces builds are out-of-date19:52
blackburnI fixed a couple of warnings but they are still here19:54
CIA-87shogun: Soeren Sonnenburg master * r226853f / src/configure : fail when java/ruby/lua are enabled but not found - http://git.io/uoLPJw19:58
@sonney2kblackburn, my computer is too unstable :(19:58
blackburnwhy?19:58
@sonney2kI cannot figure out why19:58
@sonney2kspent lots of time w/ trying19:59
@sonney2k(newer linux kernel version, very conservative options, fan running full speed, unplugged usb extensions, no more pci cards, removed / added memory ...)20:00
@sonney2krandom reboots are the evil of it all20:01
@sonney2kIf I don't figure out why these happen I will have to buy a new machine20:02
blackburnsad20:08
@sonney2kin this case it will certainly have enough power to be a buildbot, core i7 / 16G memory - but thats another 400 EUR and more fiddling time :/20:09
blackburnsonney2k: macos failure..21:16
@sonney2k?21:16
blackburnreported on trac21:16
@sonney2kblackburn, he should try git21:16
@sonney2kthat is for shogun 0.1021:16
blackburnah21:16
blackburnsonney2k: should I answer?21:17
@sonney2kplease21:17
blackburnsonney2k: but how you know it is 0.10?21:20
@sonney2kblackburn, believe me I know21:20
blackburnokay21:20
CIA-87shogun: Sergey Lisitsyn master * r37c5df6 / (4 files): Got rid of custom distance usage in Isomap and MDS - http://git.io/-Fxj6Q22:02
CIA-87shogun: Sergey Lisitsyn master * rba59f81 / (2 files): Updated dimension reduction examples - http://git.io/jH9jbA22:02
blackburnsonney2k: is there any define I could use for detecting if pthreads are here?22:22
@sonney2kblackburn, I ordered a new machine now22:43
blackburnuh22:43
blackburnexpensive22:43
@sonney2kso somewhen next week I hope we will have some better buildbot22:44
@sonney2k400 EUR22:44
blackburnI have added HAVE_PTHREAD flag22:44
blackburnwill commit soon22:44
@sonney2k3.4 GHz Quad-Core i7-2600K22:45
@sonney2kwill certainly be fater22:45
@sonney2kfaster22:45
@sonney2kblackburn, err but osx *has* pthreads22:46
@sonney2kblackburn, are you using sth non-standard?22:46
blackburnspinlock22:46
blackburnmay be it is non standart22:46
@sonney2kwhy spinlock?22:46
blackburnI don't know22:46
blackburnfaster22:46
@sonney2kisn't mutex sufficient?22:46
@sonney2kblackburn, but they do busy loops or?22:46
blackburnI don't know really22:47
blackburnbut it is faster22:47
@sonney2kblackburn, how much faster?22:47
blackburnhmm I have tested on 5000 vectors22:47
blackburnit was ~4.1s for serial22:47
blackburn~4.8s for mutex22:47
blackburnand 3.8 for spinlock22:47
@sonney2kso all the same22:48
@sonney2khave you tested on 20k vectors or more?22:48
@sonney2kI mean one second more or less doesn't matter22:48
blackburnnope22:48
blackburndo you suggest to use mutex anyway?22:48
@sonney2kblackburn, just find out if spinlocks are nonstandard22:49
@sonney2kif they are then better use mutexes22:49
@sonney2kbut you should test if it is faster then22:49
@sonney2kI mean if the overhead is say 1-2 seconds for using mutexes (independent of # vectors that you use) it doesn't matter22:49
@sonney2kbut if this overhead is badly scaling with # vectors than it is no good22:50
blackburnsonney2k: okay but HAVE_PTHREAD is a good idea anyway, right?22:51
@sonney2kblackburn, one question - couldn't you make the use of spinlocks optional?22:51
@sonney2kblackburn, that yes22:51
blackburnyeah I could but really I should check22:51
@sonney2kblackburn, I mean we could just have a macro - then one could define USE_MUTEX and mutexes will be used / otherwise spinlock22:52
blackburnpthread have _POSIX_SPIN_LOCKS definition22:52
@sonney2kjudging from http://www.alexonlinux.com/pthread-mutex-vs-pthread-spinlock  using mutexes/spinlocks is basically the same22:52
@sonney2kblackburn, google says osx doesn't know spinlocks22:54
blackburnoh I see22:54
@sonney2kso how about introducing the macro PTHREAD_LOCK22:54
@sonney2kand PTHREAD_UNLOCK22:54
blackburnwait wait22:54
blackburnI need to test overhead22:55
@sonney2kthat will then be defined as either pthread_spin_lock22:55
@sonney2kor pthread_mutex_lock22:55
@sonney2kblackburn, what for?22:55
blackburninit too22:55
blackburnI'm not sure if I really should use spinlock here22:55
@sonney2kyes PTHREAD_LOCK_T pthread_spinlock_t22:55
@sonney2kblackburn, yeah but it makes sense to use mutexes per default22:56
@sonney2kand then have a configure option to use spinlocks anyways22:56
blackburnwhere should it be defined?22:56
@sonney2kblackburn, btw the good news about the new hardware is that with 16G I can actually run 4 buildbots in parallel22:56
blackburnnice22:59
blackburnsonney2k: I don't think we need to do it globally22:59
blackburnfor example SGObject's mutex should stay mutex22:59
@sonney2kblackburn, I think it shouldn't23:00
@sonney2kI mean it is only for incrementing the reference count23:01
@sonney2kvery unlikely that two cpus even SG_REF at the same time23:01
blackburnThe best time to use a spinlock is when the protected critical section of code is performing a simple CPU bound task that should be completed as quickly as possible. An example of a simple CPU bound task would be something like incrementing counters or perhaps reading or modifying a memory mapped IO port.A spinlock should NOT be used if the critical section has code in it that is IO bound. An example of this would be waiting for keyboard input from a u23:01
@sonney2kso a spinlock would be faster23:01
@sonney2kso a spinlock would make more sense23:01
@sonney2kref++23:01
blackburnhmm okay23:02
@sonney2kblackburn, do we have mutexes at other places?23:02
blackburnI don't know23:02
blackburnwhere should it be introduced?23:02
@sonney2kblackburn, how about Parallel.h ?23:03
blackburnhmm yeah seems so23:03
blackburnokay will try now23:03
blackburnsonney2k: could you remind me how to do parameter in macro?23:06
blackburnah ()23:07
blackburnokay nevermind23:07
blackburnLOCK_PTHREAD_LOCK sounds just like die baby die :D23:12
blackburnor something crazy like that23:12
blackburnuh done23:35
@sonney2kI much prefer burn motherfucker burn.23:39
blackburnboom23:41
CIA-87shogun: Sergey Lisitsyn master * ra7f397a / examples/undocumented/python_modular/graphical/swissroll.py : Fixed swissroll example (+5 more commits...) - http://git.io/WnqnXw23:41
* sonney2k is preparing for the night...23:41
* sonney2k ... by killing mosquitos23:41
blackburnno mosquitos here already23:42
blackburntoo cold :)23:42
@sonney2kblackburn, I currently see some major installation issues23:45
@sonney2kmostly for the new interfaces but also one other issue23:46
@sonney2kone should be able to run all examples once examples are installed23:46
@sonney2kthis means data has to be shipped somehow too (but where?)23:46
blackburntoy data could be included to shogun23:47
@sonney2kand also that data has to be in some PATH maybe defined in environment variable23:47
blackburnagree23:47
@sonney2kalso examples should not include ../../../.../something23:48
@sonney2kbut expect these files under certain directories23:48
@sonney2kand another problem: we have modshogun.so as the interface lib for python, java, ...23:48
@sonney2kwe have to see if we will run into problems because of the naming clash23:49
blackburnuh I feel upsat because of no burst with parallel algos :(23:50
blackburnis there a way to name it modshogun_java?23:50
blackburnor any other modshogun_shit23:50
@sonney2kblackburn, that would mean though that one has to write import modshogun_python23:52
@sonney2kor whatever23:52
blackburnoh nono23:52
@sonney2kblackburn, parallel algos need lots of data usually to be really usefull23:53
@sonney2kblackburn, for the algos I had parallel algos made a differnce for >1e6 examples23:53
@sonney2kbut before that hmmhh well not too much23:53
blackburnnor MDS, or isomap, or .. can handle 1e6 data hah23:54
@sonney2kcould be that you overoptimize then23:55
blackburnnot really23:55
blackburnthe problem is the bottleneck = eigenproblem is not parallel23:55
blackburnother steps burst from parallel a little23:56
blackburnbut they are compartively fast23:56
--- Log closed Sun Aug 21 00:00:07 2011
« Friday, 2011-08-19 Index Sunday, 2011-08-21 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!