IRC logs of #shogun for Wednesday, 2017-06-21

--- Log opened Wed Jun 21 00:00:22 2017
-!- TingMiao [uid229534@gateway/web/irccloud.com/x-ufgcvkdtqvyqbein] has quit [Quit: Connection closed for inactivity]00:16
-!- CaBa [~Diu7saig@unaffiliated/caba] has joined #shogun00:17
-!- CaBa_ [~Diu7saig@lounge.imp.fu-berlin.de] has quit [Remote host closed the connection]00:17
-!- olinguyen [18303760@gateway/web/freenode/ip.24.48.55.96] has joined #shogun01:33
@wikingmikeling, hi05:00
mikelingwiking: Hi~05:01
mikelingDoes the benchmark looks good to you?05:01
@wikingmmm05:02
@wikingi'm just checking05:02
@wikingbut yeah that's hat i had in mind05:02
mikelingI add more random engines into the benchmark, will paste the result soon05:03
@wikingmikeling, ok05:27
@wikingmikeling, ok so i'm just wondering05:29
@wikingbased on this benchark05:29
@wikingthe existing random05:29
@wikingin shogun05:29
@wikingis actually much better right?05:29
mikelingyep05:29
mikelingactually, it's better than other engines05:29
@wikingany other engines? :)05:30
@wikingfuck05:30
mikelingyes05:30
@wikingi was hoping05:30
@wikingthat the difference is not so big :(05:30
@wikingok one more thing05:30
@wikingmmm just forgot what i wanted05:31
@wiking;(05:31
@wikingah yeah05:31
@wikingwhat is your compiler?05:31
@wikinggcc?05:31
@wikingif yes can you try the same thing with clang?05:31
@wikingonly compile the benchmark code with clang05:32
@wikingas it'll use a different libstdc++05:32
@wikingolinguyen, around?05:32
mikelingmmm, my local system is OSX and I'm using CLion to run the benchmark. So I guess my default compiler will be clang if I haven't do any specific configuration05:34
@wikingoooooh05:36
@wikingok05:36
@wikinglemme try this benchmark05:36
@wikingon a linux machine05:36
@wikingmikeling, what was the -O flag you used05:42
@wikingfor compiling benchmark.cc05:42
mikelingno, I haven't use any flag. Just press the Run button of Clion and it will output result to its debug console05:43
@wikinghow did you build shogun?05:44
mikelingyes, I already make install it. I use link_libraries(shogun) in my CMakeLists and use it then05:45
mikelingoooh05:45
@wikingyeah but how did you build it05:45
mikelingsorry05:45
@wikingdid you specify CMAKE_BUILD_TYPE05:46
mikelingcmake -DCMAKE_BUILD_TYPE=Debug -DENABLE_TESTING=ON -DBUILD_META_EXAMPLES=ON ../05:46
@wikingfor shogun's cmake?05:46
@wikingooooh05:46
@wikingok even with Debug build05:46
@wikingmmm05:46
@wikingcould you please redo on your end the benchmakring05:46
@wikingbut use a shogun that is05:46
@wikingcmake -DCMAKE_BUILD_TYPE=Release05:46
mikelingok05:46
@wikingi'm suspecting that the random might be even faster :(05:47
@wikinghoho05:47
mikelingBTW, do you want to look at the result I have now? with the Debug version05:47
@wikingyou mean you have newer ones?05:47
@wikingbecause i've check the gist05:48
@wikingyou've sent me yesterday05:48
mikelingI just add the other engines05:48
mikelingI haven't update the gist, because most of them is copy and paste05:48
mikelingI just test them locally05:48
mikelingand get the result05:49
mikelingwiking: https://gist.github.com/MikeLing/d258cf0c7e18d17639c577a695dbec41#gistcomment-212853405:49
mikelinglike this05:49
@wikingooohooo05:52
@wikingok05:52
@wikingso in case05:52
@wikingof minstd_rand05:52
@wikingit's comparable05:52
@wikingwith what we have atm05:52
mikelingwiking: mmm, yes. And it has same speed on normal_dis benchmark with shogun::random05:53
mikeling* CRandom::normal_distrib05:54
@wikingsame/05:54
@wiking?05:54
@wikingbased on the benchmark comment05:54
@wikingBM_sg_random_normal_dis  is a bit faster05:55
@wiking74 vs 12305:55
mikelingwiking: that's a mis tag. I just update it. the 74 vs 123 is the situation of int3205:56
mikelingnot normal_distrib05:56
mikelingI haven't paste that part of result for now05:56
mikelinglet me do it right now05:56
@wikingkk05:56
@wikingcool05:56
mikelingAnd then I will try the release versio05:57
@wikingcool05:57
mikelingversion05:57
@wikingthnx05:57
mikelingnp05:57
@wikingbut ok05:57
@wikingthis is a good sign05:57
@wikingwhat is weird tho05:58
@wikingif i compare the screenshot05:58
@wikinghttps://gist.github.com/MikeLing/d258cf0c7e18d17639c577a695dbec41#gistcomment-212816705:58
@wikinghere05:58
@wikingBM_random_float64 = BM_mt19937_64_float6405:58
@wikingright?05:58
mikelingyep05:59
@wikingfor some reason05:59
@wikingthere's quite a lot of diff05:59
@wiking71 vs 4205:59
mikelingyeah, I had notice that also. But I found the first benchmark been test will spend other time than others. Maybe it's because I run it on the IDE rather output it to an .out. If you look at the CPU time is 41 ns vs 51ns06:02
mikelingAnd I also try to change the sequence of the benchmark. Even I out shogun::random to the first, it will be the fastest one06:03
mikelingalmost06:03
mikeling* put shogun::random06:04
olinguyenwiking: hey!06:07
@wikingolinguyen, hihi06:08
@wikingolinguyen, this is just a gentle reminder to keep your log (daily) updated. thnx!06:08
olinguyensure, will do!06:08
@wikingthnx06:09
@wikingmikeling, OOOOOOO06:11
@wikinglook at this :)06:13
@wikingsent in private06:13
@wikingmmm06:20
@wikingso i'm now testing with Release :)06:20
-!- travis-ci [~travis-ci@ec2-54-196-109-91.compute-1.amazonaws.com] has joined #shogun06:22
travis-ciit's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/24520581906:22
-!- travis-ci [~travis-ci@ec2-54-196-109-91.compute-1.amazonaws.com] has left #shogun []06:22
-!- travis-ci [~travis-ci@ec2-54-196-109-91.compute-1.amazonaws.com] has joined #shogun07:43
travis-ciit's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/24520816707:43
-!- travis-ci [~travis-ci@ec2-54-196-109-91.compute-1.amazonaws.com] has left #shogun []07:43
-!- olinguyen [18303760@gateway/web/freenode/ip.24.48.55.96] has quit [Quit: Page closed]08:05
-!- iglesiasg [~iglesiasg@217.119.234.214] has joined #shogun09:13
-!- mode/#shogun [+o iglesiasg] by ChanServ09:13
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has joined #shogun09:31
micmnwiking: ping09:46
-!- TingMiao [uid229534@gateway/web/irccloud.com/x-lgghrupggtdqrypi] has joined #shogun10:08
@wikingmicmn, pong11:02
@wikingsorry11:02
@wikingwas away11:02
micmnnp, some weird things happening with benchmarking scripts + eigen I think11:03
micmnI'll paste my journal update11:04
micmnBenchmark script for (MC)LDA (just a trivial conversion from QDA’s script), but got stuck since Eigen hangs on a matrix product (has to do with multithreading, endless loop on this line: https://bitbucket.org/eigen/eigen/annotate/0cb43994733f413565eb6dbe1495d5ec2694436f/Eigen/src/Core/products/GeneralMatrixMatrix.h?at=default&fileviewer=file-view-default#GeneralMatrixMatrix.h-127). This happens with: python interface11:04
micmn+ second execution of MCLDA.train() (on two different instances).11:04
@wikinglemme check11:14
@wikingwhat's your benchmark script?11:14
micmnhttps://gist.github.com/micmn/7d471341bf2ffd787930035bfcd1dd0c11:16
@wikingmicmn, is this gonna be part of mlpack's benchmarking fw?11:17
micmnwhat do you mean?11:18
@wikingmicmn, whatever benchmarking you do11:18
@wikingit doesn't make sense to make some custom solution11:18
@wikingi'm fine if you wanna do something quick11:18
@wikingbut in the long run11:18
micmnoh yes11:18
@wikinganything related to benchmarking11:18
@wikingwould be great11:18
@wikingthat it becomes part of https://github.com/mlpack/benchmarks11:19
micmnit's a mlpack script11:19
@wikingcool11:19
@wikingok11:19
@wikingso there's gonna be a PR for this in mlpack's benchmarking11:19
micmnyep11:19
@wikingk11:20
@wikingmmm11:20
@wikingso we have a race condition?11:20
micmnnot sure what's going on since we don't have multiple MCLDA instances in different threads11:21
micmnbut when the scripts calls MCLDA.train() for the second time eigen hangs on a matrix product11:22
@wikingmmm11:22
@wikinghave you tried reaching out to eigen developers?11:22
@wikingbut i reckon until this is not fixed in eigen11:22
@wikingwe need to guard this from shogun ;P11:23
@wikingwe already have some such things in shogun11:23
@wikingwhere based on eigen's version there are some weird workarounds within shogun11:23
micmnI'll try to make a minimal example and reach to the developers11:24
@wikingthey are quite responsive11:24
@wikingeven on #eigen11:24
@wikingthey are all CET timezone11:24
micmngood to know11:25
@wikingbut yeah here's an example11:25
@wikingEIGEN_WITH_OPERATOR_BUG11:25
@wikingor the other macro is11:25
@wikingEIGEN_WITH_TRANSPOSITION_BUG11:25
@wikingboth defined in src/shogun/mathematics/eigen3.h11:25
@wikingso there are workarounds for this already11:26
@wikingif this is really a bug in eigen11:26
@wikingthen we need to make sure that it's gonna get fixed in eigen11:27
@wikingbut till then have a workaround in shogun for i11:27
@wiking*it11:27
geektoniping wiking11:27
@wikingpong11:28
@wikingyes11:28
@wikingyou need merging11:28
@wiking:)11:28
@wikinglemme go through the prs11:28
geektoniyeah exactly ;)11:28
@wikingok i can merge w/o problem the docker thingy11:28
@wikingboom11:28
@wikingheheh geektoni on the end of this story11:29
@wikingyou will own all the lines11:29
@wikingin shogun11:29
@wiking:D11:29
geektoniif there will be an end ;)11:29
geektoniwiking: mmh, Github updates are not sent here anymore? Were they too verbose? ;)11:31
@wikingmmm11:32
@wikinglemme dunno wtf happened11:32
@wikinglemme check11:32
@wikingi've upgraded sukey's system on the weekend11:32
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]11:33
geektoniahah, I though it was done on purpose ;)11:33
@wikingnoup11:34
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun11:37
-!- mode/#shogun [+o sukey] by ChanServ11:38
@wikingmmm11:39
@wikingsukey, flip11:39
@sukey‎(ノಥ益ಥ)ノ ┻━┻11:39
@wikingsukey, flip11:39
@sukey┬─┬ ノ( ゜-゜ノ)11:39
@wikingfuck11:39
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]11:40
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun11:41
-!- mode/#shogun [+o sukey] by ChanServ11:41
@wikingsukey, flip11:41
@sukey(/¯◡ ‿ ◡)/¯ ~ ┻━┻11:41
@wikingboom11:41
@wikingno boom? :)11:41
geektonidoes it work?11:42
@wikinggeektoni, now it should11:42
geektonigreat :D11:42
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3855 merged by vigsterkr11:43
@wikingok11:43
@wiking:)11:43
@wikingas you can see it works11:43
geektoni\o/11:44
@wikingmmm11:46
@wikingm_sigint_observable.connect();11:46
@wikingso you are thinking that in case of11:46
@wikingany signal you connect to the observable11:47
@wikingand that triggers the stopping?11:47
@wikingor i mean i'm not so sure how do you use here the subscriber/observables11:47
geektonibasically11:47
geektonithere are two observable, one for SIGINT and one for SIGURG signal11:47
geektonian algorithm subscribe to each of them11:48
geektoniand when, for instance, CTRL+C is pressed, the handler will call the connect() method on one of these observable to notify all algorithm about what to do.11:49
@wikingyeah11:49
@wikingi'm just wondering11:49
@wikingthat in case you use csignal11:49
@wikingand get any of the signals... shouldn't it just do an onNext(signalType) on an observable11:49
@wikingand the observer/subscriber would just receive it onNext11:50
@wikingand that's all11:50
@wikingso say11:50
@wikingmachine subscribes to both of the observables11:50
@wikingor even it can be one11:51
@wikingas you wish11:51
@wikingand then when the event happens11:51
@wikingit'll get the onNext(signalType)11:51
@wikingsee what i mean?11:52
geektonimmm11:53
geektoniI'm trying to figure out the on_next method11:53
@wikingwell i mean11:53
@wikingyou know in case of observables11:53
@wikingyou have11:53
@wikingonNext/onError/onComplete11:53
@wikingyou can name those different11:53
@wikingbut that's the logic11:53
geektoniyeah, but they're methods which are called when the observable emits an item11:54
@wikingyes11:54
geektoniok, so, if I understood correctly, we need a subject object instead of a pure observable11:55
geektonihttps://github.com/Reactive-Extensions/RxCpp/issues/26511:55
geektonithat if we want to implement what you said11:56
@wikingmmm11:58
@wikingwell11:58
@wikingyeah either that11:58
@wikingthe other option i had in mind11:58
@wikingis rather stupid11:58
@wiking;)11:58
geektoniwell, what is? ;)11:59
@wikingnothin12:00
@wikingas it is a wasteful way to do this12:00
geektonikk12:01
geektoniI think it's worth to switch to "subject" instead of pure observable.12:02
geektoniit'll make the code more straightforward.12:02
@wikingi mean i'm now thinking12:03
@wikingthat isn't it maybe an overkill12:04
@wikingto use for this rx?12:04
geektoniwell12:04
geektoniit's a bit an overkill12:04
geektonibecause12:04
geektonirxcpp is usually used for other purposes12:05
geektonibut, on the other hand12:05
geektoniit may come handy in the future12:05
geektoni*become12:05
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has quit [Quit: Page closed]12:06
@wikinglisitsyn, ping12:15
micmnwiking: I have isolated the problem, check https://gist.github.com/micmn/0301a7fe8833842c5ad94a16c62ba33f12:17
@wikinghohohoooooooooo12:17
@wikingfucking python multiprocess12:17
@wiking:)12:17
@wikinghave you tried the smae12:17
@wikingwith c++?12:17
micmnno, I guess it's the next step :p12:18
@wikingplz do12:18
@wikingas python's multiprocess is loco12:18
-!- geektoni [~Mutter@62.18.94.171] has joined #shogun12:19
micmnit's what mlpack's benchmarks use :(12:19
geektoniwiking: sorry, I will be mobile for a while12:19
@wikinggeektoni, that's fine12:19
@wikingmicmn, yeah in know :) plz just try the same thing12:20
@wikingwith c++12:20
@wikingand if it is still failing12:20
@wikingthen we can be sure12:20
@wikingthat it's not a python GIL story12:20
@wikingif it fails12:20
@wikingthen we need a workaround for the python's GIL12:20
@wiking:)12:20
@wikinggeektoni, so in case of connect observable12:22
@wikingyou just connect to the machine right?12:22
@wikingin case there's a ping12:22
@wikingi mean signal12:22
@wikingand that triggers a change in the boolean?12:22
geektoniThe machine connects to the observable12:23
geektoniAnd yes, when the signal arrives there is a change in the Boolean12:23
@wikingmmm12:24
@wikingyou dont have this code12:24
@wikingright in the pr12:24
geektoniNope12:24
geektoniBecause it will be another or12:24
geektoni*Pr12:24
geektoniBut12:24
geektoniI've already written the CMachine code12:25
@wikingdo you have it somewhere? :)12:25
geektoniLet me link you the branch12:25
@wikingok12:25
geektonihttps://github.com/geektoni/shogun/commit/6fd340d48695531fd1be3844657e6597e60784d412:26
geektoniThe idea is that the machine has to be linked manually to the signal handler. That's permit to do much complex things without worrying about corner cases12:27
@wikingm_cancel_computation12:28
@wikingthis is actually not the best naming12:28
@wikingfor a variable12:28
@wikingthat says that it should do premature stopping12:29
@wikingbecause you dont actually cancel the computation12:29
@wikingjust prematurely stopping it12:29
@wikingbut on the other hand12:29
@wikinglets talk abou tth ecase12:29
@wikingof this12:29
@wiking:)12:29
@wikingso maybe we can figure out why we want to have still ex12:29
@wiking*rx12:29
@wikinglet's say i wanna have an option12:29
@wikingto pause computation12:29
@wikingof a model12:30
@wikingbecause i wanna do some other stuff :)12:30
@wikinghow would we do this12:30
@wiking(and we add a new signal or whatever fuck to this to represent it)12:30
geektoniPause and the restart or just pause? :)12:31
geektoni*then12:31
@wikingwell have a flag that is12:31
@wikingpause12:31
@wikingif it's true then you flip a mutex12:32
@wikingor somethign12:32
@wikingso that you dont burn cpu time with while(1) {}12:32
geektoniWell12:32
geektoniThe code do this actually12:32
geektoniLet me link you something12:33
@wikinggeektoni, and the other thing is m_cancel_computation should be atomic_flag12:33
@wikingor at least std::atomic<bool>12:33
geektonihttps://gist.github.com/geektoni/1afd2daf69a813de64252a296997a8ea12:34
geektoniDou you mean something like this? ^12:34
@wikingloop = train_machine12:34
@wikingright?12:34
geektoniYeah12:35
@wikingcancel_computation => will just pop out of the train_machine loop12:35
@wikingbut what i wanted is12:35
@wikingjust to be able to say12:35
@wikingstop calculation12:35
@wikingi mean12:35
@wikingpause12:35
@wikingand be able to resume12:35
@wikingof course this is hard12:35
@wikingbecause you cannot have a flag for this12:35
geektoniMmmh12:36
geektoniPause and resume will be hard12:36
@wikingyes12:36
@wikingi mean we can still do it12:36
@wikinghave the framework for it12:36
@wikingand tell to the algo develoepr12:36
@wikingthat he has to do the following things12:36
@wikingin order to be able to support it12:36
@wikingjust as in case of cancel computation12:36
@wikingthe only difference is that it's a 'bit' harder12:36
@wikingor not that straightforward12:37
@wikingi would say12:37
@wikingto add something like12:37
geektoniCould you make an example of such a situation? :)12:38
geektoniJust to be sure to have understood :P12:38
@wikingif (m_pause_computation.load()) { while(m_pause_computation.load()); }12:39
@wikingthis is the very stupid version of it12:39
@wikingand this you put as the first line12:39
@wikingafter12:39
@wikingwhile(true && !cancel_computation())12:39
@wikingin your example12:39
@wikingso basically until m_pause_computation true you dont do anything in the algo12:40
@wikingi.e. hang the computation12:40
@wikingthis way you can observe the states of the algo12:40
@wikingor even update some variables12:40
@wikingand then do12:40
@wikingm_pause_computation = false12:40
geektoniAhhn okay, got it12:40
@wikingthen the algo would resume12:40
@wikingsee what i mean?12:41
geektonibut12:41
geektoniwe need to tell the developer to code like this to implement this behavior12:41
@wikingyes12:41
@wikingbut the same is true12:41
@wikingfor12:41
@wikingcancel_computation()12:41
@wikingno?12:41
geektoniyes yeas, it needs to be called to cancel the computation.12:42
@wikingbtw the very reason i wanted to introduce rxcpp in shogun is that CMachine would be an observable12:42
@wikingand so all the parameters of a CMachine12:42
@wikingcould be observed12:42
@wikingand they would be emitted12:42
@wikingso an observer12:42
@wikinggeektoni, so somehow you can trace the evolution of the parameters12:42
@wikingof a machine12:42
geektoniI see12:43
micmnwiking: same with fork :( https://gist.github.com/micmn/f1044ef5d828b6bd84ef35c6cb3dab7512:44
@wikinggeektoni, yes it's true that the cancel computation implementation is much easier to add than the pause-resume12:44
@wikingbut still it's not an automatic feature of a new algo12:44
@wikingan algo implementer has to take care of it12:44
@wikingbut of course one is easier than the other12:45
geektoniYeah sure. I'll think how to add that.12:45
@wikingok so but wait12:45
@wikingso the reason i came up with this stupid idea12:45
@wikingis to see12:45
@wikingso adding a new functionality/sginal like this12:46
@wikingwould mean that we need to add a new observable12:46
@wikingright?12:46
@wikingand i'm just thiking12:46
@wikingthat if we keep extending12:46
@wikingallt hese little shit12:46
@wikingwould that actually make a case for rxcpp12:47
@wikingand one more thing owuld be good to see12:47
@wikinghow this would get affected12:47
@wikingby trying to make the params of a CMachine observable12:47
@wiking:)12:47
@wikingmicmn, ok12:47
-!- geektoni [~Mutter@62.18.94.171] has quit [Quit: Mutter: www.mutterirc.com]12:47
@wikingso it's eigne shit12:47
@wikinglemme actually compile and run this code12:47
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has joined #shogun12:48
@wikinggeektoni, mutter :)12:48
geektoniNo more mobile ;)12:48
geektoniso12:48
geektoniI think we won't need to add a new observable12:49
geektonibecause12:49
geektonidepending on what the user select12:49
geektoniwe could add different action inside CSignal::handler12:49
@wikingmicmn, ever hanging right?12:49
micmnyeah12:49
@wikinglemme see12:50
@wikingmmm but waitpid12:50
@wikingi dont get wtf12:50
@wikingdo you see where is it hanging12:50
micmnchild's train()12:50
@wikingwhhyyyyyyyyyyy12:51
geektoniwiking: I'll be afk for a few minutes12:51
geektoniI need to eat LOL12:52
@wikinggeektoni, but i mean observable in CSignal12:52
micmnit hangs on a matrix multiplication12:53
micmnhttps://gist.github.com/micmn/6bc2304fb5ee7b6b628ff242449474b012:53
@wikinglol12:53
@wikingmmm12:55
@wikingwoaaaaah12:55
@wikingfunny12:55
@wikingif you compile shogun with Debug mode12:55
@wikingsuper slow everything12:55
@wiking:)12:55
@wikingi mean yeah expected12:55
@wikingbut not this much12:55
@wikinggeektoni, ping me when u r back12:57
@wikingmicmn, but only when you fork?12:58
@wikingi.e. if you do 2 train sequentially?12:58
@wikingw/o fork?12:58
micmnthat works12:58
micmnthe problem is 1) train 2) fork 3) train on child12:58
micmnbecause mlpack does that12:59
@wikingk13:00
@wikingweird13:00
@wikingi dont understand :O13:00
@wikinglet's go to #eigen13:00
micmnyeah :|13:00
@wikingmicmn, seems ChriSopht is not around :(13:30
@wikingmicmn, mmm the best would be of course13:30
@wikingto do this with eigen code :)13:30
@wikingand the we could fire a bug13:30
@wikingmicmn, what's your eigen version?13:30
@wikingii  libeigen3-dev                          3.2.2-3                              all          lightweight C++ template library for linear algebra13:30
@wikingthis is mine where i've observed the hanging13:31
micmnthe one that is bundled with shogun13:31
@wikingoh ok13:31
@wikingcould you just for the sanity check13:31
@wikingchange the eigen version in the bundling13:31
@wikingto the latest stable13:31
micmnbut I tried with newer versions13:31
@wiking3.3.413:31
@wikingoh ok13:31
@wikingso any 3.3.x has the same13:31
@wikingthing?13:31
micmnit doesn't hangs on the same line but always on gemm13:32
@wikingyep13:32
@wikingbut it's a 3.3.x that you've tried13:32
@wikingor still a 3.2.x?13:32
micmnyep13:32
micmn3.313:32
@wikingok cool13:33
@wikingso i think it's still an existing problem :)13:33
micmn3.3.413:33
@wikingbut i wonder wtf13:33
@wikingi mean there's 0 multithreading going on here13:33
@wikingyou just simply fork13:33
@wikinganother process13:33
micmnyeah doesn't make sense13:33
@wikingthis is super weird13:33
@wikingas i dont even see what could hold anything13:34
@wikingthat interferes with anything13:34
micmnshared libs?13:34
@wikingmmmm yeah but13:34
micmnhow can I disable openmp?13:38
@wikingmicmn, try OPENMP_NUM_THREADS=113:39
@wikingis eigen using openmp?13:39
micmnyes13:39
@wikingLOL13:40
@wikingwiking@hedonismbot:~/shogun/build$ OMP_NUM_THREADS=1 LD_LIBRARY_PATH=./src/shogun/ ./a.out13:40
@wiking3145: Training...13:40
@wiking3145: done.13:40
@wiking3145: Spawning process...13:40
@wiking3145: Joining ...13:40
@wiking3145: Training...13:40
@wiking3145: done.13:40
@wiking3145: done.13:40
@wiking:)13:40
@wikingmicmn, ^13:41
micmnmmm why is it the same process13:41
@wikingah ok oh13:41
@wikingok13:41
@wikingsorry13:41
@wikingthat's my bad13:41
@wiking:D13:41
micmnXD13:41
@wikinggood catch13:41
@wikingwas testing13:41
@wikingwhat happens w/o fork13:42
@wikingok13:42
@wikingstill hangs for me13:42
micmnyeah me too13:43
@wikingstill13:44
@wikingthis doesn't make sense13:44
@wikingwhat would cause the hang ?:)13:44
@wikingLOLFACE13:44
@wikingwiking@hedonismbot:~/shogun/build$ OMP_THREAD_LIMIT=1 OMP_NUM_THREADS=1 LD_LIBRARY_PATH=./src/shogun/ ./a.out13:44
@wiking3293: Training...13:44
@wiking3293: done.13:44
@wiking3293: Spawning process...13:44
@wiking3293: Joining ...13:44
@wiking3294: Training...13:44
@wiking3294: done.13:44
@wiking3293: done.13:44
@wikingmicmn, ^13:44
micmnwow13:44
micmnworks indeed13:45
@wikingthe problem is13:52
@wikingthat even if we find out the root cause13:52
@wikingi dont see how one could circumvent this13:53
@wikingfrom shogun13:53
@iglesiasgTingMiao: nice job with the blog post, let me know if any of the comments does not make sense :D13:59
TingMiaoThanks!14:00
@wikingwhere's the post14:09
@wikingvafanculo14:09
geektoniwiking: I'm back14:12
@wikinggeektoni, pong14:12
@wikingwhat did u eat?14:13
geektonieasy answer to this one ;)14:13
@wiking?14:13
geektoniI'm italian14:14
geektoniso14:14
@wikingyeah but still14:14
@wikingwhich pasta?14:14
@wikingwhat type?14:14
@wikingwith what sauce?14:14
geektoniahahaha14:14
@wikingcomeone14:14
@wiking*on14:14
geektoniyeah, with meat sauce14:14
geektonistandard one14:14
@wikinggood14:15
@wikingwith what?14:15
@wikingpenne?14:15
geektoniyeah, penne ;)14:15
@wikinggood14:15
@wikingok so14:15
geektoniWe were talking about pausing algorithms14:17
@wikingyeah14:18
@wikingas well as monitoring14:18
@wikingparameters14:19
@wikingthose are two different things14:19
@wikingbut in case of rx design14:19
@wikingthey might be relevant14:19
@wikinggeektoni, i mean one way or another14:19
@wikingwe gonna have a parameter map right?14:20
@wikingoldschool way we have m_parameters14:20
@wikingtags is another story14:20
@wikingbut in any ways14:20
@wikingone thing that could be great is14:20
lisitsynoh wuck14:22
lisitsynmessages everywhere14:22
geektoniwiking: ok, so, I would say I'll focus on finishing premature stopping/pausing first14:26
geektonithen I'll move on observable parameters if it'll be the case14:26
@wikinggeektoni, yeye14:30
@wikingthat i get14:30
@wikingbut14:30
@wikingi'm now a bit scared14:31
@wikingto just say14:31
@wikingeyeyey fck it14:31
@wikinggo rx14:31
@wikingwhatever14:31
@wiking:)14:31
geektoniyes, I understood :P I mean, since it is now an hard requirement, it has to be useful.14:31
geektonibuut, regarding observable parameters, as you said, shouldn't we wait until we have Tags in place?14:33
geektonibecause14:33
geektonithe current Parameter14:33
geektoniwill be thrown away soon (I imagine)14:33
@wikingyeye14:37
@wikingnono14:37
@wikingit's fine14:37
@wikingwe are just discussing here concepts14:37
@wikingno matter what actual implementation you use14:37
@wikingwhat you will have is14:37
@wikinga map14:37
@wikingwhere the params are set and stored14:37
@wikingright?14:37
geektoniyeah, sure14:38
@wikingso what i mean14:38
@wikingthat in this case14:38
@wikingsay we do a simple trick that14:38
@wikingyou can subscribe to CMachine14:38
@wikingparameters14:38
@wikingand once eombody sets14:38
@wiking'somebody' = something14:38
@wikingset's a parameter in the param map14:38
@wikingyou do an onNext14:38
@wikingor?14:38
@wikingi mean i'm just tyring to figure out what would be the nicest way14:39
@wikingand to include rx here14:39
geektonimmh14:39
geektoniideally, we could make the map an observable, and every time somebody updates a value then we output the Tag/Parameter which was modified. Of course, the map observable will have to call the on_next()/on_completed() methods of the subscribers.14:42
micmnwiking: second post https://software.intel.com/en-us/forums/intel-c-compiler/topic/54010014:43
@wikingWhen using fork, you must do so prior to issuing any OpenMP directives or omp_... library calls.14:47
micmnhttps://gcc.gnu.org/bugzilla/show_bug.cgi?id=5837814:47
@wikingmicmn, eeh14:53
@wikingmicmn, imo then this is a feature :)15:07
micmnyeah :D15:07
micmnjust wondering why it didn't come up before15:12
micmnmclda it's not the only algorithm that both uses eigen and it's tested in that way15:12
@wiking:)15:19
@wikinggeektoni, mmm but15:25
@wikingwould then actually15:25
@wikingCMachine be an observable?15:25
@wikingObservable<Any>15:25
@wiking:)15:25
lisitsynwhoa15:25
lisitsynobservable!15:25
@wikinglisitsyn, bearhunting15:25
lisitsynthings unfold15:25
geektoniwiking: what do you mean?15:27
@wikingi mean15:27
@wikingwhat i want is that i can subscribe to a machine15:28
@wikingand that would emit15:28
@wikingparameters15:28
@wiking:)15:28
lisitsynparameters?15:28
@wikingyes15:28
@wikingparameters15:28
lisitsynchanges of parameters?15:28
@wikingall of them15:28
@wikingyes15:28
lisitsynsounds good to me15:28
lisitsyn:P15:28
@wikinglisitsyn, :D15:28
@wikinggeektoni, so we would have15:28
@wikingChmachine::get_parameters_observable() ?15:28
@wikingor what?15:29
geektoniyeah, that could be an option15:29
geektonibut15:29
@wikingwhere CMachine would have a subject15:29
@wikingthat would be pushed from parameter changes15:29
@wiking?15:29
@wikingor what?15:29
@wiking:)15:29
@wikingno BUT15:29
@wiking:D15:29
geektoni CMachine is not the observable15:29
@wikingonly solutions15:29
@wikinggeektoni, i know15:29
geektonikk15:29
@wikingi'm just trying to come up with something15:29
@wikingwe can make it an observable15:29
@wikingbut we can just add a property15:30
@wikingwhich is a subject15:30
@wikingonly question is15:30
@wikingthat how one could make this automatic15:30
@wikingthat CMachine's params15:30
@wikingcoudl be emitted15:30
@wikinganyhow15:30
@wikingi'm just high level talking about this15:30
@wikingbecause currently15:30
@wikingrxcpp is a bit of an overkill15:31
@wikingimo15:31
geektonimmh15:31
geektonibecause, I was thinking about extending Tags such that every time sets() is called then the observable will send some signals.15:33
geektonibut I don't know how to do it in the general case15:33
geektoniI mean15:33
geektonilet's say you have a variable called m_something, and you modify  it inside the algorithm's code, then  you should alert the observable that m_something has changed15:34
@wiking`yes15:40
@wikingand it's fine15:40
@wikingthat you only do that15:40
@wikingwhen you do a parameter.set()15:40
@wikingor of course15:41
@wikingif we are super haxors15:41
@wikingwe'll wrap variables into classes15:41
@wikingbecause you wanna do some autodiff YOLO shit15:42
@wikingand in this case you could override operator=15:42
@wiking:)15:42
@wikinggeektoni, see what i mean?15:42
@wikinglisitsyn, ^15:43
geektoniyes, yes, I see.15:43
lisitsynuhm15:44
lisitsynwiking: sorry what exactly :)15:44
@wikinglisitsyn, ok so15:45
@wikingsay we start having magical parameters15:45
@wikingremember of15:46
lisitsynaha15:46
@wikingDoubleWithInterval15:46
lisitsynoh super cool15:46
lisitsynyes15:46
@wikingso what if we could override there15:46
@wikingoperator=15:46
@wikingand then every of these fuckers15:46
@wikingand they would be observable15:46
@wikingand once you add it to a the tags15:46
@wikingthey would get subscribed15:47
lisitsynbut operator= in swig?15:47
@wikingto the changed observable15:47
@wikinglisitsyn, mmm life is hsit :)15:47
@wikinglisitsyn, set/get :)15:47
lisitsynok15:47
lisitsynsvm.set("C", Range())?15:47
@wikinglisitsyn, http://www.swig.org/Doc3.0/SWIGPlus.html#SWIGPlus_nn2815:48
lisitsynonly python?15:48
@wikingimo it's SWIG15:48
@wikingi mean it doesnt's ay anywhere15:48
@wikingthat python only15:48
lisitsynno way to make it work in java I think15:48
@wikinglisitsyn, yeah  :)15:49
@wikinglisitsyn, ok so15:50
@wikingi mean then i'm just wondering15:50
@wikingin long term again15:50
lisitsynaha15:50
@wikinghow would we be able to support15:50
@wikingconstrained parameters15:50
@wiking?15:51
@wikingso for example that you can define an interval on which they are valid15:51
@wiking+ that it has a generator function15:51
@wikingor just15:51
@wikingisValid(value)15:51
lisitsynhmm15:52
lisitsynyes15:52
@wikingso in this case15:52
@wikingwould we have some wrapper classes over primitive types?15:52
@wikingand those would be passed to tags?15:52
lisitsynit's tough15:54
lisitsynno idea yeat15:54
lisitsynwell15:54
lisitsynwe should always treat parameters as something multiple15:54
@wikingcoz in that case15:55
@wikingwe could make the wrappers observable15:55
@wikingand then those are to be subscribed15:55
@wikingand merged into15:55
@wikinga common observable15:55
@wikingthat later the user15:55
@wikingcan observe15:55
@wikingi.e. have a full stream of parameter changes15:55
@wikingwhile training15:55
@wikinggeektoni, anyhow15:57
@wikingok lets fucking d othis15:57
@wikingwe never finish anything otherwise15:57
@wiking:)15:57
@wikingwhat do i merge? :)15:57
lisitsynoh15:57
lisitsynok I am too stupid to think that further15:58
@wiking:)15:58
lisitsynI'd start with using tags yet15:58
@wikingyeye15:58
@wikingi mean tags is the way15:58
@wikingand we can hook tehre15:58
@wikingwhen it's .set()15:58
lisitsynyeah at least make everything use tags then we will see15:58
lisitsynyes15:58
geektoniwiking: so, in my opinion, we could merge the two PRs as they are right now15:58
@wikinggeektoni, yeah lemme go through them again15:59
geektoniafter them I'll make a PR with the addtions to CMachine.16:00
geektoniwiking: ah! One thing16:00
geektoniOpenMP16:00
@wikingwho's thing16:00
@wikingsukey, flip16:00
@sukey(/¯◡ ‿ ◡)/¯ ~ ┻━┻16:00
geektonilol16:00
geektonipretty accurate16:01
geektoniIt seems that it doesn't have a proper support for signal16:01
geektonibut, it shouldn't be a problem16:01
geektoniI mean POSIX signal in general16:02
@wikingeh what do yo umean?16:02
geektoniI mean16:02
geektonithat if I'm running a for loop inside a #pragma, and I press CTRL+C, only one thread will be blocked16:02
geektonithe others will continue happily16:03
geektoniuntil the user decide what to do16:03
geektonibut, as I said, it shouldn't be a problem16:03
@wikingmmm16:03
@wikinghttps://cdn.meme.am/cache/instances/folder193/400x/66401193.jpg16:04
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun16:05
-!- mode/#shogun [+o HeikoS] by ChanServ16:05
geektoniLOL16:05
@wikingHeikoS san16:06
@wikingaren't u sicko?16:06
@HeikoSwiking: hello16:06
@HeikoSyes I am16:06
@wikingHeikoS, then sleep16:06
@HeikoSbut turned on computer16:06
@HeikoSwhich had all the irc stuff on still ;)16:06
@wikinggeektoni, well i guess c'est la vie16:07
@wikingwe could play there with a semaphore :)16:07
@wikingbut yeah i mean16:07
@wikingor we just kill them all? :)16:08
@wikingkill it with fire!16:08
@wikinglaaaaaaaaaaaaalalallaa16:08
@wikingsukey, flip16:08
@sukey(ノ ゜Д゜)ノ ︵ ┻━┻16:08
geektoniwiking: this was a long day for you, wasn't it? ;)16:09
@wikinggeektoni, tell me about it :)16:09
geektonibtw, if we use things like cancel_computation() or something similar, multi threaded programs are not a problem.16:10
geektoniI mean16:13
geektonieven if we are using multi threading16:14
geektonithe signal handler will work and it will also stop the program as planned16:14
geektonifor example16:14
geektonihttps://pastebin.com/V5FMVZLK16:14
geektoniworks perfectly even if it is multithreaded16:14
geektoniwiking: ^16:14
@wikingmmmhmmm16:15
@wikingso this is more like converging16:15
@wikingto the story16:15
@wikingof16:15
@wikingpause-resume functionality16:16
@wikingright?16:16
@wikingbecause that'd be as well not part of the loop16:16
@wikingbut rather withing the loop16:16
@wiking*within16:16
@wikinganyhow16:16
@wikingpr review time16:16
@wikingmmm btw docker image should be ready16:16
@wikingso i'll kick off a new build16:16
@wikingsee if that works16:16
@wikingon travis16:16
geektonithanks!16:17
geektoniit should :)16:17
@wikingwtf16:18
@wiking  16:18
@wiking  16:18
@wikingThis job ran on our Trusty, sudo: required environment which will be updated on Wednesday, June 21st. Please add group: edge to your .travis.yml file to try the new images and check our blog for more details about this update.16:18
@wiking:)16:18
@wikinganybody knows something about this shit?16:18
geektonierr nope :P16:20
@wikingHeikoS, what's our name officially/16:21
@wikingin gsoc?16:21
@wikinggeektoni, running https://travis-ci.org/shogun-toolbox/shogun/jobs/24468888516:31
geektoniyeah, I saw it ;)16:33
CaBawiking: you pinged me a year ago or so? ;)16:34
@wikingCaBa, yes16:34
@wikingCaBa, have you used CommWordStringKernel?16:34
CaBapretty sure i have at some point, yes16:35
@wikingok so it's not strictly related16:35
@wikingbut do you know an easy way16:35
@wikinga mapped stringfeature back to string space?16:36
@wikingfrom uint16?16:36
@wikingfeats_train.obtain_from_char16:36
@wikingthat will convert stringfeatures16:36
@wikingbased on an alphabet16:36
@wikingright?16:36
CaBahmmm...16:37
@wikingno?16:37
@wiking:)16:37
CaBalet me boot that part of my brain :D16:38
CaBawhere is that snippet from?16:38
@wikingexamples/undocumented/python_modular/kernel_comm_word_string_modular.py16:38
@wikingbut i mean this is in a way unrelated to CommWordStringKernel16:39
@wikingits' more about stringfeatures16:39
CaBayeah i know, just a sec16:39
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3812 synchronized by MikeLing16:44
@wikingmmmm16:44
CaBawiking: so i've never done it in the other direction. for a moment i wasn't sure whether it's actually possible, i thought the conversion might actually be lossy, but apparently it's not. should be possible to reverse, but i don't know if the code is there16:44
@wikingtravis is gone16:44
CaBashould i run over there and yell? :P16:45
CaBaactually works fine here16:45
@wiking:)16:46
@wikinghttps://www.nytimes.com/2017/06/21/technology/uber-ceo-travis-kalanick.html?_r=016:46
@wikingi meant16:46
@wiking:>16:46
CaBaah lol16:46
CaBawell, big asshole afaiu...16:47
@wiking:)16:47
@wikingwhat a wonderful world :)16:48
@wikingwtf17:05
@wiking      17:05
@wiking        17:05
@wiking        17:05
@wiking        17:05
@wiking        17:05
@wiking      17:05
@wiking    Running for 45 min 48 sec17:05
@wiking:<17:05
@wiking:))17:05
-!- iglesiasg [~iglesiasg@217.119.234.214] has quit [Quit: leaving]17:10
geektoniwiking: wtf. Why did Travis die?17:13
@wikinglongruntime17:15
@wiking;)17:15
@wikingThe job exceeded the maximum time limit for jobs, and has been terminated.17:15
@wiking:)17:15
geektoniwtf17:17
geektoniwiking: I don't understand why it takes so long. :/17:31
@wikingi guess17:31
@wikingcoz u include it into machine.h?17:31
@wikingor no?17:32
@wikingsrc/shogun/kernel/Kernel.h:#include <shogun/lib/Signal.h>17:32
@wiking:)17:32
@wikingi guess this is hurting a bit17:33
geektonimmh17:34
geektoniI'll give a look17:34
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3848 synchronized by geektoni17:46
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Ping timeout: 240 seconds]17:54
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun18:02
-!- mode/#shogun [+o HeikoS] by ChanServ18:02
mikelingHeikoS: ping18:47
@HeikoSmikeling: sorry I am sick, just happen to have the computer open, will get back tomorrow or so18:51
mikelingHeikoS: oh, I am sorry to hear that18:51
mikelingwe could talk about it later18:51
mikelingjust want to ask a review or merge for a pr, wiking had approve it18:52
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Ping timeout: 255 seconds]18:59
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has quit [Remote host closed the connection]19:05
-!- mikeling [uid89706@gateway/web/irccloud.com/x-nwhagerlzhenkeyg] has quit [Quit: Connection closed for inactivity]21:21
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun22:28
-!- mode/#shogun [+o HeikoS] by ChanServ22:28
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Ping timeout: 255 seconds]22:33
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun23:03
-!- mode/#shogun [+o HeikoS] by ChanServ23:03
-!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Ping timeout: 240 seconds]23:13
-!- TingMiao [uid229534@gateway/web/irccloud.com/x-lgghrupggtdqrypi] has quit [Quit: Connection closed for inactivity]23:47
--- Log closed Thu Jun 22 00:00:23 2017

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!