IRC logs of #shogun for Thursday, 2017-07-13

« Wednesday, 2017-07-12 Index Friday, 2017-07-14 »
--- Log opened Thu Jul 13 00:00:53 2017
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]00:34
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun00:35
-!- mode/#shogun [+o sukey] by ChanServ00:35
-!- zxtx [~zxtx@sunnythinking.org] has quit [Ping timeout: 251 seconds]00:37
-!- zxtx [~zxtx@sunnythinking.org] has joined #shogun00:38
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3925 merged by vigsterkr01:03
@sukey[https://github.com/shogun-toolbox/shogun] vigsterkr pushed 2 commits:01:03
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/d08660b4fb6bd60e8c3cbb579b5176b12010e7a501:03
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/52f9f20c04e6284af5d390ec2b625ffe56c961da01:03
@sukey[https://github.com/shogun-toolbox/shogun] vigsterkr pushed 23 commits:01:11
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/f7fe02f60483cec2165384560c8592e86cd4349d01:11
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/917be87d368487dc81bdf817316cc9f4aa7e32ab01:11
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/0b9aeaa9167b8f21c4b16044eb38364bb0f6256301:11
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/0a742a2809b6035dda121402282aa5c6e52c5a1a01:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/30ca8ad1144ef57338fadecb85227a5262938e7101:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/07335b1195e4a16e59633c889db8a70d66f1e76401:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/007e7371ed68ace9aadf1e9a0ea321ef378c2ae701:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/b2552b7f2d16ee2cdfcaef7c8a3e85d71644328201:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/b9c8e315ad477efd6eee9a2931b6312dc58ad16901:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/7e3a23f51161d1765ddb98d8caae084dc2be774101:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/d0a58e82538dba09a7c653ac2139c4b9d3a3d89501:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/f687607e15c0a4d7c89713d4e1f9bb93f305db4d01:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/618ad00374a2d9963a9d013b4a33a658f91a99d701:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/a616e66ed63d07269f81771c66044ba520e8447b01:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/7d9a05afd12edea56a2db9910be4f3c3c57cd07401:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/392ecd28765ab259954f49d9041f900c807a4ae301:12
@sukeyhttps://github.com/shogun-toolbox/shogun/commit/91ac0de3ab4b7bd6cfbe67d99d5709d1378a17d601:12
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]01:12
-!- travis-ci [~travis-ci@ec2-54-144-60-107.compute-1.amazonaws.com] has joined #shogun01:53
travis-ciit's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/25300733101:53
-!- travis-ci [~travis-ci@ec2-54-144-60-107.compute-1.amazonaws.com] has left #shogun []01:53
@wikingmikeling, here?02:56
mikelingwiking: yes03:02
@wikingmikeling, ok so about the pR of GMM03:06
@wikinghere https://github.com/shogun-toolbox/shogun/pull/3859/files#diff-b59cfc6c549dd52160d0ce73b6356b04R42803:07
@wikingif you do this03:07
-!- travis-ci [~travis-ci@ec2-54-159-203-70.compute-1.amazonaws.com] has joined #shogun03:08
travis-ciit's Giovanni De Toni's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/25300934503:08
-!- travis-ci [~travis-ci@ec2-54-159-203-70.compute-1.amazonaws.com] has left #shogun []03:08
@wikingauto d1 = components[1]->get_d();03:08
@wikingauto d2 = components[2]->get_d();03:08
@wikingand then call linalg03:08
@wikingwhat happens?03:08
@wikingand of course change03:08
@wikinginline SGVector<float64_t>& get_d()03:08
@wikingto inline SGVector<float64_t> get_d()03:08
@wikingso you actually return a copy not a reference there03:08
mikelingwiking: like the things I do in https://github.com/shogun-toolbox/shogun/pull/3859/files#diff-b59cfc6c549dd52160d0ce73b6356b04R377 for get_mean()03:11
mikelingright?03:11
@wikingyeah but use please auto03:12
@wikingsince it's c++103:12
@wiking103:12
mikelingok03:12
@wikinganother question03:13
@wikingwhat's happening with prng03:13
@wikingi know some of them still fail03:13
@wikingbut that's because of different seed/prng behaviour03:13
@wikingnot because there's fundamentally something wrong, like in case of mmd, right?03:14
mikelingwiking: yes03:14
@wikingok03:14
mikelingwiking: so, what should we do next for prng?03:15
-shogun-buildbot:#shogun- Build osx1 - libshogun #80 is complete: Failure [failed compile (failure)] - http://buildbot.shogun-toolbox.org:8080/#builders/25/builds/8003:16
@wikingso the next move there is: a) expose get/set_global_seed in swig interfaces (or that has been done?) b) expose get_prng in swig as well c) remove CRandom as is and move to use get_prng + c++11 distributions03:16
@wikingmikeling, lemme know if you have any questions about how to do either of those03:17
@wikingmikeling, and i've just realised that instead of going into logging there's a far more important task to be done. not so interesting but very very useful for shogun.... it's feature/BigInteger which as about changing the typedef of index_t to typedef int64_t index_t;03:18
@wikingthis is quite a cumbersome task as you need to make sure with -Woverflow that everywhere we use index_t for indexing instead of int32_t and of course this is not true in shogun in many many places :(03:19
@wikinganyhow let's try to finish up prng refactoring03:19
mikelingwiking: yeah, sure. I would happy to work on it. Thank you for telling me that :)03:20
@wikingnw03:20
-!- mikeling is now known as mikeling|brb03:23
-shogun-buildbot:#shogun- Build bsd1 - libshogun #80 is complete: Failure [failed compile (failure)] - http://buildbot.shogun-toolbox.org:8080/#builders/31/builds/8003:25
-!- mikeling|brb is now known as mikeling04:28
-!- travis-ci [~travis-ci@ec2-54-92-202-98.compute-1.amazonaws.com] has joined #shogun04:31
travis-ciit's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/25303356004:31
-!- travis-ci [~travis-ci@ec2-54-92-202-98.compute-1.amazonaws.com] has left #shogun []04:31
mikelingwiking: ping06:13
mikelingI got this error https://pastebin.mozilla.org/9026988 after I do make && sudo make install06:13
mikelingit still call 'from modshogun import *' inside. Is that ok?06:14
@iglesiasgmikeling: try import shogun instead of modshogun08:21
mikelingiglesiasg wiking: I fix it by reinstall shogun package. Thank you :)08:45
@iglesiasgmm ok08:59
@iglesiasgmikeling: it works now with 'from modshogun import *' then?09:00
mikelingnono, I do 'from shogun import * ' in the beginning. I just don't know why it somehow reference from modshogun09:01
@iglesiasgok09:01
mikelinghttps://pastebin.mozilla.org/902698809:01
@iglesiasgI see now09:02
mikeling:)09:03
@iglesiasgI removed the modshogun.so and maybe other *modshogun* files from the python installation directory to avoid trouble09:04
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has joined #shogun09:16
@wikingyeah develop is at the moment very very changing :)09:22
geektoniwiking: the last PR should fix OSX and FreeBSD errors (I have no way to test it, though).09:47
@wiking\o/09:58
@wikinggeektoni, lets see09:59
@wikingjust merge09:59
@wikinggeektoni, shogun-buildbot will let us know10:02
geektoniwiking: nice :D btw, is sukey dead?10:02
@wikinghuuuuuuh10:03
@wikingwhere the fuck is sukey10:03
@wiking:D10:03
geektonilol10:03
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun10:04
-!- mode/#shogun [+o sukey] by ChanServ10:04
@wikingsukey, flip10:06
@sukeyʇǝʞɔɐɹq ︵ヽ(`Д´)ﾉ︵ ǝʞup10:06
micmnwiking: did you read my update on kpca + dsyevr?10:47
@wikingmicmn, where wwhere?10:48
@wikingi think i've missed that one10:49
micmnjournal, anyway in short:10:49
@wikingshow show10:49
@wiking :)10:49
micmneigen's eigensolver has 2 drawbacks: implements a slow algorithm (using lapack backend doesn't help)10:49
micmnand doesn't handle computation of top-k eigenvalues10:49
@wikingmmm10:50
@wikingfucker10:50
micmnso instead using lapack's dsyevr we are as fast as scikit/mlpack10:50
@wikingand yeah as you say it doesn't matter if we enable LAPACK for eigen?10:50
micmnyeah I tried10:50
@wikinghow did you enable lapack for eigen?10:50
@wikingwhich lapack have you tried?10:50
@wikingi mean which lapack implementation?10:51
@wikingatlas or openblas10:51
@wikingor netlib10:51
@wiking?10:51
micmnI used your feature branch eigen-lapack10:51
@wikingoh ic10:51
micmnis working no?10:51
@wikingi have never confirmed that it really working lapack10:52
@wikingit should :)))10:52
@wikingbut i have never really tested it (debug)10:52
@wikingand note that there i enable only10:52
@wikingand you have 3.3.x eigne?10:52
micmnok not sure which lapack I'm using but the problem is that eigen uses syev10:53
@wikingEIGEN_USE_LAPACKE_STRICT Same as EIGEN_USE_LAPACKE but algorithms of lower numerical robustness are disabled.10:53
@wikingThis currently concerns only JacobiSVD which otherwise would be replaced by gesvd that is less robust than Jacobi rotations.10:53
@wikingso i did not enable EIGEN_USE_LAPACKE10:53
@wikingEnables the use of external Lapack routines via the Lapacke C interface to Lapack (compatible with any F77 LAPACK interface)10:53
@wikingbut it suppose to be only about svd10:54
@wikingalthough i dont know what you use actually for svd in kpca10:54
micmnno I'm using svd10:54
micmn*not using10:54
micmnI'm using selfadjointeigensolver10:55
micmnwhich with lapack's enabled maps to syev10:56
micmnthat it's not the fastest routie10:56
micmn*routine10:56
-shogun-buildbot:#shogun- Build osx1 - libshogun #84 is complete: Success [build successful] - http://buildbot.shogun-toolbox.org:8080/#builders/25/builds/8410:56
@wikinggeektoni, ^10:56
@wikingmicmn,10:57
@wikinghave you asked #eigen why dont they use10:57
@wikingdsyevr10:57
@wiking?10:57
geektoniwiking: cool! \o/10:57
@wikingmicmn, i've just asked it :)10:58
micmnnot sure but eigen should implement an algorithm similar to dsyev so that maybe the reason, I could ask anyway10:58
@wikingmicmn, but yeah in this case we should maybe default back to lapack10:58
micmnok, so my question is do we integrate it into linalg?10:59
@wikingmicmn, so come to #eigen as one of the core developers10:59
@wikingare there actually talking with me10:59
@wikingjump into the conversation10:59
-shogun-buildbot:#shogun- Build bsd1 - libshogun #84 is complete: Success [build successful] - http://buildbot.shogun-toolbox.org:8080/#builders/31/builds/8411:06
@wikingmicmn, how do you like christoph's idea ? :)11:13
micmnthat is: use lapack no? :P11:14
@wiking:D11:17
@wikingwell yeah11:17
@wikingmicmn, i mean there's the only way to do this11:18
micmnok so I'll integrate it into linalg11:18
@wikingthat you do a preference for HAVE_LAPACK11:18
@wikingi.e. if HAVE_LAPACK is available you use that eigensolver11:18
@wikingotherwise use eigen's one11:18
micmnyep11:19
@wikingmicmn, in the patch put somewhere a note why this is happening... maybe one day eigen people decide to use dsyevr11:25
@wikingand then we can drop this hack11:26
micmnsure11:26
@wikingthnx11:26
micmnwiking: btw this CSVFile bug is really annoying! https://github.com/shogun-toolbox/shogun/issues/199111:30
@wikingmicmn, :DDDDDDDDDDDDDDDDDDDDDD11:30
@wikingssssssssssssshhhhhhhhhhhhhh11:30
@wikingact like you dont see it11:31
@wiking:)11:31
micmnok I've just increased the buffer size... :P11:31
@sukey[https://github.com/shogun-toolbox/shogun] Issue https://github.com/shogun-toolbox/shogun/issues/3926 closed by vigsterkr11:31
@wiking:>11:31
@wikinglike a pro11:31
-!- travis-ci [~travis-ci@ec2-54-92-202-98.compute-1.amazonaws.com] has joined #shogun11:45
travis-ciit's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/25310389111:45
-!- travis-ci [~travis-ci@ec2-54-92-202-98.compute-1.amazonaws.com] has left #shogun []11:45
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has quit [Ping timeout: 260 seconds]12:58
TrixisOXPHOS: cool nickname13:28
Trixiswiking: i finally got everything to work, apparently, so far on numeric features only13:28
Trixiswiking: although im a bit suspicious that after running grid search to find the optimal C value, all Cs from 0 to 10 are apparently nearly equally good in performance13:30
@wiking:>13:31
Trixiswiking: that doesnt seem correct, right?13:31
@wikingwell lets say that it'd be very surprising if that's true13:32
Trixiswiking: yeah the minimum average are under PRC after 5x-validations is 0.57, and the max is 0.5913:38
Trixis*average area13:38
Trixis... and if i set c = 10000 im getting PRC of 0.7, ROC of 0.73 (as opposed to ROC of 0.61 on avg for values 0 - 10)13:44
Trixisalthough as would be expected the accuracy drops to 0.5 for the c of 10000, so no better than random guessing13:47
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has joined #shogun14:39
Trixiswell time to move the test onto the cluster and run it on the full dataset which is two orders of magnitude larger.14:44
Trixiswiking: in order to set C for mkl binary classification, i call the set_C() method in the solver (in this case SVMLight) passed to MKLClassification, right?15:32
@wikinghopefully yes15:32
@wikingdidn't really played ever with mkl15:33
Trixislet me run a few tests15:34
Trixiswell theyre running now15:34
Trixisright, calling set_C on the solver doesnt do anything, i.e. the results are /identical/ to15:35
Trixiseach other15:35
Trixiscalling set_C in MKLClassification works. Unless it's 0. then it goes up in flames and crashes15:35
-!- iglesiasg [~iglesiasg@217.119.234.214] has quit [Quit: leaving]15:41
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3928 opened by geektoni16:26
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/392816:38
@wikinggeektoni, wooooooooooooooah16:46
@wikingman that looks amazing16:46
@wiking!16:46
geektoniwiking: and this is just the beginning ;)16:48
@wikingm_enable_emit16:52
@wiking?16:52
@wikingi mean the good thing about observables16:52
@wikingthat if  you have no subscriber16:52
@wikingi.e observer16:52
@wikingit should be a nop16:53
@wiking:16:53
geektoniwiking: I didn't investigate the RxCPP behavior when there are no subscribers (or I didn't notice when I read the documentation).16:55
Trixisright 10^8 is not an acceptable value for C (numeric instability)17:00
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun17:06
-!- mode/#shogun [+o HeikoS] by ChanServ17:07
geektoniping HeikoS17:16
@HeikoSgeektoni: pong17:17
@HeikoSnice patch man!17:17
@HeikoSreally nice work this is17:17
@wikinggeektoni, http://reactive-extensions.github.io/RxCpp/classrxcpp_1_1observable.html#a5b3dcda85c609a5006fa1707da5ae3d717:17
@wikinggeektoni, just sayin' :)17:17
geektoniHeikoS: it still needs a lot of work :p17:18
@wikinggeektoni, maybe we wanna change to this as tb needs timestamp anyways17:18
@wikingright?17:18
@HeikoStimestamps++17:18
@HeikoSgeektoni: you got my comments on the folds?17:19
geektoniwiking: mmh, I need to look into it because we are currently using a subject17:19
geektoniand I don't know if we can override the subject's observable with this new one you just mentioned.17:19
geektoniHeikoS: yes yes17:19
geektoniHeikoS: that's why I pinged you ;)17:20
@wikinggeektoni, subject = observable + observer right?17:20
@wikingit has an observer and an observerable17:20
geektoniwiking: yes17:20
@wikingso once you do get_observable () const17:20
@wikingthen it's timestamp()17:20
@wikingand it'll just create a pair17:20
@wikingof whatever you have in as observed17:20
@wikingwhere .second = time_unit17:20
geektoniwiking: ah! Okay, I thought it was a factory class17:20
geektoniok, it is doable17:20
@wikingso then you can just drop the whole chrono story17:21
@wiking:P17:21
geektoniHeikoS: first of all, to quickly answer to your email, as far as i know, there is no way to store indices into Tensorboard.17:21
@HeikoScant we do a histogram on integers?17:22
@HeikoSor a time series?17:22
@HeikoSI mean17:22
@HeikoSthis is not so interesting for tensorboard17:22
@HeikoSbut more for extracting the indices manually for some custom analysis later17:22
@HeikoSyou run the xvalidation17:22
@HeikoSthen you want the indices in your python code17:22
geektoniHeikoS: yes, we can, but tensorboard's histogram are different17:23
geektoniHeikoS: https://www.tensorflow.org/get_started/tensorboard_histograms17:23
@HeikoSok i see17:25
@HeikoSwell this is not so much about tensorboard17:25
@HeikoSbut more about getting hand on the indices17:25
@HeikoSI assume one can directly access observables without going through tensorboard17:25
geektoniyes, it is possibile. However, the only ParameterObserver that are currently done works only with Tensorboard.17:26
OXPHOSTrixis: thx :D17:27
@HeikoSgeektoni: let's make one for indices in xvalidation that I can access from swig :D17:28
TrixisOXPHOS: do you do bioinformatics / comp bio?17:28
geektoniHeikoS: mmh, and what will this new ParameterObserver do with the values it receive? Will it print them to stdout?17:30
@HeikoSno store them17:30
@HeikoSso one can access later on17:30
geektonilike, store them to a variable (vector since they're indices)?17:31
@HeikoSgeektoni: some dynamically growing structure17:32
@HeikoSwhere you later one can nicely access the indices17:32
@HeikoSor the folds17:32
@HeikoSfold results17:32
@HeikoSanything really17:32
@wikinggeektoni, geektoni https://github.com/Reactive-Extensions/RxCpp/blob/master/Rx/v2/src/rxcpp/subjects/rx-subject.hpp#L18517:34
@HeikoSgeektoni: this is hugely useful for scientists that want to look inside the blackbox of algorithms, but without going the tensorvboard extra way (which is also cool)17:34
@wikinggeektoni, hence remove the emit boolean flag plz17:34
geektoniHeikoS: ok, understood. I'll come up with something.17:34
@HeikoSgeektoni: ace!17:35
geektoniwiking: ok, fair enough. I'll remove the flag then.17:40
OXPHOSTrixis: not really...only do benchwork in lab17:44
Trixishah fair17:44
Trixisit's lifesciences nevertheless17:44
Trixisi assumed either of those two because ML and so on17:44
OXPHOSmake more sense actually..didn't think that much when I joined the lab17:50
geektoniwiking: regarding the timestamp thing, I think we'll still need <chrono>, since we manually call on_next() to send values :P17:57
-!- yammerz [~myamada@64.7.137.182] has quit [Ping timeout: 255 seconds]18:22
-!- yamz [~myamada@2620:13:0:10:ad90:f873:79fb:1bee] has joined #shogun18:22
-!- mikeling [uid89706@gateway/web/irccloud.com/x-siynttqijkgauifg] has quit [Quit: Connection closed for inactivity]18:25
-!- yammer [~myamada@64.7.137.182] has joined #shogun18:26
-!- yamz [~myamada@2620:13:0:10:ad90:f873:79fb:1bee] has quit [Read error: Connection reset by peer]18:26
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has quit [Remote host closed the connection]18:30
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 255 seconds]19:45
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has joined #shogun22:08
-!- mode/#shogun [+o HeikoS] by ChanServ22:08
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has quit [Quit: Leaving.]22:16
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]23:52
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun23:53
-!- mode/#shogun [+o sukey] by ChanServ23:53
--- Log closed Fri Jul 14 00:00:54 2017
« Wednesday, 2017-07-12 Index Friday, 2017-07-14 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!