IRC logs of #shogun for Tuesday, 2019-03-19

--- Log opened Tue Mar 19 00:00:58 2019
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]00:43
-!- wuwei[m] [wuweilinma@gateway/shell/matrix.org/x-ifagvfypgfwvlxcu] has quit [Remote host closed the connection]01:28
-!- wuwei[m] [wuweilinma@gateway/shell/matrix.org/x-zjlpldhlbsckqobs] has joined #shogun01:35
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun02:38
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]02:38
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun02:38
-!- mode/#shogun [+o wiking] by ChanServ02:38
-!- lambday [a7dcee98@gateway/web/freenode/ip.167.220.238.152] has quit [Quit: Page closed]03:42
-!- lambday [a7dcee98@gateway/web/freenode/ip.167.220.238.152] has joined #shogun04:14
-!- mode/#shogun [+o lambday] by ChanServ04:14
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]04:19
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun04:38
-!- mode/#shogun [+o wiking] by ChanServ04:38
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]04:57
-!- sighingnow [~sighingno@2001:da8:203:81:b4d6:e223:34c1:836e] has joined #shogun06:44
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun06:45
-!- mode/#shogun [+o wiking] by ChanServ06:45
-!- sighingnow [~sighingno@2001:da8:203:81:b4d6:e223:34c1:836e] has quit [Remote host closed the connection]07:45
-!- gf712 [c13cdcfd@gateway/web/freenode/ip.193.60.220.253] has joined #shogun08:20
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has joined #shogun09:48
-!- novate [96421b53@gateway/web/freenode/ip.150.66.27.83] has joined #shogun09:59
-!- novate [96421b53@gateway/web/freenode/ip.150.66.27.83] has quit [Client Quit]10:00
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]10:09
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:10
-!- mode/#shogun [+o wiking] by ChanServ10:10
-!- novate [~Mutter@111.187.63.140] has joined #shogun10:12
-!- novate [~Mutter@111.187.63.140] has quit [Client Quit]10:13
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 272 seconds]10:15
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:22
-!- mode/#shogun [+o wiking] by ChanServ10:22
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 252 seconds]10:27
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:34
-!- mode/#shogun [+o wiking] by ChanServ10:34
@wikingmiju10:47
@wikingi haven't checked valgrind for a while10:47
@wikingbut i suspect that we leak quite a lot :P10:47
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has quit [Ping timeout: 256 seconds]11:16
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has joined #shogun11:27
@wikinggeektoni: ping11:30
@wikinggeektoni: https://github.com/shogun-toolbox/shogun/pull/4573/files#diff-39d6ac0c2f48ea892f1021a83b20d4d011:30
@wikingif u want i can carve this out from this commit and push it as is into develop11:31
@wiking"if u want" = if this would work for u as well :P11:31
geektoniwiking: I guess we could directly use that commit since it is more general than mine :)11:34
@wikinguff11:34
geektoniLoL11:34
@wikingthat commit is super convoluted11:34
@wiking:)))11:34
@wikingit contains a lot of shit11:34
@wikingthat is super unrealted11:35
geektoniif it is not too much trouble of course11:35
@wikingok i'll just copy paste it11:35
@wikingand push it into develop11:35
@wikingso you just rebase11:35
geektoni\o/11:35
geektonithanks11:35
@wiking799973c6d11:37
@wikingpushed11:37
geektonigood good11:41
-!- geektoni [c1cdd253@gateway/web/freenode/ip.193.205.210.83] has quit [Quit: Page closed]12:00
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]12:13
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun12:15
-!- mode/#shogun [+o wiking] by ChanServ12:15
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 264 seconds]12:20
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun12:46
-!- mode/#shogun [+o wiking] by ChanServ12:46
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]13:44
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun13:46
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]13:46
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun13:46
-!- mode/#shogun [+o wiking] by ChanServ13:46
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 246 seconds]13:50
-!- wiking [~wiking@2001:67c:10ec:5784:8000::3ff] has joined #shogun13:57
-!- wiking [~wiking@2001:67c:10ec:5784:8000::3ff] has quit [Changing host]13:57
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun13:57
-!- mode/#shogun [+o wiking] by ChanServ13:57
@wikinggf712:13:58
@wikingdo you want me to push the fix13:58
@wikingor the code13:58
@wikingor what?13:58
@wikingi mean take out the triangularview of that sgvector13:59
gf712wiking: not sure how to do that...14:07
gf712I did the mapping14:07
gf712then called triangular view14:07
gf712with Eigen::StrictlyUpper14:07
gf712but all that does is set the lower triangle to 0s14:08
gf712I still need to extract the upper triangle14:08
gf712I think I'm missing something?14:08
@wikingk14:15
@wikinglemme generate a patch14:15
@wikingeasier14:15
@wiking:)14:15
gf712aight thanks!14:16
@wikingnw14:17
@wikingbut first i just want to fix the last remaining failing unit test in deadbeaf14:17
@wiking*beef14:18
@wikingand then i'll switch to this14:18
gf712ok!14:18
gf712are you merging that whole thing in one go?14:18
@wikingprobably14:18
@wiking:)14:18
@wikingbut i mean14:18
@wikingthere's still a huge problem14:18
gf712loco14:18
@wikingthat trait14:18
@wiking:D14:18
@wikingmsvc fails big time with it14:18
@wiking:)14:18
gf712hmmm and nothing on stackoverflow?14:19
@wikinghaven't checked yet14:19
@wiking:)14:19
@wikingah no i still need to fix one more thing on top of that14:20
@wikingwriting complex128 and floatmax to json14:20
@wiking:)14:20
@wikingprobably cutting the value into two doubles14:20
gf712why floatmax?14:20
@wikinglong double14:20
gf712is it too large for json?14:20
gf712ah ok14:20
@wikingrapidjson has no interface only for double14:20
@wikingthat i assume float6414:21
@wikingbut yeah dunno the json serialization is fucking slow14:21
@wikingi mean deser14:21
@wikingbut yeah i think that's because of using std::string as a buffer14:21
@wikingneed to benchmarkit once merged14:21
gf712hmm, but std string wouldn't make it that much slower?14:23
@wikingwell perf and dtrace begs to differ14:24
@wiking:)14:24
gf712I mean it just adds some utilities around char right? the compiler should optimise?14:24
gf712are you using -O3?14:24
@wikingi mean it takes forever to deserialize a 270k json file14:24
@wikingRelease yes14:24
gf712very weird14:24
@wikingyeah especially that rapidjson is like the fastest json parser14:24
@wiking:)14:24
gf712where does it spend most of the time then?14:25
@wikingwhat output would u like to have?14:25
@wikingperf?14:25
gf712yes, if you have it?14:25
gf712just thinking what could be so costly...14:25
@wikingk14:26
@wikinggo into donbot14:28
@wikingits in /tmp/perf.data14:29
@wikingmmm wait14:29
@wikingneed chmod14:29
@wikingwtf14:29
@wikingah ok14:29
@wikingso cd /tmp and then perf report14:30
@wikingi haven't tested to read that json with rapidjsons native file input stream14:31
@wikingi guess i should do it14:31
@wiking:)14:31
gf712ok, let me try14:31
@wikingjust to compare14:31
@wikingsince now we use like an input stream adapter14:31
-!- novate [~Mutter@103.102.6.233] has joined #shogun14:33
novateHi everyone! I am quite interested in the topic of detox++ could anyone help me?14:39
wuwei[m]novate: hi14:40
@wikingnovate: what do you need help with14:40
@wiking?14:40
novateHi wiling, this is the first experience of GSoC, I am very interested in your organization and that topic, what can I do.14:42
@wikingnovate: first thing first14:42
@wikingread our instructions for the newcomers14:42
@wikingit's on the wiki14:42
@wikinghttps://github.com/shogun-toolbox/shogun/wiki/Getting-involved14:42
@wikingthis gives you a good set of tasks that you should start with14:42
novateThank you very much for your kind reply, I'm gonna read that carefully and consider.14:43
@wikingno worries14:44
-!- cruel [0e8bd4e4@gateway/web/freenode/ip.14.139.212.228] has joined #shogun14:45
-!- novate [~Mutter@103.102.6.233] has quit [Quit: Mutter: www.mutterirc.com]14:53
-!- novate [~Mutter@103.102.6.233] has joined #shogun14:59
-!- novate [~Mutter@103.102.6.233] has quit [Client Quit]15:00
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]15:01
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun15:01
-!- mode/#shogun [+o wiking] by ChanServ15:02
-!- cruel [0e8bd4e4@gateway/web/freenode/ip.14.139.212.228] has quit [Ping timeout: 256 seconds]15:10
-!- novate [~Mutter@114.87.9.16] has joined #shogun15:19
-!- novate [~Mutter@114.87.9.16] has quit [Client Quit]15:19
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]16:01
-!- wiking [~wiking@195.176.113.33] has joined #shogun16:05
-!- wiking [~wiking@195.176.113.33] has quit [Changing host]16:05
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun16:05
-!- mode/#shogun [+o wiking] by ChanServ16:05
@wikinggf712: here?16:25
@wikingsince we dont have if constexpr() in c++14 would you use templates to decide whether the compiler has support for something or else? :)16:26
@wikingworking on long double... and wanna just do a sizeof(float64_t) == sizeof(floatmax_t) check... and if that's the case do one operation or if it's false do another thing16:27
@wikingalthough with this i'll have a problem porting code :)))16:27
@wikingi mean serialized objects16:27
@wikingso i shouldn't do this16:27
-!- gf712_ [c13cdcfd@gateway/web/freenode/ip.193.60.220.253] has joined #shogun16:35
gf712_wiking: sorry got disconnected16:36
gf712_I saw you message16:36
@wikingnvm16:36
gf712_I would do tag dispatching16:36
gf712_ah ok :p16:36
-!- gf712 [c13cdcfd@gateway/web/freenode/ip.193.60.220.253] has quit [Ping timeout: 256 seconds]16:38
-!- gf712_ is now known as gf71216:40
-!- yv [7d63a1e5@gateway/web/freenode/ip.125.99.161.229] has joined #shogun17:13
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]17:14
yvhello sir! can you help me up with this procedur?17:15
-!- novate [~Mutter@103.102.6.233] has joined #shogun17:16
-!- HeikoS [3e1edec7@gateway/web/cgi-irc/kiwiirc.com/ip.62.30.222.199] has joined #shogun17:24
-!- novate [~Mutter@103.102.6.233] has quit [Quit: Mutter: www.mutterirc.com]17:25
-!- novate [~Mutter@103.102.6.233] has joined #shogun17:26
-!- novate [~Mutter@103.102.6.233] has quit [Client Quit]17:29
-!- ghaoziang [d569a78a@gateway/web/freenode/ip.213.105.167.138] has joined #shogun17:35
-!- ghaoziang [d569a78a@gateway/web/freenode/ip.213.105.167.138] has quit [Client Quit]17:35
-!- novate [~Mutter@103.102.6.233] has joined #shogun17:38
-!- novate [~Mutter@103.102.6.233] has quit [Client Quit]17:39
HeikoSgf712 yooy17:41
gf712HeikoS: hey17:42
HeikoSim in london now17:42
HeikoSbut I guess wont come in until tomorrow17:42
HeikoSthanks for the median patch17:42
gf712OK! What time are you coming in tomorrow?17:42
gf712no worries17:42
gf712I am refactoring the model selection stuff17:42
gf712trying to get rid of Parameter17:43
gf712and move all to Any/AnyParameter17:43
HeikoSah that is great17:44
HeikoSyou can totally scrap gridsearch btw17:44
HeikoSParameterTree17:44
HeikoSParameterCombination17:44
HeikoSetc17:45
HeikoSthat needs a rewrite17:45
gf712ah really? I am rewriting it already17:45
gf712moving all to stl containers17:45
gf712but with similar logic17:45
HeikoSI think the logic is flawed tbh17:45
HeikoSwhats your view on that?17:45
gf712I think there is a lot of repetition :D17:45
gf712but yea, might be worth rewrite a lot of this stuff17:46
gf712from scratch17:46
gf712can discuss tomorrow17:46
HeikoSyeah can do17:47
HeikoSso for the tree for example17:47
HeikoSI am sure something better can be done17:47
HeikoSespecially API wise17:47
HeikoSI mean the tree-like structure is needed17:47
HeikoSbut maybe that can be done a bit easier17:47
HeikoSor rather even, without an additional datastructure17:47
HeikoSand re-useing the objects instead17:47
gf712yea, I rewrote the nodes17:47
gf712to use maps17:47
gf712and vectors17:47
HeikoSso that users can specify ranges/sets for every parameter17:47
gf712so not quite a traditional tree17:47
gf712ah yes17:48
HeikoSso that a user doesnt need to build something new17:48
HeikoSbut rather does17:48
HeikoSsvm.kernel.log_grid(-5,5,1)17:48
gf712yea, can do that17:48
HeikoSsvm.kernel.types([GaussianKernel, PolyKernel])17:48
HeikoSthe recursive stuff is more tricky and requires thought17:48
HeikoSbut the 1D numerical values could be nicely represented this way17:49
HeikoSI mean the current framework can do complicated stuff like17:49
HeikoSdifferent kernels with different parameter ranges17:49
HeikoSbut is that even needed?17:49
HeikoSI feel it might be overkill17:49
HeikoSand selecting parameter combinations of flat object hierarchies is enough17:49
HeikoSand then users can put in another kernel by hand maybe?17:50
HeikoSI mean I dont know17:50
HeikoSlet's discuss17:50
gf712hmmm I am not sure17:50
HeikoSbut with lisitsyn we already thought a bit about this a while ago17:50
gf712I have something like this17:50
HeikoSand came up with this thing above17:50
gf712import shogun as sg17:50
HeikoSkernel.log_width.range(bla)17:50
gf712model = sg.machine("LibSVM") kernel = sg.kernel("GaussianKernel")17:50
gf712param_tree_root = sg.ModelSelectionParameters("SVM", model) param_tree_root.build_values("C1", 0.01, 10, sg.R_LOG, 1, 10) param_tree_root.build_values("C2", 0.01, 10, sg.R_LOG, 1, 10)17:50
HeikoSmain question is: do we need an abstraction for parameters that can be applied to models17:51
HeikoSor shall we modify the model (or cloned models) directly17:51
gf712hmm, is the abstraction for parameters necessary?17:52
gf712wouldn't it be quite complicated?17:52
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun17:52
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]17:52
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun17:52
-!- mode/#shogun [+o wiking] by ChanServ17:52
gf712something like sklearn search algos would be neat no?17:53
gf712and not too difficult...17:53
-!- novate [~Mutter@103.102.6.233] has joined #shogun17:53
HeikoSyep17:53
HeikoSI wrote a GSoC project description for this stuff a while ago17:53
HeikoSlet me find it17:53
HeikoShttps://github.com/shogun-toolbox/shogun/wiki/GSoC_2019_project_modelselection17:54
HeikoSif we had nice algorithms, we could offer shogun as a model-selection framework wrapper in all the interface langs17:55
HeikoSthat would be quite useful I think17:55
HeikoSlike you wrap your aglo in a shogun java API, and then it can learn parameters for you17:55
HeikoSmeta learning 2.0 :)17:55
gf712I thought model selection is already available in all languages?17:56
HeikoSyeah sure17:58
HeikoSbut if we had some new algos in there17:58
HeikoSlike CMA17:58
HeikoSor some nice Bayesian optimization lib17:58
HeikoSwell anyways17:58
HeikoSlet's discuss this stuff tomorrow as it is quite about time to update it :)17:58
gf712ahh17:59
gf712yes, that would be neat17:59
gf712everyone loves bayesian opt in kaggle :D17:59
gf712aight17:59
-!- novate [~Mutter@103.102.6.233] has quit [Quit: Mutter: www.mutterirc.com]17:59
gf712HeikoS: I will be here the whole day so let me know when you're at the ATI18:00
HeikoScool18:00
HeikoSoh18:00
HeikoSI had this other idea in the plane18:00
HeikoSwhile hacking stuff18:00
HeikoSlet me try to recall it18:00
HeikoSah yes18:01
HeikoSit would be cool if we could dispatch file formats18:01
HeikoScurrently we have18:01
HeikoSsg.csv_file factory18:01
HeikoSand I also just added sg.libsvm_file18:01
HeikoSand those files can be passed to the feature factories for loading18:01
HeikoSnow I wonder, couldnt we just have sg.file(fname) factory18:02
HeikoSand it figures out the format and loads appropriately?18:02
HeikoSvia trial and error18:02
gf712you mean from the extension?18:02
gf712or from the actual buffer?18:02
gf712can try, not sure how well that would work in terms of speed?18:03
gf712depends how many formats you need to try>?18:03
HeikoScsv-dense, sparse?18:03
HeikoShdf518:03
gf712HeikoS: I need to go now, but can discuss tomorrow?18:03
HeikoSyeah speed is an issue of course18:03
HeikoSyeah ok18:03
HeikoSsee you tomorrow18:03
gf712see you tomorrow18:04
-!- gf712 [c13cdcfd@gateway/web/freenode/ip.193.60.220.253] has quit [Quit: Page closed]18:04
HeikoSlisitsyn yo18:29
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]19:03
-!- HeikoS [3e1edec7@gateway/web/cgi-irc/kiwiirc.com/ip.62.30.222.199] has quit [Ping timeout: 250 seconds]19:07
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun19:19
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]19:19
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun19:19
-!- mode/#shogun [+o wiking] by ChanServ19:19
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]19:44
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun20:08
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]20:08
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun20:08
-!- mode/#shogun [+o wiking] by ChanServ20:08
-!- yv [7d63a1e5@gateway/web/freenode/ip.125.99.161.229] has quit [Ping timeout: 256 seconds]20:32
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]23:32
--- Log closed Wed Mar 20 00:00:59 2019

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!