IRC logs of #shogun for Thursday, 2016-06-09

« Wednesday, 2016-06-08 Index Friday, 2016-06-10 »
--- Log opened Thu Jun 09 00:00:32 2016
-!- travis-ci [~travis-ci@ec2-54-162-19-163.compute-1.amazonaws.com] has joined #shogun00:08
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/13614532000:08
-!- travis-ci [~travis-ci@ec2-54-162-19-163.compute-1.amazonaws.com] has left #shogun []00:08
-!- sonne|osx [~sonne@x4e33f692.dyn.telefonica.de] has quit [Quit: sonne|osx]00:13
-!- sonne|osx [~sonne@x4e33f692.dyn.telefonica.de] has joined #shogun00:14
-!- sonne|osx [~sonne@x4e33f692.dyn.telefonica.de] has quit [Client Quit]00:17
-!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has quit [Quit: Page closed]01:00
-!- arianepaola [~ariane@unaffiliated/arianepaola] has quit [Ping timeout: 252 seconds]01:37
shogun-buildbotbuild #1017 of nightly_none is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_none/builds/1017  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>03:04
-!- arianepaola [~ariane@unaffiliated/arianepaola] has joined #shogun03:05
-!- lambday [6d9941c0@gateway/web/freenode/ip.109.153.65.192] has quit [Ping timeout: 250 seconds]03:27
shogun-buildbotbuild #15 of clang - thread analysis is complete: Failure [failed compile]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang%20-%20thread%20analysis/builds/15  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>03:43
shogun-buildbotbuild #14 of clang - undefined behaviour analysis is complete: Failure [failed compile]  Build details are at http://buildbot.shogun-toolbox.org/builders/clang%20-%20undefined%20behaviour%20analysis/builds/14  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>03:47
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun04:00
-!- sanuj [~sanuj@117.204.252.181] has quit [Ping timeout: 240 seconds]04:25
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun05:00
Saurabh735 % cache misses :/05:08
sanujwiking, got time?05:12
shogun-buildbotbuild #16 of memleak - valgrind is complete: Failure [failed memory check]  Build details are at http://buildbot.shogun-toolbox.org/builders/memleak%20-%20valgrind/builds/16  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>05:30
shogun-buildbotbuild #1018 of nightly_all is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_all/builds/1018  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>05:36
-!- sanuj [~sanuj@117.204.252.181] has quit [Read error: Connection reset by peer]05:44
@wikingSaurabh7: why so many misses? :S05:56
Saurabh7wiking: no idea05:58
Saurabh7and when i do map.transpose.col(i) it goes to 47%05:58
@wikingyeah i see that05:58
@wikingmm05:58
@wikingone thing we should try05:58
@wikingi mean i know it's a bit unrelated05:58
@wikingmmmm can you give me the code/branch where this is available?05:59
Saurabh7map.transpose.col(i)  this version ?05:59
@wikingany of them06:00
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun06:00
Saurabh7my pr branch can be used06:01
Saurabh7https://github.com/Saurabh7/shogun/tree/lars06:01
@wikingwhich is the code you use for running perf?06:02
@wikingSaurabh7: ^06:03
Saurabh7wiking: https://gist.github.com/Saurabh7/74f551a4c2bd15e12f8aaaaeade7f81606:03
Saurabh7but i am using a dataset file06:03
@wikingwhere can i get taht06:04
@wikingis it part of data?06:04
Saurabh7no06:04
Saurabh7its in zoq/benchmark data06:04
@wikinglink?06:04
Saurabh7wiking: https://bitbucket.org/zoqbits/benchmark-data/src06:05
Saurabh7madelon_X.csv and madelon_y.csv06:05
-!- sanuj [~sanuj@117.204.252.181] has quit [Read error: Connection reset by peer]06:07
@wikingSaurabh7: can you give me your perf cmd line args?06:10
Saurabh7wiking: perf stat -e task-clock,cycles,instructions,cache-references,cache-misses ./a.out06:10
@wikingthnx06:11
@wikingok so with that code i get06:13
@wiking        18,429,761      cache-misses              #   35.330 % of all cache refs06:13
@wikingright?06:13
Saurabh7yesyes06:13
@wikingjust a sec06:13
shogun-buildbotbuild #1147 of nightly_default is complete: Failure [failed test test_1 notebooks]  Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/1147  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, OXPHOS <engelzora@gmail.com>06:15
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun06:27
-!- sanuj [~sanuj@117.204.252.181] has quit [Ping timeout: 260 seconds]06:38
Saurabh7hm when I argsort the col accessors it falls by 2%06:53
Saurabh7hehe06:53
Saurabh7maybe I should make a copy of X and store only the cols we acess, pretty sure no misses then06:55
@wikingmmm but it's a copy then07:14
@wikingi mean it adds memory overhead07:15
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun07:30
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has joined #shogun07:31
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/13623542707:31
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has left #shogun []07:31
-!- GandalfTheWizard [~Eva@112.10.170.58] has joined #shogun08:33
-!- GandalfTheWizard [~Eva@112.10.170.58] has quit [Quit: Leaving.]09:25
-!- sanuj [~sanuj@117.204.252.181] has quit [Ping timeout: 250 seconds]10:55
-!- sanuj [~sanuj@117.204.252.181] has joined #shogun10:58
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun11:12
-!- mode/#shogun [+o HeikoS] by ChanServ11:12
@HeikoSSaurabh7: jo11:20
Saurabh7HeikoS: ello11:20
@HeikoSSaurabh7: how are things?11:21
Saurabh7HeikoS: updated with results here: https://github.com/shogun-toolbox/shogun/pull/324311:22
@HeikoSSaurabh7: checking11:23
@HeikoSSaurabh7: cool thanks for the numbers11:24
@HeikoSone thing I wanted to ask about is this transpose thing11:24
@HeikoSdouble transpose, you said, was faster11:24
Saurabh7HeikoS: we need to decide on one or two things, other than that I have tried most of things I can with eigen11:24
Saurabh7HeikoS: doubel transpose ?11:24
@HeikoSlike transpose in beginning, do things, then another transpose at the end11:25
@HeikoSwhat viktor didnt like11:25
@HeikoSbut then you said it is faster11:25
@wikingi liked buuut11:25
@wikingnot with a nested loop11:25
@wiking;)11:25
@HeikoSwiking: I see :)11:25
@HeikoSwell, good to check this anyways11:25
@HeikoSSaurabh7: remember what I mean?11:25
Saurabh7HeikoS:  we need just te matrix in col major sotrage so we can access cols11:26
@HeikoSSaurabh7: yep11:26
@HeikoSok so it is that?11:26
Saurabh7yes when we tranpose the feature matrix11:26
Saurabh7first it was with loops11:26
Saurabh7now I have done with eigen11:27
@HeikoSSaurabh7: cool, all good then11:27
@HeikoSgreat :)11:27
Saurabh7but we have to pass it to methods :)11:27
Saurabh7the eigen matrix11:27
@HeikoSSaurabh7: you mean the column?11:27
@HeikoSSaurabh7: I dont think that is necessary11:27
@HeikoSwhy not just pass the SGMatrix11:28
@HeikoSand then create a map?11:28
@HeikoSinside the method11:28
Saurabh7then I have to transpose and store it11:28
@HeikoSno11:28
@HeikoSjust pass it11:28
@HeikoSthen create map11:28
@HeikoSand transpose the map11:28
@HeikoSits free11:28
Saurabh7yes the te transposed map has to be stored in another EigenMatrix11:28
Saurabh7or we access rows of a colmajor matrix11:29
@HeikoSSaurabh7: we are talking about two different things here11:29
@HeikoSSaurabh7: my point is: SGMatrix and EigenMatrix are both just wrappers around a memory pointer11:29
@HeikoSthey are 100% interchangable11:30
@HeikoSand can be created from one another11:30
@HeikoSso if you have an eigen3 matrix11:30
@HeikoSyou can just have an SGMatrix to the exact same memory11:30
@HeikoSwhere is the memory allocated for the matrix you pass?11:30
Saurabh7I create a eigen transpose at the start11:31
Saurabh7I can wrap a SGMatrix around that ?11:31
@HeikoSyes11:31
@HeikoSthe easiest way actually is11:31
@HeikoSrather than allocating the memory with eigen3, allocate it with shogun, then put an eigen map around it11:31
@HeikoSand then pass the sg matrix in the method11:32
@HeikoScan you show me a line?11:32
Saurabh7https://github.com/shogun-toolbox/shogun/pull/3243/files#diff-c0c82dce2edf38481495103b1ecc4174R14011:32
@HeikoSwhich line?11:33
@HeikoSI see11:33
Saurabh714011:33
@HeikoSok11:33
@HeikoSso here we have three objects11:33
@HeikoSX11:34
@HeikoSmap_Xr11:34
@HeikoSmap_X11:34
@HeikoSall of them point to the *same* memory11:34
@HeikoSso you can just pass X11:34
@HeikoSto the method11:34
@HeikoSand then create a map around it, transpose it11:34
@HeikoSthen you new map will be equal to map_X11:34
@HeikoSSaurabh7: thoughts?11:35
Saurabh7I remember doing things but it lead to some overhead11:35
Saurabh7I will check if its in stas11:35
Saurabh7stash11:35
Saurabh7lemme show numbers11:35
@HeikoSno wait11:36
@HeikoSbefore you benchmark differences11:36
@HeikoSit is important to really understand11:36
@HeikoSthere *cannot* be any overhead caused by this11:36
@HeikoSas these things all wrap the same memory11:36
@HeikoSif there is difference, it comes from something else11:36
Saurabh7so MatrixXd map_X = map_Xr.transpose(); if its same memory will it not be stored in row major way ?11:37
@HeikoSthe eigen3 map just pretends that you transposed the matrix, the memory behind it is the same11:38
@HeikoSI think there also is transposeInPlace()11:39
@HeikoSwhich actually changes the memory11:39
@HeikoSSaurabh7: I suggest you do a little mini example to check?11:39
@HeikoScreate an SGMatrix11:39
@HeikoScreate two maps on it11:39
Saurabh7so map_X = map_Xr.transpose() if called 1000 times it will be no overhead ?11:39
@HeikoSone of it, you transpose11:40
@HeikoSand then print the SGMatrix11:40
@HeikoSI think no11:40
@HeikoSthe tranpose will only take effect when you do an operation with the map11:40
@HeikoSsay a matrix multiply11:40
@HeikoSthis is adjusted11:40
@HeikoSso that the transpose is respected11:41
@HeikoSjust like the tranpose flags in cblas11:41
@HeikoSSaurabh7: but please, write a little program to verify all that11:41
@HeikoSmaybe also helps understanding11:41
sanujhey HeikoS11:45
sanujwhat's up :)11:45
@wiking++11:45
@HeikoSsanuj: jo11:46
@HeikoSsanuj: going through PRs :)11:46
@HeikoSthen disappearing, need to prepare a talk11:47
sanujokay11:47
sanujHeikoS, can you see my comments at the end of this PR https://github.com/shogun-toolbox/shogun/pull/325011:47
@HeikoSSaurabh7: think its good to do this example?11:47
@HeikoSsanuj: ye11:47
sanujthen i can push my changes accordingly11:47
Saurabh7HeikoS: yes doing :)11:47
@HeikoSSaurabh7: cool! :)11:47
@HeikoSSaurabh7: share the code and output after11:47
@HeikoSsanuj:11:49
sanujyes11:49
@HeikoSisnt it possible to just call ->init(train,test) on the combined kernel?11:50
@HeikoSwhy do we need to create a new object?11:50
sanujHeikoS, that's what i have said in the comments11:50
sanujHeikoS, if i do init() then it won't change the custom kernel11:50
@HeikoSah!11:50
sanujHeikoS, maybe we can remove the custom kernel... then it would be straight forward11:51
@HeikoScan you change the custom kernel then manually?11:51
@HeikoSno actually11:51
@HeikoSthen it is fine11:51
@HeikoSbut you should say that in the cookbook11:51
Saurabh7HeikoS: code : https://gist.github.com/Saurabh7/902878e25b0b24c2674e1ab1aacb1b2111:51
sanujHeikoS, i won't be able to change the custom kernel due to meta language restrictions11:51
Saurabh7output: https://gist.github.com/Saurabh7/e0e018db4da00af6f5229beebdb56f0111:52
@HeikoSsanuj: you can keep the original reference to it, no?11:52
@HeikoSand then call set_kernel_matrix11:52
@HeikoSon it?11:52
sanujHeikoS, i can delete and add a new custom kernel11:52
@HeikoSsanuj: no need to11:52
@HeikoSjust keep reference?11:52
sanujHeikoS, yes, that works in the python translation but not in cpp translation11:52
sanuji had tried that11:52
@HeikoScustom_kernel_train.set_kernel_matrix_from_full()11:53
@HeikoS(or similar name)11:53
sanujHeikoS, yes that only11:53
@HeikoSSaurabh7: whats the output?11:53
Saurabh7matrix new cannnot change old data11:53
@HeikoSI see11:53
@HeikoSsanuj: that only?11:54
@HeikoSwhat do you mean?11:54
sanujHeikoS, i used custom_kernel_train.set_kernel_matrix_from_full()11:54
@HeikoSbut?11:54
sanujHeikoS, it was able to set the matrix in python translation but not in cpp translation11:55
@HeikoSsanuj: you will have to explain why11:55
sanujHeikoS, okay, give me a few minutes11:56
@HeikoSsanuj: again, if you run into such things, please dont report that something doesnt work without giving the error11:56
@HeikoSmaybe there is an easy fix? :)11:56
@HeikoSmaybe not11:56
sanujokay11:58
sanuji was just telling you why i hadn't used the same combined kernel11:58
Saurabh7HeikoS: so we can assume now MatrixXd map_X = map_Xr.transpose() creates new memory ?11:58
sanujthe error that i was getting was "kernel matrix is not set"11:58
sanujbut i haven't really dived in what is causing it for cpp11:58
sanuji'll do it now11:59
@HeikoSSaurabh7: can you show me the output?12:00
Saurabh7HeikoS: https://gist.github.com/Saurabh7/e0e018db4da00af6f5229beebdb56f0112:00
Saurabh7i changed first element for both maps12:00
@HeikoSsanuj: python / cpp should behave the same12:00
sanujHeikoS, in the pr you just reviewed, the notebook diff is different due to the version difference12:01
sanujof ipython12:01
@HeikoSSaurabh7: ok so transpose allocates new memory if being assigned  to another variable12:01
@HeikoSSaurabh7: interesting12:01
@HeikoSwasnt aware of that, thanks for checking12:02
@HeikoSSaurabh7: ok then, the way to go then is: you allocate SGMatrix for the transposed memory yourself12:02
@HeikoSthen you create an eigen map12:02
@HeikoSand then you assign that map the transpose12:02
@HeikoSthen the SGMatrix contains the transposed mempry12:02
@HeikoSwhich you can pass around12:02
@HeikoSsanuj: ok thanks12:02
@HeikoSsanuj: annoying this ipython12:03
sanujyeah12:03
sanujbut don't merge it now12:03
@HeikoSsanuj: ok lets get down with this cpp/python error12:03
sanuji'll make the changes you suggested12:03
@HeikoSsanuj: yeah also need to get rid of the error12:03
@HeikoSsanuj: the error you are getting might be due to some older files floating around12:06
@HeikoStry removing the build dir12:06
@HeikoSit complains about members being protected so it cannot set them12:06
sanujHeikoS, yeah, i spent a lot of time trying to find the source of error12:06
@HeikoSso either your forgot to change some direct access things, or your swig output is outdated12:06
@HeikoSonly two possibilities12:07
@HeikoSdid you try removing build dir and re-building?12:07
sanujit seems it is outdated12:07
sanuji tried re-cmake and building12:07
sanujnow i'll try in a new build12:07
@HeikoSok12:07
@HeikoScool12:07
@HeikoSsanuj: btw12:08
sanujHeikoS, btw travis was passing, so i think it is because of my old build dir12:08
@HeikoSreally cool that you dig into all this stuff12:08
@HeikoSit is pretty messy and we appreciate your efforts a lot12:08
@HeikoSsanuj: ah there you go, travis for the rescue12:08
@HeikoSsanuj: let me know how the custom kernel stuff is going12:08
sanujHeikoS, thank you :)12:09
@HeikoSSaurabh7: did you get what I mean via pre-allocating using SGMatrix?12:09
@HeikoSand then passing that?12:09
@HeikoSsince I gotta go now12:09
@HeikoSarianepaola: hi!, any news on the setup.py, anything wiking can help you with?12:09
Saurabh7HeikoS: yes I will make changes12:09
@HeikoSSaurabh7: ok cool12:10
Saurabh7HeikoS: other than that what should be looking at12:10
@HeikoSSaurabh7: yeah I was curious about one thing in the numbers12:10
@HeikoS2000   500  0.339843    0.240674   2.52726e+09  2.52726e+09         167         16512:10
@HeikoSthi line12:10
@HeikoSsklearn is faster at the same number of iterations12:11
@HeikoSI wonder why12:11
Saurabh7No idea i have tried many things12:11
@HeikoSthe other thing is: is it possible to use the very same epsilon in the comparison?12:11
@HeikoSSaurabh7: ok12:11
@HeikoSits not a big deal for now12:11
Saurabh7HeikoS: I used the same epsilon12:12
@HeikoSsklearn uses so many more iterations....12:12
@HeikoSknow why?12:12
Saurabh7their was bigger which lead to good stopping12:12
@HeikoS"their was bigger"?12:13
Saurabh7HeikoS: I mean the epsilon value12:13
@HeikoS(11:10:34) Saurabh7: HeikoS: I used the same epsilon12:13
@HeikoS??12:13
Saurabh7we used 10-1612:13
@HeikoSI am confused now12:13
Saurabh7theirs was 10-712:13
Saurabh7I changed shoguns to 10-712:14
Saurabh7to compare12:14
@HeikoSok12:14
@HeikoSso it was the same12:14
Saurabh7sry i type weird sometimes12:14
Saurabh7yep12:14
@HeikoSthen same question: why does sklearn do more iterations?12:14
@HeikoSso many more?12:14
Saurabh7when N and D are close sklearn always does this12:15
sanujHeikoS, oh, so... mkl cookbook is working for both cpp and python12:15
@HeikoSsanuj: with updating the train kernel?12:15
sanuji must have done something wrong in the code when i had tried previously12:15
sanujbecause i wrote fresh code right now12:15
sanujHeikoS, yes12:15
@HeikoSsanuj: ok cool!12:15
@HeikoSsanuj: I will check PRs later today12:15
sanujHeikoS, okay12:15
@HeikoSSaurabh7: ok, weird12:15
@HeikoSSaurabh7: ok, now one thing to look at: LARS is also in mlpack right?12:16
Saurabh7HeikoS: yes12:16
@HeikoSSaurabh7: any idea how fast it is?12:16
Saurabh7not checked yet12:16
Saurabh7will do12:16
@HeikoSSaurabh7: cool12:17
@HeikoSSaurabh7: ah one more12:17
@HeikoSSaurabh7: does sklearn pre-compute the gram matrix?12:17
@HeikoSin lars?12:17
@HeikoSbecause it will be much faster then12:17
Saurabh7no I disabled it12:17
@HeikoSok12:17
@HeikoSif you enable it, is it much faster?12:17
@HeikoSand: can shogun precompute it?12:17
Saurabh7no shogun cannot12:17
@HeikoSto gain the same speedup if memory is available12:17
@HeikoSthink it is good to add that?12:17
Saurabh7yes might be good12:18
@HeikoSI am just looking at https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/least_angle.py12:18
@HeikoStheir code is not inline c12:18
@HeikoSit is python12:18
sanujHeikoS, what happened to "Math:init_random(1)" in gmm cookbook12:18
sanujit still fails12:18
sanujfor R12:18
@HeikoSsanuj: I think the integration testing data is out of sync with the listing btw12:18
@HeikoScpp examples fail on traivs12:18
@HeikoSsanuj: which link are you refering to?12:19
@HeikoSwill check12:19
Saurabh7HeikoS: yes I compared with that code12:19
Saurabh7almost everything is similar12:19
sanujHeikoS, i'm talking about this https://github.com/shogun-toolbox/shogun/pull/323912:19
@HeikoSSaurabh7: their solve_cholesky is from lapaclk12:19
@HeikoSSaurabh7: ok.....12:19
sanujHeikoS, it is bound to fail till we don't merge the above PR ^12:19
@HeikoSSaurabh7: maybe also profile it12:19
@HeikoSsanuj:  I see12:19
@HeikoSdid you rebase recently?12:20
sanujin the morning12:20
sanujHeikoS, shall i do it again12:20
@HeikoSsanuj: no12:20
@HeikoSsigh, seems like another error12:20
Saurabh7HeikoS: excet the difference that they do direct matrix operations L[:n_active, :n_active]12:20
Saurabh7HeikoS: like this, while we have to get the active columns and dot product12:20
sanujHeikoS, Error in Math$init_random : object of type 'closure' is not subsettable12:21
sanuj:/12:21
@HeikoSsanuj: I will check and fix12:22
@HeikoSSaurabh7: I see, so they copy the matrix12:22
sanujcool12:22
@HeikoSSaurabh7: and then do matrix operations12:22
@HeikoSSaurabh7: ?12:22
Saurabh7HeikoS: does that copy it ?12:22
@HeikoSSaurabh7: I dont know12:22
Saurabh7for example : eq_dir = np.dot(X.T[:n_active].T, least_squares)12:22
Saurabh7only dot with active columns12:23
@HeikoSwhat do we do?12:23
Saurabh7loop12:23
Saurabh7extract column12:23
Saurabh7dot12:23
@HeikoSI see12:23
@HeikoSwhy not try extracting all collumns12:23
@HeikoSand then do dot product with matrix?12:23
@HeikoSBUT12:23
@HeikoSdo it clever12:23
@HeikoShave a buffer matrix allocated at the beginning12:24
@HeikoSand then use memcpy to copy columns in there12:24
@HeikoSand then create a map that only contains the first few columns12:24
@HeikoS(while the buffer is bigger)12:24
@HeikoSso you dont have to re-allocate12:24
@HeikoSsee?12:24
Saurabh7oh12:24
Saurabh7so what about uninitialized things12:25
Saurabh7extra memory12:25
Saurabh7oh a eigen map12:25
Saurabh7okok can do that12:26
@HeikoSyeah you allocate a big buffer in the beginning12:26
@HeikoSthen fill only the first few columns12:26
@HeikoSand create a map that only uses these12:26
@HeikoSthen in next iteration, you copy another column in there and create new map12:26
@HeikoSSaurabh7: that might speed up things, might not12:26
Saurabh7yes i had tried with a new matrix in each iter12:27
Saurabh7it didnt work much12:27
Saurabh7lemme check this out12:27
@HeikoSSaurabh7: I think such things cannot be optimized by the compiler12:28
@HeikoSso it might help12:28
@HeikoSok I will be back later today12:28
@HeikoSsee you all12:28
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Remote host closed the connection]12:28
Saurabh7ok cool12:28
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun12:30
-!- mode/#shogun [+o HeikoS] by ChanServ12:30
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Remote host closed the connection]12:30
-!- lambday [8028b10a@gateway/web/freenode/ip.128.40.177.10] has joined #shogun12:48
-!- mode/#shogun [+o lambday] by ChanServ12:48
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun13:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * 0e5d0d8 / / (16 files): https://github.com/shogun-toolbox/shogun/commit/0e5d0d8602cc26b8ecd6455f1d944a6e1e0532cf13:08
shogun-notifier-shogun: working version13:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * aa577dd / / (27 files): https://github.com/shogun-toolbox/shogun/commit/aa577ddb9f86a4c343bc58b86be3f6872f24a46d13:08
shogun-notifier-shogun: set default cpubackend13:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * f6c018b / src/shogun/mathematics/linalgrefactor/ (5 files): https://github.com/shogun-toolbox/shogun/commit/f6c018bebf7e5951040b1b890a778625eda9815913:08
shogun-notifier-shogun: const13:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * 18540fb / src/shogun/mathematics/linalgrefactor/ (10 files): https://github.com/shogun-toolbox/shogun/commit/18540fbf59eb942acdfef5d6c22ad318f1e8e3da13:08
shogun-notifier-shogun: float3213:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * 177f5ea / / (20 files): https://github.com/shogun-toolbox/shogun/commit/177f5ea47890fe9a3211da6dd326bed86f42781513:08
shogun-notifier-shogun: fix style13:08
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * b73728f / src/shogun/mathematics/linalgrefactor/GPUBackend.cpp: https://github.com/shogun-toolbox/shogun/commit/b73728ffe91bccb779a0aca47d3ee0728728f11113:08
shogun-notifier-shogun: update13:08
-!- sanuj [~sanuj@117.204.252.181] has quit [Ping timeout: 260 seconds]13:10
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * 72aa846 / benchmarks/linalg_refactor_benchmark.cpp: https://github.com/shogun-toolbox/shogun/commit/72aa846f496b6ef088af75d1df33c671677741a513:24
shogun-notifier-shogun: benchmark update13:24
shogun-notifier-shogun: OXPHOS :feature/linalg_refactor * 1272d6e / benchmarks/linalg_benchmark.cpp,benchmarks/linalg_refactor_benchmark.cpp: https://github.com/shogun-toolbox/shogun/commit/1272d6ebf9731ade5fd781ddf5aef92bf54a71d513:24
shogun-notifier-shogun: update13:24
shogun-notifier-shogun: Soumyajit De :feature/linalg_refactor * 254a83f / benchmarks/linalg_benchmark.cpp,benchmarks/linalg_refactor_benchmark.cpp: https://github.com/shogun-toolbox/shogun/commit/254a83f05b332bf2998b18b73d8b02e4a147fa5a13:24
shogun-notifier-shogun: Merge pull request #3252 from OXPHOS/linalg_benchmark13:24
shogun-notifier-shogun:13:24
shogun-notifier-shogun: linalg benchmark update13:24
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has joined #shogun13:57
travis-ciit's Soumyajit De's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/13639905213:57
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has left #shogun []13:57
arianepaolahello everyone14:32
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has joined #shogun14:38
travis-ciit's Soumyajit De's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/13640222314:38
-!- travis-ci [~travis-ci@ec2-54-205-13-204.compute-1.amazonaws.com] has left #shogun []14:38
-!- besser82 [~besser82@fedora/besser82] has joined #shogun14:39
-!- mode/#shogun [+o besser82] by ChanServ14:39
arianepaolawiking: ping14:41
@wikingpong14:42
arianepaolawiking: it looks like setuptools is very bad at handling anything that is not Python related, especially binary packages14:44
arianepaolaso some very strange things14:44
arianepaolaI get to build rpm source packages of the tree, independent of what I specify as source package, when running sdist14:44
arianepaolawhen I use a manifest, it is ignored by setuptools14:45
arianepaolawhen I switch back to use distutils, it uses the MANIFEST.in only for sources that are under version control14:45
arianepaolaso basically everything in the output is ignored14:46
arianepaolaI tried package_data vs data_files on both distutils and setuptools14:46
arianepaolatried auto generated and filling the package_data or data_files dict with path + file combinations from build/install output of shogun14:46
@wikingmmm you are trying to add the .so-s as additional fixtures?14:48
arianepaolabtw, it also makes a difference if you use include_package_data=True, causing either one of distutils or setuptools to break14:48
arianepaolait doesn't add anything in the dist14:49
arianepaola:-(14:49
arianepaolaHeikoS: I saw your message14:53
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 260 seconds]15:02
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-aizjrbmwcdgxpctw] has quit [Ping timeout: 264 seconds]15:12
-!- sanuj [~sanuj@117.204.249.60] has joined #shogun15:21
-!- lambday [8028b10a@gateway/web/freenode/ip.128.40.177.10] has quit []15:25
arianepaolawiking: any idea?16:07
@wikingarianepaola: i'll get back asap16:10
@wikingjust need some time to finish up my own personal stuff16:10
arianepaolaok, sure16:10
arianepaolano problem16:10
arianepaolaIn the meanwhile, I got apparently libshogun.so included using glob.glob16:10
arianepaolaseems that somehow auto generated it got added16:11
-!- OXPHOS [9d8b1501@gateway/web/freenode/ip.157.139.21.1] has joined #shogun16:17
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]16:24
-!- mizari [~mizari@95-174-213-100.nts.su] has joined #shogun16:25
arianepaolahi OXPHOS16:36
-!- lupinix [~quassel@fedora/lupinix] has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]16:47
-!- lupinix [~quassel@v22014041761818086.yourvserver.net] has joined #shogun16:47
-!- lupinix [~quassel@v22014041761818086.yourvserver.net] has quit [Changing host]16:47
-!- lupinix [~quassel@fedora/lupinix] has joined #shogun16:47
sanujneural net notebooks take forever to run17:29
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun17:37
-!- mode/#shogun [+o HeikoS] by ChanServ17:37
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Client Quit]17:37
arianepaolawiking: when you have time, check this gist https://gist.github.com/anonymous/c3dfeb456656dd3b468fd7bc91b0205818:15
arianepaolawiking: the setup.py in the PR overloads functionality from distutils to bootstrap the cmake configuration process and the compilation18:17
arianepaolawiking: the gist just deals with the Python related content in lib. you can test it building shogun manually and then in the build dir: python setup.py bdist18:18
arianepaola^^ HeikoS: if you want to check also18:18
-!- mizari [~mizari@95-174-213-100.nts.su] has quit [Ping timeout: 246 seconds]19:08
-!- besser82 [~besser82@fedora/besser82] has joined #shogun19:09
-!- mode/#shogun [+o besser82] by ChanServ19:09
sanujhey lisitsyn19:20
lisitsynрун19:20
lisitsynhey19:20
sanujlisitsyn, so i think i should start with swig for tags19:20
lisitsynsanuj: alright19:24
lisitsynagree19:24
sanujlisitsyn, i don't know where to start19:25
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 272 seconds]19:25
sanujlike in what files shall i make changes?19:26
sanujin src/interfaces?19:26
lisitsynyes19:26
sanujlisitsyn, there is modular19:27
sanujand there are python_modular, r_modular and ....19:27
sanujlisitsyn, which ones to touch?19:28
lisitsynsanuj: src/interfaces/modular19:29
sanujlisitsyn, sgbase.i?19:29
lisitsynsanuj: modshogun.i is the entry point19:31
lisitsynit includes other ones19:31
sanujlisitsyn, okay19:32
sanuji'll do this tomorrow19:32
sanujgoing to sleep now19:32
lisitsynsanuj: I think SGBase.i is the way to go19:32
lisitsynok19:32
sanujokay19:32
lisitsynyou just need to add some renames19:32
lisitsynlike IntTag19:32
lisitsynIntType19:32
sanujokay, then it should be easy19:32
lisitsynand check if it works in python19:32
sanujlisitsyn, then shall i write meta examples for tags19:32
sanujfor integration testing or something19:33
lisitsynI think we don't have to use meta now19:33
lisitsynnot sure19:33
lisitsynok let me think19:33
sanujokay19:33
lisitsynlets discuss that tomorrow19:33
sanujlisitsyn, okay, i'll try to make this work with python19:33
arianepaolalisitsyn: hi, I have a question regarding src/shogun/lib/versionstring.h19:34
sanujlisitsyn, what time will you be ready to discuss tomorrow?19:34
arianepaolalisitsyn: is there a way to check if we are using git source? e.g. will the version change for a Release version?19:35
arianepaolaI want only to append the info from #define VERSION_REVISION 0x83970a7 when it is a development version19:36
-!- besser82 [~besser82@fedora/besser82] has joined #shogun19:37
-!- mode/#shogun [+o besser82] by ChanServ19:37
-!- sanuj [~sanuj@117.204.249.60] has quit [Quit: Leaving]19:40
lisitsynsanuj: mostly any but ping me19:40
lisitsynarianepaola: let me check19:40
arianepaolathanks lisitsyn19:40
-!- besser82 [~besser82@fedora/besser82] has quit [Remote host closed the connection]20:02
-!- besser82 [~besser82@fedora/besser82] has joined #shogun20:03
-!- mode/#shogun [+o besser82] by ChanServ20:03
-!- besser82 [~besser82@fedora/besser82] has quit [Remote host closed the connection]20:04
-!- besser82 [~besser82@fedora/besser82] has joined #shogun20:05
-!- mode/#shogun [+o besser82] by ChanServ20:05
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 260 seconds]20:24
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-nhgboshmvjngqjid] has joined #shogun21:22
arianepaolabye everyone, see you all tomorrow :-)22:07
-!- lambday [6d9941c0@gateway/web/freenode/ip.109.153.65.192] has joined #shogun22:11
-!- mode/#shogun [+o lambday] by ChanServ22:11
@lambdayOXPHOS: yo23:36
@lambdayOXPHOS: how's it going?23:36
OXPHOSlambday: hey!23:37
@lambdayOXPHOS: just going through your patch23:37
@lambdayOXPHOS: so this is supposed to be the factory method that we discussed about :)23:38
OXPHOSlambday: thanks! yes.23:38
@lambdayOXPHOS: alright.. let me comment on the patch23:38
@lambdaylooking nice already :)23:38
OXPHOSlambday: thx :) should be making progress23:39
@lambdayOXPHOS: yeah.. don't worry.. the initial bit is the toughest.. once you get the momentum then it's all smooth ride :)23:39
OXPHOSlambday: i'm still wondering, for some template methods without #ifdef, like CPUBackend.dot(), why don't we put the implementation in .h?23:40
OXPHOSseems it's gonna be faster like that23:40
@lambdayOXPHOS: well, when used in actual shogun algorithm, it won't make much difference23:43
@lambdayOXPHOS: if you wonder about this, check the other benchmark that I made : https://github.com/lambday/derby/tree/master/benchmarks/headervsimpl23:44
OXPHOSlambday: but is there any harm to put the stuff in .h?23:44
@lambdayOXPHOS: well, imagine you have a client.. and he built some code that relies on the shogun.. how if shogun devs later decides to use another cool linalg library instead of eigen, then all his code will fail if the headers are changed..23:46
@lambdaybut if that is in the cpp, then he doesn't have that problem.. he just updates his libshogun.so and he's good to go..23:47
@lambdaythat's the reason things better be in the cpp23:47
OXPHOSlambday: cool..I got it. thanks for the explain.23:48
@lambdayand that's the main motivation behind using pimpl pattern as well23:48
@lambdayideally, even the cpubackend should be implemented as pimpl23:48
@lambdaybut let's not worry about that now23:48
OXPHOSlambday: right right. yes it won't be hard to refactor23:48
OXPHOSlambday: I made up that GPU_CALC stuff and now I feel like I can just use HAVE_VIENNACL23:49
@lambdayOXPHOS: actually what I think would be best is this : get_gpu_type() would return a GPU type if a GPU backend is registered.. (just like you did there), but if it's not, it should return a cpu type instead..23:49
@lambdaywith a friendly SG_SINFO23:50
@lambday:)23:50
@lambdayso the computation will go on.. nothing fails23:50
@lambdaybut the users get a message about that23:50
@lambdayso maybe no GPU_CALC is required23:51
OXPHOSlambday: u don't want to slap the users ? ;)23:51
@lambdayOXPHOS: nope.. it's shogun's internal code.. the user don't care what we use inside.. as long as things work!23:52
@lambdaythe shogun-devs will use sg_linalg->get_gpu_type() as a more of an abstract design principle23:52
@lambdayfor algorithms that are gpu friendly23:53
@lambdaybut it still should be doable on cpu if the poor user doesn't have a gpu thingi on his machine23:53
@lambdayOXPHOS: well, I messed up something in the description of the problem of having everything in the header..23:55
@lambdayOXPHOS: what would happen is, the user would have to recompile his application..23:55
@lambdayif headers are changed23:55
@lambday(which may or may not lead to failure)23:56
OXPHOSlambday: and it would save time for devs if only implemetation is .cpp is change?23:56
OXPHOSd23:56
@lambdayOXPHOS: well, for the user, yes23:57
OXPHOSkk23:57
@lambdayimagine he built something on top of shogun and he sold his product to another customer23:57
@lambdaynow that customer updates shogun lib from repo and bam! nothing works anymore :D23:57
OXPHOShaha23:57
OXPHOSI'm thinking if Shogun enabled calculation with GPULibrary 1 and GPULibrary 2. We'll have (GPULIB1_FOUND AND ENABLE_GPULIB1) to define HAVE_GPULIB123:59
--- Log closed Fri Jun 10 00:00:03 2016
« Wednesday, 2016-06-08 Index Friday, 2016-06-10 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!