--- Log opened Wed Jun 15 00:00:40 2016 | ||
-!- lambday [0551433c@gateway/web/freenode/ip.5.81.67.60] has joined #shogun | 00:29 | |
-!- mode/#shogun [+o lambday] by ChanServ | 00:29 | |
@lambday | OXPHOS: yo | 00:29 |
---|---|---|
lisitsyn | bazdmeg! | 00:30 |
lisitsyn | daily bazdmeg time | 00:30 |
@lambday | lisitsyn: what's that? | 00:31 |
lisitsyn | lambday: wiking taught me some curses | 00:32 |
@lambday | lisitsyn: haha | 00:32 |
OXPHOS | lambday: whaaaaaat | 00:57 |
OXPHOS | lambday: who are you | 00:57 |
@lambday | OXPHOS: Kal-El | 00:58 |
@lambday | OXPHOS: checking your patch.. looks pretty good now :) | 00:59 |
@lambday | just a few minor issues, commenting | 00:59 |
OXPHOS | lambday: thx..late night work? | 01:00 |
@lambday | OXPHOS: won't be awake for that long | 01:01 |
@lambday | lisitsyn: awake? | 01:02 |
@lambday | wiking: ? | 01:03 |
lisitsyn | lambday: on my way to bed | 01:03 |
lisitsyn | just falling doooownn | 01:03 |
OXPHOS | lambday: Does coding reading promotes sleeping XD | 01:03 |
@lambday | okay | 01:03 |
lisitsyn | slloowww | 01:03 |
lisitsyn | what'su p | 01:03 |
@lambday | lisitsyn: maybe have a quick look at OXPHOS's patch? https://github.com/shogun-toolbox/shogun/pull/3277 | 01:03 |
lisitsyn | lambday: can check morning | 01:03 |
lisitsyn | if get some time | 01:03 |
@lambday | lisitsyn: coding reading will help you sleep better, as OXPHOS says :D | 01:03 |
@lambday | lisitsyn: okay thanks :) | 01:04 |
OXPHOS | haha. turned down | 01:04 |
@lambday | OXPHOS: yeah, story of my life! :D | 01:04 |
OXPHOS | lambday: ohhhhhhhhh so sad | 01:04 |
@lambday | OXPHOS: BTW did I answer your question about the transfer to cpu thingi? | 01:05 |
OXPHOS | lambday: sort of..so we have to assume the user won't switch backend/or we need a restriction? | 01:06 |
OXPHOS | I mean the dot onGPU() can be changed easily | 01:06 |
OXPHOS | I was also worried, one user transfer data to GPU with ViennaCL, and then he switched to CUDA and try to transfer it back to CPU with CUDA | 01:07 |
@lambday | OXPHOS: well, we have to assume something! if data is on GPU and user registered a GPU backend, then it is a fair assumption that operations are performed on GPU data | 01:07 |
@lambday | OXPHOS: haha let's not worry about that now | 01:07 |
OXPHOS | kk.. | 01:08 |
@lambday | OXPHOS: I am not entirely sure whether multiple gpu backend support will happen/is required anytime soon | 01:08 |
OXPHOS | lambday: also currently we cannot destroy GPU without destroying the whole Vector. So if the user decides to go GPU, he has to stick to it or construct a new Vector | 01:08 |
OXPHOS | just add a method to reset GPUptr? | 01:09 |
@lambday | OXPHOS: well, transfer to cpu can do that, no? | 01:09 |
OXPHOS | lambday: I removed to reset part.. | 01:09 |
OXPHOS | in case someone just wants to get a SGVector but also leave this Vector intact? | 01:10 |
OXPHOS | feel like they should be separate | 01:10 |
@lambday | OXPHOS: okay makes sense.. | 01:10 |
@lambday | a release gpu memory method | 01:11 |
OXPHOS | yep | 01:11 |
@lambday | cool! | 01:11 |
OXPHOS | and actually SG_FREE doesn't work in shared_ptr(SG_MALLOC(), ) | 01:11 |
OXPHOS | because it has to take arguments | 01:11 |
OXPHOS | but let me pull out the error message | 01:12 |
@lambday | OXPHOS: you can pass it a lambda | 01:12 |
@lambday | so it would be something like shared_ptr(SG_MALLOC(....), [](T* ptr) { SG_FREE(ptr); }); | 01:12 |
OXPHOS | I can try | 01:13 |
OXPHOS | BTW until last week I thought your name was lamb day for some reason | 01:13 |
@lambday | OXPHOS: I've been called worse :D | 01:13 |
OXPHOS | sad night ;( | 01:14 |
@lambday | OXPHOS: so it's not that bad :D | 01:14 |
OXPHOS | haha | 01:14 |
@lambday | OXPHOS: actually my first mail to shogun community in my first GSoC was just this : hey people, I am lambda y (not lamb day) | 01:14 |
OXPHOS | lambday: so once data is onGPU, CPU part is "masked" | 01:15 |
@lambday | OXPHOS: I think it should be fine.. | 01:15 |
OXPHOS | lambday: I must learn from the mail | 01:15 |
OXPHOS | kk | 01:15 |
OXPHOS | lambday: oh you mean your first GSoC lol. Guess you'll have to state that everytime | 01:16 |
@lambday | haha | 01:16 |
@lambday | OXPHOS: alright.. off to sleep | 01:17 |
@lambday | OXPHOS: please update the PR.. we want to merge it soon | 01:18 |
@lambday | OXPHOS: and make sure you get it reviewed by wiking or lisitsyn or HeikoS also | 01:18 |
OXPHOS | lambday: sure I'll chase them. good night! | 01:18 |
@lambday | OXPHOS: night night | 01:19 |
-!- lambday [0551433c@gateway/web/freenode/ip.5.81.67.60] has quit [Quit: Leaving.] | 01:19 | |
-!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has quit [Quit: Page closed] | 03:11 | |
-!- GandalfTheWizard [~Eva@112.10.170.90] has joined #shogun | 03:12 | |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has left #shogun [] | 03:19 | |
-!- sanuj [~sanuj@117.203.14.130] has joined #shogun | 06:06 | |
sanuj | what happened to buildbot? | 06:07 |
Saurabh7 | ye something was wrong last night too | 06:10 |
sanuj | buildbot be like: The service is not available. Please try again later. | 06:11 |
Saurabh7 | its on aws now right , dunno wasup | 06:12 |
-!- sanuj [~sanuj@117.203.14.130] has quit [Ping timeout: 276 seconds] | 06:31 | |
@wiking | hehe no buldbot for some time now | 07:44 |
@wiking | (under progress to fix it) | 07:44 |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 08:01 | |
-!- sanuj [~sanuj@117.203.14.130] has joined #shogun | 08:19 | |
shogun-buildbot | build #54 of xenial - libshogun is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/xenial%20-%20libshogun/builds/54 blamelist: Viktor Gal <vigsterkr@gmail.com>, Saurabh7 <saurabh.mahindre@gmail.com> | 08:36 |
shogun-buildbot | build #17 of clang - undefined behaviour analysis is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/clang%20-%20undefined%20behaviour%20analysis/builds/17 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Viktor Gal <vigsterkr@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com>, Saurabh7 | 08:46 |
shogun-buildbot | <saurabh.mahindre@gmail.com> | 08:46 |
-!- sanuj [~sanuj@117.203.14.130] has quit [Remote host closed the connection] | 08:48 | |
shogun-buildbot | build #0 of FC23 - libshogun - aarch64 is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/FC23%20-%20libshogun%20-%20aarch64/builds/0 blamelist: Viktor Gal <vigsterkr@gmail.com>, Saurabh7 <saurabh.mahindre@gmail.com> | 08:50 |
-!- GandalfTheWizard [~Eva@112.10.170.90] has quit [Quit: Leaving.] | 09:02 | |
@wiking | shogun-buildbot: force build --branch=develop 'CentOS7 - libshogun' | 09:32 |
shogun-buildbot | The build has been queued, I'll give a shout when it starts | 09:32 |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Remote host closed the connection] | 09:32 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 09:33 | |
shogun-buildbot | build #1021 of nightly_none is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_none/builds/1021 blamelist: Viktor Gal <vigsterkr@gmail.com>, Saurabh7 <saurabh.mahindre@gmail.com> | 09:42 |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has joined #shogun | 09:48 | |
@wiking | shogun-buildbot: dance | 10:07 |
shogun-buildbot | <(^.^<) | 10:07 |
shogun-buildbot | <(^.^)> | 10:07 |
shogun-buildbot | (>^.^)> | 10:07 |
shogun-buildbot | (7^.^)7 | 10:07 |
shogun-buildbot | (>^.^<) | 10:07 |
-!- besser82_ [~besser82@fedora/besser82] has joined #shogun | 10:16 | |
-!- mode/#shogun [+o besser82_] by ChanServ | 10:16 | |
shogun-buildbot | build #1151 of nightly_default is complete: Failure [failed test notebooks] Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/1151 blamelist: Viktor Gal <vigsterkr@gmail.com>, Saurabh7 <saurabh.mahindre@gmail.com> | 10:32 |
@wiking | Saurabh7: ping | 10:36 |
@wiking | Saurabh7: https://github.com/shogun-toolbox/shogun/pull/3288 | 10:37 |
@wiking | see the comment of Heiko | 10:37 |
Saurabh7 | hm no test for it atm | 10:59 |
Saurabh7 | but I will have to compare against some constant values then | 11:00 |
Saurabh7 | let me add one | 11:00 |
Saurabh7 | wiking: should I push the test in PR or ? | 11:18 |
Saurabh7 | in the same PR i mean | 11:22 |
-!- besser82_ [~besser82@fedora/besser82] has quit [Ping timeout: 250 seconds] | 11:25 | |
Saurabh7 | brb | 11:30 |
@wiking | Saurabh7: same same :) | 11:44 |
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 12:15 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 12:15 | |
@wiking | HeikoS: so we got buildbot back :) | 12:15 |
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.] | 12:44 | |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has quit [Excess Flood] | 13:14 | |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has joined #shogun | 13:16 | |
@wiking | arianepaola: lets merge that pr | 13:36 |
-!- sanuj [~sanuj@117.203.14.130] has joined #shogun | 13:36 | |
sanuj | lisitsyn, hey | 13:36 |
lisitsyn | sanuj: hey | 13:36 |
@wiking | arianepaola: oh wait, can you just amend that commit with the description that Heiko referred to: https://github.com/shogun-toolbox/shogun/wiki/Project-description | 13:36 |
@wiking | plz | 13:36 |
@wiking | ping me when its done | 13:37 |
@wiking | thnx | 13:37 |
sanuj | lisitsyn, did you see my email? | 13:37 |
lisitsyn | sanuj: yeah but got no time yet to check | 13:37 |
sanuj | okay | 13:37 |
sanuj | i'll start working on the unit-tests for now | 13:37 |
lisitsyn | okay | 13:37 |
sanuj | wiking, you were talking about something we have to check coverage of code.... | 13:49 |
@wiking | peeeeeeeeeeeeeeople | 13:55 |
@wiking | we have a bug | 13:55 |
@wiking | http://buildbot.shogun-toolbox.org/builders/deb4%20-%20python3/builds/882/steps/install/logs/stdio | 13:55 |
sanuj | ooooooh | 13:55 |
@wiking | it's from Heiko actually :) | 13:56 |
lisitsyn | OLD PyTHon! | 13:56 |
@wiking | yep yep | 13:57 |
-!- HeikoS [~heiko@nat-201-233.internal.eduroam.ucl.ac.uk] has joined #shogun | 14:06 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 14:06 | |
@wiking | HeikoS: bugzi | 14:09 |
@HeikoS | wiking: jo | 14:09 |
@HeikoS | wiking: nice one on the buildbot | 14:09 |
@HeikoS | so I can merge more cookbooks later | 14:09 |
@wiking | yeah and he found you the bug you've created for python3 ;) | 14:09 |
@HeikoS | wiking: ah nice | 14:09 |
@HeikoS | checking | 14:10 |
@HeikoS | arianepaola: ping | 14:11 |
@HeikoS | arianepaola: can you pls fix this: http://buildbot.shogun-toolbox.org/builders/deb4%20-%20python3/builds/881/steps/install/logs/stdio | 14:11 |
@HeikoS | arianepaola: i explicitly asked whether this is python 2 and 3 compatible, remember? :) | 14:11 |
@HeikoS | wiking: ok lets hope that is solved soon, I have to do some work now, will check back later | 14:13 |
-!- [Chris] [~Chris]@interferon.am10.uni-tuebingen.de] has joined #shogun | 14:15 | |
[Chris] | hi, I have a question regarding combined kernels and cross validation. How should I specify the features for CrossValidation if I'm using a SVM with a combined kernel consisting of two rbf kernels with different feature encodings? Is there some example code on that ? | 14:17 |
arianepaola | hello everyone | 14:26 |
arianepaola | hi wiking, I saw your message in the logs | 14:32 |
arianepaola | wiking: I will add the description | 14:32 |
arianepaola | HeikoS: Python3... I will update it | 14:33 |
@HeikoS | arianepaola: hi | 14:33 |
@HeikoS | thanks | 14:33 |
@HeikoS | whenever you write python code for shogun, it needs to work under both 2/3, like I said in the PR | 14:33 |
arianepaola | I must have changed the interpreter | 14:34 |
arianepaola | does it need to be < Python 2.6 | 14:34 |
@HeikoS | no 2.7 is fine | 14:34 |
arianepaola | or is Python >= 2.6 ok | 14:34 |
arianepaola | ok | 14:34 |
sanuj | wiking, i built shogun with ENABLE_COVERAGE=ON | 14:38 |
sanuj | now how to see coverage? | 14:38 |
-!- c4goldsw [5da420e6@gateway/web/cgi-irc/kiwiirc.com/ip.93.164.32.230] has joined #shogun | 14:40 | |
c4goldsw | Hello | 14:40 |
c4goldsw | HeikoS Hey, how are you? | 14:43 |
@HeikoS | c4goldsw: jo"! | 14:44 |
@HeikoS | pretty good | 14:44 |
c4goldsw | I saw your comments on my PR - aside from what you listed, I think it's fine. | 14:44 |
c4goldsw | But, its not being included b/c Pan is doing linalg, right? | 14:44 |
c4goldsw | If that's the case, I'll just close it. | 14:45 |
c4goldsw | HeikoS ^ | 14:51 |
@HeikoS | c4goldsw: sorry was in a call | 14:52 |
@HeikoS | ok cool | 14:52 |
@HeikoS | I mean pans stuff will refactor the very basics of linalg | 14:52 |
@HeikoS | but the operation will still be needed | 14:52 |
@HeikoS | as well as the unit tests etc | 14:52 |
@HeikoS | they will just be refactored a bit | 14:52 |
c4goldsw | Fair enough - I'll still close it anyways. | 14:53 |
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 15:07 | |
shogun-notifier- | shogun: Ariane Paola Gomes :develop * 5a6f847 / examples/meta/generator/generate.py: https://github.com/shogun-toolbox/shogun/commit/5a6f8474b3c09d52911ecc9a5a520923738cbe7c | 15:07 |
shogun-notifier- | shogun: Fix Python 2 to 3 exception conversion. | 15:07 |
shogun-notifier- | shogun: Viktor Gal :develop * cd345d1 / examples/meta/generator/generate.py: https://github.com/shogun-toolbox/shogun/commit/cd345d15ca03819c8a62c013d51fb66d9f556e7e | 15:07 |
shogun-notifier- | shogun: Merge pull request #3291 from arianepaola/fix_2to3 | 15:07 |
shogun-notifier- | shogun: | 15:07 |
shogun-notifier- | shogun: Fix Python 2 to 3 exception conversion. | 15:07 |
arianepaola | ^^ HeikoS | 15:07 |
@wiking | HeikoS: puuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuub | 15:07 |
arianepaola | ^^ wiking setup.py is ready to merge | 15:07 |
shogun-buildbot | build #718 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/718 blamelist: Ariane Paola Gomes <arianepaola@users.noreply.github.com> | 15:07 |
@wiking | why trusty faaaails fuck | 15:08 |
@HeikoS | thx :) | 15:08 |
arianepaola | shogun-buildbot: dance | 15:08 |
shogun-buildbot | <(^.^<) | 15:08 |
shogun-buildbot | <(^.^)> | 15:08 |
shogun-buildbot | (>^.^)> | 15:08 |
shogun-buildbot | (7^.^)7 | 15:08 |
shogun-buildbot | (>^.^<) | 15:08 |
arianepaola | hahaha | 15:08 |
shogun-buildbot | build #719 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/719 blamelist: Viktor Gal <vigsterkr@gmail.com> | 15:08 |
@wiking | mmmm this is weirdo http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/719/steps/compile/logs/stdio | 15:10 |
@wiking | lars_unittest.cc | 15:11 |
@wiking | wtf? | 15:11 |
@wiking | why did it started to error now?! | 15:11 |
arianepaola | it had a bad day | 15:11 |
arianepaola | lol | 15:11 |
@wiking | shogun-buildbot: force build --branch=develop 'deb4 - python3' | 15:11 |
shogun-buildbot | build #883 forced | 15:11 |
shogun-buildbot | I'll give a shout when the build finishes | 15:11 |
arianepaola | probably some teenager growing up stuff | 15:11 |
arianepaola | rebellion, etc. | 15:12 |
arianepaola | starting to break everything on buildbot now | 15:12 |
@wiking | shogun-buildbot: force build --branch=develop 'trusty - libshogun - viennacl' | 15:12 |
shogun-buildbot | build #720 forced | 15:12 |
shogun-buildbot | I'll give a shout when the build finishes | 15:12 |
@wiking | fuck | 15:12 |
@wiking | lets see now | 15:12 |
arianepaola | wiking: when using -DPythonModular=ON and release build it crashes | 15:13 |
shogun-buildbot | build #720 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/720 | 15:13 |
@wiking | :D | 15:13 |
@wiking | nice | 15:13 |
@wiking | when was this unit test changed? | 15:17 |
shogun-buildbot | build #883 of deb4 - python3 is complete: Success [build successful] Build details are at http://buildbot.shogun-toolbox.org/builders/deb4%20-%20python3/builds/883 | 15:19 |
@wiking | ok cool arianepaola \o/ fixed | 15:20 |
@wiking | HeikoS: ping ping | 15:20 |
arianepaola | wiking: good | 15:20 |
@wiking | mmm how do we add a package to pypi | 15:20 |
@wiking | ? | 15:20 |
arianepaola | wiking: setup.py PR is updated | 15:20 |
@wiking | arianepaola: yeah i'm just wanna check if HeikoS is ok with it | 15:21 |
@wiking | arghfuck | 15:21 |
@wiking | HeikoS: LETS FUCKING START DOING WHAT WE AGREED ON | 15:21 |
arianepaola | wiking: python setup.;why register | 15:21 |
@wiking | HeikoS: https://github.com/shogun-toolbox/shogun/pull/3240#issuecomment-226180683 | 15:21 |
@wiking | so i added my +1 | 15:21 |
@HeikoS | wiking: whats the matter? | 15:21 |
arianepaola | http://python-packaging.readthedocs.io/en/latest/minimal.html -> Publishing On PyPI | 15:21 |
@wiking | remember? :D | 15:21 |
@wiking | only merge when we have 2 +1s | 15:21 |
@wiking | :> | 15:22 |
@HeikoS | I am lost as in a call | 15:22 |
@HeikoS | give me some context | 15:22 |
@wiking | heheh so | 15:22 |
@wiking | remember weve agreed to only merge | 15:22 |
@wiking | a gsoc pr | 15:22 |
@HeikoS | which one? | 15:22 |
@wiking | when there are 2 +1s on | 15:22 |
@wiking | the pr from core contributers | 15:22 |
@wiking | this pr https://github.com/shogun-toolbox/shogun/pull/3240#issuecomment-226180683 | 15:22 |
@wiking | so i just need your +1 :D | 15:22 |
@HeikoS | ok checking | 15:22 |
@wiking | HeikoS: my question is about description only | 15:23 |
@wiking | if you are fine | 15:23 |
@wiking | with that | 15:23 |
@HeikoS | description | 15:23 |
@HeikoS | https://github.com/shogun-toolbox/shogun/pull/3240/files#diff-2eeaed663bd0d25b7e608891384b7298R296 | 15:23 |
@HeikoS | =? | 15:23 |
@wiking | Readme.md | 15:23 |
@wiking | +with open(os.path.join(setup_py_location, 'README.md')) as f: | 15:24 |
@wiking | + readme = f.read() | 15:24 |
@HeikoS | ah yes | 15:24 |
@HeikoS | good | 15:24 |
@HeikoS | BUT | 15:24 |
arianepaola | wiking: I think HeikoS means https://github.com/shogun-toolbox/shogun/pull/3240/files#diff-2eeaed663bd0d25b7e608891384b7298R317 | 15:24 |
@HeikoS | we should update the readme | 15:24 |
@HeikoS | guys I am lost sorry | 15:24 |
@HeikoS | half in a call | 15:24 |
@HeikoS | what exactly is the question? | 15:24 |
@wiking | ah but ok | 15:24 |
@wiking | so you wanna change some stuff in README.md | 15:24 |
@HeikoS | wiking: yes | 15:25 |
@wiking | that's ok but that should be changed | 15:25 |
@wiking | in another pr | 15:25 |
@wiking | not here | 15:25 |
@HeikoS | I want to put in the description from the wiki | 15:25 |
@HeikoS | yes | 15:25 |
@HeikoS | +1 from my side for the pr | 15:25 |
arianepaola | is this line ok? https://github.com/shogun-toolbox/shogun/pull/3240/files#diff-2eeaed663bd0d25b7e608891384b7298R317 | 15:25 |
@HeikoS | yes | 15:25 |
@wiking | put the +1 | 15:26 |
@wiking | :D | 15:26 |
@HeikoS | done | 15:26 |
@HeikoS | in text form :) | 15:26 |
@wiking | ahahha | 15:26 |
arianepaola | btw, we can add Operating System :: Microsoft :: Windows :: Windows 3.1 or Earlier | 15:26 |
arianepaola | and Operating System :: Microsoft :: MS-DOS | 15:26 |
arianepaola | rofl | 15:26 |
@wiking | heheh no | 15:26 |
@HeikoS | is that on our buildbot? ;) | 15:26 |
@wiking | we dont support hat | 15:26 |
@wiking | *that | 15:26 |
@wiking | anyhow merging | 15:26 |
@wiking | and lets fucking register ourselves into pypi | 15:26 |
arianepaola | thanks wiking | 15:26 |
shogun-notifier- | shogun: Ariane Paola Gomes :develop * 2151b9c / setup.py: https://github.com/shogun-toolbox/shogun/commit/2151b9c8cce2aca068a421460e580d8c5d398293 | 15:27 |
shogun-notifier- | shogun: Added Shogun Python package generation using setup.py. | 15:27 |
shogun-notifier- | shogun: Viktor Gal :develop * 3998da8 / setup.py: https://github.com/shogun-toolbox/shogun/commit/3998da842f988a6cc816ef4b225241c81da18591 | 15:27 |
shogun-notifier- | shogun: Merge pull request #3240 from arianepaola/feature/setup_py | 15:27 |
shogun-notifier- | shogun: | 15:27 |
shogun-notifier- | shogun: Shogun Python package using setup.py | 15:27 |
@HeikoS | whoooo :) | 15:27 |
@HeikoS | congrats | 15:27 |
@HeikoS | so now lets install on all machine on buildbot :) | 15:27 |
@wiking | :D | 15:27 |
@wiking | ah that will not happen | 15:27 |
@wiking | :> | 15:27 |
sanuj | HeikoS, i had updated this https://github.com/shogun-toolbox/shogun/pull/3281 | 15:27 |
@wiking | registering | 15:27 |
@wiking | :) | 15:27 |
@HeikoS | sanuj: thanks! | 15:27 |
@HeikoS | I will check a bit later | 15:28 |
shogun-buildbot | build #721 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/721 blamelist: Viktor Gal <vigsterkr@gmail.com>, Ariane Paola Gomes <arianepaola@users.noreply.github.com> | 15:28 |
sanuj | HeikoS, the math compiles fine on my local but not on buildbot | 15:28 |
sanuj | for svm_cookbook | 15:28 |
sanuj | wiking, i built shogun with ENABLE_COVERAGE=ON,now how to see coverage? | 15:29 |
@HeikoS | sanuj: I sent you this tool a while ago | 15:29 |
@HeikoS | that can just do it for the unit test binary | 15:29 |
sanuj | okay | 15:29 |
@HeikoS | sanuj: or you can check how the buildbot uploads the coverage | 15:29 |
-!- besser82_ [~besser82@fedora/besser82] has joined #shogun | 15:31 | |
-!- mode/#shogun [+o besser82_] by ChanServ | 15:31 | |
-!- besser82_ [~besser82@fedora/besser82] has quit [Ping timeout: 260 seconds] | 15:39 | |
-!- sanuj [~sanuj@117.203.14.130] has quit [Ping timeout: 240 seconds] | 15:43 | |
-!- besser82_ [~besser82@fedora/besser82] has joined #shogun | 15:48 | |
-!- mode/#shogun [+o besser82_] by ChanServ | 15:48 | |
-!- besser82_ [~besser82@fedora/besser82] has quit [Remote host closed the connection] | 15:58 | |
-!- besser82 [~besser82@fedora/besser82] has joined #shogun | 16:01 | |
-!- mode/#shogun [+o besser82] by ChanServ | 16:01 | |
-!- sanuj [~sanuj@117.203.14.130] has joined #shogun | 16:11 | |
sanuj | HeikoS, gcov wants optimization off | 16:11 |
sanuj | do i need to rebuild the tests | 16:12 |
@HeikoS | sanuj: yes thats ok | 16:12 |
@HeikoS | yes | 16:12 |
sanuj | how to set optimization off | 16:12 |
@HeikoS | good for developing | 16:12 |
@HeikoS | CMAKE_BUILD_MODE | 16:12 |
@HeikoS | Should be "Debug" | 16:12 |
sanuj | it is already debug | 16:12 |
sanuj | HeikoS, i had build with ==> cmake -DENABLE_TESTING=on -DPythonModular=on -DCMAKE_BUILD_TYPE=Debug .. | 16:13 |
@HeikoS | that has optimisation off | 16:13 |
@wiking | arianepaola: Server response (403): You are not allowed to store 'shogun' package information | 16:13 |
@wiking | :S | 16:13 |
arianepaola | lol | 16:13 |
arianepaola | hmmm | 16:14 |
@wiking | weird | 16:14 |
@wiking | trying to google why | 16:14 |
arianepaola | https://wiki.python.org/moin/TestPyPI | 16:15 |
arianepaola | did you try to run: python setup.py register before upload? | 16:16 |
arianepaola | ^^ wiking | 16:16 |
@wiking | ? | 16:18 |
@wiking | i ran | 16:18 |
@wiking | python setup.py register | 16:18 |
@wiking | and that was the output :) | 16:18 |
@wiking | ah ok | 16:18 |
arianepaola | there is also no shogun package on pypi | 16:18 |
@wiking | python setup.py register -r https://testpypi.python.org/pypi | 16:18 |
@wiking | so i shoudl try this first? | 16:19 |
arianepaola | it should work on pypi directly | 16:19 |
@wiking | i get 403 | 16:19 |
sanuj | HeikoS, how to run unit-test with gcov? | 16:22 |
sanuj | gcov ./shogun-unit-test | 16:23 |
@HeikoS | sanuj: I dont know, read the manual, should be in there | 16:23 |
@HeikoS | that command doesnt work? | 16:23 |
sanuj | no | 16:23 |
sanuj | ./shogun-unit-test.gcno:cannot open notes file | 16:23 |
@HeikoS | ok then manual :) | 16:24 |
-!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has joined #shogun | 16:27 | |
-!- GandalfTheWizard [~Eva@112.10.170.90] has joined #shogun | 16:33 | |
sanuj | HeikoS, it says that to use gcov, the code should be compiled with gcc options: -fprofile-arcs and -ftestcoverage | 16:34 |
sanuj | does that happen in debug mode? | 16:35 |
@wiking | oh for god sakes | 16:36 |
@wiking | https://pypi.python.org/pypi/shogun/ | 16:36 |
@wiking | :D | 16:36 |
arianepaola | yay | 16:36 |
@wiking | so this guy | 16:36 |
@wiking | created a python package | 16:36 |
@wiking | that fucking does nothing | 16:36 |
@wiking | and took our name | 16:36 |
@wiking | :) | 16:36 |
@wiking | time to message | 16:37 |
@wiking | https://github.com/portantier/shogun/issues/1 | 16:40 |
@wiking | :) | 16:40 |
sanuj | wiking, have you used shogun with gcov? | 16:40 |
sanuj | haha | 16:41 |
c4goldsw | wiking pong | 16:57 |
@wiking | c4goldsw: hey | 17:01 |
@wiking | sanuj: mmmm lemme recall this part i wrote like 3 years ago | 17:01 |
sanuj | wiking, thanks | 17:01 |
c4goldsw | I just want to clarify that this is essentially what I should be doing: https://gist.github.com/c4goldsw/fd325503b9b81a2a7f434291732e19e0 | 17:02 |
c4goldsw | as compared to https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/regression/LeastAngleRegression.cpp | 17:02 |
@wiking | c4goldsw: does it compile? :) | 17:03 |
c4goldsw | No, because it's superclass would also have to be parametereized / tempalted :( | 17:03 |
@wiking | because this is a problem for example | 17:04 |
@wiking | https://gist.github.com/c4goldsw/fd325503b9b81a2a7f434291732e19e0#file-templated-lars-L141 | 17:04 |
@wiking | which super class? | 17:04 |
@wiking | so while Xorig.matrix is now SGMatrix<T> that could be any of those args | 17:04 |
@wiking | MatrixXd | 17:04 |
c4goldsw | CLinearMachine | 17:04 |
c4goldsw | CLeastAngleRegession inherits from it | 17:05 |
@wiking | is actually typedef Matrix<double, Dynamic, Dynamic> MatrixXd; | 17:05 |
@wiking | so that's not a good mapping :( | 17:05 |
c4goldsw | hmm, okay | 17:05 |
@wiking | because you could have int32_t or any other int type there | 17:05 |
@wiking | or float32_t for that matter | 17:05 |
@wiking | but you'd be doing a double map | 17:05 |
@wiking | so that for sure has to be figured out | 17:06 |
@wiking | and as well wondering | 17:06 |
c4goldsw | But to be clear, am I taking the right approach? | 17:07 |
@wiking | whether one can do/or makes sense to do a LARS on CLeastAngleRegression<bool> or CLeastAngleRegression<char>; | 17:07 |
@wiking | mmm | 17:07 |
@wiking | i'm wondering now a bit | 17:07 |
c4goldsw | Ah, but I had that question last week - which types make sense? | 17:07 |
@wiking | because you know there are other means to figure out the type of the CFeatures | 17:07 |
c4goldsw | What do you mean by that? | 17:08 |
@wiking | https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/features/FeatureTypes.h | 17:09 |
@wiking | so if you call CFeatures->get_feature_type() | 17:09 |
@wiking | you'll get the type | 17:09 |
@wiking | same for EFeatureClass | 17:10 |
@wiking | i mean the thing is | 17:10 |
@wiking | that LARS could even get a C_COMBINED | 17:10 |
@wiking | :) | 17:10 |
@wiking | so maybe instead of templating | 17:11 |
@wiking | you could like do an actual type detection within the train_machine part | 17:11 |
@wiking | and do the rigth call | 17:12 |
@wiking | becuase i wonder how would you solve the issue of type mapping with eigen | 17:12 |
@wiking | this way btw you dont need to support templating in the partent class | 17:12 |
@wiking | *parent | 17:12 |
c4goldsw | Alright then | 17:13 |
@wiking | again this might need a bit more thinking | 17:13 |
@wiking | because actually whatever you'll write here | 17:13 |
@wiking | most of the code | 17:13 |
@wiking | shoudl be reusable | 17:13 |
@wiking | in other parts of shogun | 17:13 |
@wiking | since this is a problem as you saw in many palces | 17:14 |
c4goldsw | In what sense? | 17:15 |
c4goldsw | I understand your type detection approach | 17:15 |
c4goldsw | But I'm a little unclear of what problems could arise, as I haven't thoroughly read through train_machine | 17:15 |
@wiking | in sense thwat this type detection thing should be available in other parts | 17:15 |
@wiking | and how to map an SGMatrix<int8_t> to an eigen map | 17:16 |
c4goldsw | What possible values can eigen maps take? | 17:16 |
c4goldsw | I though they were parameterized. | 17:16 |
c4goldsw | possible types* | 17:16 |
@wiking | i guess whatever u give it :) | 17:17 |
c4goldsw | Alright then - wouldn't this simply become a problem of parameterizing everything else that uses it? | 17:18 |
c4goldsw | Which could be quite a big problem. | 17:18 |
@wiking | ? | 17:19 |
OXPHOS | wiking: hey! | 17:20 |
OXPHOS | wiking: can you have a look at the linalg when you got a sec? https://github.com/shogun-toolbox/shogun/pull/3277 | 17:20 |
OXPHOS | thx! | 17:20 |
c4goldsw | Well, you asked "how to map an SGMatrix<int8_t>" to an eigen map. Wouldn't that just involve doing : Matrix<int8_t, Dynamic, Dynamic> MatrixXd; ? | 17:21 |
c4goldsw | Or, let's use a more sensible type | 17:21 |
@wiking | no | 17:21 |
@wiking | MatrixXd is a reserved already defined type in eigen | 17:22 |
@wiking | that is Matrix<double, Dynamic, Dynamic> | 17:22 |
sanuj | lisitsyn, there? | 17:23 |
c4goldsw | Then let's take a step back - instead of templating / parameterizing all of LeastAngleRegression, you suggest that I can just check the type of CFeatures* data, then act accordingly. | 17:24 |
arianepaola | wiking: did you find any info regarding launchpad packaging format requirements? | 17:24 |
c4goldsw | check inside of train_machine*. That's what you were saying, right? | 17:24 |
@HeikoS | sanuj: jo | 17:24 |
@HeikoS | now checking PRs | 17:24 |
-!- ptizoom [~christian@host-92-21-196-52.as13285.net] has quit [Ping timeout: 258 seconds] | 17:25 | |
@wiking | c4goldsw: yep | 17:25 |
c4goldsw | Good. So, you're suggesting that the outstanding problem is that I would have to find ways of making CFeatures* of various types compatible with eigen maps, right? | 17:26 |
@wiking | arianepaola: https://developer.ubuntu.com/en/publish/other-forms-of-submitting-apps/ppa/ | 17:26 |
@wiking | c4goldsw: yep | 17:27 |
arianepaola | thanks wiking | 17:27 |
c4goldsw | So, should I continue in that direction? | 17:27 |
arianepaola | wiking: do you know https://build.opensuse.org/ ? | 17:27 |
@wiking | c4goldsw: nou | 17:27 |
@wiking | sorry | 17:27 |
@wiking | arianepaola: noup | 17:27 |
@wiking | c4goldsw: yeap since this templating what you've started with | 17:27 |
c4goldsw | wiking that's fine. | 17:27 |
@wiking | i'm not sure will lead to a solution | 17:28 |
arianepaola | wiking: seems quite large -> openSUSE Build Service hosts 44,424 projects, with 364,974 packages, in 68,967repositories and is used by 44,475 confirmed developers. | 17:28 |
c4goldsw | wiking haha with the way things are going I don't think they will | 17:28 |
@wiking | arianepaola: how do we apply/register? | 17:28 |
arianepaola | I think it is free | 17:28 |
arianepaola | I will check | 17:28 |
@wiking | mmm i mean the thing is that we wanna ha ppa | 17:29 |
@wiking | because it's an easy way for ubuntu/debian | 17:29 |
@HeikoS | +1 for ppa | 17:30 |
@HeikoS | nightly build | 17:30 |
@HeikoS | and every release triggers an automatic one | 17:30 |
c4goldsw | wiking don't think it will * | 17:31 |
@wiking | c4goldsw: wil will | 17:31 |
@wiking | just takes time | 17:31 |
shogun-notifier- | shogun: OXPHOS :develop * 9950533 / / (7 files): https://github.com/shogun-toolbox/shogun/commit/9950533524e3d393a0bd7eb94393524b419a9dd6 | 17:31 |
shogun-notifier- | shogun: means cookbook - meta examples and integration test | 17:31 |
shogun-notifier- | shogun: Heiko Strathmann :develop * 59c003f / / (7 files): https://github.com/shogun-toolbox/shogun/commit/59c003fdbb01a5f2af680c5a99de7e331195e7cb | 17:31 |
shogun-notifier- | shogun: Merge pull request #3183 from OXPHOS/cookbook | 17:31 |
shogun-notifier- | shogun: | 17:31 |
shogun-notifier- | shogun: Add kmeans page to cookbook | 17:31 |
@wiking | \o/ | 17:31 |
@wiking | cookering | 17:31 |
arianepaola | although it probably out of date: http://openlife.cc/blogs/2011/february/looking-opensuse-build-service-and-launchpad-ppa-aka-how-build-packages-mepsql | 17:31 |
arianepaola | the opensuse buildservice could be useful for rpm though | 17:32 |
shogun-buildbot | build #722 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/722 blamelist: OXPHOS <engelzora@gmail.com> | 17:32 |
arianepaola | ppa is easier to add | 17:32 |
c4goldsw | wiking No, I mean templating everything - you just said that wouldn't work. So, I'll focus on making train_machine compatible with different types (Sorry for making you talk to two people at once). Another question: which types should I make it compatible with? | 17:33 |
shogun-buildbot | build #723 of trusty - libshogun - viennacl is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/trusty%20-%20libshogun%20-%20viennacl/builds/723 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 17:33 |
c4goldsw | wiking would it even make sense for non float types? | 17:34 |
@wiking | c4goldsw: good question | 17:34 |
@wiking | it's really up to the algorithm | 17:35 |
@HeikoS | c4goldsw, wiking couldnt we just have some template methods that do the conversions for us? | 17:35 |
c4goldsw | I'm looking at it:https://en.wikipedia.org/wiki/Least-angle_regression | 17:35 |
sanuj | HeikoS, for svm cookbook, the math compiles on my local but not on buildbot | 17:35 |
@HeikoS | I also think the change between int and float is not so interesting | 17:35 |
@HeikoS | but rather float32 vs float64 | 17:35 |
@HeikoS | and int32 vs int64 | 17:35 |
@HeikoS | these 4 are like 80% for the use cases where one would be interested | 17:35 |
@HeikoS | in changing representation | 17:35 |
sanuj | i tried some online thing to translate math and it worked on that too | 17:35 |
@HeikoS | sanuj: | 17:36 |
@HeikoS | http://buildbot.shogun-toolbox.org/cookbook_pr/033f9c8fa9e07d040d0c94e6c16092a1090983c4/examples/classifier/kernel_svm.html | 17:36 |
sanuj | yeah i saw that | 17:36 |
@wiking | HeikoS: could | 17:37 |
c4goldsw | HeikoS I've been doing that, but it could potentially involve templating all superclasses and sublcasses of those superclasses. CLeastAngleRegression has CLinearMachine as a parent and I've have to template that if I'm to template LARs | 17:37 |
@HeikoS | c4goldsw: the algorithm classes would not change | 17:37 |
@HeikoS | only the features | 17:37 |
@HeikoS | and then there is a template method that gives corresponding eigen vector/matrix | 17:37 |
-!- ptizoom [~christian@host-92-21-231-167.as13285.net] has joined #shogun | 17:37 | |
@HeikoS | and we can use MatrixBase in the algorithm if eigen is used | 17:37 |
@HeikoS | or even auto | 17:38 |
@HeikoS | in the algo you then do | 17:38 |
@HeikoS | features->to_eigen_bla() | 17:38 |
@HeikoS | or actually | 17:38 |
@HeikoS | a wrapper for map would be good | 17:38 |
@HeikoS | features->to_eigen_map(num_rows, num_cols) | 17:38 |
@HeikoS | offset maybe | 17:38 |
@HeikoS | and that method is templated | 17:39 |
@HeikoS | sanuj: saw my comment in data? | 17:39 |
c4goldsw | It's a little hard for me to grasp this, could you possibly make a Gist of it? | 17:39 |
c4goldsw | HeikoS ^ | 17:39 |
sanuj | HeikoS, yeah | 17:39 |
sanuj | shall i try changing some parameters for the svm | 17:40 |
@HeikoS | c4goldsw: I see, sorry | 17:41 |
@HeikoS | c4goldsw: I am currently a bit overbusy, so no time to draft it out | 17:41 |
@HeikoS | but my point is this: | 17:41 |
@HeikoS | the feature class is templated right? | 17:41 |
c4goldsw | Yes | 17:41 |
@HeikoS | ok | 17:42 |
@HeikoS | so why not add a method to DenseFeeatures | 17:42 |
@HeikoS | that returns an eigen map to the feature matrix? | 17:42 |
@HeikoS | of the correct type | 17:42 |
@HeikoS | Like VectorXd or VectorXf | 17:42 |
@HeikoS | like for example get_feature_matrix | 17:43 |
@HeikoS | returns SGMatrix<T> | 17:43 |
c4goldsw | That makes sense | 17:43 |
@HeikoS | but for eigen its a bit different | 17:43 |
@HeikoS | we cannot just pass on the T | 17:43 |
@HeikoS | but for T=float64_t, we return MatrixXd, and for T=float32_t we return MatrixXf | 17:44 |
c4goldsw | Ah, that's clearer now. | 17:44 |
c4goldsw | HeikoS Thanks | 17:46 |
@HeikoS | c4goldsw: are you looking at a concrete example? | 17:47 |
c4goldsw | HeikoS yes: https://gist.github.com/c4goldsw/fd325503b9b81a2a7f434291732e19e0#file-templated-lars-L135 | 17:48 |
OXPHOS | HeikoS: hey! got time to check the PR? https://github.com/shogun-toolbox/shogun/pull/3277 | 17:50 |
c4goldsw | Heiko: In this file, I've templated everything so far, which has lead to errors being thrown, since superclass members are not parameterized. wiking pointed out that this line is an issue. So, as you suggested, I'll get onto making a get_feature_matrix() which can return eigen maps of differing types | 17:50 |
@HeikoS | OXPHOS: hi | 17:50 |
@HeikoS | going through things right now | 17:50 |
@HeikoS | coming back to you soon :) | 17:50 |
OXPHOS | HeikoS: thx! | 17:50 |
@HeikoS | c4goldsw: why is this line a problem? | 17:50 |
@HeikoS | lines like Map<VectorXd> are a problem | 17:51 |
@HeikoS | because there, you have a float64_t | 17:51 |
@HeikoS | whereas SGMatrix<ST> is ok, it is template | 17:51 |
@HeikoS | just need a way to create the eigen map from the SG structures in a templated way | 17:51 |
c4goldsw | HeikoS ah, excuse me | 17:51 |
@HeikoS | so I think the way to go is to have some factory where one can create eigen maps for existing SG data types | 17:52 |
@HeikoS | I think there should be something like this already somewhere | 17:52 |
@HeikoS | just need to template it | 17:52 |
@HeikoS | lisitsyn: ^ ? | 17:52 |
c4goldsw | HeikoS should it be templated for all types? | 17:52 |
@HeikoS | if we use eigen3 | 17:52 |
@HeikoS | we can only template for those that are supported by eigen | 17:52 |
@HeikoS | i dont know how many that are | 17:53 |
@HeikoS | gotta check their manual | 17:53 |
c4goldsw | I'll find out then. | 17:53 |
@HeikoS | nice one that you are digging on this btw! | 17:53 |
@HeikoS | really useful | 17:53 |
@HeikoS | to change lars accuracy to 32bit without doing anything | 17:53 |
c4goldsw | Ah, you're welcome. I'm off for now, I'll be back on tomorrow. | 17:54 |
@HeikoS | see you! :) | 17:55 |
-!- c4goldsw [5da420e6@gateway/web/cgi-irc/kiwiirc.com/ip.93.164.32.230] has quit [Quit: http://www.kiwiirc.com/ - A hand crafted IRC client] | 17:55 | |
@HeikoS | OXPHOS: ok | 17:58 |
@HeikoS | OXPHOS: can you give me a quick update what you recently did there and what I should look out for? | 17:59 |
@HeikoS | OXPHOS: whats CPU_CACHE_LINE SIZE? | 17:59 |
@HeikoS | OXPHOS: actually just talking to lambday | 18:00 |
@HeikoS | will get back after | 18:00 |
sanuj | HeikoS, do i need to check mem leak in python for this https://github.com/shogun-toolbox/shogun/pull/3283 | 18:00 |
@HeikoS | sanuj: will get back soon | 18:01 |
sanuj | okay | 18:01 |
-!- GandalfTheWizard [~Eva@112.10.170.90] has quit [Quit: Leaving.] | 18:13 | |
OXPHOS | HeikoS: hey thanks for the comments | 18:20 |
OXPHOS | HeikoS: so basically we don't allow [] visit | 18:21 |
@HeikoS | OXPHOS: yep | 18:25 |
@HeikoS | found a problem | 18:25 |
@HeikoS | thinking and commenting | 18:25 |
@HeikoS | OXPHOS: ok put final comment | 18:35 |
@HeikoS | OXPHOS: can you read the final comment first? | 18:38 |
@HeikoS | and othen go backwards from there | 18:38 |
OXPHOS | HeikoS: sure | 18:38 |
@HeikoS | OXPHOS: btw, you should aim for 100% unit test coverage of this code | 18:38 |
@HeikoS | please read everything carefully and let me know if you feel you udnerstand it | 18:39 |
sanuj | HeikoS, see https://github.com/shogun-toolbox/shogun-data/pull/102 | 18:39 |
sanuj | i updated the data | 18:39 |
sanuj | should be good now | 18:39 |
@HeikoS | what was wrong? | 18:39 |
OXPHOS | So if I have to transfer everything back to SGVector first then compare the element value by value - I have to assume everything associate with operator SGVector() works | 18:39 |
OXPHOS | HeikoS^ | 18:39 |
shogun-notifier- | shogun-data: Sanuj :master * baf949d / testsuite/meta/classifier/kernel_svm.dat: https://github.com/shogun-toolbox/shogun-data/commit/baf949d1d1449c82e63f1a39ad12d38f4a827238 | 18:39 |
shogun-notifier- | shogun-data: add data for kernel svm cookbook | 18:39 |
sanuj | HeikoS, increased the width of gaussian kernel | 18:39 |
shogun-notifier- | shogun-data: Heiko Strathmann :master * 038f080 / testsuite/meta/classifier/kernel_svm.dat: https://github.com/shogun-toolbox/shogun-data/commit/038f080e9eb720097900b34e51372798a385d594 | 18:39 |
shogun-notifier- | shogun-data: Merge pull request #102 from sanuj/master | 18:39 |
shogun-notifier- | shogun-data: | 18:39 |
shogun-notifier- | shogun-data: add data for kernel svm cookbook | 18:39 |
@HeikoS | OXPHOS: I am not sure what you mean | 18:40 |
@HeikoS | sanuj: always a good idea :) | 18:40 |
@HeikoS | OXPHOS: can you ask again? | 18:40 |
sanuj | :D | 18:40 |
@HeikoS | sanuj: any idea about the math? | 18:41 |
sanuj | i'm updating it | 18:41 |
sanuj | let's see if that works | 18:41 |
OXPHOS | HeikoS: by 100% unit test coverage - does it include every single method in the class? | 18:41 |
@HeikoS | OXPHOS: every single line of code | 18:41 |
@HeikoS | OXPHOS: there is a tool to check that | 18:41 |
@HeikoS | wiking: still awake? | 18:42 |
@HeikoS | OXPHOS: but dont worry about that for now | 18:42 |
@HeikoS | the main thing we need to change for now is the copying | 18:42 |
@HeikoS | and as a result the reference counting | 18:42 |
@HeikoS | and as a result the alignment | 18:42 |
sanuj | HeikoS, When using gcov, you must first compile your program with two special GCC options: ‘-fprofile-arcs -ftest-coverage’. | 18:43 |
sanuj | how to do this ^ | 18:43 |
OXPHOS | HeikoS: okay...but i don't feel like it's a big problem copying SGVector. They'll be done just once by user | 18:43 |
@HeikoS | sanuj: ah ok, lemme check | 18:43 |
@HeikoS | OXPHOS: it is a big problem | 18:43 |
@HeikoS | imagine you have two SGVectors | 18:43 |
@HeikoS | and you want to compute their dot product | 18:43 |
@HeikoS | via linalg::dot | 18:43 |
@HeikoS | linalg:dot accepts Vector at this point | 18:44 |
@HeikoS | upon casting SGVector to Vector, the data is copied | 18:44 |
@HeikoS | so the dot product is preceeded by copying the two vectors | 18:44 |
@HeikoS | see? | 18:44 |
OXPHOS | But this is based on the assumptions that the vector will be used (for calculation) for only once | 18:45 |
OXPHOS | Every SGVector can have an associated Vector | 18:46 |
OXPHOS | Speaking of which, since Vector is just a shallow copy of SGVector, what is the difference of having only SGVector? | 18:47 |
OXPHOS | HeikoS^ | 18:47 |
@HeikoS | sanuj: can you check CMakeLists.txt in the main dir | 18:47 |
@HeikoS | line 192, 193 | 18:47 |
@HeikoS | try adding them there | 18:47 |
@HeikoS | (re-run cmake afterwards) | 18:47 |
sanuj | okay | 18:47 |
@HeikoS | sanuj: otherwise, you could also compile and link the unit tests manually | 18:48 |
@HeikoS | ah no | 18:48 |
@HeikoS | sanuj: I dont know | 18:48 |
@HeikoS | sanuj: what about checking our covarage build? | 18:49 |
@HeikoS | OXPHOS: you are right, if memory lives on cpu, Vector wraps the *same* memory as SGVector used to construct it | 18:49 |
@HeikoS | why we should have another class for it, in my opinion is separation of concerns | 18:50 |
@HeikoS | e.g. no element-wise access | 18:50 |
@HeikoS | sanuj: we can probably also add them to the debug build, but need to ask besser82 or wiking for that | 18:51 |
@HeikoS | sanuj: http://buildbot.shogun-toolbox.org/builders/coverity%20analysis/builds/102 | 18:51 |
@HeikoS | ever checked this one? | 18:51 |
sanuj | no | 18:52 |
sanuj | this is the coverage build? | 18:52 |
@HeikoS | I think only static analysis | 18:53 |
sanuj | HeikoS, the alignment looks weird now http://buildbot.shogun-toolbox.org/cookbook_pr/708245f57c6f5a0d5385277303ff714587bc3ab6/examples/classifier/kernel_svm.html | 18:53 |
sanuj | it is again perfectly fine on my local | 18:53 |
sanuj | do we have old version of sphinx | 18:54 |
@HeikoS | can you send the pr link again? | 18:55 |
@HeikoS | alignment weird indeed | 18:55 |
@HeikoS | but maybe its just some latex command that should not be used | 18:55 |
@HeikoS | lemme check | 18:55 |
OXPHOS | HeikoS: i see. the current idea is, Vector holds two vectors, one on GPU, one on CPU. If onGPU is true, CPUvec part is masked. As the calculation is done on GPU and CPU won't be updated | 18:55 |
sanuj | HeikoS, PR is here https://github.com/shogun-toolbox/shogun/pull/3281 | 18:55 |
OXPHOS | HiekoS: that's why we transfer_to_CPU everytime we try to copy it to a new Vector | 18:55 |
@HeikoS | OXPHOS: lets have a private chat | 18:56 |
@HeikoS | I get confused otherwise ;) | 18:56 |
OXPHOS | HeikoS: Can't use this command in this window..can you receive me? | 18:57 |
OXPHOS | "Can't use this command in this window" | 18:57 |
@HeikoS | ok then :) | 18:57 |
@HeikoS | nevermind | 18:57 |
@HeikoS | Vector holds two pointers | 18:57 |
@HeikoS | CPU one points to the data that was handed in on construction | 18:58 |
@HeikoS | i.e. the one in the SGVector | 18:58 |
sanuj | HeikoS, normally when do you get time to review/merge PRs and chat on irc? | 18:58 |
@HeikoS | sanuj: nowish | 18:58 |
OXPHOS | HeikoS: yes | 18:58 |
sanuj | HeikoS, earlier than this not possible? | 18:59 |
@HeikoS | OXPHOS: I did not get what you mean by this transfer thing you said | 18:59 |
@HeikoS | sanuj: sometimes :) | 18:59 |
sanuj | just asking because my sleeping time starts :D | 18:59 |
@HeikoS | sanuj: haha | 18:59 |
@HeikoS | yeah I am usually around earlier as well | 18:59 |
@HeikoS | but at some point I also need to work | 18:59 |
sanuj | yeah | 18:59 |
@HeikoS | so sometimes I turn off irc | 18:59 |
sanuj | i understand | 19:00 |
@HeikoS | (remember I am not paid for this, but for my normal job ;) ) | 19:00 |
sanuj | :D | 19:00 |
sanuj | HeikoS, you are the best :D | 19:00 |
OXPHOS | HeikoS: I was trying to justify that transfer stuff - and also the copy problem - let me start over again | 19:00 |
@HeikoS | OXPHOS: ok | 19:00 |
@HeikoS | so a copy should only ever happen | 19:01 |
@HeikoS | when data is put on gpu | 19:01 |
@HeikoS | sanuj: its the && | 19:01 |
@HeikoS | remove it and all will be good | 19:01 |
sanuj | okay | 19:01 |
OXPHOS | HeikoS: so CPU and GPUptr of Vector should be synchronized some time, because the calculation is done on one end | 19:01 |
OXPHOS | HeikoS: if Vector is a shallow copy of SGVector, there's chance that the vector on CPU is modified, like SGVector[i] = 0 | 19:02 |
OXPHOS | and the *Vector.CPUptr is also changed | 19:03 |
@HeikoS | no need for that | 19:03 |
OXPHOS | HeikoS: but there is no way for Vector to mark this change as it is done in SGVector | 19:03 |
@HeikoS | for the syncronisation I mean | 19:03 |
@HeikoS | you copy data to GPU, then you can forget about the data in CPU (it can even be freed, i.e. ref-counter decreased) | 19:04 |
@HeikoS | you move data to GPU, then you do stuff with it, and at some point you transfer it back to main memory, then you have to allocate memory for that again (or use previously allocated) | 19:04 |
OXPHOS | If CPUptr is not freed - re-use previous allocated memory - SGVector is changed; if the CPUptr is freed - new memory - new SGVector | 19:05 |
OXPHOS | This should be something user decides? or we should? | 19:06 |
OXPHOS | HeikoS^ | 19:07 |
@HeikoS | user | 19:07 |
@HeikoS | option a) allocate new block for transfering back | 19:07 |
@HeikoS | with fresh ref-counter as it was just created | 19:07 |
@HeikoS | option b) user provides pointer (or SGVector) that has pre-allocated memory | 19:08 |
OXPHOS | HeikoS: i see | 19:09 |
@HeikoS | about this thing of re-using the memory that was used to copy things to GPU, i need to think | 19:10 |
@HeikoS | but thats not a big difference | 19:10 |
@HeikoS | and easy to change | 19:10 |
@HeikoS | the main point is: if data is never transferred to GPU, it is never copied | 19:10 |
OXPHOS | HeikoS: cool. make sense | 19:11 |
sanuj | HeikoS, is this fine http://buildbot.shogun-toolbox.org/cookbook_pr/c29762cbcd29c568b084a32be454a1b7c5bf5319/examples/classifier/kernel_svm.html | 19:11 |
sanuj | HeikoS, how did you know && was causing the problem? | 19:12 |
@HeikoS | can you add a new line for the constraints and say "subject to" in the text? | 19:12 |
@HeikoS | sanuj: intuition | 19:12 |
sanuj | can i include "subject to" in the math area | 19:12 |
@HeikoS | sanuj: | 19:12 |
@HeikoS | no | 19:12 |
@HeikoS | do two math sections | 19:12 |
@HeikoS | and normal text in between | 19:12 |
sanuj | cool | 19:12 |
OXPHOS | HeikoS: honestly now I prefer the old idea again - have separate CPUVector and GPUVector. you see if we're doing calculation on GPU, we have to go through a CPUptr step | 19:12 |
@HeikoS | OXPHOS: so what makes this bad is | 19:13 |
@HeikoS | that things are decided at compile time them | 19:13 |
@HeikoS | and we dont want that | 19:13 |
@HeikoS | think: what would be the signature of linalg::dot? | 19:13 |
@HeikoS | would it accept CPUVector or GPUVector? | 19:13 |
@HeikoS | and how to change this at runtime | 19:14 |
@HeikoS | impossible | 19:14 |
@HeikoS | thats the idea of not having separate classes for CPU/GPU | 19:14 |
@HeikoS | OXPHOS: see that? | 19:14 |
OXPHOS | HeikoS sry im not familiar with compile/runtime stuff..it won't work even we have a factory method and base class, and everything was passed as base class ptr(also linalg::dot takes baseptr)? | 19:16 |
@HeikoS | nope | 19:16 |
OXPHOS | ;( | 19:16 |
@HeikoS | OXPHOS: it is very easy to understand | 19:16 |
@HeikoS | think about it this way: | 19:17 |
@HeikoS | the difference is WHEN you take the decision which method is called | 19:17 |
@HeikoS | linalg:dot(CPUVec) or linalg:dot(GPUVec) | 19:17 |
@HeikoS | compile time, this is decided when you compile shogun | 19:17 |
@HeikoS | and cannot be changed afterwards | 19:17 |
@HeikoS | runtime: you can change the behaviour at runtume | 19:18 |
@HeikoS | now with a base class pointer, you would think that can be avoided | 19:18 |
@HeikoS | as one can pass both CPU and GPU | 19:18 |
@HeikoS | BUT | 19:18 |
@HeikoS | the factory, which one does it return? this is also decided at compile time | 19:18 |
@HeikoS | which hard-wires the backend into shogun binary | 19:18 |
@HeikoS | and cannot be changed anymore after it has been compiled | 19:19 |
OXPHOS | okay so it's not like we can switch backend anytime after compiling | 19:19 |
OXPHOS | can i try, in Vector constructor, if HAVE_VIENNACL, everything goes to GPU directly? skipping that CPUptr step? | 19:21 |
@HeikoS | yes | 19:22 |
sanuj | HeikoS, what do you think about this http://buildbot.shogun-toolbox.org/cookbook_pr/c4ab6ffb384ea563327f64612b659033e9e21f13/examples/classifier/kernel_svm.html | 19:22 |
@HeikoS | OXPHOS: nope | 19:22 |
@HeikoS | every copy to GPU is expensive, right? | 19:22 |
@HeikoS | so it should never happen implicitly | 19:22 |
@HeikoS | but always require the user to call a method | 19:23 |
@HeikoS | that has a warning sign on it | 19:23 |
@HeikoS | sanuj: thats it :) | 19:23 |
@HeikoS | sanuj: check whether travis executes you integration test, then ping me and I wil lmerge | 19:23 |
OXPHOS | HeikoS: okay i see. thanks so much! | 19:24 |
sanuj | HeikoS, travis is going to take time | 19:24 |
sanuj | and i'm going to sleep :D | 19:24 |
OXPHOS | HeikoS: hope wiking won't be upset if i remove that aligned stuff XD | 19:24 |
@HeikoS | sanuj: good night :) | 19:24 |
@HeikoS | OXPHOS: yeah align.... | 19:24 |
@HeikoS | mmh | 19:25 |
OXPHOS | sanuj: night night | 19:25 |
@HeikoS | I dont know much about it tbh | 19:25 |
sanuj | but one more thing before i sleep | 19:25 |
sanuj | https://github.com/shogun-toolbox/shogun/pull/3283 | 19:25 |
sanuj | HeikoS, do i need to check for mem leaks ^ | 19:25 |
@HeikoS | OXPHOS: but I dont see how we can align the data itself (!= the Vector class), without having to copy | 19:25 |
@HeikoS | OXPHOS: we can put that back in later | 19:26 |
@HeikoS | but for sure, wiking will be angry | 19:26 |
OXPHOS | HeikoS: yeah I was thinking the same..but he told me to use align before I have the deepcopy? but anyway it's easy to add back | 19:26 |
@HeikoS | yeah | 19:26 |
@HeikoS | keep the snippets | 19:26 |
@HeikoS | we cant copy, thats for sure | 19:26 |
@HeikoS | I mean copy in cpu case | 19:26 |
@HeikoS | before the move_to_gpu() call | 19:26 |
OXPHOS | okay | 19:26 |
@HeikoS | OXPHOS: ok, Ill head home now | 19:27 |
@HeikoS | OXPHOS: thanks for the discussion :) | 19:27 |
@HeikoS | OXPHOS: and pls let me know if there are more questions | 19:27 |
sanuj | home sweet home | 19:27 |
@HeikoS | OXPHOS: important to getthis right | 19:27 |
@HeikoS | sanuj: indeed :) | 19:27 |
@HeikoS | climbing wall is my home :D | 19:27 |
OXPHOS | HeikoS it's a nice one. thanks! | 19:27 |
sanuj | climbing wall ? | 19:28 |
sanuj | HeikoS, mem leaks? | 19:29 |
sanuj | okay sleeping | 19:30 |
sanuj | HeikoS, OXPHOS goodnight :) | 19:31 |
-!- sanuj [~sanuj@117.203.14.130] has quit [Quit: Leaving] | 19:31 | |
-!- HeikoS [~heiko@nat-201-233.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.] | 19:39 | |
-!- HeikoS [~heiko@nat-201-233.internal.eduroam.ucl.ac.uk] has joined #shogun | 19:40 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 19:40 | |
-!- HeikoS [~heiko@nat-201-233.internal.eduroam.ucl.ac.uk] has quit [Client Quit] | 19:41 | |
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 258 seconds] | 20:32 | |
arianepaola | bye everyone, see you tomorrow | 21:21 |
OXPHOS | arianepaola: good night | 21:23 |
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout] | 21:39 | |
@wiking | OXPHOS: u wrote kmeans cookbook? http://buildbot.shogun-toolbox.org/cookbook_pr/cec96964e162b2442e7d7515d2610b883f00fba6/examples/clustering/kmeans.html ? | 22:35 |
@wiking | see argmin | 22:35 |
OXPHOS | wiking: yes? | 22:36 |
@wiking | that argmin has a problem right? | 22:37 |
@wiking | lol | 22:37 |
@wiking | not | 22:37 |
@wiking | (my browser/internet) | 22:37 |
@wiking | sorry | 22:37 |
OXPHOS | wiking haha | 22:37 |
OXPHOS | so it's fine? | 22:37 |
@wiking | yeo | 22:39 |
-!- tctara [~quassel@128.199.61.169] has joined #shogun | 23:53 | |
OXPHOS | wiking: there? | 23:58 |
--- Log closed Thu Jun 16 00:00:42 2016 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!