| --- Log opened Sat May 28 00:00:15 2016 | ||
| -!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 250 seconds] | 03:26 | |
| shogun-buildbot | build #7 of clang - thread analysis is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/clang%20-%20thread%20analysis/builds/7 blamelist: Viktor Gal <viktor.gal@maeth.com> | 03:45 |
|---|---|---|
| shogun-buildbot | build #6 of clang - undefined behaviour analysis is complete: Failure [failed compile] Build details are at http://buildbot.shogun-toolbox.org/builders/clang%20-%20undefined%20behaviour%20analysis/builds/6 blamelist: Viktor Gal <viktor.gal@maeth.com> | 03:49 |
| Saurabh7 | Update : Got parallel xval working for svm, trying to generalize it now for Machine* and Features*, and looking for possible memleaks | 05:09 |
| Saurabh7 | will make some cookbooks today, working on LDA one. | 05:10 |
| -!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has quit [Quit: Page closed] | 05:45 | |
| shogun-buildbot | build #7 of memleak - valgrind is complete: Failure [failed memory check] Build details are at http://buildbot.shogun-toolbox.org/builders/memleak%20-%20valgrind/builds/7 blamelist: Viktor Gal <viktor.gal@maeth.com> | 06:16 |
| shogun-buildbot | build #1009 of nightly_none is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_none/builds/1009 blamelist: Viktor Gal <viktor.gal@maeth.com> | 06:27 |
| shogun-buildbot | build #1138 of nightly_default is complete: Failure [failed test notebooks] Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_default/builds/1138 blamelist: Viktor Gal <viktor.gal@maeth.com> | 07:36 |
| -!- sanuj [~sanuj@59.97.245.155] has joined #shogun | 07:55 | |
| -!- sanuj [~sanuj@59.97.245.155] has quit [Ping timeout: 264 seconds] | 08:53 | |
| -!- sanuj [~sanuj@117.203.3.118] has joined #shogun | 09:05 | |
| Saurabh7 | sanuj: hey | 10:48 |
| sanuj | Saurabh7, hello :) | 10:48 |
| Saurabh7 | how did you run the cookbook exmaples | 10:49 |
| Saurabh7 | generate.py ? | 10:49 |
| sanuj | no | 10:49 |
| sanuj | use make cookbook to build them | 10:49 |
| sanuj | make test to run | 10:49 |
| Saurabh7 | ah ok | 10:50 |
| sanuj | Saurabh7, since they are integration tests, should run by make test | 10:50 |
| -!- sonne|osx [~sonne@x4db3de16.dyn.telefonica.de] has joined #shogun | 10:58 | |
| Saurabh7 | sanuj: whats your sphinx version? | 11:01 |
| -!- sanuj [~sanuj@117.203.3.118] has quit [Ping timeout: 264 seconds] | 11:01 | |
| -!- sanuj [~sanuj@117.203.3.118] has joined #shogun | 11:45 | |
| sanuj | wiking, there? | 12:34 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 12:40 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 12:40 | |
| @HeikoS | wiking, lisitsyn jo | 12:43 |
| sanuj | HeikoS, hi :) | 12:43 |
| @HeikoS | sanuj: hi there! | 12:43 |
| sanuj | i need help | 12:43 |
| @HeikoS | I am here to help ;) | 12:43 |
| sanuj | have you seen the code in aer? | 12:43 |
| @HeikoS | not yet, what exactly? | 12:43 |
| sanuj | HeikoS, okay, i'll put it in a gist and share it with you in 5 mins | 12:44 |
| @HeikoS | cool | 12:44 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.] | 12:46 | |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 12:49 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 12:49 | |
| sanuj | HeikoS, https://gist.github.com/sanuj/d58a90e327516c11449cfba597e62316 | 12:52 |
| sanuj | HeikoS, i think the error is due to this | 12:52 |
| sanuj | HeikoS, https://github.com/shogun-toolbox/aer/blob/master/src/aer/util/d_ptr.hpp#L19 | 12:53 |
| @HeikoS | sanuj: checking just a sec | 12:57 |
| sanuj | cool | 12:57 |
| @HeikoS | Saurabh7: jo | 12:57 |
| Saurabh7 | HeikoS: hi! | 12:58 |
| @HeikoS | Saurabh7: hey, I am just looking at KMeans | 12:59 |
| Saurabh7 | HeikoS: I sent a PR | 12:59 |
| Saurabh7 | oh | 12:59 |
| Saurabh7 | about? | 12:59 |
| @HeikoS | just to fix some problem with the meta examples | 12:59 |
| @HeikoS | I saw that all the mini batch stuff is in the main class | 12:59 |
| Saurabh7 | ok whats the problem | 13:00 |
| Saurabh7 | yes its done via enum | 13:00 |
| @HeikoS | Saurabh7: I remember that we changed the class sturcutre a bit | 13:00 |
| @HeikoS | can you remind me what we did there? | 13:00 |
| Saurabh7 | HeikoS: no its not done | 13:00 |
| @HeikoS | I see | 13:00 |
| Saurabh7 | i mean not merged | 13:00 |
| @HeikoS | Ah | 13:00 |
| @HeikoS | ok | 13:00 |
| @HeikoS | so I am looking at the old | 13:00 |
| @HeikoS | structure | 13:00 |
| @HeikoS | I see | 13:01 |
| @HeikoS | I will send a few mini changes to KMeans | 13:01 |
| Saurabh7 | yes | 13:01 |
| @HeikoS | will have to rebase the patch | 13:01 |
| @HeikoS | okok | 13:01 |
| @HeikoS | will check your other PR soon | 13:01 |
| Saurabh7 | HeikoS: I think we can extend the parallel xval to other machines too | 13:02 |
| Saurabh7 | ust need to create similar objects | 13:02 |
| @HeikoS | yep | 13:02 |
| @HeikoS | shallow copy | 13:02 |
| @HeikoS | which reminds us of the need for copy constructors | 13:02 |
| @HeikoS | that exactly do this | 13:03 |
| Saurabh7 | HeikoS: yes Iam doing it manually for now | 13:03 |
| Saurabh7 | or by virtual methods | 13:03 |
| @HeikoS | Saurabh7: do it via the copy constructor | 13:03 |
| @HeikoS | then we can go through a few one by one and make sure they work | 13:03 |
| @HeikoS | but let's start with the SVM for now | 13:03 |
| @HeikoS | should have a quick discussion with lisitsyn and wiking about how to do it best | 13:04 |
| -!- leagoetz [~leagoetz@host-92-0-162-192.as43234.net] has joined #shogun | 13:04 | |
| @HeikoS | leagoetz: hello | 13:04 |
| @HeikoS | Saurabh7: btw thanks for sending the pr | 13:04 |
| @HeikoS | useful for discussion | 13:04 |
| @HeikoS | will write some feedback soon | 13:04 |
| @HeikoS | just fixing something | 13:05 |
| @HeikoS | sanuj: and then also check your gist | 13:05 |
| sanuj | okay | 13:06 |
| @HeikoS | Saurabh7: btw what keeps your kmeans patch from merging? | 13:09 |
| Saurabh7 | HeikoS: It was big and github comparison was not displaying properly the file renames | 13:10 |
| Saurabh7 | so it looked messy | 13:10 |
| @HeikoS | Saurabh7: shall I create a feature branch for you to push into? | 13:11 |
| Saurabh7 | ok, will it help ? | 13:11 |
| @HeikoS | Saurabh7: but looking through the log, I think this should be possible to be merged | 13:11 |
| @HeikoS | Saurabh7: if all tests still pass etc | 13:11 |
| Saurabh7 | they wer passing iirc | 13:11 |
| Saurabh7 | I will ahve a look then if all looks good ? | 13:12 |
| -!- leagoetz [~leagoetz@host-92-0-162-192.as43234.net] has quit [Remote host closed the connection] | 13:15 | |
| Saurabh7 | ye everything was passing | 13:16 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Ping timeout: 260 seconds] | 13:16 | |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 13:17 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 13:17 | |
| Saurabh7 | ah yes we can check buildobt with feature branch | 13:18 |
| @HeikoS | Saurabh7: exactly | 13:19 |
| @HeikoS | so let me create one, then you can send the PR against it | 13:19 |
| @HeikoS | and then we can run on buildbot | 13:19 |
| @HeikoS | test | 13:19 |
| @HeikoS | and then merge if it is fine | 13:19 |
| Saurabh7 | cool | 13:20 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Ping timeout: 250 seconds] | 13:24 | |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 13:27 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 13:27 | |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 13:32 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 13:32 | |
| sanuj | lisitsyn, !!!! | 13:43 |
| lisitsyn | sup | 13:43 |
| sanuj | i need your help | 13:43 |
| lisitsyn | yeah sure | 13:43 |
| sanuj | i'm bringing in tags from aer to shogun | 13:44 |
| sanuj | got this error | 13:44 |
| sanuj | https://gist.github.com/sanuj/d58a90e327516c11449cfba597e62316 | 13:44 |
| sanuj | lisitsyn, i think the error is because of https://github.com/shogun-toolbox/aer/blob/master/src/aer/util/d_ptr.hpp#L19 | 13:44 |
| lisitsyn | yes | 13:45 |
| lisitsyn | it is because of absent operator= | 13:46 |
| lisitsyn | ehmm ok | 13:46 |
| sanuj | wait | 13:46 |
| sanuj | doorbell | 13:46 |
| sanuj | will be back in 2 mins | 13:46 |
| sanuj | lisitsyn, yes | 13:48 |
| sanuj | i'm back | 13:48 |
| sanuj | yeah i know it is because of absent operator= | 13:48 |
| sanuj | but i dunno how to fix this | 13:48 |
| sanuj | lisitsyn, what do you think | 13:48 |
| lisitsyn | sanuj: I think we've got to fix this multitask kernel normalizer | 13:49 |
| sanuj | okay | 13:50 |
| lisitsyn | sanuj: checking | 13:50 |
| lisitsyn | sanuj: you can try making CTaxonomy independent of CSGObject | 13:53 |
| lisitsyn | currently it inherits csgobject but probably there is no reason to do so | 13:53 |
| sanuj | lisitsyn, okay, i'll do it now | 13:54 |
| lisitsyn | src/shogun/transfer/multitask/MultitaskKernelTreeNormalizer.h | 13:54 |
| sanuj | lisitsyn, do we need to add something in dptr.h | 13:54 |
| lisitsyn | no, why? | 13:54 |
| sanuj | there is some TODO comment there | 13:55 |
| lisitsyn | sanuj: what does it tell? | 13:58 |
| sanuj | lisitsyn, TODO avoid allocation for stateless 'private's | 13:58 |
| lisitsyn | hah no idea | 14:00 |
| lisitsyn | nevermind | 14:00 |
| sanuj | lisitsyn, new error now | 14:00 |
| sanuj | lisitsyn, http://pastebin.com/tvP2ACxY | 14:01 |
| lisitsyn | sanuj: use SG_SERROR | 14:02 |
| -!- sonne|osx [~sonne@x4db3de16.dyn.telefonica.de] has quit [Quit: sonne|osx] | 14:09 | |
| @HeikoS | lisitsyn: hi | 14:13 |
| lisitsyn | HeikoS: ho | 14:13 |
| @HeikoS | lisitsyn: we need shallow copies | 14:13 |
| @HeikoS | use copy constructor for that? | 14:13 |
| @HeikoS | or what? | 14:13 |
| @HeikoS | can the plugin system solve that? | 14:14 |
| lisitsyn | yes | 14:14 |
| lisitsyn | not plugin | 14:14 |
| lisitsyn | but parameter | 14:14 |
| @HeikoS | parameter? | 14:14 |
| lisitsyn | HeikoS: what shallow copy? | 14:14 |
| @HeikoS | lisitsyn: we need this | 14:14 |
| @HeikoS | I have a machine | 14:14 |
| @HeikoS | want to create a copy, with exactly the same parameters | 14:14 |
| lisitsyn | can you make it immutable? | 14:14 |
| lisitsyn | why shallow? | 14:15 |
| @HeikoS | but then change the feautres | 14:15 |
| @HeikoS | another example is feature | 14:15 |
| @HeikoS | s | 14:15 |
| lisitsyn | it would work quite good if you can make the copy immutable | 14:15 |
| @HeikoS | need to create a copy of the feautre object | 14:15 |
| @HeikoS | but then should have the same underlying memory | 14:15 |
| @HeikoS | yeah would be ok | 14:16 |
| @HeikoS | since it uses the same memory | 14:16 |
| @HeikoS | but I dont quite know what you mean by that | 14:16 |
| @HeikoS | and why it would help | 14:16 |
| lisitsyn | because it would be a mess | 14:16 |
| lisitsyn | if you have two objects | 14:16 |
| lisitsyn | pointing to the same parameters | 14:16 |
| lisitsyn | you change X and Z changes as well | 14:16 |
| lisitsyn | HeikoS: how soon would you need it? | 14:17 |
| lisitsyn | HeikoS: sanuj can work it out once parameters are integrated I think | 14:22 |
| lisitsyn | but it would be available end of next week or so | 14:22 |
| sanuj | lisitsyn, what do you mean by use sg_error? | 14:22 |
| lisitsyn | sanuj: SG_ERROR -> SG_SERROR | 14:23 |
| sanuj | i did #include SGIO.h | 14:23 |
| sanuj | oh | 14:23 |
| sanuj | lisitsyn, it worked ! | 14:24 |
| lisitsyn | cool | 14:24 |
| Saurabh7 | lisitsyn: by parameters you mean ? | 14:24 |
| Saurabh7 | is kernel parameter of svm ? | 14:24 |
| sanuj | lisitsyn, why are dptr unique_ptr? | 14:24 |
| lisitsyn | sanuj: exactly to avoid unwanted copies | 14:24 |
| Saurabh7 | beacuse right now i use diferent kernel for each object | 14:24 |
| lisitsyn | sanuj: to avoid having two objects getting the same implementation | 14:25 |
| sanuj | okay | 14:25 |
| lisitsyn | Saurabh7: yeah, a kernel would be a parameter of svm | 14:25 |
| lisitsyn | they can be different | 14:25 |
| lisitsyn | parameter = member field | 14:26 |
| sanuj | lisitsyn, but why do we need to strictly have one unique pointer own a resource | 14:26 |
| lisitsyn | sanuj: which alternatives are you comparing to? | 14:26 |
| sanuj | if we had not used unique_ptr, the error would not have occurred right? | 14:27 |
| lisitsyn | sanuj: yes but this way we detected wrong usages of sgobjects | 14:27 |
| sanuj | lisitsyn, so correct usage is to have only one pointer point to a particular object? | 14:28 |
| lisitsyn | sanuj: yeah | 14:28 |
| lisitsyn | sanuj: we do not use value semantics | 14:28 |
| lisitsyn | we use pointer semantics | 14:28 |
| @HeikoS | lisitsyn: it is not urget | 14:28 |
| @HeikoS | lisitsyn: we can do things by hand for now | 14:29 |
| sanuj | lisitsyn, thanks :) | 14:29 |
| lisitsyn | HeikoS: ok then we will keep it in mind | 14:29 |
| @HeikoS | lisitsyn: so for now we just do copy ctor | 14:29 |
| sanuj | lisitsyn, i'll write some unit tests and run them | 14:29 |
| lisitsyn | HeikoS: there would be some shallow copy function available | 14:29 |
| sanuj | for tags | 14:29 |
| @HeikoS | lisitsyn: ? | 14:29 |
| lisitsyn | sanuj: would be cool! | 14:29 |
| @HeikoS | what do you mean? | 14:29 |
| lisitsyn | HeikoS: obj.view() may be | 14:29 |
| lisitsyn | this would create a copy | 14:29 |
| @HeikoS | lisitsyn: no | 14:29 |
| @HeikoS | ah | 14:29 |
| @HeikoS | well | 14:30 |
| lisitsyn | but immutable one | 14:30 |
| lisitsyn | you can get | 14:30 |
| lisitsyn | but can't set | 14:30 |
| @HeikoS | I see | 14:30 |
| @HeikoS | yes | 14:30 |
| @HeikoS | good | 14:30 |
| @HeikoS | yeah thats ok for our purpose | 14:30 |
| @HeikoS | we just want this multicore paradigma to work | 14:30 |
| @HeikoS | that is: | 14:31 |
| @HeikoS | i have one machine and set of features | 14:31 |
| @HeikoS | and then want to train/test in different threads | 14:31 |
| @HeikoS | so I need copies on the machine | 14:31 |
| @HeikoS | of | 14:31 |
| @HeikoS | and the features | 14:31 |
| @HeikoS | the feature copies will then get different subsets assigned | 14:31 |
| lisitsyn | ah cool | 14:31 |
| @HeikoS | and the machines use their own state to train | 14:31 |
| @HeikoS | lisitsyn: but the point is | 14:31 |
| @HeikoS | they have to have their own variables | 14:32 |
| @HeikoS | (state) | 14:32 |
| @HeikoS | as it is change by training | 14:32 |
| lisitsyn | yeah I start to see a problem | 14:32 |
| @HeikoS | so immutable maybe not | 14:32 |
| @HeikoS | we actually need a copy of the object, which just re-uses the feature matrix memory | 14:32 |
| lisitsyn | HeikoS: well ok no problem | 14:32 |
| lisitsyn | we can create mutable copy | 14:32 |
| @HeikoS | but like the alpha vector of a kernel machine has to be different for each copy | 14:32 |
| lisitsyn | we just inherit all parameters | 14:32 |
| Saurabh7 | and also kernel ? | 14:32 |
| lisitsyn | yeah everything it has | 14:33 |
| Saurabh7 | I mean the kernel has to diferent too for each copy | 14:33 |
| lisitsyn | why? | 14:33 |
| lisitsyn | ah | 14:33 |
| Saurabh7 | beacuse ti can only looka t one subset at a time | 14:34 |
| @HeikoS | ay | 14:34 |
| lisitsyn | shheatz | 14:34 |
| @HeikoS | yeah the kernel also has to be copied | 14:34 |
| @HeikoS | so its not really shallow copy | 14:34 |
| lisitsyn | haha | 14:34 |
| @HeikoS | the only thing that is shared | 14:34 |
| @HeikoS | is the feature memory | 14:34 |
| Saurabh7 | is feature matrix | 14:34 |
| @HeikoS | how to do that | 14:34 |
| Saurabh7 | yep | 14:34 |
| lisitsyn | well you still copy | 14:34 |
| @HeikoS | it is not a deep copy either | 14:34 |
| @HeikoS | as we dont want to duplicate feature data | 14:34 |
| lisitsyn | but then clone kernel | 14:34 |
| Saurabh7 | just new feature obj pointing at the feature matrix | 14:35 |
| @HeikoS | cloning kernel close the assigned features | 14:35 |
| lisitsyn | you just explicitly re-clone what you need | 14:35 |
| @HeikoS | lisitsyn: no good | 14:35 |
| lisitsyn | why? | 14:35 |
| @HeikoS | lisitsyn: has to work for every machine we have | 14:35 |
| @HeikoS | lisitsyn: so need to implement this via virtual calls | 14:35 |
| lisitsyn | yeah but all this kernel stuff has to be implemented somewhere | 14:36 |
| lisitsyn | do you want something really generic? | 14:36 |
| @HeikoS | lisitsyn: what is "kernel stuff" | 14:36 |
| lisitsyn | like copying kernels blabla | 14:36 |
| @HeikoS | yeah | 14:36 |
| @HeikoS | mmh | 14:36 |
| @HeikoS | maybe deep copy is best after all | 14:36 |
| lisitsyn | I mean it sounds like something adhoc | 14:36 |
| @HeikoS | that is: copy feature data as well? | 14:36 |
| @HeikoS | but no | 14:36 |
| @HeikoS | think about precomputed kernel matrix or kernel cache | 14:37 |
| lisitsyn | HeikoS: we can make features copy-on-write | 14:37 |
| lisitsyn | once you change it you copy it | 14:37 |
| @HeikoS | I mean we have to deep copy everything, apart from features, which need to be shallow copied | 14:37 |
| lisitsyn | HeikoS: we can always do deep copies | 14:37 |
| lisitsyn | but support sharing of immutable stuff | 14:37 |
| @HeikoS | lisitsyn: yeah | 14:38 |
| @HeikoS | that would be the way | 14:38 |
| @HeikoS | lisitsyn: how to do that? | 14:38 |
| lisitsyn | I mean as long as you don't change the matrix | 14:38 |
| lisitsyn | HeikoS: well any method that can change features | 14:38 |
| lisitsyn | would copy it | 14:38 |
| @HeikoS | yeah | 14:39 |
| @HeikoS | lisitsyn: so if we had proper setup of read/write methods | 14:39 |
| lisitsyn | yes | 14:39 |
| @HeikoS | and how is that done? | 14:39 |
| @HeikoS | copy on write? | 14:39 |
| lisitsyn | HeikoS: well any operation that writes also copies | 14:39 |
| lisitsyn | I mean it copies before writing | 14:40 |
| @HeikoS | lisitsyn: example? | 14:40 |
| lisitsyn | ok | 14:40 |
| lisitsyn | say you have features X | 14:40 |
| lisitsyn | 1 2 3 4 | 14:40 |
| lisitsyn | 2 3 4 5 | 14:40 |
| lisitsyn | you can assign Z = X | 14:40 |
| lisitsyn | they would still use the same matrix | 14:40 |
| lisitsyn | once you want to change Z | 14:41 |
| lisitsyn | Z[1,3] = 2 | 14:41 |
| lisitsyn | or anything like that | 14:41 |
| lisitsyn | whole matrix is copied | 14:41 |
| lisitsyn | and then changed | 14:41 |
| lisitsyn | so now they are pointing to different matrices | 14:41 |
| @HeikoS | and Z is replaced with copy? | 14:41 |
| lisitsyn | yes | 14:41 |
| lisitsyn | now we have two matrices | 14:41 |
| @HeikoS | I see the idea but I dont quite see how this is implemented | 14:41 |
| @HeikoS | seems tricky | 14:41 |
| lisitsyn | well it is just sgmatrix | 14:42 |
| lisitsyn | with all non-const methods | 14:42 |
| @HeikoS | I see | 14:42 |
| lisitsyn | spawning a copy | 14:42 |
| @HeikoS | so we would do this on all SG* structures that actually manage memory | 14:42 |
| @HeikoS | and the all CSGObject are deep copied | 14:42 |
| lisitsyn | yeah | 14:42 |
| lisitsyn | it sounds good to me | 14:43 |
| @HeikoS | lisitsyn: so then the deep copy needs to copy the SG* stuff in this way | 14:43 |
| @HeikoS | so it is not really deep copy | 14:43 |
| @HeikoS | but another function | 14:43 |
| @HeikoS | deepView | 14:43 |
| @HeikoS | ;) | 14:43 |
| lisitsyn | maybe | 14:43 |
| @HeikoS | mmh | 14:44 |
| @HeikoS | lisitsyn: need prototype | 14:44 |
| @HeikoS | lisitsyn: Ill draft one | 14:44 |
| @HeikoS | Saurabh7: | 14:44 |
| @HeikoS | for now just use clone then | 14:44 |
| @HeikoS | we can start with that and then optimise this later | 14:44 |
| lisitsyn | HeikoS: yeah let me know | 14:44 |
| Saurabh7 | clone does cop the features too i think | 14:45 |
| @HeikoS | lisitsyn: yes | 14:45 |
| Saurabh7 | for machine->clone | 14:45 |
| lisitsyn | leaving for a bit now, late breakfast | 14:45 |
| @HeikoS | ehm | 14:45 |
| @HeikoS | Saurabh7: yes | 14:45 |
| @HeikoS | lisitsyn: ok enjoy | 14:45 |
| @HeikoS | Saurabh7: we do deep copy for now | 14:45 |
| @HeikoS | that is: everything is duplicated | 14:45 |
| @HeikoS | since we cannot distinguish between read and write operations at the moment | 14:45 |
| @HeikoS | Saurabh7: I will try to draft something for that, we can then update the version with close | 14:45 |
| @HeikoS | clone | 14:45 |
| @HeikoS | Saurabh7: so this should make things very easy for you | 14:46 |
| @HeikoS | Saurabh7: you just clone the machine | 14:46 |
| Saurabh7 | with clone it will be completely generic | 14:46 |
| @HeikoS | then ask for its features (which were also cloned) | 14:46 |
| Saurabh7 | ust use Machine * | 14:46 |
| @HeikoS | and add subsets there | 14:46 |
| @HeikoS | exactly | 14:46 |
| @HeikoS | change it and see whether it works for SVM | 14:46 |
| Saurabh7 | yes it works | 14:46 |
| Saurabh7 | I had tried already | 14:46 |
| @HeikoS | ok cool | 14:46 |
| @HeikoS | then send a PR for discussion | 14:47 |
| -!- leagoetz [~leagoetz@host-92-0-162-192.as43234.net] has joined #shogun | 14:47 | |
| @HeikoS | Saurabh7: once things are discussed, we have to test this with as many machine as possible | 14:47 |
| @HeikoS | Saurabh7: might even want to write an automatically generated unit test for every machine instance for that | 14:47 |
| Saurabh7 | HeikoS: clone version ? | 14:47 |
| @HeikoS | Saurabh7: and see what things break | 14:47 |
| @HeikoS | Saurabh7: ?? | 14:48 |
| @HeikoS | sanuj: you still need help? | 14:48 |
| Saurabh7 | i mean test for the clone version ? | 14:48 |
| Saurabh7 | ok i will do that | 14:48 |
| sanuj | HeikoS, no :) | 14:48 |
| sanuj | HeikoS, sergey helped me | 14:48 |
| @HeikoS | sanuj: great! :) | 14:48 |
| @HeikoS | Saurabh7: yeah, something that runs every machine with x-validation | 14:48 |
| @HeikoS | Saurabh7: but later | 14:48 |
| @HeikoS | start with a simple test | 14:49 |
| @HeikoS | and we can discuss in the PR | 14:49 |
| @HeikoS | Saurabh7: one thing also is: parallelise over runs as well | 14:49 |
| @HeikoS | not just folds | 14:49 |
| @HeikoS | that is, the pragma mp parallel is around the runs loop | 14:49 |
| Saurabh7 | ok | 14:49 |
| Saurabh7 | yep | 14:49 |
| @HeikoS | and the pragma omp for around the fold loop | 14:49 |
| @HeikoS | Saurabh7: and then pray that nothing explodes ;) | 14:50 |
| @HeikoS | Saurabh7: maybe it is not the best idea actually | 14:50 |
| Saurabh7 | that will intorduce | 14:50 |
| @HeikoS | since the permutation of features changes | 14:50 |
| Saurabh7 | new complciations | 14:50 |
| @HeikoS | but it is read only | 14:50 |
| Saurabh7 | yep not sure we can do on both | 14:50 |
| @HeikoS | I think should work | 14:50 |
| @HeikoS | but start with fold parallel | 14:50 |
| @HeikoS | Saurabh7: thanks for your cookbook | 14:58 |
| @HeikoS | just put some comments | 14:58 |
| @HeikoS | should be quick to fix | 14:58 |
| Saurabh7 | HeikoS: Thanks ! | 14:59 |
| Saurabh7 | ok i need to pull data | 15:00 |
| @HeikoS | Saurabh7: yeah | 15:00 |
| @HeikoS | git submodule init | 15:00 |
| @HeikoS | git submodule update | 15:01 |
| @HeikoS | then put the generated integration test file from your LDA to the data folder | 15:01 |
| @HeikoS | send a patch for data | 15:01 |
| @HeikoS | update the data version in your cookbook commit | 15:01 |
| Saurabh7 | yep | 15:02 |
| Saurabh7 | HeikoS: Is parallelize over runs alwasy useful ? | 15:03 |
| Saurabh7 | if onyl 2 core then it wont parallelize folds | 15:03 |
| Saurabh7 | ust thinking | 15:03 |
| @HeikoS | Saurabh7: yes it is | 15:03 |
| @HeikoS | imagine you have 4 cores | 15:03 |
| @HeikoS | and do 3 folds | 15:03 |
| @HeikoS | then one core waits | 15:03 |
| @HeikoS | whereas it could already start doing the first fold of the second run | 15:04 |
| @HeikoS | so what you need to do, is to precompute the folds for each run | 15:04 |
| @HeikoS | folds_for_all_runs = precompute() | 15:04 |
| @HeikoS | #pragma omp parallel | 15:04 |
| @HeikoS | for loop over runs | 15:04 |
| @HeikoS | #pragma omp for | 15:05 |
| @HeikoS | loop over folds | 15:05 |
| @HeikoS | see what I mean? | 15:05 |
| Saurabh7 | ok | 15:05 |
| Saurabh7 | I was assuming it will assign all threads to runs | 15:05 |
| Saurabh7 | and eventually fold loop will run onl on 1 thread | 15:05 |
| @HeikoS | no no | 15:07 |
| @HeikoS | each thread will have exactly one fold | 15:07 |
| @HeikoS | but you dont do the runs sequentially | 15:07 |
| @HeikoS | imagine you create a pool of tasks (here one task is one fold) | 15:07 |
| @HeikoS | then you can add the tasks of all runs | 15:07 |
| @HeikoS | and make your threadpool solve all of them, without fixing the order | 15:08 |
| @HeikoS | pragma omp parallel is just like starting a thread pool | 15:08 |
| @HeikoS | and #pragma omp for just adds tasks to that pool | 15:08 |
| @HeikoS | Saurabh7: did I confuse you with that? :D | 15:11 |
| Saurabh7 | HeikoS: yes :D | 15:11 |
| @HeikoS | Saurabh7: ok then, let us start with parallelising over folds only | 15:11 |
| @HeikoS | polish a bit | 15:11 |
| Saurabh7 | its ok i will get it eventually | 15:11 |
| @HeikoS | and then change | 15:11 |
| Saurabh7 | yup | 15:11 |
| @HeikoS | ok? | 15:11 |
| @HeikoS | cool | 15:12 |
| @HeikoS | Saurabh7: I gotta go now, ping me when you update the PR | 15:12 |
| -!- leagoetz [~leagoetz@host-92-0-162-192.as43234.net] has quit [Remote host closed the connection] | 15:16 | |
| shogun-buildbot | build #16 of xenial - libshogun is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/xenial%20-%20libshogun/builds/16 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 15:21 |
| shogun-buildbot | build #2878 of bsd1 - libshogun is complete: Failure [failed configure] Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/2878 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 15:22 |
| sanuj | lisitsyn, are we going to have macros defined for Type<int>, Type<double> etc | 15:35 |
| lisitsyn | what's type? | 15:35 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.] | 15:35 | |
| sanuj | lisitsyn, https://github.com/shogun-toolbox/aer/blob/master/src/aer/base/type.hpp | 15:37 |
| sanuj | used to get parameters | 15:37 |
| lisitsyn | hmm is it really needed? | 15:37 |
| lisitsyn | don't remember how I used it | 15:37 |
| sanuj | get(string name, type) | 15:37 |
| sanuj | i was wondering the same | 15:37 |
| sanuj | lisitsyn, usage is here https://github.com/shogun-toolbox/aer/blob/master/src/aer/base/object.hpp#L44 | 15:38 |
| lisitsyn | ohh | 15:39 |
| lisitsyn | ok | 15:39 |
| lisitsyn | yes | 15:39 |
| lisitsyn | makes sense | 15:39 |
| lisitsyn | sanuj: but not typedefs | 15:39 |
| lisitsyn | erghh | 15:39 |
| sanuj | cool | 15:39 |
| lisitsyn | not macroses | 15:39 |
| lisitsyn | but typedefs | 15:39 |
| sanuj | okay | 15:39 |
| sanuj | :) | 15:39 |
| lisitsyn | typedef Type<int> IntType; | 15:39 |
| lisitsyn | ... | 15:39 |
| sanuj | lisitsyn, this can be done in swig | 15:40 |
| lisitsyn | or something like that | 15:40 |
| lisitsyn | yeah | 15:40 |
| sanuj | for c++ no typedefs? | 15:40 |
| lisitsyn | idk if they are needed now | 15:41 |
| sanuj | lisitsyn, i won't put them now | 15:41 |
| sanuj | let's see later | 15:41 |
| lisitsyn | ok | 15:41 |
| sanuj | Type<int> int_type(); gauss_kernel->get("distance", int_type); | 15:48 |
| sanuj | lisitsyn, gives error ^ | 15:48 |
| lisitsyn | int_type()?? | 15:48 |
| lisitsyn | why () here? you defined a pointer to function that returns Type<int> this way | 15:49 |
| sanuj | lisitsyn, Type<int> int_type; gives seg fault | 15:50 |
| sanuj | lisitsyn, int_type() --> calls constructor to create object right? | 15:50 |
| lisitsyn | no | 15:50 |
| lisitsyn | Type<int>(); calls constructor | 15:50 |
| lisitsyn | T something(); is a pointer to function that returns T | 15:50 |
| sanuj | lisitsyn, Tag<int> distance("distance"); | 15:51 |
| lisitsyn | that's correct | 15:51 |
| lisitsyn | but not () | 15:51 |
| sanuj | okay | 15:51 |
| sanuj | lisitsyn, so do i use new then? | 15:51 |
| lisitsyn | no | 15:52 |
| lisitsyn | this can't segfault -> Type<int> int_type; | 15:52 |
| lisitsyn | you sure? | 15:52 |
| sanuj | this segfaults gauss_kernel->get("distance", int_type) | 15:52 |
| sanuj | when i do abovee | 15:52 |
| sanuj | lisitsyn, Type<int> int_type; | 15:54 |
| sanuj | EXPECT_EQ(gauss_kernel->get("distance", int_type), 10); | 15:54 |
| sanuj | ^^ seg faults when i run the unit test | 15:54 |
| lisitsyn | yeah well at least it compiles right? ;) | 15:54 |
| sanuj | yeah compiles | 15:54 |
| sanuj | :) | 15:54 |
| lisitsyn | I am not sure what can be wrong | 15:54 |
| lisitsyn | try to run it using valgrind | 15:55 |
| sanuj | lisitsyn, how to run one individual test | 15:55 |
| lisitsyn | go to build/ | 15:55 |
| lisitsyn | ok don't remember | 15:56 |
| lisitsyn | a sec | 15:56 |
| lisitsyn | recompiling | 16:02 |
| lisitsyn | in a bit | 16:02 |
| sanuj | lisitsyn, cool | 16:02 |
| sanuj | lisitsyn, you have more time on weekends :) | 16:03 |
| sanuj | for shogun | 16:03 |
| lisitsyn | sanuj: yeah sometimes | 16:04 |
| shogun-buildbot | build #7 of FC23 - libshogun - aarch64 is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/FC23%20-%20libshogun%20-%20aarch64/builds/7 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 16:04 |
| lisitsyn | well at least not at job | 16:04 |
| lisitsyn | which saves me from a lot of distractions :D | 16:04 |
| sanuj | eyah | 16:04 |
| sanuj | yeah | 16:04 |
| lisitsyn | sanuj: do you have sth like | 16:25 |
| lisitsyn | tests/unit/shogun-unit-tests | 16:25 |
| lisitsyn | in your build? | 16:26 |
| sanuj | lisitsyn, yes | 16:26 |
| lisitsyn | sanuj: --gtest_filter=*Something* | 16:27 |
| lisitsyn | that's the option you need | 16:27 |
| lisitsyn | just some regexp | 16:27 |
| sanuj | alright | 16:28 |
| sanuj | lisitsyn, i'm going to have dinner | 16:28 |
| sanuj | will try after that | 16:28 |
| lisitsyn | ok | 16:28 |
| lisitsyn | won't be there probably | 16:28 |
| lisitsyn | so if you need to debug it | 16:28 |
| lisitsyn | run valgrind tests/unit/shogun-unit-tests | 16:28 |
| lisitsyn | or use gdb | 16:28 |
| sanuj | lisitsyn, okay | 16:28 |
| sanuj | i'll start a PR before sleeping | 16:28 |
| lisitsyn | ok cool | 16:29 |
| sanuj | you can give feedback there | 16:29 |
| lisitsyn | sanuj: can you email me once done? | 16:29 |
| sanuj | lisitsyn, yes i'll | 16:29 |
| lisitsyn | I can't follow all the PRs so they go directly to some kind of /dev/null | 16:29 |
| lisitsyn | :D | 16:29 |
| sanuj | haha | 16:29 |
| sanuj | yeah i'll send you a mail | 16:29 |
| sanuj | see you tomorrow | 16:29 |
| lisitsyn | see you | 16:30 |
| -!- sanuj [~sanuj@117.203.3.118] has quit [Remote host closed the connection] | 16:31 | |
| -!- sanuj [~sanuj@117.203.3.118] has joined #shogun | 17:20 | |
| -!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 244 seconds] | 17:44 | |
| shogun-buildbot | build #276 of deb1 - libshogun - PR is complete: Failure [failed cookbook] Build details are at http://buildbot.shogun-toolbox.org/builders/deb1%20-%20libshogun%20-%20PR/builds/276 blamelist: sanuj | 19:36 |
| -!- lupinix [~quassel@fedora/lupinix] has quit [Remote host closed the connection] | 19:41 | |
| sanuj | lisitsyn, there? | 19:46 |
| -!- lupinix [~quassel@fedora/lupinix] has joined #shogun | 19:46 | |
| lisitsyn | ыф | 19:46 |
| lisitsyn | sanuj: apparently yes | 19:46 |
| sanuj | got time? | 19:46 |
| lisitsyn | yes | 19:46 |
| sanuj | i sent you a mail just now | 19:46 |
| sanuj | i was about to go to sleep | 19:46 |
| -!- lupinix [~quassel@fedora/lupinix] has quit [Remote host closed the connection] | 19:46 | |
| sanuj | but we can fix the bug if you have time | 19:46 |
| sanuj | lisitsyn, check gmail inbox | 19:47 |
| lisitsyn | sanuj: its up to you | 19:47 |
| -!- lupinix [~quassel@v22014041761818086.yourvserver.net] has joined #shogun | 19:47 | |
| -!- lupinix [~quassel@v22014041761818086.yourvserver.net] has quit [Changing host] | 19:47 | |
| -!- lupinix [~quassel@fedora/lupinix] has joined #shogun | 19:47 | |
| lisitsyn | uhmm for some reason this branch looks crappy | 19:47 |
| lisitsyn | I see other commits | 19:47 |
| sanuj | yes | 19:47 |
| lisitsyn | okay this is fixable | 19:47 |
| sanuj | just see my commit | 19:47 |
| -!- lupinix [~quassel@fedora/lupinix] has quit [Remote host closed the connection] | 19:48 | |
| -!- lupinix [~quassel@fedora/lupinix] has joined #shogun | 19:49 | |
| lisitsyn | sanuj: where does it fail? | 19:52 |
| lisitsyn | so get by tag works | 19:52 |
| lisitsyn | but get by string + type does not? | 19:52 |
| sanuj | lisitsyn, yes get by string + type does not work | 19:53 |
| lisitsyn | sanuj: ok have to go now, let me try to reproduce that on my machine | 19:54 |
| lisitsyn | I'll put a comment then | 19:54 |
| sanuj | lisitsyn, you can try to replicate this in aer | 19:54 |
| sanuj | it will be easy | 19:54 |
| sanuj | to debug | 19:54 |
| sanuj | even i'm going to sleep | 19:55 |
| sanuj | will see your comments tomorrow | 19:55 |
| sanuj | lisitsyn, goodnight | 19:55 |
| -!- sanuj [~sanuj@117.203.3.118] has quit [Remote host closed the connection] | 19:56 | |
| shogun-buildbot | build #277 of deb1 - libshogun - PR is complete: Success [build successful] Build details are at http://buildbot.shogun-toolbox.org/builders/deb1%20-%20libshogun%20-%20PR/builds/277 | 21:27 |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 21:51 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 21:51 | |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 22:01 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 22:01 | |
| @wiking | y0 | 22:45 |
| @wiking | HeikoS: how's the ccache? | 22:45 |
| @wiking | :) | 22:45 |
| shogun-buildbot | build #2879 of bsd1 - libshogun is complete: Failure [failed configure] Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/2879 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 22:49 |
| shogun-buildbot | build #17 of xenial - libshogun is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/xenial%20-%20libshogun/builds/17 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 22:49 |
| -!- travis-ci [~travis-ci@ec2-54-211-50-200.compute-1.amazonaws.com] has joined #shogun | 23:21 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/133639681 | 23:21 |
| -!- travis-ci [~travis-ci@ec2-54-211-50-200.compute-1.amazonaws.com] has left #shogun [] | 23:21 | |
| shogun-buildbot | build #8 of FC23 - libshogun - aarch64 is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/FC23%20-%20libshogun%20-%20aarch64/builds/8 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 23:31 |
| -!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 252 seconds] | 23:32 | |
| -!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.] | 23:37 | |
| -!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has joined #shogun | 23:38 | |
| -!- OXPHOS [9d8b131c@gateway/web/freenode/ip.157.139.19.28] has quit [Client Quit] | 23:39 | |
| --- Log closed Sun May 29 00:00:16 2016 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!