| --- Log opened Tue Sep 06 00:00:37 2016 | ||
| -!- yorkerlin [ae73d4c2@gateway/web/freenode/ip.174.115.212.194] has joined #shogun | 04:03 | |
| yorkerlin | ping wiking | 04:04 |
|---|---|---|
| @wiking | pong what's up | 04:04 |
| yorkerlin | about the optimization framework | 04:04 |
| @wiking | yep | 04:05 |
| yorkerlin | r you working on kkt solvers? | 04:05 |
| @wiking | heheh wish | 04:05 |
| @wiking | till now it was pushing bits areound | 04:05 |
| @wiking | for gsoc | 04:05 |
| @wiking | and some javascript related stuff with shogun + windows port of shogun | 04:06 |
| @wiking | so not yet | 04:06 |
| yorkerlin | ic. so we can discuss about the framework | 04:06 |
| @wiking | definitely | 04:07 |
| @wiking | so the thing is | 04:07 |
| @wiking | that since now we have a full refact | 04:07 |
| @wiking | of the linalg backend | 04:07 |
| @wiking | we should rely on that | 04:07 |
| @wiking | heavily | 04:07 |
| @wiking | so for any matrix factorization | 04:07 |
| @wiking | or basically any linalg operations we should use the new linalg backend | 04:08 |
| @wiking | in the beginning it's going to be a pain a bit | 04:08 |
| @wiking | because it'll require us to implement some of the linalg methods of our own | 04:08 |
| @wiking | but it's worth it in the long run | 04:08 |
| @wiking | because then one will be able to choose between the backend being used for linalg calculations (like cpu, gpu etc.) | 04:08 |
| yorkerlin | do we assume that we can use the linalg lib in the opt framework | 04:09 |
| @wiking | yes | 04:09 |
| @wiking | definitely | 04:09 |
| @wiking | as well as openmp | 04:09 |
| @wiking | and c++11 | 04:09 |
| yorkerlin | maybe we should list most of operators used in the opt framework | 04:10 |
| @wiking | for kkt there are usually LDL, QR factorization | 04:10 |
| @wiking | so basically matrix factorization | 04:10 |
| yorkerlin | I may need LDLT | 04:11 |
| yorkerlin | Heiko told me that now tensorflow supports API for building computational graphs in C++ | 04:12 |
| @wiking | yeah | 04:12 |
| @wiking | we could rely on that | 04:14 |
| yorkerlin | so we should have some backends for gradient computation (eg, tensorflow and hand gradient ) | 04:14 |
| @wiking | yep | 04:15 |
| @wiking | so in my mind | 04:15 |
| @wiking | if some of the heavy computation stuff we cannot only cover with linalg lib | 04:15 |
| @wiking | we should wrap it somehow | 04:16 |
| @wiking | so we could later easily factor it out | 04:16 |
| @wiking | and have a different implementation of it | 04:16 |
| @wiking | since now most of the case is the problem is that all the computation is hard-coded | 04:16 |
| @wiking | hence it's very hard to factor it out | 04:16 |
| @wiking | ok not very hard | 04:16 |
| @wiking | but cumbersome | 04:16 |
| @wiking | and nowadays if i look at libsvm, svmlight implementation | 04:16 |
| @wiking | my heart starts to bleed :) | 04:17 |
| yorkerlin | so add a tensorflow backend in the linalg lib? | 04:17 |
| @wiking | mmm i'm not so sure if we wanna add it there | 04:17 |
| @wiking | but i'll check how we can integrate it | 04:18 |
| @wiking | my only question is | 04:18 |
| @wiking | for example | 04:18 |
| @wiking | in case of TF | 04:18 |
| @wiking | would it be possible to distribute the computations? | 04:19 |
| @wiking | i mean it says "data flow graphs for scalable machine learning" | 04:19 |
| @wiking | but to be honest if one asks me i much more prefer an actor based framework for distributing the computations between nodes | 04:19 |
| yorkerlin | multiple GPUs? | 04:19 |
| @wiking | gpus, cpus whatever :) | 04:20 |
| yorkerlin | the focus of Shogun is single machine, right? | 04:21 |
| @wiking | mmm should not be :) | 04:21 |
| @wiking | i mean yes currently we only support that | 04:22 |
| @wiking | but would be good to start not only support that | 04:22 |
| @wiking | hence why i'm telling here that all the computations | 04:22 |
| @wiking | should be wrapped | 04:22 |
| @wiking | into some abstraction | 04:22 |
| @wiking | so that later having different way of doing the computations | 04:22 |
| @wiking | would be easy to change | 04:22 |
| yorkerlin | do we pay attention to PC clusters instead of HPC? | 04:24 |
| @wiking | yep | 04:24 |
| @wiking | but we should not focus now on | 04:25 |
| @wiking | how the clustering is being done | 04:25 |
| @wiking | just make it possible | 04:25 |
| @wiking | from the code point of view | 04:25 |
| yorkerlin | ic | 04:26 |
| yorkerlin | we should do a lot of refactoring. | 04:27 |
| @wiking | yep :) | 04:27 |
| yorkerlin | I will be back in 10 mins. | 04:28 |
| @wiking | k | 04:29 |
| @wiking | the best is to do what we did with the linalg refactor gsoc project | 04:29 |
| yorkerlin | does the linalg refactor project listed in the wiki? | 04:36 |
| yorkerlin | https://github.com/shogun-toolbox/shogun/wiki/GSoC_2015_project_linalg ? | 04:37 |
| @wiking | https://github.com/shogun-toolbox/shogun/wiki/Google%20Summer%20of%20Code%202016%20Projects | 04:38 |
| @wiking | https://github.com/shogun-toolbox/shogun/wiki/GSoC_2015_clean_up_infrastructure | 04:38 |
| @wiking | it was this one | 04:38 |
| @wiking | oxphos wrote a whole wiki page | 04:39 |
| @wiking | about the project | 04:39 |
| @wiking | you can find it here... lemme try to find t | 04:39 |
| @wiking | it | 04:39 |
| yorkerlin | we should investigate how to integrate tensoflow in the meantime | 04:40 |
| @wiking | yorkerlin, | 04:41 |
| @wiking | https://github.com/OXPHOS/shogun/wiki/GSoC-2016-Final-Report-:-The-Shogun-Detox | 04:41 |
| @wiking | this is the report of that project | 04:41 |
| yorkerlin | I will take a look at the report | 04:42 |
| yorkerlin | we may need a tensorflow backend if gradient information is needed. | 04:43 |
| yorkerlin | BTW, do you know this project, https://github.com/GPflow/GPflow | 04:45 |
| yorkerlin | GPflow uses the python API from tensorflow. BTW, matrix solvers are needed in GP. | 04:47 |
| yorkerlin | I mean that tensowflow should support many matrix solvers | 04:48 |
| yorkerlin | at least in the python API | 04:49 |
| yorkerlin | Once the C++ API is polished, we could use the C++ API to build a tensorflow backend. | 04:50 |
| @wiking | mmm | 04:57 |
| @wiking | but what's tensorflow giving | 04:57 |
| @wiking | in extra here? | 04:57 |
| yorkerlin | maybe we should have a gradient computation framework? | 04:58 |
| yorkerlin | for hand gradient, the linalg is used. for AD, tensorflow is used | 04:59 |
| @wiking | AD? | 04:59 |
| yorkerlin | automatic differentiation | 04:59 |
| @wiking | ah | 05:00 |
| @wiking | ok i need to dig in | 05:00 |
| @wiking | how we could use tf's computation graph | 05:00 |
| @wiking | via the c++ api | 05:00 |
| @wiking | because that support just recently has been released | 05:00 |
| @wiking | till now it was only via python api | 05:00 |
| yorkerlin | no. accodring to the release note | 05:01 |
| yorkerlin | https://github.com/tensorflow/tensorflow/releases | 05:01 |
| yorkerlin | Added graph-construction C API Major revision to the graph-construction C++ API | 05:02 |
| yorkerlin | I am looking for examples about how to use the C++/C API | 05:02 |
| @wiking | no? | 05:03 |
| @wiking | i mean it has been released | 05:03 |
| @wiking | recently | 05:03 |
| @wiking | or/ | 05:03 |
| @wiking | Added graph-construction C API | 05:03 |
| @wiking | this wasnt part till now | 05:04 |
| yorkerlin | I mean tensorflow now has C++/C API for graph construction | 05:04 |
| @wiking | yep | 05:04 |
| @wiking | that's what i mean that till now it wasn't there | 05:04 |
| @wiking | so i have 0 idea | 05:04 |
| @wiking | how the api is like | 05:04 |
| yorkerlin | yes. it is true. that is why I am looking for examples about the usage of the API | 05:05 |
| yorkerlin | There are two ways to add a tensorflow backend. | 05:07 |
| yorkerlin | one way is that each algorithm has at least two implemenations. one for hand gradient, the other for tensorflow. | 05:08 |
| yorkerlin | There are two ways to use tensorflow in shogun. one way is that each algorithm has at least two implemenations. one for hand gradient, the other for tensorflow. another way is to add a tensorflow backend in the linalg lib. | 05:09 |
| yorkerlin | do you think which way is better? or other suggestion? | 05:11 |
| yorkerlin | or an AD framework (eg, tensorflow backend, CNTK backend, and Stan backend) | 05:17 |
| yorkerlin | Let me say it again. | 05:24 |
| yorkerlin | 1 a tensorflow backend for the linalg; 2 an implementation for algorithms in shogun using tensorflow; 3 an AD framework which can use tensorflow, or CNTK, or STAN? | 05:25 |
| yorkerlin | I have to leave now. I will be in ICR again soon. | 05:26 |
| -!- yorkerlin [ae73d4c2@gateway/web/freenode/ip.174.115.212.194] has quit [Quit: Page closed] | 05:26 | |
| @wiking | lisitsyn, ping | 12:53 |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 13:16 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 13:16 | |
| @wiking | HeikoS: pong ping ping pong | 13:37 |
| @wiking | :) | 13:37 |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Ping timeout: 265 seconds] | 13:45 | |
| -!- sanuj [~sanuj@59.91.128.161] has joined #shogun | 14:02 | |
| -!- jones_ [5befb10e@gateway/web/freenode/ip.91.239.177.14] has joined #shogun | 14:44 | |
| -!- jones_ [5befb10e@gateway/web/freenode/ip.91.239.177.14] has quit [Ping timeout: 264 seconds] | 14:51 | |
| -!- jones_ [5befb10e@gateway/web/freenode/ip.91.239.177.14] has joined #shogun | 14:52 | |
| jones_ | Hi, is there anybody here who could help me with loading a trained CMulticlassLibSVM after saving it to a file? Saving seems to work fine, but I'm not sure how you are supposed to load it. I couldn't find an example on the web or in the docu. Here is my current approach http://pastebin.com/YHbyEKJk | 14:58 |
| @wiking | and i guess model is an std::string that contains the path for the model file? | 15:07 |
| jones_ | Oh sry, yes its just a path | 15:08 |
| @wiking | ok and what's the error? | 15:09 |
| jones_ | I get the following error: [ERROR] In file .../shogun/src/shogun/kernel/Kernel.h line 211: LinearKernel::kernel(): index out of Range: idx_a=240/240 idx_b=0/240 | 15:10 |
| @wiking | you get this while loading? | 15:10 |
| jones_ | No its occuring in apply_multiclass | 15:10 |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 15:11 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 15:11 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Client Quit] | 15:12 | |
| @wiking | ah | 15:12 |
| jones_ | But I'm not sure if I'm setting the kernel and responses correctly, how are you supposed to set those while predicting? | 15:12 |
| @wiking | mmm | 15:12 |
| @wiking | mmm | 15:24 |
| @wiking | but what are you actually applying? | 15:24 |
| @wiking | the kernel that was in the trained machine? | 15:24 |
| @wiking | shogunsvm->load(f); | 15:24 |
| @wiking | will overwrite your CMulticlassLibSVM | 15:24 |
| @wiking | so you should set the kernel | 15:24 |
| jones_ | I initialize the kernel before with the test data: kernel->init(testfeatures,testfeatures); | 15:25 |
| @wiking | yeah but you set the kernel of the CMulticlassLibSVM | 15:26 |
| @wiking | before | 15:26 |
| @wiking | the loading of the pretrained data | 15:26 |
| @wiking | *model | 15:26 |
| jones_ | shogunsvm->load(f); kernel->init(testfeatures,testfeatures); shogunsvm->set_kernel(kernel); CMulticlassLabels* labels=shogunsvm->apply_multiclass(); | 15:31 |
| jones_ | has the same effect, hm | 15:31 |
| jones_ | maybe the saved model isn't even right, the support vector indices are kinda high | 15:32 |
| @wiking | dunno when and how you have created the svm | 15:37 |
| jones_ | I did an apply_multiclass with the trained svm and I get repsonses, so it should work | 15:40 |
| @wiking | mmm | 15:41 |
| @wiking | you could btw do a shogunsvm->apply_multiclass(testfeatures) | 15:41 |
| @wiking | right? | 15:41 |
| jones_ | yes, I also tried a single apply_one(0). Same error | 15:42 |
| @wiking | apply_one(0) ? | 15:43 |
| @wiking | it will give you the same error | 15:43 |
| @wiking | if you give a null input | 15:43 |
| jones_ | how am I supposed to input the test data? kernel->init? | 15:44 |
| @wiking | no | 15:44 |
| @wiking | just have the testfeatures | 15:45 |
| jones_ | so shogunsvm->apply_multiclass(testfeatures); ? | 15:45 |
| @wiking | virtual CMulticlassLabels* apply_multiclass(CFeatures* data=NULL); | 15:45 |
| @wiking | and yes | 15:45 |
| jones_ | and how do I set the kernel then? | 15:45 |
| jones_ | If I don't initialize the kernel I will get errors about its left and right hand sizes | 15:47 |
| jones_ | *sides | 15:47 |
| jones_ | [ERROR] In file .../shogun/src/shogun/multiclass/MulticlassSVM.cpp line 86: MulticlassLibSVM: No left hand side specified | 15:49 |
| @wiking | kernel should have been serialized | 15:55 |
| @wiking | already | 15:55 |
| @wiking | as part of the model | 15:55 |
| jones_ | I get: "shogun/src/shogun/multiclass/MulticlassSVM.cpp line 82: No kernel assigned! " if i don't set an explicit kernel and as far I can follow it, CMulticlassSVM::load doesn't seem to set the kernel | 16:02 |
| -!- sanuj [~sanuj@59.91.128.161] has quit [Ping timeout: 255 seconds] | 16:08 | |
| @wiking | mmm that's actually a problem :( | 16:15 |
| @wiking | jones_: only good news i have now for you | 16:23 |
| @wiking | that soon we'll gonna have a new release :) | 16:23 |
| @wiking | and that has new serialization backend :) | 16:23 |
| jones_ | hooray, thanks for your help | 16:25 |
| @wiking | well | 16:26 |
| @wiking | not much i could helped it seems | 16:26 |
| jones_ | you answered my questions, its help enough ;) and you could report the bug. anyway I must leave now, thanks again and bye | 16:28 |
| @wiking | jones_: could you fill a bug | 16:30 |
| @wiking | in issues plz | 16:30 |
| @wiking | would be really appreciated | 16:30 |
| @wiking | and you would have an update | 16:30 |
| @wiking | as soon as it's fixed | 16:30 |
| jones_ | I will look into it | 16:31 |
| jones_ | cya | 16:31 |
| @wiking | thnmx | 16:31 |
| -!- jones_ [5befb10e@gateway/web/freenode/ip.91.239.177.14] has quit [Quit: Page closed] | 16:31 | |
| -!- sanuj [~sanuj@59.91.128.161] has joined #shogun | 16:46 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 18:35 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 18:35 | |
| -!- sanuj [~sanuj@59.91.128.161] has quit [Ping timeout: 240 seconds] | 18:37 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Quit: Leaving.] | 18:49 | |
| -!- sanuj [~sanuj@117.204.253.59] has joined #shogun | 18:50 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 19:01 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 19:01 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Quit: Leaving.] | 20:48 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 20:52 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 20:52 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Quit: Leaving.] | 20:57 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has joined #shogun | 21:12 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 21:12 | |
| -!- sanuj [~sanuj@117.204.253.59] has quit [Remote host closed the connection] | 21:12 | |
| -!- HeikoS [~heiko@host-92-0-178-129.as43234.net] has quit [Client Quit] | 21:14 | |
| --- Log closed Wed Sep 07 00:00:38 2016 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!