| --- Log opened Wed Apr 27 00:00:31 2016 | ||
| -!- HeikoS [~heiko@p5797249D.dip0.t-ipconnect.de] has joined #shogun | 00:03 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 00:03 | |
| -!- travis-ci [~travis-ci@ec2-54-87-104-96.compute-1.amazonaws.com] has joined #shogun | 00:11 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/125959349 | 00:11 |
|---|---|---|
| -!- travis-ci [~travis-ci@ec2-54-87-104-96.compute-1.amazonaws.com] has left #shogun [] | 00:11 | |
| -!- HeikoS [~heiko@p5797249D.dip0.t-ipconnect.de] has quit [Quit: Leaving.] | 00:45 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 04:10 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Ping timeout: 250 seconds] | 06:07 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 07:23 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Ping timeout: 250 seconds] | 07:30 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 07:36 | |
| @wiking | sanuj: yo | 08:05 |
| @wiking | sanuj: how's going with the prototyping | 08:05 |
| @wiking | do you have a place where you do it and shared in public | 08:05 |
| sanuj | wiking: hi! | 08:05 |
| @wiking | ? | 08:05 |
| @wiking | (dont wanna push you and i guess you are busy with other stuff now, just for future refernce) | 08:06 |
| sanuj | wiking: i had managed to get Tags working | 08:06 |
| sanuj | https://github.com/sanuj/shogun-plugin-arch/blob/master/tags-string/tests/tests.cpp | 08:06 |
| sanuj | next step is to make SWIG work with this | 08:06 |
| @wiking | ok | 08:06 |
| @wiking | about swig | 08:06 |
| @wiking | yeah the tags i saw | 08:07 |
| @wiking | but with swigs we should do some thourough testing | 08:07 |
| sanuj | and allow string-based parameters setters/getters in python | 08:07 |
| @wiking | you mean in swig interfaces :P | 08:07 |
| sanuj | yeah :P | 08:07 |
| @wiking | anyhow | 08:07 |
| @wiking | so | 08:07 |
| sanuj | yeah | 08:07 |
| @wiking | swig allows you to do http://www.swig.org/Doc3.0/Modules.html | 08:08 |
| sanuj | i was reading swig docs but got busy with exams.... | 08:08 |
| @wiking | yeah sure | 08:08 |
| sanuj | i see | 08:08 |
| @wiking | so anyhow | 08:08 |
| @wiking | we can do modules | 08:08 |
| @wiking | BUT | 08:08 |
| @wiking | sonne|work: tried this in an earlier version of swig (1.3) | 08:09 |
| @wiking | and it wasn't working | 08:09 |
| sanuj | okay | 08:09 |
| @wiking | but | 08:09 |
| @wiking | since then we started to depend on 2.0.4 or later | 08:09 |
| @wiking | but actually you should test it | 08:09 |
| @wiking | using 3.x | 08:09 |
| @wiking | and if it works stable | 08:09 |
| @wiking | then we should go with modules | 08:10 |
| sanuj | yes i'll...if it works then we can make swig 3.0 a necessary dependancy | 08:10 |
| @wiking | (as many other things got fixed in recent swig versions) | 08:10 |
| @wiking | so it really doesn't matter that if we fix that we require 3.0+ | 08:10 |
| sanuj | okay | 08:10 |
| @wiking | just this needs to be tested | 08:10 |
| sanuj | okay | 08:11 |
| sanuj | wiking: btw i would be interested in GP notebook but first i need to get a grip on my project | 08:12 |
| sanuj | wiking: and regarding plugins | 08:14 |
| sanuj | bjorn will finish the basic codebase in dynaplugz by the end of this week | 08:15 |
| sanuj | we talked about it yesterday | 08:15 |
| sanuj | then i can start building on top of that | 08:15 |
| sonne|work | wiking, sanuj - the problem with swig and modules is the following: if you have a module say Kernels and a module say Features. Then the kernel module is using features | 08:19 |
| sonne|work | So you need to make features known to kernels. | 08:20 |
| sonne|work | End you end up having to include features again (to some extend). | 08:20 |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 08:29 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 08:29 | |
| sanuj | sonne|work: i'm not sure if i fully understand it | 08:31 |
| sanuj | you need to include Features module in Kernel | 08:32 |
| sanuj | but why is this a problem | 08:32 |
| sanuj | sorry if i'm missing something obvious :P | 08:32 |
| -!- besser82 [~besser82@fedora/besser82] has quit [Remote host closed the connection] | 08:50 | |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 08:51 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 08:51 | |
| -!- sonne|work [~sonnenbu@ip5b414dfc.dynamic.kabel-deutschland.de] has quit [Quit: Leaving.] | 09:20 | |
| -!- sonne|work [~sonnenbu@ip5b414dfc.dynamic.kabel-deutschland.de] has joined #shogun | 09:21 | |
| sonne|work | sanuj: the problem is that the kernel module will grow big since it has to wrap all feature classes too | 09:21 |
| sonne|work | so you have maybe a small features module | 09:22 |
| sonne|work | but then a kernel module which is the size of kernels and features | 09:22 |
| sonne|work | -> no gain | 09:22 |
| sanuj | sonne|work: i see | 09:30 |
| sonne|work | sanuj: the other (fixable) problem is that we don't have clear modules so we might have cycles as in that features might need to know kernels for something | 09:31 |
| sonne|work | that is pretty painful :/ | 09:31 |
| sanuj | yeah | 09:31 |
| sonne|work | and of course classifiers require kernels and features etc... | 09:32 |
| -!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 276 seconds] | 10:06 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Ping timeout: 250 seconds] | 10:17 | |
| @wiking | sonne|work: yeah | 10:17 |
| @wiking | but then we want to load every | 10:17 |
| @wiking | .so of the plugin | 10:17 |
| @wiking | ? | 10:17 |
| @wiking | say we have .so for every different kernel | 10:18 |
| @wiking | how do we do this with swig interfaces? | 10:18 |
| lisitsyn | we would have kernel interface in the base shogun | 10:20 |
| lisitsyn | everything else goes to plugins | 10:21 |
| @wiking | yeah | 10:21 |
| @wiking | but how would a swig wrapper | 10:21 |
| @wiking | for all the plugins look like? | 10:21 |
| lisitsyn | no single wrapper | 10:22 |
| lisitsyn | kernels, features, etc are all base classes | 10:22 |
| @wiking | yeah but what i mean | 10:22 |
| @wiking | we would have for each plugin | 10:22 |
| @wiking | a swig interface | 10:22 |
| @wiking | no? | 10:22 |
| lisitsyn | no | 10:22 |
| lisitsyn | just for the base shogun | 10:22 |
| @wiking | k | 10:23 |
| lisitsyn | that would simplify things | 10:23 |
| @wiking | and when creating kernel = GaussianKernel(...) | 10:23 |
| @wiking | ? | 10:23 |
| @wiking | in python for example | 10:23 |
| lisitsyn | yeah but you don't write gaussiankernel anymore | 10:23 |
| lisitsyn | you do something like kernel('gaussian') | 10:23 |
| lisitsyn | because it is runtime now | 10:23 |
| @wiking | k | 10:23 |
| @wiking | and then i guess that big switch is autogenerated? | 10:24 |
| lisitsyn | oh well | 10:26 |
| lisitsyn | not sure everything will go smooth :) | 10:26 |
| @wiking | k :) | 10:26 |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 10:28 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 10:28 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 10:31 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Ping timeout: 250 seconds] | 10:38 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 11:22 | |
| sonne|work | wiking, lisitsyn - yeah that sounds indeed like the only feasible solution... | 11:27 |
| lisitsyn | sonne|work: yeah I've failed to come up with any other | 11:28 |
| sonne|work | and we don't have nightmares with dependencies either | 11:28 |
| sonne|work | lisitsyn: still kernels will require features etc in swig | 11:28 |
| lisitsyn | there is a thing that we can't have 'custom' methods but there must be a workaround | 11:28 |
| lisitsyn | well we can call them by strings as well :) | 11:29 |
| sonne|work | lisitsyn: custom as in extended by python? | 11:34 |
| sonne|work | like we had with swig directors? | 11:35 |
| lisitsyn | sonne|work: yeah this thing as well | 11:35 |
| lisitsyn | but mostly things like precompute_AA_matrix | 11:35 |
| lisitsyn | whatever it means | 11:35 |
| lisitsyn | some cache_oligomers_whatever | 11:35 |
| sonne|work | lisitsyn: yeah all strings... | 11:35 |
| lisitsyn | we can't put such things into the base interface so we have troubles with non-standard stuff | 11:36 |
| lisitsyn | but again | 11:36 |
| lisitsyn | we can call them using strings | 11:36 |
| lisitsyn | not that convenient but maybe it is good enoguh | 11:36 |
| sonne|work | lisitsyn: well the advantage of having a tiny piece of c++ code aka .so file to be dynloaded and have the full features shogun power including typemaps etc for $language is worth it ... | 11:38 |
| lisitsyn | yeah agree | 11:39 |
| lisitsyn | sonne|work: there is another good thing | 11:58 |
| lisitsyn | plugins allow remote execution as well | 11:58 |
| lisitsyn | I mean if we have limited interface it is much easier to make it work over some REST API or RPC | 11:58 |
| sonne|work | lisitsyn: botherd :P | 11:59 |
| lisitsyn | sonne|work: well I find it useful to train something on other machine | 11:59 |
| lisitsyn | although industry stuff needs more care anyway | 12:00 |
| lisitsyn | like queueing | 12:01 |
| lisitsyn | we don't want it | 12:01 |
| sonne|work | but I guess we don't want to re-invent the wheel but just ran shogun multiple times... | 12:02 |
| sonne|work | or let it be remote-controlled by sth else at least | 12:02 |
| -!- HeikoS [~heiko@87.151.41.66] has joined #shogun | 12:48 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 12:48 | |
| -!- HeikoS [~heiko@87.151.41.66] has quit [Ping timeout: 246 seconds] | 13:59 | |
| -!- HeikoS [~heiko@p57972942.dip0.t-ipconnect.de] has joined #shogun | 14:01 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 14:01 | |
| @wiking | HeikoS: oooohooohoooooooooooo | 14:08 |
| @wiking | in .de | 14:08 |
| @wiking | good | 14:08 |
| @wiking | so its ev time | 14:08 |
| @wiking | :) | 14:08 |
| @HeikoS | yep :) | 14:08 |
| @HeikoS | actually just called the court | 14:08 |
| @HeikoS | but only machine answers ;) | 14:08 |
| @wiking | :> | 14:08 |
| @wiking | wait are you in venezuela? :) | 14:08 |
| @wiking | because there only monday & tuesday are workdays | 14:09 |
| @HeikoS | haha | 14:09 |
| @HeikoS | german state employees | 14:09 |
| @HeikoS | same thing | 14:09 |
| @HeikoS | only work 13:29-13:30 on Thursdays | 14:09 |
| @wiking | lol | 14:10 |
| @wiking | :> | 14:10 |
| @HeikoS | ah crap | 14:17 |
| @HeikoS | looks like will have to pay | 14:17 |
| @HeikoS | wiking: Ill ask my father's friends. maybe one of them can do that for free | 14:17 |
| @wiking | ? | 14:18 |
| @wiking | how much $ you need? | 14:18 |
| @wiking | i'm happy to contribute | 14:18 |
| @wiking | just gimme your iban+swift | 14:19 |
| @HeikoS | still need to ask | 14:23 |
| @HeikoS | will let you know | 14:23 |
| @HeikoS | but thanks! | 14:23 |
| @HeikoS | :) | 14:23 |
| sanuj | HeikoS: you moved to germany from london? | 14:26 |
| @wiking | HeikoS: ok | 14:31 |
| @HeikoS | sanuj: just for a week | 14:32 |
| @HeikoS | wiking: ok looks good | 14:32 |
| @HeikoS | I will bring them the paperwork today | 14:32 |
| @HeikoS | they will get back to me tomorrow with price | 14:32 |
| @HeikoS | and whether sören has to appear in person (which kind of would kill it;) ) | 14:33 |
| @HeikoS | but I think it should be ok if it is just me | 14:33 |
| @HeikoS | Thats what I read a while ago | 14:33 |
| @HeikoS | will print things now | 14:33 |
| @HeikoS | see you in a bit | 14:33 |
| lisitsyn | wiking: will you contribute to my bank account as well? | 14:41 |
| sanuj | :P | 14:43 |
| @wiking | HeikoS: cool! | 14:59 |
| @wiking | lisitsyn: for you anything <3 | 14:59 |
| @wiking | :) | 14:59 |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Ping timeout: 250 seconds] | 16:01 | |
| sonne|work | wiking: we are talking about liquid donations aren't we :P | 16:03 |
| @HeikoS | sonne|work: so will you send me the thing? :) | 16:07 |
| @HeikoS | going to the notar now to give them all the docs, lets see what they say ... they promised me an appointment end of this or early next week which is great | 16:08 |
| @HeikoS | wiking: ^ | 16:08 |
| -!- HeikoS [~heiko@p57972942.dip0.t-ipconnect.de] has quit [Ping timeout: 268 seconds] | 16:13 | |
| -!- OXPHOS [8ca3fe9e@gateway/web/freenode/ip.140.163.254.158] has joined #shogun | 16:15 | |
| @wiking | sonne|work: :) | 16:23 |
| @wiking | Heiko \o/ (for the record :P) | 16:24 |
| -!- Netsplit *.net <-> *.split quits: @ChanServ | 16:44 | |
| -!- Netsplit over, joins: @ChanServ | 16:57 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has joined #shogun | 17:40 | |
| sanuj | wiking: Is this about all about taking donations for shogun? | 17:47 |
| -!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 260 seconds] | 19:17 | |
| sanuj | anyone interested in reinforcement learning? | 19:19 |
| sanuj | https://openai.com/blog/openai-gym-beta/ | 19:19 |
| sanuj | lisitsyn: ^ | 19:19 |
| CaBa | hi | 19:26 |
| CaBa | i have real value features for my observations in a large pandas dataframe (rows are observations, columns are features). i wonder in what data structure i have to plug this data in order for shogun to be able to work with it - in the examples for the python modular interface i came across a class called 'RealFeatures' which intuitively sounds like a match, but it doesn't show up anywhere in the API documenta | 19:29 |
| CaBa | tion... | 19:29 |
| sanuj | wiking: lisitsyn ^^ | 19:33 |
| sanuj | CaBa: I think RealFeatures should work | 19:33 |
| CaBa | sanuj: any idea where I can find info on that class? i.e. how i have to hand over my data, is a dataframe okay, do i have to transform it, etc? | 19:37 |
| sanuj | CaBa: you can see Ipython-notebooks for some examples | 19:37 |
| sanuj | https://github.com/shogun-toolbox/shogun/blob/develop/doc/ipython-notebooks | 19:37 |
| sanuj | CaBa: like | 19:40 |
| sanuj | https://github.com/shogun-toolbox/shogun/blob/develop/doc/ipython-notebooks/regression/Regression.ipynb | 19:40 |
| CaBa | hm thanks, i'll look into it | 19:54 |
| -!- sonne|osx [~sonne@x4e37eb9c.dyn.telefonica.de] has joined #shogun | 20:10 | |
| -!- Saurabh7 [~Saurabh7@103.225.100.51] has joined #shogun | 20:13 | |
| -!- sanuj [0e8bc402@gateway/web/freenode/ip.14.139.196.2] has quit [Quit: Page closed] | 20:15 | |
| @wiking | CaBa: numpy | 20:19 |
| -!- sonne|osx [~sonne@x4e37eb9c.dyn.telefonica.de] has quit [Quit: sonne|osx] | 20:19 | |
| @wiking | of course if you have categoricals in your pandas df | 20:20 |
| @wiking | then you'll have to transform that before actually feeding it to the models | 20:20 |
| CaBa | wiking: i only have floating point values in my df. so i pass multiple numpy arrays (one per feature?) instead of one single df, right? | 20:33 |
| @wiking | mmmm | 20:35 |
| @wiking | CaBa: features is a matrix | 20:35 |
| @wiking | columns: feature vectors | 20:36 |
| @wiking | row each feature | 20:36 |
| @wiking | but yeah that should be a matrix | 20:36 |
| CaBa | ok, so more or less the transformed case | 20:36 |
| CaBa | columns are observations, rows are features | 20:36 |
| @wiking | :) | 20:36 |
| @wiking | yeah | 20:36 |
| CaBa | wiking: i'm fairly new to python and numpy/pandas. the fm seems to be represented by an array of arrays (numpy). at least that's what i get when I call get_feature_matrix() on a RealFeatures object from one of your examples... feeding a matrix seems to work nevertheless, it is then transformed... | 20:42 |
| @wiking | CaBa: lemme try to find you an example | 20:43 |
| @wiking | CaBa: check this example: https://github.com/shogun-toolbox/shogun/blob/develop/examples/undocumented/python_modular/features_dense_modular.py | 20:45 |
| @wiking | that show | 20:45 |
| @wiking | s | 20:45 |
| @wiking | RealFeatures(np.array([[],...], dtype=np.float64) | 20:46 |
| @wiking | ) | 20:46 |
| CaBa | should i have one RealFeatures object per feature? | 20:46 |
| @wiking | CaBa: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.as_matrix.html | 20:47 |
| @wiking | nono | 20:47 |
| @wiking | RealFeatures contains all the feature vectors | 20:47 |
| CaBa | ok | 20:47 |
| CaBa | hence the plural ;) | 20:47 |
| @wiking | indeed | 20:48 |
| @wiking | :) | 20:48 |
| CaBa | in the example you posted - we have 3 feats / 6 obs or the other way round? | 20:49 |
| CaBa | 3 feats 6 obs i guess... set_feature_vector() is probably for one feature, right? | 20:50 |
| CaBa | hm no.. | 20:50 |
| CaBa | it sets the feature vector for one observation? | 20:50 |
| CaBa | from modshogun import RealFeatures, LongIntFeatures, ByteFeatures | 20:51 |
| CaBa | from numpy import array, float64, int64, uint8, all | 20:51 |
| CaBa | oops | 20:51 |
| CaBa | sorry | 20:51 |
| CaBa | confused irssi and ipython :P | 20:51 |
| @wiking | :) | 20:51 |
| @wiking | 3 obs and 6 features | 20:52 |
| CaBa | yes, makes sense :) | 20:53 |
| CaBa | wiking: okay, fed data.as_matrix().transpose() into the RealFeatures constructor and i get proper feature vectors with get_feature_vector()... thanks :) | 21:07 |
| @wiking | no worries | 21:09 |
| -!- besser82 [~besser82@fedora/besser82] has joined #shogun | 21:11 | |
| -!- mode/#shogun [+o besser82] by ChanServ | 21:11 | |
| -!- sonne|osx [~sonne@x4e37eb9c.dyn.telefonica.de] has joined #shogun | 22:12 | |
| -!- HeikoS [~heiko@p4FED5271.dip0.t-ipconnect.de] has joined #shogun | 22:21 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 22:21 | |
| -!- HeikoS [~heiko@p4FED5271.dip0.t-ipconnect.de] has quit [Quit: Leaving.] | 22:32 | |
| -!- sonne|osx [~sonne@x4e37eb9c.dyn.telefonica.de] has quit [Quit: sonne|osx] | 23:34 | |
| -!- OXPHOS [8ca3fe9e@gateway/web/freenode/ip.140.163.254.158] has quit [Quit: Page closed] | 23:42 | |
| --- Log closed Thu Apr 28 00:00:32 2016 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!