| --- Log opened Tue Oct 21 00:00:33 2014 | ||
| -!- Phawx [~Phawx@91.102.9.98] has quit [Ping timeout: 245 seconds] | 00:26 | |
| -!- HeikoS [~heiko@nat-165-26.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.] | 00:45 | |
| -!- Phawx [~Phawx@91.102.9.98] has joined #shogun | 00:47 | |
| -!- Phawx [~Phawx@91.102.9.98] has quit [Remote host closed the connection] | 01:06 | |
| -!- Floatingman [~Floatingm@c-68-52-34-232.hsd1.tn.comcast.net] has quit [Remote host closed the connection] | 06:58 | |
| -!- Floatingman [~Floatingm@c-68-52-34-232.hsd1.tn.comcast.net] has joined #shogun | 07:01 | |
| -!- PirosB3 [~pirosb3@195.101.111.125] has joined #shogun | 10:37 | |
| -!- HeikoS [~heiko@2.216.173.249] has joined #shogun | 11:25 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 11:25 | |
| -!- Netsplit *.net <-> *.split quits: @lisitsyn | 13:37 | |
| -!- Netsplit over, joins: lisitsyn | 13:38 | |
| -!- mode/#shogun [+o lisitsyn] by ChanServ | 13:38 | |
| -!- PirosB3 [~pirosb3@195.101.111.125] has quit [Quit: PirosB3] | 14:41 | |
| -!- PirosB3 [~pirosb3@195.101.111.125] has joined #shogun | 14:50 | |
| -!- HeikoS [~heiko@2.216.173.249] has quit [Quit: Leaving.] | 16:20 | |
| -!- lambday [67157f4f@gateway/web/freenode/ip.103.21.127.79] has joined #shogun | 16:29 | |
| @lisitsyn | lambday: hey | 16:52 |
|---|---|---|
| lambday | lisitsyn: hola | 16:52 |
| @lisitsyn | lambday: how are you doing? | 16:52 |
| lambday | lisitsyn: how do you do, sir? | 16:52 |
| lambday | lisitsyn: haha.. good.. reading submodular functions is killing me | 16:53 |
| @lisitsyn | lambday: hahah sir! | 16:53 |
| -!- arne-cl [~arne@f052135204.adsl.alicedsl.de] has joined #shogun | 16:53 | |
| lambday | (trying to add a British touch) | 16:53 |
| @lisitsyn | lambday: before gsoc I receive like a billion of DEAR SIR | 16:53 |
| @lisitsyn | :D | 16:53 |
| @lisitsyn | no offense but guess who writes this way :) | 16:53 |
| lambday | lisitsyn: haha that's common.. I too got a few from anonymous people regarding asking tips for cracking the IIT entrance exam for masters :| | 16:54 |
| @lisitsyn | lambday: did you manage to solve your issue with virtual stuff? | 16:54 |
| lambday | lisitsyn: thanks to you.. I did - here it is | 16:54 |
| lambday | lisitsyn: https://github.com/lambday/flash/blob/develop/src/flash/statistics/internals/DataManager.h#L118 | 16:55 |
| @lisitsyn | std::shared_ptr<shogun::CFeatures>(feats) | 16:55 |
| @lisitsyn | uhmm don't do that ;) | 16:55 |
| @lisitsyn | make_shared | 16:55 |
| lambday | lisitsyn: ah that's just to save time | 16:55 |
| lambday | ohh | 16:55 |
| lambday | alrighty | 16:55 |
| @lisitsyn | ahh no | 16:55 |
| @lisitsyn | this would be ok as well | 16:56 |
| lambday | but this doesn't leak | 16:56 |
| @lisitsyn | yeah yeah my bad | 16:56 |
| lambday | lisitsyn: why don't we start using shared_ptr in shogun as well whenever we can? | 16:56 |
| @lisitsyn | lambday: I think we should | 16:57 |
| @lisitsyn | lambday: we actually should convert to d-ptrs | 16:57 |
| lambday | lisitsyn: and do we need a wrapper for the containers as well? | 16:57 |
| @lisitsyn | sgvector? | 16:57 |
| @lisitsyn | or what? | 16:57 |
| @besser82 | lisitsyn, lambday: I'm currently accumulating stuff to cleanup ^^ | 16:57 |
| @lisitsyn | ahh dynamicarray etc? | 16:57 |
| lambday | nah normal std::vector std::pair std::map | 16:58 |
| @lisitsyn | lambday: swig is slow with std | 16:58 |
| @lisitsyn | that's just the only issue | 16:58 |
| @besser82 | lisitsyn, sgvector is custom impl. written in C \o/ | 16:58 |
| @lisitsyn | besser82: hah well there were some reasons I guess | 16:58 |
| lambday | besser82: also our CDynamicObjectArray is pretty heavyweight... | 16:58 |
| lambday | otherwise it solves the same problems | 16:59 |
| @besser82 | lisitsyn, yes ---> No invented here XD | 16:59 |
| @lisitsyn | I am like into training some models here now | 16:59 |
| @lisitsyn | and actually for me what matters is command line :D | 16:59 |
| @besser82 | lisitsyn, lambday: one thing is kick out all that legacy C*****-shiatz... | 16:59 |
| @lisitsyn | so I start to think we should have cool command line for shogun | 16:59 |
| @besser82 | that was just for those ugly $binding_static | 16:59 |
| @besser82 | lisitsyn, real gangstaz sip on yacc ^^ | 17:00 |
| @lisitsyn | besser82: heh | 17:00 |
| @lisitsyn | I mean VW is just an ugly thing with command line | 17:00 |
| @lisitsyn | but you can integrate it anywhere | 17:00 |
| @besser82 | the whole cli-thingie is pretty ugly... | 17:01 |
| lambday | I think that's a great idea.. | 17:01 |
| @lisitsyn | yeah cli is ugly | 17:01 |
| @lisitsyn | but writing huge programs for data processing is hard as well | 17:01 |
| @lisitsyn | I mean you need to grep something sort it blabla and then feed to your classifier | 17:02 |
| @lisitsyn | it is just a waste of time to write a program for that | 17:02 |
| lambday | lisitsyn: how good is our cmdline_static? | 17:02 |
| @besser82 | lisitsyn, that's what python_modular is for ;) | 17:02 |
| @lisitsyn | nah not really | 17:02 |
| @lisitsyn | just one CLI utility | 17:03 |
| @lisitsyn | and your python data processing code is a mess :D | 17:03 |
| arne-cl | hi! could someone help me compile/install shogun with python_modular bindings? | 17:03 |
| @besser82 | lisitsyn, Y? | 17:03 |
| @lisitsyn | arne-cl: yeah sure what's the problem? | 17:03 |
| @besser82 | lisitsyn, import sh | 17:03 |
| arne-cl | i tried foo@bar:/tmp/shogun-3.2.0/build# cmake -DCMAKE_INSTALL_PREFIX="/opt/shogun-install" -DPythonModular=ON .. which ran fine, but... | 17:03 |
| lambday | arne-cl: hey | 17:03 |
| @lisitsyn | besser82: yeah but still ugly | 17:03 |
| @besser82 | arne-cl, which distro? | 17:04 |
| arne-cl | ... after make and make install, i cant import modshogun in python | 17:04 |
| lambday | arne-cl: set PYTHONPATH | 17:04 |
| arne-cl | linux mint 16 | 17:04 |
| @lisitsyn | yeah | 17:04 |
| @lisitsyn | you have some strange prefix for shogun | 17:04 |
| @lisitsyn | you'd have to let python know of it | 17:04 |
| @besser82 | arne-cl, export PYTHONPATH=/opt/shogun-install/usr/lib/pythonX.X/ | 17:05 |
| @besser82 | arne-cl, replace X.X with real python version ^^ | 17:06 |
| lambday | lisitsyn: did you happen to give any thoughts on the plugin arch? | 17:07 |
| lambday | how should we go about that... etc? | 17:07 |
| @besser82 | lambday, I'd propose void** XD | 17:08 |
| lambday | besser82: hahaha I C | 17:08 |
| @lisitsyn | lambday: well still think | 17:09 |
| @lisitsyn | we should have a few interfaces | 17:09 |
| @lisitsyn | lke base interfaces we have | 17:09 |
| @lisitsyn | others go away | 17:09 |
| arne-cl | besser82: i tried export PYTHONPATH=/opt/shogun-install/lib/python2.7/ but i still get an import error in python | 17:09 |
| @lisitsyn | you do some require - it loads .so (.dylib, .dll) | 17:10 |
| lambday | so... the way I was thinking is to make the shogun code partitioned into wrappers and internals.. so in that case, the wrappers are the interfaces that you're talking about | 17:10 |
| @lisitsyn | yeah | 17:10 |
| @besser82 | arne-cl, you missed that **/usr/** :D | 17:10 |
| lambday | fits pretty well then.. | 17:10 |
| lambday | arne-cl: do a quick find :) | 17:10 |
| arne-cl | besser82: but in /opt/shogun-install, there are only these dirs: include, lib and share | 17:11 |
| @lisitsyn | lambday: it is not clear for me how parameters should look like | 17:11 |
| @lisitsyn | kernel.set("width", 3.0)? | 17:11 |
| lambday | lisitsyn: in many cases I think we can make the internals header only... so highly optimized templated code can be written there.. | 17:11 |
| lambday | and wrappers are just clients using that internal library | 17:12 |
| @lisitsyn | yeah whenever it makes sense we can optimize | 17:12 |
| @besser82 | arne-cl, then I'm out of advice, sry... | 17:12 |
| -!- Floatingman [~Floatingm@c-68-52-34-232.hsd1.tn.comcast.net] has quit [Read error: Connection reset by peer] | 17:12 | |
| lambday | lisitsyn: you did something for that in aer, no? | 17:12 |
| @lisitsyn | lambday: yeah kind of | 17:12 |
| arne-cl | besser82: thanks anyway, i'll retry without a custom install dir | 17:13 |
| lambday | lisitsyn: is there any off-the-shelf parameter framework that we can use? | 17:13 |
| @lisitsyn | lambday: uhmm I don't know | 17:13 |
| @lisitsyn | lambday: that's just about the interface | 17:13 |
| @lisitsyn | storing parameters is simple and stupid | 17:13 |
| @lisitsyn | you do some map or anything like that and that's it | 17:14 |
| lambday | lisitsyn: so what's the issue? | 17:14 |
| @lisitsyn | lambday: the interface | 17:14 |
| lambday | kernel.set("width", 3.0) is fine | 17:14 |
| lambday | pretty python-like :D | 17:14 |
| @lisitsyn | lambday: the signature of such method is | 17:15 |
| @lisitsyn | set(const char*, T) | 17:15 |
| @lisitsyn | so we just generate all the possible things | 17:15 |
| -!- Floatingman [~Floatingm@c-68-52-34-232.hsd1.tn.comcast.net] has joined #shogun | 17:15 | |
| @lisitsyn | I don't know if that's cool | 17:15 |
| lambday | yeah | 17:15 |
| @lisitsyn | no compile time checks at all | 17:15 |
| lambday | ummmm | 17:16 |
| lambday | yeah that's an issue :/ | 17:17 |
| @lisitsyn | lambday: we can support both tags | 17:18 |
| lambday | lisitsyn: so you don't want to do anything specific for each class? | 17:18 |
| @lisitsyn | and strings | 17:18 |
| @lisitsyn | that's easy | 17:18 |
| @lisitsyn | yeah no specific methods if we want plugins | 17:19 |
| @lisitsyn | okay that's actually possible | 17:19 |
| @lisitsyn | but would be ugly | 17:19 |
| @lisitsyn | call('precompute_shit_together') | 17:19 |
| @lisitsyn | lambday: ^ like that | 17:20 |
| @lisitsyn | in python we can add things dynamically | 17:20 |
| @lisitsyn | not in java | 17:20 |
| lambday | you gotta have something for the classes to specify | 17:25 |
| lambday | make register_params mandatory for each class.. | 17:26 |
| @lisitsyn | lambday: I think we can avoid any register params | 17:26 |
| @lisitsyn | lambday: you just don't specify field exactly | 17:26 |
| @lisitsyn | you only specify their names and type | 17:27 |
| lambday | I mean yes that should do | 17:27 |
| lambday | and you store as Any or void** | 17:27 |
| lambday | and cast it back | 17:27 |
| @lisitsyn | yeah Any and void** is the same :D | 17:27 |
| @lisitsyn | lambday: you can do smart things if you know all the fields of a class | 17:28 |
| lambday | so if the cast function is available then can't we static assert it with the templated set(...) function | 17:28 |
| lambday | :-/ | 17:28 |
| @lisitsyn | 1) you reserve exactly this amount of memory | 17:28 |
| @lisitsyn | 2) you re-order them to be aligned | 17:28 |
| @lisitsyn | and 3) you can add some spaces to avoid threading issues | 17:28 |
| @lisitsyn | lambday: ^ 3) is the funny thing actually | 17:29 |
| @lisitsyn | I am talking about false sharing | 17:29 |
| lambday | lisitsyn: what's that? | 17:30 |
| @lisitsyn | lambday: say you have | 17:30 |
| @lisitsyn | struct { int a; int b; } | 17:30 |
| @lisitsyn | and you access it from different threads | 17:30 |
| @lisitsyn | your CPU will block both a and b just because they are in same cache line | 17:31 |
| @lisitsyn | struct { int a; char space[1024]; int b; } would be much faster :D | 17:31 |
| lambday | really | 17:31 |
| @lisitsyn | lambday: so the thing is that ideally you'd just | 17:32 |
| @lisitsyn | put some lines like that | 17:32 |
| @lisitsyn | addParameter<int>("width"); | 17:32 |
| @lisitsyn | addParameter<std::string>("name"); | 17:32 |
| @lisitsyn | it reserves sizeof(int) + sizeof(std::string) + sizeof(alignment junk) bytes | 17:33 |
| lambday | what if some single threaded program tries to use it then? locality is lost... | 17:33 |
| @lisitsyn | lambday: yeah | 17:33 |
| lambday | or it may be possible to use different layouts for single threaded or multithreaded if we know that too in advance | 17:34 |
| @lisitsyn | lambday: I don't think we should bother about that | 17:34 |
| @lisitsyn | lambday: the thing is that we can do that | 17:35 |
| @lisitsyn | in one place | 17:35 |
| @lisitsyn | if we have plain old C++ classes you know | 17:35 |
| @lisitsyn | we have to do all the stuff manually | 17:35 |
| lambday | lisitsyn: how vital would be this alignment in some, say, data-oriented job? | 17:36 |
| lambday | data hungry | 17:36 |
| @lisitsyn | not really an issue I think | 17:36 |
| @lisitsyn | lambday: we just let compiler know of constness of get | 17:37 |
| @lisitsyn | and it optimizes redundant calls | 17:37 |
| @lisitsyn | you don't really use parameters that often I think | 17:37 |
| lambday | lisitsyn: hmm... this is cool stuff man | 17:38 |
| @lisitsyn | lambday: ahh yeah another thing about storing everything in one piece of memory | 17:39 |
| @lisitsyn | lambday: we can (de)serialize with one strcpy | 17:39 |
| @lisitsyn | (if it is one arch) | 17:39 |
| lambday | strcpy or std::copy? | 17:40 |
| @lisitsyn | lambday: what's the difference :D | 17:40 |
| lambday | lol.. null :D | 17:40 |
| @lisitsyn | lambday: pointers would be updated though | 17:43 |
| lambday | lisitsyn: yep.. | 17:45 |
| lambday | lisitsyn: could we write some poc within aer for this stuff? | 17:45 |
| @lisitsyn | lambday: yeah I'll try to find some time for that | 17:45 |
| lambday | lisitsyn: if you think I can help code-wise for some places let me know.. | 17:48 |
| @lisitsyn | lambday: would assigning some issues work? | 17:48 |
| lambday | lisitsyn: I guess so.. | 17:48 |
| @lisitsyn | lambday: cool then I'll try | 17:49 |
| lambday | lisitsyn: awesome.. looking forward to this stuff.. very psyched | 17:49 |
| @lisitsyn | :D | 17:49 |
| lambday | lisitsyn: reading all these mathematics is giving me headache :( | 17:49 |
| lambday | could use some break.. | 17:49 |
| @lisitsyn | lambday: oh why? maths is kind of cool | 17:49 |
| lambday | lisitsyn: it is... but I feel like way too much new information is being hashed inside my brain within a very short period of time.. and I'm telling you I don't feel much good :D | 17:50 |
| lambday | lisitsyn: working on feature selection btw which is cool stuff :D | 17:51 |
| -!- arne-cl [~arne@f052135204.adsl.alicedsl.de] has left #shogun [] | 17:51 | |
| lambday | lisitsyn: going to grab a beer... see you soon :) | 17:52 |
| @lisitsyn | see ya | 17:53 |
| -!- PirosB3 [~pirosb3@195.101.111.125] has quit [Quit: PirosB3] | 19:13 | |
| @wiking | ping | 19:19 |
| -!- lambday [67157f4f@gateway/web/freenode/ip.103.21.127.79] has quit [Ping timeout: 246 seconds] | 23:32 | |
| --- Log closed Wed Oct 22 00:00:35 2014 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!