| --- Log opened Fri May 03 00:00:59 2019 | ||
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has quit [Remote host closed the connection] | 10:00 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 10:00 | |
| -!- gf712 [9052086c@gateway/web/freenode/ip.144.82.8.108] has joined #shogun | 10:03 | |
| -!- geektoni [c1cdd24a@gateway/web/freenode/ip.193.205.210.74] has joined #shogun | 11:02 | |
| -!- wiking [~wiking@2001:67c:10ec:5784:8000::3ff] has joined #shogun | 11:08 | |
| wiking | gf712: segfault on import shogun :( | 11:13 |
|---|---|---|
| gf712 | wiking: with shared_ptr stuff? | 11:15 |
| wiking | yeah | 11:15 |
| wiking | but i mean it's segfaulting on | 11:15 |
| gf712 | did you add shared_ptr to everything? | 11:16 |
| wiking | SwigPyObjectType | 11:16 |
| wiking | so i really dont get it | 11:16 |
| wiking | yeye if u dont add everything it doesn't compile | 11:16 |
| wiking | this is some swig related stuff | 11:17 |
| wiking | so i'm not so sure where it goes really to the crazy bin | 11:17 |
| wiking | as essentially this has nothing to do with shared_ptr yet | 11:17 |
| gf712 | oh right | 11:18 |
| gf712 | hmm I am pretty sure develop works fine though no? | 11:18 |
| gf712 | did you shared_ptr the templates too? | 11:19 |
| wiking | i think so | 11:19 |
| wiking | yeah but again | 11:19 |
| wiking | this is still w/o any shogun related stuff | 11:19 |
| wiking | it's the very beginning of importing a swig module | 11:19 |
| gf712 | are you using the new swig? | 11:19 |
| wiking | literally no shogun objects or methods are called yet | 11:19 |
| wiking | no | 11:19 |
| wiking | i use the one that works with develo | 11:19 |
| wiking | p | 11:19 |
| gf712 | mhhh very odd. did you delete other stuff in the swig files? | 11:22 |
| gf712 | stuff just snowballs in swig | 11:22 |
| wiking | nah but now i basically removed almost all the stuff from swig | 11:22 |
| wiking | lets see what happens | 11:22 |
| wiking | meaning no shogun objects | 11:22 |
| gf712 | aight, and then add bit by bit? | 11:23 |
| gf712 | wiking: I got a deserialiser for arff working btw | 11:23 |
| gf712 | but I don't know how to handle Nan values? | 11:23 |
| wiking | ah fuuuuck | 11:23 |
| wiking | what is this factory wrappers shit | 11:23 |
| wiking | that's why i have 2 definitions | 11:24 |
| wiking | of the same thing in factory | 11:24 |
| wiking | src/interfaces/python/swig_typemaps.i | 11:24 |
| wiking | ah man | 11:24 |
| wiking | i'll throw this out for now | 11:24 |
| gf712 | what do you mean? | 11:25 |
| wiking | there's literally 2 definition of every factory method for some reason in the codebase | 11:26 |
| wiking | i couldnt yet track down where is it coming from | 11:26 |
| wiking | but this part i'll throw out for the time being | 11:26 |
| wiking | as i'm sure the new there and stuff | 11:26 |
| wiking | is not shared_ptr ready | 11:26 |
| wiking | def _internal_factory(name, *args, **kwargs): | 11:26 |
| wiking | new_obj = _obj(name, *args) | 11:26 |
| wiking | for k,v in kwargs.items(): | 11:26 |
| wiking | new_obj.put(k, v) | 11:26 |
| wiking | return new_obj | 11:27 |
| wiking | if docstring: | 11:27 |
| wiking | _internal_factory.__doc__ = docstring | 11:27 |
| wiking | else: | 11:27 |
| wiking | _internal_factory.__doc__ = _obj.__doc__.replace(object_name, new_name) | 11:27 |
| wiking | _internal_factory.__qualname__ = new_name | 11:27 |
| wiking | but dont have patient to fix this for now | 11:27 |
| wiking | first i wanna be able to import shogun | 11:27 |
| wiking | w/o segfault | 11:27 |
| gf712 | yea that's the wrapper I wrote to support kwargs in python | 11:28 |
| wiking | ok this works now... i'll one by one | 11:28 |
| wiking | yeye i know | 11:28 |
| wiking | realised | 11:28 |
| wiking | btw | 11:29 |
| wiking | CGaussianKernel is dropped from swig | 11:29 |
| wiking | right? | 11:29 |
| gf712 | let me know if you manage to instantiate a factory | 11:29 |
| gf712 | yes | 11:29 |
| wiking | as /home/wiking/shogun/src/shogun/kernel/GaussianShiftKernel.h:37: Warning 401: Nothing known about base class 'CGaussianKernel'. Ignored. | 11:29 |
| gf712 | all other kernels will be gone soon | 11:30 |
| gf712 | I have a pr where I removed everything | 11:30 |
| gf712 | managed to get swig bellow 1M lines... | 11:30 |
| wiking | hehe yeah just now gsk is for example not in swig | 11:35 |
| wiking | not that anybody is missing it | 11:35 |
| wiking | D: | 11:35 |
| wiking | :> | 11:35 |
| gf712 | haha | 11:35 |
| gf712 | it'll be back soon don't worry | 11:36 |
| gf712 | did you manage to import shogun? | 11:36 |
| gf712 | wiking: btw i am starting to work on openml curl stuff | 11:37 |
| wiking | sure thingy | 11:37 |
| gf712 | I have some spare time the whole of today | 11:37 |
| gf712 | I'll write some stuff and then you can take it apart | 11:38 |
| gf712 | it's mostly the tedious task of getting all the get/post paths from openml | 11:38 |
| gf712 | wiking: but yea seems quite simple to use curl | 11:38 |
| gf712 | I thought it was a really obscure old lib, but it's pretty neat | 11:39 |
| wiking | no | 11:39 |
| wiking | its good | 11:39 |
| wiking | libcurl is defacto www shita | 11:39 |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has quit [Read error: Connection reset by peer] | 11:40 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 11:40 | |
| gf712 | wiking: have you seen this https://www.reddit.com/r/freefolk/comments/bjjjjj/actual_footage_of_the_writers_after_episode_3/ ? | 11:40 |
| wiking | hahahahahah | 11:43 |
| wiking | this is tooooo GOOD | 11:45 |
| wiking | laughing loud in the office | 11:45 |
| gf712 | haha yea too good | 11:46 |
| gf712 | but so true | 11:47 |
| wiking | not even podrick died :D | 11:47 |
| wiking | lOOOL | 11:47 |
| gf712 | hahah | 11:47 |
| wiking | ok now i'm bisecting swig .i imports | 11:48 |
| wiking | as one of it causes the segfault apparently | 11:48 |
| gf712 | so removing the python code didn't help? | 11:48 |
| wiking | yeah that wasnt the one | 11:49 |
| wiking | now adding Classifier.i | 11:49 |
| wiking | :D | 11:49 |
| wiking | lets see | 11:49 |
| gf712 | did you fix the gpl stuff too then? | 11:51 |
| gf712 | wiking: which one would be easier to parse json or xml? | 11:58 |
| gf712 | seems like openml supports both | 11:58 |
| wiking | hahaha gpl... | 12:00 |
| wiking | not even bsd is really fixed | 12:00 |
| wiking | since wednesday i'm struggling with swig | 12:00 |
| wiking | gf712: since i guess they have special tags/attributes it doesn't really matter | 12:01 |
| wiking | either is going to require a custom serializer | 12:01 |
| gf712 | aight, and in terms in speed it doesn't really matter in guess | 12:02 |
| gf712 | the number of params will always be quite small | 12:02 |
| gf712 | wiking: does classifier work? | 12:02 |
| wiking | yeah | 12:03 |
| wiking | going one by one | 12:03 |
| wiking | well json serializer is super fast :) | 12:03 |
| wiking | but yeah it doesnt really matter | 12:03 |
| wiking | btw i should try to create an sgo | 12:04 |
| wiking | the next time i dont get segfault | 12:04 |
| wiking | as i just try import for now :D | 12:04 |
| wiking | oh man all these _protocols.i needs fixing as well | 12:04 |
| wiking | :< | 12:04 |
| gf712 | I think you should try to instantiate one | 12:06 |
| gf712 | that's probably why it failed the first time | 12:06 |
| gf712 | when the code in shogun.py called the factory | 12:06 |
| wiking | >>> x = shogun.F1Measure() | 12:06 |
| wiking | >>> x | 12:06 |
| wiking | Traceback (most recent call last): | 12:06 |
| wiking | File "<stdin>", line 1, in <module> | 12:06 |
| wiking | TypeError: in method 'CSGObject::tp_print', argument 1 of type 'CSGObject *' | 12:06 |
| wiking | ok getting there | 12:06 |
| gf712 | mhh | 12:07 |
| wiking | static int print_sgobject(PyObject *pyobj, FILE *f, int flags) { | 12:07 |
| gf712 | do you have to wrap it in shared_ptr? | 12:07 |
| wiking | needs fixing | 12:07 |
| wiking | :) | 12:07 |
| gf712 | ah | 12:07 |
| gf712 | ok | 12:07 |
| gf712 | and then all the other targets | 12:07 |
| gf712 | erf | 12:07 |
| -!- geektoni [c1cdd24a@gateway/web/freenode/ip.193.205.210.74] has quit [Quit: Page closed] | 12:08 | |
| gf712 | wiking: are we adding fmt as a dependency btw? | 12:09 |
| wiking | gf712: finally when we have spdlog | 12:11 |
| wiking | then its there | 12:11 |
| wiking | mmm something is wrong in Structured.i | 12:38 |
| wiking | :O | 12:38 |
| gf712 | is that where the swig error comes from? | 12:44 |
| wiking | "error" | 12:51 |
| wiking | segfault | 12:51 |
| wiking | but dunno what's the reason i guess the only way to figure it out is to bisect that file as well | 12:52 |
| gf712 | wiking: but does shogun import work + instantiation work fine without it then> | 12:54 |
| wiking | indeed | 12:54 |
| wiking | there are some problems still with the meta examples but hopefully with the next compile those are fixed | 12:55 |
| wiking | it's a super pain to debug this swig interface | 12:55 |
| wiking | as any changes requires a full recompile | 12:55 |
| wiking | :D | 12:55 |
| gf712 | yea... but there is no way to have a swig module that loads the different libs is there? | 12:56 |
| wiking | imo yeah | 12:56 |
| wiking | swig has modules feature | 12:56 |
| wiking | which we dont use at all | 12:56 |
| gf712 | ah really? | 12:56 |
| gf712 | so you can split it all | 12:56 |
| gf712 | that would be useful... | 12:56 |
| gf712 | anyway, another day "SD | 12:57 |
| gf712 | :D | 12:57 |
| wiking | :D | 12:57 |
| wiking | ok put is not working | 12:57 |
| wiking | wonder why | 12:57 |
| wiking | https://pastebin.com/5T1sp7dL | 12:58 |
| gf712 | yea... I guess too much dispatching stuff going all the way | 12:58 |
| gf712 | wiking: should that be calling shogun::CSGObject::put< shogun::CLabels,shogun::CLabels,void >(std::string const &,std::shared_ptr< shogun::CLabels >) ? | 12:59 |
| wiking | one would hope | 12:59 |
| wiking | but imo casting fails | 13:00 |
| wiking | or something | 13:00 |
| gf712 | and type is clabels? | 13:00 |
| gf712 | not sgobject? | 13:00 |
| gf712 | does as_labels help? | 13:00 |
| wiking | (gdb) p *argv[2] | 13:00 |
| wiking | $5 = {ob_refcnt = 2, ob_type = 0x7ffff2e18f00 <SwigPyBuiltin__shogun__CLabels_type>} | 13:00 |
| gf712 | mmmh | 13:01 |
| gf712 | im how does swig wrap it in a shared_ptr | 13:01 |
| gf712 | ? | 13:01 |
| wiking | ah no | 13:01 |
| wiking | of course it's wrong | 13:02 |
| wiking | the put first param wsn't a string | 13:02 |
| gf712 | lol | 13:02 |
| wiking | local hacks | 13:02 |
| wiking | 5 generated example fails | 13:02 |
| wiking | rest passes :) | 13:03 |
| gf712 | nice! | 13:03 |
| gf712 | segfault? | 13:03 |
| gf712 | have you checked for mem leaks? | 13:03 |
| wiking | lol | 13:03 |
| wiking | 300: Traceback (most recent call last): | 13:03 |
| wiking | 300: File "/home/wiking/shogun/build/examples/meta/python/multiclass/random_forest.py", line 19, in <module> | 13:03 |
| wiking | 300: p.set_num_threads(1) | 13:03 |
| wiking | 300: AttributeError: 'SwigPyObject' object has no attribute 'set_num_threads' | 13:03 |
| wiking | 300: swig/python detected a memory leak of type 'std::shared_ptr< shogun::Parallel > *', no destructor found. | 13:03 |
| wiking | this is one of the failing example | 13:03 |
| wiking | 304: File "/home/wiking/shogun/build/examples/meta/python/neural_nets/convolutional_net_classification.py", line 4, in <module> | 13:04 |
| wiking | 304: from shogun import NNOM_GRADIENT_DESCENT | 13:04 |
| wiking | 304: ImportError: cannot import name NNOM_GRADIENT_DESCENT | 13:04 |
| wiking | heheh heaps of valgrind errors :) | 13:05 |
| wiking | dunno if it's related | 13:05 |
| wiking | but | 13:05 |
| gf712 | it shouldn't be importing NNOM_GRADIENT_DESCENT | 13:05 |
| wiking | ==15051== LEAK SUMMARY: | 13:05 |
| wiking | ==15051== definitely lost: 0 bytes in 0 blocks | 13:05 |
| wiking | ==15051== indirectly lost: 0 bytes in 0 blocks | 13:05 |
| gf712 | thats weird | 13:05 |
| wiking | valgrind python cross_validation.py | 13:05 |
| wiking | so | 13:05 |
| wiking | i guess we are goodish | 13:05 |
| wiking | :)) | 13:05 |
| gf712 | that's cool! | 13:05 |
| gf712 | at last | 13:05 |
| wiking | yeah need to go back to c++ part | 13:05 |
| wiking | to fix the errors | 13:05 |
| gf712 | really need that valgrind check on bot working | 13:05 |
| wiking | but conceptually | 13:06 |
| wiking | swig works | 13:06 |
| gf712 | after all this work you had :p | 13:06 |
| gf712 | good | 13:06 |
| gf712 | all those for something :D | 13:06 |
| wiking | yep | 13:06 |
| wiking | ok so before continuing | 13:06 |
| wiking | i'm dropping C | 13:06 |
| wiking | as that shit is crazy | 13:06 |
| wiking | doesn't make any more sense | 13:06 |
| gf712 | yay | 13:07 |
| wiking | and then all the %rename stuff could be just | 13:07 |
| wiking | %shared_ptr | 13:07 |
| wiking | and in swig files | 13:07 |
| wiking | but yeah first i check with structured.i | 13:07 |
| gf712 | makes the swig files more readable | 13:07 |
| wiking | wtf is happening | 13:07 |
| wiking | https://github.com/tensorflow/tensorflow/pull/26782#discussion_r280580154 | 13:09 |
| wiking | amazing comment | 13:09 |
| wiking | :D | 13:09 |
| wiking | i wonder if he is a developer of TF or what | 13:10 |
| wiking | yep | 13:10 |
| wiking | a python coder | 13:10 |
| wiking | :facepalm: | 13:10 |
| wiking | https://www.tensorflow.org/api_docs/python/tf/errors/FailedPreconditionError | 13:12 |
| wiking | :>>> | 13:12 |
| wiking | sometimes i wonder about things | 13:13 |
| wiking | :) | 13:13 |
| gf712 | wat? raise a python error from c++ code? | 13:15 |
| wiking | ok plif related stuff which is not a wonder ... | 13:15 |
| wiking | gf712: dunno :D | 13:15 |
| wiking | and anyways that's an OpError | 13:15 |
| wiking | not an IOError | 13:15 |
| wiking | anyhow | 13:15 |
| wiking | it's crazy :)) | 13:15 |
| gf712 | lol | 13:15 |
| gf712 | the plif stuff was already complaining before no? | 13:15 |
| wiking | yeye it's crazy stuff | 13:15 |
| wiking | needs lot of fix on c++ level anyways | 13:15 |
| wiking | so ok i'm gone back to fix more c++ shiatz | 13:16 |
| wiking | and once unit tests are done | 13:16 |
| wiking | hopefully all the generated stuff will pass as well | 13:16 |
| gf712 | maybe just ifdef 0 for now? :D | 13:16 |
| wiking | hahahah | 13:16 |
| wiking | that's why it fails | 13:16 |
| wiking | half of plif is #if 0 | 13:16 |
| wiking | :D | 13:16 |
| gf712 | hahah | 13:17 |
| wiking | btw | 13:31 |
| wiking | is there a point to return char* instead of std::string for get_name()? | 13:31 |
| gf712 | no, i don't think so | 13:34 |
| gf712 | I mean, char* would be cheaper | 13:34 |
| gf712 | but it's to get a name so it doesn't matter if we use string instead | 13:34 |
| -!- gf712 [9052086c@gateway/web/freenode/ip.144.82.8.108] has quit [Ping timeout: 256 seconds] | 14:24 | |
| -!- gf712 [9052086c@gateway/web/freenode/ip.144.82.8.108] has joined #shogun | 15:46 | |
| gf712 | wiking: can you glance over https://github.com/shogun-toolbox/shogun/pull/4628 quickly and see if this is going in the right direction please? :) | 15:51 |
| wiking | just a sec | 15:51 |
| gf712 | nw | 15:51 |
| wiking | mmm void OpenMLReader::post(const std::string& request, const std::string& data) | 15:52 |
| wiking | reader would never need post or? | 15:52 |
| wiking | would be a weird rest api | 15:52 |
| wiking | :) | 15:52 |
| wiking | look good | 15:53 |
| wiking | small things like if camelcase then OpenMLFlow instead of OpenmlFlow | 15:54 |
| gf712 | ah no | 15:54 |
| gf712 | no post in reader :D | 15:54 |
| gf712 | cool | 15:54 |
| gf712 | wiking: curl is good but no callback support with lambda :( | 15:55 |
| gf712 | they need to write it in C++ :D | 15:55 |
| gf712 | otherwise it fails silently | 15:55 |
| wiking | :<> | 15:56 |
| wiking | hahah | 15:56 |
| wiking | missing values | 15:56 |
| gf712 | yea... | 15:56 |
| wiking | most of the machines does not handle it | 15:56 |
| wiking | aaaand | 15:56 |
| wiking | we dont have any imputer | 15:56 |
| wiking | \o/ | 15:56 |
| wiking | imo maaaaybe some trees can handle missing values | 15:56 |
| gf712 | I need to check out the arff parsers | 15:57 |
| gf712 | see what they do | 15:57 |
| gf712 | because it should be up to the user to decide | 15:57 |
| wiking | do we want xxHash or stick with murmur3? | 15:57 |
| wiking | and btw i'm thinking to extend any with hash :) | 15:57 |
| gf712 | what will you be using to hash in any? | 15:58 |
| wiking | same as Parameter.cpp | 15:58 |
| gf712 | ah ok | 15:58 |
| gf712 | as for algo | 15:58 |
| gf712 | not sure tbh | 15:58 |
| wiking | basically to calculate the hash of the object | 15:58 |
| wiking | on put i guess somehow we could update the hash | 15:59 |
| wiking | the problem is that what's with c++ api | 15:59 |
| wiking | and then again propagating the changes is not possible :) | 15:59 |
| wiking | bbl | 16:02 |
| gf712 | ttyl | 16:02 |
| -!- wiking [~wiking@2001:67c:10ec:5784:8000::3ff] has quit [Remote host closed the connection] | 16:02 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun | 16:18 | |
| gf712 | wiking: we are definitely switching to rapidjson right? | 16:18 |
| wiking | yes | 16:18 |
| gf712 | ok, is it ok if I steal some of your deadbeef cmake stuff | 16:19 |
| gf712 | just to add it as dependency | 16:19 |
| gf712 | and then use it for parsing of openml response? | 16:19 |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Remote host closed the connection] | 16:36 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun | 16:48 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Remote host closed the connection] | 16:57 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun | 16:59 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Remote host closed the connection] | 17:11 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun | 17:14 | |
| -!- gf712 [9052086c@gateway/web/freenode/ip.144.82.8.108] has quit [Ping timeout: 256 seconds] | 18:15 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Remote host closed the connection] | 18:47 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun | 18:51 | |
| -!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Client Quit] | 18:53 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has quit [Read error: Connection reset by peer] | 19:29 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 19:29 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has quit [Read error: Connection reset by peer] | 23:06 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 23:06 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has quit [Read error: Connection reset by peer] | 23:10 | |
| -!- durovo [~durovo@8e.68.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 23:10 | |
| --- Log closed Sat May 04 00:00:00 2019 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!