IRC logs of #shogun for Wednesday, 2019-01-23

« Tuesday, 2019-01-22 Index Thursday, 2019-01-24 »
--- Log opened Wed Jan 23 00:00:39 2019
-!- tangy [491d7f98@gateway/web/freenode/ip.73.29.127.152] has joined #shogun01:09
-!- tangy [491d7f98@gateway/web/freenode/ip.73.29.127.152] has quit [Client Quit]01:10
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]02:40
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun02:42
-!- mode/#shogun [+o wiking] by ChanServ02:42
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 245 seconds]02:46
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun02:50
-!- mode/#shogun [+o wiking] by ChanServ02:50
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 246 seconds]02:54
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun03:31
-!- mode/#shogun [+o wiking] by ChanServ03:31
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 246 seconds]03:35
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun04:28
-!- mode/#shogun [+o wiking] by ChanServ04:28
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 245 seconds]04:32
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun04:49
-!- mode/#shogun [+o wiking] by ChanServ04:49
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 250 seconds]04:54
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun06:11
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]06:11
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun06:11
-!- mode/#shogun [+o wiking] by ChanServ06:11
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 250 seconds]06:15
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun06:32
-!- mode/#shogun [+o wiking] by ChanServ06:32
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 245 seconds]06:36
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun07:48
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]07:48
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun07:48
-!- mode/#shogun [+o wiking] by ChanServ07:48
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 244 seconds]07:55
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun08:24
-!- mode/#shogun [+o wiking] by ChanServ08:24
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 245 seconds]08:28
-!- gf712 [8028b333@gateway/web/freenode/ip.128.40.179.51] has joined #shogun08:49
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun09:19
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]09:19
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:19
-!- mode/#shogun [+o wiking] by ChanServ09:19
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]09:23
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:24
-!- mode/#shogun [+o wiking] by ChanServ09:24
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]09:32
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun09:47
-!- mode/#shogun [+o wiking] by ChanServ09:47
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 272 seconds]09:51
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun10:10
-!- mode/#shogun [+o wiking] by ChanServ10:10
gf712wiking are you there?10:28
@wikinggf712: yes10:28
gf712I was wondering if there was some page where there is a breakdown of what works for swig in the target languages we support?10:29
@wikingi dont think so10:29
@wikingit's all in a distributed cache in our minds10:30
@wikingafaik10:30
gf712OK, thanks!10:30
gf712btw do you know if the people from OpenML are thinking about merging your PR?10:30
gf712started to work on a shogun "translator"10:31
@wikingah i need to talk with them10:35
@wikingbut yeah we can get that merged10:35
@wikingquite rapidly10:35
@wikingjust need to justify some of the design decisions10:36
@wikingbut since the last ping i haven't got back to them that's why its stale atm10:36
gf712OK! that's cool, I just need to wrap it all up in a class then at some point10:37
gf712but the functions will be similar10:37
gf712I think now we have all that is needed10:37
@wikingok so your new interface10:49
@wikinghas the ability to query things from swig? :)10:49
@wikingi.e. name, type, value, description?10:49
@wiking:)10:49
gf712not description yet but the rest yest10:50
gf712yes*10:50
gf712oh, unless you mean the docstring?10:51
@wikingnono10:51
@wikingi mean we need to get out the description of a value10:51
@wikingprogramatically10:52
@wikingvalue = parameter10:52
gf712the string we pass to SG_ADD?10:53
@wikingyes10:55
@wikingthe 2nd string10:55
@wikingnot the first :)10:55
@wikingbut we could use the 1st string as the name of the variable10:55
@wikingso we need this tuple basically: (value_name, value_description, value_type, value)10:55
gf712SG_ADD3(param, name, description) -> you mean description = value_description10:56
gf712value_name = name10:56
@wikingyes10:57
gf712that should be doable10:57
gf712is it worth introducing these sklearn-like methods directly in shogun, like get_params(deep=bool)?11:09
gf712or is it somethig that should sit in a seperate python script just for openml stuff?11:11
@wikingmmm i'm not so sure why would we need something like that at all11:19
@wikingi mean the reason i started to do some separation in openml-python is that under the hood11:20
@wikingwe could do anything11:20
@wikingto expose a param11:20
gf712ah ok!11:20
gf712I was just pretty much translating stuff to make it work11:20
gf712I'll have a look at the base class you wrote in your PR11:21
gf712wiking: Anyway all good to access the descriptions11:22
@wiking+!11:23
@wiking+111:23
gf712same principle as the parameter_names and parameter_types11:23
gf712just added a parameter_descriptions11:23
-!- Lefteris [836fb90d@gateway/web/freenode/ip.131.111.185.13] has joined #shogun11:30
@wikingok11:43
@wikingperfect11:43
@wikingi'm working now on exporting things to coreml11:55
@wikingi've added a minimal new thing to machine11:55
@wikingthat marks whether a machine has been trained11:55
@wikingas we dont have things for this11:55
@wikingi'm thinking about making the train/apply thread safe11:55
@wikingbut that's up for discussion11:55
gf712ah that's good11:57
gf712have you11:57
gf712btw to check whether it's trainable it should be enough to check if the shogun object has a .train method?11:58
@wikingmmmmm12:00
@wikingafaik yes12:00
@wikingwhy?12:00
gf712just wondering. I was checking the SKLearnConverter and in sklearn you can check if something is trainable with fit12:01
@wikingmmm12:02
@wikingbut in our case it's enough that it's a mchine12:02
@wiking*machine12:02
@wikingsorry12:02
@wikingmachine or a transformer12:02
@wikingbut both has fit/train12:02
@wikingbut12:02
gf712what about the gridsearch?12:02
@wikingoh that's another story12:03
@wikingwe'll have to figure that out12:03
@wikingbut in my case12:03
@wikingnow i'm working with this12:03
@wikingEMachineType12:03
@wikingfor me that's a good selector for how to map a given machien12:03
@wikingand maybe we should just extend that12:03
@wikingget_solver_type12:04
@wikingmachine.get_solver_type()12:04
@wikingsome of the machines are not marked good atm12:04
gf712oh right, yes we need that12:04
@wikingbut that's fixable12:04
@wikingit's more like a bug atm12:04
gf712I can't remember where I found that I needed it12:04
@wikingfor example CLibLinearRegression is not marked as any machine12:04
@wikingthat is a bug12:04
gf712I think it was when I was translating undocumented12:04
gf712ah ok12:05
@wikingbut i've just realised that using that enum12:05
@wikingwould be a better thing to do12:05
@wikingand imo that's the same for u12:05
@wikinginstead of checking whether an object has a given method12:05
@wikingit's more robust12:05
@wiking:)12:05
@wikingimo12:05
@wikingtransformers atm does not have that enum12:06
@wikingnow the problem with this is actually the following12:06
@wikingand we need to figure out something with this12:06
@wikingsay we have the base interface12:06
@wikingand base library like shogun-core12:06
@wikingthis is not going to be extenable12:06
@wikingif we rely on an enum12:06
@wikingsince that enum is fixed in the core interface12:06
@wikingthat would mean that in that case12:07
@wikingwe would need to do a new release12:07
@wikingif we have a new pluging12:07
@wiking*plugin12:07
@wiking(see what i mean?)12:07
@wikingsorry i've just realised this.... so currently just stating facts12:07
gf712I need a minute to absorb the information :p12:08
@wikinghttps://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/machine/Machine.h#L3212:08
@wikingso i'm talking about this12:08
gf712yup12:08
@wikingso this is not really supporting a dynamically loadable12:09
@wikingmachien12:09
@wikingright? :)12:09
@wikingsay tomorrow we implement WonderMachine12:09
gf712oh right12:09
@wikingthat is some magic stuff12:09
gf712yes yes12:09
@wikinghow do we add this to the system correct way12:09
@wikingonly way is that we extend this enum12:09
@wikingbut that's against the actual design12:09
@wikingright12:09
@wiking?12:09
@wikingbecause then it's not a plugin12:09
@wikingbecause we cannot extend this runtime12:10
@wikingof course we can always use the return value of get_name()12:10
@wiking:D12:10
@wikingbut that's kind of crazy imo12:10
gf712i am just not sure why you need to extend it at runtime? i get that you need to extend everytime you implement something12:10
gf712I am lost in that part :D12:10
@wikingok12:10
@wikingso the long term plan is12:11
@wikingthat we'll have a libshogun-core12:11
@wikingthat only contains very base interfaces12:11
@wikinglike machine, features etc12:11
@wikingand adding a new model12:11
@wikingor method or whatever12:11
@wikingis basically just implementing these interfaces12:11
@wikingand shipping a .so or dll12:11
gf712ahh ok, it's starting to click in my head12:12
@wikingand then in runtime the framework would be able to load that .soi12:12
@wikingsince it implements the interface we need12:12
@wikingnow12:12
@wikingif we would keepo this12:13
@wikingEMachineType12:13
@wikingwe have a problem12:13
@wikingbecause we cannot extend this12:14
@wikingas it should be in the core12:14
@wikingright?12:14
gf712well you could do the bitmasking thing again12:14
@wikingin what sense12:14
@wikingi mean we cannot use an enum right?12:14
@wiking:)12:14
gf712hmm ok, let me think a bit12:15
@wikingso we could go with a magic number12:15
@wiking:P12:15
@wikinglike in case of serialUUID in java12:15
@wikingof course that has the same 'problematic' as a string12:15
@wikingas nothing can enforce you12:15
gf712what do you mean magic number?12:15
@wikinguint32_T12:15
@wikingand you choose some random uuid12:16
@wikingsee UUID12:16
gf712so just randomly attribute a number from 0 to 2**32 - 1?12:16
@wikingyes12:16
@wikingbut with that ht eproblem is12:16
@wikingthat you can still collide right12:17
@wiking(same as get_name())12:17
@wikingi mean there's no enforcing12:17
@wikingthat you cannot choose the same id12:17
@wikingso this is a minor problem with the current design12:17
@wiking;)12:17
@wikingthat's what i'm tryign to say here12:17
gf712so can you generate it from the hash of the name of each implementation?12:18
@wikingand we should not rely on this enum12:18
@wikingbecause this enum should go12:18
@wikingyeah12:18
@wikinglike in java12:18
gf712but it should be collusion free no?12:18
@wikingwhere every obj has a .hash() method12:18
@wiking:)12:18
gf712collision haha12:18
@wikinghahahah collusion12:18
@wikingTHER WAS NO collusion12:18
@wiking:D12:18
@wikingNONE12:18
@wiking:>12:18
@wikingok12:18
gf712haha12:19
@wikingbut yeah .hash is still a good question12:19
@wikingwhat do we hash?12:19
gf712just the name12:19
gf712it should be unique no?12:19
@wikingi can name myself whatever i want no?12:20
@wikingi mean there's no way to enforce things from the core12:20
@wikingthat's what i'm trying to say12:20
@wikingso there's no way i can enforce that 2 totally different implementation12:20
@wikingcannot appear as the same thing12:20
@wikingright?12:20
gf712not if you name them the same way..12:20
@wikingyeah12:20
@wikingso anyways12:20
@wikingwe are gonna have this problem12:20
gf712__FILENAME__ macro?12:21
@wikingwhat filename? :)12:21
@wikingi mean this is just saying that12:21
@wikingi can always create a malicious plugin12:21
@wikingthat appears to be the same thing12:21
@wikingbut does totally different stuff12:21
@wikingonly way to do this12:21
@wikingto do some sort of public/private key signing12:21
@wikingand then we check whether the signature is there12:21
@wiking:)12:21
@wikingbut yeah that's another level12:21
@wikingi was just wondeirng if there's a sane way12:22
gf712true, but that is quite advanced12:22
@wikinginstead of a string compare12:22
@wiking:D12:22
@wikingbut seems not12:22
gf712as in would take some work12:22
@wikingand the enum should good sooner or later12:22
@wikingas that's against the design what we have in mind12:22
gf712when are you considering doing the split?12:23
@wikingmmm12:23
@wikinggood q12:23
@wikingnow i've mentioned this more about12:23
gf712could write a proposal for gsoc?12:23
@wikingthe coreml using that enum12:23
@wikingthat shoudl not happen12:23
@wiking:D12:23
gf712when do you use it in coreml btw? just to situate myself a bit better...12:25
@wikingto figure out into what object i would need to serialize the given machine12:28
@wikingsay i see an SVM12:28
@wikingthen i need to serialize that into an SVM + kernel12:29
@wikingin coreml12:29
@wikingnow unfortunately it seems this'll be done by specifying a list of strings12:29
@wiking(get_name())12:29
@wikingand if machine.get_name() in set(svms) then -.12:29
@wikingetc etc12:29
@wikingi just feel that this is ugly12:30
@wikingbut it'll be like this it seems12:30
gf712is that how they do it for sklearn?12:31
@wikingdunno12:32
@wiking:)12:32
@wikingbut probably something like this12:32
@wikingyou need a mapping12:33
@wikingbetween the objects and the coreml objects12:33
gf712and then use this switch https://github.com/apple/coremltools/blob/master/coremltools/converters/sklearn/_svm_common.py12:33
gf712btw are you going to ask them to extend their kernel support?12:34
@wikinghttps://github.com/apple/coremltools/blob/master/coremltools/converters/sklearn/_converter_internal.py#L26412:35
@wikinggf712: we could create a PR :)12:35
@wikingbut for the time being having a full coverage between coreml objs and shogun12:35
@wikingwould be nice12:35
gf712yup!12:36
gf712coreml seems to be more comments than code.. :D12:37
@wikingheheh yeah12:38
@wikingi mean it's more like a protobuf12:38
@wikingand some patches between some libs12:38
@wikingthat's all12:38
@wikingway too good https://www.linkedin.com/feed/update/urn:li:ugcPost:648952333111974297612:50
gf712ahahaha12:53
@wikingso true13:08
-!- abhinav-7 [76b9ec98@gateway/web/freenode/ip.118.185.236.152] has joined #shogun13:10
@wikingthere's something strange with the Pipeline.cpp13:26
@wikingas that is being recompiled every time of make13:26
@wikingi'll need to check why is that13:26
@wikingwuwei[m]: any ideas? :)13:26
@wikinghence the whole .so is being relinked13:27
@wikingetc etc13:27
wuwei[m]hmm13:27
wuwei[m]that's weird13:27
wuwei[m]I can reproduce it locally13:29
wuwei[m]i guess variant is re-installed everytime you make13:31
wuwei[m]wiking: btw are we moving to c++17 ?13:38
@wikingmmm we should soon13:43
@wikingbut i think we could patch the variant reinstall part13:44
@wikingso that if it's there then there's no trigger13:44
wuwei[m]yes I will change the cmake file13:44
@wikingshared_mutex13:45
@wikingnooooooo13:45
@wikingit's c++17?!13:45
@wikingreally13:46
@wikingwhy shared_mutex is not c++14 ? :>13:46
@wikinganybody knows of a good list of compilers that support c++1713:47
@wikingshared_timed_mutex is c++1413:47
@wikingdoh13:47
gf712full c++17 support?13:47
@wikingnot full13:47
@wikingbut what are the supports13:47
@wikingi think full nobody has13:48
gf712yea, i dont think so13:48
gf712i had a list somewhere13:48
gf712let me have a look13:48
gf712https://en.cppreference.com/w/cpp/compiler_support13:48
gf712it's what I use13:48
@wikingmmm13:50
@wikinglemme see about common distros13:50
@wiking:D13:50
@wikinghttps://packages.ubuntu.com/search?keywords=gcc13:50
@wikinghttps://packages.debian.org/search?keywords=gcc&searchon=names&suite=stable&section=all13:51
@wikingfuck debian stable only has gcc613:51
@wiking:S13:51
gf712I thought gcc6 is the one that has best support no?13:51
@wikingits a bit shitty as variant, any, string_view etc is in 713:51
gf712in any case you can get gcc-8 now quite easily on ubuntu13:52
@wikinghehe13:53
@wikingyeah but debian?13:53
@wikingfor example i dont use ubuntu on any server13:53
@wikingi prefer deb13:53
@wikingand it's not only me13:53
@wikingon the other hand https://packages.debian.org/search?keywords=clang-4.013:54
@wikingwoudl work13:54
@wikinglisitsyn: ^13:54
@wikinghttps://packages.debian.org/stretch/clang-4.013:55
@wiking Clang implements all of the ISO C++ 1998, 11 and 14 standards and also provides most of the support of C++17.13:55
@wikingbut again... not having gcc is a bit icky13:55
@wikingi feel13:55
@wikingin any case creating a feature branch that has c++17 requirements would not hurt imo14:01
@wikingthere14:05
gf712yup, I am guessing some stuff would break from 14 to 17?14:05
@wikinghttps://github.com/shogun-toolbox/shogun/tree/feature/cpp1714:07
@wikinglets see :D14:07
@wikinghttps://dev.azure.com/shogunml/shogun/_build/results?buildId=30714:08
@wikinghere's the pipeline14:08
@wikingwe'll see14:08
gf712that was quick :D14:10
@wiking2019-01-23T13:09:25.9364249Z CMake Error in src/shogun/CMakeLists.txt:14:10
@wiking2019-01-23T13:09:25.9364619Z   Target "libshogun" requires the language dialect "CXX17" , but CMake does14:10
@wiking2019-01-23T13:09:25.9364665Z   not know the compile flags to use to enable it.14:10
@wiking2019-01-23T13:09:25.9364694Z14:10
@wikingyeah14:10
@wikingi'm guessing the msvc will go14:10
@wikingand probably the osx as well14:11
gf712need  cmake version > 3.814:11
@wikingbut anyhow i'll try to make this branch in sync and we'll see then what we do with this14:11
@wikingrather 4.014:11
@wikingah cmake14:11
@wikingwhy?14:11
@wikingthat's only in case of the problem with MSVC14:11
@wikingno?14:11
gf712not sure, just checked stackoverflow14:12
@wikingyeah14:12
gf712and people said 3.814:12
@wikingthis one: https://stackoverflow.com/questions/45688522/how-to-enable-c17-in-cmake14:12
@wiking?14:12
@wikingbecause that's more about msvc14:12
@wikingbut we have the trick for that14:12
@wikingand based on the cmake error14:12
@wikingit detected correctly14:12
@wikingthat it doesn't know how to enable 17 for that compiler14:13
gf712https://stackoverflow.com/a/4725141814:13
@wikingit should be tested though14:13
@wikingif there's clang 4.014:13
@wikingthen what happens14:13
@wikinglemme see on donbot14:13
gf712https://cmake.org/cmake/help/v3.8/prop_tgt/CXX_STANDARD.html14:14
gf712https://cmake.org/cmake/help/v3.7/prop_tgt/CXX_STANDARD.html14:14
gf712support for 17 appears in 3.814:15
@wikingyeah ok14:15
@wikingno workie14:16
@wiking:D14:16
@wikingCMake Error in tests/meta/CMakeLists.txt:14:16
@wiking  Target "meta_example_integration_tester" requires the language dialect14:16
@wiking  "CXX17" , but CMake does not know the compile flags to use to enable it.14:16
@wikingeven in case of clang 4.014:16
@wikinghttps://packages.debian.org/stretch-backports/cmake14:17
@wikingok there is at least backport14:17
@wiking:)14:17
gf712we are just ahead of our time14:19
@wikingmmmm14:20
@wikingi've ran now with gcc 6.3 on debian and cmake 3.1314:20
@wikingand it didn't complain14:20
@wikinglemme see how it dies with variant14:21
@wikingdunno how we can enforce certain part of 1714:21
@wikingcoz yeah 6.3.0 gcc has some c++1714:21
@wikingbut not variant etc14:21
@wikingwoops14:22
@wikingthis is interesting14:22
@wiking(note that we use this for SWIG)14:22
gf712is that the build that is running now?14:23
@wiking2019-01-23T13:10:46.9540290Z ? noexcept(false)?2019-01-23T13:10:46.9539740Z ? ^~~~~~~~~~~~~~~~~~~~~~?2019-01-23T13:10:46.9539140Z         bool has(const std::string& name) const throw(ShogunException)2019-01-23T13:10:46.9539060Z ?/Users/vsts/agent/2.144.2/work/1/s/src/shogun/base/SGObject.h:329:42: ?note: ?use 'noexcept(false)' instead?2019-01-23T13:10:46.9538430Z ? ^~~~~~~~~~~~~~~~~~~~~~?2019-01-23T13:10:46.9537740Z14:23
@wiking      bool has(const std::string& name) const throw(ShogunException)2019-01-23T13:10:46.9537150Z ?/Users/vsts/agent/2.144.2/work/1/s/src/shogun/base/SGObject.h:329:42: ?error: ?ISO C++17 does not allow dynamic exception specifications [-Wdynamic-exception-spec]?14:23
@wikingso this is on macos14:23
gf712so need to replace all the throw(..) with noexcept(false)?14:27
gf712https://en.cppreference.com/w/cpp/language/except_spec14:27
gf712even just throw() will disappear14:27
gf712in c++2014:27
@wikingyes14:28
@wikingmmm anyhow it might work14:29
@wikingneeds regex14:29
@wiking:)14:29
gf712haha, it should be as simple as find and replace14:30
gf712for openml, how can you tell if an object is a pipeline?14:31
gf712other than get_name() == "pipeline"?14:32
@wikingyes14:32
@wikingok lets see if this helped14:38
@wikinghttps://dev.azure.com/shogunml/shogun/_build/results?buildId=30914:38
@wikingregex fail14:42
@wiking:D14:42
gf712so are you testing only macos right now?14:50
@wikingbasically14:50
@wikingbut that is almost the same compiler that you would have on ubuntu14:50
@wikingif u use clang14:50
@wikingand of course we'll see what happens on msvc as well14:50
@wikingbut that should be fine14:50
gf712oh the register issue14:54
gf712that is going to be annoyng with Python2.714:54
@wikingwhy?15:00
@wikingworst case we need to drop 2.7 support :D15:01
gf712well, if it is that easy to drop support then it's not an issue :D15:02
@wiking:>15:03
@wikingbut good to know what will break if we move the 1715:03
gf712so are shogun's dependencies also being compiled with 17?15:04
gf712i.e. stanmath15:04
@wikingyeah15:05
@wikingbut since we dont include it anywhere15:05
@wikingit shoudl be ok15:05
@wiking:D15:05
gf712no std::ptr_fun, time to get used to lambdas :D15:10
wuwei[m]wiking: do you have idea what I should do with variant ?15:10
wuwei[m]what's the standard way for external project15:11
@wikingmmm lemme check15:18
@wikingcopy_directory is the problem i think15:19
@wikingas that will always copy15:20
@wikingthat makes things changed15:20
wuwei[m]yes15:20
@wikingand that's why then things are being called again15:20
@wikingfor compilation15:20
wuwei[m]I used copy_if_different for single file version previously15:20
@wikingyes15:20
@wikingthe thing is that we need to copy more than one file right?15:21
wuwei[m]yes15:21
@wikingits only 4 files15:21
@wikingor?15:21
wuwei[m]but it's okay to switch to the single file version for variant15:22
@wikingcan we?15:22
wuwei[m]it has a new single file release15:22
@wikingok15:22
wuwei[m]https://github.com/mpark/variant/blob/single-header/v1.4.0/variant.hpp15:22
@wikingcan you try switching to that15:22
@wikingand do a copy_if_different15:22
@wikingfor the install15:22
@wikingthat way i think it should be fixed15:22
wuwei[m]okay15:22
@wikingthnx15:23
@wikinglisitsyn: tapkee is not c++17 ready :D15:26
lisitsynoh really?15:26
lisitsynwiking: what breaks?15:26
@wikingyep fails15:26
@wikinghttps://dev.azure.com/shogunml/shogun/_build/results?buildId=31215:26
@wikingsee the macos build15:26
@wikingthere's the log15:26
@wikingmem_fun_ref_t is removed in c++1715:27
lisitsynoh15:27
@wikingrandom_shuffle needs an engine15:28
@wikingand tapkee uses register as well15:28
@wikingin covertree15:28
lisitsynnah that's some strange covertree code15:28
lisitsyn:D15:28
@wikingok anyhow there's some things that are breaking stuff now in tapkee15:29
@wikingso i guess we'll need to change both in shogun and as well in your repo15:29
lisitsynyeah let me check15:29
@wikingkk15:30
-!- HeikoS [5aae0475@gateway/web/cgi-irc/kiwiirc.com/ip.90.174.4.117] has joined #shogun15:32
HeikoSgf712 yo15:32
gf712HeikoS hey!15:32
HeikoShow are things going?15:33
HeikoSLefteris hi!15:33
gf712fine! we are pretty much ready for openml15:33
gf712so working on that15:33
HeikoSIll be online for a while now and will do reviews, ping me if you wanna discuss anything15:33
HeikoSgf712 nice! looking forward to see that?15:33
gf712things are working, but need to figure out Pipelines15:33
LefterisHello15:33
gf712and crossvalidators, etc..15:33
HeikoSI see15:33
HeikoSyeah that will be tricky15:34
gf712but models work fine15:34
HeikoSas it is not within the parameter framework in an exposable way15:34
HeikoSok let me dig into those PRs15:34
gf712OK, i'll let you have a look and the figure things out15:34
gf712then*15:34
@wikingwuwei[m]: does that work then?15:35
-!- mode/#shogun [+o HeikoS] by ChanServ15:35
wuwei[m]yes it works15:35
@wikinggreat15:35
wuwei[m]I sent a pr15:35
@wikingthnx15:35
@wikingHeikoS: i wouldn't merge this15:45
@wikingas it adds more crazyness15:45
@wikinginto it15:45
@wikingadn later cleanup15:45
@HeikoSok sure15:45
@HeikoSpitty for him though15:45
@HeikoSlots of work15:46
@wikingdunno why he went ahead15:46
@HeikoSLet me push him again then to have a look at the feature branch15:46
@HeikoScuriosity I guess15:46
@wikingyeah but i told him15:46
@wikingthat there's another thing going on15:46
@wikingand that should be the prio15:46
@wikingif15:46
@HeikoSLefteris any updates on the string feature madness?15:51
LefterisI am about to hit enter for a new push15:53
LefterisI think now the python version is fine15:53
LefterisI could not fix the java version yet.15:54
@HeikoSdid you add the method to swig?15:55
@HeikoSthe methods you added were guarded so that swig only saw them for python15:55
Lefterisnot but why do we want to use DoubleMatrix?15:58
Lefterisisn't this for double numbers?15:59
@HeikoSLefteris well we don't want to use it16:04
@HeikoSthis is why we want to change it16:04
@HeikoSThere is a typemap for stringlist16:04
@HeikoSin java16:04
@HeikoSgf712 just made some comments around this map16:05
@wikingHeikoS: i dont agree with this16:05
@HeikoSgf712 since std::map doesnt work, I think we should not use a map16:05
@wikingFurthermore, we could then either have Python/specific lang extensions that turn it into a local datastructure (such as an explicit typemap for std::map in Python) or whatever if we want.16:05
@wikingits a lot of extra work16:05
@HeikoSwiking I think we shouldnt use CMap16:05
@wikingcmap is somewhat swigable16:05
@HeikoSbecause it makes later changes harder16:05
@wikingwhich changes?16:06
@HeikoSchange to std:: for example16:06
@wikingstd16:06
@wikingmap you mean?16:06
@wikingthat we can never do16:06
@wikingfor swig16:06
@wikingbecause that would mean that somebody will implement those typemaps16:06
@HeikoSexactly16:06
@wikingwithin swig16:06
@wikingthat will not happen16:06
@wikingas its still missing in many target langs16:06
@HeikoSbut for this use case I dont think it is necessary to use/explose cmap16:07
@HeikoSwe just want some descriptions16:07
@HeikoSand we want to minimise using those hand-baked data structures we have16:07
@wikingyeah16:07
@HeikoSespecially if not really necessary16:07
@wikingbut we can move cmap16:07
@wikinginto the swig16:07
@wikingif we want16:07
@wikingbut still16:07
@wikinghaving an 'ordered' std::list/vector16:08
@HeikoSno thats shitz16:08
@HeikoSgetter for single descr is my fav16:08
@HeikoSI dont like it16:08
@HeikoSbut I like it more than the cmap16:08
@HeikoS@wik16:09
@HeikoS@wiking or you prefer the cmap thingi for real?16:09
@wikingwell that would work :)16:10
@wikingi mean somewhere there will be the mapping16:10
@wiking:)16:10
@wikingor no mapping at all16:10
@wikingand you need to do a [o.get_descr(n) for n in o.param_names()]16:10
@wikingin all the target langs16:10
@wikingif you want to expose the description of all the params16:11
@wikingof a machine param16:11
@HeikoSwhich sucks as well16:11
@HeikoSsign16:11
@HeikoSwell at least no cmap so I prefer it16:11
@HeikoSwe can add the specific target lang code anyways for that then btw16:12
@HeikoSadd a method that returns a python dict16:12
@wikingthat sucks16:12
@wikingi mean because this is getting more and more16:12
@wikingpython stuff16:12
@HeikoSit is somewhat simmilar to the get16:12
@wikingand we and up the same way16:12
@wikingas with examples16:12
@HeikoStrue16:12
@wikingthere will be python target that is super maintained16:12
@wikingeverything else will just linger around16:12
@wikingbut that's the same for get already16:13
@wiking:)16:13
@wikingjust wanna avoid to have this super python oriented solutions16:13
@HeikoSget is quite findamental so it is useful there I think16:13
@wikingyeah but would be useful in all the langauges16:13
@wiking(none-typed)16:13
@wiking:>16:13
@wikingwuwei[m]: yeah great i've just merged the pr worked nicely16:14
wuwei[m]great16:14
@HeikoSok I think then we should go for the getter16:15
@HeikoSgf712 see discussion above16:15
@wikinghttps://foonathan.net/meetingcpp2018.html16:15
@wikingcool talk16:15
Lefterisunfortunately, I will be able to work on this on Monday again.16:15
wuwei[m]wiking: StanMath might have the same issue16:15
@HeikoSLefteris ok sure16:16
LefterisHeikoS: Thank you!16:16
@HeikoSgf712 ah you already wrote a typemap for CMap I see16:16
lisitsynwiking: I fixed tapkee in the upstream16:34
lisitsynis C++17 a branch?16:34
@wikinglisitsyn: is it copy paste that we did?16:34
@wikingfeature/cpp1716:34
lisitsynok16:34
lisitsynI can send a pr16:34
@wikingnono16:34
@wikingjust push it directly16:34
@wiking:D16:34
lisitsynok16:34
@wikingthnx16:34
@wikinglisitsyn: https://github.com/lisitsyn/tapkee/commit/749c6e1edf026d5f88ddcaa61cca49c210828492#r3203793616:35
@wikinglisitsyn: adding some comments in tapkee16:35
lisitsynoh thanks16:35
lisitsynwiking: check the constexpr thing16:36
@wikinghaha16:36
@wikingwas about to comment16:36
lisitsynI had to do really crazy stuff16:36
@wikingcrazy hacko16:36
@wiking:)16:36
lisitsynit was barely 11 these days16:36
lisitsynamazing that rather advanced code I wrote back then is now such a total crap lol16:36
@wikingstd::mt19937_64 is better btw16:36
@wikingfaster that is16:36
@wikingand you need  cmake_minimum_required (VERSION 3.8) to be able to set 17 standard in cmake16:37
@wikinglalala16:37
lisitsynwiking: yeah thanks just updtaed16:37
@wikingkk16:37
@wikinglisitsyn: register16:38
@wikingor where is that covertree16:38
lisitsynwiking: I think it is already dropped in the upstream16:38
lisitsynyeah it is16:38
@wikingajh yeah16:38
@wikingcool cool16:38
lisitsynwiking: I just need to update16:38
@wikingwe had a sync problem16:38
lisitsynwiking: which is16:38
@wikinghahahah16:38
lisitsynwell16:38
@wikingnot a copy16:39
@wikingi guess16:39
@wiking:D16:39
lisitsynwiking: I lost my regular expression to do that for sure16:39
lisitsynbut idea is that we need to replace everything in between of16:39
lisitsynthese /* Tapkee includes */16:39
@wiking:D16:39
LefterisBye guys. I'll be back on Monday.16:41
@wikingttyl16:41
-!- Lefteris [836fb90d@gateway/web/freenode/ip.131.111.185.13] has left #shogun []16:42
gf712HeykoS sorry was away from my desk16:51
@HeikoSgf712 no worries16:55
gf712so shall I drop the map in favour of a single param getter?16:55
@HeikoSI would prefer, but not sure about you andor wiking16:56
@HeikoSmy final thoughts are: then at least we have the same API for all langs without having to do anything16:56
@HeikoS(and avoid spreading the cmap)16:57
gf712I like the dictionary, because it avoids having to write a loop/comprehension to get all the param value pairs16:57
@HeikoSwell16:57
@HeikoShow to access the map?16:57
@HeikoSin say java?16:57
@HeikoSwith a loop over the swig api of the CMap16:57
@HeikoSso we don't really get anything from doing that16:57
@HeikoSof course we could write typemaps for all langs16:58
gf712ok! I can replace it then16:58
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]16:58
gf712for me it is not a big issue16:58
@HeikoSbut I dont think this would happen, and if it only happens for python we will have the old problem of different APIs back16:58
gf712was more out of convenience, but I only use the python interface16:58
@HeikoSkeep the old code maybe in case the discussions goes back (it sometimes does :)  )16:59
gf712Hmm, I can have a look at the other langs to be honest16:59
gf712always wanted to be polylingual16:59
gf712:p16:59
@HeikoSlol, well there are a few16:59
@HeikoSand all this needs to be maintained17:00
@HeikoSso you write the typemaps and then leave and then they will never be touched17:00
@HeikoSadding a new lang will be harder17:00
@HeikoSetc17:00
gf712OK, i'll stick to std::strings as return types17:00
@HeikoSas said keep the old code (and psych) just in case ;)17:00
gf712Do you know what other std structures cause issues btw?17:01
gf712basically stick to primitives and std::string?17:01
@HeikoSthe only thing we can rely on is vector atm I think17:02
@HeikoSnot sure what the state of Some is17:02
gf712OK17:03
-!- abhinav-7 [76b9ec98@gateway/web/freenode/ip.118.185.236.152] has quit [Quit: Page closed]17:03
@HeikoSgf712 all this is subject to change if someone has better ideas :D17:04
gf712I would return a vector instead of a map but I am always scared that the order might mess up somehow17:04
@HeikoSyeah let's not go there indeed17:04
-!- abhinav_7 [76b9ec98@gateway/web/freenode/ip.118.185.236.152] has joined #shogun17:05
gf712alright, I'll replace the SWIG getters and stash the current code17:05
@HeikoScool!17:06
gf712and fix the error message17:06
gf712in CSGObject.cpp17:06
@HeikoScool!17:06
@HeikoSI like having those 3 methods for name, type, and descr17:07
gf712HeikoS also need to talk about pipelines and crossvalidators at some point17:07
@HeikoSsure17:07
@HeikoSwhat do you want to know?17:07
gf7121) how to find out it's a pipeline (get_name() == "Pipeline")17:08
gf7122) how to iterate over transformers with the interface17:08
@HeikoS1) you mean the only way to find out is to ask for its name?17:09
gf7123) is it possible to have nested pipelines? I don't think it is from the C++ code, but just to make sure17:09
gf712yes17:09
gf712if I have an object in Python17:09
@HeikoSAs of now, we don't have a Swig type pipeline i.e. it is not part of the base types iirc17:10
gf712and need to know if it is a sg.Pipeline17:10
@HeikoSyeah I see17:10
@HeikoSthat is since you need to extract stuff that only a pipeline has17:10
@HeikoSIn this case, I think we need to add this to the base types17:10
@HeikoSso that SWIG knows the type17:11
gf712this seems to work type(pipeline) == sg.Pipeline17:11
@HeikoSyes because we export everything to swig17:12
gf712where pipeline is instantiated from sg.PipelineBuilder()17:12
@HeikoSand this needs to stay like that17:12
@HeikoSi.e. pipeline needs to a be a base type17:12
@HeikoSand also we need a factory for the builder in factory.h17:12
gf712OK!17:12
@HeikoSactually we dont if the type is exposed to swig17:12
gf712how would the builder handle multiple transformers though?17:12
@HeikoSwe can just use the ctor17:12
gf712sorry factory not builder17:12
gf712OK!17:12
abhinav_7Could anyone tell how to subscribe to the mailing list? Thanks in advance!17:13
@HeikoSgf712 so the type(...) == sg.Pipeline works?17:13
@HeikoSabhinav_7 the instructions on the website dont work?17:13
gf712HeikoS yes that works17:13
@HeikoSgf712 ok that would be ok then or?17:14
gf712yup, I just hadn't thought about it before haha17:14
@HeikoShehe ok17:14
gf712what about 2)?17:14
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun17:14
-!- mode/#shogun [+o wiking] by ChanServ17:14
@HeikoSok then how to iterate over its elements17:15
@HeikoSI think you need to ask wiking17:15
@HeikoSnot sure there is yet a python api for that but I might be wrong17:15
@HeikoSwuwei[m] actually wrote it so he can probably help as well17:15
abhinav_7@HeikoS The website links to this email address shogun-list-subscribe@shogun-toolbox.org, I don't know what do with it.17:15
gf712because you can get by name17:15
gf712but I can't find a way to get a list of the names17:16
gf712other than from __repr__17:16
@HeikoSabhinav_7 send an email with "subscribe" in subject I think17:16
abhinav_7I have done that. I think I should wait. Thanks for the help!17:16
@HeikoSgf712 I see, you might have to add that, should be useful for other cases or?17:16
@HeikoSabhinav_7 an empty email might also work17:17
gf712potentially, I mean it is used internally by pipeline so could expose it17:17
@HeikoSyeah why not, just make sure the others have a look17:17
gf712and then nested pipelines? Is that a thing?17:18
gf712sorry about the question bombardment, I have been going through what is still needed for openml17:18
@HeikoSi dont know the answer, haven't seen it though, seems complicated17:18
@HeikoSno worries17:19
gf712OK!17:19
gf712thanks :)17:19
-!- wiking [~wiking@huwico/staff/wiking] has quit [Ping timeout: 245 seconds]17:20
wuwei[m]HeikoS gf712 hey17:21
@HeikoShi17:21
@HeikoSsee questions above17:22
wuwei[m]what's the status of pipeline, is it a base class17:22
wuwei[m]?17:22
gf712yes17:22
gf712I think Pipeline can only be instantiated from PipelineBuilder17:22
gf712which returns a Pipeline base class17:23
wuwei[m]this is what we are doing I think17:23
@HeikoSwuwei[m] it atm is just exposed to swig17:24
@HeikoSand I think this will stay like this17:24
wuwei[m]I remember the issue of cross validation with pipeline is that we need CMachine*17:24
@HeikoSyes17:24
wuwei[m]see https://github.com/shogun-toolbox/shogun/pull/4380#issuecomment-43789617817:24
@HeikoSah yes this issue17:25
@HeikoSthe problem is that "put" is very strict on types17:25
@HeikoSno subclasses allowed17:25
@HeikoSif something is stored as CMachine you have to give CMachine, cannot give CPipeline17:26
@HeikoSthis will not change unfortunately, it is down to the design of the tags framework17:27
@HeikoShaving something like "as" is the only way out I see atm17:27
@HeikoSeither as_pipeline or as_machine17:28
@HeikoSone alternative would be to hardcode this particular case into the dispatcher for "put" ... not the nicest either17:28
@HeikoSthe alternative of just leaving it sitting there is not good as well finally17:28
@HeikoSactually due to the stuff gf712 is doing, we should not return a pipeline as a machine, since then it will be impossible to extract its members for openml17:29
@HeikoSso we need the pipeline type17:31
@HeikoSalthough get_name also does the job17:32
gf712Either work for now! As long as we can rely on get_name17:32
@HeikoSgf712 we can due to class_list in shogun17:33
@HeikoSbut then, note that "get" doesnt expose the pipeline elements17:33
@HeikoSso we need the type anyways17:33
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun17:38
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]17:38
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun17:38
-!- mode/#shogun [+o wiking] by ChanServ17:38
@wikingabhinav_7: hi! please write to the mailing list17:38
@HeikoSwuwei[m] what about we have a factory for xvalidation that accepts pipeline?17:40
wuwei[m]HeikoS: what do you mean?17:43
@HeikoSwuwei[m] have a factory method17:43
@HeikoSCCrossValidation* xval(CPipeline* pipeline, ... )17:43
@HeikoSand inside it sets the "machine" member17:43
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]17:45
wuwei[m]I think this works, we can only add it to swig17:45
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun17:45
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]17:45
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun17:45
-!- mode/#shogun [+o wiking] by ChanServ17:45
wuwei[m]this is the current factory method: machine_evaluation("CrossValidation", machine=pipeline, ...)17:46
wuwei[m]so we will have another factory that take a pipeline argument17:47
@HeikoSyes exactly17:47
@HeikoSthis way libshogun stay proper but we still can stick things together17:48
wuwei[m]machine_evaluation("CrossValidation", pipeline=pipeline, ...), this still translates to xval.put17:51
@HeikoSsure17:51
@HeikoSit will have to be17:52
@HeikoSmachine_evaluation("CrossValidation, my_pipeline_instance)17:52
@HeikoStranslating to a c++ factory with 2 arguments17:52
@HeikoSand then all the put stuff afterwards17:52
@HeikoSmaybe even this works17:52
wuwei[m]i see17:53
@HeikoSmachine_evaluation("CrossValidation", pipeline, foo=bar)17:53
@HeikoSfinally, there is the option to have an exception is the dispatcher for the interfaces17:53
@HeikoSwuwei[m] check out shogun.i17:54
@HeikoSthere are lots of such exceptions there already17:54
@HeikoSyou could sneak in a method put_machine_from_pipeline17:54
@HeikoSsimilar to say put_vector_or_matrix_from_double_matrix_dispatcher17:55
@HeikoSso you actually define c++ code that overloads the original "put"17:55
@HeikoSand then checks the type, potentially casts it and then calls the CSGObject::put17:55
wuwei[m]I think users may get confused with two factories, they have to use keyward argument machine=..., instead of positional one, if that is not a pipeline17:56
@HeikoSyep17:56
@HeikoSagree17:56
@HeikoSso then dispatch within put17:56
@HeikoSnote that if you do that, the c++ meta example will complain17:56
@HeikoSand if you put it into libshogun, then viktor will complain17:57
@HeikoSwhich reminds me17:57
wuwei[m]haha17:57
@HeikoSwiking maybe we should drop the cpp meta examples17:57
@HeikoSthey dictate a lot of shitty stuff to libshogun arising from the "unified API" concept17:57
@HeikoSsee discussion above17:58
@HeikoSshogun.i overloading of put for example17:58
@HeikoSwould be fine with putting another dispatcher for put into the interface code, but it shouldnt really be in libshogun imo17:58
@HeikoSbut then meta examples are kinda blocked for pipeline and xvalidation, so maybe the cpp ones should be just removed17:59
@HeikoSin fact the API  *is* different between libshogun and swig17:59
@HeikoSso maybe not show them as the same but have a second set of libshognu examples, or even no examples instead17:59
@HeikoSthoughts?17:59
@HeikoSlisitsyn ^17:59
@HeikoSwuwei[m] I am signing off, maybe discuss this with the guys so we can move forward with the issue18:02
@HeikoSgf712 see you later18:02
wuwei[m]see you later18:03
gf712see you later18:05
-!- wiking [~wiking@huwico/staff/wiking] has quit [Remote host closed the connection]18:07
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has joined #shogun18:08
-!- wiking [~wiking@c-185-45-237-122.customer.ggaweb.ch] has quit [Changing host]18:08
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun18:08
-!- mode/#shogun [+o wiking] by ChanServ18:08
-!- HeikoS [5aae0475@gateway/web/cgi-irc/kiwiirc.com/ip.90.174.4.117] has quit [Ping timeout: 240 seconds]18:12
-!- gf712 [8028b333@gateway/web/freenode/ip.128.40.179.51] has quit [Quit: Page closed]18:15
abhinav_7@wiking I have already sent a mail to shogun-list-subscribe@shogun-toolbox.org with the subject "Subscribe". But I haven't got any reply till now.18:27
@wikingmmm18:27
@wikinglemme check18:27
-!- abhinav_7 [76b9ec98@gateway/web/freenode/ip.118.185.236.152] has quit [Ping timeout: 256 seconds]19:39
-!- gf712 [560fc4e5@gateway/web/freenode/ip.86.15.196.229] has joined #shogun21:42
-!- gf712 [560fc4e5@gateway/web/freenode/ip.86.15.196.229] has quit [Ping timeout: 256 seconds]21:48
-!- Trixis [~Trixis@unaffiliated/trixis] has quit [Ping timeout: 240 seconds]21:58
-!- Trixis [~Trixis@unaffiliated/trixis] has joined #shogun22:00
--- Log closed Thu Jan 24 00:00:41 2019
« Tuesday, 2019-01-22 Index Thursday, 2019-01-24 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!