IRC logs of #shogun for Wednesday, 2018-05-16

« Tuesday, 2018-05-15 Index Thursday, 2018-05-17 »
--- Log opened Wed May 16 00:00:08 2018
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4285 synchronized by vinx1305:48
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/428505:51
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4235 synchronized by shubham80808:25
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4285 synchronized by vinx1308:56
@wikingwuwei, make sure that you use the right indenting for the python examples09:35
@wikingsome of them are wrong atm09:35
wuweiall right, let me fix them09:36
-!- travis-ci [~travis-ci@ec2-54-81-74-116.compute-1.amazonaws.com] has joined #shogun09:42
travis-ciit's Shubham Shukla's turn to pay the next round of drinks for the massacre he caused in shubham808/shogun: https://travis-ci.org/shubham808/shogun/builds/37956633209:42
-!- travis-ci [~travis-ci@ec2-54-81-74-116.compute-1.amazonaws.com] has left #shogun []09:42
-!- travis-ci [~travis-ci@ec2-54-81-74-116.compute-1.amazonaws.com] has joined #shogun10:03
travis-ciit's Shubham Shukla's turn to pay the next round of drinks for the massacre he caused in shubham808/shogun: https://travis-ci.org/shubham808/shogun/builds/37956633210:03
-!- travis-ci [~travis-ci@ec2-54-81-74-116.compute-1.amazonaws.com] has left #shogun []10:03
wuweihi wiking, i'll be out a bit, can we delay our meeting an hour later (1800 beijing time)10:33
@wikingsire10:36
@wikingsure10:36
@wikingno worries10:36
-!- micmn [uid216702@gateway/web/irccloud.com/x-gjcsjjkiileleoin] has joined #shogun11:16
wuweiwiking: hey12:01
@wikinghey hey12:01
@wikingjust q asec12:03
@wikingok here12:04
wuweisure12:04
@wikingok so i haven't got time yet to thoroughly go trough the patch12:05
@wikingbut the question i just had in mind thinking about transformers and machines12:05
@wikingis the following12:05
@wikingah ok nothing12:05
@wikingor maybe yeah i'm not so sure12:05
@wikingso i'm thinking about say we have a DAG (directed graph) where u define your pipeline12:06
@wikingwhere each node is a simple Machine or Transformer12:06
wuweiok12:06
@wikingso my problem is that how we would store this DAG12:08
wuweiso u want everything, features, labels, in a single pipeline?12:10
@wikingno12:13
@wikingPipeline: CMachine12:14
@wikingso it has trian and apply12:14
@wikingbut internally it just executes the stack of transformers and machines12:14
wuweii see12:15
wuweii'm not sure, api for operating a graph may be complex for users12:18
wuweicompared with a linear list12:19
@wikingyeah i mean interally it's a graph12:20
@wikingbut as a use12:20
@wikingyou would rather just do12:20
@wikingPipeline::append(M/T)12:20
@wikingor ::add12:20
@wikingbut the question is more how do you store those objs internally12:21
@wikingi mean you could do a std::vector<SGObjectU*>12:21
@wikingbut then you need to infer the type every time you execute the pipeline12:21
wuweiyeah12:23
wuweibut we have to use some 'if' in runtime, because both preproc and machine are allowed12:26
wuweiby either RTTI or storing the type in some way12:28
@wikingwuwei, yeah12:39
@wikingwhich would not be the case if everything woudl be a machine12:39
@wiking:)12:39
wuweiokay but I haven't got some idea now12:43
wuweibtw could u elaborate a bit how this graph is built internally12:44
@wikingok say that we only now support linear12:45
@wikingpipeline12:45
@wikingso it's a simple A->B->C->D12:45
@wikingas an example12:45
@wikingi want to do a simple standardScaling(Transformer)->LinearSVM(Machine)(12:46
@wikingso you do12:46
@wikingauto p = new CPipeline();12:46
@wikingp->add(standardScaling); p->add(LinearSVM);12:46
@wikingand then you would do12:46
@wikingp->train(Features)12:46
@wikingand later do p->apply(featureS)12:46
@wikingnow the question internally12:46
@wikinghow do you represent this pipeline12:46
wuweiin this simple example I can use a list, and iterate over the list when applying12:49
@wikingwuwei, yeah but what do you store in std::vector?13:21
wuweithat should be SGObject*, which is bad :)13:23
-!- wiking [~wiking@huwico/staff/wiking] has quit [Read error: Connection reset by peer]13:43
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun13:44
-!- mode/#shogun [+o wiking] by ChanServ13:44
@wikingwuwei, yeah that's my oprobllem actually... that it's way too general13:44
@wikingbecause every time you train13:44
@wikingor apply13:44
@wikingyou'll do a type inference for every obj13:44
@wikingwuwei, of course you can store an std::variant :P13:45
@wikingthe problem that it's c++1713:46
@wikinghttps://github.com/mpark/variant13:48
wuweii prefer variant, as we don't actually need SGObject*, for only preproc and machine13:49
@wikingyep13:49
@wikingso this way it's possible13:49
@wikingdunno what lisitsyn thinks ^13:49
@wikingwuwei, note that plz only use this implementation of variant via a macro13:54
@wikingsomething like13:54
@wiking#define SG_VARIANT mpark::variant13:54
@wikingso that flipping to std::variant should be easy13:54
wuweisure13:54
@wikingor of course we can do this nicer13:54
@wikingwith13:54
@wikingtemplate<VARAGS... > using variant = typename ...13:55
@wikingright?13:55
@wiking:)13:55
wuweiyeah13:56
@wikingok13:57
@wikingdo you have any blockers atm?13:57
wuweia type alias, right?13:57
@wikingyes13:57
@wikinga type alias13:57
@wikingwuwei, can you actually prepare a patch against develop?13:58
@wikingnot urgent13:58
@wikingbut when you have some idle time13:58
@wikingand actually make it so that you check in cmake whether std::variant is available or not13:58
wuweii'm trying to support inplace mode for transformers now13:59
@wikingsee for example HAVE_LGAMMAL13:59
@wikingand then something like HAVE_STD_VARIANT13:59
@wikingand basically do the corresponding type alias in the code14:00
@wikingso if somebody uses a compile that supports c++17 then it would default to std::variant14:00
@wikingwuwei, ok great :)14:00
@wikingthat's gonna be a bit tricky14:00
@wiking:P14:00
@wikingbut yeah for things like standardization14:00
@wikingwe need that14:00
@wikingbtw14:00
@wikingwhile you are changing so much stuff14:00
@wikingcan we actually standardize the names of some of our preprocessors14:01
@wiking:)14:01
@wikingfor example CPruneVarSubMean14:01
@wikingwe should have a simple CStandardizer14:01
@wikingor something liek that14:01
@wiking(see StandardScaler in sklearn)14:01
@wikingthat has a flag for doing unit variance (true/false)14:02
@wikingand dunno if we wanna extend this class to a CPruneFeature class14:02
@wikingthat does the pruning in case the variance is lower than a given threshold14:02
wuweisure14:02
@wikingi dont really like the idea having 2 flags in CStandardizer14:03
@wikingunit variance = true/false; prune = true/false14:03
@wiking:)14:03
@wikingbut yeah there are other preprocessors that has some cryptic name14:03
@wikingfeel free to suggest different naming14:03
@wikingwe can discuss that in the PR14:03
@wiking:)14:03
@wikingwuwei, great! thnx a lot!14:04
wuweimy problem for now is14:04
wuweiimplementing inplace mode, and now i'm trying to create a clone in base class and calls implementation like apply_to_feature_matrix of subclass14:05
@wikingoh i see14:05
wuweiand my idea is to share underlying data, but create a new instance instead14:06
wuweiso features can still be immutable14:06
wuweiim not so sure14:06
@wikingah14:07
@wikingit's a bit tricky14:07
@wiking:)14:07
wuweibut this can be a bit more refactor14:07
@wikingsure14:07
@wikingi mean any idea you have just put it in14:07
@wikingit's better to just do it14:07
@wiking(not spend too much time atm on either this or that)14:07
wuweisure14:07
@wikingand just see how it works14:07
@wikingand discuss later14:07
wuweibtw there are some apis in subclasses of preproc like apply_to_string() apply_to_feature_matrix, how about remove or make them protected? i think them confusing14:09
@wikingyes14:10
@wikingthose are old apis14:10
@wikingshouldn't be exposed like that anymore14:11
wuweigreat!14:11
-!- sukey [~nodebot@ks312251.kimsufi.com] has quit [Remote host closed the connection]17:48
-!- sukey [~nodebot@ks312251.kimsufi.com] has joined #shogun17:49
-!- mode/#shogun [+o sukey] by ChanServ17:49
@wikinghttps://www.vitavonni.de/blog/201503/2015031201-the-sad-state-of-sysadmin-in-the-age-of-containers.html18:44
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4286 opened by shubham80820:51
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4286 synchronized by shubham80820:55
--- Log closed Thu May 17 00:00:10 2018
« Tuesday, 2018-05-15 Index Thursday, 2018-05-17 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!