Open in new window / Try shogun cloud
--- Log opened Thu Aug 10 00:00:32 2017
micmnwiking: hey10:10
@wikinghihi10:11
micmndid you get my mails in the past few days?10:11
@wikingyes10:11
@wikingI got the one10:11
@wikingwas there more than one?10:11
@wikingi've pinged you here yesterday10:12
@wikingbut i think we've missed each otehr10:12
@wiking*other10:12
micmnone yesterday and one 4-5 days ago that's all10:12
micmnmmm sorry didn't see it in the log10:13
@wikingnevermind10:14
@wikingi'm just having a minor ssytem crash10:14
@wikingso backing up stuff10:14
@wikingand trying to recover10:15
@wikingbut if you are around in 2-3 hours10:15
@wikingwe could discuss the email10:15
micmnsure10:15
@wikingthnx10:22
@wikingmicmn, here11:11
@wikingso11:11
@wikingping me when you are around11:11
micmnwiking: ping!11:13
@wikingk11:14
@wikingso let's discuss the things now about the features11:14
@wikinglemme pull up your stuff11:15
micmnpreprocessors: as Heiko said they should either apply features on-the-fly or create a new features instance using fresh memory, right?11:17
@wikingok so here's a story11:17
@wikingthat is actually in a way related11:17
@wikingmicmn, so this11:20
@wikinghttps://github.com/shogun-toolbox/shogun/issues/395211:20
@wikingmicmn, so see what sorts of troubles we might have actually :)(11:21
micmnchecking11:22
micmnwiking: so debugging this might be useful to see how features should not be designed? is that the point?11:26
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3960 synchronized by MikeLing11:26
@wikingmicmn, indeed11:29
@wikingmicmn, lemme know when had a chance to read about it11:44
@wikingand then let's talk a bit more11:44
micmnwiking: ok I'm checking what's going on11:45
@wikingi mean dont need to debug11:45
@wikingjust the problem there11:45
@wikingto understand11:45
@wikinghonestly i'm not so sure if the exact problem is what the person says there11:46
@wikingbut there's something for sure11:46
micmnsure I've read it11:47
@wikingok so i'm just wondering how would an on-the-fly solution look like?11:54
micmnmmm11:56
micmnspeaking of iterators, they should remove mean/pruned feats before doing i.e. the dot product11:58
micmnI think the idea is having like a stack of preprocessors11:59
micmnapplied to the features11:59
micmnbut not sure how it would work internally12:04
micmnwiking: ok the problem is here https://github.com/micmn/shogun/blob/0b713865cf064e6a4fdc80cf4aae6c511455b34e/src/shogun/regression/LeastAngleRegression.cpp#L148-L15112:12
micmnmy guess is: PruneStuff does features->set_num_features(m) == features.num_features = m12:14
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3960 synchronized by MikeLing12:15
micmnthen lars features->get_feature_matrix() blows everything since features->set_num_features() doesn't have any effect on the matrix12:16
micmnin fact by adding feature_matrix.num_rows = num; to set_num_features(num) the script work12:36
@wikingmicmn, sorry man i've got ambushed by some people for some meetings15:42
micmnnw :P15:42
@wikingmmm ok i'm trying to see where's the problem15:45
@wiking"in fact by adding feature_matrix.num_rows = num; to set_num_features(num) the script work"15:45
@wikingwhat script?15:45
micmnthe example reported by that guy15:46
@wikingoh man15:52
@wikingCPruneVarSubMean15:52
@wikinghas some serious hard coded stuff as well15:52
@wikingmicmn,  about SGMatrix iterators16:04
@wikingalthough for the time being we'll have to go with that16:04
@wikingbut the ideas is to remove most of the functionalities from SGMatrix as that's really just a wrapper for SWIG interfaces16:05
@wikingbut yeah until we dont have a typeless Matrix16:05
@wikingwe'll have to make due with SGMatrix16:05
@wikingi'm just wondering how would we do the on the fly mutations with preprocessors?16:05
micmnyeah me too!16:05
@wikingah ok :)16:06
@wikingyeah i mean if we could force iterator only access16:06
@wikingthat's actually not something impossible :P16:06
micmnanother point is: say we apply two preprocessors not on-the-fly16:07
micmnwhat happens? one copy is useless16:07
@wikingmm you mean preprocessor that does a copy of the feature?16:08
micmnyeah16:08
@wikingPrune(normalize(feature))16:08
@wiking?16:08
@wikingyeah this is the story16:08
@wikingof eigen :)16:08
@wikingi mean that you can optimise these things out that no need for the temporary varialb16:08
@wiking*variable16:08
micmnif the users do feats=...; zero_mean_feats=feats.preprocess(zero_mean_preprocessor); norm_one_feats=zero_mean_feats.preprocess(norm_one_preprocessor)16:09
@wikingyep yep16:09
@wikingwell we should start somewhere16:11
@wikingi mean there's not enough time to wrap up evyerhint16:11
@wiking*everything but at least a way to go/start would be nice :)16:12
micmnso how would you design the interaction between features and preprocessor?16:12
@wikingyeah that's a good question16:12
@wikingas said16:12
@wikingif an algo16:12
@wikingcan only interract with features16:12
@wikingvia iterators16:12
@wikingthings are a bit easier16:12
@wikingbecause you are just wrapping iterators right? :)16:13
@wikingi'm just wondering if we could enforce this16:13
micmnyeah but of course there are linalg algorithms16:13
micmnthat use svd, qr etc16:13
@wiking:D16:13
@wikingthat work directly on the matrix :D16:13
@wikingand than it's kabooooooom16:13
micmnso in such a case16:13
micmnwe enforce a copy16:14
@wikingyep16:14
@wikingthat's the only way16:14
@wikingimmutable features16:14
@wiking:P16:14
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has joined #shogun16:40
-!- mode/#shogun [+o HeikoS] by ChanServ16:40
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has quit [Client Quit]16:40
-!- iglesiasg [~iglesiasg@217.119.234.214] has quit [Quit: Lost terminal]16:40
micmnwiking: "i mean that you can optimise these things out that no need for the temporary variable", how? is it the lazy evaluation story or?17:14
@wikingmmm compiler does it for eigen17:15
@wiking:)17:15
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has joined #shogun17:16
-!- mode/#shogun [+o HeikoS] by ChanServ17:16
-!- HeikoS [~heiko@host-92-0-169-11.as43234.net] has quit [Ping timeout: 240 seconds]17:20
micmnah expression templates cool :)17:27
--- Log closed Fri Aug 11 00:00:34 2017