| --- Log opened Fri Feb 09 00:00:54 2018 | ||
| -!- witness [uid10044@gateway/web/irccloud.com/x-vtjaiunahhborlaw] has joined #shogun | 03:08 | |
| -!- HeikoS [~heiko@host86-129-231-92.range86-129.btcentralplus.com] has joined #shogun | 10:41 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 10:41 | |
| @HeikoS | lisitsyn: nice! | 10:53 |
|---|---|---|
| lisitsyn | HeikoS: do you want me to fix the message or we merge it as is? ;) | 10:57 |
| @HeikoS | nah merge it | 10:57 |
| @HeikoS | Ill do that :) | 10:57 |
| lisitsyn | unit tests passed locally | 10:58 |
| lisitsyn | but travis always timeouts | 10:58 |
| lisitsyn | HeikoS: can we maybe ask travis to raise the limit? | 10:58 |
| lisitsyn | we lack just ten minutes usually | 10:58 |
| @HeikoS | yep | 10:58 |
| @HeikoS | i dont know how to do that :) | 10:58 |
| lisitsyn | lets yell until they raise | 10:58 |
| lisitsyn | TRAVIS RAISE THE LIMIT | 10:58 |
| lisitsyn | TRAVIS | 10:58 |
| lisitsyn | TRAVIS | 10:58 |
| @HeikoS | TRAAAVIIIIISSS | 10:58 |
| lisitsyn | didn't help yet | 10:58 |
| @HeikoS | you didnt yell loud enough | 10:59 |
| @HeikoS | LOUDER! | 10:59 |
| @HeikoS | lisitsyn: merge! | 11:03 |
| lisitsyn | ok | 11:03 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4162 merged by lisitsyn | 11:03 |
| @sukey | [https://github.com/shogun-toolbox/shogun] New commit https://github.com/shogun-toolbox/shogun/commit/d1763b83fbc029bae1487201ef56977efdfe43a0 by lisitsyn | 11:03 |
| lisitsyn | HeikoS: okk cast is next | 11:03 |
| @HeikoS | yep do it | 11:03 |
| @HeikoS | I will prepare the interface in the meantime | 11:03 |
| @HeikoS | swig will only have sgobj | 11:03 |
| @HeikoS | c++ will have all types | 11:03 |
| @HeikoS | and we register as sgobj | 11:03 |
| @HeikoS | so SG_ADD stays | 11:03 |
| lisitsyn | yeap | 11:11 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4163 opened by syashakash | 11:16 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4163 | 11:17 |
| lisitsyn | HeikoS: do you do permutation tests often? | 11:17 |
| @HeikoS | lisitsyn: yes | 11:18 |
| lisitsyn | HeikoS: should we have something for that in shogun? | 11:18 |
| @HeikoS | yes | 11:18 |
| @HeikoS | ultrafast mmd permutation test | 11:18 |
| @HeikoS | but it is based on kernel-test | 11:18 |
| @HeikoS | lambday wrote this cache friendly implementation | 11:18 |
| lisitsyn | HeikoS: you know I teach some kind of data science | 11:18 |
| lisitsyn | and instead of `statistics` I teach two things | 11:19 |
| lisitsyn | permutation test for A/B | 11:19 |
| @HeikoS | http://www.shogun-toolbox.org/notebook/latest/mmd_two_sample_testing.html | 11:19 |
| lisitsyn | and bootstrapping for confidence | 11:19 |
| lisitsyn | :) | 11:19 |
| lisitsyn | that's it | 11:19 |
| @HeikoS | I think this might be useful | 11:19 |
| @HeikoS | the test statistic is based on kernels | 11:19 |
| @HeikoS | but that doesnt matter to much | 11:19 |
| @HeikoS | it is a non-parametric two -sample test | 11:19 |
| lisitsyn | yeah but this is a bit more advanced | 11:19 |
| @HeikoS | and you need to permute to compute the null | 11:19 |
| @HeikoS | if you treat the test statistic as a black box? | 11:20 |
| @HeikoS | just say: "it is a t-test in a feature space" | 11:20 |
| lisitsyn | haha | 11:20 |
| lisitsyn | say to whom? | 11:20 |
| @HeikoS | the notebook samples from null and alternative via permutation | 11:20 |
| @HeikoS | to your students? :) | 11:20 |
| lisitsyn | t-test in a feature space? :) | 11:20 |
| lisitsyn | I explicitly avoid t-tests | 11:21 |
| lisitsyn | my point is that | 11:21 |
| lisitsyn | if you don't want to be smart | 11:21 |
| lisitsyn | just use your computer | 11:21 |
| lisitsyn | :) | 11:21 |
| -shogun-buildbot:#shogun- Build trusty - libshogun - viennacl #321 is complete: Failure [failed test (failure)] - http://buildbot.shogun-toolbox.org:8080/#builders/6/builds/321 | 11:21 | |
| lisitsyn | like don't care about distribution of statistics, just use permutation | 11:21 |
| lisitsyn | HeikoS: apart from shogun | 11:23 |
| lisitsyn | do you think it is a good idea to avoid all the classical stats | 11:23 |
| lisitsyn | but permutation and bootstrap | 11:23 |
| @HeikoS | lisitsyn: then use the kernel test | 11:23 |
| @HeikoS | that's exactly the point of it :) | 11:23 |
| @HeikoS | lisitsyn: I think you might be mixing two different things | 11:23 |
| lisitsyn | what things? | 11:23 |
| @HeikoS | permutation is just to get your hands on the calibration level of the test | 11:24 |
| @HeikoS | but the test itself has some kind of classes of problems that it can solve associated to it | 11:24 |
| @HeikoS | the permutation is a tool that makes your test work | 11:24 |
| @HeikoS | not a test itself | 11:24 |
| lisitsyn | HeikoS: no-no I am not mixing :) | 11:24 |
| @HeikoS | maybe I dont understand what you mean by permutation :) | 11:24 |
| lisitsyn | HeikoS: ok I say sometimes you have A/B problem | 11:24 |
| @HeikoS | what is that? | 11:24 |
| lisitsyn | you say measure means of something | 11:25 |
| lisitsyn | like A/B testing | 11:25 |
| lisitsyn | two samples | 11:25 |
| lisitsyn | :) | 11:25 |
| @HeikoS | ah ok | 11:25 |
| @HeikoS | yes | 11:25 |
| lisitsyn | HeikoS: so you get difference of 0.3 | 11:25 |
| @HeikoS | is p(a)=p(b)? | 11:25 |
| @HeikoS | what different? | 11:25 |
| @HeikoS | in mean? | 11:25 |
| lisitsyn | yeah means are different by 3% | 11:25 |
| lisitsyn | can you believe it or it is by chance | 11:25 |
| @HeikoS | that's a t-test 8-) | 11:25 |
| lisitsyn | HeikoS: yes in case of means | 11:25 |
| lisitsyn | but | 11:25 |
| lisitsyn | HeikoS: I say we need P(D|H0) | 11:26 |
| lisitsyn | so lets make H0 true | 11:26 |
| @HeikoS | yes sure | 11:26 |
| @HeikoS | now I know what you want | 11:26 |
| lisitsyn | permute the thing | 11:26 |
| @HeikoS | you want to know how much variation is in there by chance under H0 | 11:26 |
| lisitsyn | HeikoS: exactly | 11:26 |
| lisitsyn | and how often we get such D | 11:26 |
| lisitsyn | by chance | 11:26 |
| lisitsyn | aaand p-value | 11:26 |
| lisitsyn | HeikoS: that's how I avoid t-test, chi2 test whatever | 11:27 |
| lisitsyn | I just say if you have two samples and you can compare them | 11:27 |
| lisitsyn | permute and measure few times | 11:27 |
| lisitsyn | do you like this approach? | 11:27 |
| lisitsyn | (I have limited time to teach stats) | 11:27 |
| @HeikoS | but wait | 11:27 |
| @HeikoS | that IS a t test | 11:27 |
| @HeikoS | but an expensive way to do it | 11:27 |
| @HeikoS | if you compare the mean | 11:28 |
| lisitsyn | HeikoS: in case of mean yes | 11:28 |
| lisitsyn | but you can replace mean with something else | 11:28 |
| @HeikoS | then you know the asymptotic distribution of the difference under Hss | 11:28 |
| lisitsyn | and follow the same approach | 11:28 |
| @HeikoS | H0 | 11:28 |
| @HeikoS | yeah | 11:28 |
| @HeikoS | true | 11:28 |
| lisitsyn | that's the point | 11:28 |
| @HeikoS | so the kernel thing | 11:28 |
| lisitsyn | say comparing medians (which might be stupid) | 11:28 |
| @HeikoS | does mean in kernel space | 11:28 |
| @HeikoS | which captures everythign you want | 11:28 |
| lisitsyn | HeikoS: oh well I understand that but I don't have any clue how to teach that at the same time | 11:29 |
| lisitsyn | but having that in shogun is good for sure :) | 11:29 |
| @HeikoS | lol | 11:29 |
| @HeikoS | in shogun | 11:29 |
| @HeikoS | there is a framework to do permutation tests with any statistic | 11:29 |
| @HeikoS | so you could modify it for mean/median | 11:29 |
| @HeikoS | whatever you want | 11:29 |
| lisitsyn | HeikoS: yeah but I am checking your opinion on that teaching approach :) (separate from shogun) | 11:30 |
| @HeikoS | ah I see | 11:30 |
| @HeikoS | I think you want to first explain t-test :) | 11:30 |
| @HeikoS | and that it is comparing mean | 11:30 |
| @HeikoS | and then you want to explain the problem of calibration, i.e. how much variation is to expect | 11:30 |
| @HeikoS | then you introduce permutation test | 11:30 |
| @HeikoS | but then you also say, that for t-test the distribution of the statistic under H0 is closed form | 11:30 |
| lisitsyn | HeikoS: ok ok but we're talking about people who desperately want to avoid maths | 11:31 |
| lisitsyn | :) | 11:31 |
| lisitsyn | it is not students from uni | 11:31 |
| lisitsyn | it is data science course | 11:31 |
| @HeikoS | you don't need to explain math I think | 11:31 |
| @HeikoS | just say THAT this thing is also known in case of comparing mean | 11:32 |
| @HeikoS | no good? :) | 11:32 |
| lisitsyn | yeah I say that actually | 11:32 |
| lisitsyn | I say that if you're sure it is the test you want | 11:32 |
| lisitsyn | use it | 11:32 |
| lisitsyn | but if you don't care - permute | 11:32 |
| @HeikoS | but wait | 11:33 |
| @HeikoS | permute is just a tool | 11:33 |
| @HeikoS | what matters is the test statistic | 11:33 |
| @HeikoS | if you permute and compute difference in means | 11:33 |
| @HeikoS | that is ttest | 11:33 |
| @HeikoS | so what you care about is the statistic you use | 11:33 |
| @HeikoS | the statement "if you are unsure, just permute" I think it doesnt make sense | 11:34 |
| @HeikoS | since you still need to decide on what is the test statistic | 11:35 |
| -shogun-buildbot:#shogun- Build deb1 - libshogun #299 is complete: Failure [failed test (failure)] - http://buildbot.shogun-toolbox.org:8080/#builders/10/builds/299 | 11:38 | |
| durovo1 | HeikoS: Hi, the follwing paper was used as a reference for MulticlassStrategy and MulticlassMachine prob calibration: <https://hal.inria.fr/inria-00103955/document> | 11:42 |
| durovo1 | Currently, for OVA_SOFTMAX, the parameters fitted on a Sigmoid (the parameters learned at line 132 of MulticlassMachine.cpp and used for softmax at line 158, the implementation of softmax is in MultilcassStrategyOneVsRest.cpp line 108) are being used. However, the probability output in softmax is different from sigmoid (see eq 7 in the paper). Is it correct to use parameters fitted on a sigmoid function in a softmax function? | 11:42 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4163 | 11:45 |
| @HeikoS | durovo1: hello | 11:45 |
| @HeikoS | nopr | 11:46 |
| @HeikoS | nope | 11:46 |
| @HeikoS | probably not | 11:46 |
| @wiking | lisitsyn, bazdmeg! | 11:46 |
| @HeikoS | durovo1: you have a suggestion to fix? :) | 11:46 |
| @wiking | lisitsyn, bazdmeg i'm serious bazdmeg! | 11:46 |
| lisitsyn | HeikoS: uhmm yes you still have to decide but you might not care about the distribution of statistics | 11:47 |
| lisitsyn | wiking: bazdmeg? | 11:47 |
| @wiking | bazdmeg why u ignore me? :) | 11:47 |
| @wiking | bazdmeg | 11:47 |
| @wiking | :D | 11:47 |
| durovo1 | HeikoS: I'll try searching for one. | 11:48 |
| lisitsyn | HeikoS: you still can compare quantiles of 0.25 of two samples with permutation, right? | 11:48 |
| @HeikoS | lisitsyn: I think if you know that someone figured out the distribution, then you use that. Otherwise, you just permute, agreed | 11:48 |
| @HeikoS | lisitsyn: you can | 11:48 |
| lisitsyn | what is the distribution of that I don't even know | 11:48 |
| @HeikoS | but it is expensive if someone has figured out that the distrubution is gaussian with known mean and variance | 11:48 |
| @HeikoS | like when you compare the means | 11:48 |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has joined #shogun | 11:48 | |
| travis-ci | it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339384345 | 11:48 |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has left #shogun [] | 11:48 | |
| @HeikoS | you *know* it | 11:48 |
| lisitsyn | HeikoS: true | 11:48 |
| lisitsyn | that's exactly my point | 11:48 |
| @HeikoS | durovo1: maybe you can put up an example as a gist? | 11:49 |
| @HeikoS | durovo1: or give me a link to the source files you mean | 11:49 |
| lisitsyn | choosing statistics + permutation is a bit easier but if you know then you know :) | 11:49 |
| @HeikoS | lisitsyn: yes | 11:49 |
| @HeikoS | lisitsyn: one thing | 11:49 |
| @HeikoS | really important | 11:49 |
| @HeikoS | if you do permutation | 11:49 |
| @HeikoS | that needs to be done on a hold out dataset | 11:49 |
| @HeikoS | and then the test itself has to be performed on different data | 11:49 |
| durovo1 | HeikoS: I am a bit confused here .. are you talking about the source code of the solution when I find it? | 11:50 |
| @HeikoS | so the p-value is computed on training data, and then the test is computed on test data | 11:50 |
| @HeikoS | durovo1: no the lines in shogun code, a github link | 11:50 |
| lisitsyn | HeikoS: uhm | 11:50 |
| durovo1 | HeikoS: Oh, sure | 11:50 |
| lisitsyn | say you have two samples | 11:50 |
| lisitsyn | what do you do then? | 11:50 |
| @HeikoS | durovo1: like this https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/clustering/Hierarchical.cpp#L5 | 11:50 |
| lisitsyn | train/test split and then permute test and compare test? | 11:51 |
| @HeikoS | lisitsyn: you have 1000 samples each from A and B | 11:51 |
| @HeikoS | and you are interested in dist(A)==dist(B) | 11:51 |
| lisitsyn | 1000 permuted samples? | 11:51 |
| @HeikoS | nono data | 11:51 |
| lisitsyn | examples/points? | 11:51 |
| @HeikoS | yes | 11:51 |
| lisitsyn | ok | 11:51 |
| @HeikoS | then you divide each A and B into half | 11:51 |
| @HeikoS | then you do your permutation thingi on the first half of each A and B | 11:51 |
| lisitsyn | uhmm | 11:51 |
| @HeikoS | this gives you a threshold | 11:51 |
| lisitsyn | yes | 11:52 |
| @HeikoS | and then you compute the test statistic on the second half of A and B | 11:52 |
| lisitsyn | I don't do that | 11:52 |
| lisitsyn | :) | 11:52 |
| @HeikoS | and compare it against the threshold that you computed on the first half | 11:52 |
| lisitsyn | but this makes sense indeed | 11:52 |
| @HeikoS | then your test is wrong :) | 11:52 |
| lisitsyn | how wrong? | 11:52 |
| @HeikoS | pretty badly :) | 11:52 |
| @HeikoS | not consistent | 11:52 |
| @HeikoS | the permutation gives wrong answer | 11:52 |
| @HeikoS | and as you permute more and more, it doesnt converge to the true | 11:52 |
| durovo1 | HeikoS: understood, will do that in a few minutes | 11:52 |
| lisitsyn | HeikoS: ok let me cry for a bit | 11:53 |
| @HeikoS | lisitsyn: hehe | 11:53 |
| @HeikoS | man! | 11:53 |
| @HeikoS | it's all in the shogun notebook :D | 11:53 |
| @HeikoS | you should have read it :D | 11:54 |
| @HeikoS | lisitsyn: btw your error msg doesnt work | 11:54 |
| @HeikoS | 628: SystemError: [ERROR] In file /home/heiko/git/shogun/src/shogun/base/SGObject.h line 357: Type for parameter with name "kernel" is not correct. | 11:54 |
| lisitsyn | HeikoS: argh | 11:55 |
| @HeikoS | lisitsyn: because it does has first | 11:55 |
| @HeikoS | "has<..:>(name)" | 11:55 |
| lisitsyn | HeikoS: didn't change the message in put | 11:55 |
| @HeikoS | dont worry I can do | 11:55 |
| durovo1 | HeikoS: | 11:56 |
| durovo1 | fit sigmoid call in multiclass machine <https://github.com/shogun-toolbox/shogun/blob/d1763b83fbc029bae1487201ef56977efdfe43a0/src/shogun/machine/MulticlassMachine.cpp#L132> | 11:56 |
| @HeikoS | Ill take a little break, ill be back in 10 | 11:56 |
| durovo1 | understood | 11:57 |
| @HeikoS | lisitsyn: you didnt read the error message it produced :) | 12:12 |
| lisitsyn | HeikoS: I fixed the get :) | 12:12 |
| lisitsyn | not put | 12:12 |
| lisitsyn | that's the problem | 12:12 |
| @HeikoS | no I mean the one in put :) | 12:12 |
| lisitsyn | eh? | 12:12 |
| @HeikoS | 3 times %s but only two populated | 12:13 |
| @HeikoS | :D | 12:13 |
| lisitsyn | argh | 12:13 |
| lisitsyn | in get? | 12:13 |
| lisitsyn | ok happens | 12:13 |
| lisitsyn | :) | 12:13 |
| lisitsyn | HeikoS: well you wanted to change it anyway :P | 12:13 |
| @HeikoS | :P | 12:14 |
| lisitsyn | HeikoS: put is a bit different actually | 12:14 |
| lisitsyn | no exception is thrown | 12:14 |
| @HeikoS | yes | 12:14 |
| @HeikoS | BUT | 12:14 |
| @HeikoS | I changed that | 12:14 |
| lisitsyn | you can just try | 12:14 |
| lisitsyn | instead of checking | 12:14 |
| @HeikoS | there is this has<T>(tag) | 12:14 |
| @HeikoS | try | 12:14 |
| @HeikoS | { | 12:14 |
| @HeikoS | ref_value(&value); | 12:14 |
| @HeikoS | update_parameter(_tag, make_any(value)); | 12:14 |
| @HeikoS | } | 12:14 |
| @HeikoS | will this throw the same exception? | 12:15 |
| lisitsyn | HeikoS: should do | 12:15 |
| @HeikoS | I replaced "has" with try | 12:15 |
| @HeikoS | kk | 12:15 |
| @HeikoS | lisitsyn: ah no doesnt compile | 12:16 |
| @HeikoS | I think I will just try the any_cast you did in get? | 12:16 |
| lisitsyn | HeikoS: yes | 12:16 |
| lisitsyn | just baaam | 12:16 |
| lisitsyn | and if it fails | 12:16 |
| lisitsyn | ok | 12:16 |
| lisitsyn | :) | 12:16 |
| lisitsyn | HeikoS: can you point me to any book that has that permutation train/test split? | 12:17 |
| lisitsyn | is it always true? | 12:17 |
| @HeikoS | only math books | 12:17 |
| @HeikoS | yes | 12:17 |
| lisitsyn | yes but I can read math you know | 12:17 |
| lisitsyn | HeikoS: if you glance over things like http://genomicsclass.github.io/book/pages/permutation_tests.html | 12:18 |
| lisitsyn | no mention of that | 12:19 |
| lisitsyn | I can believe people mistake that but are you sure? :) | 12:19 |
| lisitsyn | can that be it is only the case of kernel mmd or something? | 12:20 |
| lisitsyn | I don't really get why you get *that* bad probabilities if you do t-test thingy with permutation | 12:20 |
| @HeikoS | let me think a bit | 12:21 |
| @HeikoS | if it is only kernel test | 12:22 |
| @HeikoS | since that is all I ever did | 12:22 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has joined #shogun | 12:22 | |
| travis-ci | it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339384345 | 12:22 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has left #shogun [] | 12:22 | |
| @HeikoS | lisitsyn: we had to do it as we were learning the kernel | 12:22 |
| durovo1 | HeikoS: fit_sigmoid call in MulticlassMachine: <https://github.com/shogun-toolbox/shogun/blob/d1763b83fbc029bae1487201ef56977efdfe43a0/src/shogun/machine/MulticlassMachine.cpp#L132> this calls is used to store the params in As and Bs. | 12:23 |
| durovo1 | Then, As and Bs are used to rescale outputs in this line: <https://github.com/shogun-toolbox/shogun/blob/d1763b83fbc029bae1487201ef56977efdfe43a0/src/shogun/machine/MulticlassMachine.cpp#L157> | 12:23 |
| durovo1 | This is only done when the probability heuristic is OVA_SOFTMAX | 12:23 |
| durovo1 | Finally, this is the rescale implementation (which is correct I believe) : <https://github.com/shogun-toolbox/shogun/blob/d1763b83fbc029bae1487201ef56977efdfe43a0/src/shogun/multiclass/MulticlassOneVsRestStrategy.cpp#L108> | 12:23 |
| @HeikoS | lisitsyn: forgive me | 12:24 |
| @HeikoS | I was wrong ! | 12:24 |
| @HeikoS | it is only if you are learning your test statistic | 12:24 |
| @HeikoS | but if it is fixed, all good | 12:24 |
| @HeikoS | sorry | 12:24 |
| @HeikoS | should have been more careful | 12:24 |
| @HeikoS | durovo1: checking | 12:24 |
| @HeikoS | durovo1: ok got the context, now what was the problem? | 12:25 |
| @HeikoS | that the sigmoid is computed differently that in some reference? | 12:25 |
| @HeikoS | i.e. we are comparing apples and pears? | 12:25 |
| durovo1 | yeah | 12:26 |
| @HeikoS | durovo1: I guess that is true | 12:26 |
| @HeikoS | so we need to have two different methods and use them where appropriate | 12:26 |
| durovo1 | parameters learnt for sigmoid are used in exp(A*out + B)/sum(A...) | 12:26 |
| durovo1 | Yeah | 12:26 |
| @HeikoS | durovo1: cool! | 12:28 |
| @HeikoS | can you make that a github issue? | 12:28 |
| @HeikoS | good to find these things! | 12:28 |
| durovo1 | Understood, i'll make that a github issue :slightly_smiling_face: | 12:29 |
| @HeikoS | lisitsyn: you hate me now? :D | 12:30 |
| @HeikoS | lisitsyn: btw is this reasonable to do in "put" | 12:30 |
| @HeikoS | try | 12:30 |
| @HeikoS | { | 12:30 |
| @HeikoS | any_cast<T>(get_parameter(_tag).get_value()); | 12:30 |
| @HeikoS | } | 12:30 |
| @HeikoS | nah will not work | 12:31 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Issue https://github.com/shogun-toolbox/shogun/issues/4164 opened by durovo | 12:39 |
| -!- king [5862c914@gateway/web/freenode/ip.88.98.201.20] has joined #shogun | 12:50 | |
| king | Hello | 12:50 |
| king | is it possible to get true positive rate etc out of shogun? I cant find it anywhere | 12:51 |
| @HeikoS | lisitsyn: | 12:52 |
| @HeikoS | 338: SystemError: [ERROR] In file /home/heiko/git/shogun/src/shogun/base/SGObject.h line 357: Setting parameter LibSVM::kernel failed. Provided type is shogun::CSGObject*, but actual type is shogun::CSGObject const*. | 12:52 |
| @HeikoS | 338: | 12:52 |
| @HeikoS | king: http://www.shogun-toolbox.org/api/latest/classshogun_1_1CContingencyTableEvaluation.html | 12:52 |
| @HeikoS | from those you can get it | 12:53 |
| king | thanks a lot! | 12:53 |
| king | legend | 12:53 |
| @HeikoS | king: or you compute it yourself in one line | 12:53 |
| king | how would i do that? | 12:54 |
| @HeikoS | ask google :) | 12:54 |
| king | lol | 12:54 |
| king | thanks for the link | 12:55 |
| king | sorry one more noob question while im here | 12:55 |
| king | how do i get info out of get_confusion_matrix | 12:56 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has joined #shogun | 12:56 | |
| travis-ci | it's Sergey Lisitsyn's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339385157 | 12:56 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has left #shogun [] | 12:56 | |
| king | i want to print the results | 12:56 |
| @HeikoS | what info? | 12:56 |
| king | but i know im not doing it right cos it just prints <built-in method get_confusion_matrix of SwigPyObjectType object at 0x00007FFC43572740> | 12:57 |
| king | or am i missing the point | 12:57 |
| @HeikoS | let me see | 12:57 |
| king | i do | 12:57 |
| king | conf_mat_knn = acc.get_confusion_matrix | 12:58 |
| king | print("Confusion Matrix: ", conf_mat_knn) | 12:58 |
| lisitsyn | HeikoS: ahaa so it is only for kernel | 12:58 |
| king | returns <built-in method get_confusion_matrix of SwigPyObjectType object at 0x00007FFC43572740> | 12:58 |
| @HeikoS | acc.get_confision_matrix() | 12:58 |
| king | ahh | 12:59 |
| @HeikoS | eval.get_TP() | 12:59 |
| @HeikoS | is what you want | 12:59 |
| king | you are a hero thanks so much | 13:01 |
| @HeikoS | :) | 13:10 |
| @HeikoS | lisitsyn: python and c++ now work | 13:11 |
| -!- king [5862c914@gateway/web/freenode/ip.88.98.201.20] has quit [Quit: Page closed] | 13:19 | |
| -!- king [5862c914@gateway/web/freenode/ip.88.98.201.20] has joined #shogun | 13:21 | |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has joined #shogun | 13:31 | |
| travis-ci | it's Sergey Lisitsyn's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339385157 | 13:31 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has left #shogun [] | 13:31 | |
| @HeikoS | lisitsyn: looks like we don't need casting patch for any | 13:43 |
| lisitsyn | HeikoS: ok cool | 14:53 |
| @HeikoS | lisitsyn: you know what it was? | 15:17 |
| @HeikoS | const vs non-const CSGObject | 15:17 |
| @HeikoS | for python at least | 15:18 |
| @HeikoS | just re-doing the octave fix in swig .i files and then see how it goes | 15:19 |
| @HeikoS | tag_float | 15:22 |
| @HeikoS | >> cast(4.0, "double") | 15:22 |
| @HeikoS | ans = 4 | 15:22 |
| @HeikoS | octave is amazing | 15:22 |
| -!- king [5862c914@gateway/web/freenode/ip.88.98.201.20] has quit [Quit: Page closed] | 15:22 | |
| @HeikoS | lisitsyn: you there? | 16:02 |
| lisitsyn | HeikoS: what works what not? | 16:23 |
| lisitsyn | :) | 16:23 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4165 opened by karlnapf | 16:30 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4165 | 16:31 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4165 | 16:31 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4165 | 16:32 |
| lisitsyn | HeikoS: tell me! | 17:06 |
| @wiking | HeikoS, | 17:09 |
| @wiking | https://github.com/shogun-toolbox/shogun/pull/4165/commits/1723a3bbf5b5932310e6ef1e5285fd1c6c9a6f1f#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R390 | 17:09 |
| @wiking | this is nopoint | 17:09 |
| @wiking | i mean the problem is that you are presumoing | 17:09 |
| @wiking | this way that there are 2 ways of bulding | 17:10 |
| @wiking | shogun itself :( | 17:10 |
| @wiking | skipping some stuff in the interface is cool | 17:10 |
| @wiking | but doing something else if it's swig is danger | 17:10 |
| @wiking | i would actually change https://github.com/shogun-toolbox/shogun/pull/4165/commits/1723a3bbf5b5932310e6ef1e5285fd1c6c9a6f1f#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R403 | 17:11 |
| @wiking | that it'd take anything and the tag above would have some magicmonkey | 17:11 |
| @wiking | to be actually taken for all classes | 17:11 |
| @wiking | HeikoS, the reason i kept these things out of a PR | 17:12 |
| @wiking | is because exactly the case | 17:12 |
| @wiking | that we can collaborate | 17:12 |
| @wiking | now you have moved this into your own repo | 17:12 |
| @wiking | i'll cherry pick your commit | 17:12 |
| @wiking | into the typedo | 17:12 |
| @wiking | and push it there | 17:12 |
| @wiking | and push on top what i mean what to do instead of #else | 17:12 |
| @HeikoS | wiking: mmh ok | 17:24 |
| @HeikoS | let me fix | 17:24 |
| @sukey | [https://github.com/shogun-toolbox/shogun] karlnapf pushed 6 commits: | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/bf4a64caacf5ac2fdd4d5f9cec4bc2f7c4f66404 | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/cfb220c62b091ec27a1aaaaf03e4d1f6ab60b15c | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/a7a79f16442336cd21ed6bcd470d93c57e9430df | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/8e84e681a2321dc3b34c14c22898b947d63a68e0 | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/e0436be3e6c2916e59110253ba5023d1fc257e79 | 17:32 |
| @sukey | https://github.com/shogun-toolbox/shogun/commit/cc07657ed940e39f6e657c368c4c0493b0c8c62b | 17:32 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4165 closed by karlnapf | 17:32 |
| @HeikoS | wiking: put it in typedo | 17:36 |
| @HeikoS | fixing the #else | 17:36 |
| @HeikoS | and also java | 17:36 |
| @HeikoS | and then ruby | 17:36 |
| @HeikoS | and all the others ... | 17:36 |
| @wiking | ruby? | 17:41 |
| @HeikoS | yep | 17:42 |
| @HeikoS | travis will tell | 17:42 |
| @wiking | ah i know | 17:43 |
| @wiking | yeye | 17:43 |
| @wiking | i get it | 17:43 |
| @wiking | yeah that one was a huge mess gsoc push :) | 17:43 |
| @wiking | but yeah i mean party they both worked | 17:43 |
| @wiking | the last for the c++ the prior for the others | 17:43 |
| @wiking | only the traits need to be a bit tricked | 17:43 |
| @wiking | and then it'll be fine | 17:43 |
| @HeikoS | haha | 17:44 |
| @HeikoS | ok I have the #else remove and it still works | 17:44 |
| @HeikoS | just doing a few checks and pushing | 17:44 |
| @HeikoS | java vector is a bit more tricky | 17:44 |
| @HeikoS | unless we restrict to float64, then it should be easy | 17:44 |
| @HeikoS | lisitsyn: jo! | 17:45 |
| lisitsyn | HeikoS: what is going on | 17:45 |
| lisitsyn | :) | 17:45 |
| @HeikoS | I think we might need the any cast after all | 17:45 |
| @HeikoS | :D | 17:45 |
| @HeikoS | Setting parameter LibSVM::kernel failed. Provided type is shogun::CKernel*, but actual type is shogun::CSGObject*. | 17:46 |
| @HeikoS | I can easily work around that for now | 17:46 |
| @HeikoS | but would be good to have | 17:46 |
| lisitsyn | HeikoS: ok but what works now? | 17:47 |
| @HeikoS | object works | 17:48 |
| @HeikoS | when we downcast it explicitly | 17:49 |
| @HeikoS | for all subclasses of SGObject | 17:49 |
| @HeikoS | since everything is registered as SGObject using SG_ADD | 17:49 |
| @HeikoS | but would be nicer if any could accept types that are castable into each other | 17:49 |
| @HeikoS | lisitsyn: https://github.com/shogun-toolbox/shogun/commit/cc07657ed940e39f6e657c368c4c0493b0c8c62b#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R384 | 17:50 |
| lisitsyn | HeikoS: ok | 17:50 |
| lisitsyn | ah | 17:50 |
| lisitsyn | tricky | 17:50 |
| lisitsyn | ! | 17:50 |
| @HeikoS | lisitsyn: if we don't downcast in put, we get | 17:52 |
| @HeikoS | Setting parameter LibSVM::kernel failed. Provided type is shogun::CKernel*, but actual type is shogun::CSGObject*. | 17:52 |
| @HeikoS | Ill put a todo note in there | 17:52 |
| @HeikoS | for now it works as SG_ADD downcasts | 17:52 |
| lisitsyn | HeikoS: ok but having it downcasted is mostly fine | 17:52 |
| lisitsyn | a | 17:52 |
| lisitsyn | but | 17:52 |
| @HeikoS | lisitsyn: it is ok | 17:52 |
| lisitsyn | we can set kernel to distance? | 17:53 |
| lisitsyn | right? | 17:53 |
| lisitsyn | put a kernel into features blabla? | 17:53 |
| @HeikoS | yeah | 17:53 |
| @HeikoS | because any just checks whether type matches | 17:53 |
| @HeikoS | not if is can be converted | 17:53 |
| lisitsyn | ok this would be fixed if we used actual type | 17:53 |
| @HeikoS | yes | 17:53 |
| lisitsyn | ok I'll check if I can fix that | 17:53 |
| @HeikoS | in the assignment, it needs to check whether the classes match | 17:54 |
| @HeikoS | then we can leave the SG_ADD | 17:54 |
| @HeikoS | make all new registers using the actual type | 17:54 |
| @HeikoS | and everything works | 17:54 |
| lisitsyn | HeikoS: no, no need to register with the actual type | 17:54 |
| @HeikoS | lisitsyn: I dont mean changing the old | 17:55 |
| @HeikoS | but for new classes | 17:55 |
| @HeikoS | we can avoid the static cast everywhere | 17:55 |
| @HeikoS | nicer | 17:55 |
| @wiking | ok done with grading | 17:57 |
| @wiking | lemme check the stuff | 17:57 |
| lisitsyn | HeikoS: oh trickier than I thought | 17:59 |
| @HeikoS | wiking: push incoming in 1 min | 17:59 |
| @wiking | k | 17:59 |
| @sukey | [https://github.com/shogun-toolbox/shogun] New commit https://github.com/shogun-toolbox/shogun/commit/08434857d7ae183e94d8b9c9c4d90b1b4b8193a1 by karlnapf | 18:09 |
| @wiking | HeikoS, idgetit | 18:10 |
| @wiking | https://github.com/shogun-toolbox/shogun/commit/08434857d7ae183e94d8b9c9c4d90b1b4b8193a1#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R384 | 18:10 |
| @wiking | ? | 18:10 |
| @wiking | why | 18:10 |
| @wiking | why [[deprecated("asdf")]] | 18:10 |
| @wiking | is not good? | 18:10 |
| @wiking | indeed | 18:10 |
| @wiking | it should be that | 18:10 |
| @wiking | so that every time you use that you get a compiler asdf | 18:10 |
| @wiking | btw this could be mosdef const | 18:11 |
| @wiking | https://github.com/shogun-toolbox/shogun/commit/08434857d7ae183e94d8b9c9c4d90b1b4b8193a1#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R399 | 18:11 |
| @wiking | sorry that i'm commenting here | 18:11 |
| @wiking | see this https://github.com/shogun-toolbox/shogun/commit/2f488e4558fe3d84e3babaa670714f3bd35a5803#diff-1689b0bd3ceb28355cc3c05ed7ffe4f0R369 | 18:12 |
| @HeikoS | ah noooo | 18:26 |
| @HeikoS | just hitting the same problem as for the explicit template version | 18:26 |
| @HeikoS | cannot make swig to match such methods: | 18:26 |
| @HeikoS | void put(const std::string& name, const SGMatrix<float64_t>& value) | 18:26 |
| @HeikoS | doesnt use typemap | 18:26 |
| @HeikoS | lisitsyn: ^:( | 18:26 |
| lisitsyn | HeikoS: again? :) | 18:27 |
| @HeikoS | yes | 18:27 |
| @HeikoS | now it is coming from the swig %extend | 18:27 |
| @HeikoS | (argument mismatch; DoubleMatrix cannot be converted to SWIGTYPE_p_SGMatrixT_double_t) | 18:28 |
| @wiking | HeikoS, const! | 18:28 |
| @HeikoS | is it the const? | 18:28 |
| @wiking | HeikoS, mmm | 18:28 |
| @wiking | i mean i dont get it | 18:28 |
| @HeikoS | it wasnt the const before | 18:28 |
| @wiking | lemme check your branch | 18:28 |
| @HeikoS | let me push | 18:28 |
| @wiking | but i dont get it | 18:28 |
| @HeikoS | 1 sec | 18:28 |
| @wiking | why did you change? :) | 18:28 |
| @wiking | there was a good reason | 18:28 |
| @wiking | why const and & has been removed | 18:28 |
| @wiking | :) | 18:28 |
| @wiking | i mean it was all working | 18:29 |
| @wiking | in swig | 18:29 |
| @wiking | :D | 18:29 |
| @HeikoS | it is still | 18:29 |
| @HeikoS | this is a different issue | 18:29 |
| @HeikoS | but maybe that fixes it | 18:29 |
| @wiking | this | 18:29 |
| @wiking | https://github.com/shogun-toolbox/shogun/commit/2f488e4558fe3d84e3babaa670714f3bd35a5803 | 18:29 |
| @HeikoS | let me push | 18:29 |
| @wiking | definitely | 18:29 |
| @wiking | worked with swig | 18:29 |
| @wiking | apart from | 18:29 |
| @wiking | SGO | 18:29 |
| @wiking | but that's obvious why | 18:29 |
| @wiking | i dont get | 18:29 |
| @wiking | what is the diff issue? | 18:29 |
| @HeikoS | no rememer when the typemap didnt match the SGMatrix | 18:30 |
| @wiking | when/ | 18:30 |
| @wiking | ? | 18:30 |
| @wiking | which case | 18:30 |
| @HeikoS | nevermind | 18:30 |
| @wiking | SGM has and was matched | 18:30 |
| @HeikoS | I will push and you can see | 18:30 |
| @HeikoS | what I mean | 18:30 |
| @wiking | that's 100% | 18:30 |
| @HeikoS | different thing I am talking about | 18:31 |
| @wiking | k push | 18:31 |
| @wiking | cbut | 18:31 |
| @wiking | can you put back | 18:31 |
| @wiking | the deprecation? | 18:31 |
| @wiking | plz | 18:31 |
| @sukey | [https://github.com/shogun-toolbox/shogun] New commit https://github.com/shogun-toolbox/shogun/commit/9e4a42f23b3c8224c6691980d7beae81a34d642f by karlnapf | 18:33 |
| @HeikoS | java still has this missing import ( I put it in by hand locally) | 18:33 |
| @HeikoS | gotta run now | 18:33 |
| @HeikoS | will be back on it tomorrow or so | 18:33 |
| @HeikoS | see you all | 18:34 |
| -!- HeikoS [~heiko@host86-129-231-92.range86-129.btcentralplus.com] has quit [Ping timeout: 264 seconds] | 18:38 | |
| -!- king [5862c914@gateway/web/freenode/ip.88.98.201.20] has joined #shogun | 18:45 | |
| king | hey anyone there? | 18:45 |
| @wiking | yes | 19:04 |
| @wiking | king, what'sup? | 19:04 |
| king | dont worry resolved it now, thanks | 19:05 |
| @wiking | :) k | 19:06 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4163 synchronized by syashakash | 19:08 |
| king | i do have a question actually | 19:16 |
| king | so I would like to use AccuracyMeasure() to evaluate KNN results, but it wont accept multiclass labels or binary? | 19:16 |
| king | i can get confusion matrix with MulticlassAccuracy and it works fine, but I would like to use AccuracyMeasure, or is this not possible | 19:18 |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has joined #shogun | 19:25 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339555625 | 19:25 |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has left #shogun [] | 19:25 | |
| king | it just gives me error saying SystemError: [ERROR] In file C:\bld\shogun-cpp_1512689666832\work\shogun-shogun_6.1.3\src\shogun\machine\Machine.cpp line 217: This machine does not support apply_binary() | 19:41 |
| king | sorry, this error SystemError: [ERROR] In file C:\bld\shogun-cpp_1512689666832\work\shogun-shogun_6.1.3\src\shogun\evaluation\MulticlassAccuracy.cpp line 23: assertion ground_truth->get_label_type() == LT_MULTICLASS failed in shogun::CMulticlassAccuracy::evaluate file C:\bld\shogun-cpp_1512689666832\work\shogun-shogun_6.1.3\src\shogun\evaluation\MulticlassAccuracy.cpp line 23 | 19:43 |
| @sukey | [https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/4166 opened by durovo | 19:48 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has joined #shogun | 19:52 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339555625 | 19:52 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has left #shogun [] | 19:52 | |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has joined #shogun | 19:57 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339566031 | 19:57 |
| -!- travis-ci [~travis-ci@ec2-54-166-128-254.compute-1.amazonaws.com] has left #shogun [] | 19:57 | |
| -!- durovo1 [~durovo@57.65.2ea9.ip4.static.sl-reverse.com] has quit [Remote host closed the connection] | 19:58 | |
| -!- durovo [~durovo@7d.78.2ea9.ip4.static.sl-reverse.com] has joined #shogun | 19:59 | |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has joined #shogun | 20:26 | |
| travis-ci | it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/339566031 | 20:26 |
| -!- travis-ci [~travis-ci@ec2-54-196-244-45.compute-1.amazonaws.com] has left #shogun [] | 20:26 | |
| @wiking | lisitsyn, https://www.youtube.com/watch?time_continue=56&v=RQZr2NgKPiU :) | 23:36 |
| --- Log closed Sat Feb 10 00:00:56 2018 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!