--- Log opened Fri May 13 00:00:53 2016 | ||
-!- mizari [~mizari@95-174-213-100.nts.su] has joined #shogun | 05:02 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has quit [Read error: Connection reset by peer] | 07:08 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has joined #shogun | 07:09 | |
-!- HeikoS [~heiko@176.12.107.140] has joined #shogun | 07:23 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 07:23 | |
@HeikoS | wiking: jo | 07:43 |
---|---|---|
@HeikoS | Saurabh7: jo | 07:44 |
@HeikoS | arianepaola: jo | 07:44 |
@HeikoS | lisitsyn: hi | 07:44 |
@wiking | helo | 08:04 |
@wiking | wtf r u doing up at 6:42am :> | 08:04 |
@HeikoS | hehe | 08:25 |
@HeikoS | I was up 5am | 08:25 |
@HeikoS | travelling for work | 08:26 |
@HeikoS | north england | 08:26 |
@HeikoS | meeting time 1030 gnaaa | 08:26 |
@HeikoS | wiking: whats the time for you atm? | 08:26 |
@wiking | HeikoS: i'm still in eu | 08:37 |
@wiking | for another 3 weeks | 08:37 |
@wiking | then i'm back to sg | 08:37 |
@HeikoS | I see | 08:37 |
@HeikoS | why were you up so early | 08:37 |
@wiking | i wake up with the sun | 08:38 |
@wiking | :) | 08:38 |
@HeikoS | wiking: hehe | 08:38 |
@wiking | no joek :) | 08:38 |
@wiking | it's a shit here in eu | 08:38 |
@wiking | because the sun comes up at 5am | 08:38 |
@HeikoS | I like that | 08:38 |
@wiking | sg it's a stable 7am | 08:38 |
@wiking | so that's ok | 08:38 |
@HeikoS | I need light in my face for at least an hour before my body can start ;) | 08:38 |
@wiking | hahahah | 08:38 |
@wiking | yeah you live in england | 08:38 |
@wiking | :)))))))))) | 08:38 |
@wiking | btw when you come to sg visit ? :) | 08:38 |
@HeikoS | hehe | 08:38 |
@wiking | do this | 08:39 |
@HeikoS | cool | 08:39 |
@HeikoS | would love to | 08:39 |
@HeikoS | when is good? | 08:39 |
@wiking | anytime | 08:39 |
@wiking | just not this xmas | 08:39 |
@HeikoS | hehe | 08:39 |
@wiking | beause then we are off to nz | 08:39 |
@wiking | https://fellowships.data.gov.sg/ | 08:39 |
@HeikoS | and not before 7am | 08:39 |
@HeikoS | haha | 08:39 |
@wiking | ^ HeikoS do this | 08:39 |
@HeikoS | what is that? | 08:39 |
@wiking | it's a 3+ month | 08:40 |
@wiking | fellowship in sg | 08:40 |
@HeikoS | ay cool | 08:40 |
@wiking | "Expertise in computer and social sciences, software development, data science, UI/UX design, or technology (wireless and sensor networks, ICT and cybersecurity, Internet of Things)" | 08:40 |
@wiking | they'll pay you everything | 08:40 |
@HeikoS | cool idea in fact | 08:40 |
@wiking | so come over for 3 months :P | 08:40 |
@wiking | while i'm tehre | 08:40 |
@HeikoS | how are you there fore? | 08:41 |
@HeikoS | and what is nz? | 08:41 |
@wiking | https://scontent-vie1-1.xx.fbcdn.net/t31.0-8/12314291_1027579657265195_2268379013512691573_o.jpg | 08:41 |
@wiking | if you come in time we can go with my boat a ride :P | 08:41 |
@wiking | nz = new zealand | 08:41 |
@wiking | i'm there because my partner is doing her phd ther | 08:42 |
@wiking | *there | 08:42 |
@HeikoS | ah man amazing the boat | 08:43 |
@HeikoS | ah cool, heard it is a nice place | 08:43 |
@wiking | so i'm off for xmas there | 08:44 |
@HeikoS | ok | 08:44 |
@wiking | but i'm back in jan | 08:44 |
@wiking | so you can come any other time basically | 08:44 |
@HeikoS | so autumn it has to be | 08:44 |
@wiking | we have a guest room as well | 08:44 |
@wiking | you can come next spring as well | 08:44 |
@wiking | :) | 08:44 |
@HeikoS | maybe I'll come for a visit trip fist | 08:44 |
@HeikoS | cool man, appreciated! :) | 08:44 |
@wiking | HeikoS: ideas where to place the cluster? | 09:20 |
@wiking | i was thinking to have it in the ireland centre of aws | 09:20 |
@HeikoS | mmmh | 09:21 |
@HeikoS | rationale? | 09:21 |
@wiking | well it is still eu | 09:21 |
@HeikoS | maybe the one that people like most? | 09:21 |
@wiking | but has good connectivity to the us | 09:21 |
@wiking | i dont wanna host the thing in usa :) | 09:21 |
@HeikoS | yeah I see that | 09:21 |
@HeikoS | so yes fine then | 09:21 |
@HeikoS | germany also ok | 09:21 |
@HeikoS | strong privacy there | 09:21 |
@wiking | we can move it there as well | 09:22 |
@wiking | anyhow | 09:22 |
@wiking | first i need to figure out | 09:22 |
@wiking | why we cannot have instances atm in ireland | 09:22 |
@wiking | or german | 09:22 |
@wiking | y | 09:22 |
@wiking | :) | 09:22 |
@HeikoS | hehe I see | 09:22 |
@HeikoS | hey btw, what is the state of the file io stuff | 09:23 |
@HeikoS | just asking, not pushing :) | 09:23 |
@HeikoS | btw sonney has got the ltter from foundation | 09:23 |
@HeikoS | he needs to go to notar and sign it with him | 09:23 |
@HeikoS | we need to push him | 09:23 |
@wiking | cool | 09:23 |
@wiking | yeah i have a local commit | 09:23 |
@wiking | but that needs fixing | 09:23 |
@wiking | still | 09:23 |
@wiking | i refactored the whole thing | 09:23 |
@HeikoS | wiking: serious fixing or just glitches? | 09:23 |
@wiking | nono it's a full refactor | 09:24 |
@wiking | because the thing was getting out of hand | 09:24 |
@HeikoS | ok | 09:24 |
@HeikoS | no wait, I mean what fixing does it still need | 09:24 |
@wiking | ah | 09:24 |
@HeikoS | like how much work needs to be done on nit? | 09:24 |
@wiking | in the output format | 09:24 |
@wiking | i need to fix a buffering mechanism | 09:24 |
@wiking | because currently it can only read with this | 09:24 |
@wiking | we dont operate anymore on FILE* | 09:24 |
@wiking | we have CFile | 09:25 |
@HeikoS | ok | 09:25 |
@wiking | that has write/read | 09:25 |
@wiking | and we have Codec | 09:25 |
@wiking | that is basically the parent of all output formats | 09:25 |
@HeikoS | so that sounds messy ;) | 09:25 |
@HeikoS | just wondering how to manage time wise with OXPHOS | 09:25 |
@HeikoS | because she should look at this stuff soon | 09:25 |
@HeikoS | maybe already push into feature branch so that she can read? | 09:25 |
@wiking | yep yep | 09:26 |
@HeikoS | while you are doing the aws? | 09:26 |
@wiking | that was the plan | 09:26 |
@wiking | but it's still a messy state | 09:26 |
@HeikoS | hehe | 09:26 |
@wiking | she doesn't really need to take care of any IO stuff | 09:26 |
@HeikoS | dont use swearwords in comments ;) | 09:26 |
@HeikoS | ok then | 09:26 |
@HeikoS | then its not important to do that | 09:26 |
@wiking | why? :) | 09:26 |
@HeikoS | haha ,) | 09:26 |
@HeikoS | grep "fuck off" in NEWS | 09:27 |
@wiking | iss there? :) | 09:27 |
@wiking | noup | 09:27 |
@wiking | git grep says nothing for "fuck off" | 09:27 |
@HeikoS | ah | 09:27 |
@HeikoS | not there anymore? | 09:27 |
@HeikoS | let me investigate ;) | 09:27 |
@HeikoS | not there anymore indeed | 09:28 |
@HeikoS | there used to be a "fuck off sören" in there | 09:28 |
@wiking | :DDDDDDDDDDDDDD | 09:28 |
@HeikoS | by this guy who wrote the serialization stuff | 09:28 |
@HeikoS | and then somehow turned evil | 09:28 |
@wiking | who wrote the serialization stuff? | 09:29 |
@HeikoS | and threatened shogun leagally to remove some code he wrote in his spare time | 09:29 |
@HeikoS | yeah | 09:29 |
@wiking | man we need to prepare so many images for aws | 09:31 |
@wiking | like | 09:31 |
@wiking | fedoraXX, centos6-7, ubuntu..., debian... | 09:31 |
@wiking | good that half of that can be automated | 09:31 |
lisitsyn | in the danger of sound stupid | 09:32 |
lisitsyn | what's the plan with aws | 09:32 |
@wiking | move buildbot there | 09:32 |
@wiking | + bring back shogun-cloud | 09:32 |
@wiking | and then the current machines can run benchmarks | 09:33 |
@wiking | instead of builds | 09:33 |
@HeikoS | lisitsyn: build a spambot to your email ;) | 09:33 |
@HeikoS | cloud will be cool | 09:33 |
lisitsyn | that's ok I stopped reading mail already | 09:33 |
@HeikoS | lisitsyn: haha | 09:34 |
@wiking | but cloud needs rewamp | 09:34 |
@HeikoS | lisitsyn: btw when are you coming to london? | 09:34 |
@wiking | because i would start adding support for jupyter | 09:34 |
@HeikoS | lisitsyn: remember we planned that | 09:34 |
@HeikoS | wiking: yep ! | 09:34 |
lisitsyn | I was reading a self-help book recently and it says it is okay to not answer mails | 09:34 |
@HeikoS | wiking: you can also come | 09:34 |
lisitsyn | and I was like ooh I am a cool guy | 09:34 |
@HeikoS | rahul is there | 09:34 |
@HeikoS | so we can have a hackathoin | 09:34 |
@HeikoS | he stays 2.5 more months | 09:34 |
@HeikoS | any time slots for you guys? | 09:34 |
lisitsyn | not sure now | 09:34 |
@wiking | not me | 09:35 |
@wiking | i'm going back to sg on the end of this month | 09:35 |
@wiking | asedf fucker aws | 09:35 |
@wiking | 3rd instance | 09:35 |
@wiking | that doesn't ping | 09:35 |
@wiking | \o/ | 09:35 |
lisitsyn | I could be coming to nips on my own so not sure I can handle both london and barcelona | 09:35 |
@wiking | but an instance for 0.02 usd/hour | 09:36 |
@wiking | that 16k will last long | 09:36 |
@wiking | although we can only have it for 1 year | 09:37 |
@wiking | so lets mine some bitcoin | 09:37 |
@wiking | any objections? | 09:37 |
@HeikoS | wiking: hehe :) | 09:37 |
@HeikoS | lisitsyn: nips? | 09:37 |
@HeikoS | ah yes | 09:37 |
@HeikoS | but that in dec man | 09:37 |
@HeikoS | I am talking July | 09:37 |
lisitsyn | yeah but I am in the poorest oil country :D | 09:37 |
@wiking | :D | 09:38 |
@wiking | isn't that iraq? | 09:38 |
@HeikoS | lisitsyn: you can stay at my place | 09:38 |
@HeikoS | just need flight | 09:38 |
lisitsyn | HeikoS: ok I'll think | 09:39 |
@HeikoS | come over a long weekend or so | 09:40 |
@wiking | amazing new spams | 09:42 |
@wiking | Hello, my name is Miss Jovita Gomez, I saw your profile email at github.com I am interested in having a good friendship with you. I'll tell you more about me and send my pictures to you as soon as you reply to my email.(jovitagomez120@hotmail.com) I'll be waiting for your mail soon. Honestly Jovita. | 09:42 |
@wiking | :> | 09:43 |
@HeikoS | wiking: how/where do you do the aws images? | 09:43 |
lisitsyn | HeikoS: is it like anytime now as you all are there? | 09:43 |
@HeikoS | lisitsyn: more or less | 09:43 |
@HeikoS | I am away a few days here and there | 09:43 |
@HeikoS | but anytime for the next 2.5 months in general | 09:43 |
@HeikoS | wiking: hahaha | 09:43 |
@wiking | HeikoS: packer.io | 09:43 |
@HeikoS | wiking: cool | 09:43 |
@wiking | it'll get into shogun-infra | 09:43 |
@wiking | but now we have some problemos | 09:43 |
@HeikoS | cool, looking forward to see that | 09:43 |
@wiking | with the limitations of aws | 09:44 |
@wiking | i've fired a request | 09:44 |
@wiking | lets see how long it'll take them to reply | 09:44 |
@HeikoS | you already broke it? :D | 09:44 |
@wiking | nono i mean | 09:44 |
@wiking | we had 0 slots | 09:44 |
@wiking | of running any instance | 09:44 |
@wiking | in ireland | 09:44 |
@wiking | for god knows what reason | 09:45 |
@wiking | ok i give up | 09:45 |
@wiking | i've tried to bring up a fucking m4.large | 09:45 |
@wiking | with ubuntu 16.04 | 09:45 |
@wiking | for the 4th time | 09:45 |
lisitsyn | wiking: github friendship is magic | 09:50 |
@wiking | lisitsyn: :d | 09:51 |
lisitsyn | wiking: recvd it as well | 09:51 |
@wiking | hahahaha | 09:52 |
@wiking | pworked the first instance | 09:56 |
@wiking | fuck | 09:56 |
@wiking | :) | 09:56 |
-!- ptizoom [~christian@host-92-21-235-55.as13285.net] has quit [Quit: Ex-Chat] | 09:59 | |
@HeikoS | lisitsyn: btw I have a questions regarding plugin | 10:01 |
lisitsyn | yes | 10:01 |
@HeikoS | lisitsyn: are there any plans for the "base interface" | 10:01 |
@HeikoS | that is which methods are in shogun's swig interface? | 10:01 |
lisitsyn | HeikoS: yes wiking and I asked sanuj just yesterday or so | 10:01 |
lisitsyn | to have a gist | 10:01 |
@HeikoS | lisitsyn: to draft one | 10:01 |
lisitsyn | HeikoS: https://gist.github.com/sanuj/56f03cd242473137fad851e68fa0f2c1 | 10:02 |
lisitsyn | yes | 10:02 |
lisitsyn | this is it | 10:02 |
@HeikoS | since it is a chance to re-work our supervised interfaces | 10:02 |
@HeikoS | better structure | 10:02 |
lisitsyn | yeah it would enforce better structure | 10:02 |
@HeikoS | cool | 10:02 |
@HeikoS | will comment | 10:02 |
@HeikoS | thanks | 10:02 |
-!- HeikoS [~heiko@176.12.107.140] has quit [Ping timeout: 250 seconds] | 10:28 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has quit [Ping timeout: 276 seconds] | 11:43 | |
-!- HeikoS [~heiko@82-132-214-18.dab.02.net] has joined #shogun | 13:52 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 13:52 | |
-!- HeikoS [~heiko@82-132-214-18.dab.02.net] has quit [Client Quit] | 13:55 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has joined #shogun | 13:56 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-sselotqfitbansqv] has quit [Ping timeout: 260 seconds] | 14:07 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-phwhlfgrjdxqamgy] has joined #shogun | 14:08 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has quit [Read error: Connection reset by peer] | 14:41 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has joined #shogun | 14:42 | |
arianepaola | Hi HeikoS | 15:30 |
-!- travis-ci [~travis-ci@54.166.179.21] has joined #shogun | 16:52 | |
travis-ci | it's lambday's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/130004571 | 16:52 |
-!- travis-ci [~travis-ci@54.166.179.21] has left #shogun [] | 16:52 | |
-!- HeikoS [~heiko@176.12.107.140] has joined #shogun | 17:48 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 17:48 | |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has quit [Excess Flood] | 17:58 | |
-!- arianepaola [~ariane@unaffiliated/arianepaola] has joined #shogun | 17:59 | |
@HeikoS | Saurabh7: around? | 18:16 |
Saurabh7 | HeikoS: hi ! | 18:20 |
Saurabh7 | HeikoS: i saw your comment about modelselection | 18:33 |
@HeikoS | hi | 18:33 |
@HeikoS | yeah | 18:33 |
@HeikoS | so we need a plan | 18:33 |
@HeikoS | with concrete findings | 18:33 |
@HeikoS | more than the proposal | 18:33 |
@HeikoS | more concrete | 18:34 |
Saurabh7 | I was wondering how much the new plugin thing will affect the framework | 18:34 |
@HeikoS | more technical | 18:34 |
@HeikoS | doesnt matter for now | 18:34 |
@HeikoS | more important is to make it multicore for now | 18:34 |
Saurabh7 | ok | 18:34 |
@HeikoS | this requires some thinking about thread safety | 18:34 |
@HeikoS | maybe start with a kernel svm? | 18:34 |
@HeikoS | this requires the subset on features to be thread safe | 18:34 |
@HeikoS | i.e. one needs to register a subset and then access via an identifier | 18:35 |
Saurabh7 | sure | 18:35 |
Saurabh7 | but we ned thread safe views | 18:35 |
Saurabh7 | yes | 18:35 |
Saurabh7 | last discussion i remember there was something like a shallow copy and adding subset | 18:35 |
@HeikoS | do you need help with that? | 18:35 |
Saurabh7 | and a method create_view for features | 18:35 |
@HeikoS | not yet views (also need those), but let's start with making the thing we have work | 18:36 |
@HeikoS | this will teach us a bit on how to do it | 18:36 |
@HeikoS | and then we can do more sophisticated | 18:36 |
@HeikoS | the point is: | 18:36 |
@HeikoS | data should not be doubled in memory | 18:36 |
@HeikoS | and if the kernel matrix is pre-computed, this should also not be duplicated in memory | 18:36 |
@HeikoS | furthermore | 18:36 |
@HeikoS | x-validation should be cache friendly (currently it is not) -- this is even true before we start doing things in parallel | 18:37 |
@HeikoS | maybe that is a good start? | 18:37 |
@HeikoS | to find out how bad it is? :D | 18:37 |
Saurabh7 | ok so the kernel matrix will differ by alogirthm right | 18:37 |
Saurabh7 | i mean different algos will ahve diffrent compuattions that need not be duplicated | 18:37 |
Saurabh7 | ok so you think we should start with one ? | 18:38 |
@HeikoS | yes | 18:38 |
@HeikoS | but it illustrates the issues of doing things multicore | 18:38 |
@HeikoS | so it is a good one to start with | 18:38 |
@HeikoS | see what I mean? | 18:38 |
@HeikoS | as in: some data structures will be shared along model selection | 18:39 |
@HeikoS | x-validation | 18:39 |
Saurabh7 | yes and those should be thread safe | 18:39 |
@HeikoS | step1: make x-validation on a kernel machine work (without precomputed kernel matrix) | 18:39 |
@HeikoS | this involves making the subsets thread safe | 18:39 |
@HeikoS | step2: make it work with precomputed kernel matrix | 18:39 |
@HeikoS | step3: make bagging machine work with the thread safe subset features | 18:40 |
@HeikoS | all required for gsoc | 18:40 |
@HeikoS | but for the draft: we can start with step 1 | 18:40 |
@HeikoS | and find out what we can gain there | 18:40 |
@HeikoS | step0: make x-validation cache friendly | 18:40 |
@HeikoS | that is first in fact | 18:40 |
@HeikoS | can you add those points to the doc and fill them with details? | 18:40 |
Saurabh7 | alright | 18:40 |
@HeikoS | btw is there a benchmark on x-validation in the mlpack framework | 18:41 |
Saurabh7 | yes i have added points i will ad these details there | 18:41 |
@HeikoS | rcurtin: ^ | 18:41 |
@HeikoS | Saurabh7: cool | 18:41 |
@HeikoS | and for the toher algos | 18:42 |
@HeikoS | it would be good to have some kiund of summary of which algos are slow and what causes it, i.e. profiler information | 18:42 |
@HeikoS | or benchmarks in mlpacks lib | 18:42 |
Saurabh7 | i might need help with cache i will ask as i come accross | 18:42 |
Saurabh7 | so i have to profile to understand the implications ? | 18:42 |
@HeikoS | doesnt need to fix things now, but the WHY is important | 18:42 |
@HeikoS | Saurabh7: yeah definitely | 18:42 |
@HeikoS | whats interesting in profiling: | 18:42 |
Saurabh7 | HeikoS: i dont think there xvalidation in the framework | 18:43 |
@HeikoS | a) which calls is most time spent on (we are interested in overhead here) | 18:43 |
@HeikoS | b) what is the fraction of cache misses | 18:43 |
@HeikoS | Saurabh7: yeah that would be bets | 18:43 |
@HeikoS | c) where can we parallelise | 18:43 |
@HeikoS | c) is independent of profiler, can can mostly be answered with looking at implementation | 18:43 |
@HeikoS | ? | 18:44 |
@HeikoS | ah I see | 18:44 |
Saurabh7 | yes i was using callgrind thats how i came across the linalg::dot thing :D but havent looked at the cache miss thing | 18:44 |
@HeikoS | Saurabh7: ok then, profiling is needed then | 18:44 |
Saurabh7 | yet | 18:44 |
@HeikoS | its just another statistic | 18:44 |
@HeikoS | nbut you need to isolate the particular part of the program | 18:44 |
Saurabh7 | yup wasnt aware of it much | 18:44 |
@HeikoS | I expect speedups up to one order of magnitude | 18:44 |
@HeikoS | Saurabh7: ok that also added to plan | 18:45 |
@HeikoS | Saurabh7: then whats next? | 18:45 |
@HeikoS | you were talking about the random forests a while ago | 18:45 |
@HeikoS | what is the matter there ? | 18:45 |
Saurabh7 | hm they require the same thread safe views to parallelize | 18:46 |
Saurabh7 | after that it should be straight forward | 18:46 |
@HeikoS | i see cool | 18:46 |
@HeikoS | Saurabh7: also would be good to profile them | 18:47 |
@HeikoS | decision trees can be optimized a lot even single core | 18:47 |
Saurabh7 | HeikoS: but other decision tree implementation id3, cart etc did look well in comparison | 18:47 |
Saurabh7 | so that maybe another thing to add to test | 18:48 |
-!- HeikoS [~heiko@176.12.107.140] has quit [Quit: Leaving.] | 18:48 | |
-!- HeikoS [~heiko@176.12.107.140] has joined #shogun | 18:48 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 18:48 | |
@HeikoS | Saurabh7: sorry got disconnected ,) | 18:49 |
@HeikoS | in train atm | 18:49 |
Saurabh7 | wifi in train, one day i enjoy it too :) | 18:49 |
@HeikoS | hehe | 18:49 |
Saurabh7 | HeikoS: but other decision tree implementation id3, cart etc did look well in comparison | 18:49 |
@HeikoS | first class only | 18:49 |
Saurabh7 | didnt * | 18:49 |
@HeikoS | Saurabh7: also profile it | 18:49 |
@HeikoS | the thing I recently realized is: you should always pass through the data linear in memory | 18:50 |
@HeikoS | and then check what to do with it | 18:50 |
@HeikoS | rather than letting the algorithm decide the order in which you pass | 18:50 |
@HeikoS | so that should be made sure | 18:50 |
@HeikoS | and it will reduce the cache misses | 18:50 |
@HeikoS | you should also check this for the kmeans we have | 18:50 |
@HeikoS | Saurabh7: what is great about all this is: it cannot really be done in Python | 18:51 |
Saurabh7 | yup i was thinking about it | 18:51 |
@HeikoS | so we can win big time | 18:51 |
Saurabh7 | but how to ensure it passes linearlly ? | 18:51 |
Saurabh7 | that would depend i guess | 18:52 |
@HeikoS | for (auto i=0 ..... ) | 18:52 |
@HeikoS | do something with X[i] | 18:52 |
@HeikoS | you will need to precompute something for that | 18:52 |
@HeikoS | for kmeans, e.g. that would be in which cluster X[i] currently is | 18:52 |
@HeikoS | and then do the corresponding thing | 18:52 |
@HeikoS | another idea is to always sort indices so that they ressemble the data structure in memory | 18:53 |
@HeikoS | Saurabh7: do you see what I mean? | 18:54 |
@HeikoS | say you need to touch all data (as e.g. in x-validation) | 18:54 |
@HeikoS | and you have some vector of indices | 18:54 |
@HeikoS | call it perm | 18:54 |
@HeikoS | then you can do | 18:54 |
Saurabh7 | HeikoS: yup i get the idea | 18:54 |
@HeikoS | for (auto i=0 .....) | 18:55 |
@HeikoS | do something with X[perm[i]] | 18:55 |
@HeikoS | for | 18:55 |
@HeikoS | or | 18:55 |
@HeikoS | for (auto i=0 .....) | 18:55 |
@HeikoS | do something with X[i] | 18:55 |
@HeikoS | such that the second is equal to the first | 18:55 |
@HeikoS | in x-validation, sorting the combined folds is e.g. an idea | 18:56 |
Saurabh7 | ah so whatever you have in that fold its good to have them in an order | 18:57 |
@HeikoS | yes exactly | 18:57 |
@HeikoS | Saurabh7: BUT | 18:58 |
@HeikoS | that has to be benchmarked | 18:58 |
@HeikoS | and compared | 18:58 |
@HeikoS | and that needs to be in the plan before GSoC start | 18:58 |
@HeikoS | s | 18:58 |
@HeikoS | then optimizing it is part after the 22nd | 18:58 |
@HeikoS | this is what I mean by spelling out the details | 18:58 |
Saurabh7 | HeikoS: right , i will try a minimal example with kernel svm then | 18:59 |
@HeikoS | great | 19:00 |
@HeikoS | this is much about exploring the problem and learning what we can do | 19:00 |
@HeikoS | because after GSoC we want it to be as close as possible to being optimal | 19:00 |
@HeikoS | so that we are faster than python implementations | 19:01 |
@HeikoS | and its a useful skill ;) | 19:01 |
Saurabh7 | we deifnelty want that :) | 19:01 |
@HeikoS | I mean you saw the table in the slides I sent | 19:02 |
@HeikoS | lots to gain | 19:02 |
@wiking | gimem table | 19:02 |
@wiking | HeikoS: ^ | 19:02 |
@HeikoS | wiking: qualitative, but will send it | 19:03 |
@wiking | k | 19:03 |
@HeikoS | internal talk at gatsby I did | 19:03 |
@HeikoS | on this sequential stuff | 19:03 |
@HeikoS | did some benchmark on my laptop | 19:03 |
@HeikoS | the cache actually killed one of my supervisors nips paper ;D | 19:05 |
rcurtin | HeikoS: no cross-validation in mlpack | 19:05 |
rcurtin | would be cool but nobody has done it | 19:05 |
@wiking | HeikoS: :D | 19:05 |
@HeikoS | rcurtin: ok Saurabh7 might help there if we find time | 19:05 |
@HeikoS | wiking: rahul pulled that, pretty cool | 19:05 |
@wiking | cool | 19:06 |
@HeikoS | reduced mem from D^2 to D and is 10 times faster than my old c code | 19:06 |
@HeikoS | started with this idea of only visiting data once and do the rest from there | 19:06 |
@HeikoS | only possible in that algo though | 19:06 |
@HeikoS | but good example | 19:06 |
-!- HeikoS1 [~heiko@176.12.107.140] has joined #shogun | 20:17 | |
-!- HeikoS [~heiko@176.12.107.140] has quit [Ping timeout: 260 seconds] | 20:19 | |
-!- HeikoS1 [~heiko@176.12.107.140] has quit [Quit: Leaving.] | 20:41 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-phwhlfgrjdxqamgy] has quit [Ping timeout: 260 seconds] | 21:06 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-jshlvjxezurkfmjs] has joined #shogun | 21:07 | |
-!- mizari [~mizari@95-174-213-100.nts.su] has quit [Ping timeout: 240 seconds] | 21:57 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting] | 22:04 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:04 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Client Quit] | 22:05 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:05 | |
@wiking | shogun-buildbot: force build --branch=develop 'CentOS7 - libshogun' | 22:05 |
shogun-buildbot | The build has been queued, I'll give a shout when it starts | 22:05 |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Client Quit] | 22:07 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:07 | |
@wiking | shogun-buildbot: force build --branch=develop 'CentOS7 - libshogun' | 22:07 |
shogun-buildbot | The build has been queued, I'll give a shout when it starts | 22:07 |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Client Quit] | 22:12 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:12 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting] | 22:18 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:19 | |
@wiking | \o/ | 22:26 |
@wiking | first aws spot instance based build is on its way | 22:26 |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-jshlvjxezurkfmjs] has quit [Ping timeout: 260 seconds] | 22:36 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-oquzmzvbcnckrxke] has joined #shogun | 22:36 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Quit: buildmaster reconfigured: bot disconnecting] | 22:56 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 22:57 | |
-!- Saurabh7 [Saurabh7@gateway/shell/panicbnc/x-oquzmzvbcnckrxke] has quit [Ping timeout: 260 seconds] | 22:59 | |
-!- Saurabh7 [~Saurabh7@gateway/shell/panicbnc/x-uwgvtddqiqhiabma] has joined #shogun | 23:00 | |
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun | 23:21 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 23:21 | |
--- Log closed Sat May 14 00:00:55 2016 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!