IRC logs of #shogun for Friday, 2017-03-10

« Thursday, 2017-03-09 Index Saturday, 2017-03-11 »
--- Log opened Fri Mar 10 00:00:58 2017
-!- wudayoda_ [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Read error: Connection reset by peer]00:45
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun00:46
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Remote host closed the connection]01:41
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun02:23
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Remote host closed the connection]03:06
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun03:47
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds]03:52
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun04:40
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds]04:49
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun06:46
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 240 seconds]06:50
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun06:59
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has quit [Ping timeout: 256 seconds]07:04
-!- rads23 [8984030c@gateway/web/freenode/ip.137.132.3.12] has joined #shogun07:41
-!- rads23 [8984030c@gateway/web/freenode/ip.137.132.3.12] has quit [Ping timeout: 260 seconds]07:50
-!- suhas2go [uid201652@gateway/web/irccloud.com/x-rezivwilyytbgaks] has joined #shogun07:51
-!- wudayoda [~goksinen@rrcs-50-75-193-138.nyc.biz.rr.com] has joined #shogun09:00
-!- ashu [616375c8@gateway/web/freenode/ip.97.99.117.200] has joined #shogun09:56
-!- ashu [616375c8@gateway/web/freenode/ip.97.99.117.200] has quit [Client Quit]09:57
-!- rads23 [8984030a@gateway/web/freenode/ip.137.132.3.10] has joined #shogun10:14
-!- suhas2go [uid201652@gateway/web/irccloud.com/x-rezivwilyytbgaks] has quit [Quit: Connection closed for inactivity]10:16
-!- rads23 [8984030a@gateway/web/freenode/ip.137.132.3.10] has quit [Quit: Page closed]10:39
CaBawiking: how was the presentation?11:06
@wikinggood11:06
@wikingi should share the presentation somewhere11:10
@wiking;)11:10
-!- geektoni [~Mutter@93-34-234-212.ip52.fastwebnet.it] has joined #shogun11:32
-!- geektoni [~Mutter@93-34-234-212.ip52.fastwebnet.it] has quit [Client Quit]11:35
-!- ShellcatZero [~ShellcatZ@cpe-76-93-136-67.san.res.rr.com] has quit [Read error: Connection reset by peer]12:00
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun12:02
-!- mode/#shogun [+o HeikoS] by ChanServ12:02
-!- ShellcatZero [~ShellcatZ@cpe-76-93-136-67.san.res.rr.com] has joined #shogun12:03
@wikingHeikoS, y012:03
@HeikoSjojo12:03
@HeikoSwiking:  how are things?12:03
@wikingtired12:03
@wikingbut good12:04
@HeikoSgood :D12:04
@wikinglemme know what is the expected number of people12:04
@HeikoS3012:04
@wikinglook into that stuff12:04
@wikingohohoooo12:04
@wikingwhen?12:04
@HeikoStuesday12:04
@wikingkk12:04
@wikingno problemo12:04
@wikingbtw12:04
@HeikoScan we test this somehow?12:04
@wikingwhat?12:04
@HeikoSwould be a bit nervous not doing that12:04
@wikingi mean there's that12:04
@HeikoS30 ppl logging on12:04
@HeikoSand running stuff12:05
@wikingah well u have 30 github accounts? :D12:05
@HeikoSno12:05
@wikingk12:05
@wikingthen no12:05
@wikingbut i mean12:05
@wikingit is simple12:05
@wikingjust there are some stuff12:05
@wikingu should clarify12:05
@wikinga) is there any special stuff u need on shogun feature wise?12:05
@HeikoSI will finish the notebook on the weekend12:05
@wikingb) other then some notebooks do you wanna change the docker container's content12:05
@HeikoSwhat happens if I do need to push something to develop and that needs to be in there?12:06
@wikingc) how much resources one would need for running that stuff?12:06
@HeikoSressources are minimal, like a laptop powered thing per person12:06
@HeikoS(same computing power)12:06
@wikingHeikoS, nono12:06
@wikingdefine12:06
@wiking:D12:06
@wikingnot like some vague shit12:06
@wikingdefine12:06
@wikingvpcu12:06
@wiking*vcpu12:06
@wikingand ram12:06
@HeikoShow can I?12:06
@wikinghttps://github.com/shogun-toolbox/docs/pull/512:06
@wikingthis is what happens when u push shit into develop12:07
@wikingwe have nightly debian packages12:07
@HeikoSah cool will review12:07
@HeikoSok then12:07
@wikingof course i can update them faster12:07
@wikingthan nightly12:07
@wikingi mean anytime12:07
@HeikoSif I push on Monday, is that in there Tuesday?12:07
@wikingon demand :)12:07
@HeikoScool12:07
@wikingman12:07
@wikingshogun-buildbot dance12:07
@wikingshogun-buildbot: dance12:07
shogun-buildbot<(^.^<)12:07
shogun-buildbot<(^.^)>12:07
shogun-buildbot(>^.^)>12:07
shogun-buildbot(7^.^)712:07
@HeikoSgood12:07
shogun-buildbot(>^.^<)12:07
@wikingso yeah he will dance when you want12:07
@HeikoSok so ressources12:07
@wikingand generate package when u instruct it12:08
@wikingbut by default it's nightly12:08
@HeikoSI think 4GB ram and 2cpu per person for the afternoon12:08
@HeikoSthats way more than is needed but a good guideline12:08
@HeikoSis that good?12:08
-!- mikeling [uid89706@gateway/web/irccloud.com/x-clbrxoaqgcvnoeqc] has joined #shogun12:09
@wikingHeikoS, you know12:10
@wikingi dont  :)12:10
@HeikoSwiking: I mean as in is that specified enough?12:11
@HeikoSthe ressources will be enough12:12
@wikingk12:12
@HeikoSwiking: and re the saving stuff12:12
@wiking?12:14
@HeikoSso you said that will all go then, i.e. people's notebook will be saved under their "account"12:15
@HeikoS?12:15
@HeikoSwiking: another question: what happens if there is a problem (i.e. the service doenst work or so)12:15
@wikingwhat do yo umean?12:18
@wikingi mean what happens usually when a service is down? :D12:18
@HeikoSwiking: I meant: can I call you? :D12:18
@wikingHeikoS, well yeah the saving thing i have a solution... just currenlty it's too slow12:18
@wikingthat's why it's now as it is12:18
@HeikoSok12:18
@wikingbut i'll try to fix this over the weekend12:19
@HeikoScool12:19
@wikingif the service goes down12:19
@wikingu can call me12:19
@HeikoSwiking: whats you opinion on dataset, we usually have them on some server and people download them by hand12:19
@wiking?12:19
@wikingit's there12:19
@wikingnow12:19
@HeikoShttp://www.gatsby.ucl.ac.uk/~gretton/adaptModel/ShortCourseLab.pdf12:19
@wikingwe can do it the same way12:20
@wikingjust mount it in12:20
@wikingread only12:20
@wikingbtw12:20
@HeikoSokok12:20
@wikingwe need a different domain12:20
@wikingbtw12:20
@wikingjust to be clear12:20
@wikingthat shogun12:20
@wikinghas no svmlight12:20
@HeikoSyes12:21
@HeikoSno worries12:21
@wikingyou are aware of it right?12:21
@HeikoSjust need any kernel svm12:21
@wikingk12:21
@HeikoSand naive bayes should work, but I will check that12:21
@wikingkk12:21
@wikingi mean go through the thing12:21
@wikingif u need 2vcpu + 4 gis of ram12:22
@wikingthat is12:22
@wiking60 cores and 120 gig ram12:22
@wikingright12:22
@wiking?12:22
@HeikoSbut that is auto allocated right?12:22
@HeikoSyes12:22
@HeikoSI dont know *exact* number of people, but this will be more that sufficient12:22
@wikingwhat is auto located/12:22
@wiking?12:22
@HeikoSthe number of instances12:22
@HeikoSrunning12:22
@wikingin what sense?12:22
@wikingyeah they should scale12:22
@HeikoSlike if only 10 people show up12:22
@wikingbut it takes some time to scale up an instance12:23
@HeikoSwe dont run 60 cores the while time?12:23
@wikinglike 2 minutes or so12:23
@HeikoSokok12:23
@wikingso it's better have some stuff already running there12:23
@HeikoSI understand12:23
@HeikoSwill it turn itself off after a while if not used?12:23
@wikingyes it should downscale12:23
@HeikoSok cool12:26
@HeikoSthis will be pretty awesome12:26
@HeikoSsee all the mex shit in the pdf?12:26
@wikingbtw12:26
@wikingyes12:26
@wikingthat is ugly12:26
@wikingso12:26
@wikingdo u think12:26
@wikingwe should do a benchmark on this: https://aws.amazon.com/ec2/instance-types/x1/12:26
@wiking:)12:26
@wikingx1.32xlarge = 1,952 Gb of ram12:27
@HeikoSoh wow12:27
@wikingit'd be cool to do a test with http://labs.criteo.com/2013/12/download-terabyte-click-logs/12:27
@wikingit's 13 usd/h12:27
@HeikoSwe have lots of funds still right? :)12:27
@wikingyes12:28
@wikingwe cannot index that amount of data though :)12:28
@wikingi have BigInput branch12:28
@wikingit's a lot of fucking replace for int32_t -> index_t12:28
@wikingbut would be cool to have a benchmark12:28
@HeikoSagreed12:28
@wikingto see not only accuracy12:29
@wikingbut time12:29
@wiking:)12:29
@wikingcompared to other horizontally scalable stuff12:29
@HeikoSwiking: btw12:30
@HeikoSwhat do we need to release ? :D12:30
@wikingmmm12:30
@wikingfix for cloud12:30
@wikingand there are all the read shit12:31
@wiking*red shit12:31
@wikingin the buildbot12:31
@wikingnightlies are failing12:31
@wikingPRs are failing12:31
@wikingdocumentations are failing12:31
@HeikoSokok12:31
@HeikoSwill fix some of that12:31
@HeikoSwhat do you think about opencv notebook?12:31
@HeikoSneeds some installation update12:31
@HeikoSof opencv as they split some things out of the lib12:31
@HeikoSand cmake newer version for PRE12:31
@HeikoSPR12:31
@wikingah12:32
@wikingand still have no idea12:32
@wikingwhat to do with octave12:32
@wikingand we still hve some stuff going on12:32
@wikingwith CV12:32
@HeikoSyeah12:32
@HeikoStests pass12:32
@wikinghttp://buildbot.shogun-toolbox.org/builders/nightly_default/builds/60/steps/test/logs/stdio12:32
@wikinghere they dont12:33
@HeikoSoh wow didnt realise12:33
@wikingand there's omething12:33
@wikingwith nihgtly_none12:33
@HeikoSwell ok that needs fi12:33
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has left #shogun []12:33
@wikingthat rahul was working on12:33
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun12:33
-!- mode/#shogun [+o HeikoS] by ChanServ12:33
@HeikoSwiking: tweet about the cloud when done! :)12:37
@wikingyeye12:38
@wikingi deliberately didn't12:38
@wikingtill it's all nice and good12:38
@wikingbtw12:38
@wikingwhat is the plan12:38
@wikingshouldn't we have like12:38
@wikingR, java, scala and maybe c++ kernels12:38
@wikingfor jupyter?12:38
@wikingall in one? :)12:38
@HeikoSyeah that would be cool12:38
@HeikoSso cool in fact12:38
@HeikoSthough we dont have notebook demos yet12:39
@HeikoSbut well12:39
@HeikoSstill cool to run shogun12:39
@wikingyes12:39
@wikingsomebody should test12:39
@wikingwhether it's possible to have12:39
@wikingmultiple kernels12:39
@wikingi know you can have12:39
@wikingpython2 and 312:39
@HeikoSyes that definitely12:39
@HeikoSI dont know much aobut jupyter and the other kernels12:40
@HeikoSmeta notebooks would be so cool12:40
@wikinghttps://github.com/michhar/rpy2_sample_notebooks/blob/master/TestingRpy2.ipynb12:41
@wikingbut this is not really12:41
@wikingi mean i dont want to mix12:41
@wikinglanguage in one notebook12:42
@wikingjust to have the option12:42
@wikingof different language12:42
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has left #shogun []12:42
@wikingin different notebook12:42
@wikings12:42
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun12:42
-!- mode/#shogun [+o HeikoS] by ChanServ12:42
@sukeyWiki page: Google-Summer-of-Code-2017-Projects edited on shogun-toolbox/shogun by karlnapf12:47
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 240 seconds]12:55
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun12:58
-!- mode/#shogun [+o HeikoS] by ChanServ12:58
@wikingHeikoS, y013:59
@HeikoSjo13:59
@wikingwhat do u think we should be more mem heavy13:59
@wikingor cpu heavy?13:59
@wikingwith the instances?13:59
@HeikoSmem wont be too large13:59
@wikingin general13:59
@wikingnot about13:59
@HeikoSin general13:59
@HeikoSphew13:59
@HeikoSnot sure13:59
@wikingthe course13:59
@wikingcurrenlty14:00
@wikingu get14:00
@wiking1 vpcu14:00
@HeikoSit doesnt need to be super fast14:00
@wikingand 3 gigs of ram14:00
@HeikoSyeah thats fine I think14:00
@HeikoSwe can adjust later14:00
@HeikoSif people complain?14:00
@wikingmmm of course14:00
@wikingthe problem is this14:00
@wikinghttps://aws.amazon.com/ec2/instance-types/14:00
@wikingcurrenlty i use m4.xlarge14:01
@wikingbut maybe we should be using c4.xlarge14:01
@wikingsee what i mean?14:02
@wikingbecause m4.xlarge = 4 people roughly....14:02
@HeikoSyeah I see14:02
@HeikoSI think we cannot really decide this now14:02
@wikingnono14:02
@wikingwe can14:02
@wikinglater14:02
@wikinghow this is devided up14:02
@wikingdivided14:02
@wikingthat's another story14:03
@wikingbut i have a fealing14:03
@wiking*feeling14:03
@wikingthat we are too heavy per instance14:03
@wikingon memory14:03
@wikingit's not being used14:03
@wikingwhile cpu is .......well not enough14:03
@HeikoSyeah14:04
@HeikoStrue14:04
@HeikoSmodel selection and stuff14:04
@HeikoSpeople wont do huge datasets in there14:04
@HeikoSso agreed14:04
@wikingi have an arm64 at hom atm14:12
@wikingshould try to get shogun running on it :)14:12
@wikingi wonder if it works14:12
@HeikoSphew :)14:21
@HeikoSmaybe a good idea14:21
@HeikoSthough I wonder about priorities :D14:21
@HeikoSprobably wont work14:21
@HeikoShaha14:21
@HeikoSbut I love more architechtures and OS14:21
@wikingHeikoS, why not?14:27
@wikingi mean14:27
@wikingon fedora14:28
@wikingi think we have all that compiled14:28
@wikinghttps://koji.fedoraproject.org/koji/buildinfo?buildID=85977914:29
@wikingthere14:29
@wikingshogun-5.0.0-0.5.fc26.aarch64.rpm14:30
@wiking:)14:30
@HeikoSwiking: jo14:57
@wikingpo14:58
@HeikoSwiking:  just talked to arthur a bit14:58
@HeikoSabout getting some $$$ to shogun for setting up the cloud for the course14:58
@wikinggood :)14:58
@wikingi should write the blog post soon14:58
@HeikoSif you can estimate number of hours14:58
@wikingabout the whole story14:58
@wikingmmm14:58
@HeikoSyou did14:58
@wikingwe can do the accounting14:58
@HeikoSand me too14:58
@wikingaaah14:58
@HeikoSthere is two ways14:58
@wikingnoo it's ok14:58
@wikingi can give you the aws accoutning14:58
@HeikoSone is: we write an invoice over numfocus (in the future)14:58
@HeikoSother option is: we are hired by UCL CS department14:59
@wikingmmmm look14:59
@HeikoS(thats how I am paid for the course stuff I do)14:59
@wikingfor me it's only thing is14:59
@HeikoSand then can donate to shogun account14:59
@HeikoSpoint is to get something for it14:59
@wikingthat i would like to have this for future shit for shogun14:59
@wikingliek14:59
@wikingsonney2k, have u reimbursed14:59
@HeikoSyeah I see, just sayintg it is a nice side effect14:59
@wikingmy fossasia stuff? :D14:59
@wikingi guess i should ping him via email/faceboko15:00
@HeikoSso we can definitely do that via ucl human ressources and then each of us donates15:00
@wikingmmm15:00
@wikingif  they can hold up with shit15:00
@HeikoSand even cooler would be if ucl donates to shogun for that15:00
@wikingthen lets wait till april15:00
@HeikoSyeah agreed15:00
@wikingand by then15:00
@HeikoSjust wanted to share15:00
@wikingwe know something about numfocus15:00
@HeikoSsince I just discussed15:00
@HeikoSsure sure15:00
@wikingkk15:00
-!- lambday [6a3311c5@gateway/web/freenode/ip.106.51.17.197] has joined #shogun15:13
-!- mode/#shogun [+o lambday] by ChanServ15:13
@lambdayHeikoS: yo15:13
@lambdayHeikoS: agreed on reallocation.. removing that...15:14
@lambdayshould fail if not enough memory is there15:14
@HeikoSlambday: jojo15:15
@HeikoSlambday: just check num rows columns15:15
@HeikoScaller responsibility15:15
@lambday yeah15:15
@lambdaythis one is way too flexible.. inviting callers to screw things up :D15:16
@HeikoSREQUIRE can help15:16
@lambdayBTW did you notice that this patch reduced the number of memallocs from N to 1?15:16
@HeikoSI mean if somebody gives a matrix with less memory allocated than in num_cols they deserve a segfault15:16
@HeikoSnope, can you poiunt me to it15:17
@lambdaythe way it worked before... you allocate the total memory once.. then copy the current.... then copy the return values of the get_feature_matrix() of other features.. which themselves did one memalloc extra...15:18
@lambdaynow, since you can pass pre-allocated things, the other features don't alloc those matrices, rather uses the pre-allocated one, with column offset :)15:19
@lambdayHeikoS: https://github.com/lambday/shogun/blob/221cdcac33b7b08899322f3060e692b248f768c0/src/shogun/features/DenseFeatures.cpp#L106115:19
@HeikoSlambday: jojo15:25
@HeikoSlambday: hey I have a question15:25
@HeikoSor more like wiking has15:26
@lambdayHeikoS: yeah15:26
@HeikoShe wants to showcase some of the fast testing speed stuff at a talk next week15:26
@HeikoSso we was asking for code to reproduce the plots in the paper15:26
@HeikoSso the code for that is kinda messed up / not in a state to share(/reproduce15:26
@lambdayHeikoS: it's on gatsby's machine15:26
@HeikoSbut maybe you can point viktor to the script that does the comparisons?15:26
@lambdayyeah15:26
@HeikoSlambday: like especially comparing old shogun vs new shogun15:27
@lambdayHeikoS: alright.. maybe we can have a separate script for that one.. which takes two libshogun.so as params...15:27
@lambdayso, the speed-up with number of threads won't help.. he needs something that he can compare on single core..15:28
@lambdayotherwise it's not just to the old code15:28
@HeikoSlambday: so optimal world: you clean up the scripts so that there is a single clean one that reproduces the plots15:30
@wikingmmm15:30
@HeikoSlambday: and then wiking can take this script and modify it to generate some cool vis for the presentation15:30
@wikingif possible15:30
@wikingjust a string of data15:31
@wikingis much better15:31
@wikingno need for the graph15:31
@wiking:D15:31
@wikingjust tell me what's axis x and y15:31
@wikingand the data points15:31
@wiking:D15:31
@wikingafter that i can generate the graphs15:31
@lambday haha lovely :D15:31
@wikingof course15:31
@wikingi'm happy to modify the sciprt as well15:31
@HeikoSlambday: we need these scripts anyways15:31
@wikingwhichever is the easiest atm15:31
@lambdayHeikoS: I agree.. but remember we wanted to do it using pandas15:31
@HeikoSlambday: in dougals git15:31
@wikingpandas15:32
@wikingfuck python15:32
@HeikoSlambday: nah, text file stuff is ok15:32
@wiking:D15:32
@HeikoSlambday: just no piping :)15:32
@HeikoSjust a script that generates the data for the plot15:32
@HeikoScan take a while is OK15:32
@HeikoSbut just reproduce15:32
@HeikoSshould be minimal effort no?15:32
@lambdaygotta see.. do I still have the login? :P15:32
@HeikoSdougal also wanted to re-create the plot for camera ready version15:32
@HeikoSlambday: you do15:32
@lambdayHeikoS: cool then.. will clean things up15:33
@HeikoSif you could put the stuff on your github somewhere? Then we could copy it15:33
@HeikoScool man thx15:33
@lambdayHeikoS: yeah sure15:33
@lambdayHeikoS: u435a was it?15:33
@HeikoSlambday: doesnt matter filesystem is the same15:33
@HeikoSi am on sperry15:33
@lambdayno but if I wanted to run..15:34
@HeikoSu435a is the machine you used15:34
@HeikoSsperry then15:34
@lambdayHeikoS: cool let me try loggin in15:34
-!- suhas2go [uid201652@gateway/web/irccloud.com/x-tptznqwnhoqaakye] has joined #shogun15:34
@HeikoSlambday: let me know if you need help or something15:35
@lambdayHeikoS: working15:35
@lambdayHeikoS: sure15:35
@HeikoScool15:35
@lambdayHeikoS: shall I merge the patch first, once I rename the method and remove the reallocation feature?15:36
@lambdayHeikoS: also, should we really make this method public?15:36
@lambdayI have kept it protected15:36
@HeikoSlambday: why?15:36
@HeikoSah15:36
@HeikoSyeah why not15:36
@HeikoSmight be useful to copy out the subsetted matrix15:36
@HeikoSbut15:37
@HeikoSmaybe only the one that doesnt accept pre-aloocated15:37
@HeikoSotherwise messy15:37
@HeikoSor hide from SWIG15:37
@HeikoSor just leave protected15:37
@lambdayHeikoS: wonder how the typemap works for SGMatrix.. does it use the same memory that, say, numpy passes or reallocates and copies in SGMatrix15:37
@HeikoSnono15:37
@HeikoScopy15:37
@HeikoSso definitely no SWIG for this one15:37
@lambdayHeikoS: yeah so it will be messy15:37
@HeikoSonly for a wrapper that allocates new mem15:37
@lambdayyeah15:38
@HeikoSleave protected15:38
@lambdaycool15:38
@lambdayHeikoS: BTW the point I was talking about reducing the number of memalloc : https://github.com/lambday/shogun/blob/221cdcac33b7b08899322f3060e692b248f768c0/src/shogun/features/DenseFeatures.cpp#L106115:38
@HeikoSI see15:39
@HeikoSI mean you can do that15:39
@HeikoSbut pls make it a sep method15:39
@HeikoSsep. of concerns15:39
@HeikoSand then call it explicitly in the loop15:39
@lambday HeikoS nono this one does not require realloc.. that's the point15:39
@HeikoSI see15:39
@lambdayeven if I remove realloc, this method works15:39
@HeikoSok then15:39
@lambdaycause it is called on the "other" features15:40
@lambdaywhich re-uses the pre-allocated matrix15:40
@lambdayearlier, each of those "other" features was allocating one matrix for itself...15:40
@lambdaysee what I mean/15:40
@HeikoSI get it15:42
@HeikoSnice one15:43
@wikingok15:44
@wikingpeople15:44
@lambdaywiking: we, the people?15:44
@lambday;)15:44
@wikingyes15:45
@wikingso i'm tired15:45
@wikingand crashing15:45
@wikingbut could you just drop me15:45
@wikingan email15:45
@wikingabout the stuff15:46
@wiking?15:46
@wikingwould appreciate15:46
@lambdaywiking: yeah.. sure15:46
@lambdaybrb.. my laptop is fucked up..15:47
@HeikoSgood night15:47
-!- lambday [6a3311c5@gateway/web/freenode/ip.106.51.17.197] has quit []15:47
-!- lambday [6a3311c5@gateway/web/freenode/ip.106.51.17.197] has joined #shogun15:53
-!- mode/#shogun [+o lambday] by ChanServ15:53
@HeikoSlambday: you followed the team list?15:54
@lambdayHeikoS: yo.. there is already a method named copy_feature_matrix.. which copies the matrix arg to "this" one15:58
@lambdayHeikoS: yeah15:58
@lambdayHeikoS: umm what do you mean?15:58
@lambdayHeikoS: how about renaming the existing one to "use_feature_matrix" .... it is asking the class to "use" the matrix that it provides..15:59
@lambdayHeikoS: then "copy_feature_matrix" could mean it copies the existing one16:00
@lambdayhttps://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/features/DenseFeatures.h#L27216:02
@lambdaywhy is it virtual16:03
@lambday:/16:03
@lambdaynot being overridden anywhere16:03
@HeikoSlambday: well ....16:05
@HeikoSbad SE16:05
@HeikoSwe should fix things16:05
@HeikoSbut maybe in sep patches16:05
@HeikoSso issue these things16:05
@lambdayHeikoS: tell me a good name for this method16:05
@lambdaycan't use copy_feature_matrix anymore16:05
@HeikoSI see16:05
@lambday:(16:05
@HeikoSuse_feature_matrix for the other one seems better16:06
@HeikoSor even: set_feature_matrix16:06
@lambdayyeah16:06
@lambdayyeah set16:06
@HeikoSbut that needs a separate PR16:06
@lambdayso natural16:06
@HeikoSand then a rebase16:06
@HeikoSin your one16:06
@lambdayerr16:06
@HeikoSI mean such API changes should be separated from your changes16:06
@lambdayHeikoS: well, actually it doesn't really "set"... it clones it first16:06
@HeikoSsee what I mean?16:06
@HeikoSah I  see16:07
@lambdayHeikoS: so, set would convey something different16:07
@HeikoSah such a method doesnt even make sense16:07
@HeikoSright?16:07
@HeikoSwhy would one want such function16:07
@lambdayHeikoS: let me check whether it is being used anywhere16:07
@HeikoScaller can copy itself and then set16:07
@HeikoSsounds like I added this shit :D16:07
@HeikoSbut yeah good to check16:07
@lambdayKmeans16:07
@HeikoSand overloading doesnt do it?16:07
@lambdaythat's the only one place16:07
@HeikoSlambday: ok then what about a PR that fixes that16:08
@lambdayHeikoS: well, with overloading, you lose the default argument value for column_offset = 016:08
@HeikoSand then you name it copy_*16:08
@HeikoSokok16:08
@HeikoSthen I think it is OK to change16:08
@lambdayHeikoS: change it to "use_feature_matrix" ?16:08
@HeikoSno16:08
@HeikoScaller clones matrix and then calls set16:09
@lambdaymakes sense16:09
@HeikoSor whatever16:09
@HeikoSjust a setter16:09
@lambdayyeah16:09
@lambdaywhy clone I don't really understand :D16:09
@lambdayanyway16:09
@lambdaymaybe it was required16:09
@lambdayah multithreaded16:10
@HeikoSlambday: I think it is good to check that16:10
@HeikoSand then fix :D16:10
@lambdayyeah... so caller calls clone..16:10
@lambdaymakes zero sense to clone it within densefeature16:10
@lambdayyeah just checked16:11
@lambdayit's used within a openmp for16:11
@lambdaywe shouldn't accept such patches in the first place16:11
@HeikoSlambday: there you go :D16:11
@HeikoSI agree16:11
@HeikoSbut things slip through16:11
@HeikoSso just fixing16:11
@HeikoSjust read only saves the day there16:12
@HeikoSbut the const stuff can be fixed later16:12
@lambdayHeikoS: read only for which case?16:12
@HeikoSto avoid cloning data16:12
@HeikoSopenmp16:12
@HeikoSfeatures rarely are modified16:12
@HeikoSso read only views are needed16:13
@lambdayHeikoS: but the algorithm is updating the matrix over and over.... I wonder why16:13
@HeikoSah it is16:13
@HeikoSok then16:13
@HeikoSwell leave it16:13
@HeikoSas it is16:13
@HeikoSjust refactor16:13
@lambdayI mean, do we really need it to be a dense feature16:13
@lambdaylet me check16:13
@HeikoSprobably not16:14
@lambdayah man this code16:14
@HeikoShahaha16:16
@lambdayonly set would suffice.. no clone required16:16
@HeikoSyes16:16
@lambdayas far as I can see16:16
@HeikoSno threading issues?16:16
@lambdaylet me change that, this method was added recently.. should be unit-tested properly16:17
@HeikoScool,16:17
@lambdayHeikoS: nah.. within the openmp multithreaded zone, the features object is not even used.. only the matrix16:17
@HeikoSnew pr16:17
@HeikoSkk16:17
@lambdayHeikoS: yeah new pr..16:18
@lambday:(16:18
@HeikoScool16:19
@HeikoSI like PRs where code gets better16:19
@HeikoSrather than more bloated16:19
@lambdayyeah.. when I was looking at our dense feature code.. I was feeling like changing most of it.. to make it more readable16:25
@lambdaythere was this book which says somewhere "always leave the code in a better state than it was whenever you touch it".. guess it applies nicely if we want things to be manageable ..16:26
@lambdayhave to be strict with our merging policies.. all of the current changes in my pr is happening because you didn't let me merge in the first place :D16:27
-!- mikeling [uid89706@gateway/web/irccloud.com/x-clbrxoaqgcvnoeqc] has quit [Quit: Connection closed for inactivity]16:30
@HeikoSwiking: I invited the lab17:19
@HeikoSbut cannot see anything where ppl log on17:19
@HeikoShow many etc17:19
@sukeyIssue #3683 "Memcheck KMeans" opened by lambday - https://github.com/shogun-toolbox/shogun/issues/368317:23
@sukeyIssue #3683 "Memcheck KMeans" lambday added label: "bugfixing" - https://github.com/shogun-toolbox/shogun/issues/368317:23
@sukeyIssue #3683 "Memcheck KMeans" lambday added label: "entrance" - https://github.com/shogun-toolbox/shogun/issues/368317:23
@sukeyIssue #3683 "Memcheck KMeans" lambday added label: "Cleanups" - https://github.com/shogun-toolbox/shogun/issues/368317:23
-!- tj_ [67e3630c@gateway/web/freenode/ip.103.227.99.12] has joined #shogun17:28
@HeikoSlambday: memcheck?17:28
@HeikoSerrors? :D17:28
@sukeyIssue #3683 "Memcheck KMeans"- https://github.com/shogun-toolbox/shogun/issues/368317:29
@lambdayHeikoS: first I thought I introduced it.. then checked it with old code.. still got the same error..17:29
@HeikoScool17:30
@HeikoSsigh17:30
-!- tj_ [67e3630c@gateway/web/freenode/ip.103.227.99.12] has quit [Ping timeout: 260 seconds]17:35
@sukeyPull Request #3684 "Removed copy_feature_matrix method from DenseFeatures"  opened by lambday - https://github.com/shogun-toolbox/shogun/pull/368417:38
@sukeyPull Request #3684 "Removed copy_feature_matrix method from DenseFeatures" - https://github.com/shogun-toolbox/shogun/pull/368417:43
-!- lambday [6a3311c5@gateway/web/freenode/ip.106.51.17.197] has quit [Ping timeout: 260 seconds]18:14
-!- tj_ [67e3630c@gateway/web/freenode/ip.103.227.99.12] has joined #shogun18:16
-!- deshalb [58e9944a@gateway/web/freenode/ip.88.233.148.74] has joined #shogun18:45
-!- deshalb [58e9944a@gateway/web/freenode/ip.88.233.148.74] has left #shogun []18:47
@sukeyPull Request #3564 "Temp"  closed by karlnapf - https://github.com/shogun-toolbox/shogun/pull/356419:01
-!- OXPHOS [92bd305b@gateway/web/freenode/ip.146.189.48.91] has joined #shogun19:55
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 240 seconds]20:26
tj_Hi, I am trying to resolve issue https://github.com/shogun-toolbox/shogun/issues/3663. As far as I have understood, InputParser uses the ParseBuffer(which is a ring buffer) to hold the data and does the parsing(with the help of Streaming*File classes) in a separate thread when started. Now the Streaming*Features fetches example objects one by one through a InputParser object.20:36
tj_ So there is only one parsing thread(producer) and only one streaming*feature thread(consumer) at all times? Not multiple producer/multiple consumer?20:37
-!- tj_ [67e3630c@gateway/web/freenode/ip.103.227.99.12] has quit [Quit: Page closed]20:54
--- Log closed Sat Mar 11 00:00:59 2017
« Thursday, 2017-03-09 Index Saturday, 2017-03-11 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!