IRC logs of #shogun for Thursday, 2015-07-16

« Wednesday, 2015-07-15 Index Friday, 2015-07-17 »
--- Log opened Thu Jul 16 00:00:55 2015
-!- PirosB3 [~pirosb3@host145-212-dynamic.26-79-r.retail.telecomitalia.it] has joined #shogun00:24
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has joined #shogun01:16
-!- PirosB3 [~pirosb3@host145-212-dynamic.26-79-r.retail.telecomitalia.it] has quit [Quit: PirosB3]01:19
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has quit [Remote host closed the connection]04:21
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has joined #shogun05:22
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has quit [Ping timeout: 240 seconds]05:26
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has joined #shogun09:18
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has quit [Ping timeout: 246 seconds]09:22
-!- HeikoS [~heiko@054421fa.skybroadband.com] has joined #shogun11:32
-!- mode/#shogun [+o HeikoS] by ChanServ11:32
-!- HeikoS [~heiko@054421fa.skybroadband.com] has quit [Quit: Leaving.]11:38
-!- HeikoS [~heiko@054421fa.skybroadband.com] has joined #shogun12:06
-!- mode/#shogun [+o HeikoS] by ChanServ12:06
lisitsynHeikoS: hey13:16
@HeikoShi lisitsyn13:16
lisitsynhowdy?13:17
lisitsynHeikoS: I've got some time probably we could discuss things if you get some time too13:17
@HeikoSlisitsyn: yeah I have13:17
@HeikoSlets say in 10 min?13:17
lisitsynyeah sure13:18
@HeikoScool13:18
-!- shaochuan [~shaochuan@2601:647:4600:fac5:e97c:656f:3d93:f749] has joined #shogun13:21
-!- shaochuan [~shaochuan@2601:647:4600:fac5:e97c:656f:3d93:f749] has quit [Ping timeout: 248 seconds]13:25
@HeikoSlisitsyn: re13:30
lisitsynHeikoS: yeap me too13:31
@HeikoSlisitsyn:  want to talk about the modules?13:32
lisitsynHeikoS: anything you're interested in13:32
@HeikoSlisitsyn: yeah thats good, and also maybe a priority list of things that are important?13:34
@HeikoSshall we start with latter?13:34
@HeikoSkind of big picture13:34
lisitsynHeikoS: hm yeah may be13:34
@HeikoSbecause we got quite a few things started and not yet completely done13:34
@HeikoSlow hanging fruits13:34
@HeikoSwhats your opinion on general progress and all that?13:34
lisitsynok so the thing is that13:35
lisitsynwe probably overengineered things a bit13:35
lisitsyni've got a feeling we got more than we need with CI and stuff13:35
lisitsynwe don't use it properly now13:35
@HeikoSCI?13:36
@HeikoSas in testing etc?13:37
@HeikoSlisitsyn: is that blocking us from anything then?13:37
lisitsynah no13:37
lisitsynjust a random thought13:37
@HeikoSi see13:37
@HeikoSyeah maybe13:37
@HeikoSI personally would live to drop all but the unit tests anyways13:37
lisitsynHeikoS: there is a thing that blocks us13:38
lisitsynit is that we don't have good concept or idea or whatever13:38
lisitsynwhy we are better than something else13:38
@HeikoSlisitsyn: not globally at least13:38
@HeikoSI have an opinion on that though13:38
lisitsynI am a bit lost about that13:39
@HeikoSits the languages13:39
lisitsynso would be happy to join :)13:39
@HeikoSand the c++ core13:39
@HeikoSi.e. modular while fast13:39
@HeikoSwe dont exploit the latter to much anymore13:39
@HeikoSand the first we dont expose it13:39
@HeikoSbut I think thats the core strength of shogun13:39
lisitsynwhat are the main languages you think?13:40
@HeikoSa third one could be the amount of algorithms we implement13:40
lisitsynit looks like python is the main thing while others are not interesting at all13:40
@HeikoSlisitsyn: python, R, java, C++, lua, octave, and MATLAB13:40
@HeikoSI dont agree13:40
@HeikoSits just we dont support this properly13:40
@HeikoSi.e. too hard to set up13:41
@HeikoSand not exposed13:41
lisitsynok another thought then13:41
@HeikoSso thats why people dont use them13:41
@HeikoSI think for that13:41
lisitsynwe are too frameworkish13:41
@HeikoSthe two major improvements would be:13:41
@HeikoS1. manual with the languages as we started13:41
@HeikoS2. MATLAB Swig support13:41
@HeikoSif we had that, we Shogun would have a massive leap over other projects13:42
@HeikoSso thats something we could be good at13:42
@HeikoSanother thing we could be good at if we focussed is13:42
@HeikoSsingle-machine large-scale13:42
lisitsynhmm yeah13:42
@HeikoSi.e. very efficient memory managment and intelligent GPU integration13:42
@HeikoSbut we dont have that in our development process13:42
@HeikoSwe focus more in reliability at the moment, which is good and comes first13:43
lisitsynI have a feeling we should not dive into gpu things13:43
@HeikoSbut I think there is room to speed things up quite a bit13:43
lisitsynor rely on some magic library13:43
@HeikoSlisitsyn: why?13:43
lisitsynit is a shitload of work13:43
@HeikoStrue13:43
@HeikoSbut I think there is lots of low hanging fruits again13:43
lisitsynI don't have gpu13:44
@HeikoSi.e. massive gains with little effort13:44
lisitsynneither personally nor at job13:44
@HeikoSlike the basic operations in linalg13:44
@HeikoSmatrix product or inversion13:44
@HeikoSno need to be super fancy13:44
@HeikoSevery compute has a GPU these days13:44
@HeikoSmaybe not a fast, but still gains13:44
lisitsynHeikoS: you mean that thing in sandy bridge?13:44
@HeikoSyeah13:44
lisitsynI wouldn't bother13:44
@HeikoSstill way faster for those basic things13:45
lisitsynnot many things can exploit it very efficiently13:45
lisitsynI meet IO bounds much more often13:45
@HeikoSlisitsyn: yeah sure, but I am not talking about those cases13:45
@HeikoSbut those where IO doesnt block and the gains are big13:45
@HeikoSI would ignore others13:46
@HeikoSbut if we have say a CG solver13:46
@HeikoSits just faster on GPU without a lot of effort13:46
lisitsynhmm13:46
@HeikoSbut you know what13:46
lisitsynaren't matrices big?13:46
@HeikoSthats details13:46
@HeikoSI think more is to be gained when coding c++ properly13:46
@HeikoSor more general, write performant code13:46
@HeikoScurrently we dont really do that13:47
lisitsynyeah that's true13:47
@HeikoSwe dont have means of ensuring that we do that13:47
@HeikoSlike the unit tests for reliability13:47
lisitsynto do that we should make a lot of tested functions13:48
lisitsynto perform basic things13:48
@HeikoSyeah13:48
@HeikoSI dont really know how to do this13:48
lisitsyngood old C++ like we had13:48
lisitsynis not really testable13:48
@HeikoSI think the other two points are more important anyways13:48
lisitsynwell e.g. I pushed python-like range13:48
lisitsynit kills a few possible problems13:48
lisitsynsuch small things reduce complexity and improve code quality13:49
@HeikoSthe main problem for users is still 1) cannot install 2) dont know what shogun's strengths are13:49
lisitsynoh13:49
lisitsynyes that's true13:49
@HeikoSinternal I think 1) swig is too big 2) efficiency of linalg and others13:49
@HeikoSso I see the aer as the most important internal change13:50
@HeikoSand the manual thing as the most important external13:50
lisitsynyeah agree13:50
@HeikoSand in general I think external is more important than internal13:50
@HeikoSthe installation thing is going on for years13:50
lisitsynyes13:50
lisitsynok actually13:50
lisitsynpretty interface is not stable usually13:50
lisitsynif we get really simple api13:51
lisitsynwe can stabilize it13:51
lisitsynso users just use whether version they want13:51
@HeikoSyeah13:52
@HeikoSAPI is actuall another problem13:52
@HeikoSpoint 3.) for external13:52
@HeikoSwe in fact need a major re-design of the public API13:52
lisitsynyes13:52
@HeikoSbut well13:52
@HeikoSthats not top priority13:52
@HeikoSbecause we at least have something13:52
@HeikoSas with the other points, we dont have it ;)13:52
lisitsynI am personally interested :)13:53
lisitsynhmm btw13:53
lisitsynhave you seen balasz kegl article13:53
lisitsynon data science?13:53
@HeikoSnope send it13:53
lisitsynjust a sec13:53
lisitsynhttps://medium.com/@balazskegl/the-data-science-ecosystem-678459ba601313:53
lisitsynHeikoS: the thing is any person has some incentive :)13:53
lisitsynmay be we should all have our own incentive13:54
@HeikoSoh btw and another thing that affect priority is projects that are started and then stalled. We do this quite often, starting something and then not finishing. If we finished all the stuff we started, that would already be a major step13:54
lisitsynI am psycho about api :)13:54
@HeikoSlisitsyn: yeah sure13:54
@HeikoSlisitsyn: something I realised about open-source,13:54
@HeikoSpeople do what they like13:54
@HeikoSnot what sneeded13:54
lisitsynso we could have an angel for each point you listed13:54
lisitsynso e.g. you push people to make it installable13:55
lisitsynand I am obsessed with api, so on13:55
@HeikoSlisitsyn: yeah true13:55
@HeikoSI mean13:55
@HeikoSits just about small steps in the end13:55
@HeikoSbut making sure the process of those is designed in such way that we gain something by those steps13:55
@HeikoSoften we invest work which actually pushed is back, because it made Shogun less maintainable or unreliable or etc13:56
@HeikoSso I like to think in those processes13:56
@HeikoS(rather than in the details)13:56
@HeikoSbut anyways13:56
@HeikoSlets talk API13:56
lisitsynjust a sec13:56
@HeikoSHow would you like to proceed?13:57
lisitsynI have no idea :)13:57
lisitsynI started to send mails may be its the way13:57
lisitsyndid you get that property idea?13:58
@HeikoSreading13:58
@HeikoSget it14:00
@HeikoSlisitsyn: sounds like a good idea14:00
@HeikoSbut I cannot judge14:00
@HeikoSMy first question would be: why not do the simplest way14:00
@HeikoSwhich is14:00
@HeikoSto only compile the parts of Shogun that one needs14:00
@HeikoSand then expose all methods?14:01
@HeikoSwe dont have many getters/setters in fact14:01
@HeikoSlisitsyn: and if you only want Shogun for say GPs, then its just like 20 classes anyways14:02
@HeikoSmaybe its a naive question, but what would speak against that?14:02
lisitsynHeikoS: hmm yeah that's simple idea14:03
lisitsynHeikoS: how do you want to enforce it is possible to compile different parts of shogun?14:04
@HeikoSlisitsyn: isnt this possible through namespaces?14:04
@HeikoSlisitsyn: or even automatically14:04
@HeikoSlisitsyn: if doxygen can do that, we cannot cmake do ?14:04
@HeikoSthoralf manages with bash14:05
lisitsyn??14:05
lisitsynoops14:05
lisitsynyeah I think it is easy14:05
@HeikoSlisitsyn: see email14:05
lisitsynwe just compile separate parts14:05
@HeikoSlisitsyn: just wondering how much effort that would be and how much we would gain14:05
@HeikoSI have the feeiling its hard to beat in that respect14:06
@HeikoSlisitsyn: another option that I thought about:14:06
lisitsynI think we should use properties and that thing14:06
@HeikoSpull out shogun-base14:06
@HeikoSi.e. a library with the core parts of shogun14:06
@HeikoSand then sub-parts cannot depend on each other, only on the base14:06
lisitsynyes14:07
@HeikoSbut maybe thats much harder14:07
lisitsynits harder but should be possible14:07
@HeikoSlisitsyn: btw thoralfs thing compiled a LibSVM shogun in a minute14:07
@HeikoSsince just 10 classes or so14:07
lisitsynok sorry have to go14:07
@HeikoSlisitsyn: another reason why shogun is so big is the ridiculous base class14:07
@HeikoSlisitsyn: ah ok14:07
@HeikoSlisitsyn: talk later then14:07
lisitsynI ll be back in 20 mins14:07
lisitsynor 2514:07
@HeikoSi might be here then still14:07
lisitsynok cool14:08
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has joined #shogun15:22
-!- shaochuan [~shaochuan@c-50-184-81-180.hsd1.ca.comcast.net] has quit [Ping timeout: 244 seconds]15:26
-!- HeikoS [~heiko@054421fa.skybroadband.com] has quit [Quit: Leaving.]16:12
-!- HeikoS [~heiko@host-92-25-65-117.as13285.net] has joined #shogun17:15
-!- mode/#shogun [+o HeikoS] by ChanServ17:15
-!- HeikoS [~heiko@host-92-25-65-117.as13285.net] has quit [Quit: Leaving.]18:07
-!- HeikoS [~heiko@193.61.17.254] has joined #shogun18:17
-!- mode/#shogun [+o HeikoS] by ChanServ18:17
-!- HeikoS1 [~heiko@dab-glb1-h-75-9.dab.02.net] has joined #shogun18:23
-!- HeikoS1 [~heiko@dab-glb1-h-75-9.dab.02.net] has quit [Client Quit]18:24
-!- HeikoS [~heiko@193.61.17.254] has quit [Ping timeout: 244 seconds]18:26
-!- HeikoS [~heiko@nat-163-62.internal.eduroam.ucl.ac.uk] has joined #shogun18:41
-!- mode/#shogun [+o HeikoS] by ChanServ18:41
-!- HeikoS [~heiko@nat-163-62.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 252 seconds]19:56
-!- HeikoS [~heiko@dab-glb1-h-75-3.dab.02.net] has joined #shogun20:03
-!- mode/#shogun [+o HeikoS] by ChanServ20:03
-!- HeikoS [~heiko@dab-glb1-h-75-3.dab.02.net] has quit [Quit: Leaving.]20:27
--- Log closed Fri Jul 17 00:00:56 2015
« Wednesday, 2015-07-15 Index Friday, 2015-07-17 »

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!