IRC logs of #shogun for Wednesday, 2012-08-15

--- Log opened Wed Aug 15 00:00:17 2012
shogun-buildbotbuild #355 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/355  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>00:12
shogun-buildbotbuild #284 of deb2 - static_interfaces is complete: Failure [failed test cmdline_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/284  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>00:14
@sonney2kblackburn, yes01:04
blackburnsonney2k: I forgot what I wanted to say, so nevermind :D01:05
@sonney2khah01:05
@sonney2kdid you get the bb happy?01:05
blackburnno, not yet01:05
blackburnthere are a DUALLIBQPBMRMSOMCSVM01:05
-!- cwidmer [45c9b18c@gateway/web/freenode/ip.69.201.177.140] has joined #shogun01:05
blackburnbug01:06
@sonney2kwow now everything crashes01:06
blackburnsonney2k: it was hard but I managed to carry out all these crashes01:06
blackburnbelieve me it is hard to break everything :D01:06
@sonney2kblackburn, from experience I think you are joking :D01:08
blackburnI will fix cmdline static01:09
blackburnpython modular won't be fixed before our SO guys negotiate anything01:09
@sonney2kgrmpf01:29
-!- cwidmer [45c9b18c@gateway/web/freenode/ip.69.201.177.140] has quit [Quit: Page closed]02:27
CIA-21shogun: Sergey Lisitsyn master * rf7116ba / src/shogun/lib/malsar/malsar_clustered.cpp : Restored termination check in clustered MTL - http://git.io/VK8TsA02:33
shogun-buildbotbuild #356 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/356  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>03:02
-!- blackburn [~blackburn@109.226.80.43] has quit [Quit: Leaving.]03:03
shogun-buildbotbuild #285 of deb2 - static_interfaces is complete: Failure [failed test cmdline_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/285  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>03:04
shogun-buildbotbuild #55 of nightly_none is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_none/builds/5503:06
shogun-buildbotbuild #64 of nightly_default is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/6403:39
shogun-buildbotbuild #51 of nightly_all is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_all/builds/5104:08
-!- gsomix_ [~gsomix@88.200.214.210] has joined #shogun08:17
-!- gsomix_ [~gsomix@88.200.214.210] has quit [Client Quit]08:17
-!- uricamic [~uricamic@2001:718:2:1634:88c4:9daf:4b80:763c] has joined #shogun08:43
-!- n4nd0 [~nando@98.Red-2-137-39.dynamicIP.rima-tde.net] has joined #shogun09:06
uricamicn4nd0: hi09:17
n4nd0hi uricamic09:17
n4nd0how are you doing?09:17
uricamicgood, how are you?09:17
n4nd0I am fine too09:18
n4nd0enjoying my last days in Spain09:18
uricamicI installed mosek, but shogun configuration script haven't found it, is there something what shall I do to help it?09:18
n4nd0I am coming back to Sweden on Friday09:18
uricamicnice :)09:18
n4nd0yes09:18
n4nd0I didn't change anything in the configure script to detect mosek (soeren told me not to worry about it)09:19
n4nd0although Sergey added something09:19
n4nd0what I do is just to run ./configure09:19
n4nd0and later to add manually a couple of changes for mosek in the file .config09:19
n4nd0I can tell you about them if you want09:19
uricamicI see, I was confused because there is info in configuration output that MOSEK was not found09:20
n4nd0mmm ok09:20
n4nd0that must be probably what Sergey added09:20
uricamicI wanted to try your python example, because I have figured out, that my example crashes on exit09:20
n4nd0but I don't know whether it works09:20
n4nd0all right09:21
n4nd0then just do these modifications09:21
uricamicand I have tried to run valgrind on it and pointed out some lines in MulticlassSOLabels so I just wanted to compare with your example if it will aslo be the case09:21
n4nd0aham ok09:21
n4nd0I am sure that the HMSVM doesn't leak memory09:22
n4nd0but it might be that the Multiclass example does because now that I think of it I didn't check this with valgrind, have to do that soon!09:22
n4nd0uricamic: did you get the mail?09:22
n4nd0I have just forwarded you the same instructions I sent to Nico some time ago to build with MOSEK09:23
uricamicyes, thx. And if you run your python multiclass example does it exit normally?09:23
uricamicin my case everything works nicely, but on exit it crashes with segmentation fault.09:24
uricamicI can send you valgrind output on that09:24
n4nd0uricamic: we are talking about so_multiclass.py right?09:25
uricamicyep09:26
uricamicbut I have run valgrind so far only on so_multiclass_BMRM.py09:26
uricamicsince I have not yet managed to get so_multiclass.py working with MOSEK yet09:26
n4nd0but you are right09:27
n4nd0something explodes in that example09:27
n4nd0I get a *** glibc detected *** python: free(): invalid pointer: 0x09e33d38 ***09:27
n4nd0once I started with hm-svm I never tried compatibility with the multiclass thing with the new changes :(09:27
n4nd0sorry about that09:28
n4nd0let me see09:28
uricamichttps://gist.github.com/c8e5335419f15ada9f8909:28
n4nd0mmm09:28
n4nd0let me try with the libshogun example09:28
n4nd0C++ traces are far more readable...09:28
uricamicit is ok, I was just trying to figure out what is wring, since I have never written anything in python :)09:29
n4nd0ok :)09:29
n4nd0the error appears in the C++ example too, which is good09:30
uricamicreally?09:31
uricamicbecause for me my C++ example worked fine09:31
n4nd0my so_multiclass.cpp had a *** glibc detected *** ./so_multiclass: free(): invalid pointer: 0x09d25fc8 ***09:31
uricamicI see, I haven't checked the C++ example with valgrind yet, but at least it haven't produced any segfault on running09:34
uricamicn4nd0: btw what do you mean by C++ traces?09:35
n4nd0like the gist you sent me09:35
n4nd0valgrind detects lot of things with python programs09:35
n4nd0just because of the python language09:35
n4nd0(one can either use a suppresions file or re-compile python with some flags)09:35
uricamicI see09:36
n4nd0so those are normally hard to read09:36
uricamicok09:36
n4nd0however, C++ ones are cleaner09:36
uricamicyep, sure09:36
uricamicAnd finally the last thing concerning the merging of risk function and structured models :)09:37
n4nd0sure, tell me09:38
uricamicI think there have just been some misunderstanding at the beginning of GSoC or we only forgot our original intentions09:38
uricamicI remember that at the beginning, we have been discussing the risk function and I have told you, that the risk function consists of argmax with loss and building psi vectors09:39
n4nd0ok09:39
uricamicbut because we had to submit some code already, this divergence happened :)09:39
n4nd0all right09:40
n4nd0so then09:40
uricamicbut then somehow, we forgot that we wanted to put it back together09:40
n4nd0is it possible to do a generic risk function in StructuredModel right?09:40
n4nd0there is no need to re-define in multiclass model, hm-svm, etc09:40
n4nd0because the parts that change are argmax, loss, and psi09:41
n4nd0or?09:41
uricamicit is, but I guess sometimes, the risk function can be written more efficiently, so there should still be possibility to redefine it09:41
n4nd0sure09:41
n4nd0virtual method :D09:41
uricamicyep :)09:41
n4nd0ok09:41
n4nd0let's do that then09:41
uricamicthat was the intention of possibility to pass some user data to the risk function09:41
n4nd0mm I don't understand that09:42
n4nd0can you explain me more what do you mean with user data to the risk function?09:42
uricamicbecause you can e.g. precompute a lot of things because it is done in each iteration of BMRM again09:42
n4nd0I see09:43
uricamicsure for example in one of my applications of SO-SVM for facial landmark detection, the computation of risk is quite complex, I mean time demanding09:43
n4nd0ok09:43
n4nd0and what do you there then?09:43
uricamicbut if u have a computer with a lot of memory, you can save a lot of computations by precomputing some matrix multiplications09:43
uricamicin this particular example I now can afford to precompute loss for all examples09:44
-!- n4nd0 [~nando@98.Red-2-137-39.dynamicIP.rima-tde.net] has quit [Quit: leaving]09:44
-!- n4nd0 [~nando@98.Red-2-137-39.dynamicIP.rima-tde.net] has joined #shogun09:45
uricamicand also precompute all psi vectors in the last term of evaluation of risk09:45
n4nd0ok09:45
n4nd0uricamic: I have to go now09:46
uricamicmoreover we have done lately some experiments with incompletely annotated examples, which also needs some other things to be passed to risk function09:46
uricamicn4nd0: ok, see ya09:46
n4nd0uricamic: I have not managed to detect what is wrong with the multiclass model, why that glibc appears09:46
n4nd0but I'll come back to this tonight09:46
uricamicok09:46
n4nd0bye!09:46
-!- n4nd0 [~nando@98.Red-2-137-39.dynamicIP.rima-tde.net] has quit [Client Quit]09:46
-!- heiko [~heiko@host86-179-59-65.range86-179.btcentralplus.com] has joined #shogun09:58
wikinguricamic: mmm waht was now the consensus for the risk function? :)11:54
wikingpure virtual or virtual? :)11:54
-!- heiko1 [~heiko@host86-185-9-87.range86-185.btcentralplus.com] has joined #shogun13:20
uricamicwiking: I guess virtual13:20
-!- heiko [~heiko@host86-179-59-65.range86-179.btcentralplus.com] has quit [Ping timeout: 260 seconds]13:21
wikingokei14:58
uricamicwiking: and have you seen yesterday the message about change in risk function?14:59
uricamicthat insted of void the retrun type should be flaot64_t ?14:59
wikinguricamic: ah yeah15:11
wikingi saw that15:11
wikingbut what should be the value?15:11
wikingi mean it represents what?15:11
uricamicinstead of passing argument R which is there just to store the result of the value of risk function it will return this value15:11
wikingah ok15:12
wikingso R gets out from the function args15:12
uricamici.e. R will pop out from the argument list and instead of it this value will be returned15:12
wikingokok15:12
uricamicyep, exactly15:12
wikingi'll do that change15:12
wikingfirst i'll do a change in latentlabels15:12
wikingand then i'll send in the new patch for the risk as well15:12
uricamicso it requires also to change 2 lines in each libbmrm.cpp, libppbm.cpp and libp3bm.cpp15:13
uricamicok, thanks15:13
wikingyeps15:13
wikingno worries15:13
uricamicthank you very much, for this inconveniences15:13
wikingno no it's cool15:14
wikingon the end it'll help me as well15:14
uricamicin the end everything will be probably nicely merged together :)15:15
wikinghehehe yeah i'm hoping that i can do something similar with latent and structured model ;)15:16
wikingwe'll see how that'll end up15:16
wikingbtw libbmrm.cpp, libppbm.cpp and libp3bm.cpp are all solving the same minimization problem right?15:16
wikingjust in a different way?15:16
uricamicyep15:17
uricamicwell, more or less15:17
wikingokok15:17
uricamicin libppbm and libp3bm is added proximal term to the function that is optimized15:18
uricamici.e. instead of just minimizing \lambda/2 || w ||^2 + R(w)15:19
uricamicwe are minimizing \lambda/2 || w ||^2  + \alpha || w - w_t ||^2 + R(w)15:20
uricamicand the difference between libppbm and libp3bm is that libp3bm uses multiple cutting plane models15:20
uricamicbut when number of cp models is set to 1, then those two are identical15:21
-!- uricamic [~uricamic@2001:718:2:1634:88c4:9daf:4b80:763c] has quit [Quit: Leaving.]17:44
-!- blackburn [~blackburn@109.226.80.43] has joined #shogun18:51
wikingblackburn: !19:09
blackburnwiking: !19:09
wikingblackburn: merge :)19:13
blackburnok19:15
blackburnbe back in 15 minutes19:16
CIA-21shogun: iglesias master * rccc31d9 / (8 files in 2 dirs): * doc fixes and less warnings - http://git.io/5nO2Ew19:50
CIA-21shogun: Sergey Lisitsyn master * r09b33ad / (8 files in 2 dirs): Merge pull request #716 from iglesias/so - http://git.io/PSwwhQ19:50
CIA-21shogun: Viktor Gal master * r5689042 / (11 files in 5 dirs): Redesign LatentLabels This way LatentLabels not only store actually the h_i (latent labels) but as well as in a general way store the actual y_i (labels). - http://git.io/kj8cGw19:59
CIA-21shogun: Sergey Lisitsyn master * reee036e / (11 files in 5 dirs): Merge pull request #718 from vigsterkr/latent - http://git.io/quttLA19:59
blackburnwiking: around?20:00
shogun-buildbotbuild #357 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/357  blamelist: iglesias <fernando.iglesiasg@gmail.com>20:32
shogun-buildbotbuild #286 of deb2 - static_interfaces is complete: Failure [failed test cmdline_static]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/286  blamelist: iglesias <fernando.iglesiasg@gmail.com>, Sergey Lisitsyn <lisitsyn.s.o@gmail.com>20:35
CIA-21shogun: Sergey Lisitsyn master * r42f6150 / data : Updated data - http://git.io/N2m4gQ20:40
-!- naywhaya1e [~ryan@spoon.lugatgt.org] has joined #shogun20:40
-!- zxtx [~zv@c-24-18-130-24.hsd1.wa.comcast.net] has quit [Ping timeout: 240 seconds]20:44
-!- Netsplit *.net <-> *.split quits: naywhayare20:45
-!- gsomix [~gsomix@178.45.80.114] has joined #shogun20:58
gsomixgood evening20:59
-!- gsomix_ [~gsomix@109.169.131.21] has joined #shogun21:00
shogun-buildbotbuild #358 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/358  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>21:00
-!- gsomix [~gsomix@178.45.80.114] has quit [Read error: Connection reset by peer]21:02
shogun-buildbotbuild #347 of deb1 - libshogun is complete: Failure [failed git]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb1%20-%20libshogun/builds/347  blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>, Viktor Gal <viktor.gal@maeth.com>21:03
@sonney2kgsomix_, hey there!21:09
@sonney2kgsomix_, how is 0-copy?21:09
gsomix_sonney2k, there is one small problem. I need overload free_feature_matrix in swig. now I'm trying resolve. :)21:11
-!- gsomix_ is now known as gsomix21:12
@sonney2kgsomix, hmmhh indeed IIRC blackburn also did nasty things when preprocessing freatures (freeing the feature matrix)21:13
@sonney2kgsomix, is there a way in python to 'own' the numpy array?21:13
-!- heiko1 [~heiko@host86-185-9-87.range86-185.btcentralplus.com] has quit [Quit: Leaving.]21:14
gsomixsonney2k, own? what do you mean? I just get internal buffer from numpy array and keep it.21:15
@sonney2kgsomix, so you Py_INCREF?21:17
@sonney2kor how do you ensure that this buffer is still valid later?21:18
gsomixyep, automatically in PyObject_GetBuffer21:18
@sonney2kok21:18
gsomixand decref in ReleaseObject.21:18
gsomixonly one problem - refcount in DenseFeatures.21:18
gsomix*ReleaseBuffer21:19
gsomixI need something like SGMatrix with refcount=2, because buffer is external21:20
gsomixand I need overload free_feature_matrix for proper behavior21:20
@sonney2kgsomix, why do you need to overload free_feature_matrix?21:25
gsomixhm, waaaaaait...21:25
@sonney2kisn't increasing the refcount in SGMatrix sufficient?21:25
@sonney2kgsomix, ^21:35
gsomixnope. hm, but how can I increase refcount?21:36
gsomixI just create new SGMatrix with numpy array's buffer21:36
@sonney2kgsomix, why not?21:37
gsomixsonney2k, hack? I just  don't need to free buffer in DenseFeatures (free_feature_matrix or destructor).21:39
gsomixmaybe it's sufficient, but it's hack.21:40
@sonney2kgsomix, the buffer is not freed21:41
@sonney2kgsomix, just the refcount is decreased21:41
@sonney2kgsomix, or do you want to call ReleaseBuffer?21:42
gsomixI need to call ReleaseBuffer.21:42
gsomixI figured out one solution.21:47
@sonney2kgsomix, ?21:47
gsomixI can to keep one more copy of new SGMatrix in Py_buffer object, that I getting from numpy array.21:48
gsomixrefcount++21:48
@sonney2kgsomix, for refcounts it is easy - you just make the ref/unref functions public in SGReferencedData21:48
@sonney2kthat is even better :D21:48
gsomixsonney2k, public? are you sure? :)21:48
gsomixsonney2k, so what do you prefer?21:49
@sonney2kbut for ReleaseBuffer you would have to either modify dense features to store the py object (awful :( or register some kind of callback function21:50
@sonney2klike void release_buffer() :D21:50
@sonney2kgsomix, yours!21:50
gsomixok21:50
@sonney2kgsomix, ok with the callback idea? or do you have any other?21:55
gsomixat this moment - other, but I'm not sure. need to experiment.21:56
gsomixthanks! :)21:56
gsomixsonney2k, ah!21:57
blackburnsonney2k: he needs an LHC to check that21:57
gsomixafter all I need to overload free_feature_matrix. I need to call ReleaseBuffer in this method21:58
gsomixblackburn, pffff. I have sent curiosity for checking some my shogun's ideas. :)22:00
gsomix*Curiosity22:00
@sonney2kgsomix, so ok with the release_buffer callback idea?22:04
gsomixI don't know, why I need this callback func.22:04
gsomixbecause ReleaseBuffer don't free buffer's memory.22:06
gsomixit seems I'm a little misunderstood or confused.22:06
gsomix:)22:06
@sonney2kgsomix, how do you call ReleaseBuffer?22:09
@sonney2kor better where is it called?22:09
gsomixsonney2k, in destructor and free_feature_matrix.22:10
@sonney2kgsomix, of DenseFeatures you mean?22:10
gsomixyep22:10
@sonney2kbut you can you make DenseFeatures to be aware of that - I mean it knows nothing about python or anything22:10
gsomixI can overload destructor and free_feature_matrix in swig22:11
@sonney2kgsomix, how?22:11
gsomixmmm, builtin and little hack22:12
gsomixI have overload destructor already22:12
@sonney2kgsomix, I think it is sufficient to overload free_feature_matrix22:12
@sonney2ksince that is called in destructor22:12
gsomixyep =____=22:12
gsomixhehe, I'm stupid22:13
@sonney2kok - then it is all good22:13
@sonney2kgsomix, do it and PR it!22:13
gsomixaha22:13
-!- zxtx [~zv@c-67-170-78-150.hsd1.wa.comcast.net] has joined #shogun22:24
-!- blackburn [~blackburn@109.226.80.43] has quit [Quit: Leaving.]22:37
-!- blackburn [~blackburn@109.226.80.43] has joined #shogun22:37
-!- blackburn1 [~blackburn@109.226.78.36] has joined #shogun22:39
-!- blackburn [~blackburn@109.226.80.43] has quit [Ping timeout: 245 seconds]22:41
-!- blackburn [~blackburn@62.106.106.114] has joined #shogun22:42
-!- blackburn1 [~blackburn@109.226.78.36] has quit [Ping timeout: 272 seconds]22:46
-!- blackburn [~blackburn@62.106.106.114] has quit [Quit: Leaving.]23:19
gsomixsonney2k, around?23:20
@sonney2kgsomix, yes?23:38
gsomixsonney2k, what do you think about runtime hack? I can only to modify DenseFeatures method's table in init (runtime) section of module.23:39
gsomixis it awful?23:41
@sonney2kgsomix, for free_feature_matrix you mean?23:51
gsomixyep23:51
@sonney2kgsomix, I think it is already hacky anyways - so just do it :D23:51
gsomixsonney2k, btw, did you read Levi "Hackers: heroes of computer revolution"? :)23:57
--- Log closed Thu Aug 16 00:00:17 2012

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!