| --- Log opened Tue Aug 21 00:00:17 2012 | ||
| @sonney2k | yoh_, alright. uploaded shogun 1.1.0-6 | 00:53 |
|---|---|---|
| * sonney2k starts an upgrade to wheezy | 00:54 | |
| shogun-buildbot_ | build #330 of deb2 - static_interfaces is complete: Failure [failed test octave_static] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/330 blamelist: Soeren Sonnenburg <sonne@debian.org> | 00:59 |
| yoh_ | sonney2k: haven't consciously used shogun for a while :-/ | 01:33 |
| yoh_ | btw -- remembering our discussion awhile ago -- have you come up with a built-in breakage of the ties for multi-class problems relying on voting of pair-wise classifiers (e.g. how it was in SVMs) | 01:34 |
| yoh_ | ? | 01:34 |
| shogun-buildbot_ | build #420 of deb3 - modular_interfaces is complete: Failure [failed test python_modular] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/420 blamelist: Soeren Sonnenburg <sonne@debian.org> | 01:34 |
| shogun-buildbot_ | build #331 of deb2 - static_interfaces is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/331 | 01:40 |
| @sonney2k | yoh_, actually the whole multiclass code got rewritten this summer by pluskid (not online right now...) - we have tons of MC methods now and I simply don't know - ask on the mailinglist he will know | 01:46 |
| -!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has joined #shogun | 02:18 | |
| shogun-buildbot_ | build #421 of deb3 - modular_interfaces is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/421 | 02:18 |
| shogun-buildbot_ | build #61 of nightly_none is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_none/builds/61 | 03:01 |
| -!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has quit [Ping timeout: 244 seconds] | 03:43 | |
| shogun-buildbot_ | build #72 of nightly_default is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_default/builds/72 | 03:44 |
| -!- emrecelikten [~emre@trir-5d800761.pool.mediaWays.net] has quit [Ping timeout: 244 seconds] | 04:26 | |
| -!- emrecelikten [~emre@trir-4d0d941d.pool.mediaWays.net] has joined #shogun | 04:41 | |
| -!- zxtx [~zv@c-24-6-91-131.hsd1.ca.comcast.net] has quit [Ping timeout: 244 seconds] | 05:24 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 07:27 | |
| -!- uricamic [~uricamic@2001:718:2:1634:e560:dc03:82b4:748f] has joined #shogun | 07:38 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Read error: Operation timed out] | 09:09 | |
| -!- gsomix_ [~gsomix@80.234.27.140] has joined #shogun | 09:10 | |
| -!- gsomix [~gsomix@109.169.157.174] has quit [Read error: Connection reset by peer] | 09:10 | |
| -!- gsomix_ [~gsomix@80.234.27.140] has quit [Quit: Ex-Chat] | 09:38 | |
| CIA-52 | shogun: Sergey Lisitsyn master * r3a54e3f / src/shogun/classifier/FeatureBlockLogisticRegression.h : Updated doc of FeatureBlockLogisticRegression - http://git.io/KlGRsw | 09:57 |
| -!- pluskid [72fa5039@gateway/web/freenode/ip.114.250.80.57] has joined #shogun | 09:58 | |
| -!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has joined #shogun | 10:19 | |
| yoo | hi alll | 10:43 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 10:48 | |
| -!- pluskid [72fa5039@gateway/web/freenode/ip.114.250.80.57] has quit [Quit: Page closed] | 11:18 | |
| CIA-52 | shogun: Sergey Lisitsyn master * r64f1532 / src/shogun/features/SubsetStack.h : Added get_last_subset doc - http://git.io/Zo3oEg | 11:54 |
| _____________ | oh CIA-52 you are back | 11:56 |
| _____________ | n4nd0: hey how is it going? | 11:59 |
| CIA-52 | shogun: Sergey Lisitsyn master * rf9ca4a4 / src/shogun/transfer/multitask/MultitaskLeastSquaresRegression.h : Added doc for MultitaskLeastSquaresRegression - http://git.io/V3Yehw | 12:03 |
| CIA-52 | shogun: Sergey Lisitsyn master * r2a89ec1 / src/shogun/structure/StateModel.h : Removed Math class reference in doc to avoid warnings - http://git.io/gI08Tw | 12:08 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 245 seconds] | 12:22 | |
| @sonney2k | _____________, do you know if this is caused by the - sign in front of CMath? https://github.com/shogun-toolbox/shogun/commit/2a89ec12acb3c5946ec5c726ad3794bebba3e1d6 | 12:31 |
| * wiking is talking on the phone german... after 4 years of not speaking at all :))) | 12:35 | |
| wiking | pure fun that is :D | 12:35 |
| _____________ | sonney2k: still warning?? | 13:08 |
| _____________ | wiking: ich spreche etwas deutsch! :D | 13:09 |
| @sonney2k | _____________, no but I was wondering if it is possible to keep CMath::INFTY there if one just moves the - sign | 13:12 |
| _____________ | sonney2k: I had no idea so removed that | 13:13 |
| -!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun | 13:13 | |
| -!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has quit [Quit: Page closed] | 13:20 | |
| CIA-52 | shogun: Sergey Lisitsyn master * r8d0d4e2 / src/shogun/lib/IndexBlockGroup.h : Updated doc of IndexBlockGroup - http://git.io/gmU9tg | 14:37 |
| CIA-52 | shogun: Sergey Lisitsyn master * rd5c67f9 / src/shogun/latent/LatentSOSVM.h : Updated doc of LatentSOSVM - http://git.io/XxPDvA | 14:40 |
| CIA-52 | shogun: Sergey Lisitsyn master * r4ec5ff5 / src/shogun/latent/LatentSOSVM.h : Updated LatentSOSVM apply doc causing warning - http://git.io/WSZk7g | 15:08 |
| -!- pluskid [72f6b389@gateway/web/freenode/ip.114.246.179.137] has joined #shogun | 15:18 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 15:47 | |
| n4nd0 | hey _____________ | 15:48 |
| _____________ | nice nickname I have isn't it? | 15:49 |
| _____________ | :) | 15:49 |
| _____________ | n4nd0: what's up? | 15:58 |
| n4nd0 | yeah really cool nickname | 15:59 |
| n4nd0 | _____________: not much, I just came from university | 15:59 |
| n4nd0 | _____________: you at job? | 15:59 |
| _____________ | yeah | 15:59 |
| _____________ | are your studies started already? | 16:00 |
| n4nd0 | no, not yet | 16:00 |
| n4nd0 | it was a meeting for first year students | 16:00 |
| n4nd0 | I went there to give tips and stuff like that | 16:00 |
| _____________ | did you give a tip about shoguning? | 16:02 |
| _____________ | and vodka of course | 16:03 |
| n4nd0 | haha | 16:04 |
| n4nd0 | I just told my professor about my summer in SHOGUN | 16:04 |
| _____________ | what does he think about it? :) | 16:04 |
| n4nd0 | aah he just said, oh coding summer ;) | 16:05 |
| -!- emrecelikten [~emre@trir-4d0d941d.pool.mediaWays.net] has quit [Ping timeout: 244 seconds] | 16:18 | |
| _____________ | n4nd0: do you expect you manage to put HM-SVM based on BMRM before release? | 16:38 |
| n4nd0 | _____________: I don't think so since now I am focusing more on the ASR application | 16:39 |
| _____________ | I see | 16:39 |
| n4nd0 | _____________: do you think it is important to have it for now? | 16:39 |
| _____________ | well I don't know | 16:39 |
| n4nd0 | aham ok | 16:40 |
| _____________ | I'd ask you to check mosek implementation for bugs | 16:40 |
| _____________ | have you any test you used before? | 16:40 |
| n4nd0 | I've tested and compared results with the hm-svm toolbox (the one written by Gunnar and Georg) | 16:41 |
| _____________ | can there be any regression due to last changes? | 16:41 |
| n4nd0 | what do you mean with regression? | 16:41 |
| _____________ | in soft engineering mean :) | 16:42 |
| n4nd0 | still :) | 16:42 |
| n4nd0 | what do you mean? hehe | 16:42 |
| _____________ | bug introduced by some change in other code | 16:42 |
| n4nd0 | like to come back? | 16:42 |
| n4nd0 | mmm I hope not, at least I believe it isnt' | 16:43 |
| _____________ | okay | 16:43 |
| n4nd0 | at least the hm-svm application returns exactly the same results as the toolbox | 16:43 |
| _____________ | I think we will manage to release in healthy state at 1st of september | 16:43 |
| n4nd0 | the only thing I think I am going to add before release is the PLiF support | 16:44 |
| n4nd0 | this is already in a branch of my fork | 16:44 |
| -!- alexlovesdata [~binder@194.95.174.230] has joined #shogun | 16:46 | |
| -!- vojtech [9320543b@gateway/web/freenode/ip.147.32.84.59] has joined #shogun | 16:47 | |
| CIA-52 | shogun: Chiyuan Zhang master * r2590cf5 / (8 files in 4 dirs): added doc to disable warning. - http://git.io/ZhnLIQ | 16:48 |
| CIA-52 | shogun: Sergey Lisitsyn master * r4cbdfa4 / (8 files in 4 dirs): Merge pull request #750 from pluskid/multiclass - http://git.io/KVKzpQ | 16:48 |
| -!- pluskid [72f6b389@gateway/web/freenode/ip.114.246.179.137] has quit [Quit: Page closed] | 16:49 | |
| wiking | alexlovesdata: hey hey | 16:50 |
| -!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has joined #shogun | 16:53 | |
| -!- vojtech [9320543b@gateway/web/freenode/ip.147.32.84.59] has quit [Quit: Page closed] | 17:13 | |
| -!- uricamic [~uricamic@2001:718:2:1634:e560:dc03:82b4:748f] has quit [Quit: Leaving.] | 17:22 | |
| -!- yoo [2eda6d52@gateway/web/freenode/ip.46.218.109.82] has quit [Quit: Page closed] | 17:44 | |
| n4nd0 | see you later guys | 18:24 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] | 18:25 | |
| -!- yoh_ is now known as yoh | 18:25 | |
| -!- blackburn [~blackburn@83.234.54.182] has joined #shogun | 18:48 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 19:37 | |
| n4nd0 | hey guys, one question about the final evaluation: is there a part to submit by the mentors too? | 19:39 |
| blackburn | ooooohh nice you mentioned that i totally forgot | 19:42 |
| blackburn | yes mentors do submit it too | 19:42 |
| @sonney2k | yoh, https://buildd.debian.org/status/package.php?p=shogun looks good - powerpc still has the clang error but otherwise all good | 19:42 |
| @sonney2k | yoh, so now where do you think shall I ask? | 19:43 |
| @sonney2k | n4nd0, I can tell that it is important to have BMRM based SO stuff... otherwise we won't have many users... | 19:44 |
| n4nd0 | sonney2k: ok, I understand | 19:44 |
| blackburn | n4nd0: if you want I could help you somehow with it | 19:45 |
| n4nd0 | blackburn: sure | 19:45 |
| @sonney2k | n4nd0, it is like one is invited to that *great* party but one needs a car to get there (or boat if you want :) | 19:46 |
| blackburn | n4nd0: for example I could introduce a risk for your HM model | 19:46 |
| blackburn | and you could test it | 19:46 |
| blackburn | I mean compare with that G&G :D implementation | 19:46 |
| n4nd0 | blackburn: I think that the generic risk function in the StructuredModel should work | 19:46 |
| n4nd0 | blackburn: otherwise, what risk would you like to introduce in the HM model? | 19:47 |
| blackburn | well the same but not generic | 19:47 |
| blackburn | I could add generic if you want :) | 19:49 |
| blackburn | and then you could test | 19:49 |
| blackburn | is that better for you? | 19:50 |
| n4nd0 | ok | 19:50 |
| blackburn | sonney2k: hey we have a shogun party on mars | 19:50 |
| blackburn | but no curiosities left though | 19:51 |
| blackburn | so you have to swim | 19:51 |
| -!- octopine is now known as audy | 19:59 | |
| blackburn | n4nd0: did you ever eat cat food? | 20:14 |
| blackburn | I did but nobody else did :D | 20:15 |
| blackburn | didn't* | 20:15 |
| blackburn | sonney2k: may be you? :D | 20:15 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 244 seconds] | 20:16 | |
| -!- alexlovesdata [~binder@194.95.174.230] has left #shogun [] | 20:34 | |
| yoh | sonney2k: well -- #debian-release or whatever it is ... ? or the mailing list... or 'reportbug release.debian.org' although that one probably would be stale for a while and IRC or ML might be a better choice for the discussion | 20:46 |
| -!- blackburn1 [~blackburn@188.168.13.81] has joined #shogun | 21:13 | |
| blackburn1 | sonney2k: any idea what is wrong wtih gp regression and apply_regression extend ? | 21:14 |
| -!- blackburn [~blackburn@83.234.54.182] has quit [Ping timeout: 240 seconds] | 21:14 | |
| CIA-52 | shogun: Sergey Lisitsyn master * r6d6770b / (2 files): Removal for last multiclass warnings - http://git.io/aFKt-Q | 21:17 |
| blackburn1 | n4nd0 hmm multiclass risk is more efficient as it is now | 21:38 |
| blackburn1 | it uses no compute joint feature vector | 21:39 |
| -!- yoo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has joined #shogun | 21:55 | |
| yoo | hi all | 21:55 |
| blackburn1 | yoo: hey | 21:56 |
| yoo | blackburn1: hey! how is it going ? | 21:56 |
| blackburn1 | pretty busy last days because of getting back to job I had before gsoc :) | 21:56 |
| blackburn1 | lets do that multiclass output stuff you wanted tonight | 21:58 |
| yoo | nice | 21:58 |
| yoo | I am figuring out stuff with yaml format | 21:58 |
| yoo | since today I ahve only work with shogun c++ | 21:58 |
| blackburn1 | so you switched to C++? | 21:58 |
| yoo | but plotting is a pretty hard task, using boost python .. | 21:59 |
| yoo | I try to switch to python modular | 21:59 |
| blackburn1 | okay - what makes it hard to use in python? | 21:59 |
| yoo | sorry, you didnt understand, I usually use c++ but now I try python | 22:00 |
| blackburn1 | oh :D | 22:00 |
| blackburn1 | I mix things now because quite a few users use python or C++ | 22:00 |
| yoo | what did people use usually ? | 22:00 |
| blackburn1 | I use python mainly | 22:01 |
| blackburn1 | but recently I've used C++ for problem where opencv is required | 22:01 |
| blackburn1 | opencv's python interface is crappy | 22:01 |
| yoo | I am from image processing then I have opencv routines as well | 22:01 |
| yoo | yep it is ! | 22:01 |
| yoo | thats why I am trying to export xml or yml (opencv) format to python | 22:02 |
| blackburn1 | recently I switched to vlfeat because it has quite efficient and clear phow (dense sift aka hog) implementation | 22:02 |
| blackburn1 | opencv's hog is something like WTF | 22:02 |
| yoo | I have coded my own HOG last year | 22:03 |
| blackburn1 | it is much easier to try to convert your problem to pedestrian detection than extract HOG features from it :D | 22:03 |
| yoo | opencv's hog uses some weird buffer etc .. | 22:03 |
| yoo | I/O in opencv is crappy for everythg except c++ and I must admit that the opencv community is not as "kind" as shogun one :p | 22:05 |
| blackburn1 | yoo: there is http://www.vlfeat.org/api/dsift.html I like most | 22:05 |
| blackburn1 | I never talked to any of them - what is wrong? | 22:05 |
| yoo | maybe that because opencv community is very big ^^ | 22:05 |
| blackburn1 | yeah we have only a few developers | 22:05 |
| yoo | it seems that the bindings for python are not very well update | 22:06 |
| yoo | File storage is very messy for python | 22:06 |
| blackburn1 | why? | 22:06 |
| yoo | do you know YML format ? | 22:10 |
| blackburn1 | no | 22:10 |
| blackburn1 | what is it for? | 22:10 |
| yoo | it permits to store data like matrix (data, size, type) and sequences .. in a hash talbe style | 22:10 |
| yoo | format | 22:11 |
| yoo | very efficient to parse | 22:11 |
| yoo | BUT python bindings are not up-to-date then ndoes are not well recognize etc .. | 22:12 |
| blackburn1 | I see | 22:12 |
| blackburn1 | sonney2k: okay if everything is correct HM SVM should work with BMRM now | 22:13 |
| blackburn1 | I will commit in a min | 22:13 |
| yoo | BMRM ? | 22:13 |
| blackburn1 | bundle method for risk minimization | 22:13 |
| blackburn1 | currently HM works only with mosek | 22:13 |
| yoo | yes I have seen that | 22:14 |
| yoo | I try to keep an eye on whats new here every weeks :p | 22:14 |
| blackburn1 | things will stop now unfortunately | 22:14 |
| blackburn1 | however I am thinking about CRF and other stuff like that research so may be I will extend it | 22:15 |
| yoo | yet, YML format is a right way to store data for large scale machine learning. | 22:16 |
| yoo | I was quite surprise that shogun dont have is own i/O format | 22:16 |
| yoo | for now it is only libsvm style right ? | 22:17 |
| blackburn1 | for serialization? we have quite a few | 22:17 |
| blackburn1 | json, xml, pure ascii | 22:17 |
| blackburn1 | hdf5 | 22:18 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 22:19 | |
| yoo | <blackburn1> things will stop now unfortunately : what do you mean ? End of gsoc ? | 22:19 |
| blackburn1 | yeah exactly | 22:19 |
| blackburn1 | n4nd0: I am done with generic risk probably - will commit in a minute | 22:20 |
| n4nd0 | blackburn1: I have just seen logs about the BMRM | 22:20 |
| n4nd0 | blackburn1: nice! | 22:20 |
| blackburn1 | waiting it to finish compilation | 22:20 |
| blackburn1 | resultset you added is pretty useful | 22:20 |
| blackburn1 | implementation is really straightforward with it - I thought of something more difficult | 22:21 |
| n4nd0 | blackburn1: well after that conversation we had, it seemed it was going to be something easy | 22:21 |
| blackburn1 | lets check if everything is correct | 22:21 |
| blackburn1 | in a minute more | 22:21 |
| blackburn1 | :D | 22:21 |
| n4nd0 | maybe we have misunderstood something | 22:22 |
| n4nd0 | blackburn1: let me know when you commit please | 22:23 |
| blackburn1 | I am pretty sure we didn't do any mistake out there | 22:25 |
| blackburn1 | at least it fits with multiclass things just perfectly | 22:25 |
| n4nd0 | yeah | 22:25 |
| blackburn1 | n4nd0: generic risk in multiclass is really bad | 22:26 |
| blackburn1 | it would make very sparse vectors go all around | 22:26 |
| blackburn1 | so specific should stay | 22:26 |
| n4nd0 | blackburn1: what do you mean with very sparse vectors go all around? | 22:27 |
| n4nd0 | what is the difference between making it generic and specific? | 22:27 |
| blackburn1 | n4nd0: okay imagine we have 5000 dimensional vectors and 50 classes | 22:28 |
| blackburn1 | what is dim of joint feature vector | 22:28 |
| blackburn1 | and how many zeros out there? | 22:28 |
| blackburn1 | current multiclass risk do not allocate anything | 22:28 |
| blackburn1 | generic will allocate two 250000 dimensional vectors for each feature vector | 22:29 |
| blackburn1 | and will add that to subgrad | 22:29 |
| n4nd0 | I understand what you mean | 22:30 |
| n4nd0 | although using sparse data structures that shouldn't be a problem, right? | 22:31 |
| blackburn1 | yeah but currently we are not using that | 22:32 |
| n4nd0 | ok | 22:32 |
| n4nd0 | do you want to carry out the test for HM-SVM in any case? | 22:33 |
| n4nd0 | using the StructuredAccuracy class it shouldn't take much to get the feeling it the idea is correct at least | 22:33 |
| n4nd0 | just as proof of concept | 22:33 |
| blackburn1 | so you want me to test it too? :D | 22:35 |
| blackburn1 | I don't mind but I don't know how | 22:35 |
| n4nd0 | I can test it | 22:35 |
| CIA-52 | shogun: Sergey Lisitsyn master * r530155d / src/shogun/structure/StructuredModel.cpp : Added generic risk function implementation for structured model class - http://git.io/96jvMg | 22:35 |
| n4nd0 | I was not saying it like "do this" | 22:35 |
| blackburn1 | do we have an example? | 22:35 |
| n4nd0 | now that is commited I can make it ;) | 22:35 |
| blackburn1 | oh we need to add an example I guess | 22:36 |
| blackburn1 | hmm | 22:36 |
| blackburn1 | is HMSVMModel guarded for mosek? | 22:36 |
| blackburn1 | oh that's cool | 22:37 |
| blackburn1 | nothing to change | 22:37 |
| n4nd0 | if a bundle method is used, of course | 22:37 |
| blackburn1 | n4nd0: I will add an example right now | 22:37 |
| n4nd0 | blackburn1: I am trying to work out one too :) | 22:37 |
| blackburn1 | n4nd0: what is accuracy in that example you did add to python modular? | 22:38 |
| n4nd0 | 99% | 22:38 |
| n4nd0 | do we have BMRM in modular interfaces? | 22:38 |
| blackburn1 | http://www.reactionface.info/sites/default/files/images/YvEN9.png | 22:39 |
| blackburn1 | yes | 22:39 |
| n4nd0 | haha | 22:40 |
| blackburn1 | ohhh | 22:41 |
| blackburn1 | matrix features issue | 22:41 |
| blackburn1 | michal's svm wants to have dot features | 22:41 |
| n4nd0 | game over | 22:41 |
| blackburn1 | nope | 22:42 |
| n4nd0 | any good guess for lambda? | 22:42 |
| blackburn1 | I don't have any | 22:42 |
| n4nd0 | let's change DotFeatures -> Features? | 22:42 |
| blackburn1 | yeah seems that is required | 22:43 |
| n4nd0 | yes | 22:44 |
| n4nd0 | in the DualLibQPBMSOSVM it seems that it doesn't affect much | 22:44 |
| n4nd0 | btw, CDualLibQPBMSOSVM::train_machine looks weird to me | 22:46 |
| blackburn1 | n4nd0: problem is that dlqpbmsosvm is linear SO machine | 22:46 |
| n4nd0 | blackburn1: what problem do you find with it? | 22:46 |
| blackburn1 | n4nd0: why? | 22:46 |
| blackburn1 | ahh I though linear SO machine needs dot features | 22:47 |
| blackburn1 | why do we need to have features in svm at all? | 22:47 |
| blackburn1 | if we have model that have features? | 22:47 |
| n4nd0 | I was wondering that right now too | 22:48 |
| n4nd0 | and this is actually a decision I made... | 22:48 |
| n4nd0 | I don't think we need them there too | 22:50 |
| n4nd0 | it is like the labels | 22:51 |
| n4nd0 | probably they shouldn't be given in the constructor of SOMachine | 22:52 |
| n4nd0 | but just take the ones that are in the model | 22:52 |
| n4nd0 | blackburn1: around | 22:53 |
| n4nd0 | ? | 22:53 |
| blackburn1 | sorry got interrupted by email | 22:53 |
| blackburn1 | okayy | 22:53 |
| blackburn1 | I don't know to be sure | 22:53 |
| blackburn1 | to be honest :D | 22:53 |
| n4nd0 | hehe | 22:54 |
| blackburn1 | n4nd0: we could remove it completely - they are not needed for training | 22:54 |
| n4nd0 | I agree | 22:54 |
| n4nd0 | blackburn1: do you want to do the change or do you want me to do it? | 22:54 |
| blackburn1 | I will do | 22:54 |
| n4nd0 | ok | 22:55 |
| blackburn1 | one thing is unclear here | 22:55 |
| blackburn1 | n4nd0: take a look at | 22:55 |
| blackburn1 | apply_structured | 22:55 |
| blackburn1 | in linear so machine | 22:56 |
| blackburn1 | n4nd0: we have to set features for model, rgiht? | 22:56 |
| blackburn1 | currently it looks like something wrong for me | 22:57 |
| n4nd0 | set_features sets the ones for the model too | 22:57 |
| blackburn1 | ah got it | 22:57 |
| n4nd0 | blackburn1: is sonney2k still around? | 22:58 |
| blackburn1 | n4nd0: okay I think best way is to emulate it has m_features | 22:58 |
| blackburn1 | but handle model's features | 22:58 |
| n4nd0 | blackburn1: ok | 22:59 |
| blackburn1 | good for you? | 22:59 |
| n4nd0 | blackburn1: so you are just planning to remove them from the constructor? | 22:59 |
| n4nd0 | the features param I mean | 22:59 |
| blackburn1 | from class | 22:59 |
| n4nd0 | the member too? | 22:59 |
| blackburn1 | ?? | 23:00 |
| blackburn1 | oops | 23:00 |
| blackburn1 | yes I mean | 23:00 |
| n4nd0 | ok | 23:01 |
| n4nd0 | so get_features and set_features are still there | 23:01 |
| n4nd0 | just calling the ones of the model | 23:01 |
| n4nd0 | is that what you meant with emulate? | 23:01 |
| -!- blackburn [~blackburn@37.61.181.133] has joined #shogun | 23:02 | |
| blackburn | here | 23:02 |
| blackburn | okay | 23:04 |
| blackburn | n4nd0: so we don't need features in dual lib qp bm so svm too, right? | 23:04 |
| n4nd0 | blackburn: if they go out of the LinearSOMachine, they go out of this one too | 23:04 |
| n4nd0 | dual lib qp bm so svm gets it by inheritance | 23:04 |
| blackburn | yes | 23:05 |
| yoo | re | 23:05 |
| -!- blackburn1 [~blackburn@188.168.13.81] has quit [Ping timeout: 244 seconds] | 23:05 | |
| blackburn | okay lets see if that works | 23:05 |
| n4nd0 | blackburn: how does it look like? | 23:09 |
| blackburn | compiled lets check | 23:10 |
| blackburn | I need to adjust examples now | 23:10 |
| blackburn | uh | 23:10 |
| blackburn | hm svm objective grows exponentially :D | 23:10 |
| n4nd0 | wow, really? | 23:11 |
| n4nd0 | I am not sure if that is good | 23:11 |
| blackburn | yeah I think it is caused by some error | 23:11 |
| n4nd0 | blackburn: let me know if I can help with something | 23:13 |
| blackburn | n4nd0: you could check a risk function I added | 23:15 |
| blackburn | are signs of psi_pred and psi_truth I have added correct? | 23:16 |
| n4nd0 | http://git.io/96jvMg? | 23:16 |
| blackburn | yes | 23:16 |
| blackburn | Psi(x,hat y) - Psi(x,y), right? | 23:17 |
| n4nd0 | yeah, I think so | 23:18 |
| n4nd0 | blackburn: I think there might be a problem with the definition though | 23:20 |
| blackburn | n4nd0: definition of? | 23:21 |
| n4nd0 | http://pastebin.com/PUfyBp9P | 23:21 |
| n4nd0 | the risk function | 23:21 |
| blackburn | ahhhhh | 23:22 |
| n4nd0 | it is the max of that thing there | 23:22 |
| blackburn | I forgot delta | 23:22 |
| n4nd0 | but are you sure that using the argmax is correct? | 23:22 |
| blackburn | yes, pretty sure | 23:22 |
| blackburn | I forgot delta - that is the bug for sure | 23:23 |
| n4nd0 | shouldn't it be checked for every y which one is it that maximizes l + <w, psi(x,y) - psi(x,y)> | 23:23 |
| n4nd0 | blackburn: mmmm why are you so pretty sure? | 23:23 |
| n4nd0 | I mean | 23:23 |
| blackburn | I can't see nothing wrong here | 23:23 |
| n4nd0 | the argmax will tend to minimize the first term | 23:24 |
| n4nd0 | Delta | 23:24 |
| n4nd0 | while maximize the second | 23:24 |
| n4nd0 | right? | 23:24 |
| blackburn | I think I got what you mean | 23:25 |
| blackburn | so you mean we minimize only <w,predict-truth> | 23:25 |
| blackburn | not delta? | 23:25 |
| n4nd0 | look that in his code Michal is actually maximizing the whole thing | 23:25 |
| n4nd0 | what I mean is that | 23:25 |
| n4nd0 | what you did for the risk function assumes | 23:26 |
| n4nd0 | that max_y [Delta(yi, y) + <w, Psi(xi, y) - Psi(xi,yi)>] | 23:26 |
| n4nd0 | is always given by the argmax | 23:26 |
| n4nd0 | and I think that is not true | 23:27 |
| blackburn | are you going to say delta changes the game? | 23:27 |
| n4nd0 | blackburn: do you understand what I mean? | 23:27 |
| n4nd0 | yes, it is Delta what changes it :) | 23:28 |
| blackburn | I have no proof for that but it is true for multiclass and I induce it for other so stuff | 23:28 |
| n4nd0 | blackburn: what is true for multiclass? | 23:29 |
| n4nd0 | it is not true that the max of the equation | 23:29 |
| n4nd0 | \max_{y \in \mathcal{Y}} \left[ \ell(y_i, y) + \langle {\bf w}, \Psi(x_i, y) - \Psi(x_i, y_i) \rangle \right] | 23:30 |
| blackburn | argmax_y [ <w, Psi(xi,y) - Psi(xi,yi) > ] = argmax_y [ Delta(yi,y) + <w, Psi(xi,y) - Psi(xi,yi) > ] | 23:30 |
| n4nd0 | that is not true | 23:30 |
| n4nd0 | because the term Delta(yi,y) is going to be minimized by the argmax_y | 23:31 |
| blackburn | no it is maximized | 23:31 |
| n4nd0 | mmm why? | 23:31 |
| wiking | mmm woah i need to read this conversation :D | 23:31 |
| n4nd0 | wiking: haha | 23:32 |
| blackburn | because you maximize difference between dot products | 23:32 |
| wiking | btw: anybody knows why tar -t <tarfile> doesn't work :) | 23:32 |
| blackburn | you make them farther from each other | 23:32 |
| blackburn | and loss grows | 23:32 |
| blackburn | something like that in my mind | 23:32 |
| n4nd0 | blackburn: look just at the term delta | 23:32 |
| n4nd0 | Delta(yi,yi) = 0 | 23:32 |
| n4nd0 | right? | 23:32 |
| blackburn | yes | 23:32 |
| n4nd0 | ok, now Delta(yi,y) | 23:32 |
| n4nd0 | the closer y is to yi -> the smaller Delta(yi,y) is | 23:33 |
| n4nd0 | ok? | 23:33 |
| blackburn | yes | 23:33 |
| n4nd0 | when you have a good model | 23:33 |
| n4nd0 | i.e. a good w | 23:33 |
| n4nd0 | you will get good predictions | 23:34 |
| blackburn | yes | 23:34 |
| -!- yoo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has quit [Ping timeout: 245 seconds] | 23:34 | |
| n4nd0 | i.e argmax_y for the example xi will be close to yi | 23:34 |
| blackburn | yes | 23:34 |
| n4nd0 | then Delta(yi,y) is minimized for y = argmax | 23:34 |
| n4nd0 | ? | 23:34 |
| blackburn | no, it is maximized by argmax of <w,Psi(x_i,y_i)-Psi(x_i,y)> | 23:35 |
| n4nd0 | wait | 23:35 |
| n4nd0 | we are not talking about that term now | 23:35 |
| blackburn | Delta is max then y_i is far away from y | 23:35 |
| n4nd0 | we are just seeing how Delta(yi,y) behaves | 23:35 |
| n4nd0 | exactly | 23:36 |
| blackburn | I see no problem out there | 23:36 |
| n4nd0 | the contrary happens for the other term | 23:36 |
| blackburn | the second term is zero then y_i is close to y | 23:36 |
| blackburn | and vice versa if y_i is far away from y | 23:37 |
| @sonney2k | wiking, tar -tf | 23:37 |
| n4nd0 | blackburn: I am writing it down and thinking of it :D | 23:38 |
| blackburn | okay I will try to find true bmrm minimizer meanwhile | 23:38 |
| blackburn | okay unstoppable grow of objective was caused by too small lambda | 23:40 |
| blackburn | that happened to me with other method so I guess that is ok | 23:40 |
| n4nd0 | blackburn: ok | 23:40 |
| blackburn | no I get reasonable w | 23:40 |
| blackburn | but I broke apply probably | 23:40 |
| blackburn | oh yes I did | 23:41 |
| blackburn | sonney2k: captain we are fixing SO! | 23:41 |
| blackburn | oh segfault now cool | 23:42 |
| blackburn | okay fixed | 23:43 |
| n4nd0 | blackburn: look at this | 23:43 |
| n4nd0 | blackburn: in your reasoning | 23:43 |
| blackburn | Accuracy = 0.9948 | 23:43 |
| n4nd0 | blackburn: you say that when y_i is far away from y | 23:43 |
| blackburn | n4nd0: my reasoning works :D | 23:43 |
| n4nd0 | blackburn: hahaha | 23:43 |
| n4nd0 | blackburn: c'mon ... it cannot be true! | 23:44 |
| blackburn | n4nd0: they have the same interface for increasing/decreasing | 23:44 |
| n4nd0 | blackburn: I need you to tell this | 23:44 |
| blackburn | both decrease when y_i is close | 23:44 |
| blackburn | yes tell me | 23:44 |
| n4nd0 | both are zero when y_i = y | 23:44 |
| n4nd0 | I am ok with that | 23:44 |
| n4nd0 | but when y_i is very different from y, what happens then? | 23:44 |
| n4nd0 | Delta is maximized | 23:45 |
| blackburn | both delta and loss are big | 23:45 |
| n4nd0 | but what about the second term? | 23:45 |
| n4nd0 | delta and loss are the same, aren't they? | 23:45 |
| blackburn | the same? | 23:45 |
| blackburn | why the same? | 23:45 |
| n4nd0 | what about < w, Psi(xi, y) - Psi(xi,yi)> | 23:45 |
| n4nd0 | let's sync notation | 23:45 |
| n4nd0 | for me the loss is Delta(y,yi) -> that's why I said that Delta and the loss are the same :D | 23:46 |
| n4nd0 | I guess that you meant the dot product with delta | 23:46 |
| blackburn | ah | 23:46 |
| blackburn | yeah | 23:46 |
| n4nd0 | ok | 23:46 |
| n4nd0 | so now | 23:46 |
| blackburn | okay let delta be 1st | 23:46 |
| blackburn | and w, psi the second | 23:46 |
| n4nd0 | good | 23:46 |
| n4nd0 | then | 23:46 |
| n4nd0 | what happens with the second when yi and y are very far from each other | 23:47 |
| n4nd0 | ? | 23:47 |
| blackburn | second is really big then | 23:47 |
| n4nd0 | why so? | 23:47 |
| -!- yooo [575b08cb@gateway/web/freenode/ip.87.91.8.203] has joined #shogun | 23:48 | |
| blackburn | because they are far away in feature space | 23:48 |
| n4nd0 | well | 23:48 |
| n4nd0 | it could be that by chance, this Psi(xi,y) ~ 0 | 23:48 |
| n4nd0 | then | 23:48 |
| n4nd0 | <w, psi(xi,y) - psi(xi,yi)> ~ -<w, psi(xi,yi)> | 23:49 |
| blackburn | it is not the case to help the understanding | 23:49 |
| blackburn | :) | 23:49 |
| n4nd0 | hahaha | 23:49 |
| n4nd0 | :D | 23:49 |
| blackburn | imagine classes | 23:49 |
| blackburn | to the left | 23:49 |
| blackburn | and to the right | 23:49 |
| blackburn | what happens with the second term | 23:49 |
| blackburn | then one is to the left | 23:50 |
| n4nd0 | yes | 23:50 |
| blackburn | but it should be to the right | 23:50 |
| n4nd0 | but you are assuming that | 23:50 |
| n4nd0 | y very different from yi => psi(x,y) very different from psi(x,yi) | 23:50 |
| blackburn | yes, it is | 23:50 |
| n4nd0 | there is no condition for that | 23:50 |
| blackburn | try to get back to typical svm | 23:51 |
| n4nd0 | let me give you a more formal argument a moment | 23:51 |
| -!- yoo [~eric@bdv75-2-87-91-8-203.dsl.sta.abo.bbox.fr] has joined #shogun | 23:51 | |
| -!- yoo [~eric@bdv75-2-87-91-8-203.dsl.sta.abo.bbox.fr] has quit [Client Quit] | 23:52 | |
| n4nd0 | blackburn: let's go | 23:53 |
| n4nd0 | so we have | 23:53 |
| n4nd0 | max_y [ Delta(yi, y) + < w, Psi(xi,y) - Psi(xi,yi) > ] | 23:53 |
| n4nd0 | now, the second Psi doesn't depend on y, then | 23:53 |
| n4nd0 | max_y [ Delta(yi, y) + < w, Psi(xi, y) > ] - Psi(xi, yi) | 23:54 |
| n4nd0 | agree? | 23:54 |
| blackburn | seems to be | 23:55 |
| n4nd0 | ok | 23:55 |
| n4nd0 | by definition | 23:55 |
| n4nd0 | the argmax_y maximizes < w, Psi(xi, y) > | 23:55 |
| n4nd0 | AND | 23:55 |
| n4nd0 | providing that we have a good model | 23:55 |
| n4nd0 | the argmax_y makes y close to yi; therefore, minimizes Delta(yi, y) | 23:56 |
| blackburn | no, it is a loss term | 23:56 |
| blackburn | it finds most violating y | 23:57 |
| blackburn | not most complying one | 23:57 |
| n4nd0 | http://www.cs.cornell.edu/people/tj/publications/tsochantaridis_etal_04a.pdf | 23:57 |
| n4nd0 | equations (1) and (2) | 23:57 |
| n4nd0 | I think that the concept of finding the mos violating one is related to the way of optimization | 23:58 |
| n4nd0 | cutting plane algorithm | 23:58 |
| n4nd0 | it is not related to the argmax definition | 23:58 |
| blackburn | (1) and (2) are decision functions | 23:58 |
| blackburn | they are not related with risk | 23:59 |
| n4nd0 | it was just to tell you that the idea of mos violated doesn't appear there | 23:59 |
| n4nd0 | most* | 23:59 |
| --- Log closed Wed Aug 22 00:00:05 2012 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!
