| --- Log opened Tue Jun 19 00:00:41 2012 | ||
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has joined #shogun | 00:08 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has quit [Changing host] | 00:08 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 00:08 | |
| shogun-buildbot_ | build #612 of java_modular is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/java_modular/builds/612 blamelist: blackburn91@gmail.com | 00:11 |
|---|---|---|
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 00:36 | |
| -!- blackburn [~blackburn@62.106.111.221] has quit [Quit: Leaving.] | 01:31 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] | 01:55 | |
| -!- emrecelikten [~emre@176.40.254.205] has quit [Read error: Connection reset by peer] | 02:55 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 03:21 | |
| -!- cheng [80fa500f@gateway/web/freenode/ip.128.250.80.15] has joined #shogun | 03:54 | |
| -!- cheng [80fa500f@gateway/web/freenode/ip.128.250.80.15] has quit [Ping timeout: 245 seconds] | 05:21 | |
| -!- shogun-buildbot_ [~shogun-bu@7nn.de] has quit [Ping timeout: 245 seconds] | 05:28 | |
| -!- uricamic [~uricamic@2001:718:2:1634:397b:935c:85c:2381] has joined #shogun | 08:48 | |
| gsomix | optics! | 10:08 |
| -!- gsomix [~gsomix@37.61.181.80] has quit [Quit: Ex-Chat] | 10:08 | |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 10:42 | |
| -!- emrecelikten [~emre@176.40.254.205] has joined #shogun | 12:48 | |
| -!- emrecelikten [~emre@176.40.254.205] has quit [Quit: Leaving.] | 13:59 | |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 14:06 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has joined #shogun | 14:56 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has quit [Changing host] | 14:56 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 14:56 | |
| -!- blackburn [d55783a2@gateway/web/freenode/ip.213.87.131.162] has joined #shogun | 15:11 | |
| blackburn | wiking: you promised to send report ;) | 15:14 |
| wiking | sended | 15:14 |
| wiking | :) | 15:14 |
| wiking | should b there in any seconds | 15:15 |
| blackburn | why didn't you do that yesterday? :) | 15:16 |
| wiking | hacking too late | 15:16 |
| wiking | i've just woken up man | 15:16 |
| wiking | :) | 15:16 |
| blackburn | it seems you like to get us worried :D | 15:16 |
| wiking | nono i like to work on this project | 15:17 |
| wiking | that's why the all-nighters | 15:18 |
| wiking | currently trying to work with libqp somehow :S | 15:18 |
| blackburn | ok | 15:18 |
| wiking | but that's still shit :( | 15:18 |
| wiking | n4nd0: have u started on that | 15:18 |
| wiking | or u are still working with mosek? | 15:19 |
| blackburn | what are you doing this time? | 15:19 |
| blackburn | to get you in shape and make some progress we really would like to merge anything | 15:19 |
| n4nd0 | wiking: libqp doesn't solve the problem I need t osolve | 15:19 |
| wiking | blackburn: well i'm trying already the SO implementation + m3e porting (http://jmlr.csail.mit.edu/proceedings/papers/v22/miller12/miller12.pdf) | 15:19 |
| wiking | n4nd0: how so? | 15:19 |
| blackburn | wiking: IIRC you had some simple latent svm before? | 15:20 |
| wiking | yeye | 15:20 |
| wiking | i should pull it though | 15:20 |
| n4nd0 | wiking: we need the box constraints case with inequalities | 15:20 |
| wiking | so you can start marging true | 15:20 |
| wiking | n4nd0: s.t. \sum x_i <= b_k ? | 15:21 |
| n4nd0 | wiking: check in your mail the pdf that nico sent | 15:22 |
| wiking | n4nd0: ok | 15:22 |
| n4nd0 | wiking: or even better | 15:22 |
| n4nd0 | http://iglesiashogun.wordpress.com/ | 15:22 |
| n4nd0 | second weekly report | 15:22 |
| wiking | http://s0.wp.com/latex.php?latex=s.t.~%5Cforall+i~%5Cforall+y+%5Cin+%5Cmathcal%7BY%7D+%5Csetminus+y_i+%3A+%5Cvec%7Bw%7D+%5Ccdot%5Cdelta%5CPsi_i%28y%29+%5Cge+%5CDelta%28y_i%2C+y%29+-+%5Cxi_i&bg=eeeae8&fg=4a4a49&s=0 ? | 15:24 |
| wiking | :D | 15:24 |
| wiking | mmm so does this mean that you'll actually stick with mosek for good? | 15:24 |
| n4nd0 | it looks like there is no open source available for that problem | 15:28 |
| n4nd0 | the plan is to use mosek now and later, if there's time, do a simple solver for it integrated into shogun | 15:29 |
| wiking | mmm | 15:29 |
| blackburn | n4nd0: you may try nlopt now btw | 15:30 |
| blackburn | no idea if it is any reasonable but at least open source | 15:30 |
| wiking | mmm shit then actually | 15:30 |
| wiking | because then my stuff with SO will have some problems :((( | 15:30 |
| wiking | well anyhow it's not that major atm | 15:31 |
| n4nd0 | blackburn: I will see if it can be used for this | 15:32 |
| n4nd0 | blackburn: thank you :) | 15:32 |
| blackburn | n4nd0: it can be used I believe but performance would be bad probably | 15:32 |
| blackburn | cause it is quite general | 15:32 |
| wiking | n4nd0: any plan for a pull request? | 15:33 |
| n4nd0 | https://github.com/shogun-toolbox/shogun/pull/580 | 15:34 |
| wiking | mmm | 15:34 |
| wiking | so it's pending | 15:34 |
| wiking | ok | 15:34 |
| wiking | because now i'm cherry picking your so branch | 15:34 |
| n4nd0 | yeah | 15:34 |
| n4nd0 | does it give trouble for you that's not yet merged? | 15:35 |
| wiking | well currently it's ok | 15:35 |
| wiking | just stupid in a way | 15:35 |
| n4nd0 | ? | 15:35 |
| wiking | that i'm maintaining 2 dev branches | 15:35 |
| wiking | 1 for general latent stuff | 15:35 |
| wiking | and 1 for SO latent stuff based on your code | 15:36 |
| n4nd0 | I see | 15:36 |
| wiking | since the current SO in the main branch of shogun | 15:36 |
| wiking | is 'deprecated' ;) | 15:36 |
| wiking | anyhow as soon as it's in i could merge it afaik | 15:36 |
| wiking | *afais | 15:36 |
| n4nd0 | it will be good if you start to use it | 15:37 |
| n4nd0 | I want to see it in action | 15:37 |
| n4nd0 | debug more | 15:37 |
| n4nd0 | etc | 15:37 |
| wiking | well i'm already having some fun with it | 15:38 |
| wiking | :))) | 15:38 |
| wiking | but was hoping | 15:38 |
| wiking | you'll be able to use libqp :s | 15:38 |
| -!- ckwidmer [8ca3fe9d@gateway/web/freenode/ip.140.163.254.157] has joined #shogun | 15:43 | |
| wiking | n4nd0: u have some time now? | 15:45 |
| n4nd0 | wiking: yes | 15:46 |
| n4nd0 | tell me | 15:46 |
| wiking | can u check this | 15:46 |
| wiking | http://jmlr.csail.mit.edu/proceedings/papers/v22/miller12/miller12.pdf | 15:46 |
| wiking | algo 2 | 15:46 |
| wiking | (page 5) | 15:46 |
| wiking | so i need a solver for that :) | 15:47 |
| n4nd0 | haha ok | 15:48 |
| n4nd0 | I am looking to equation 13 | 15:48 |
| n4nd0 | definetely, it has some similarities with the primal sosvm | 15:48 |
| wiking | yep | 15:48 |
| CIA-18 | shogun: Christian Widmer master * r59068d6 / (4 files in 3 dirs): extended SGSparseVector to make items accessible with operator[] - http://git.io/0tROCg | 15:49 |
| CIA-18 | shogun: Christian Widmer master * rb5c22d6 / src/interfaces/octave_modular/swig_typemaps.i : updated type map for octave to use new index map - http://git.io/3K94aw | 15:49 |
| CIA-18 | shogun: Christian Widmer master * rb89a58b / (2 files in 2 dirs): some clean-up - http://git.io/0WwzIg | 15:49 |
| CIA-18 | shogun: Sergey Lisitsyn master * r43ea7ab / (5 files in 4 dirs): Merge pull request #588 from cwidmer/master - http://git.io/060Wsw | 15:49 |
| n4nd0 | but it is not the same problem | 15:49 |
| n4nd0 | I am not handling contrainst of type ||w - w_t ||^2 <= \mu | 15:50 |
| n4nd0 | but I believe it should be possible to do | 15:50 |
| n4nd0 | the norm is the common Euclidean one right? | 15:51 |
| wiking | yeps | 15:51 |
| n4nd0 | ||w - w_t||^2 = < w-w_t, w-w_t > | 15:51 |
| n4nd0 | ? | 15:51 |
| n4nd0 | the dot product simply | 15:52 |
| wiking | since it's a l2-norm | 15:53 |
| n4nd0 | aham | 15:54 |
| n4nd0 | so it's not the same that I solve :( | 15:54 |
| n4nd0 | I am sorry | 15:54 |
| n4nd0 | you can solve that with mosek though | 15:55 |
| wiking | heheh | 15:55 |
| n4nd0 | http://docs.mosek.com/6.0/capi/node007.html#329747704 | 15:55 |
| wiking | i know that :) | 15:55 |
| n4nd0 | look there | 15:55 |
| n4nd0 | yeah, wait | 15:55 |
| n4nd0 | for my current problem | 15:55 |
| n4nd0 | Q^k = 0 | 15:55 |
| n4nd0 | but in your problem is not like that | 15:55 |
| n4nd0 | do you see my point? | 15:56 |
| n4nd0 | you have quadratic constraints I think | 15:56 |
| n4nd0 | while in the primal so-svm those don't appear | 15:56 |
| wiking | this is already some extra funky stuff the stuff i'll need with your solver is actually the algo 1 on page 4. afaik that's a simple structural svm (equation 9) | 15:57 |
| n4nd0 | yes | 15:57 |
| wiking | i mean this other one i've mentioned before i'll need as well, but i thought i could go around your SO so that's why i've started that | 15:58 |
| n4nd0 | that's my problem | 15:58 |
| wiking | ok | 15:58 |
| wiking | actually | 15:58 |
| n4nd0 | the second one is a bit different | 15:58 |
| wiking | it'll be great now | 15:58 |
| wiking | i thought it'll be a problem that the latent SO and the basic latent stuff | 15:58 |
| wiking | will have very different implementation | 15:58 |
| wiking | but now since you are depending on mosek | 15:59 |
| wiking | i think it's even good | 15:59 |
| wiking | that they are so 'separate' | 15:59 |
| n4nd0 | I don't understand what you mean | 15:59 |
| n4nd0 | what do you mean with separate? | 15:59 |
| wiking | well | 16:00 |
| wiking | the basic latent svm implementation has nothing to do with your SO | 16:00 |
| wiking | obviously | 16:00 |
| wiking | but now when i've started to use your SO framework | 16:00 |
| wiking | the whole latent SO svm is a complete rewrite of the basic latent implementation | 16:01 |
| wiking | there's almost nothing they share in the implementation part | 16:01 |
| n4nd0 | aham ok | 16:01 |
| wiking | so for me it wasn't 'elegant' enough since essentially they are similar | 16:01 |
| wiking | so i wanted to have it implemented as one | 16:02 |
| wiking | but now with the whole mosek story | 16:02 |
| wiking | i even actually like the idea that it's like this | 16:02 |
| n4nd0 | but is it just mosek what makes them so 'separate'? | 16:02 |
| wiking | since if one does not have mosek, he can still have basic latent svm | 16:02 |
| wiking | no of course not | 16:02 |
| wiking | it's the whole SO | 16:02 |
| n4nd0 | ok | 16:03 |
| -!- blackburn [d55783a2@gateway/web/freenode/ip.213.87.131.162] has quit [Ping timeout: 245 seconds] | 16:04 | |
| -!- blackburn [d55783a2@gateway/web/freenode/ip.213.87.131.162] has joined #shogun | 16:05 | |
| n4nd0 | wiking: I am going to be away some minutes now | 16:05 |
| wiking | that's ok | 16:05 |
| -!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 16:10 | |
| shogun-buildbot | build #946 of r_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/r_static/builds/946 blamelist: cwidmer@tuebingen.mpg.de | 16:15 |
| shogun-buildbot | build #857 of octave_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/octave_static/builds/857 blamelist: cwidmer@tuebingen.mpg.de | 16:21 |
| wiking | n4nd0: ping me when u r back | 16:21 |
| wiking | plz | 16:21 |
| n4nd0 | wiking: back | 16:21 |
| wiking | n4nd0: do u have an example implementation of so? | 16:22 |
| wiking | so_multiclass.cpp? | 16:22 |
| n4nd0 | yes | 16:23 |
| wiking | ok so i could use that actually | 16:23 |
| n4nd0 | what do you want to do? | 16:23 |
| wiking | well to have a working example for so latent svm :) | 16:24 |
| n4nd0 | ok | 16:24 |
| wiking | mmm | 16:24 |
| n4nd0 | tell me | 16:24 |
| wiking | data :D | 16:25 |
| wiking | i'd like to have a meaningful input for the example | 16:25 |
| shogun-buildbot | build #933 of python_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/python_static/builds/933 blamelist: cwidmer@tuebingen.mpg.de | 16:27 |
| n4nd0 | what latent datasets are around there? | 16:27 |
| -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has joined #shogun | 16:27 | |
| wiking | none :D | 16:28 |
| n4nd0 | haha there must be something c'mon | 16:28 |
| n4nd0 | btw, it looks like the mailing list has a looong delay or something :P | 16:28 |
| blackburn | it must be russian post | 16:29 |
| wiking | n4nd0: well there's one i've mentioned already | 16:30 |
| wiking | last week | 16:30 |
| wiking | but that's a simple latent svm example | 16:30 |
| n4nd0 | aham ok | 16:31 |
| wiking | there's supposedly 2 so svm | 16:31 |
| n4nd0 | but it could be good to start with something simple and see if the results are ok | 16:31 |
| wiking | (to which the author of the paper refers to) | 16:31 |
| wiking | but that's not available | 16:31 |
| n4nd0 | remember that my implementation has not been thoroughly tested yet | 16:31 |
| wiking | anywhere... and there's still no answer from the author about the dataset | 16:32 |
| wiking | yeah i know that's why i actually want to use it on a dataset so that i could actually compare | 16:32 |
| wiking | haaah! | 16:35 |
| wiking | bounced! | 16:35 |
| alexlovesdata | we can create a latent structured problem easily | 16:40 |
| alexlovesdata | using a small part of existing datasets | 16:41 |
| alexlovesdata | small enough by trainnig data size so that a generic solver would do it for testing code sanity and interfaces | 16:41 |
| wiking | 2012-06-19 15:30:23 1SgzVe-0002PU-KF <= wiking@maeth.com H=wsip-184-185-108-33.ri.ri.cox.net [184.185.108.33] P=esmtpsa X=TLSv1:AES128-SHA:128 A=cram:wiking@info2k1.hu S=2102 id=B340BD45-8B1D-43AE-91ED-8E7F60B7C22F@maeth.com | 16:41 |
| wiking | 2012-06-19 16:34:14 1SgzVe-0002PU-KF => shogun-list@shogun-toolbox.org R=dnslookup T=remote_smtp H=mail.nn7.de [85.214.94.156] X=TLSv1:DHE-RSA-AES256-SHA:256 | 16:41 |
| wiking | 2012-06-19 16:34:14 1SgzVe-0002PU-KF Completed | 16:41 |
| wiking | this is what happened | 16:41 |
| wiking | :> | 16:41 |
| wiking | yes | 16:41 |
| wiking | alexlovesdata: how? | 16:42 |
| alexlovesdata | can you explain pls what hapneed in detail ;) | 16:42 |
| alexlovesdata | happened | 16:42 |
| wiking | this is my exim's log | 16:43 |
| wiking | alexlovesdata: how would u create the dataset? | 16:43 |
| alexlovesdata | what is a exim log? | 16:43 |
| wiking | alexlovesdata: exim = my mailserver | 16:43 |
| alexlovesdata | ahh, ok that has to do with the email delays ... I was not interested in that, that is not so important for me | 16:44 |
| n4nd0 | haha good answer | 16:44 |
| alexlovesdata | no going for code is more important for me | 16:44 |
| wiking | alexlovesdata: so my problem is that i want to be able to compare our results | 16:44 |
| n4nd0 | alexlovesdata: have you some experience trying out SO for multiclass classification? | 16:45 |
| alexlovesdata | no, mostly I work on multilabel problems | 16:45 |
| wiking | alexlovesdata: so that's why i wanted to get the at least on of those latent so datasets that chun nam is referring to | 16:45 |
| alexlovesdata | there is a paper about SO for multilabel problems by Christoph Lampert NIPS2012 | 16:45 |
| n4nd0 | multilabel is not multiclass right? | 16:46 |
| alexlovesdata | right: multi-class = mutually exclusive | 16:46 |
| alexlovesdata | multi-label: one image can have many labels at the same time | 16:46 |
| alexlovesdata | I have used the code of joachims for multi-class SO | 16:47 |
| n4nd0 | ok | 16:47 |
| n4nd0 | struct svm? | 16:47 |
| alexlovesdata | but it was (as expected) not better than one-vs-all | 16:47 |
| n4nd0 | svm struct probably... | 16:47 |
| alexlovesdata | yes | 16:47 |
| n4nd0 | it is because my multiclass so kind of sucks a bit ... | 16:47 |
| alexlovesdata | what means sucks? | 16:47 |
| n4nd0 | the results look bad to me | 16:47 |
| n4nd0 | I don't know if it is normal or my implementation is messed up | 16:48 |
| n4nd0 | or both ... | 16:48 |
| alexlovesdata | bad compared to what? | 16:48 |
| alexlovesdata | for multi-class SO you have to use a higher regconst compared to one vs all | 16:48 |
| alexlovesdata | because you \|w\ | 16:48 |
| n4nd0 | one-vs-rest multiclass built with liblinear binary classifiers | 16:49 |
| n4nd0 | wait a moment there | 16:49 |
| alexlovesdata | \|w\|^2 is a sum of sub-ws compared to one vs all | 16:49 |
| n4nd0 | higher regconst? | 16:49 |
| alexlovesdata | yes, the C | 16:49 |
| n4nd0 | the C in C*sum_i slack_i? | 16:49 |
| alexlovesdata | because \|w_{mc}\|^2 = \sum_c \|w_c\|^2 | 16:50 |
| n4nd0 | mmm I don't think I understand what you mean, sorry :S | 16:50 |
| alexlovesdata | so as a quick fix: take a good C from 1vsall and multiply it with number of classes | 16:50 |
| alexlovesdata | yes the C in the slack sum | 16:51 |
| wiking | n4nd0: hahah i have to stop you there :) | 16:52 |
| wiking | alexlovesdata: so how to create the data set :) | 16:52 |
| wiking | n4nd0: i've taken over the priority for my mentor :> ask the rest later :P | 16:52 |
| alexlovesdata | my experience, nando is: struct mc is in the same order as 1 vs all if you set the regcnost well | 16:52 |
| alexlovesdata | ok wiking I listen | 16:52 |
| wiking | alexlovesdata: dataset for latent so svm | 16:52 |
| n4nd0 | wiking: all yours ;) | 16:53 |
| alexlovesdata | multi-class object detection or multi-label object detection | 16:53 |
| wiking | there's the two that chun nam is talking about... but no access to them... | 16:53 |
| alexlovesdata | latent variable is position | 16:53 |
| wiking | stupid question: how's that structural? | 16:53 |
| alexlovesdata | or: take images of object classes with background (pascal VOC) and rotate them ... latent variable would be rotation | 16:54 |
| alexlovesdata | or maybe: mosaics of images with structure | 16:54 |
| wiking | ok the last one i get :) | 16:54 |
| alexlovesdata | need to demosaic the image (that would be a fun paper for a conference!) | 16:54 |
| wiking | but like mult-class object detection | 16:54 |
| wiking | how's that structural | 16:54 |
| alexlovesdata | it is structural in the same way as struct multi-class is structural | 16:55 |
| wiking | ok so your loss function ... | 16:55 |
| wiking | ok | 16:55 |
| alexlovesdata | or multi-task object detection ... | 16:55 |
| alexlovesdata | we can check: http://pub.ist.ac.at/~chl/papers/lampert-nips2011.pdf | 16:56 |
| wiking | n4nd0: actually how's your perf for SO multi class vs svm one-vs-rest ? | 16:56 |
| alexlovesdata | whether you can do an object detection version of it | 16:56 |
| wiking | n4nd0: i should read the log :) | 16:56 |
| wiking | ok multi-label could be fun actually | 16:57 |
| wiking | alexlovesdata: but then again then we could actually stick to the dataset i've already have | 16:57 |
| wiking | as there i have 6 mammals | 16:57 |
| wiking | i.e. multi-class problem | 16:57 |
| n4nd0 | naywhayare: perf? | 16:57 |
| wiking | n4nd0: performance | 16:57 |
| alexlovesdata | I have no problem with your mammals | 16:57 |
| n4nd0 | wiking: I don't like at all | 16:57 |
| wiking | alexlovesdata: ok then i guess i need a loss function for that | 16:58 |
| alexlovesdata | if they have varying positions in the images | 16:58 |
| wiking | yep | 16:58 |
| n4nd0 | SO-SVM: 38.10% | 16:58 |
| n4nd0 | MC: 57.70% | 16:58 |
| wiking | n4nd0: :( | 16:58 |
| n4nd0 | that is using the same data both for training and testing | 16:58 |
| n4nd0 | 10 Gaussians with 100 samples each | 16:58 |
| alexlovesdata | for multi-label detection: you can use pascal voc datasets (a small part of it, say 5 object classes where one can expect correlations) | 16:58 |
| wiking | n4nd0: hahah i'll try my stuff then | 16:58 |
| wiking | n4nd0: meaning my mammals :D | 16:58 |
| alexlovesdata | n4ndo: which Cs did you use??? | 16:58 |
| wiking | alexlovesdata: people + car? | 16:59 |
| alexlovesdata | n4ndo: you can try joachims svm multiclass with same parameters to countercheck your results | 16:59 |
| n4nd0 | alexlovesdata: I am using C = number of classes in so | 16:59 |
| n4nd0 | alexlovesdata: C = 1 in the other | 16:59 |
| alexlovesdata | for seeing whether the problem sucks or you have a bug in your code | 16:59 |
| alexlovesdata | I guess in SO-SVM one class is dominating the predictions | 17:00 |
| alexlovesdata | n4ndo: did you check duality gap of your solutions??? | 17:00 |
| alexlovesdata | a bad duality gap may hint at too early termination | 17:01 |
| n4nd0 | alexlovesdata: I don't know what is the duality gap :S | 17:01 |
| alexlovesdata | this might be important to check | 17:01 |
| alexlovesdata | duality gap is (primal objective - dual objective)/dual objective | 17:01 |
| alexlovesdata | for a convex problem close to the optimum this should be very small | 17:02 |
| alexlovesdata | if it is 0.1 or larger your problem is not close to convergence | 17:02 |
| n4nd0 | ok | 17:02 |
| wiking | alexlovesdata: so actually then one more question: how would a SOLabel look like | 17:02 |
| n4nd0 | alexlovesdata: how can I compute the primal and udal objectives? | 17:03 |
| alexlovesdata | SOLabel for multi-class would be a vector | 17:03 |
| alexlovesdata | for multi-label same a vector | 17:03 |
| wiking | alexlovesdata: yeah but how would u construct that vector | 17:03 |
| alexlovesdata | well i-th entry = i-th class | 17:03 |
| n4nd0 | wiking: that is already done I think | 17:03 |
| n4nd0 | wiking: check MulticlassSOLabels | 17:04 |
| n4nd0 | under structure | 17:04 |
| wiking | n4nd0: mmm | 17:04 |
| wiking | lesse | 17:04 |
| n4nd0 | each label is a real number | 17:04 |
| wiking | n4nd0: i was checking your example | 17:04 |
| alexlovesdata | N4ndo: what is the precise mathemtical formulation of your SO-SVM for multi-class? | 17:04 |
| alexlovesdata | can you give a pdf with its formulation? | 17:05 |
| alexlovesdata | for a general treatment of duality gap | 17:05 |
| alexlovesdata | Boyd Vandenberghe (the book about convex optimization) is a good starting point | 17:05 |
| -!- blackburn [d55783a2@gateway/web/freenode/ip.213.87.131.162] has quit [Quit: Page closed] | 17:05 | |
| n4nd0 | alexlovesdata: http://iglesiashogun.wordpress.com/2012/06/04/second-weekly-report-gsoc-2012/ | 17:06 |
| n4nd0 | I have there a formulation of the optimziation problem | 17:06 |
| wiking | n4nd0: so basically you create a SGVector and for i-tth example you set the i-th value of the vector to the given real number (that corresponds to the label)? | 17:06 |
| n4nd0 | in any case I screwed before | 17:06 |
| n4nd0 | because the C I was changing is the C in min w'*C*w | 17:06 |
| n4nd0 | not in C*sum_i slack_i | 17:06 |
| n4nd0 | wiking: it is not a SGVector | 17:07 |
| wiking | n4nd0: i know, but the way to create it... | 17:07 |
| alexlovesdata | oops! your C! | 17:07 |
| wiking | n4nd0: i was just checking the constructor | 17:07 |
| n4nd0 | wiking: ok | 17:07 |
| alexlovesdata | then you need: C_struct=1/numclasses in that case | 17:07 |
| n4nd0 | wiking: I mean if you tell me you create an SGVector for the labels, I have to tell you that I don't, because it is not done ;) | 17:07 |
| n4nd0 | alexlovesdata: are both Cs equivalent in that sense? | 17:08 |
| alexlovesdata | is C a scalar or a matrix??? | 17:08 |
| n4nd0 | one the inverse of the other? | 17:08 |
| n4nd0 | matrix | 17:08 |
| wiking | n4nd0: no i was just asking how that SGVector in the ctor is interpreted | 17:08 |
| alexlovesdata | in one vs all you have w^2 + C/n \sum_i slack_i | 17:09 |
| alexlovesdata | why is your C a matrix??? | 17:09 |
| alexlovesdata | a diagonal one? | 17:09 |
| n4nd0 | alexlovesdata: that matrix is something that Nico wrote in a paper he did for the problem formulation, I have just seen it there | 17:10 |
| n4nd0 | right now for me it is just the identity matrix | 17:10 |
| alexlovesdata | it cannot be a general symmetric matrix!!! then your problem gets degenerated IF C has negative eiogenvalues | 17:10 |
| n4nd0 | so like if it was not there | 17:10 |
| alexlovesdata | can you give me link to rthe formula of nico? (Alex sighs) | 17:11 |
| alexlovesdata | nando: someting is strange about your formulation | 17:12 |
| alexlovesdata | if my C is a scalar --- how do you initialize the C matrix in yuor approach - I think this is the mistake | 17:12 |
| -!- blackburn [d5578951@gateway/web/freenode/ip.213.87.137.81] has joined #shogun | 17:12 | |
| n4nd0 | https://dl.dropbox.com/u/11020840/shogun/ideas.pdf | 17:12 |
| alexlovesdata | thx nando | 17:12 |
| wiking | n4nd0: ok i think i've got it: so in the vector for each sample you set the each class' membership value? so if i have 3 classes and 10 samples then my input vector is 30, right? | 17:12 |
| n4nd0 | is input vector = weight vector | 17:13 |
| n4nd0 | = | 17:13 |
| n4nd0 | ? | 17:13 |
| wiking | n4nd0: yes for the CMulticlassSOLabels | 17:13 |
| alexlovesdata | n4ndo: pls try C= diag(1)* number of samples/number of classes | 17:13 |
| alexlovesdata | diag(1) = identity matrix | 17:13 |
| wiking | n4nd0: figured this from the example it's in the repo | 17:14 |
| n4nd0 | wiking: 3 classes with feature dim = 10, gives a weight vector of 30 | 17:14 |
| n4nd0 | not with 10 samples | 17:14 |
| n4nd0 | alexlovesdata: ok, thank you, let me try | 17:14 |
| wiking | n4nd0: so_multiclass.cpp line 39->54 | 17:15 |
| wiking | especially labs vector | 17:15 |
| wiking | :) | 17:15 |
| wiking | SGVector< float64_t > labs(NUM_CLASSES*NUM_SAMPLES); | 17:15 |
| alexlovesdata | n4ndo: how did you set the C matrix before??? | 17:15 |
| wiking | n4nd0: and u use this labs vector to construct CMulticlassSOLabels | 17:16 |
| wiking | that's why i said that this vector will contain the membership value for each class for each sample | 17:16 |
| n4nd0 | alexlovesdata: see init_opt, at the end, https://github.com/iglesias/shogun/blob/so/src/shogun/structure/MulticlassModel.cpp | 17:17 |
| wiking | n4nd0: or I'm flying here with the pink elephants here in my head | 17:18 |
| n4nd0 | wiking: ??? | 17:19 |
| wiking | n4nd0: http://www.pascalkolkman.com/delirium/images/image1.jpg | 17:19 |
| wiking | n4nd0: i'm trying to figure out what's the sgvector content for CMulticlassSOLabels | 17:20 |
| n4nd0 | wiking: the labels | 17:20 |
| wiking | yes | 17:20 |
| n4nd0 | wiking: the class labels | 17:20 |
| n4nd0 | alexlovesdata: wow the results have improved!! | 17:21 |
| n4nd0 | I never got a 40% haha | 17:21 |
| wiking | n4nd0: so for example if i 1 one sample and 2 labels then it would look like: 1,0 | 17:21 |
| wiking | or 1, 0.5 | 17:21 |
| wiking | i.e. the real values are there something like a membership value for a given class | 17:21 |
| wiking | or? | 17:22 |
| n4nd0 | alexlovesdata: I think something is still wrong though | 17:22 |
| alexlovesdata | with which C? | 17:22 |
| alexlovesdata | you can compare against svm multiclass of joachims using an equivalent C | 17:22 |
| n4nd0 | alexlovesdata: but joachims' is in the dual | 17:22 |
| alexlovesdata | (joachims has the C on the slacks) | 17:22 |
| alexlovesdata | no, he has a primal solver, too, I am sure | 17:22 |
| alexlovesdata | this is a parameter in svm multiclass | 17:23 |
| n4nd0 | alexlovesdata: I tried to use the primal but I didn't manage | 17:23 |
| n4nd0 | wiking: I don't really understand what you mean man | 17:23 |
| n4nd0 | wiking: we have labeled data | 17:23 |
| wiking | n4nd0: :DDDDd | 17:23 |
| n4nd0 | wiking: the class labels are 0, 1, 2, 3 ..... | 17:23 |
| wiking | this cannot be that difficult as it is now | 17:23 |
| wiking | yes | 17:23 |
| n4nd0 | wiking: from 0 to num_classes-1 | 17:23 |
| n4nd0 | wiking: that's all | 17:24 |
| wiking | so i have 2 labels and 1 sample | 17:24 |
| n4nd0 | but that is not possible | 17:24 |
| wiking | why? | 17:24 |
| n4nd0 | if you have two samples | 17:24 |
| n4nd0 | you need two labels | 17:24 |
| wiking | ??? | 17:24 |
| wiking | i'm sorry but what? :) | 17:24 |
| n4nd0 | your training data would be then {(x1, y1), (x2, y2)} | 17:24 |
| wiking | i have one fucking example | 17:24 |
| n4nd0 | I think we need to make a dictionary | 17:24 |
| -!- uricamic [~uricamic@2001:718:2:1634:397b:935c:85c:2381] has quit [Quit: Leaving.] | 17:25 | |
| wiking | {(x1, y1}} | 17:25 |
| wiking | that's it | 17:25 |
| wiking | but my |Y| = 2 | 17:25 |
| wiking | ok? | 17:25 |
| n4nd0 | ok | 17:25 |
| blackburn | one training example for two classes?? | 17:25 |
| wiking | how do you construct that sgvector: 1.0, 0 | 17:25 |
| n4nd0 | you cannot | 17:25 |
| wiking | blackburn: doesn'tmatter dummy example | 17:25 |
| n4nd0 | because there should be at least one training example for every class | 17:26 |
| n4nd0 | call me crazy if you like ... | 17:26 |
| wiking | ok | 17:27 |
| wiking | i think i've got it | 17:27 |
| wiking | but i don't understand it | 17:27 |
| wiking | :D | 17:27 |
| n4nd0 | .... | 17:27 |
| wiking | so your label vector would actually be something like 0,0,1,1 | 17:27 |
| wiking | for 2 labels 2 samples | 17:28 |
| wiking | or? | 17:28 |
| n4nd0 | I think that what you are saying doesn't make much sense | 17:28 |
| wiking | lol :D | 17:28 |
| n4nd0 | because you want to train a machine to recognize examples | 17:28 |
| n4nd0 | for a class for which you are not giving any sample | 17:28 |
| wiking | {(x1,y1), (x1,y2)} | 17:28 |
| wiking | i don't want to do this at all | 17:28 |
| wiking | i just want to understand your input vector | 17:28 |
| n4nd0 | that is what you were saying before | 17:28 |
| wiking | i don't want to create this | 17:29 |
| n4nd0 | I have already told you what the input vector is | 17:29 |
| n4nd0 | it's exactly the same as the one for labels/MulticlassLabels | 17:29 |
| wiking | but i wanted to narrow down the misunderstanding factor to the most stupidest example | 17:29 |
| wiking | ok | 17:29 |
| wiking | and then i've asked back | 17:29 |
| wiking | about those lines | 17:29 |
| wiking | in the xample | 17:29 |
| wiking | labs[c*NUM_SAMPLES+i] = c; | 17:30 |
| n4nd0 | ok | 17:30 |
| wiking | whether that is actually a membership value (that's why it's real variable) or why? | 17:30 |
| n4nd0 | what's the problem there? | 17:30 |
| wiking | but no | 17:30 |
| n4nd0 | membership value? | 17:30 |
| wiking | apparently it's not a membership value of a sample for a class | 17:30 |
| n4nd0 | what is taht? | 17:30 |
| n4nd0 | the label? | 17:30 |
| wiking | n4nd0: http://en.wikipedia.org/wiki/Membership_function_(mathematics) | 17:30 |
| wiking | fuzzy? | 17:31 |
| wiking | you have a membership value for a given class? | 17:31 |
| n4nd0 | just tell me what you mean with it | 17:31 |
| wiking | i cannot it seems | 17:31 |
| wiking | because if you scroll back this conversation | 17:31 |
| blackburn | why labs is a matrix? | 17:31 |
| wiking | nothing makes sense | 17:31 |
| alexlovesdata | sorry I am with a physical student for 30 minutes | 17:31 |
| blackburn | wiking: c is the class number | 17:32 |
| wiking | blackburn: i've got that | 17:32 |
| wiking | so that it's not membership | 17:32 |
| n4nd0 | labs[c*NUM_SAMPLES+i] = c | 17:32 |
| wiking | but this way | 17:32 |
| n4nd0 | NUM_SAMPLES = number of training examples of each class | 17:32 |
| blackburn | one thing I do not understand too is why labs is a matrix? | 17:32 |
| n4nd0 | blackburn: it is not a matrix | 17:33 |
| wiking | n4nd0: it is | 17:33 |
| blackburn | ah long vector | 17:33 |
| n4nd0 | blackburn: I might be indexing it so it looks like that, but it is not | 17:33 |
| n4nd0 | wiking: void gen_rand_data(SGVector< float64_t > labs, SGMatrix< float64_t > feats) | 17:33 |
| wiking | n4nd0: i'm reading that yes | 17:33 |
| n4nd0 | (SGVector< float64_t > labs | 17:33 |
| wiking | yes | 17:33 |
| wiking | still | 17:33 |
| n4nd0 | labs[c*NUM_SAMPLES+i] | 17:33 |
| wiking | OMFG | 17:33 |
| n4nd0 | is just a way to index | 17:33 |
| wiking | n4nd0: NUM_SAMPLES = number of training examples of each class | 17:33 |
| blackburn | so it has NUM_SAMPLES*NUM_CLASSES length? | 17:33 |
| n4nd0 | it doesn't make it a matrix | 17:34 |
| n4nd0 | blackburn: yes | 17:34 |
| wiking | n4nd0: ok now i've got it :D | 17:34 |
| wiking | i thought you have NUM_SAMPLES | 17:34 |
| wiking | in sum | 17:34 |
| n4nd0 | wiking: dude .... you should read the full example before | 17:34 |
| n4nd0 | :) | 17:34 |
| wiking | n4nd0: i've read it | 17:34 |
| wiking | n4nd0: but it's impossible that it wasn't understandable what i was asking | 17:34 |
| blackburn | ok I think I got it too | 17:34 |
| blackburn | so you use it to simplify that setting | 17:34 |
| blackburn | to determine which examples are of some class | 17:35 |
| blackburn | right? | 17:35 |
| n4nd0 | ok so I want to generate distinct data for each class | 17:35 |
| blackburn | I didn't read anything from the example but it may be confusing a little | 17:35 |
| n4nd0 | I loop for each of the classes | 17:36 |
| n4nd0 | generate a mean and a variance that I will use for that class (data distributed as Gaussian) | 17:36 |
| n4nd0 | and later I loop again | 17:36 |
| wiking | brb | 17:36 |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 17:36 | |
| n4nd0 | generating the feature vectors for that class | 17:36 |
| -!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Ping timeout: 260 seconds] | 17:55 | |
| -!- ckwidmer [8ca3fe9d@gateway/web/freenode/ip.140.163.254.157] has quit [Ping timeout: 245 seconds] | 17:57 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 18:01 | |
| wiking | doh n4nd0 is off | 18:01 |
| wiking | alexlovesdata: here? | 18:02 |
| wiking | alexlovesdata: so anyhow, about the multi-class data set i have: as said i have that mammals data set with 6 different mammals, so now that i *think* i understand how the CMulticlassSOLabels is constructed i'll try to use that dataset with latent so svm for testing. although i'll try it first with simple CPrimalMosekSOSVM to see the the accuracy w/o the latent var | 18:08 |
| -!- gsomix [~gsomix@188.168.2.34] has joined #shogun | 18:18 | |
| gsomix | hey | 18:18 |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 18:31 | |
| -!- gsomix [~gsomix@188.168.2.34] has quit [Remote host closed the connection] | 18:49 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 18:58 | |
| wiking | alexlovesdata: ping? | 19:06 |
| @sonney2k | blackburn, hmmhh why did you merge chris' pull request? | 19:21 |
| @sonney2k | it will explode sparsevector | 19:21 |
| @sonney2k | and uses std crap | 19:21 |
| blackburn | sonney2k: chris asked me to merge that today - I'll change it later | 19:22 |
| blackburn | he needed that for some project | 19:22 |
| blackburn | I do not like it too but he asked and I'll change so no worries | 19:22 |
| blackburn | ok? | 19:23 |
| @sonney2k | blackburn, I would say we introduce some big overhead sparse vector that can have chris' stuff | 19:27 |
| @sonney2k | as we discussed | 19:27 |
| @sonney2k | but this should really not be in shogun until done properly | 19:27 |
| @sonney2k | blackburn, any news from n4nd0 ? | 19:28 |
| @sonney2k | can I merge his SO stuff? | 19:28 |
| blackburn | sonney2k: I promise I'll remove that just after I get BSc;) | 19:28 |
| blackburn | sonney2k: he said results improved | 19:28 |
| blackburn | today he played with C and it became better | 19:29 |
| @sonney2k | blackburn, I don't care about correctness (for now) just the framework | 19:29 |
| @sonney2k | others need it so it is better to have it in main | 19:29 |
| blackburn | just merge it then | 19:29 |
| @sonney2k | k | 19:29 |
| wiking | sonney2k: n4nd0 said that it's ok and it works/compiles and the example is running as well | 19:30 |
| wiking | (for me) | 19:30 |
| CIA-18 | shogun: Soeren Sonnenburg master * r24aad4d / (38 files in 10 dirs): Merge pull request #580 from iglesias/so (+29 more commits...) - http://git.io/DO21UA | 19:31 |
| @sonney2k | wiking, alright - it is in then | 19:31 |
| wiking | ok great | 19:31 |
| wiking | thnx a lot! | 19:32 |
| @sonney2k | wiking, so now I want to see your PR! | 19:32 |
| wiking | yep | 19:32 |
| wiking | let's see if it merges... pulling | 19:32 |
| wiking | DUUUDE!!! | 19:34 |
| wiking | The data for NPcoref can be found at: | 19:34 |
| wiking | \o/ | 19:34 |
| wiking | finally! | 19:34 |
| * wiking has latent so dataset! | 19:34 | |
| wiking | alexlovesdata: ping ping | 19:35 |
| wiking | alexlovesdata: chun nam just dropped me an email now finally! | 19:35 |
| CIA-18 | shogun: Michal Uricar master * rd6c2696 / (6 files): DualLibQPBMSOSVM interface for libbmrm - http://git.io/J57Bbw | 19:36 |
| CIA-18 | shogun: Michal Uricar master * rc504412 / (4 files): Function get_dim removed from RiskFunction (will be available in StructuredModel) - http://git.io/PL1h5A | 19:36 |
| CIA-18 | shogun: Michal Uricar master * r4c424e4 / (4 files): Indentation fixed + getter/setter for bmrm parameters in CDualLibQPBMSOSVM - http://git.io/a9u69Q | 19:36 |
| CIA-18 | shogun: Soeren Sonnenburg master * rc16fde6 / (6 files): Merge pull request #582 from uricamic/BM_SOL - http://git.io/3A8T9A | 19:36 |
| shogun-buildbot | build #1073 of libshogun is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/1073 blamelist: fernando.iglesiasg@gmail.com | 19:36 |
| blackburn | it is the merge day | 19:36 |
| wiking | doh what is this build error?! | 19:36 |
| @sonney2k | wiking, you never know - maybe fernando forgot to use git commit --amend ... | 19:38 |
| wiking | mmm | 19:38 |
| @sonney2k | blackburn, do you know what the state of puffin's PR is? | 19:38 |
| @sonney2k | except for the live user in a git commit :D | 19:38 |
| wiking | shogun/kernel/ScatterKernelNormalizer.h | 19:38 |
| wiking | so it seems it's rather the kernel normalizer related stuff... | 19:39 |
| @sonney2k | n4nd0 certainly did not touch those | 19:40 |
| @sonney2k | blackburn, are the kernel normalizers now in kernel/normalizer ? | 19:40 |
| wiking | kmcquisten/master | 19:40 |
| blackburn | not I didn't that yet | 19:40 |
| wiking | sonney2k: aa9ad6a30c9e4cb08aec27bdf00c39e4332834be | 19:40 |
| wiking | blackburn: it's your merge ;P | 19:40 |
| blackburn | sonney2k: we shall wait for puffin444 cause he was stucked with porting nlopt | 19:40 |
| shogun-buildbot | build #1074 of libshogun is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/1074 | 19:41 |
| blackburn | I have no access to git now so please fix ;) | 19:41 |
| wiking | brb | 19:42 |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 19:42 | |
| @sonney2k | blackburn, uricamic's stuff is still in so folder | 19:45 |
| shogun-buildbot | build #1075 of libshogun is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/1075 blamelist: sonne@debian.org, uricar.michal@gmail.com | 19:45 |
| @sonney2k | not structure | 19:45 |
| @sonney2k | so will break anyways | 19:45 |
| @sonney2k | Test | 19:50 |
| @sonney2k | Very good works | 19:51 |
| @sonney2k | Finally online via mobile phone | 19:51 |
| CIA-18 | shogun: Soeren Sonnenburg master * rc000355 / (12 files in 2 dirs): fix compile error due to so -> structure rename - http://git.io/a8cq_Q | 20:02 |
| shogun-buildbot | build #949 of r_static is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/r_static/builds/949 | 20:05 |
| shogun-buildbot | build #1076 of libshogun is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/1076 | 20:20 |
| shogun-buildbot | build #950 of r_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/r_static/builds/950 blamelist: sonne@debian.org | 20:30 |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has joined #shogun | 20:41 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has quit [Changing host] | 20:41 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 20:41 | |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 20:50 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has joined #shogun | 20:51 | |
| -!- wiking [~wiking@wsip-184-185-108-33.ri.ri.cox.net] has quit [Changing host] | 20:51 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 20:51 | |
| -!- emrecelikten [~emre@176.40.254.205] has joined #shogun | 21:10 | |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 21:29 | |
| -!- alexlovesdata [82955843@gateway/web/freenode/ip.130.149.88.67] has quit [Quit: Page closed] | 21:43 | |
| -!- blackburn [d5578951@gateway/web/freenode/ip.213.87.137.81] has quit [Ping timeout: 245 seconds] | 22:14 | |
| -!- n4nd0 [53b32c87@gateway/web/freenode/ip.83.179.44.135] has joined #shogun | 23:34 | |
| -!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 23:40 | |
| -!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 23:53 | |
| --- Log closed Wed Jun 20 00:00:41 2012 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!
