--- Log opened Wed May 16 00:00:40 2012 | ||
-!- emrecelikten [~Anubis@217.131.4.22] has joined #shogun | 00:14 | |
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 02:04 | |
-!- av3ngr [av3ngr@nat/redhat/x-yunsofbtsbsqzqam] has joined #shogun | 02:15 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has joined #shogun | 02:47 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has quit [Changing host] | 02:47 | |
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 02:47 | |
-!- emrecelikten [~Anubis@217.131.4.22] has quit [Read error: Connection reset by peer] | 03:07 | |
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 05:11 | |
-!- av3ngr [av3ngr@nat/redhat/x-yunsofbtsbsqzqam] has quit [Remote host closed the connection] | 05:29 | |
-!- av3ngr [av3ngr@nat/redhat/x-zfabiwenfrymddtr] has joined #shogun | 05:31 | |
CIA-113 | shogun: iglesias master * r22685c6 / src/shogun/features/DotFeatures.cpp : * minor indent fix in DotFeatures - http://git.io/3SJosQ | 05:33 |
---|---|---|
CIA-113 | shogun: iglesias master * r07bc029 / (2 files): * minor indent fixes - http://git.io/0Yrjng | 05:33 |
CIA-113 | shogun: iglesias master * rf586c7e / (2 files): * minor fixes - http://git.io/dMYIaQ | 05:33 |
CIA-113 | shogun: Soeren Sonnenburg master * r6051a71 / (5 files in 2 dirs): Merge pull request #530 from iglesias/fix-indent - http://git.io/m9hqOA | 05:33 |
-!- gsomix [~gsomix@188.168.2.22] has quit [Ping timeout: 245 seconds] | 05:57 | |
-!- VarAgrawal [cf2e371e@gateway/web/freenode/ip.207.46.55.30] has joined #shogun | 07:55 | |
-!- VarAgrawal [cf2e371e@gateway/web/freenode/ip.207.46.55.30] has quit [Client Quit] | 07:56 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has joined #shogun | 08:49 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has quit [Changing host] | 08:49 | |
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 08:49 | |
-!- wiking [~wiking@huwico/staff/wiking] has quit [Client Quit] | 08:52 | |
-!- av3ngr [av3ngr@nat/redhat/x-zfabiwenfrymddtr] has quit [Quit: That's all folks!] | 09:39 | |
-!- blackburn [~qdrgsm@188.122.250.167] has joined #shogun | 09:48 | |
CIA-113 | shogun: Sergey Lisitsyn master * rf4b1c9e / (9 files in 2 dirs): Restored data in ASP (+7 more commits...) - http://git.io/sSAn7A | 09:51 |
blackburn | whoa I shouldn't did that | 09:55 |
shogun-buildbot | build #943 of libshogun is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/943 blamelist: vipin@cbio.mskcc.org | 09:57 |
-!- n4nd0 [~nando@n173-p203.kthopen.kth.se] has joined #shogun | 12:15 | |
-!- gsomix [~gsomix@188.168.13.216] has joined #shogun | 12:33 | |
gsomix | hi all | 12:33 |
blackburn | the optics expert in da house | 12:35 |
gsomix | blackburn, huh :] | 12:38 |
n4nd0 | hi gsomix | 12:38 |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has joined #shogun | 12:50 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has quit [Changing host] | 12:50 | |
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 12:50 | |
-!- wiking [~wiking@huwico/staff/wiking] has quit [Quit: wiking] | 13:18 | |
-!- n4nd0 [~nando@n173-p203.kthopen.kth.se] has quit [Read error: Operation timed out] | 13:53 | |
-!- n4nd0 [~nando@n173-p203.kthopen.kth.se] has joined #shogun | 14:01 | |
-!- wiking [~wiking@70.42.157.22] has joined #shogun | 14:02 | |
-!- wiking [~wiking@70.42.157.22] has quit [Changing host] | 14:02 | |
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 14:02 | |
-!- wiking [~wiking@huwico/staff/wiking] has quit [Client Quit] | 14:02 | |
n4nd0 | sonne|work: hi! do you have a moment? | 14:17 |
sonne|work | n4nd0: whats up? | 14:17 |
n4nd0 | sonne|work: so I have been thinking about the features issue we talked about | 14:18 |
n4nd0 | sonne|work: I am not sure if I am getting the design properly | 14:18 |
n4nd0 | take a look here a moment | 14:18 |
n4nd0 | http://dl.dropbox.com/u/11020840/FeaturesSO.pdf | 14:18 |
n4nd0 | so in the VanillaSOMachine | 14:20 |
n4nd0 | I see that we can use this CJointFeatures::dot and pass it the weight vector | 14:20 |
n4nd0 | to compute a dot product w*psi(x,y) | 14:20 |
n4nd0 | BUT | 14:21 |
n4nd0 | inside this method dot, I don't think we can avoid computing psi(x,y) explicitily | 14:21 |
n4nd0 | we need a function there that computes psi(x,y) and returns an SGVector or an SGSparseVector | 14:21 |
n4nd0 | sonne|work: or did I miss something? | 14:23 |
sonne|work | n4nd0: that is half true | 14:25 |
n4nd0 | tell me the other half then ;) | 14:25 |
sonne|work | n4nd0: you only have to have a way to compute the non-zero indices of psi(x,y) | 14:25 |
sonne|work | so for very sparse Psi/x/y this can be *a lot* less work | 14:26 |
sonne|work | for example consider you have sparse x with say 1% of the components non-zero | 14:26 |
sonne|work | and Psi(x,y) is some block thing (x for each y stacked together) you have to touch just 1% of the data | 14:28 |
n4nd0 | well I understand that is an example of when one would like to return an SGSparseVector | 14:28 |
sonne|work | why? | 14:29 |
n4nd0 | then a function Psi(x,y) that returns a vector is required in the model | 14:30 |
sonne|work | the dot product returns just a scalar | 14:30 |
sonne|work | so you need to touch only 1% of the components of w | 14:30 |
n4nd0 | sorry not to return ^, but to use SGSparseVector in the dot product | 14:30 |
sonne|work | n4nd0: the dense_dot() function is supposed to return just one scalar | 14:31 |
n4nd0 | yes | 14:31 |
sonne|work | so you never have to compute anything else other than this dot product | 14:31 |
sonne|work | but instead of precomputing some vector you directly compute the dot product | 14:31 |
sonne|work | so instead of computing a sparse vector, you look up the indices of w - and you only need to look at these indexes that Psi(x,y) is non-zero) multiply with the Psi value and sum them up and return the result | 14:32 |
n4nd0 | mmm woulnd't the trick be not to compute the elements of Psi(x,y) which corresponding w element is zero? | 14:33 |
sonne|work | no | 14:35 |
n4nd0 | I am reading it as: instead of Psi(x,y)?w, do f(x,y,w) | 14:35 |
sonne|work | the trick is to not compute Psi(x,y) for dims which are 0 | 14:35 |
n4nd0 | how can you know where Psi(x,y) is zero before computing it? | 14:35 |
sonne|work | n4nd0: because you know psi | 14:36 |
sonne|work | in the simplest case psi is the identity | 14:36 |
sonne|work | wrt x and block structure wrt y | 14:36 |
n4nd0 | I am sorry but I think I don't see it | 14:38 |
sonne|work | have a look at e.g.shogun/src/shogun/features/SparseFeatures.cpp line 307ff | 14:39 |
sonne|work | or shogun/features/PolyFeatures.cpp line 143ff | 14:40 |
sonne|work | or shogun/features/ImplicitWeightedSpecFeatures.cpp lines 172ff | 14:40 |
n4nd0 | sonne|work: I am afraid that we may be talking about different things | 14:47 |
n4nd0 | the function Psi is not a doc product | 14:48 |
n4nd0 | dot* | 14:48 |
sonne|work | n4nd0: we are talking about Psi(x,y)?w | 14:48 |
sonne|work | right? | 14:48 |
n4nd0 | we are talking about the possibility of not computing Psi(x,y) explicitily and doing the product Psi(x,y)?w | 14:49 |
sonne|work | which is computing Psi(x,y)?w :) | 14:49 |
n4nd0 | ok :) | 14:49 |
sonne|work | and that is what these functions above do | 14:49 |
sonne|work | with a more simple Psi not dependent on y | 14:50 |
n4nd0 | I but all those functions above have the equivalent to Psi(x,y) already computed | 14:50 |
sonne|work | no | 14:50 |
n4nd0 | remove I ^ | 14:50 |
sonne|work | they compute the index to the non-zero idx=Psi(x) | 14:51 |
sonne|work | idx of Psi(x) | 14:51 |
n4nd0 | from PolyFeatures | 14:52 |
n4nd0 | for (int j=0; j<vec2_len; j++) | 14:52 |
n4nd0 | { | 14:52 |
n4nd0 | float64_t output=m_multinomial_coefficients[j]; | 14:52 |
n4nd0 | for (int k=0; k<m_degree; k++) | 14:52 |
n4nd0 | { | 14:52 |
n4nd0 | output*=vec[m_multi_index[cnt]]; | 14:52 |
n4nd0 | cnt++; | 14:52 |
n4nd0 | } | 14:52 |
n4nd0 | sum+=output*vec2[j]; | 14:52 |
n4nd0 | } | 14:52 |
n4nd0 | there the loop is done for all the elements | 14:53 |
n4nd0 | the loop on j | 14:53 |
sonne|work | n4nd0: well x is considered to be dense here | 14:53 |
n4nd0 | and in the case it is sparse you will just jump over the zero elements | 14:55 |
n4nd0 | but that can be done because you know the elements that are zero | 14:55 |
n4nd0 | then you already know this Psi(x) | 14:55 |
sonne|work | n4nd0: no I don't jump over zero elements | 14:56 |
sonne|work | I only touch the non-zero ones | 14:56 |
sonne|work | because I only store those | 14:56 |
sonne|work | that would be the sparsefeatures example | 14:56 |
sonne|work | please paste it too :) | 14:57 |
n4nd0 | yes, I understand that case | 14:57 |
sonne|work | ok | 14:57 |
sonne|work | by doing it this way you can support dense and sparse x | 14:57 |
sonne|work | and even strings etc | 14:58 |
sonne|work | in the same framework | 14:58 |
n4nd0 | so what you suggest is, instead of defining Psi(x,y), an application dependent function in SO | 14:59 |
n4nd0 | let's define Psi(x,y)*w | 15:00 |
sonne|work | yes | 15:01 |
sonne|work | whcih also defines w <- w + alpha*psi(x,y) | 15:02 |
sonne|work | same principle... | 15:02 |
sonne|work | n4nd0: note that this enabled me to train on 50mio examples in a 10 million dimensional feature space | 15:03 |
sonne|work | of-the-shelve-machine | 15:03 |
n4nd0 | sonne|work: :) | 15:03 |
sonne|work | I can tell that this trick is much better than thinking about kernels and SO | 15:04 |
n4nd0 | sonne|work: where does this come from by the way? | 15:04 |
sonne|work | what? | 15:04 |
n4nd0 | the trick | 15:05 |
sonne|work | it is mine... | 15:05 |
sonne|work | n4nd0: this paper with vojtech http://sonnenburgs.de/soeren/publications/SonFra10.pdf | 15:05 |
n4nd0 | sonne|work: I am going to read it, I hope it will help me to understand better what I've to do here | 15:06 |
n4nd0 | and more important, why ;) | 15:07 |
CIA-113 | shogun: Soeren Sonnenburg master * ra60c648 / (9 files in 3 dirs): Merge pull request #527 from pluskid/multiclass-ecoc (+5 more commits...) - http://git.io/aeaOjg | 15:38 |
sonne|work | n4nd0: one big advantage is that you have to develop just one SO learning algorithm | 15:50 |
sonne|work | n4nd0: that algorithm will work with all feature types, dense, sparse, strings etc | 15:50 |
sonne|work | no need to change a line | 15:50 |
n4nd0 | sonne|work: that sounds very nice | 15:51 |
sonne|work | n4nd0: for example linear svms are usually tweaked for sparse data | 15:51 |
sonne|work | but since we have dotfeatures in shogun | 15:51 |
sonne|work | we have them tweaked for sparse, dense, strings... | 15:51 |
sonne|work | and one can even mix (as in concatenate) dense and sparse features... | 15:51 |
n4nd0 | CombinedDotFeatures | 15:52 |
sonne|work | so if you want speed and kernels are no option (which they are not for SO) you want that | 15:52 |
n4nd0 | sonne|work: why is it that kernels cannot be used in SO? | 15:53 |
n4nd0 | where does the lack of speed come from? | 15:53 |
sonne|work | kernel evaluations everywhere | 15:54 |
-!- nicococo [~nico@lacedcoffee.ml.tu-berlin.de] has joined #shogun | 16:08 | |
n4nd0 | hey nicococo! | 16:08 |
nicococo | buenos fernando | 16:08 |
n4nd0 | buenas Nico ;) | 16:09 |
n4nd0 | nicococo: sonne|work suggested to move our discussions to the channel here, is that ok for you? | 16:09 |
nicococo | the sonney.. well why not :) | 16:09 |
-!- emrecelikten [~Anubis@217.131.4.22] has joined #shogun | 16:10 | |
n4nd0 | nicococo: so a couple of things I wanted to let you know about | 16:10 |
sonne|work | nicococo: any objections in using COFFIN aka dotfeatures from the beginning for SO stuff? | 16:10 |
nicococo | one thing i would change for so toolbox is the CStructuredLoss | 16:10 |
sonne|work | s/in/to | 16:10 |
n4nd0 | nicococo: it is about the loss that I wanted to tell you | 16:12 |
n4nd0 | nicococo: look http://www.shogun-toolbox.org/doc/en/current/classshogun_1_1CLossFunction.html | 16:12 |
n4nd0 | I think this is exactly what we need, and it is already in shogun | 16:12 |
nicococo | i suspected that ;) | 16:12 |
n4nd0 | I should have looked for it from the beginning, my bad :D | 16:13 |
-!- Francis_Chan [~Adium@58.194.224.120] has joined #shogun | 16:13 | |
nicococo | but i am not sure it is applicable here .. | 16:13 |
n4nd0 | why so? | 16:13 |
nicococo | loss(prediction, label) | 16:13 |
nicococo | that's fine for svm loss(pred,label) = max(0,1-label*pred) but not so for so | 16:14 |
nicococo | (structured output) | 16:14 |
n4nd0 | nicococo: maybe this one then http://www.shogun-toolbox.org/doc/en/current/classshogun_1_1CLoss.html | 16:15 |
nicococo | we need a loss(x) | 16:15 |
nicococo | yeah.. that is what i am looking for :) | 16:16 |
n4nd0 | but I don't really like how it is implemented there ... with macros | 16:16 |
n4nd0 | I would prefer something similar to the CLossFunction I showed you before | 16:16 |
nicococo | yep and second derivative is missing | 16:17 |
nicococo | i know you think it is more elegant, right.. | 16:17 |
nicococo | but they don't make a big difference.. so it is unlikely to play a lot (at least with convex ones) with loss functions in your application | 16:18 |
n4nd0 | nicococo: what about what you said in the beginning, what would you change of CStructuredLoss? | 16:18 |
n4nd0 | aham, I see | 16:18 |
nicococo | when you say structured loss everybody expects the delta loss and not the surrounding loss function | 16:18 |
nicococo | just a matter of naming | 16:20 |
n4nd0 | nicococo: yeah, I agree with you, it is confusing to use that name | 16:20 |
nicococo | anyway, is there a possibility to extend the CLossFunction in our sense?? | 16:20 |
n4nd0 | yes | 16:20 |
n4nd0 | I think that is the best possibility | 16:20 |
nicococo | it would be nice to use them, right? | 16:20 |
n4nd0 | to put there some methods loss(z) | 16:20 |
n4nd0 | first_derivative(z) | 16:20 |
n4nd0 | and so on | 16:20 |
nicococo | and some indicator functions (convex,smooth,...) | 16:21 |
n4nd0 | these are just like constants for every loss function right? | 16:21 |
nicococo | yep | 16:21 |
n4nd0 | is it interesting to have them as indicator functions for any special reason? | 16:22 |
n4nd0 | or boolean members are just fine | 16:22 |
n4nd0 | stupid question, already in gist... | 16:22 |
nicococo | well when you use a CRF and have a loss function which states smooth=1 then you can use LBFGS which should be faster than any non-smooth optimization | 16:23 |
n4nd0 | all right | 16:24 |
n4nd0 | nicococo: now related to what sonne|work asked you about COFFIN, we have been discussing about it for quite a while :D | 16:25 |
nicococo | ohh.. wait one more issue | 16:26 |
n4nd0 | yeah tell me | 16:26 |
nicococo | you added an apply-function for the SOMachine.. put this should be only a wrapper calling the apply-function of SOModel (which is missing) | 16:27 |
nicococo | the reason is that in SOModel is already the argmax (=apply) code that is application specific | 16:27 |
n4nd0 | aham I see | 16:29 |
n4nd0 | not exactly a wrapper though, I think | 16:29 |
n4nd0 | since argmax does it just for one feature vector and apply may work on a whole set of features | 16:30 |
n4nd0 | but it should be basically to call iteratively argmax | 16:30 |
n4nd0 | or? | 16:30 |
nicococo | so, if apply is for multiple samples then yes | 16:31 |
n4nd0 | it may be for multiple or just for one, it depends on the arguments (it is overloaded) | 16:31 |
n4nd0 | apply() -> the whole CFeatures* member | 16:32 |
n4nd0 | apply(CFeatures*) -> all the features in the argument | 16:32 |
nicococo | okay.. | 16:32 |
n4nd0 | apply(int) -> just to the indexed | 16:32 |
nicococo | so, overall a pretty nice job so far :) i am almost sure it will work | 16:33 |
nicococo | 99% | 16:33 |
n4nd0 | :) | 16:33 |
nicococo | wanna talk about the test application? | 16:33 |
n4nd0 | can we talk a moment about the COFFIN thing? | 16:34 |
n4nd0 | I have been a bit stuck with that | 16:34 |
n4nd0 | I understand that the idea would we to provide functions in CStructuredModel to compute w*Psi(x,y) and w <- w + alpha*Psi(x,y) | 16:35 |
n4nd0 | instead of the actual Psi(x,y) | 16:36 |
nicococo | okay | 16:37 |
nicococo | is the sunny-boy here and can give some ideas? | 16:37 |
n4nd0 | sonne|work: around? | 16:37 |
nicococo | coffin | 16:38 |
nicococo | did you already talked to fernando about that? | 16:38 |
sonne|work | y | 16:39 |
-!- sonne|work [~sonnenbu@194.78.35.195] has left #shogun [] | 16:39 | |
n4nd0 | nicococo: but do you thing that the function definitions will change a lot? | 16:40 |
nicococo | well that was a short.. n4and0 soeren suggested to put additional functions into SOModel? | 16:40 |
n4nd0 | nicococo: http://www.shogun-toolbox.org/irclogs/%23shogun.2012-05-16.log.html | 16:42 |
n4nd0 | take a look from 14:59 | 16:42 |
n4nd0 | I think that is interesting that | 16:43 |
n4nd0 | sonne|workn4nd0: one big advantage is that you have to develop just one SO learning algorithm15:50 | 16:43 |
n4nd0 | sonne|workn4nd0: that algorithm will work with all feature types, dense, sparse, strings etc | 16:43 |
n4nd0 | since it covers what you said the last day about supporting sparse representation of the joint features | 16:44 |
nicococo | yep, i have read the coffin paper but never worked with it, so i have no experience at all here ... i guess the trick is to find a way around dot-product here, right? | 16:46 |
nicococo | is it an application thing (SOModel) or more a linear learning machine thing (LinearSOMachine). i prefer the latter one | 16:48 |
n4nd0 | I thought of that | 16:48 |
n4nd0 | I think it is an application thing | 16:48 |
n4nd0 | since we cannot make an abstraction on how Psi looks internally for this | 16:49 |
nicococo | it is not completely uncoupled. but it is concerned with the representation while learning which is a machine thing .. | 16:49 |
nicococo | computation of the dotproduct w'psi | 16:50 |
n4nd0 | in order to do w*Psi(x,y) without computing Psi(x,y) explicitily, it is required to know what Psi does | 16:50 |
n4nd0 | I am trying to think of it as a representation thing not learning | 16:52 |
nicococo | thats true, if you need to know, then it has to be in the SOModel.. but what about the CResult struct .. it is rendered useless, right | 16:52 |
n4nd0 | yeah, this definetely changes CResult | 16:52 |
nicococo | i mean, you need a special SOModel and a special LearningMachine | 16:52 |
n4nd0 | I don't think you need a spacial LearningMachine for it | 16:53 |
nicococo | then we have to change the interface again... | 16:53 |
n4nd0 | it is just like instead of making a function Psi(x,y) in the model | 16:53 |
n4nd0 | we do another f(v,x,y) = v*Psi(x,y) | 16:53 |
n4nd0 | where v is any vector | 16:54 |
nicococo | instead of computing w'psi in the learning machine it is moved to SOModel | 16:54 |
n4nd0 | yes | 16:54 |
n4nd0 | later, from the learning machine, one call it as f(w,xi,yi) | 16:54 |
n4nd0 | does that make sense for you? | 16:54 |
nicococo | yes | 16:55 |
n4nd0 | I think it makes too, but this consideration is quite general | 16:55 |
nicococo | well maybe it works perfectly in general | 16:55 |
nicococo | but i don't know | 16:55 |
nicococo | yet | 16:55 |
n4nd0 | it could be a good idea to think how this would change the function in a particular example, let's say hmm | 16:56 |
nicococo | (crf, so-svm, optimization schemes.. what about latent structures) | 16:56 |
n4nd0 | at first sight it could be applied to latent defining a function g(w,h,x,y) | 16:57 |
nicococo | for hmm the computations are okay with that (i am pretty sure) | 16:57 |
n4nd0 | where h in the latent feature vector | 16:57 |
n4nd0 | but I should talk to wiking about it first | 16:57 |
nicococo | but you need to construct one contraint per iteration where you need aaccess to the psi(x_i,y_i)-psi(x_i,y_hat) | 16:58 |
n4nd0 | is it that in the latent or the general case? | 16:59 |
nicococo | in the general case | 16:59 |
n4nd0 | do you mean https://gist.github.com/2634487 lines 35 and 36? | 16:59 |
nicococo | okay.. i think the coffin idea may work for algorithm where you can explicitly access the dot-products, right | 17:00 |
nicococo | ? | 17:00 |
nicococo | but when you have a generic QP solver that is not the case | 17:00 |
n4nd0 | when you said psi(x_i,y_i)-psi(x_i,y_hat) | 17:01 |
n4nd0 | is it that for doing w*( psi(x_i,y_i)-psi(x_i,y_hat) ) ?? | 17:02 |
n4nd0 | or the other way round in the difference according to the gist | 17:02 |
nicococo | in the gist on line 40 | 17:02 |
n4nd0 | solve qp?? | 17:02 |
nicococo | you call a QP solver with a inequality constraint matrix A | 17:03 |
nicococo | Ax <= b | 17:03 |
nicococo | this matrix A has a block structure and containts all psi(x_i,y_i)-psi(x_i,y_hat) ever constructed | 17:03 |
nicococo | (here x has also a block structure and contains our solutaion vector w) | 17:04 |
n4nd0 | what do you mean with block structure? | 17:04 |
nicococo | if you solve a primal svm | 17:05 |
nicococo | the solution vector x (delivered by QP solver) consists of [w, slacks] | 17:06 |
nicococo | so this is a block structure | 17:06 |
n4nd0 | ok | 17:06 |
nicococo | nothing special, just that the vector consists of several variables | 17:06 |
nicococo | but if we need the psi-psi_hat then for this machine the coffin makes no sense, right? | 17:07 |
n4nd0 | why? In order to do that operation? | 17:08 |
nicococo | because we cannot replace the dot-product calculation of the qp-solver | 17:09 |
n4nd0 | but we can give it directly psi-psi_hat | 17:10 |
nicococo | but then we need an explicit vector for that.. the coffin idea is to avoid that if i understand it correctly | 17:11 |
nicococo | (e.g. when using 100mio dimensions) | 17:11 |
n4nd0 | we can still avoid it when we do w*Psi(x,y) | 17:12 |
n4nd0 | in the other cases, to pass it to the QP solver, we can just give it Psi-Psi_hat directly | 17:13 |
nicococo | still an explicit vector | 17:13 |
n4nd0 | yes, that's true ... | 17:13 |
n4nd0 | I don't know, I am kind of confused with this coffin idea | 17:13 |
n4nd0 | :( | 17:13 |
nicococo | me too :)) | 17:14 |
@sonney2k | guys then read the paper | 17:14 |
nicococo | read it already.. there are some brackets missing :) | 17:14 |
@sonney2k | nicococo, heh | 17:14 |
n4nd0 | sonney2k: I have read half of it | 17:14 |
nicococo | table 1 ;) | 17:15 |
@sonney2k | nicococo, isn't Psi huge in practise? | 17:15 |
@sonney2k | c | 17:15 |
@sonney2k | how can you even solve it? | 17:15 |
nicococo | well if you think 2000 is huge | 17:15 |
@sonney2k | I mean when you have Psi-Psi_hat's - and I guess one per iteration? | 17:15 |
nicococo | yes | 17:15 |
@sonney2k | so your feature space is exploding with #iterations? | 17:16 |
@sonney2k | or do you compress them somehow? | 17:16 |
nicococo | no not my feature space it is the amount of constraints given to the qp solver | 17:16 |
nicococo | feature space still doesn't exceed 10000 dims | 17:16 |
nicococo | and stays the same in one application | 17:17 |
nicococo | so, i understand when using coffin, one can use high dim feature spaces, right? | 17:18 |
nicococo | but you have to replace any dot-product | 17:18 |
@sonney2k | yeah but it is #iterations * dim(Psi-Psi_hats) | 17:18 |
@sonney2k | so the only idea I have here is to return a sparse result for a Psi-Psi_hat difference | 17:18 |
@sonney2k | nicococo, yup | 17:18 |
nicococo | well, the difference is usually dense | 17:19 |
@sonney2k | w <- alpha w*Phi(x) | 17:19 |
nicococo | this means that using coffin in the whole application and then calling a qp solver makes no sense, right? | 17:20 |
@sonney2k | nicococo, these ops http://shogun-toolbox.org/doc/en/current/classshogun_1_1CDotFeatures.html | 17:20 |
@sonney2k | nicococo, not true | 17:20 |
@sonney2k | it makes sense | 17:21 |
nicococo | ? | 17:21 |
@sonney2k | one could use million dim feature spaces | 17:21 |
@sonney2k | on many sequences /inputs | 17:21 |
nicococo | (with qp solver i mean mosek or something where i don't have access to) | 17:21 |
@sonney2k | that fit in memory | 17:21 |
@sonney2k | which wouldn't otherwise | 17:21 |
@sonney2k | you just have to have few iterations to get convergence | 17:21 |
@sonney2k | we have the same problem with ocas | 17:22 |
@sonney2k | (primal CP svm) | 17:22 |
nicococo | if i cannot access the solveer to replace the w'psi ? | 17:22 |
@sonney2k | but usually things converge fast | 17:22 |
@sonney2k | nicococo, yes you explicitly compute psi-psi_hat | 17:22 |
@sonney2k | but you just have these things a few hundred times | 17:23 |
@sonney2k | and so can probably only use million dim spaces | 17:23 |
nicococo | again, for using that trick i need to avoid expand the feature vectors, right or wrong? | 17:24 |
@sonney2k | wrong | 17:24 |
@sonney2k | you can do that | 17:24 |
@sonney2k | just not very often | 17:24 |
@sonney2k | if you have say 100000 sequences | 17:25 |
@sonney2k | you don't need to compute psi for all of them upfront | 17:25 |
nicococo | okay, i see what you mean.. | 17:26 |
@sonney2k | just for your solver | 17:26 |
@sonney2k | so you can squeeze them in memory | 17:26 |
@sonney2k | can work with any kind of input features (strings, sparse, dense vectors) | 17:27 |
nicococo | but for the vanilla linear so svm with say mosek i don't gain anything | 17:27 |
@sonney2k | yes for the solver not | 17:27 |
@sonney2k | but I suspect that won't take most time anyway but argmax etc | 17:27 |
blackburn | oh how many messages.. | 17:27 |
@sonney2k | blackburn, have you been hiding under a rock again? | 17:27 |
nicococo | but when i start writing my own solver based on unconstrained formulations i can possibly gain a lot | 17:28 |
@sonney2k | nicococo, indeed | 17:28 |
blackburn | sonney2k: something like that | 17:28 |
@sonney2k | liblinear etc can directly use that | 17:28 |
@sonney2k | nicococo, ocas also has the same problem | 17:28 |
@sonney2k | we have to store the CP | 17:28 |
* sonney2k has to leave the train | 17:28 | |
@sonney2k | ...soon | 17:28 |
nicococo | a bientot | 17:28 |
@sonney2k | so I guess uricamics | 17:29 |
@sonney2k | stuff will help here too | 17:29 |
@sonney2k | (fewer iterations) | 17:29 |
@sonney2k | and vojtech knows ocas and coffin... | 17:29 |
* sonney2k off | 17:29 | |
n4nd0 | nicococo: so what do you think then? | 17:30 |
n4nd0 | should we define these functions w*Psi(x,y) too? | 17:30 |
nicococo | n4nd0: it is much clearer know :) and still interesting but need more time to think about it therefore i suggest to start the test multiclass application, what do you think? | 17:31 |
n4nd0 | nicococo: agree | 17:32 |
nicococo | should we talk friday again?? | 17:32 |
n4nd0 | nicococo: sure | 17:32 |
n4nd0 | nicococo: I will fix the loss issue and start with the vanilla SO finally | 17:33 |
n4nd0 | luckily we will be able to discuss about the multiclass application on Friday ok? | 17:33 |
nicococo | n4nd0: good idea.. and good work too! | 17:34 |
n4nd0 | thank you! | 17:34 |
n4nd0 | I have to go now running :) | 17:34 |
nicococo | see you on friday (i go biking) | 17:34 |
-!- Francis_Chan1 [~Adium@2001:da8:203:1823:ddae:d847:31d9:bf46] has joined #shogun | 17:34 | |
n4nd0 | running = I'm a rush :P | 17:34 |
nicococo | :) | 17:34 |
n4nd0 | in | 17:34 |
n4nd0 | bye! | 17:34 |
nicococo | bye | 17:34 |
-!- n4nd0 [~nando@n173-p203.kthopen.kth.se] has quit [Quit: leaving] | 17:34 | |
-!- nicococo [~nico@lacedcoffee.ml.tu-berlin.de] has left #shogun [] | 17:34 | |
-!- Francis_Chan [~Adium@58.194.224.120] has quit [Ping timeout: 244 seconds] | 17:37 | |
-!- gsomix [~gsomix@188.168.13.216] has quit [Ping timeout: 245 seconds] | 17:58 | |
-!- Francis_Chan1 [~Adium@2001:da8:203:1823:ddae:d847:31d9:bf46] has quit [Quit: Leaving.] | 18:24 | |
blackburn | sonney2k: I am really dying with removing vec_index! | 18:38 |
-!- blackburn [~qdrgsm@188.122.250.167] has quit [Quit: Leaving.] | 18:48 | |
-!- blackburn [~blackburn@188.122.250.167] has joined #shogun | 18:50 | |
-!- puffin444 [62e3926e@gateway/web/freenode/ip.98.227.146.110] has joined #shogun | 19:08 | |
-!- puffin444 [62e3926e@gateway/web/freenode/ip.98.227.146.110] has quit [Quit: Page closed] | 19:17 | |
-!- Marty28 [~marty@cable-158-181-77-81.cust.telecolumbus.net] has joined #shogun | 19:22 | |
-!- Marty28 [~marty@cable-158-181-77-81.cust.telecolumbus.net] has quit [Quit: Colloquy for iPad - http://colloquy.mobi] | 19:32 | |
-!- blackburn [~blackburn@188.122.250.167] has quit [Quit: Leaving.] | 19:44 | |
-!- blackburn [~qdrgsm@188.122.250.167] has joined #shogun | 19:46 | |
@sonney2k | blackburn, dying is no option! | 20:04 |
@sonney2k | shall I do it or what? | 20:04 |
blackburn | sonney2k: no I'll continue | 20:06 |
blackburn | sonney2k: but I would rather do it sgreferenced data first | 20:06 |
@sonney2k | blackburn, it really looks like piece of cake | 20:08 |
blackburn | sonney2k: removing index? | 20:08 |
@sonney2k | blackburn, give me 10 minutes - it really should be easy | 20:09 |
@sonney2k | yes | 20:09 |
blackburn | hmm it is referenced from sparse features, ascii file, etc | 20:09 |
blackburn | and even in typemaps | 20:09 |
blackburn | I gave up in typemaps | 20:09 |
@sonney2k | whatever let me just do it | 20:10 |
blackburn | sonney2k: it is up to you - I can continue | 20:10 |
@sonney2k | IMHO very little work but we will see | 20:10 |
@sonney2k | woah git pull is taking ages.. | 20:11 |
blackburn | yes it is slow today | 20:11 |
blackburn | sonney2k: have you seen my commit on merging new asp version? | 20:11 |
@sonney2k | yeah | 20:11 |
blackburn | I am afraid it was mistake but no idea whether I should revert it | 20:12 |
blackburn | student of gunnar asked me to merge it | 20:12 |
@sonney2k | no idea - I don't have time to maintain it... | 20:12 |
@sonney2k | vipin? | 20:12 |
blackburn | yes | 20:12 |
@sonney2k | yeah | 20:12 |
@sonney2k | he takes care of these things at least a bit | 20:12 |
@sonney2k | pull finished | 20:12 |
blackburn | I said that he should rather use PR | 20:12 |
* sonney2k removes stuff | 20:13 | |
blackburn | but merged for the first time | 20:13 |
blackburn | sonney2k: wait wait | 20:13 |
blackburn | I can commit my changes | 20:13 |
blackburn | shall I? | 20:13 |
@sonney2k | no | 20:13 |
blackburn | I'll continue with openmp then | 20:22 |
-!- gsomix [~gsomix@37.61.181.74] has joined #shogun | 20:36 | |
@sonney2k | blackburn, I think there is one possible use case for vec_index ... when one uses caching of vectors... | 20:45 |
@sonney2k | together with subsets | 20:46 |
@sonney2k | then one would need the orig index... | 20:46 |
blackburn | sonney2k: yes I saw something like that | 20:46 |
@sonney2k | but crap we should simplify that | 20:46 |
blackburn | sonney2k: I do not think we should focus on that right now | 20:46 |
@sonney2k | and remove caches etc and only when we really need them do better versions... | 20:46 |
blackburn | just stay as it is | 20:46 |
blackburn | sonney2k: how cool compilation is with ccache :D I found out what was wrong 1-2 weeks ago | 20:56 |
blackburn | 'sudo make' breaks everything | 20:57 |
@sonney2k | ? | 20:57 |
blackburn | sonney2k: if I do 'sudo make' ccache is not used | 20:58 |
CIA-113 | shogun: Soeren Sonnenburg master * red95843 / (26 files in 6 dirs): drop vec_index from SGSparseVector - http://git.io/1QhrrQ | 20:59 |
blackburn | hmm so simple | 20:59 |
blackburn | gsomix: so how it is with directors? | 20:59 |
@sonney2k | blackburn, I hope I didn't miss a corner case though | 21:03 |
* sonney2k continues with Labels | 21:04 | |
blackburn | hmm my openmp dotfeatures are seems to be b0rken | 21:04 |
shogun-buildbot | build #239 of nightly_all is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/nightly_all/builds/239 | 21:08 |
blackburn | sonney2k: OctaveInterface.cpp:352: error: 'struct shogun::SGSparseVector<double>' has no member named 'vec_index' | 21:08 |
gsomix | blackburn, nohow. | 21:14 |
shogun-buildbot | build #946 of libshogun is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/libshogun/builds/946 | 21:16 |
shogun-buildbot | build #866 of cmdline_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cmdline_static/builds/866 blamelist: sonne@debian.org | 21:24 |
shogun-buildbot | build #845 of r_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/r_static/builds/845 blamelist: sonne@debian.org | 21:29 |
shogun-buildbot | build #755 of octave_static is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/octave_static/builds/755 blamelist: sonne@debian.org | 21:34 |
shogun-buildbot | build #832 of python_static is complete: Failure [failed test_1] Build details are at http://www.shogun-toolbox.org/buildbot/builders/python_static/builds/832 blamelist: sonne@debian.org | 21:38 |
blackburn | nice | 21:39 |
CIA-113 | shogun: Soeren Sonnenburg master * r3db552b / examples/undocumented/libshogun/features_copy_subset_sparse_features.cpp : just use i for vector index in examples - http://git.io/9TIR0w | 21:40 |
-!- shogun-buildbot [~shogun-bu@7nn.de] has quit [Ping timeout: 244 seconds] | 21:45 | |
-!- shogun-buildbot [~shogun-bu@7nn.de] has joined #shogun | 21:45 | |
shogun-buildbot | build #833 of python_static is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/python_static/builds/833 | 21:56 |
shogun-buildbot | build #867 of cmdline_static is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cmdline_static/builds/867 | 22:01 |
shogun-buildbot | build #846 of r_static is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/r_static/builds/846 | 22:07 |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has joined #shogun | 22:47 | |
-!- wiking [~wiking@c-98-231-171-175.hsd1.md.comcast.net] has quit [Changing host] | 22:47 | |
-!- wiking [~wiking@huwico/staff/wiking] has joined #shogun | 22:47 | |
wiking | morning :D | 22:48 |
blackburn | O_O | 22:48 |
blackburn | ah right use | 22:48 |
blackburn | usa :D | 22:48 |
wiking | :P | 23:05 |
blackburn | wiking: what is conf? | 23:53 |
CIA-113 | shogun: Sergey Lisitsyn master * r7e00d0e / (src/shogun/features/Labels.cpp src/shogun/features/Labels.h): Added get_binary_for_class method that presents multiclass labels as binary subtask - http://git.io/4n-DYA | 23:59 |
--- Log closed Thu May 17 00:00:40 2012 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!