--- Log opened Fri Apr 22 00:00:36 2011 | ||
serialhex_ | evening all... how is everyone (whos awake :D ) | 02:34 |
---|---|---|
-!- serialhex_ is now known as serialhex | 02:37 | |
serialhex | hmm... i guess everyone is afk or asleep | 02:41 |
* serialhex thinks it sucks that when he's awake everyone else is asleep, or vice versa | 02:41 | |
-!- sploving [~sploving@124.16.139.196] has left #shogun [] | 02:56 | |
-!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Quit: Page closed] | 03:00 | |
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun | 08:23 | |
-!- confused [8092a059@gateway/web/freenode/ip.128.146.160.89] has joined #shogun | 08:40 | |
confused | hello | 08:41 |
confused | need some help with linearstring kernel .. can anyone help? | 08:42 |
-!- confused [8092a059@gateway/web/freenode/ip.128.146.160.89] has left #shogun [] | 08:45 | |
-!- akhil_ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 09:13 | |
-!- sploving [~sploving@124.16.139.196] has joined #shogun | 09:43 | |
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 260 seconds] | 10:08 | |
-!- akhil_ [75d35896@gateway/web/freenode/ip.117.211.88.150] has quit [Quit: Page closed] | 10:11 | |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 10:54 | |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has left #shogun [] | 11:51 | |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 12:42 | |
@sonney2k | hmmhh just a little more than 8hrs until the deduplication meeting http://www.timeanddate.com/counters/customcounter.html?msg=Gsoc+2011+-+End+of+reviews%2C+deduplication+meeting+on+IRC&day=22&month=04&year=2011&hour=19&min=00&sec=00&p0=0 | 12:43 |
sploving | yeap. In our coutry, we are sleeping. | 12:49 |
sploving | Are there many students applied for more than one org? | 12:51 |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has quit [Quit: Page closed] | 13:29 | |
-!- emrecelikten [c18cf902@gateway/web/freenode/ip.193.140.249.2] has joined #shogun | 13:37 | |
josip | sonney2k: we still won't know until Monday :/ | 13:53 |
-!- emrecelikten [c18cf902@gateway/web/freenode/ip.193.140.249.2] has quit [Quit: Page closed] | 14:16 | |
@sonney2k | sploving, in our case so far only you told us. | 14:19 |
-!- blackburn [~qdrgsm@188.168.2.13] has joined #shogun | 14:21 | |
@sonney2k | anyway I hope there won't be any surprises after the meeting | 14:25 |
blackburn | hello | 14:26 |
@bettyboo | hey | 14:26 |
blackburn | what I missed? which surprises? ;) | 14:26 |
josip | surprise = someone got into 10 projects ? :)) | 14:46 |
josip | that's a very nice surprise .. for the student at least :) | 14:46 |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 14:55 | |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has quit [Quit: Page closed] | 15:01 | |
blackburn | sonney2k: need some help | 15:13 |
-!- dvevre [~in3xes@27.107.145.70] has joined #shogun | 16:38 | |
-!- dvevre [~in3xes@27.107.145.70] has quit [Quit: Ooops..!!] | 16:58 | |
-!- lionelc_ [4c681efd@gateway/web/freenode/ip.76.104.30.253] has joined #shogun | 17:10 | |
-!- sploving_1 [~sploving@124.16.139.196] has joined #shogun | 17:48 | |
-!- sploving [~sploving@124.16.139.196] has quit [Ping timeout: 240 seconds] | 17:50 | |
-!- sploving [~sploving@124.16.139.196] has joined #shogun | 18:18 | |
-!- sploving_1 [~sploving@124.16.139.196] has quit [Ping timeout: 252 seconds] | 18:20 | |
-!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 18:54 | |
-!- dvevre_ [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 19:05 | |
-!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Ping timeout: 252 seconds] | 19:07 | |
-!- dvevre_ is now known as dvevre | 19:07 | |
@sonney2k | blackburn, yes? | 20:06 |
blackburn | sonney2k: :) I think it is not necessary now | 20:06 |
josip | <3 | 20:06 |
josip | fuck, wrong window. | 20:06 |
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun | 20:06 | |
josip | lol. | 20:06 |
blackburn | we heart you too | 20:07 |
blackburn | sonney2k: hm.. but may be you can help me to grasp one thing in ROC | 20:07 |
@sonney2k | blackburn, what was the problem (though I am loving it when questions resolve themselves) | 20:08 |
blackburn | sonney2k: the problem was my not_understanding how to return matrix in modular, but it is solved now | 20:08 |
@sonney2k | blackburn, great! | 20:08 |
@sonney2k | so how can I help about ROC? | 20:09 |
blackburn | just looked to typemaps and now understand | 20:09 |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 20:09 | |
blackburn | in that algo (do you recall it?) there is if f_prev!=f of i-th example | 20:09 |
blackburn | so if there will be equal answers ROC will be 'shorter' | 20:09 |
josip | blackburn: it has as many points as there are unique thresholds + 1 | 20:10 |
josip | in my understanding | 20:10 |
blackburn | josip: yes but why it should? | 20:11 |
blackburn | and the problem for me is how to determine it earlier | 20:11 |
josip | count the number of unique thresholds | 20:11 |
@sonney2k | err, it should have the same number of points as you have predictions | 20:12 |
blackburn | some 'overhead' for counting it | 20:12 |
josip | sonney2k: then you'll copies of the same point | 20:12 |
josip | assume all of the points have the same likelihood. Then there is only one fp rate and only one tp rate | 20:13 |
@sonney2k | josip, if you used the underestimate/overestimated ROC then yes but for expect ROC curve it is ok | 20:13 |
* sonney2k is looking for the roc paper link | 20:13 | |
blackburn | sonney2k: so may be I should remove that check? | 20:13 |
josip | blackburn: it depends on how you document it, the graph will look the asme and the integral will be the same | 20:14 |
josip | either way | 20:14 |
blackburn | josip: I see, thank you | 20:15 |
josip | but I'm not the ultimate authority, so I might be mistaken | 20:15 |
@sonney2k | josip, page 14 on http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf | 20:16 |
josip | yes, so the algorithm cramps all of them in one point | 20:17 |
josip | lemme recheck it | 20:17 |
@sonney2k | josip, when you have multiple points with the same output then you can get into this problem - so you either have a jump to the right or use the expected curve | 20:17 |
@sonney2k | then it is not just one point | 20:17 |
@sonney2k | but a diagonal | 20:17 |
blackburn | hehe how silly I am.. it is described in fawcett's paper | 20:18 |
@sonney2k | blackburn, yes he is the roc guru (gave tutorials etc on this) | 20:18 |
blackburn | sonney2k: so that check is necessary, right? | 20:18 |
@sonney2k | blackburn, he also has this nice algorithm (algorithm 2 in that paper) | 20:19 |
josip | but assume that you plot it by varying the threshold and not from the sample which you will do in "production". Then you will classify all points with the same threshold either +1 or -1 | 20:19 |
@sonney2k | not necessary no | 20:19 |
blackburn | I'm so ashamed with so late acknowledgement with ROC :) | 20:19 |
@bettyboo | funny | 20:19 |
josip | blackburn: they're mostly used in medical stuff, because the classes are very disproportionate (i.e. 0.1% chance of having breast cancer) | 20:19 |
josip | s/in/for | 20:19 |
@sonney2k | josip, yes when you apply in practise you give a point either +1 or -1 | 20:19 |
josip | blackburn: and a classifier that just classifiers everyone as not sick, it will have a 99.9% accuracy on an iid sample from the distribution | 20:20 |
blackburn | josip: I see | 20:21 |
@sonney2k | I can only recommend to always use ROC curves when comparing methods | 20:21 |
blackburn | thank you, it became clearer to me | 20:21 |
@sonney2k | accuracy etc is a useless measure (if that is not exactly what counts latero on) | 20:22 |
josip | sonney2k: but most people do use only accuracy | 20:22 |
@sonney2k | note that ROC curves are still not optimal when you have *really* unbalanced data sets | 20:22 |
@sonney2k | then you shoudl use precision recall curves | 20:23 |
blackburn | PRC is my next work :) | 20:23 |
@sonney2k | josip, that is bad. | 20:23 |
@sonney2k | josip, the user should decide which fp-rate he can afford or how many tp's he needs. | 20:24 |
josip | yup | 20:24 |
josip | I agree | 20:25 |
josip | a single number (+ a CI maybe ..) is just not enough, unless classes are evenly distributed | 20:25 |
* serialhex wishes he could matrix-style download some ML textbooks into his head :P | 20:26 | |
josip | I got confused for a moment :) You mean The Matrix-style :D | 20:27 |
serialhex | yeah, that too | 20:27 |
@sonney2k | blackburn, reading fawcetts description about the algorithm - he indeed suggests to drop points with same output and to draw a connecting line instead | 20:28 |
@sonney2k | blackburn, so indeed output can be shorter | 20:28 |
* serialhex will bbl, to learn some more... | 20:29 | |
blackburn | sonney2k: will count different outputs for that | 20:29 |
@sonney2k | blackburn, however I would just allocate an array #outputs | 20:29 |
@sonney2k | doesn't matter | 20:29 |
@sonney2k | I mean you can allocate a slightly bigger array and then just return the shortened size | 20:29 |
blackburn | hmm.. I can, but is it a good practice..? | 20:30 |
@sonney2k | it rarely happens that you have 1e6 outputs that are all the same and so you only have (0,0) and (1,1) in the end | 20:30 |
@sonney2k | I don't see a problem... it is wasting memory but that's all (no leaks no other side effects) | 20:31 |
blackburn | okay, just didn't know how you consider it ;) | 20:31 |
blackburn | thank you, sonney2k, josip | 20:32 |
blackburn | have to go now, will test that ROCEvaluation a bit later and will make a pull request | 20:33 |
blackburn | some wasting sounds better for me too, because checking will be another one O(n) | 20:34 |
@sonney2k | josip, what I don't know though is how one treats duplicate outputs in PR-Curves | 20:37 |
@sonney2k | I only know that each point in ROC space can be translated to a point in PR space | 20:37 |
josip | well the graph will look the same and also the integral | 20:38 |
josip | which is what you care about, right? | 20:39 |
@sonney2k | josip, not true unfortunately | 20:40 |
@sonney2k | if you don't skip points with same output you run into problems | 20:40 |
josip | that's true, I thought you meant something different | 20:41 |
josip | I gtg, ttyl | 20:42 |
-!- sonney2k is now known as shogun|sonney2k | 20:42 | |
* shogun|sonney2k prepares for the dedup meeting | 20:43 | |
@shogun|sonney2k | bye josip | 20:44 |
@shogun|sonney2k | it is a crazy amount of messages per minute in #gsoc | 20:46 |
-!- warpy [~warpy@bzq-79-180-56-86.red.bezeqint.net] has joined #shogun | 20:59 | |
warpy | anyone here developed a typemap before ? | 20:59 |
@shogun|sonney2k | sure | 21:00 |
@shogun|sonney2k | 627 people in #gsoc - just crazy | 21:03 |
warpy | oh hey there, didnt see you | 21:04 |
warpy | long nick name | 21:04 |
@shogun|sonney2k | great isn't it | 21:05 |
@shogun|sonney2k | required in the dedup thing going on in #gsoc | 21:05 |
warpy | shogun|sonney2k, someone i work with is trying to figure out how to do a c# port | 21:05 |
warpy | i told him i'd ask a few questions, many things are still very much a blur to us | 21:05 |
@shogun|sonney2k | ask ... but don't expect me to answer immediately - I have to check if we get any dupes due to the reshuffling going on in #gsoc | 21:07 |
warpy | would you mind explaining the core difference between the static and dynamic interfaces | 21:07 |
@shogun|sonney2k | warpy, one is swig based and each C++ object will become an object in $LANGUAGE | 21:08 |
@shogun|sonney2k | static is just some global variables with a few objects assigned. | 21:08 |
warpy | a c++ object is (a function, a class?)? | 21:09 |
@shogun|sonney2k | class | 21:09 |
@shogun|sonney2k | + associated functions | 21:09 |
warpy | and $LANGUAGE is ? | 21:10 |
@shogun|sonney2k | what you want C# or python or... | 21:10 |
warpy | so basically every c++ object will become another object in the designated language | 21:10 |
@shogun|sonney2k | yes | 21:11 |
warpy | does it mean that we have to create the code of the $language's object that deals with the c++ object? | 21:12 |
warpy | for each and every one ? | 21:12 |
@shogun|sonney2k | no | 21:12 |
@shogun|sonney2k | swig does that for you | 21:12 |
warpy | can you explain a little on how, i am guessing it doesnt convert c++ to c# | 21:13 |
@shogun|sonney2k | warpy, no I cannot - check the swig homepage... | 21:15 |
-!- shogun|sonney2k is now known as sonney2k | 21:16 | |
@sonney2k | dedup meeting is over | 21:17 |
@sonney2k | hurray | 21:17 |
warpy | regarding static interfaces. how do they work in concept. so you have global variables and a few classes. do you need a wrapper class to interface with a c++ class ? | 21:17 |
warpy | maybe we should take some class as an example | 21:18 |
@sonney2k | warpy, we have set/get* interface functions for all kinds of things (scalars, vectors, matrices,...) | 21:18 |
@sonney2k | and each $lang has to implement these | 21:18 |
warpy | okay so both interface types static and dynamic need to create new classes to deal with all the input types ? | 21:19 |
@sonney2k | global variables are then for e.g. classifiers - so one can only have one classifier | 21:19 |
@sonney2k | no | 21:19 |
@sonney2k | in static interfaces you need these set/get functions (that we have defined in our SGInterface) | 21:20 |
@sonney2k | in the modular ones you write typemaps | 21:20 |
warpy | so in the static one you would write your set get functions in the designated language, this is actually a wrapper for the c++ functions, correct ? | 21:21 |
@sonney2k | warpy, no you write them in C++ | 21:22 |
@sonney2k | and utilize you native interface (e.g. in java JNI, in python c extensions etc) | 21:22 |
-!- akhil__ [75d35896@gateway/web/freenode/ip.117.211.88.150] has quit [Quit: Page closed] | 21:23 | |
warpy | so the static interface is always written in c++ but uses libraries from the designated language ? | 21:24 |
@sonney2k | warpy, yes - but the typemaps are also C or C++ and utilize the libraries | 21:24 |
@sonney2k | so in that respect it is similar though modular is more powerful as you really have access to all C++ objects in your $lang | 21:25 |
warpy | okay so the main difference between static and dynamic is that static is limited to functions and classes that you yourself predefined and dynamic by using swig has access to everything | 21:26 |
@sonney2k | warpy, yes | 21:26 |
warpy | awesome, regarding the type map itself. the whole development process. | 21:28 |
warpy | basically what i would need is to load the project in a c++ ide and first be able to compile it | 21:29 |
warpy | so far so good ? | 21:29 |
@sonney2k | yes | 21:31 |
warpy | is it possible to use vc++, or any windows based ide ? | 21:31 |
@sonney2k | I've only ever compiled it onder posix platforms + cygwin | 21:31 |
@sonney2k | warpy, I suspect that will not work out of the box | 21:31 |
@sonney2k | we use posix threads and signals for example | 21:32 |
warpy | sorry, posix ? | 21:32 |
@sonney2k | http://en.wikipedia.org/wiki/POSIX | 21:32 |
warpy | cygwin i have used before | 21:32 |
warpy | okay so that the only way you have for windows so far | 21:33 |
warpy | so lets say it compiles under windows using cygwin etc | 21:33 |
warpy | i believe you have a tutorial on your site for windows compiling, correct ? | 21:34 |
@sonney2k | warpy, just get all the dependencies... and compile | 21:35 |
warpy | alright | 21:36 |
warpy | if you remember, you prepared shogun to support c#. how does a typemap point to that. is it somekind of directive inside the code ? | 21:37 |
@sonney2k | warpy, you need to write the file csharp/swig_typemaps.i | 21:41 |
@sonney2k | warpy, sploving is the expert to ask here - he actually contributed to swig | 21:41 |
blackburn | sonney2k: have you had some activity with what dedup? | 21:45 |
@sonney2k | blackburn, no dupes... | 21:45 |
blackburn | hehe | 21:45 |
blackburn | sonney2k: forgot one awful thing.. I have to know size of matrix because it is 2d | 21:49 |
@sonney2k | blackburn, argh... you are right | 21:50 |
blackburn | seems I have to count num of different labels | 21:50 |
warpy | okay so lets imagine we wrote the typemap and recompiled shogun and everything works | 21:53 |
warpy | now for the interesting part | 21:53 |
warpy | how do i use it with in c# (vc# for example) | 21:54 |
warpy | is it a matter of doing "using shogun;" | 21:55 |
warpy | sonney2k, still there? | 22:17 |
@sonney2k | warpy, yes - no idea. never used csharp berfore | 22:17 |
blackburn | such irritated with not having much time for finish issues with evaluation! | 22:18 |
warpy | what it would be like in other languages ? | 22:19 |
blackburn | sonney2k: how do you use valgrind when running tests? | 22:31 |
blackburn | I mean when I ran it, there are many 'pythonic' errors, and I don't see any useful information | 22:31 |
@sonney2k | warpy, you really should look at the examples http://www.shogun-toolbox.org/doc/examples.html | 22:31 |
@sonney2k | warpy, it is intuitive to say the least | 22:32 |
@sonney2k | blackburn, you either use a reasonable suppressions file or you dump everything into a file and grep for shogun | 22:32 |
@sonney2k | (later on) | 22:32 |
blackburn | ah | 22:32 |
blackburn | thank you | 22:32 |
blackburn | *getting mad cause some memory allocation error* | 22:33 |
blackburn | rghhh | 22:33 |
blackburn | oh | 22:34 |
blackburn | I'm such a fool :D | 22:34 |
warpy | okay looking thanks | 22:35 |
blackburn | sonney2k: we both failed! I'm _not_ have to know the size of matrix if I will return first N columns.. | 22:42 |
@sonney2k | blackburn, ahh you are right it is column first so UI#H@OI#UH@!!!! | 22:43 |
* sonney2k 's brain is off | 22:43 | |
blackburn | :D was? | 22:43 |
blackburn | sonney2k: you better kick me off cause I'm stupid :D | 22:44 |
blackburn | sonney2k: yesterday (for you today) was the day shogun's github received no commits | 22:47 |
@sonney2k | blackburn, hurray :) | 22:48 |
blackburn | sonney2k: is it? ;) do you like when we arent commiting? | 22:51 |
blackburn | oh, I finally find the error | 22:54 |
@sonney2k | blackburn, well we had a screw up commit on that day and I had to force push ... so better no push/pull than such ones! | 22:55 |
blackburn | hahhaha | 22:55 |
blackburn | sonney2k: what is the commit? | 22:56 |
@sonney2k | you won't see it :) | 22:56 |
blackburn | sonney2k: what it screwed? | 22:56 |
@sonney2k | and I am too ashamed to disclose what we did :) | 22:56 |
blackburn | hmm | 22:56 |
blackburn | I already had a proper ROC, but now it returned: | 22:57 |
blackburn | http://img823.imageshack.us/i/hehee.png/ | 22:57 |
blackburn | sonney2k: seems to be a strange ROC :D | 22:57 |
@sonney2k | blackburn, modern art | 22:59 |
blackburn | yeah | 22:59 |
@sonney2k | blackburn, at least it is between 0 and 1 | 22:59 |
blackburn | hehe, I screwed something when doing some 'refactoring' | 23:00 |
serialhex | blackburn: wtf is that image _supposed_ to be??? :P | 23:07 |
blackburn | serialhex: ROC curve :) | 23:07 |
blackburn | serialhex: may be like http://publications.csail.mit.edu/abstracts/abstracts07/patrycja/roc4.jpg | 23:07 |
serialhex | http://en.wikipedia.org/wiki/Receiver_operating_characteristic is this what your talking about??? | 23:08 |
blackburn | yeap | 23:08 |
serialhex | ahh, ok cool | 23:08 |
* serialhex goes and reads about roc's, and not this kind: http://en.wikipedia.org/wiki/Roc_(mythology) | 23:09 | |
@sonney2k | serialhex, cool bird :D | 23:11 |
serialhex | yes i know | 23:11 |
blackburn | someone is stealing my ROC | 23:11 |
* serialhex has played too much D&D | 23:11 | |
blackburn | serialhex: is it you? | 23:12 |
serialhex | i'm not stealing it :D | 23:12 |
blackburn | serialhex: I'm watching! | 23:12 |
serialhex | i'm sure you are | 23:12 |
blackburn | it draws some fantastic things :D | 23:13 |
serialhex | really? such as??? | 23:14 |
* serialhex waits to see some fantastic drawings | 23:14 | |
blackburn | serialhex: http://img64.imageshack.us/f/hehek.png/ | 23:15 |
blackburn | the only thing i'm pretty sure: it isn't ROC, it is some memory 'slice' :D | 23:15 |
serialhex | yeah... you can start a new form of art with it: abstract graph art :D | 23:15 |
* serialhex loves wikipedia | 23:29 | |
* sonney2k yawns | 23:39 | |
@sonney2k | blackburn, btw wikipedia lists the other measures we have on the ROC page too http://en.wikipedia.org/wiki/Receiver_operating_characteristic | 23:43 |
dvevre | sonney2k: are the irc logs online somewhere? | 23:44 |
@sonney2k | dvevre, not yet | 23:44 |
@sonney2k | didn't find the time... | 23:44 |
dvevre | hmm.. okay.. | 23:45 |
serialhex | dvevre: if i can figure out how to extract them from quassel i have a pretty complete log of the chat | 23:45 |
* serialhex has logs since ~4 days after the channel opened | 23:45 | |
dvevre | serialhex: you never disconnect? nice! | 23:45 |
serialhex | dvevre: yeah i almost never turn my computer off | 23:45 |
@bettyboo | nite | 23:46 |
dvevre | serialhex: i'm mostly stuck with the webchat version here.. and that too on a lab computer | 23:47 |
serialhex | :-/ that kind of sucks dvevre! | 23:47 |
dvevre | hence have to miss out on much of the conversation.. | 23:47 |
blackburn | sonney2k: eh.. they all are already implemented | 23:53 |
blackburn | except MCC | 23:53 |
@sonney2k | blackburn, I think that one too - we just named it cross_correlation_coefficient or so | 23:54 |
blackburn | hehe | 23:54 |
blackburn | right | 23:54 |
@sonney2k | I just meant to say - nice table :) | 23:54 |
blackburn | see now | 23:54 |
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 252 seconds] | 23:54 | |
--- Log closed Sat Apr 23 00:00:36 2011 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!