| --- Log opened Sat Apr 23 00:00:36 2011 | ||
| @sonney2k | anyway - need to sleep | 00:12 |
|---|---|---|
| @sonney2k | cu all | 00:12 |
| blackburn | see you | 00:14 |
| serialhex | later | 00:15 |
| -!- lionelc_ [4c681efd@gateway/web/freenode/ip.76.104.30.253] has quit [Quit: Page closed] | 00:25 | |
| blackburn | going to sleep too | 00:25 |
| blackburn | later | 00:25 |
| -!- blackburn [~qdrgsm@188.168.2.13] has quit [Quit: Leaving.] | 00:25 | |
| -!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun | 00:33 | |
| @sonney2k | ok I lied - we have irclogs now http://shogun-toolbox.org/irclogs/ | 00:41 |
| @sonney2k | dvevre, ^^ | 00:41 |
| @sonney2k | enjoy and now really good night | 00:41 |
| dvevre | sonney2k: awesome! | 00:41 |
| dvevre | :) | 00:41 |
| dvevre | and good night! | 00:41 |
| sploving | good night | 00:43 |
| serialhex | YAY irc logs!!! | 01:09 |
| serialhex | now i dont have to slave over learning SQL for a little while longer :D | 01:09 |
| -!- sploving [~sploving@124.16.139.196] has quit [] | 01:24 | |
| -!- dave718 [48e50367@gateway/web/freenode/ip.72.229.3.103] has joined #shogun | 01:30 | |
| dave718 | Anyone have an example of working python modular code for RealFileFeatures? I've tried building the binary file by hand and adding the data to RealFileFeatures() without args but no luck. | 01:31 |
| dave718 | i.e. I've tried both (making the binary file, and building it up within python). | 01:32 |
| -!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 240 seconds] | 02:14 | |
| -!- dave718 [48e50367@gateway/web/freenode/ip.72.229.3.103] has quit [Ping timeout: 252 seconds] | 03:37 | |
| -!- vivekp [~vivekp@14.195.118.49] has joined #shogun | 03:46 | |
| -!- vivekp [~vivekp@14.195.118.49] has quit [Read error: Connection reset by peer] | 04:09 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Quit: Page closed] | 05:31 | |
| -!- vivekp [~vivekp@180.149.49.229] has joined #shogun | 05:49 | |
| -!- vivekp [~vivekp@180.149.49.229] has quit [Ping timeout: 240 seconds] | 06:04 | |
| -!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun | 06:19 | |
| -!- vivekp [~vivekp@14.195.111.99] has joined #shogun | 06:20 | |
| -!- siddharth [~siddharth@117.211.88.150] has joined #shogun | 06:41 | |
| -!- vivekp [~vivekp@14.195.111.99] has quit [Read error: Connection reset by peer] | 07:24 | |
| -!- akhil_ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 07:51 | |
| -!- akhil_ [75d35896@gateway/web/freenode/ip.117.211.88.150] has quit [Ping timeout: 252 seconds] | 08:06 | |
| -!- sploving [sploving@210.77.26.83] has joined #shogun | 09:27 | |
| sploving | hello sonney2k | 09:29 |
| -!- vivekp [~vivekp@180.149.49.229] has joined #shogun | 09:43 | |
| -!- vivekp [~vivekp@180.149.49.229] has quit [Quit: Leaving] | 09:52 | |
| -!- siddharth [~siddharth@117.211.88.150] has quit [Ping timeout: 240 seconds] | 10:12 | |
| -!- sploving [sploving@210.77.26.83] has quit [] | 10:34 | |
| -!- siddharth [~siddharth@117.211.88.150] has joined #shogun | 10:52 | |
| siddharth | hi all | 10:52 |
| -!- warpyy [~warpy@bzq-79-180-56-86.red.bezeqint.net] has joined #shogun | 11:30 | |
| -!- warpyy [~warpy@bzq-79-180-56-86.red.bezeqint.net] has quit [Quit: The computer fell asleep] | 11:48 | |
| -!- blackburn [~qdrgsm@188.168.4.237] has joined #shogun | 12:14 | |
| -!- sploving [sploving@210.77.26.83] has joined #shogun | 12:22 | |
| sploving | hello sonney2k, are you here? | 12:22 |
| -!- siddharth [~siddharth@117.211.88.150] has quit [Ping timeout: 252 seconds] | 12:33 | |
| -!- vivekp [~vivekp@14.195.125.35] has joined #shogun | 12:48 | |
| -!- siddharth [~siddharth@117.211.88.150] has joined #shogun | 12:48 | |
| -!- blackburn [~qdrgsm@188.168.4.237] has quit [Ping timeout: 276 seconds] | 12:49 | |
| -!- sploving [sploving@210.77.26.83] has quit [] | 12:55 | |
| -!- vivekp [~vivekp@14.195.125.35] has quit [Quit: going to class !] | 13:01 | |
| -!- warpy [~warpy@bzq-79-180-56-86.red.bezeqint.net] has quit [Quit: HydraIRC -> http://www.hydrairc.com <- Po-ta-to, boil em, mash em, stick em in a stew.] | 13:17 | |
| -!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 240 seconds] | 13:25 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 14:02 | |
| -!- bettyboo [~bettyboo@bane.ml.tu-berlin.de] has quit [Remote host closed the connection] | 14:31 | |
| -!- bettyboo [~bettyboo@bane.ml.tu-berlin.de] has joined #shogun | 14:32 | |
| -!- mode/#shogun [+o bettyboo] by ChanServ | 14:32 | |
| @mlsec | great you are back, betty | 14:32 |
| @bettyboo | mlsec: I sent an email quite a while back, given that the code had more to do with my potential project than with anything else in shogun | 14:32 |
| @mlsec | okay | 14:32 |
| -!- siddharth [~siddharth@117.211.88.150] has quit [Ping timeout: 260 seconds] | 14:53 | |
| -!- abc [53173767@gateway/web/freenode/ip.83.23.55.103] has joined #shogun | 15:08 | |
| -!- abc is now known as Guest22040 | 15:08 | |
| Guest22040 | Hello | 15:10 |
| Guest22040 | Is there anyone online who knows a bit of Shogun? ;) | 15:11 |
| -!- sploving [~sploving@210.77.26.83] has joined #shogun | 15:14 | |
| Guest22040 | sploving: Hi. Do you have any experience with Shogun? | 15:22 |
| josip | Guest22040: don't ask to ask - just ask. There might be people around that can help you | 15:26 |
| sploving | Guest22040, yeap. | 15:28 |
| Guest22040 | It is written that Parallelized Code and k-means algorithm are supported by Shogun. Do you know if I can use Shogun to process >6GB datasets and to parallelize the computation process? | 15:30 |
| sploving | I am not a expert about it. I am familiar with the modular typemap | 15:32 |
| josip | you need to fit >6GBs in memory | 15:34 |
| josip | given that they're not sparse | 15:35 |
| josip | if I'm not mistaken | 15:36 |
| josip | you might also want to look at #hadoop if you have really massive datasets | 15:36 |
| josip | well, it will start swapping out otherwise and it will probably make it much slower | 15:38 |
| josip | but you should better wait until someone more knowledable comes | 15:38 |
| josip | :) | 15:38 |
| Guest22040 | I know about the hadoop but looking for sth where I do not have to install the hdfs | 15:38 |
| Guest22040 | the dataset could be 4GB but could be also 6, 8, 20 GB | 15:39 |
| josip | well, if you can fit it in memory it should work I think - but might be very slow if a lot of it is swapped out | 15:39 |
| Guest22040 | ok | 15:39 |
| josip | try it on a small subset first | 15:39 |
| Guest22040 | what about the complexity | 15:39 |
| Guest22040 | Are clustering algorithms parallelized? | 15:40 |
| josip | in general? K-means can be parallelized | 15:40 |
| Guest22040 | I know. Is it? :P | 15:41 |
| Guest22040 | in Shogun? Do you know maybe? ;) | 15:41 |
| josip | http://permalink.gmane.org/gmane.comp.ai.machine-learning.shogun/1521 not yet I guess | 15:42 |
| josip | or rather not yet ~6 months ago | 15:42 |
| Guest22040 | ok | 15:43 |
| Guest22040 | thx | 15:43 |
| josip | you should wait for sonney2k tho | 15:44 |
| -!- sploving [~sploving@210.77.26.83] has quit [] | 15:45 | |
| josip | Guest22040: hadoop is to troublesome to install? | 15:49 |
| Guest22040 | no but do not have access to such cluster | 15:50 |
| Guest22040 | I assume no ;) | 15:50 |
| josip | anyway http://users.eecs.northwestern.edu/~wkliao/Kmeans/index.html | 15:50 |
| Guest22040 | I haven't tries | 15:50 |
| Guest22040 | d | 15:50 |
| josip | there's even a link to a CUDA implementation if you have an nvidia card | 15:51 |
| -!- serialhex [~quassel@99-101-149-136.lightspeed.wepbfl.sbcglobal.net] has quit [Remote host closed the connection] | 15:51 | |
| Guest22040 | ok thx | 15:52 |
| josip | np | 15:52 |
| -!- Guest22040 [53173767@gateway/web/freenode/ip.83.23.55.103] has left #shogun [] | 16:20 | |
| -!- sploving [sploving@210.77.26.83] has joined #shogun | 16:31 | |
| -!- blackburn [~qdrgsm@109.226.117.183] has joined #shogun | 16:44 | |
| -!- sploving [sploving@210.77.26.83] has quit [] | 16:46 | |
| @mlsec | Hiho | 17:50 |
| -!- akhil_ [75d35896@gateway/web/freenode/ip.117.211.88.150] has joined #shogun | 17:54 | |
| -!- dvevre_ [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 17:56 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Quit: Page closed] | 17:57 | |
| -!- dvevre_ is now known as dvevre | 17:59 | |
| -!- vetoc [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 18:07 | |
| vetoc | Hi dvevre :) | 18:07 |
| -!- akshayb [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 18:08 | |
| vetoc | . | 18:09 |
| -!- vetoc [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Client Quit] | 18:09 | |
| akshayb | blackburn chutiya hai | 18:10 |
| akshayb | maaf karna dvevre lode hai! | 18:10 |
| -!- vetoc [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 18:11 | |
| -!- akshayb [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Quit: Page closed] | 18:13 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Quit: Page closed] | 18:14 | |
| -!- vetoc [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Client Quit] | 18:15 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 18:22 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has left #shogun [] | 18:22 | |
| blackburn | WTF | 18:30 |
| @sonney2k | blackburn, I heard my name? | 19:38 |
| blackburn | sonney2k: ehh? | 19:38 |
| @sonney2k | WTF? | 19:39 |
| blackburn | (08:10:05 PM) akshayb: blackburn chutiya hai | 19:39 |
| blackburn | (08:10:48 PM) akshayb: maaf karna dvevre lode hai! | 19:39 |
| blackburn | it's all about this :D | 19:39 |
| @bettyboo | strange | 19:39 |
| @sonney2k | blackburn, not a language you understand? | 19:39 |
| blackburn | yeah ;) | 19:39 |
| blackburn | even don't know what it is, hindu? | 19:40 |
| blackburn | sonney2k: how it is going? | 19:41 |
| @sonney2k | blackburn, live is a mess ... was weeding in the garden (and everyone except myself is sick here). | 19:43 |
| blackburn | sick? why? I heard it is warm in Deutschland | 19:43 |
| blackburn | damn segfault! | 19:45 |
| @sonney2k | yes it is very nice weather... no idea why just now. | 19:47 |
| @sonney2k | but 40 C fever is no fun... | 19:47 |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 19:48 | |
| blackburn | 40? I hope all there will recover fast and you will not sicken | 19:49 |
| blackburn | sonney2k: now looks like ROC? ;) http://img808.imageshack.us/f/hehegd.png/ | 19:56 |
| @sonney2k | blackburn, why are there so many steps in there? | 19:57 |
| @sonney2k | doesn't look correct to me (more like a overestimated ROC curve) | 19:57 |
| blackburn | hm.. | 19:57 |
| blackburn | sonney2k: I randomly placed +1 where -1 was and vice versa | 19:58 |
| blackburn | in labels | 19:58 |
| blackburn | does it depend on this? | 19:58 |
| @sonney2k | in the predicted labels or the true ones? | 20:00 |
| blackburn | in true ones | 20:00 |
| @sonney2k | blackburn, I would start with the following labels: | 20:01 |
| @sonney2k | -1 +1 for true ones | 20:01 |
| @sonney2k | and outputs +1 +1 | 20:01 |
| blackburn | sonney2k: btw it is LDA for modified label_train_twoclass.dat | 20:02 |
| @sonney2k | blackburn, just don't use any classifier at all for the test | 20:02 |
| @sonney2k | but only manually set labels | 20:03 |
| blackburn | eh.. sonney2k, is it a good example? | 20:05 |
| blackburn | in that case we have only one point | 20:05 |
| @sonney2k | it should be a diagonal line | 20:06 |
| blackburn | oh, sorry, 2 | 20:06 |
| @sonney2k | from 0,0 to 1,1 | 20:06 |
| blackburn | rgh! found bug | 20:06 |
| blackburn | ROC [[ NaN 0.] | 20:06 |
| blackburn | [ 1. 1.]] | 20:06 |
| blackburn | sonney2k: yeap, it is | 20:08 |
| blackburn | sonney2k: can ROC be lower than diagonal..? | 20:09 |
| blackburn | i tested it on (true: -1 1 1) (predicted: 1 1 -1) | 20:10 |
| blackburn | and the points are (0,0) (1,0.5) (1,1) | 20:11 |
| josip | sonney2k: someone asked if there is a parallel implementation of k-means in Shogun. is it implemented as of now>? | 20:11 |
| blackburn | found mistake | 20:11 |
| blackburn | josip: iirc it uses distancemachine class which are parallel | 20:13 |
| josip | so only the calculation of pairwise is distributed? | 20:14 |
| josip | pairwise distance* | 20:15 |
| josip | err parallel* | 20:15 |
| blackburn | yeap | 20:15 |
| josip | http://news.ycombinator.com/item?id=2476983 | 20:16 |
| blackburn | josip: seems cluster distance is parallel too, but we could better wait for answer of Soeren (cause he is author) :D | 20:16 |
| @sonney2k | josip, it is parallel but not memory efficient (computes distance matrix) | 20:24 |
| blackburn | sonney2k: can you give me an another test for ROC? ;) | 20:29 |
| @sonney2k | true -1 +1 , pred. -1 +1 :) | 20:30 |
| blackburn | sonney2k: 1 1 -1 both true and predicted gives ROC (0,0) (0,1) (1,0) and auROC 1.0 | 20:30 |
| @sonney2k | and +1 -1 for pred :) | 20:30 |
| blackburn | sonney2k: eh.. about last one | 20:31 |
| blackburn | is it good that I have (0,0) (1,0) (1,1)? | 20:31 |
| blackburn | auROC 0.0 | 20:31 |
| blackburn | it seems to be right, but don't know exactly | 20:32 |
| @sonney2k | me neither but at least auROC is ok | 20:33 |
| -!- siddharth [~siddharth@117.211.88.150] has joined #shogun | 20:33 | |
| blackburn | hm.. okay, will push it just after some doc | 20:35 |
| @sonney2k | blackburn, just compare it to the python script on some realistic data sets | 20:36 |
| blackburn | sonney2k: can i trust it? you said it have bug | 20:37 |
| @sonney2k | the python one? it should be ok, just not when there are multiple outputs that are the same | 20:37 |
| blackburn | sonney2k: okay | 20:38 |
| blackburn | tested | 20:53 |
| blackburn | sonney2k: https://github.com/shogun-toolbox/shogun/pull/67 | 20:54 |
| blackburn | ready for 'execution' ;) | 20:55 |
| blackburn | *using axe or any other weapon | 20:55 |
| * sonney2k of course I will be using hattori hanzo manufactured swords if necessary | 20:56 | |
| @sonney2k | as any shogun would. | 20:56 |
| blackburn | oh so I will drink vodka | 21:00 |
| blackburn | as any russian do :D | 21:00 |
| -!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun | 21:04 | |
| -!- dvevre_ [b49531e3@gateway/web/freenode/ip.180.149.49.227] has joined #shogun | 21:16 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Ping timeout: 252 seconds] | 21:19 | |
| -!- dvevre_ is now known as dvevre | 21:26 | |
| @mlsec | Sorry guys, but ROC is best evaluated using continuous scores | 22:00 |
| @mlsec | ROC is deeply rooted in signal processing | 22:01 |
| blackburn | eh.. what you mean? | 22:04 |
| blackburn | because there is no difference in evaluation algorithm when continuous or not | 22:06 |
| @mlsec | I was referring to: sonney2k: [20:30:10] true -1 +1 , pred. -1 +1 :) | 22:06 |
| blackburn | ah | 22:06 |
| blackburn | I tested it on 1.1, 1.2, -1.3, etc | 22:07 |
| blackburn | the other reason why I made scores this way: mldata-utils ROC don't handle with equal scores | 22:07 |
| @mlsec | That's better. | 22:07 |
| @mlsec | The interesting part about ROC curves is the interpolation for continuous scores | 22:08 |
| @mlsec | Eg pessimistic, average and optimistic | 22:09 |
| blackburn | ah. read some about that in fawcett's paper | 22:09 |
| @mlsec | Yes. Good one | 22:09 |
| @mlsec | Is there also a section averaging ROCs? | 22:10 |
| @mlsec | That's also not trivial | 22:10 |
| blackburn | where 'there'? ;) | 22:10 |
| @mlsec | In the paper of Fawcett? | 22:11 |
| blackburn | yeap, it has a section about it | 22:11 |
| @mlsec | I am keeping msg short, as I am writing from a smartphone | 22:12 |
| blackburn | ok, just not understood where exactly, in class I made or in fawcett's paper | 22:12 |
| blackburn | I wonder how you use irc on your smartphones :) it seems to be not so convenient | 22:14 |
| blackburn | *Soeren did last week too | 22:14 |
| @mlsec | hehe. it's funny | 22:14 |
| @mlsec | Anyway. I had a lot of fun with writing ROC code (interpolation, AUC bounded at FP, averaging) | 22:16 |
| @mlsec | So I am looking forward to Shogun contributions | 22:17 |
| blackburn | oh I had a lot of struggles doing simple ROC | 22:17 |
| blackburn | made a pull request with it | 22:17 |
| blackburn | now i'm doing some 'refactoring' at shogun.Evaluation | 22:18 |
| -!- blackburn [~qdrgsm@109.226.117.183] has quit [Quit: Leaving.] | 23:06 | |
| -!- dvevre [b49531e3@gateway/web/freenode/ip.180.149.49.227] has quit [Ping timeout: 252 seconds] | 23:06 | |
| -!- siddharth [~siddharth@117.211.88.150] has quit [Read error: Connection reset by peer] | 23:28 | |
| --- Log closed Sun Apr 24 00:00:36 2011 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!
