--- Log opened Mon Jun 16 00:00:34 2014 | ||
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has quit [Quit: Konversation terminated!] | 00:56 | |
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has joined #shogun | 00:58 | |
shogun-buildbot | build #29 of nightly_fedora is complete: Failure [failed git] Build details are at http://buildbot.shogun-toolbox.org/builders/nightly_fedora/builds/29 blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com>, Parijat Mazumdar <mazumdarparijat@gmail.com> | 03:00 |
---|---|---|
-!- khalednasr [~k.nasr92@41.69.233.193] has quit [Ping timeout: 272 seconds] | 04:16 | |
shogun-buildbot | build #1794 of rpm1 - libshogun is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/rpm1%20-%20libshogun/builds/1794 blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com> | 04:21 |
-!- zxtx_ [~zv@37.205.63.158] has joined #shogun | 04:49 | |
-!- abinash_panda [~abinash@182.66.31.108] has quit [Read error: Connection reset by peer] | 04:56 | |
-!- abinash_panda [~abinash@182.66.18.16] has joined #shogun | 05:00 | |
-!- zxtx_ [~zv@37.205.63.158] has quit [Ping timeout: 245 seconds] | 05:07 | |
-!- zxtx_ [~zv@37.205.63.158] has joined #shogun | 05:09 | |
-!- zxtx_ [~zv@37.205.63.158] has quit [Ping timeout: 240 seconds] | 05:30 | |
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has quit [Remote host closed the connection] | 05:37 | |
-!- abinash_panda [~abinash@182.66.18.16] has quit [Ping timeout: 264 seconds] | 06:27 | |
-!- lisitsyn [~qdrgsm@85.114.187.192] has quit [Quit: Leaving.] | 06:56 | |
-!- abinash_panda [~abinash@117.200.235.81] has joined #shogun | 07:28 | |
-!- abinash_panda [~abinash@117.200.235.81] has quit [Ping timeout: 240 seconds] | 07:45 | |
-!- Saurabh7 [~Saurabh7@117.222.13.170] has joined #shogun | 08:42 | |
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has joined #shogun | 08:48 | |
thoralf | Heyho. | 08:48 |
-!- thoralf [~thoralf@91-65-141-60-dynip.superkabel.de] has quit [Quit: Konversation terminated!] | 09:03 | |
-!- abinash_panda [~abinash@117.200.238.190] has joined #shogun | 09:43 | |
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 10:09 | |
shogun-notifier- | shogun-data: Parijat Mazumdar :master * 8384def / multiclass/categorical_dataset/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/8384def9f612372b070d9f310f51856e8274e202 | 10:09 |
shogun-notifier- | shogun-data: servo dataset added | 10:09 |
shogun-notifier- | shogun-data: Fernando Iglesias :master * 2838474 / multiclass/categorical_dataset/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/28384743f9903745cec24caca8df901e46daa500 | 10:09 |
shogun-notifier- | shogun-data: Merge pull request #52 from mazumdarparijat/master | 10:09 |
shogun-notifier- | shogun-data: | 10:09 |
shogun-notifier- | shogun-data: servo dataset added | 10:09 |
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun | 11:00 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 11:00 | |
-!- khalednasr [~k.nasr92@41.69.130.43] has joined #shogun | 11:11 | |
@wiking | HeikoS: btw cloud is back online for a week now but it's outdated since we dont have a way yet to generate nightly packages :S | 11:22 |
-!- soumyaC [uid15286@gateway/web/irccloud.com/x-rsvcwvepjgaihcpc] has joined #shogun | 11:26 | |
-!- zxtx_ [~zv@95-91-192-118-dynip.superkabel.de] has joined #shogun | 11:32 | |
-!- zxtx_ [~zv@95-91-192-118-dynip.superkabel.de] has quit [Ping timeout: 255 seconds] | 11:41 | |
@HeikoS | wiking: at least it runs! nice! | 11:43 |
@HeikoS | wiking: what did you do? | 11:43 |
-!- abinash_panda [~abinash@117.200.238.190] has quit [Ping timeout: 255 seconds] | 11:48 | |
@wiking | HeikoS: just fixed some minor bug in the website's code and updated some components on fatbot | 11:49 |
@HeikoS | wiking: cool! | 11:53 |
@HeikoS | will check soon | 11:53 |
@wiking | kk | 11:53 |
@HeikoS | now meeting a ML meetup person who wants to do a shogun hands on workshop here soon | 11:54 |
@wiking | btw we need to do something about our preprocessors | 11:55 |
-!- HeikoS [~heiko@90.201.233.229] has quit [Ping timeout: 264 seconds] | 11:58 | |
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun | 11:59 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 11:59 | |
@wiking | HeikoS: almost all our preprocessors are dense feature preprocessors | 12:02 |
@wiking | like PNorm | 12:02 |
@wiking | that should be available for sparse case as well :S | 12:02 |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun | 12:10 | |
@HeikoS | wiking: yep! also feature selection as lambday currently builds | 12:11 |
lambday | HeikoS: hey! | 12:11 |
@wiking | mmm feature selection | 12:11 |
@wiking | how where? | 12:11 |
lambday | what did I miss ... checking log | 12:11 |
@wiking | since now we have random forest and that could be part of feature selection as well | 12:11 |
@wiking | jesus i'm going to benchmark this fucking scikit-learn as it's amazing how long it does take to train a liblinear... | 12:13 |
lambday | wiking: what's your idea about preprocessor? | 12:14 |
@HeikoS | lambday: hey! I currently am waiting for a call, but will be there afterwards.- | 12:14 |
lambday | wiking: I'm planning to add some feature selection algorithms under CDimensionReductionPreprocessor | 12:14 |
lambday | HeikoS: alright :) | 12:15 |
@wiking | mmm shit | 12:15 |
@wiking | CDimensionReductionPreprocessor is based on CDensePreprocessor | 12:15 |
@wiking | :( | 12:15 |
@wiking | so actually we cannot do either pca nor kpca on sparse matrix | 12:15 |
lambday | wiking: oh... well it doesn't necessarily be under dense per se... | 12:17 |
lambday | I mean what I was planning... | 12:17 |
-!- Saurabh7 [~Saurabh7@117.222.13.170] has quit [Remote host closed the connection] | 12:18 | |
@wiking | lambday: yeah no i mean it's not your fault | 12:18 |
@wiking | i mean that we should refactor | 12:18 |
@wiking | a bit the preprocessing | 12:18 |
@wiking | as it's quite bad that we cannot do atm pca on sparse matrix | 12:19 |
@wiking | lambday: i'll need another month and i'll be back in shogun full time :P | 12:19 |
lambday | wiking: hehe yeah.. :) | 12:19 |
lambday | wiking: I was also thinking to work on the plugins project with lisitsyn and make it ready before next year gsoc | 12:20 |
@wiking | lambday: i'm up for it | 12:20 |
@wiking | after mid july | 12:20 |
lambday | wiking: yeah I am hopefully will be a bit free after this gsoc... | 12:20 |
lambday | currently juggling many balls at once :/ | 12:21 |
@wiking | oh woah | 12:22 |
@wiking | In [4]: f = SparseRealFeatures() | 12:22 |
@wiking | [1] 13334 segmentation fault PYTHONPATH=/Users/wiking/shogun-latest-build/lib/python2.7/site-packages/ = | 12:22 |
@wiking | :DDD | 12:22 |
@wiking | git HEAD | 12:22 |
lambday | wtf | 12:23 |
lambday | no unit-tests? | 12:24 |
@wiking | dunno | 12:25 |
@wiking | do we support scipy.sparse -> shogun sparse? | 12:25 |
lambday | wiking: I just checked... works fine here :/ | 12:25 |
lambday | wiking: yep | 12:26 |
lambday | >>> from modshogun import SparseRealFeatures >>> f=SparseRealFeatures() >>> f SparseFeatures >>> | 12:26 |
@wiking | lambday: how do i create SparseRealFeatures from <60000x80371 sparse matrix of type '<type 'numpy.float64'>' with 426868 stored elements in Compressed Sparse Row format> | 12:27 |
@wiking | ? | 12:27 |
lambday | wiking: wait checking | 12:28 |
lambday | wiking: check this examples/undocumented/python_modular/mathematics_logdet.py | 12:29 |
@wiking | mmm python is constantly crashing on osx :( | 12:31 |
@wiking | with shogun | 12:31 |
lambday | that sucks!... | 12:34 |
lambday | it works on linux.. just tested | 12:35 |
@wiking | X_train, X_test, y_train, y_test = cross_validation.train_test_split(feats_norm, labels, test_size=0.4, random_state=0) | 12:36 |
@wiking | b_train = BinaryLabels(y_train) | 12:36 |
@wiking | this should work afaik | 12:36 |
@wiking | as y_train is a simple np array | 12:36 |
@wiking | i.e. SGVector | 12:36 |
@wiking | but it segfaults | 12:36 |
@wiking | fuck no more gdb on osx | 12:37 |
@wiking | let's see what lldb can do | 12:37 |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds] | 12:41 | |
@wiking | oh woah this crashes: b_train = BinaryLabels(np.array([1.0, -1.0])) | 12:43 |
@wiking | :D | 12:43 |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun | 12:44 | |
-!- lisitsyn [~lisitsin@mxs.kg.ru] has joined #shogun | 12:48 | |
-!- khalednasr [~k.nasr92@41.69.130.43] has quit [Quit: Leaving] | 12:48 | |
@wiking | btw we need to start generating python api docs as well | 12:49 |
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout] | 13:09 | |
@wiking | ppl we need osx buildbot ASAP | 13:14 |
lambday | wiking: well this may seem naive but how travis works actually? they run on our machines or on machines they provide? | 13:19 |
-!- zxtx_ [~zv@g225077069.adsl.alicedsl.de] has joined #shogun | 13:37 | |
@wiking | lambday: they provide the machines | 13:37 |
lambday | okay.. | 13:38 |
-!- kislay [~Abhijeet@103.240.205.204] has joined #shogun | 14:04 | |
@HeikoS | lambday: hey | 14:06 |
lambday | HeikoS: hi | 14:06 |
@HeikoS | lambday: how are things? | 14:06 |
lambday | HeikoS: ran into a problem while fixing streaming mmd cov bug... - https://github.com/shogun-toolbox/shogun/pull/2309 | 14:06 |
@wiking | HeikoS: we need a sponsor that gives us an osx buildbot | 14:07 |
lambday | HeikoS: but I think its minor issue and I spent too much time on figuring out what's wrong :( | 14:07 |
@HeikoS | wiking: what exactly does that mean? a physical machine? or some money to pay for a cloud VM? | 14:08 |
@HeikoS | lambday: checking the PR | 14:08 |
lambday | HeikoS: okay | 14:08 |
@HeikoS | lambday: so its about the burst mode? | 14:09 |
@HeikoS | lambday: since now there is lots of overhead? | 14:09 |
@wiking | HeikoS: it's either $$ or just to run a buildslave on an osx machine that is online 24/7 | 14:09 |
@HeikoS | wiking: ok, the latter seems more tricky actually | 14:09 |
@HeikoS | wiking: how much is that? | 14:09 |
lambday | HeikoS: yeah - so instead of streaming one block at a time, I changed it to stream many such blocks... but computation works blockwise | 14:10 |
@HeikoS | lambday: ok yes, thats what I did before, right? | 14:10 |
lambday | HeikoS: this way it speeded up like before | 14:10 |
@HeikoS | lambday: and the problem now are integration tests? | 14:10 |
lambday | HeikoS: exactly | 14:10 |
lambday | HeikoS: so I think it won't be hard to accomodate earlier implementation within this for testing purpose | 14:10 |
@wiking | HeikoS: a lot ~80 usd/month | 14:11 |
lambday | HeikoS: yeah but I tried with more samples/changing the seed and it seems to work just fine | 14:11 |
lambday | as in, gives reasonable values | 14:11 |
@HeikoS | lambday: so precomputing the fill kernel doesnt make sense you are right, but rather do it block wise | 14:13 |
lambday | HeikoS: yeah that sounded reasonable to me so did it this way | 14:14 |
@HeikoS | lambday: well then its fine, currently reading but your summary sounds totally reasonable | 14:14 |
@HeikoS | wiking: thats too much for us to pay | 14:14 |
@HeikoS | wiking: why is it so expensive? | 14:14 |
@wiking | HeikoS: i know that's why i said that we need a sponsor | 14:14 |
@wiking | HeikoS: dunno.. i couldn't find cheaper | 14:14 |
@HeikoS | wiking: why can't we put OSx on fatbot? | 14:14 |
lambday | HeikoS: I am sending data PR then... | 14:14 |
@HeikoS | wiking: okok, I will ask for that next time I meet people. | 14:15 |
@HeikoS | wiking: and about 24/7 macos | 14:15 |
@HeikoS | how reliable must tha tbe? | 14:15 |
@wiking | HeikoS: because a) fatbot is already full with other tasks b) running osx in a VM is pretty fucking hard... | 14:15 |
@HeikoS | can I put one here? like an old laptop or so? | 14:15 |
@wiking | yeah anything is ok | 14:16 |
@wiking | just to have it online 24/7 | 14:16 |
@HeikoS | wiking: how reliable? | 14:20 |
@HeikoS | wiking: sometimes internet fails here etc | 14:20 |
@HeikoS | cleaner might unplug | 14:20 |
@HeikoS | wiking: not for long, but might happen once in a while | 14:20 |
@HeikoS | wiking: and how old can the laptop be maximum? | 14:20 |
@HeikoS | lambday: patch seems fine though I might be missing details | 14:21 |
@HeikoS | lambday: so what about the integration problems? | 14:21 |
@HeikoS | lambday: before you send the PR | 14:21 |
@HeikoS | lambday: what exactly fails? | 14:22 |
@wiking | HeikoS: well at least it should be intel | 14:22 |
@wiking | :P i.e. not powerpc | 14:22 |
@wiking | and that it should have at least 5G free mem | 14:22 |
@wiking | because of the modular interface | 14:22 |
@HeikoS | lambday: so the nice thing about the null approximations is that we have ground truth (permutation test) | 14:22 |
@HeikoS | wiking: ok, will ask my friends for one | 14:22 |
@HeikoS | as every is using max that should be possible | 14:22 |
lambday | HeikoS: well, it was giving weird results.. in the statistic_linear_time_mmd.py test, p-value was coming out to be weird (more than 0.5)... | 14:23 |
@HeikoS | lambday: that should be checked | 14:24 |
lambday | HeikoS: and the test failed to detect that p!=q | 14:24 |
@HeikoS | lambday: which nul approximation was that? | 14:24 |
lambday | HeikoS: with both | 14:24 |
lambday | but I changed the seed and it was fine | 14:24 |
lambday | in fact, with more samples, it always succeeds | 14:24 |
@HeikoS | lambday: what about a unit (or even python) test that compares the null approximations? | 14:25 |
@HeikoS | to ground truth (permutation) one | 14:25 |
@HeikoS | lambday: since as long as the statistic is computed correctly, te permutation test is correct | 14:25 |
@HeikoS | and this p-value issue seems werd | 14:25 |
@HeikoS | weird | 14:25 |
@HeikoS | though, if changing the seed solves the issue, this might be a problem of not having enough data | 14:26 |
lambday | HeikoS: yep.. I was stuck with this for quite a long time but didn't understand what was going so fishy... I mean,... I am just streaming more blocks at once | 14:26 |
lambday | shouldn't make such difference. | 14:26 |
@wiking | wtf is this: SGSparseVector.cpp line 395: Computing sparse_dot(a,b) on unsorted vectors is very expensive: O(n^2) | 14:26 |
lambday | yes that's what I figured finally | 14:26 |
lambday | compared streamed features by both earlier method and current method | 14:27 |
@wiking | lambday: ^ | 14:29 |
lambday | HeikoS: and what you mean by comparing null approximation.. I guess its already done in existing python tests | 14:30 |
lambday | wiking: umm.. I did that? let me check | 14:30 |
lambday | wiking: sorry man I didn't write this code... :( | 14:32 |
-!- HeikoS1 [~heiko@90.201.233.229] has joined #shogun | 14:33 | |
lambday | wiking: but having a hashmap it can be improved I guess :/ | 14:34 |
HeikoS1 | lambday: ah yeah, that might be true. I think I did that before | 14:34 |
HeikoS1 | wiking: ask thoralf about that, we introduced it | 14:34 |
lambday | wiking: HeikoS1: at least a break statement could be introduced after SGSparseVector.cpp line 408 | 14:35 |
-!- HeikoS [~heiko@90.201.233.229] has quit [Ping timeout: 240 seconds] | 14:35 | |
HeikoS1 | lambday, wiking I think this method is fallback and should be avoided | 14:36 |
HeikoS1 | thats why the warning | 14:36 |
@wiking | HeikoS1: :)))) but afaik it's wrong because the features are sorted | 14:36 |
@wiking | or i have to figure out wtf is happening | 14:36 |
HeikoS1 | wiking: I see, well that might be true. | 14:39 |
HeikoS1 | wiking: would be good to find the reason | 14:39 |
HeikoS1 | wiking: did you get the scripts from sonne|work or besser82 yet? | 14:43 |
@wiking | nada | 14:44 |
HeikoS1 | rrr | 14:45 |
HeikoS1 | wiking: btw whats the state of you currently? | 14:46 |
lisitsyn | ohmy I was reading last two lines and wondering what's the language you're speaking in :D | 14:46 |
@wiking | HeikoS1: mmm moving | 14:46 |
@wiking | HeikoS1: off to .sg within a month | 14:46 |
lisitsyn | wiking: how long will you stay there btw? | 14:47 |
@wiking | 3-4 years | 14:47 |
-!- Saurabh7 [~Saurabh7@117.222.13.170] has joined #shogun | 14:47 | |
lisitsyn | wah wah | 14:47 |
lisitsyn | that's quite a while! | 14:48 |
@wiking | yeah ... | 14:48 |
lambday | HeikoS1: sent a data PR... https://github.com/shogun-toolbox/shogun-data/pull/53/files | 14:48 |
lambday | also updated PR with data | 14:49 |
@wiking | btw i'm planning to create a wheel pacakge of shogun | 14:49 |
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 14:49 | |
shogun-notifier- | shogun-data: lambday :master * b65fecd / testsuite/ (12 files): https://github.com/shogun-toolbox/shogun-data/commit/b65fecdac7d15f961868fb11b5517414882dddf8 | 14:49 |
shogun-notifier- | shogun-data: added data for PR #2309 | 14:49 |
shogun-notifier- | shogun-data: Heiko Strathmann :master * 6df1281 / testsuite/ (12 files): https://github.com/shogun-toolbox/shogun-data/commit/6df12810fb0a2a806b5fd47354d8d086ccfbf0f5 | 14:49 |
shogun-notifier- | shogun-data: Merge pull request #53 from lambday/master | 14:49 |
shogun-notifier- | shogun-data: | 14:49 |
shogun-notifier- | shogun-data: added data for PR #2309 | 14:49 |
HeikoS1 | lambday: feel free to merge the main one once travis says ok | 14:49 |
@wiking | http://wheel.readthedocs.org/en/latest/ | 14:49 |
HeikoS1 | wiking: whats weheel? | 14:49 |
lambday | HeikoS1: alright :) | 14:49 |
@wiking | HeikoS1: http://wheel.readthedocs.org/en/latest/ | 14:50 |
HeikoS1 | wiking: so pip install | 14:50 |
HeikoS1 | wiking: nice! | 14:50 |
HeikoS1 | super nice! | 14:50 |
lambday | HeikoS1: btw did you happen to take a look at the class diagram for feature selection? it works as a subclass of CDimensionReductionPreprocessor | 14:50 |
HeikoS1 | lambday: not yet but about to | 14:51 |
lambday | HeikoS1: Dino suggested that we should also think about three-variable interaction ... but I think we need another subclass for that | 14:51 |
HeikoS1 | lambday: one thing: do we want to use DenseFeatures as subclass? | 14:51 |
HeikoS1 | lambday: yes, thats subclass | 14:51 |
lambday | HeikoS1: that's the point - we don't need to :/ | 14:51 |
lambday | well, no | 14:51 |
HeikoS1 | lambday: I think it would be cooler to allow doing this for other features types | 14:51 |
HeikoS1 | lambday: at least sparse | 14:51 |
HeikoS1 | lambday: though I dont know of any algorithms for that :;D | 14:52 |
HeikoS1 | wiking: sparse feature selection? | 14:52 |
HeikoS1 | wiking: or more precise: feature selection on sparse data | 14:52 |
@wiking | HeikoS1: well i would need a lot of shit sparsed | 14:52 |
@wiking | :)) | 14:52 |
lambday | HeikoS1: wiking ws worried about that :D | 14:52 |
@wiking | HeikoS1: like why p-norm is only working currently in shogun for densefeatures? | 14:53 |
@wiking | totally bullshit | 14:53 |
@wiking | HeikoS1: btw what's the eta for a gaussian binary GP for a sparse feature dataset with 60k training examples? :) | 14:54 |
@wiking | should i try even? :) | 14:54 |
@wiking | libsvm with gaussian kernel is running for ages already | 14:54 |
HeikoS1 | wiking: 60k | 14:54 |
HeikoS1 | crash :) | 14:54 |
@wiking | hehehe ok | 14:55 |
lambday | HeikoS1: errr... travis failed in git submodule update :/ could you please restart the build? :( | 14:55 |
HeikoS1 | wiking: GP need to invert the covariance matrix which in this case doesnt fit into memory | 14:55 |
HeikoS1 | lambday: I think you can do that too | 14:55 |
HeikoS1 | no? | 14:55 |
lambday | HeikoS1: via web interface? | 14:55 |
HeikoS1 | lambday: yes in travis, press "sign in with github" and then you can start/stop jobs | 14:55 |
lambday | HeikoS1: alright.. cool | 14:56 |
HeikoS1 | wiking: for such a large amount of data, I would go for SVM or even KRR | 14:56 |
@wiking | HeikoS1: is it possible to horizontally scale the inversion of the cov matrix ? | 14:56 |
HeikoS1 | wiking: horizontally? | 14:56 |
HeikoS1 | wiking: one can scale up these things quite a bit | 14:57 |
@wiking | i.e. run it distributed | 14:57 |
HeikoS1 | wiking: but not in the naive (current) implementation | 14:57 |
HeikoS1 | wiking: there are SGD variants of this | 14:57 |
-!- Saurabh7 [~Saurabh7@117.222.13.170] has quit [Ping timeout: 240 seconds] | 14:57 | |
HeikoS1 | and variational methods, but not in Shogun | 14:57 |
HeikoS1 | wiking: Wu is working on nice things for that | 14:58 |
HeikoS1 | wiking: but all not really large | 14:58 |
HeikoS1 | wiking: maybe next year :) | 14:58 |
@wiking | that's why i'm asking | 14:58 |
@wiking | would it be possible to run that distributed | 14:58 |
@wiking | like MR (map-reduce) | 14:58 |
HeikoS1 | wiking: probably there exist some things for that | 14:58 |
HeikoS1 | this one here is also very interesting: | 14:58 |
HeikoS1 | wiking: http://arxiv.org/abs/1305.5029 | 14:59 |
HeikoS1 | super nice | 14:59 |
HeikoS1 | we should have that | 14:59 |
HeikoS1 | a few things can be done map-reduce like in Shogun | 14:59 |
HeikoS1 | grid-search for SVM | 14:59 |
HeikoS1 | x-validation | 14:59 |
HeikoS1 | this KRR | 15:00 |
HeikoS1 | maybe a GP | 15:00 |
@wiking | oh fuck | 15:00 |
@wiking | this scipy.sparse -> SparseRealFeatures conversion does not work the best | 15:00 |
HeikoS1 | wiking: haha ,whenever one starts using Shogun, this is what happens ;) | 15:01 |
@wiking | but WTF | 15:01 |
@wiking | it's just unacceptable that we cannot mix scipy.sparse and shogun's sparse | 15:01 |
lambday | wiking: it works fine for logdet examples :/ | 15:02 |
@wiking | lambday: RealSparseMatrixOperator ? | 15:02 |
@wiking | lambday: how should i feed that for GaussianKernel? :) | 15:02 |
lambday | wiking: well you don't need to use that | 15:02 |
lambday | wait I'm checking | 15:02 |
lambday | wiking: | 15:05 |
lambday | you need SparseRealFeatures working, right? | 15:05 |
lambday | so this works - | 15:05 |
lambday | from modshogun import SparseRealFeatures | 15:05 |
lambday | from scipy.sparse import eye | 15:05 |
lambday | A=eye(10,10) | 15:05 |
lambday | f=SparseRealFeatures(A.tocsc()) | 15:06 |
@wiking | lambday: A.T.tocsc() i suppose | 15:08 |
@wiking | i mean in case of eye it doesn't matter | 15:08 |
@wiking | but any other sane dataset would require to have the .T | 15:08 |
lambday | wiking: well, yes.. | 15:08 |
lambday | but this should work, right? | 15:08 |
@wiking | yeah seems liek that | 15:08 |
@wiking | it seems fucking sklearn is making crazy sutff | 15:09 |
@wiking | *stuff | 15:09 |
lambday | crazy good or...? | 15:09 |
@wiking | it reverts the order of the sparse vector values | 15:09 |
lambday | reverts as in? | 15:09 |
@wiking | well i have 0:1 123:1 222:2 ... | 15:10 |
@wiking | and when i do this | 15:10 |
-!- Saurabh7 [~Saurabh7@117.200.185.14] has joined #shogun | 15:10 | |
@wiking | cross_validation.train_test_split(..) | 15:10 |
@wiking | the created sparse matrices' value are going to be reverted | 15:10 |
@wiking | i.e. 222:2 123:1 0:1 | 15:10 |
lambday | wtf :/ | 15:10 |
@wiking | dunno why | 15:11 |
lambday | shouldn't matter though | 15:11 |
lambday | but weird | 15:11 |
@wiking | yeah it does matter | 15:11 |
@wiking | lambday: as you see from the warning :P | 15:11 |
@wiking | O(n^2) | 15:11 |
@wiking | ;P | 15:11 |
lambday | oh that! :D | 15:12 |
* lambday afk | 15:14 | |
-!- HeikoS1 [~heiko@90.201.233.229] has quit [Quit: Leaving.] | 15:55 | |
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has joined #shogun | 16:03 | |
-!- pickle27 [~pickle27@192-0-136-118.cpe.teksavvy.com] has quit [Read error: Connection reset by peer] | 16:03 | |
@wiking | lisitsyn: terminate called after throwing an instance of 'tapkee::not_enough_memory_error' | 16:30 |
@wiking | :DDDD | 16:30 |
@wiking | lisitsyn: what do i dooooo? :) | 16:30 |
lisitsyn | wiking: whoa | 16:33 |
lisitsyn | wiking: how did you do that? | 16:33 |
-!- PirosB3 [~pirosb3@host80-223-dynamic.246-95-r.retail.telecomitalia.it] has joined #shogun | 16:33 | |
@wiking | lisitsyn: sparse matrix | 16:37 |
@wiking | lisitsyn: with 60k elements | 16:37 |
@wiking | and about 80k different values | 16:37 |
@wiking | i.e. if you convert that matrix to dense then matrix.shape = (80k, 60k) | 16:38 |
@wiking | and i guess that's what happens | 16:38 |
@wiking | or? | 16:38 |
@wiking | i.e. sparse -> dense matrix conversion | 16:38 |
lisitsyn | wiking: what method? | 16:44 |
lisitsyn | it is quite possible it does this conversion | 16:44 |
lisitsyn | if you use something linear | 16:44 |
@wiking | TDistributedStochasticNeighborEmbedding | 16:44 |
lisitsyn | ahh | 16:45 |
lisitsyn | yes I didn't patch it enough | 16:45 |
lisitsyn | to work on distances or whatever | 16:45 |
@wiking | ? | 16:45 |
lisitsyn | you know it is just ported code | 16:45 |
lisitsyn | which takes float* Y | 16:45 |
@wiking | ah ok | 16:45 |
@wiking | yeah | 16:45 |
lisitsyn | it is possible to change it though | 16:45 |
lisitsyn | but involves 90% rewrite of code :D | 16:46 |
@wiking | heheheh | 16:46 |
lisitsyn | wiking: it is quite ready to do that stuff though | 16:48 |
-!- zxtx_ [~zv@g225077069.adsl.alicedsl.de] has quit [Ping timeout: 245 seconds] | 16:49 | |
lisitsyn | I mean it already uses vantage point tree to find neighbors | 16:49 |
@wiking | i need to b able to do dim reduction | 16:49 |
@wiking | on a very very sparse matrix | 16:49 |
@wiking | :) | 16:49 |
lisitsyn | well try KLLE | 16:49 |
lisitsyn | it won't fail on that matrix as you just need a kernel | 16:50 |
lisitsyn | though 80k eigenproblem could be quite slow to solve | 16:50 |
lisitsyn | err 60k | 16:50 |
lisitsyn | wiking: you'd also try landmark isomap or landmark mds | 16:50 |
lisitsyn | though this is very gaussian shit :D | 16:51 |
lisitsyn | if you reduce to 2d you probably will see gaussian blob | 16:52 |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds] | 17:04 | |
-!- Anand__ [2a683cfa@gateway/web/freenode/ip.42.104.60.250] has joined #shogun | 17:05 | |
Anand__ | Hi! I am new here and need some help regarding the Shogun Naive Bayes classifier. | 17:07 |
Anand__ | Does Shogun have a method in NBC to get the probabilities of a particular test instance lying in all the classes? | 17:09 |
-!- HeikoS [~heiko@90.201.233.229] has joined #shogun | 17:14 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 17:14 | |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun | 17:15 | |
-!- HeikoS [~heiko@90.201.233.229] has quit [Quit: Leaving.] | 17:25 | |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds] | 17:44 | |
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout] | 17:49 | |
-!- khalednasr [~k.nasr92@41.69.237.204] has joined #shogun | 17:53 | |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun | 17:58 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 17:58 | |
-!- khalednasr [~k.nasr92@41.69.237.204] has quit [Quit: Leaving] | 18:08 | |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 240 seconds] | 18:11 | |
-!- Saurabh7 [~Saurabh7@117.200.185.14] has quit [Ping timeout: 264 seconds] | 18:17 | |
-!- abinash_panda [~abinash@117.200.225.204] has joined #shogun | 18:17 | |
-!- zxtx_ [~zv@f052018195.adsl.alicedsl.de] has joined #shogun | 18:18 | |
-!- zxtx_ [~zv@f052018195.adsl.alicedsl.de] has quit [Ping timeout: 272 seconds] | 18:26 | |
-!- HeikoS [~heiko@dab-yat1-h-70-3.dab.02.net] has joined #shogun | 18:28 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 18:28 | |
-!- Saurabh7 [~Saurabh7@117.200.180.114] has joined #shogun | 18:28 | |
-!- abinash_panda [~abinash@117.200.225.204] has quit [Read error: Connection reset by peer] | 18:40 | |
-!- Saurabh7 [~Saurabh7@117.200.180.114] has quit [Quit: Leaving] | 18:42 | |
-!- HeikoS [~heiko@dab-yat1-h-70-3.dab.02.net] has quit [Ping timeout: 240 seconds] | 18:43 | |
-!- abinash_panda [~abinash@117.200.235.238] has joined #shogun | 19:00 | |
-!- abinash_panda [~abinash@117.200.235.238] has quit [Client Quit] | 19:05 | |
-!- abinash_panda [~abinash@117.200.235.238] has joined #shogun | 19:05 | |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun | 19:06 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 19:06 | |
-!- Anand__ [2a683cfa@gateway/web/freenode/ip.42.104.60.250] has quit [Quit: Page closed] | 19:09 | |
-!- abinash_panda [~abinash@117.200.235.238] has quit [Ping timeout: 252 seconds] | 19:12 | |
-!- abinash_panda [~abinash@182.66.24.191] has joined #shogun | 19:17 | |
-!- abinash_panda [~abinash@182.66.24.191] has quit [Ping timeout: 244 seconds] | 19:31 | |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 245 seconds] | 19:44 | |
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 19:55 | |
shogun-notifier- | shogun-data: Heiko Strathmann :master * 4fc0eb7 / testsuite/python2-tests/ (2 files): https://github.com/shogun-toolbox/shogun-data/commit/4fc0eb7698f3b2b0303b5eaeb9a59a5020aeccd9 | 19:55 |
shogun-notifier- | shogun-data: added registered parameter | 19:55 |
-!- HeikoS [~heiko@dab-yat1-h-70-1.dab.02.net] has joined #shogun | 20:01 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 20:01 | |
@HeikoS | wiking: around? | 20:14 |
-!- lambday [67157e4d@gateway/web/freenode/ip.103.21.126.77] has joined #shogun | 20:18 | |
-!- abinash_panda [~abinash@223.176.23.125] has joined #shogun | 20:28 | |
@HeikoS | lambday: hi! | 20:36 |
lambday | HeikoS: hey | 20:37 |
lambday | HeikoS: integration tests again failed | 20:37 |
lambday | checking some more | 20:37 |
-!- PirosB3 [~pirosb3@host80-223-dynamic.246-95-r.retail.telecomitalia.it] has quit [Quit: PirosB3] | 20:38 | |
lambday | HeikoS: regarding the complex wrapper thing, I noticed that last year.. but don't know what do we have to do there to avoid this warning.. :/ | 20:39 |
@HeikoS | lambday: I see, well its not important for now | 20:41 |
@HeikoS | lambday: integration tests are more important | 20:41 |
lambday | HeikoS: yeah... running locally | 20:41 |
@HeikoS | and? | 20:41 |
lambday | HeikoS: yep.. worked fine.. | 20:42 |
lambday | HeikoS: made some changes | 20:42 |
@HeikoS | lambday: cool! :) | 20:42 |
@HeikoS | lambday: will soon take a much closer look to the feature selection stuiff you wrote down | 20:43 |
lambday | HeikoS: just one test was failing in python modular.. generated data for that | 20:43 |
lambday | HeikoS: yeah.. I should try to finish that before mid term | 20:43 |
lambday | b-test will be finished by today | 20:43 |
lambday | (just copy paste CLinearTimeMMD and change two lines :D) | 20:43 |
@HeikoS | nice! | 20:45 |
@HeikoS | lambday: do you have any idea how to start the demos locally? | 20:57 |
@HeikoS | shogun-demo? | 20:57 |
@HeikoS | its django, but I dont know about that stuff | 20:57 |
lambday | HeikoS: no man! :( | 20:58 |
lambday | there's no README? | 20:58 |
@HeikoS | lambday: there is, but I know too little about django to make sense of it :( | 21:00 |
lambday | I know nothing about django :( | 21:00 |
lambday | HeikoS: btw did you watch the match? :D germany 4 portugal 0 :D | 21:01 |
lambday | HeikoS: do you know how to run ctest locally for python3? | 21:03 |
@HeikoS | lambday: did not watch it, but not surprised haha :) | 21:05 |
@HeikoS | no idea :) | 21:05 |
@HeikoS | lambday: man I just want to run demos ..... | 21:05 |
@HeikoS | cant be so hard | 21:05 |
lambday | HeikoS: the only demo I know is these ones - http://shogun-toolbox.org/page/documentation/demo | 21:07 |
lambday | HeikoS: which ones are you trying to run? | 21:07 |
@HeikoS | saurabh added much more stuff to them | 21:07 |
@HeikoS | but the latest on the website dont include this | 21:07 |
@HeikoS | so I wanna run locally to see | 21:07 |
lambday | kevin is the guy :/ | 21:08 |
lambday | HeikoS: I sent a new data PR for fixing the integration tests... shall I merge it? | 21:11 |
-!- HeikoS [~heiko@dab-yat1-h-70-1.dab.02.net] has quit [Ping timeout: 240 seconds] | 21:13 | |
shogun-notifier- | shogun-data: lambday :master * e8b5520 / testsuite/ (6 files): https://github.com/shogun-toolbox/shogun-data/commit/e8b5520b8ef7946863cf84719b8b8e0e51a7caf4 | 21:16 |
shogun-notifier- | shogun-data: added data for PR #2309 | 21:16 |
shogun-notifier- | shogun-data: Soumyajit De :master * b6d9995 / testsuite/ (6 files): https://github.com/shogun-toolbox/shogun-data/commit/b6d99954619587cfa7db68d4631108141274fd2a | 21:16 |
shogun-notifier- | shogun-data: Merge pull request #54 from lambday/master | 21:16 |
shogun-notifier- | shogun-data: | 21:16 |
shogun-notifier- | shogun-data: added data for PR #2309 | 21:16 |
-!- abinash_panda [~abinash@223.176.23.125] has quit [Quit: leaving] | 21:26 | |
-!- abinash_panda [~abinash@223.176.23.125] has joined #shogun | 21:29 | |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has joined #shogun | 21:32 | |
-!- mode/#shogun [+o HeikoS] by ChanServ | 21:32 | |
@HeikoS | sonney2k: around? | 21:34 |
@HeikoS | sonne|work: arpound? | 21:35 |
@HeikoS | wiking: around? | 21:35 |
@HeikoS | besser82: around? | 21:35 |
@HeikoS | lisitsyn: around? | 21:35 |
-!- lisitsyn1 [~qdrgsm@80.252.20.67] has joined #shogun | 21:36 | |
lambday | multiclass-libsvm fails - unrelated... phew!! | 21:36 |
-!- lisitsyn1 is now known as nystisil | 21:36 | |
nystisil | hey lambday | 21:37 |
-!- lambday is now known as yadbmal | 21:37 | |
nystisil | :D | 21:37 |
yadbmal | ney nystisil | 21:37 |
yadbmal | :D | 21:37 |
shogun-notifier- | shogun: khalednasr :develop * f018255 / src/ (3 files): https://github.com/shogun-toolbox/shogun/commit/f018255fd2008ae5de398f3bb27ca5c542c80dda | 21:37 |
shogun-notifier- | shogun: Added support for softmax visible units and sampling with evidence to CRBM | 21:37 |
shogun-notifier- | shogun: Sergey Lisitsyn :develop * 31f5609 / src/ (3 files): https://github.com/shogun-toolbox/shogun/commit/31f5609f7a7345ca05b5c1f8c7425236da2270df | 21:37 |
shogun-notifier- | shogun: Merge pull request #2315 from khalednasr/rbms | 21:37 |
shogun-notifier- | shogun: | 21:37 |
shogun-notifier- | shogun: Added support for softmax visible units and sampling with evidence to CRBM | 21:38 |
nystisil | I have no idea what I merged | 21:38 |
nystisil | :D | 21:38 |
nystisil | gonna talk to khaled about it :D | 21:38 |
@HeikoS | yadbmal: nice one :) | 21:39 |
yadbmal | HeikoS: nystisil started it :D | 21:39 |
@HeikoS | yadbmal: haha :) | 21:40 |
@HeikoS | nystisil: you should read the code! ;);) | 21:40 |
nystisil | HeikoS: what code? | 21:40 |
nystisil | ahh I've read it | 21:40 |
@HeikoS | nystisil: the one your merge | 21:40 |
yadbmal | HeikoS: reading the documentation of CStreamingMMD... reads nice now | 21:40 |
nystisil | it is not about how code looks | 21:40 |
@HeikoS | nystisil: not that I get you, how to start the shogun-demo django stuff locally | 21:41 |
@HeikoS | nystisil: thats why I suggest to read it haha ;) | 21:41 |
nystisil | HeikoS: doesnt' help | 21:41 |
nystisil | HeikoS: khaled is deep into contrastive divergence of restricted boltzmann machines | 21:41 |
@HeikoS | nystisil: anyways, demo stuff, any idea? | 21:41 |
nystisil | and other smart words | 21:41 |
nystisil | demo hmm | 21:41 |
nystisil | I didn't try to run it | 21:41 |
nystisil | I am not sure I am of any help with that :D | 21:42 |
@HeikoS | nystisil: ok | 21:42 |
nystisil | HeikoS: I bet wiking is the man | 21:43 |
shogun-buildbot | build #2377 of bsd1 - libshogun is complete: Failure [failed test] Build details are at http://buildbot.shogun-toolbox.org/builders/bsd1%20-%20libshogun/builds/2377 blamelist: Sergey Lisitsyn <lisitsyn.s.o@gmail.com>, khalednasr <k.nasr92@gmail.com> | 21:44 |
-!- pickle27f [~pickle27@216.191.231.218] has joined #shogun | 21:57 | |
-!- pickle27f [~pickle27@216.191.231.218] has quit [Client Quit] | 21:58 | |
-!- pickle27 [~pickle27@216.191.231.218] has joined #shogun | 21:59 | |
@HeikoS | pickle27: hey! | 22:16 |
pickle27 | hey! | 22:16 |
pickle27 | did you get the demos working? | 22:16 |
@HeikoS | pickle27: nope | 22:16 |
@HeikoS | the server runs | 22:16 |
@HeikoS | but it just echos my browsers requests | 22:16 |
pickle27 | hmm what is it complaining about | 22:16 |
@HeikoS | pickle27: see pm | 22:17 |
@HeikoS | how do I get a list of demos, or link to find them behind? | 22:17 |
@HeikoS | the ones in the readme dont work | 22:17 |
pickle27 | so the program boots the django web server | 22:17 |
pickle27 | if you want to use it you need to go to localhost:8000 I think | 22:17 |
pickle27 | oh except you don't know what the urls are | 22:18 |
@HeikoS | yep | 22:18 |
@HeikoS | pickle27: so as said, my terminal prints my browser requests | 22:18 |
@HeikoS | but I dont know what to do now | 22:18 |
pickle27 | the urls.py maps urls to python functions | 22:18 |
pickle27 | so if you look at the first one | 22:19 |
pickle27 | regression/svr/ | 22:19 |
pickle27 | localhost:8000/regression/svr/ should run the code in the regression.svr.handler function | 22:19 |
pickle27 | man I hate django | 22:19 |
pickle27 | so ugly | 22:19 |
pickle27 | is that working for you? | 22:20 |
@HeikoS | pickle27: kind of | 22:21 |
@HeikoS | it doesnt look right | 22:21 |
@HeikoS | not like on our website | 22:21 |
@HeikoS | and buttons dont do things | 22:21 |
@HeikoS | pickle27: could you try on your machine? | 22:22 |
@HeikoS | maybe something is wrong with my setup ? | 22:22 |
pickle27 | I don't have django setup on here unfortunately | 22:25 |
pickle27 | I haven't ever played with the demo app | 22:26 |
pickle27 | for starters it should be flask too and not django ... | 22:26 |
pickle27 | maybe I'll do that some day soon | 22:26 |
pickle27 | I *really* hate how hard it is to clone any of our web repos and work on them | 22:27 |
pickle27 | waaaaay to much config specific to the exact server running those apps | 22:27 |
pickle27 | are you trying to add a new demo? | 22:27 |
@HeikoS | pickle27: I agree with you | 22:32 |
@HeikoS | pickle27: I try to inspect what Saurabh did | 22:32 |
pickle27 | ahh gotcha | 22:32 |
@HeikoS | He added some really nice stuff | 22:32 |
@HeikoS | but I cannot check it | 22:32 |
pickle27 | maybe he can help you more? | 22:32 |
pickle27 | I'll port that app to flask someday soon | 22:32 |
@HeikoS | pickle27: yes I asked him already but just had some time | 22:32 |
pickle27 | things will be better | 22:32 |
pickle27 | flask == simple django == gross | 22:33 |
@HeikoS | the application basically doesnt run here | 22:33 |
@HeikoS | just prints the text and renders all buttons/boxes | 22:33 |
@HeikoS | but nothing happens in the back | 22:33 |
@HeikoS | pickle27: cool sounds good. I mean this should be easy to use | 22:33 |
pickle27 | yeah exactly | 22:35 |
pickle27 | flask is the perfect framework for our stuff | 22:36 |
pickle27 | it has almost zero conceptual overhead to deal with in order to work with it | 22:36 |
@HeikoS | ok | 22:37 |
-!- HeikoS [~heiko@nat-180-209.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.] | 22:38 | |
-!- chakri [srichakrad@gateway/shell/bnc4free/x-jqfqzylcqabpvnjk] has quit [Remote host closed the connection] | 22:51 | |
-!- besser82 [quassel@fedora/besser82] has quit [Remote host closed the connection] | 22:51 | |
-!- travis-ci [~travis-ci@ec2-54-83-107-201.compute-1.amazonaws.com] has joined #shogun | 23:21 | |
travis-ci | [travis-ci] it's Sergey Lisitsyn's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/27711411 | 23:21 |
-!- travis-ci [~travis-ci@ec2-54-83-107-201.compute-1.amazonaws.com] has left #shogun [] | 23:21 | |
-!- khalednasr [~k.nasr92@41.69.237.204] has joined #shogun | 23:27 | |
-!- chakri [srichakrad@gateway/shell/bnc4free/x-msubycjynfywgrma] has joined #shogun | 23:29 | |
-!- yadbmal [67157e4d@gateway/web/freenode/ip.103.21.126.77] has quit [Ping timeout: 246 seconds] | 23:34 | |
-!- pickle27 [~pickle27@216.191.231.218] has quit [Quit: This computer has gone to sleep] | 23:58 | |
--- Log closed Tue Jun 17 00:00:36 2014 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!