| --- Log opened Wed Aug 21 00:00:24 2013 | ||
| shogun-buildbot | build #954 of rpm1 - libshogun is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/954 | 00:01 |
|---|---|---|
| -!- foulwall [~user@2001:da8:215:c252:7846:786c:36da:abfc] has quit [Ping timeout: 245 seconds] | 00:10 | |
| shogun-buildbot | build #1278 of cyg1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1278 blamelist: Viktor Gal <viktor.gal@maeth.com> | 00:32 |
| shogun-buildbot | build #1572 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1572 blamelist: Viktor Gal <viktor.gal@maeth.com> | 01:16 |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 260 seconds] | 01:58 | |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun | 02:03 | |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 276 seconds] | 02:08 | |
| -!- travis-ci [~travis-ci@ec2-54-227-44-173.compute-1.amazonaws.com] has joined #shogun | 02:14 | |
| travis-ci | [travis-ci] it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/10424566 | 02:14 |
| -!- travis-ci [~travis-ci@ec2-54-227-44-173.compute-1.amazonaws.com] has left #shogun [] | 02:14 | |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun | 02:16 | |
| -!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout] | 02:18 | |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun | 02:32 | |
| travis-ci | [travis-ci] it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/10424611 | 02:32 |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun [] | 02:32 | |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 248 seconds] | 02:32 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun | 02:33 | |
| -!- zxtx [~zv@rrcs-74-62-200-195.west.biz.rr.com] has quit [Ping timeout: 264 seconds] | 04:12 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun [] | 04:33 | |
| -!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has joined #shogun | 06:58 | |
| -!- gsomix [~gsomix@95.67.149.216] has quit [Ping timeout: 264 seconds] | 07:53 | |
| -!- gsomix [~gsomix@80.234.99.172] has joined #shogun | 08:05 | |
| -!- lisitsyn1 [~lisitsyn@fb2-lo1.global63.net] has left #shogun [] | 08:21 | |
| -!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun | 08:26 | |
| @wiking | sonney2k: ping | 09:02 |
| -!- gsomix [~gsomix@80.234.99.172] has quit [Ping timeout: 264 seconds] | 09:17 | |
| -!- gsomix [~gsomix@80.234.99.172] has joined #shogun | 09:19 | |
| -!- gsomix_ [~gsomix@109.169.154.231] has joined #shogun | 09:56 | |
| -!- gsomix [~gsomix@80.234.99.172] has quit [Ping timeout: 260 seconds] | 09:59 | |
| -!- gsomix_ is now known as gsomix | 11:08 | |
| -!- yo [4e352d55@gateway/web/freenode/ip.78.53.45.85] has joined #shogun | 11:21 | |
| -!- yo is now known as Guest60747 | 11:21 | |
| gsomix | good morning? | 11:22 |
| -!- Guest60747 [4e352d55@gateway/web/freenode/ip.78.53.45.85] has quit [Ping timeout: 250 seconds] | 11:26 | |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun | 11:36 | |
| @wiking | hey | 11:38 |
| gsomix | wiking, hey ho | 11:43 |
| gsomix | sonney2k, around? | 11:44 |
| gsomix | wiking, are there nice and safe methods for use SGVector with offset? I want use it for Parsing. | 11:59 |
| @wiking | gsomix: whatyaneed exactly | 11:59 |
| @wiking | i mean SGVector.vector is just a simple T* array where T is a builtin type... so u can offset that for sure quite easily the only thing you'll always have to check yourself is that you are not trying to offset more than the length of the vector... | 12:00 |
| gsomix | something like SGVector<char> substring=string.get_vector_with_offset(start); where string is SGVector. | 12:01 |
| gsomix | wiking, yeah, but I want refcount for this. | 12:01 |
| @wiking | gsomix: aaaaah lool | 12:01 |
| @wiking | you want to raise the refcount of the sgvector if you get a subset of it | 12:02 |
| @wiking | mmm | 12:02 |
| gsomix | yep, of course I can to make unsafe offset with pointers... | 12:02 |
| @wiking | mmm | 12:03 |
| @wiking | there's this command | 12:03 |
| @wiking | SGVector<T> get() | 12:03 |
| @wiking | { | 12:03 |
| @wiking | return *this; | 12:03 |
| @wiking | } | 12:03 |
| @wiking | and according to the doxygen: * @return the refcount increased vector | 12:04 |
| @wiking | which i dont see yet how it is done if you just return *this :P | 12:04 |
| @wiking | but then again you should check it with .ref_count() | 12:05 |
| @wiking | but if that works | 12:05 |
| @wiking | then basically you can write the same type of function | 12:05 |
| @wiking | SGVector<T> get_vector_with_offset(start) { return } | 12:06 |
| @wiking | just do the offsetting yourself | 12:06 |
| @wiking | awith the pointer of *this | 12:06 |
| @wiking | *with | 12:06 |
| @wiking | gsomix: but then again this is really just an assumption... you should check it simply with the ref_count() :P | 12:07 |
| gsomix | wiking, hm, ok. but there is needed to store offset in SGVector. additional data... I need think about it. | 12:09 |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Quit: Leaving.] | 12:11 | |
| -!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has joined #shogun | 12:36 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 12:36 | |
| @wiking | HeikoS: sorry man but that's not shogun's fault | 13:02 |
| lisitsyn | wiking: yes it raises the RC | 13:17 |
| lisitsyn | wiking: it *has* to call the constructor | 13:17 |
| lisitsyn | RVO whatever but you need to create a copy | 13:18 |
| @wiking | ah ok so it's calling the copy ctr | 13:21 |
| @wiking | lisitsyn: still here? | 13:29 |
| -!- xBeast [~xBeast@den-69-171-160-70.evdo.leapwireless.net] has joined #shogun | 13:47 | |
| -!- gsomix [~gsomix@109.169.154.231] has quit [Ping timeout: 256 seconds] | 13:56 | |
| -!- xBeast [~xBeast@den-69-171-160-70.evdo.leapwireless.net] has left #shogun ["Leaving"] | 13:57 | |
| lisitsyn | wiking: yes | 13:58 |
| @wiking | lisitsyn: so i was thinking about adding a new serilization support, but then i realised that actually what would b good to have is to be able to (de)serialize not only to hdfs but as well deserialize a mahout Dense/SparseVector into SGVector/sparseVector | 14:03 |
| @wiking | the first part is quite easy... i mean we just have to add like a SerializeAvroFile/Reader.h with the right implementation | 14:04 |
| @wiking | insights? | 14:06 |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun | 14:06 | |
| lisitsyn | wiking: sorry don't get: what is the first part and the second part? | 14:13 |
| -!- gsomix [~gsomix@109.169.154.231] has joined #shogun | 14:14 | |
| @wiking | 1) support serialization to hdfs | 14:14 |
| @wiking | 2) support serialization of mahout's vector into shogun's vector | 14:14 |
| @HeikoS | wiking: well, I still cannot compile it, mpi *is* installed | 14:39 |
| lisitsyn | wiking: I am not sure I have any insight on that actually | 14:57 |
| lisitsyn | is there any specific problem? | 14:57 |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Quit: Leaving.] | 15:25 | |
| lisitsyn | HeikoS: are you going to nips this year? | 15:31 |
| @HeikoS | lisitsyn: hi! not clear yet, waiting for rewie answers | 15:32 |
| lisitsyn | HeikoS: http://mloss.org/workshop/nips13/ | 15:32 |
| @HeikoS | but optimistic | 15:32 |
| lisitsyn | I think it is for you then :) | 15:32 |
| @HeikoS | ah cool! :) | 15:32 |
| @HeikoS | thanks for the info | 15:32 |
| lisitsyn | HeikoS: lets prepare something | 15:32 |
| @HeikoS | lisitsyn: yeah why not :) | 15:32 |
| @HeikoS | shogun parallel stuff :D | 15:33 |
| lisitsyn | I would like to go as well but that's too much travelling already :D | 15:33 |
| @HeikoS | why? | 15:35 |
| @HeikoS | travelling is nice :) | 15:36 |
| lisitsyn | HeikoS: $$ :) | 15:36 |
| @HeikoS | hehe | 15:37 |
| -!- foulwall [~user@2001:da8:215:c252:893c:1950:94a2:dc48] has joined #shogun | 16:19 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun | 16:32 | |
| -!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 16:47 | |
| shogun-notifier- | shogun: Thoralf Klein :develop * d0bbb8f / src/shogun/features/streaming/ (2 files): https://github.com/shogun-toolbox/shogun/commit/d0bbb8ff4f40c453e79f6a8f71b2a32de608bf91 | 16:47 |
| shogun-notifier- | shogun: Refactoring of StreamingSparseFeatures: Removed current_vector and current_length | 16:47 |
| shogun-notifier- | shogun: * Removed code duplication; simpler initialization. | 16:47 |
| shogun-notifier- | shogun: * Using non-refcounted sparse vector to avoid double-frees when using get_vector(). | 16:47 |
| shogun-notifier- | shogun: Heiko Strathmann :develop * eb72b69 / src/shogun/features/streaming/ (2 files): https://github.com/shogun-toolbox/shogun/commit/eb72b6932aec410817eb76af44b781a9fc67de7e | 16:47 |
| shogun-notifier- | shogun: Merge pull request #1449 from tklein23/refactoring_streaming_sparse_features | 16:47 |
| shogun-notifier- | shogun: | 16:47 |
| shogun-notifier- | shogun: Refactoring of StreamingSparseFeatures: Removed members current_vector/_length | 16:47 |
| shogun-buildbot | build #1457 of deb2 - static_interfaces is complete: Failure [failed configure] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1457 blamelist: Thoralf Klein <thoralf.klein@zib.de> | 16:54 |
| shogun-buildbot | build #1488 of bsd1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1488 blamelist: Thoralf Klein <thoralf.klein@zib.de> | 16:57 |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun | 16:57 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun [] | 16:59 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun | 17:10 | |
| @wiking | re | 17:25 |
| -!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 246 seconds] | 17:31 | |
| shogun-buildbot | build #1279 of cyg1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1279 blamelist: Thoralf Klein <thoralf.klein@zib.de> | 17:34 |
| -!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has joined #shogun | 17:36 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 17:37 | |
| shogun-buildbot | build #1458 of deb2 - static_interfaces is complete: Failure [failed configure] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1458 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 17:38 |
| shogun-buildbot | build #1489 of bsd1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1489 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 17:42 |
| @wiking | gsomix: here? | 17:45 |
| gsomix | wiking, yep | 17:45 |
| @wiking | gsomix: so just a quick question as you are really into this | 17:45 |
| @wiking | say i want a new class that can generate SGVectors or SGMatrixes from files | 17:46 |
| @wiking | currently this is implemented with classes that are subclasses of File.h | 17:46 |
| @wiking | right? | 17:46 |
| gsomix | hm, yep. but it works with plain arrays, not SGVector's or SGMatrices. | 17:47 |
| @wiking | yeah i saw that | 17:48 |
| @wiking | but then how one would get like a CFeatures* from a CSVFile at the moment? | 17:48 |
| gsomix | wiking, CFeatures' object should have method load(CFile*) | 17:49 |
| @wiking | and it does \o/ | 17:49 |
| @wiking | ok so basically what i would need then is open the file with CSVFile and then create a CFeatures object and call .load(CSVFile) on it | 17:50 |
| @wiking | and that'll basically load the csvfile content into a CFeatures. | 17:51 |
| @wiking | ok and that is the same for SGMatrix | 17:51 |
| @wiking | i can see | 17:51 |
| gsomix | wiking, yeo | 17:52 |
| gsomix | https://github.com/shogun-toolbox/shogun/blob/develop/examples/undocumented/python_modular/kernel_anova_modular.py | 17:52 |
| gsomix | for example | 17:52 |
| @wiking | ok this is great | 17:53 |
| @wiking | thnx | 17:53 |
| -!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.] | 17:54 | |
| -!- van511 [~van51@ppp-94-66-88-196.home.otenet.gr] has joined #shogun | 18:14 | |
| -!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Ping timeout: 248 seconds] | 18:17 | |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun | 18:19 | |
| travis-ci | [travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/10455297 | 18:19 |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun [] | 18:19 | |
| shogun-buildbot | build #1280 of cyg1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1280 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 18:19 |
| gsomix | ok, all another works are done today. | 18:33 |
| gsomix | time for shogun | 18:34 |
| -!- lambday [67157c37@gateway/web/freenode/ip.103.21.124.55] has joined #shogun | 18:57 | |
| lambday | HeikoS: hi | 18:57 |
| lambday | wiking: thanks man! :) | 19:01 |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun | 19:03 | |
| travis-ci | [travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/10455297 | 19:03 |
| -!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun [] | 19:03 | |
| @HeikoS | lambday: hi! | 19:04 |
| @HeikoS | lambday: I have been starting to prototype the independent computation stuff for PBS in python, might be useful for us later | 19:06 |
| lambday | HeikoS: PBS? | 19:06 |
| @HeikoS | thats a batch based cluster | 19:06 |
| @HeikoS | where one can submit independent jobs that are queued etc | 19:06 |
| lambday | :D | 19:06 |
| lambday | wow! | 19:06 |
| @HeikoS | seems to work nice with our ideas | 19:07 |
| @HeikoS | but still building things | 19:07 |
| @HeikoS | lambday: anyway, how are things? | 19:07 |
| lambday | HeikoS: I was checking out ColPack | 19:07 |
| @HeikoS | lambday: ah cool, what do you think about it? | 19:07 |
| lambday | HeikoS: and to use it, we need CRS format | 19:07 |
| lambday | which eigen3 provides | 19:08 |
| lambday | and we already have a method which convers SGSparseMatrix to eigen3 one | 19:08 |
| @HeikoS | lambday: nice! | 19:08 |
| lambday | but extra memory usage is a drawback | 19:08 |
| lambday | anyway | 19:08 |
| @HeikoS | well | 19:08 |
| lambday | but computing matrix power | 19:08 |
| @HeikoS | thats life | 19:08 |
| lambday | is a bad thing :( | 19:08 |
| @HeikoS | I know | 19:08 |
| @HeikoS | krylstat do it by hand? | 19:09 |
| -!- van511 [~van51@ppp-94-66-88-196.home.otenet.gr] has quit [Quit: Leaving.] | 19:09 | |
| lambday | HeikoS: yeah they simply multiply matrices | 19:09 |
| lambday | HeikoS: I wanted to test something | 19:09 |
| @HeikoS | lambday: what better ideas are there? | 19:09 |
| lambday | since its psd, we can use eigenvalues and eigenvectors | 19:10 |
| @HeikoS | for matrix powers? | 19:10 |
| lambday | and we already have Lanczos for that | 19:10 |
| lambday | yes | 19:10 |
| @HeikoS | ah | 19:10 |
| @HeikoS | that would be sweet | 19:10 |
| @HeikoS | would be good to compare the speed | 19:10 |
| lambday | HeikoS: yes... otherwise I can add matrix-matrix product for SGSparseMatrix... | 19:11 |
| lambday | using mat-vec product that I already added | 19:11 |
| @HeikoS | lambday: yep | 19:11 |
| lambday | so, I am thinking of adding just the coloring first | 19:12 |
| @HeikoS | lambday: ok | 19:12 |
| lambday | and then the matrix power - straight forward | 19:12 |
| @HeikoS | the only thing open is then what is fastest | 19:12 |
| lambday | then will add the eigenvalue/eigenvectors thing | 19:12 |
| @HeikoS | lambday: ok this sounds all very promising | 19:12 |
| lambday | HeikoS: yes | 19:12 |
| @HeikoS | has colpack many dependencies? | 19:13 |
| lambday | HeikoS: oh one thing... while passing SGVector as a param, is all the data copied or just the refcount increases? | 19:13 |
| @HeikoS | can we cmake it into shogun?, wiking? | 19:13 |
| lambday | HeikoS: that's already done.... :D I talked to wiking yesterday and asked him to add it | 19:13 |
| lambday | and he did :D | 19:13 |
| @HeikoS | lambday: SGVector copy by value creates another SGVector instance with the pointer pointing to the same memory | 19:13 |
| @HeikoS | and yes recount is increased | 19:13 |
| @HeikoS | lambday: whoooo | 19:13 |
| @HeikoS | wiking rocks doesnt he? :) | 19:13 |
| lambday | no I can run ColPack without any dependency | 19:13 |
| lambday | HeikoS: you bet :D | 19:14 |
| lambday | the cmake thing is frickin | 19:14 |
| lambday | awesome! | 19:14 |
| lambday | HeikoS: I just checked that wiking added ColPack check... now gotta do some test.. | 19:14 |
| @HeikoS | lambday: indeed! | 19:15 |
| @HeikoS | lambday: ok then | 19:15 |
| lambday | HeikoS: once I add this.. I'll be relieved! :D theoretically all modules will be implemented | 19:15 |
| @HeikoS | I am really curious how long this takes with colpack and your lancos idea | 19:15 |
| lambday | rest, test and more tests | 19:15 |
| @HeikoS | since with matlab it took ages | 19:15 |
| lambday | HeikoS: yeah! but the power is small, then Lanczos can be a overkill | 19:16 |
| @HeikoS | so that I just dumped a fixed matrix colouring to disc and loaded it all over again | 19:16 |
| @HeikoS | lambday: lets see | 19:16 |
| @HeikoS | lambday: the ozone matrix is a good benchmark, use it | 19:16 |
| lambday | alright.. yes | 19:16 |
| lambday | HeikoS: can I add to shogun?? | 19:16 |
| @HeikoS | its quite sparse so the conversion also should not be a problem | 19:16 |
| lambday | add the ozone matrix as data? | 19:16 |
| @HeikoS | lambday: good question | 19:16 |
| @HeikoS | would be good let me check | 19:17 |
| lambday | its 80 MB almost | 19:17 |
| @HeikoS | sonney2k, ^ too much? | 19:18 |
| @HeikoS | lambday: I just asked daniel lets see whether its even allowed | 19:18 |
| @HeikoS | I mean the matrix at least is ok | 19:18 |
| lambday | HeikoS: what is? | 19:18 |
| @HeikoS | but we rather want the dataset to generate the matrix from for different parameters | 19:18 |
| @HeikoS | the matrix itself is not protected o whatever | 19:18 |
| @HeikoS | allowed to publish the data | 19:19 |
| lambday | okay...hmm | 19:19 |
| @HeikoS | I will also ask Daniel to for advice on hard synthetic datasets | 19:19 |
| @HeikoS | lambday: btw the notebooks should also illustrate how to use all the solvers you wrote, people might be interested in that as a seperate module in shogun | 19:19 |
| @HeikoS | lambday: but that later, first lets make things work :) | 19:20 |
| lambday | currently for log-det, we can only use 3 solvers... | 19:20 |
| lambday | direct one, cocg, and cg-m | 19:20 |
| @HeikoS | ah yes not the individual one | 19:22 |
| lambday | I am still confused about why the paper calls it cocg-m... :-/ if the data is real, then vec1.transpose()*vec2 is same as vec1.dot(vec2) :( | 19:22 |
| @HeikoS | lambday: dont know, dont worry about this | 19:22 |
| lambday | the only thing that changes from cocg and cg is this step! | 19:22 |
| @HeikoS | lambday: the individual with preconditioning should be optional, once all the rest works | 19:22 |
| @HeikoS | since time is getting more tight these days | 19:22 |
| lambday | yeah! :( | 19:23 |
| lambday | and things are getting tougher and tougher on my side :( | 19:23 |
| @HeikoS | lambday: indeed, so lets focus on getting the core to work | 19:23 |
| @HeikoS | then we can extend | 19:23 |
| @HeikoS | oh and documentation is also very important | 19:23 |
| @HeikoS | we should go over all class docs etc to make things very clear | 19:24 |
| @HeikoS | so | 19:24 |
| lambday | alright | 19:24 |
| lambday | okay... how long would it take for me to add swig interfaces? (stupid question but I have no idea about this) | 19:24 |
| @HeikoS | 1.) some tests on solvers | 19:24 |
| @HeikoS | 2.) matrix product | 19:24 |
| @HeikoS | 3.) colpack | 19:24 |
| @HeikoS | 4.) implement probing | 19:24 |
| @HeikoS | 5.) final example | 19:24 |
| @HeikoS | 6.) tests | 19:24 |
| @HeikoS | 7.) docs | 19:24 |
| @HeikoS | lambday: they are done extremley fast | 19:24 |
| @HeikoS | just a few lines to add | 19:25 |
| @HeikoS | all built automagically | 19:25 |
| lambday | classic! | 19:25 |
| @HeikoS | static interfaces are more work though | 19:25 |
| @HeikoS | and I would like to have at least some basic ones | 19:25 |
| @HeikoS | for matlab and R people | 19:25 |
| lambday | wow this list seems great!... I am copying it to my mail :D | 19:25 |
| lambday | err.. | 19:25 |
| lambday | for the ipython demo, we need python modular interface to work, right? | 19:26 |
| @HeikoS | lambday: yes | 19:26 |
| @HeikoS | lambday: but as said, modular interfaces are only 1h work | 19:26 |
| lambday | HeikoS: phew! :D | 19:26 |
| @HeikoS | if you want, have a look into src/interfaces/modular | 19:26 |
| @HeikoS | there are .i files | 19:26 |
| @HeikoS | GaussianProcess.i GaussianProcess_includes.i | 19:26 |
| @HeikoS | you have to create similar ones | 19:27 |
| @HeikoS | or even put it into math | 19:27 |
| @HeikoS | for static ones, no idea, I have in fact never created some, but ask pickle27 I think he created some static ones | 19:27 |
| lambday | wah | 19:28 |
| lambday | alright... so, in the final example, would we just compute log-det or run the whole thing? | 19:28 |
| @HeikoS | lambday: but thats also not top priority | 19:28 |
| @HeikoS | lambday: yes | 19:28 |
| lambday | and plot that graph that I saw in the ozone examplepage? | 19:28 |
| @HeikoS | lambday: so I would like a few things in fact | 19:28 |
| @HeikoS | lambday: first some basic examples how the log-det framework is used in c++ and in python modular | 19:29 |
| @HeikoS | with a few cases covered ( no plots though, just basic usage) | 19:29 |
| lambday | alright | 19:29 |
| @HeikoS | and then, have one big ipython notebook where you introduce your whole project, illustrate many of the methods, how they work, how they behave, have some examples, convergence plots, maybe the ozone data | 19:30 |
| @HeikoS | maybe even a little MCMC on a GMRF | 19:30 |
| @HeikoS | even the ozone maybe | 19:30 |
| @HeikoS | depending on how much time we have | 19:30 |
| @HeikoS | should leave 1-2 weeks at the end just for the notebook and examples | 19:30 |
| lambday | HeikoS: alright... | 19:31 |
| @HeikoS | lambday: will be very cool! :) | 19:31 |
| lambday | yeah! :) | 19:31 |
| lambday | feels bad that I could have done much more if I could give all my time but these f**king people! :'( | 19:32 |
| lambday | anyway I will try to finish coloring by this week only | 19:33 |
| lambday | last night I checked what I had to do, now just gotta implement | 19:33 |
| @HeikoS | lambday: dont spend too much energy being angry, not worth it. You will finish the project anyhow and did a great job already, so dont feel bad | 19:33 |
| @HeikoS | lambday: cool, sounds good! | 19:33 |
| lambday | :( | 19:34 |
| @HeikoS | lambday: I hope to finish the cluster stuff this week so then next week I will test your stuff :D | 19:34 |
| lambday | yeah you're right... better use my energy in something constructive | 19:34 |
| lambday | hehe.... that's superb! | 19:34 |
| lambday | alright.. | 19:35 |
| @HeikoS | cool man, keep me update how it is going | 19:35 |
| lambday | HeikoS: are we gonna have another meeting before endsem? | 19:35 |
| @HeikoS | really excited about the graph colouring stuff :) | 19:35 |
| @HeikoS | lambday: yes I will organize one soon | 19:35 |
| lambday | HeikoS: official meeting | 19:35 |
| lambday | alright :) | 19:36 |
| lambday | me too | 19:36 |
| lambday | lol@endsem, I meant endterm :D | 19:36 |
| @HeikoS | hehe :) | 19:38 |
| @HeikoS | students ;) | 19:38 |
| @wiking | HeikoS ok so i was digging | 19:40 |
| @wiking | HeikoS: got some time now? | 19:40 |
| @HeikoS | wiking: sorry just about to leave home | 19:42 |
| @HeikoS | wiking: what did you dig out? | 19:42 |
| @wiking | okok | 19:42 |
| @wiking | no worries we talk another time | 19:42 |
| @HeikoS | wiking: ok then, see you tomorrow! | 19:43 |
| @HeikoS | wiking: btw we could have another dev meeting some time soon! :) | 19:43 |
| -!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.] | 19:43 | |
| -!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has quit [Ping timeout: 245 seconds] | 19:43 | |
| -!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout] | 19:47 | |
| shogun-buildbot | build #1575 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1575 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com> | 19:54 |
| @wiking | lambday: ping | 19:57 |
| lambday | wiking: pong | 19:57 |
| @wiking | lambday: does the color graph library detection work for u? | 19:57 |
| lambday | wiking: checking... :) | 19:57 |
| lambday | wiking: it did detect | 19:58 |
| @wiking | cool | 19:58 |
| -!- foulwall [~user@2001:da8:215:c252:893c:1950:94a2:dc48] has quit [Ping timeout: 245 seconds] | 19:58 | |
| lambday | you rock! | 19:58 |
| @wiking | lambday: do u have time a bit now to talk about the computation engine you've developed? | 19:59 |
| lambday | wiking: sure | 20:01 |
| lambday | its still its infancy :( | 20:02 |
| @wiking | lambday: yeah but still cool | 20:02 |
| @wiking | i just want to talk about the backend | 20:02 |
| lambday | but this can be extended... | 20:02 |
| @wiking | what u had in mind for the real distributed engine | 20:03 |
| lambday | wiking: please ask/suggest | 20:03 |
| @wiking | so let's say we have now the serialize engine for all the objects | 20:03 |
| @wiking | that currently only supports serialization via file but that can be extended | 20:03 |
| @wiking | my only question here is what should we implement as a parallel engine | 20:04 |
| @wiking | i.e. the new subclass of IndependentComputationEngine | 20:04 |
| lambday | wiking: so far me and Heiko had discussed, it uses a job queue... and dispatches the jobs to different nodes... all these jobs has a compute method and they all know how to compute the individual results... | 20:05 |
| lambday | and all these jobs submits their results to the job aggregator.. the computation engine waits until all the jobs are computed | 20:06 |
| @wiking | yep true | 20:06 |
| @wiking | but what's the job engine | 20:06 |
| @wiking | ? | 20:06 |
| @wiking | because so far it's abstract | 20:07 |
| @wiking | we need a library or something that does all the underlying job | 20:07 |
| @wiking | like finding a free node and sending the job there | 20:07 |
| lambday | wiking: yeah and I guess these all will be within the parallel implementation of the job engine.. yes as of now its abstract only, the serial computation engine which is currently there just computes these jobs sequentially | 20:08 |
| @wiking | lambday: yeah i wouldn't recommend implementing the scheduling and all the networking stuff ourselves | 20:09 |
| lambday | so, we need implementation of the submit_job method and wait_for_all... | 20:09 |
| @wiking | i was wondering whether you've been talking about what libraries to interface with | 20:09 |
| lambday | wiking: we planned using mpi for that | 20:09 |
| @wiking | because for sure there are like millions of engines like this out there | 20:09 |
| @wiking | lambday: ookey that's an option | 20:10 |
| lambday | wiking: Heiko is currently working on that for a project that needs log-det computation on cluster.. | 20:10 |
| @wiking | yeah i got that T:) | 20:11 |
| lambday | wiking: if you have any other suggestion please let us know... we've kept this outside the scope of gsoc, so not much we discussed about the future of it | 20:11 |
| @wiking | lambday: yeah that's totally out of scope of it | 20:12 |
| @wiking | i'm just thinking what we should support/implement in the future | 20:12 |
| lambday | but the job/engine design is flexible enough as per we think | 20:12 |
| lambday | what idea do you have in mind for dispatching jobs other that mpi (I don't have experience in this yet) | 20:13 |
| @wiking | lambday: well that's the thing | 20:13 |
| @wiking | i'm looking into other cluster infrastructure implementations | 20:14 |
| @wiking | what we could use/support | 20:14 |
| lambday | that's super cool! | 20:14 |
| @wiking | i'm wondering now about hadoop | 20:14 |
| lambday | ah I was just about to say this | 20:15 |
| @wiking | because it's quite hot nowadays | 20:15 |
| @wiking | for big data computation | 20:15 |
| lambday | yeah it is... | 20:15 |
| @wiking | but in the past when i was doing hadoop pipes | 20:15 |
| @wiking | (the c++ interface of hadoop) i had quite some troubles | 20:15 |
| @wiking | but now i'm checking into it | 20:16 |
| lambday | what's the main difference in openmpi and hadoop anyway? | 20:16 |
| @wiking | anyhow i'd like to support to read/write mahout vector,matrix so we'll have hdfs support in shogun for sure | 20:16 |
| @wiking | lambday: well there's a lot :) | 20:16 |
| @wiking | for one hadoop is java native :P | 20:16 |
| lambday | yeah that but functionally? | 20:16 |
| @wiking | lambday: and hadoop is bascially giving you a filessystem | 20:17 |
| @wiking | a distributed fs | 20:17 |
| @wiking | while openmpi is really just purely doing the communication among the nodes | 20:17 |
| lambday | wiking: but when we have independent computation jobs (like the ones I created for log-det) we don't need to have any msg massing at all... do you think openmpi won't be a bottleneck for this case? | 20:18 |
| @wiking | lambday: what do you mean you dont need msg massing? | 20:18 |
| lambday | err | 20:18 |
| lambday | passing | 20:18 |
| @wiking | lambday: who's gonna take care of communicating amongst the nodes? | 20:19 |
| @wiking | and who's gonna know at all about the nodes | 20:19 |
| lambday | ya but these nodes can compute each of the tasks independently, so in that sense.. | 20:20 |
| lambday | I mean, in this scenario, what advantages does hadoop have over mpi | 20:21 |
| @wiking | yeah but then after the computation is finished | 20:21 |
| @wiking | somehow it has to be signaled and sent back the result | 20:21 |
| lambday | yeah that's true | 20:21 |
| @wiking | well hadoop takes care of lot of things | 20:22 |
| @wiking | like having a fault tolerant computation engine (i.e. when a node dies while doing a job) | 20:22 |
| @wiking | and then having the hdfs which all the nodes share | 20:22 |
| @wiking | and some funky optimization | 20:22 |
| @wiking | like the scheduler of the job will assign the node to do the computation | 20:23 |
| @wiking | that has the fastest way of accessing the data in the hdfs | 20:23 |
| lambday | okay... that sounds great | 20:24 |
| lambday | but you had some troubles with the c++ interface... so would it fit in? | 20:24 |
| @wiking | lambday: i'm just cehcking the latest stable | 20:25 |
| @wiking | the last time i've used this was like 1.5 years ago | 20:25 |
| lambday | if it works - awesome.. I am sure Heiko can give some inputs... I will be really happy if I can be of any help in implementing any of these stuffs | 20:26 |
| @wiking | yeah i'll do today a test with this | 20:27 |
| @wiking | see if i can get something out of the new version of hadoop and it's c++ api | 20:27 |
| @wiking | seems to be much more mature now | 20:27 |
| lambday | that's great | 20:27 |
| -!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun | 20:30 | |
| -!- naywhayare [~ryan@spoon.lugatgt.org] has quit [Ping timeout: 276 seconds] | 20:37 | |
| -!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has quit [Ping timeout: 256 seconds] | 20:38 | |
| -!- naywhayare [~ryan@spoon.lugatgt.org] has joined #shogun | 20:43 | |
| shogun-buildbot | build #23 of osx1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/23 blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Thoralf Klein <thoralf.klein@zib.de> | 20:58 |
| -!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has joined #shogun | 21:39 | |
| -!- zxtx [~zv@rrcs-76-79-81-162.west.biz.rr.com] has joined #shogun | 22:00 | |
| -!- lambday [67157c37@gateway/web/freenode/ip.103.21.124.55] has quit [Ping timeout: 250 seconds] | 22:03 | |
| -!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun [] | 22:33 | |
| -!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has joined #shogun | 23:12 | |
| -!- mode/#shogun [+o iglesiasg] by ChanServ | 23:12 | |
| @iglesiasg | good evening people | 23:12 |
| -!- shogun-notifier- [~irker@7nn.de] has joined #shogun | 23:13 | |
| shogun-notifier- | shogun: Kevin :develop * 53014e3 / / (10 files): https://github.com/shogun-toolbox/shogun/commit/53014e3b94e82d366c5a9cdb730ec628eb813f3d | 23:13 |
| shogun-notifier- | shogun: added UWedgeSep and python modular examples for all current ICA methods | 23:13 |
| shogun-notifier- | shogun: Sergey Lisitsyn :develop * 7d6ee1d / / (10 files): https://github.com/shogun-toolbox/shogun/commit/7d6ee1dc3ce1d9cd1905c6b71e2dd81a314a20c5 | 23:13 |
| shogun-notifier- | shogun: Merge pull request #1447 from pickle27/develop | 23:13 |
| shogun-notifier- | shogun: | 23:13 |
| shogun-notifier- | shogun: added UWedgeSep Alg and python modular examples for all current ICA methods | 23:13 |
| lisitsyn | hey iglesiasg | 23:14 |
| @iglesiasg | lisitsyn, how is it going? | 23:15 |
| shogun-notifier- | shogun: hushell :develop * 6badc95 / / (5 files): https://github.com/shogun-toolbox/shogun/commit/6badc95efdf9816841dd1cc82e8e04cbde7e49d3 | 23:15 |
| shogun-notifier- | shogun: max product inference for factor graph model | 23:15 |
| shogun-notifier- | shogun: Fernando Iglesias :develop * 636a71b / / (5 files): https://github.com/shogun-toolbox/shogun/commit/636a71bb190549cdb43ec28f263cc5811a6bd7b0 | 23:15 |
| shogun-notifier- | shogun: Merge pull request #1379 from hushell/fg_dev | 23:15 |
| shogun-notifier- | shogun: | 23:15 |
| shogun-notifier- | shogun: max product inference for factor graph model | 23:15 |
| lisitsyn | ha | 23:15 |
| lisitsyn | iglesiasg: ok, what about you? | 23:16 |
| @iglesiasg | lisitsyn, I am good. This is my last night of holidays :( | 23:16 |
| lisitsyn | iglesiasg: I am really curious why you have chose poland ;) | 23:17 |
| @iglesiasg | lisitsyn, well it is cheap, close to home and with beach holidays ;) | 23:18 |
| lisitsyn | iglesiasg: c'mon you are from spain, it is common destination for 99% of mid class russians ;) | 23:19 |
| @iglesiasg | lisitsyn, Spain or Poland? | 23:19 |
| lisitsyn | spain | 23:19 |
| @iglesiasg | as you may understand I have been in Spain quite a bit already hehe | 23:19 |
| @iglesiasg | I like to visit new places | 23:19 |
| lisitsyn | haha | 23:19 |
| lisitsyn | yeah but beaches are better in spain right? | 23:20 |
| @iglesiasg | but yeah, there are places left to visit in Spain for me | 23:20 |
| gsomix | iglesiasg, >> I like to visit new places | how about Russia next year? >:3 | 23:20 |
| @iglesiasg | lisitsyn, there are better beaches in Spain than the one I am here | 23:20 |
| @iglesiasg | gsomix, hehe why not! | 23:21 |
| lisitsyn | yeah we discussed st. petersburg once :D | 23:21 |
| @iglesiasg | gsomix, St. Petersburgh looks awesome | 23:21 |
| lisitsyn | it is pretty swedish though | 23:21 |
| @iglesiasg | it will take me years before I get right the spelling hehe | 23:21 |
| lisitsyn | iglesiasg: you should get to gsomix's origin place | 23:22 |
| @iglesiasg | where is it? | 23:22 |
| gsomix | huh! | 23:22 |
| @iglesiasg | Kamchatka is also impressive! | 23:22 |
| lisitsyn | well all around here | 23:22 |
| lisitsyn | but it is pure russia | 23:22 |
| lisitsyn | not the city so more pure for sure | 23:22 |
| @iglesiasg | I checked a bit more about both St. Petersburg and Kamchatka | 23:22 |
| lisitsyn | kamchatka should be quite expensive | 23:22 |
| gsomix | iglesiasg, lisitsyn pure russia... no, no cities! I'm here now: 52.337542,49.923922 | 23:24 |
| lisitsyn | iglesiasg: yo yo saint P should be not that expensive comparing to kamchatka | 23:25 |
| @iglesiasg | gsomix, google maps doesn't show me much about that location hehe | 23:25 |
| lisitsyn | iglesiasg: there is freaking nothing :D | 23:25 |
| @iglesiasg | lisitsyn, saint P is da city | 23:25 |
| gsomix | iglesiasg, indeed. | 23:25 |
| @iglesiasg | haha | 23:25 |
| lisitsyn | it is like 50 kms and you are in kazakhstan | 23:26 |
| @iglesiasg | Borat?! | 23:26 |
| lisitsyn | the borat yes | 23:26 |
| lisitsyn | hmm I think more | 23:27 |
| @iglesiasg | I have to start saving money, I want to go up to Kilimanjaro | 23:27 |
| shogun-buildbot | build #1490 of bsd1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1490 blamelist: hushell <hushell@hushell-U510.(none)> | 23:27 |
| lisitsyn | maybe ~100 or whatever | 23:27 |
| shogun-buildbot | build #1459 of deb2 - static_interfaces is complete: Failure [failed configure] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1459 blamelist: hushell <hushell@hushell-U510.(none)> | 23:27 |
| @iglesiasg | crazy expensive | 23:27 |
| shogun-buildbot | build #24 of osx1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/24 blamelist: hushell <hushell@hushell-U510.(none)> | 23:28 |
| lisitsyn | iglesiasg: like gsoc I guess :D | 23:28 |
| @iglesiasg | lisitsyn, so so | 23:29 |
| shogun-buildbot | build #1491 of bsd1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1491 blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com> | 23:32 |
| shogun-buildbot | build #25 of osx1 - libshogun is complete: Failure [failed test] Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/25 blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com> | 23:33 |
| hushell | iglesiasg: Thanks! :) | 23:43 |
| @iglesiasg | hushell, thanks to you ;) | 23:44 |
| hushell | iglesiasg: Next PR coming soon, I see how hard your job is, haha | 23:45 |
| @iglesiasg | hehe | 23:45 |
| -!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has quit [Quit: Leaving.] | 23:49 | |
| shogun-buildbot | build #1281 of cyg1 - libshogun is complete: Failure [failed compile] Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1281 blamelist: hushell <hushell@hushell-U510.(none)> | 23:50 |
| @iglesiasg | all right guys, going to bed now. See you tomorrow afternoon! | 23:59 |
| --- Log closed Thu Aug 22 00:00:25 2013 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!