--- Log opened Thu Aug 30 00:00:17 2012 | ||
-!- gsomix [~gsomix@178.45.88.247] has quit [Ping timeout: 256 seconds] | 00:03 | |
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has quit [Ping timeout: 268 seconds] | 03:54 | |
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun | 04:08 | |
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has quit [Remote host closed the connection] | 04:15 | |
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun | 04:25 | |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 07:09 | |
n4nd0 | hi naywhayare | 07:09 |
---|---|---|
n4nd0 | I read in logs you were asking about KNN yeseterday | 07:10 |
-!- blackburn [5bdfb203@gateway/web/freenode/ip.91.223.178.3] has joined #shogun | 07:38 | |
blackburn | oh matplotlib's author john hunter passed away | 07:39 |
n4nd0 | :( | 07:47 |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] | 08:05 | |
-!- gsomix [~gsomix@178.45.74.12] has joined #shogun | 08:46 | |
-!- gsomix [~gsomix@178.45.74.12] has quit [Quit: Ex-Chat] | 08:52 | |
-!- gsomix [~gsomix@178.45.74.12] has joined #shogun | 08:52 | |
CIA-52 | shogun: Evgeniy Andreev master * r037f1ed / (114 files): many fixes in examples - http://git.io/uxpkLA | 08:56 |
CIA-52 | shogun: Evgeniy Andreev master * r4cac666 / (204 files in 2 dirs): added env path - http://git.io/mSpu0Q | 08:56 |
CIA-52 | shogun: Sergey Lisitsyn master * r41d2350 / (207 files in 2 dirs): Merge pull request #761 from gsomix/examples - http://git.io/GOHytw | 08:56 |
@sonney2k | blackburn, what? | 08:58 |
@sonney2k | what happened to john? | 08:58 |
blackburn | sonney2k: he passed away after cancer treatment | 08:58 |
gsomix | good morning | 08:59 |
blackburn | he was diagnosed with cancer in July and there were some complications during treatment | 08:59 |
blackburn | sonney2k: http://numfocus.org/johnhunter/ | 09:00 |
@sonney2k | blackburn, horrible | 09:01 |
blackburn | yeah it is | 09:01 |
@sonney2k | blackburn, I knew him in person | 09:01 |
@sonney2k | we organized an mloss workshop and invited him | 09:01 |
@sonney2k | there is this nice video on videolectures from him showing the power of matplotlib | 09:01 |
@sonney2k | I even think he was subscribed to the shogun mailinglist | 09:02 |
blackburn | yeah OSS is kind of big family and I am saddened | 09:02 |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 09:04 | |
@sonney2k | blackburn, yeah he is still subscribed :/ | 09:08 |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] | 09:19 | |
blackburn | sonney2k: http://rosalind.info/problems/as-table/ | 09:25 |
shogun-buildbot | build #485 of deb3 - modular_interfaces is complete: Failure [failed test python_modular] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/485 blamelist: Evgeniy Andreev <gsomix@gmail.com> | 09:27 |
gsomix | shogun-buildbot, aha, next commit, please :) | 09:28 |
CIA-52 | shogun: Evgeniy Andreev master * r148781d / (2 files): fixed order of imports - http://git.io/eXh_0A | 09:52 |
CIA-52 | shogun: Sergey Lisitsyn master * rd321e36 / (2 files): Merge pull request #762 from gsomix/fix_examples - http://git.io/34ycqg | 09:52 |
shogun-buildbot | build #486 of deb3 - modular_interfaces is complete: Failure [failed test python_modular] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/486 blamelist: Evgeniy Andreev <gsomix@gmail.com>, Sergey Lisitsyn <lisitsyn.s.o@gmail.com> | 09:59 |
shogun-buildbot | build #487 of deb3 - modular_interfaces is complete: Success [build successful] Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/487 | 10:32 |
gsomix | shogun-buildbot, good work! | 10:39 |
shogun-buildbot | What you say! | 10:39 |
-!- hoijui [~hoijui@dslb-092-078-182-230.pools.arcor-ip.net] has joined #shogun | 12:07 | |
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has joined #shogun | 12:24 | |
-!- av3ngr [~av3ngr@60-241-222-244.static.tpgi.com.au] has left #shogun [] | 12:24 | |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has joined #shogun | 13:10 | |
-!- blackburn [5bdfb203@gateway/web/freenode/ip.91.223.178.3] has quit [Ping timeout: 245 seconds] | 13:42 | |
-!- blackburn [5bdfb203@gateway/web/freenode/ip.91.223.178.3] has joined #shogun | 13:46 | |
-!- gsomix [~gsomix@178.45.74.12] has quit [Quit: Ex-Chat] | 15:20 | |
@sonney2k | n4nd0, do you know when bern4rd has time? | 16:47 |
@sonney2k | or will be around? | 16:47 |
n4nd0 | sonney2k: I think he finished his internship yesterday | 16:49 |
n4nd0 | so probably he is coming back to Spain today, if he didn't do so yesterday | 16:49 |
n4nd0 | apart from that, I know nothing else | 16:49 |
n4nd0 | sonney2k: do you want me to ask him something? | 16:50 |
-!- in3xes [~in3xes@122.174.88.209] has joined #shogun | 16:54 | |
-!- in3xes [~in3xes@122.174.88.209] has quit [Ping timeout: 244 seconds] | 17:01 | |
-!- in3xes [~in3xes@122.174.88.209] has joined #shogun | 17:11 | |
naywhayare | n4nd0: are you Dr. KNN for shogun? | 17:42 |
n4nd0 | naywhayare: haha not a Dr. :P | 17:42 |
n4nd0 | but I put the cover tree there and test some other things so I may be able to help you | 17:42 |
naywhayare | okay; my goal is to find the distances and indices of the k nearest neighbors of a single dataset as simply and quickly as possible | 17:43 |
naywhayare | sonney2k suggested I build the tree by hand | 17:44 |
n4nd0 | I recommend you to use JL covertree | 17:44 |
n4nd0 | there is no fastest algorithm for that | 17:44 |
naywhayare | I mean, the fastest way to do it with shogun's available code specifically | 17:44 |
n4nd0 | ok | 17:44 |
naywhayare | JL's code does that, definitely, but his code is an unmaintainable nightmare | 17:44 |
naywhayare | I do not have curse words descriptive enough for what I think of that work | 17:45 |
n4nd0 | hehe it takes time but I finally managed to use it in shogun | 17:45 |
naywhayare | are you calling out to it, or did you port it? | 17:45 |
n4nd0 | I call it | 17:45 |
n4nd0 | I think it is the best ... I mean that code there has crazy stuff for optimization... | 17:45 |
n4nd0 | that's why it is so fast | 17:46 |
naywhayare | mmm, yes and no | 17:46 |
naywhayare | I don't like the CFLAGS he uses | 17:46 |
naywhayare | -ffast-math is technically unsafe in some cases though it does give a 3x speedup (on the trials I ran) | 17:46 |
naywhayare | when you take that away, the gains from the partial distance computation and the absurd stack-of-already-allocated-memory are not incredibly huge | 17:46 |
n4nd0 | you can just not compile with that? | 17:47 |
naywhayare | yeah, modify the Makefile | 17:47 |
n4nd0 | ok | 17:47 |
n4nd0 | anyway | 17:47 |
n4nd0 | how large is your dataset and what dimension does your data have? | 17:47 |
naywhayare | there are many datasets ranging from 100x3 to millions x hundreds | 17:47 |
n4nd0 | and maybe if you tell me a bit more of your application we can discover what is best to do using shogun | 17:47 |
naywhayare | well, I am using shogun to time it for comparison | 17:48 |
naywhayare | I would prefer to use kd-trees to keep the algorithms the same across all the libraries I am comparing | 17:48 |
n4nd0 | aham | 17:48 |
n4nd0 | we don't have kd-trees here I believe | 17:48 |
naywhayare | however I didn't see any kd-tree support anytmore | 17:48 |
naywhayare | sonne suggested that they did exist | 17:48 |
-!- in3xes [~in3xes@122.174.88.209] has quit [Ping timeout: 240 seconds] | 17:49 | |
n4nd0 | aham | 17:49 |
n4nd0 | I think we don't have them | 17:49 |
naywhayare | I need to step out for a little while... I'll be back later | 17:51 |
naywhayare | thanks for the pointers so far | 17:51 |
n4nd0 | all right | 17:51 |
n4nd0 | let me know later | 17:51 |
n4nd0 | in any case, covertree is faster than kd-tree, AFAIK | 17:51 |
-!- blackburn1 [~blackburn@188.168.14.142] has joined #shogun | 18:14 | |
-!- in3xes [~in3xes@122.174.88.209] has joined #shogun | 18:18 | |
CIA-52 | shogun: Sergey Lisitsyn master * r4c43de1 / src/shogun/mathematics/Statistics.h : Fixed doc mistake in statistics - http://git.io/49h53A | 18:25 |
blackburn1 | n4nd0: around? | 18:27 |
n4nd0 | yeah | 18:28 |
n4nd0 | tell me | 18:28 |
blackburn1 | n4nd0: wanted to think out loud :D | 18:28 |
blackburn1 | n4nd0: about serialization | 18:29 |
n4nd0 | yeah | 18:29 |
blackburn1 | okay the case is | 18:29 |
blackburn1 | we have a machine (say linear) | 18:29 |
blackburn1 | with w = vector of length 200 | 18:29 |
blackburn1 | we use it somehow and then load from file | 18:30 |
blackburn1 | with different w length | 18:30 |
n4nd0 | ok | 18:31 |
blackburn1 | I am confused what is happening there and what to do | 18:31 |
blackburn1 | :D | 18:31 |
blackburn1 | first of all we corrupt SGVector | 18:31 |
-!- puffin444 [180317bb@gateway/web/freenode/ip.24.3.23.187] has joined #shogun | 18:31 | |
n4nd0 | in terms of the ref counter? | 18:31 |
blackburn1 | yes | 18:31 |
blackburn1 | it is a different vector already | 18:32 |
blackburn1 | so we should create new reference counter I guess | 18:32 |
puffin444 | hey blackburn | 18:32 |
n4nd0 | but should it be a different vector? | 18:32 |
blackburn1 | puffin444: hey, any success with your issue? | 18:32 |
puffin444 | Yes in fact. | 18:32 |
blackburn1 | n4nd0: in simplest case "new machine then load" it is a different vector | 18:32 |
n4nd0 | blackburn1: I am assuming the vector you load from file is into the first vector you were talking about | 18:33 |
n4nd0 | yeah sure, in that case it ise | 18:33 |
puffin444 | It runs fine now. The only issue is that using the SGMatrices results in less precision. | 18:33 |
blackburn1 | yes but it is not true in general | 18:33 |
blackburn1 | puffin444: why? | 18:33 |
puffin444 | I really am not sure. It's the brent minimzer which seems to be where the precision is a little off. | 18:34 |
blackburn1 | but what is different with eigen matrices? | 18:34 |
puffin444 | All the calculations appear to be correct, is just that small numerical differences appear to result in an answer which is just slightly off that of using purely eigen matrices. | 18:34 |
blackburn1 | but still seems so strange | 18:35 |
puffin444 | It is absolutely essential that there are no eigen references in the headers, correct? | 18:35 |
blackburn1 | yes for now | 18:35 |
CIA-52 | shogun: Sergey Lisitsyn master * r80fee54 / src/shogun/lib/slep/slep_solver.cpp : Fixed memleak in slep solver - http://git.io/JFTiAA | 18:35 |
puffin444 | Yes it does seem strange I took 4 hours to see what was wrong. | 18:35 |
blackburn1 | I can imagine | 18:35 |
puffin444 | And it just seems that the difference is due to numerical error. | 18:36 |
blackburn1 | how much is the error? | 18:36 |
puffin444 | The gradient search comes up with a mimimum likelihood of something like 5.35 instead of 5.345 | 18:36 |
blackburn1 | huh that's pretty big | 18:37 |
puffin444 | Initially the two resuls are exactly the same but over time those numerical diffferences add up. | 18:37 |
puffin444 | Through gradient search. | 18:37 |
CIA-52 | shogun: Sergey Lisitsyn master * r78f693c / data : Updated data - http://git.io/hf-Leg | 18:38 |
puffin444 | I can look at it more. | 18:38 |
puffin444 | It's just that I don't think I can get this in by Sept 1st | 18:38 |
blackburn1 | puffin444: we have some troubles with serialization actually | 18:38 |
puffin444 | Uh Oh - the hash function? | 18:39 |
puffin444 | Are you talking about my parameter hash function? | 18:39 |
blackburn1 | not really, there is a problem with reference counting and some more | 18:39 |
puffin444 | Oh okay. | 18:39 |
blackburn1 | I didn't detect anything with hash stuff | 18:39 |
puffin444 | Just to let you know I really want to make this part of a release for Shogun, and I hope to continue to contribute for the time being. | 18:40 |
puffin444 | I may be able to add more stuff as part of a class project this semester too, but I am not sure. | 18:40 |
blackburn1 | that's awesome to know because you are the only man that knows GPs here | 18:41 |
blackburn1 | :D | 18:41 |
puffin444 | heh lol | 18:41 |
blackburn1 | I am totally out of GPs | 18:41 |
blackburn1 | puffin444: are you going to have masters study now? | 18:41 |
puffin444 | No it's actually Ph.D. | 18:41 |
blackburn1 | ahh so phd | 18:41 |
blackburn1 | I see | 18:41 |
puffin444 | It's probably going to focus on Computer Vision, but machine learning is part of everything here, so I will be definitely be working with it in some fashion. | 18:42 |
blackburn1 | I am hanging with CV as well so we could always discuss something | 18:43 |
blackburn1 | :) | 18:43 |
puffin444 | Yeah sure :) | 18:43 |
-!- blackburn2 [~blackburn@188.168.2.186] has joined #shogun | 18:45 | |
-!- gsomix [~gsomix@178.45.74.12] has joined #shogun | 18:45 | |
gsomix | good evening | 18:45 |
blackburn2 | too many of me | 18:46 |
puffin444 | So I have to go now. Just wanted to give you an update on stuff. I'll see if I can get rid of the imprecision. | 18:46 |
-!- gsomix is now known as blackburn4 | 18:46 | |
puffin444 | lol | 18:46 |
blackburn2 | puffin444: I don't really know whether we could release on 1st | 18:46 |
puffin444 | Okay. Are you guys thinking about pushing back the date? | 18:47 |
n4nd0 | puffin444: btw, did you find something strange in valgrind's trace at the end? | 18:47 |
-!- blackburn1 [~blackburn@188.168.14.142] has quit [Ping timeout: 244 seconds] | 18:47 | |
puffin444 | Oh valgrind detected a few read errors, which was the cause of the problem. | 18:48 |
n4nd0 | cool! | 18:48 |
puffin444 | There are still memory leaks though from Vector<CMap>. | 18:48 |
blackburn2 | puffin444: yes, I also had an idea of releasing RC (release candidate) first | 18:48 |
n4nd0 | I got angry some months ago too because my program didn't crash on valgrind but yes alone :D | 18:48 |
puffin444 | Okay good. I am working as much as I can to get this ready. I may be held back a bit by university stuff however. Am I holding you guys back? | 18:49 |
blackburn2 | puffin444: no, not really | 18:50 |
blackburn2 | we wanted a major version so better make it correct | 18:51 |
blackburn2 | :) | 18:51 |
blackburn2 | I will try to handle serialization leaks next days | 18:51 |
blackburn2 | puffin444: anyway thanks :) | 18:52 |
puffin444 | Thank you for being understanding. I'll see what I can do to increase precision. | 18:53 |
n4nd0 | let's continue with serialization conversation? | 18:53 |
-!- puffin444 [180317bb@gateway/web/freenode/ip.24.3.23.187] has quit [Quit: Page closed] | 18:54 | |
blackburn2 | n4nd0: yeah | 18:57 |
blackburn2 | n4nd0: I am totally confused with reference counting | 18:57 |
blackburn2 | do you have an understanding? | 18:57 |
n4nd0 | the trivial one | 18:58 |
blackburn2 | okay let me check what is leaking | 19:00 |
blackburn2 | n4nd0: okay serialization_basic_tests | 19:00 |
-!- blackburn1 [~blackburn@188.168.14.135] has joined #shogun | 19:02 | |
blackburn1 | piece of honey, useful wired connection | 19:02 |
blackburn1 | ooooookay | 19:03 |
blackburn1 | n4nd0: here/ | 19:03 |
n4nd0 | :) | 19:04 |
-!- blackburn2 [~blackburn@188.168.2.186] has quit [Ping timeout: 268 seconds] | 19:05 | |
n4nd0 | I have no idea how these functions Parameter::Load work :S | 19:06 |
blackburn1 | n4nd0: I could try to describe | 19:06 |
blackburn1 | I can't say I like how it works but it is there :) | 19:06 |
n4nd0 | yeah sure | 19:07 |
blackburn1 | n4nd0: each time you call load/save_serializable | 19:07 |
blackburn1 | in basic case it iterates through all parameters | 19:07 |
blackburn1 | you set via SG_ADD | 19:07 |
n4nd0 | mmm | 19:07 |
blackburn1 | in case of vectors and matrices it stores pointer to pointer to data and pointer to lengths | 19:08 |
n4nd0 | I need a break, I've got sort of headache | 19:08 |
blackburn1 | lol | 19:08 |
blackburn1 | already? | 19:08 |
blackburn1 | :D | 19:08 |
n4nd0 | haha not for this | 19:08 |
blackburn1 | okay sure | 19:08 |
n4nd0 | I have been reading on the computer for a long while before | 19:08 |
blackburn1 | do you have headache after that? | 19:08 |
n4nd0 | sometimes | 19:08 |
blackburn1 | I can easily handle 14-16 hours using computer :D | 19:09 |
blackburn1 | you are not geek enough! | 19:09 |
n4nd0 | haha no, I don't think so :P | 19:09 |
n4nd0 | see you in a while | 19:09 |
blackburn1 | more geekness! | 19:09 |
-!- blackburn4 is now known as blackburn2 | 19:18 | |
n4nd0 | all righty | 19:24 |
-!- in3xes [~in3xes@122.174.88.209] has quit [Quit: Leaving] | 19:32 | |
n4nd0 | do you think the use of m_parameters->add might be an issue? | 19:33 |
n4nd0 | I remember Heiko sent a mail saying to use SG_ADD | 19:33 |
n4nd0 | I think we could try to think what the state of the reference counters should be after each step of that test | 19:38 |
n4nd0 | e.g. how it should be after to_save->save_serializable(file) | 19:38 |
blackburn1 | n4nd0: okay leaking in serialization_basic_test is caused by wrong reference count after load | 19:38 |
blackburn1 | case is | 19:39 |
blackburn1 | SGObject -> DenseFeatures -> matrix | 19:39 |
n4nd0 | ok | 19:39 |
blackburn1 | after reading SGObject matrix's refcount is -1 | 19:39 |
blackburn1 | however I have no idea why | 19:39 |
n4nd0 | is it correct after to_save->save ? | 19:40 |
blackburn1 | yes | 19:41 |
blackburn1 | save does not corrupt | 19:41 |
n4nd0 | what is the ref count after save? | 19:41 |
blackburn1 | 1 | 19:42 |
blackburn1 | 2 when I get it | 19:42 |
n4nd0 | I don't understand what you mean with the last thing, sorry | 19:42 |
n4nd0 | 2 when you get it? | 19:42 |
blackburn1 | with get_feature_matrix | 19:42 |
blackburn1 | :) | 19:42 |
n4nd0 | aham ok :) | 19:43 |
blackburn1 | yeah it was hard to parse probably | 19:43 |
n4nd0 | I will compile with refcount enabled | 19:43 |
n4nd0 | may be helpful | 19:43 |
blackburn1 | what do you mean with refcountenabled? | 19:44 |
n4nd0 | I didn't use the right name but I don't find in configure what I wanted to say | 19:45 |
n4nd0 | wait | 19:45 |
blackburn1 | trace memory allocs? | 19:45 |
blackburn1 | no that won't really help | 19:45 |
n4nd0 | maybe it was that one yes | 19:45 |
blackburn1 | valgrind detects the error already | 19:45 |
n4nd0 | but I think it would be nice to see when the refcount is modified | 19:46 |
n4nd0 | it should get us to the root of the problem | 19:46 |
blackburn1 | yes that makes sense | 19:46 |
blackburn1 | n4nd0: it can be done with SG_GCDEBUG | 19:46 |
blackburn1 | no need to recompile | 19:46 |
n4nd0 | that was it :) | 19:47 |
n4nd0 | will we see something related to SGVector there as well? I remember I use it for SGObject | 19:47 |
blackburn1 | ahh | 19:47 |
blackburn1 | yes it is DEBUG_SGVECTOR and it requires to recompile | 19:47 |
n4nd0 | do you think it will be helpful? | 19:48 |
blackburn1 | yeah could be | 19:48 |
blackburn1 | I do not really understand how can it be -1 | 19:49 |
blackburn1 | :) | 19:49 |
n4nd0 | DEBUG_SGVECTOR is an option in configure? don't find it | 19:50 |
blackburn1 | no you should put it manually to lib/config.h if you want | 19:50 |
n4nd0 | all right | 19:50 |
n4nd0 | compiling now | 19:52 |
n4nd0 | we'll be back after dinner | 19:52 |
blackburn1 | yeah I will be back later too | 19:52 |
blackburn1 | haha | 20:41 |
CIA-52 | shogun: Sergey Lisitsyn master * r4520986 / src/shogun/lib/SGMatrix.h : Fixed SGMatrix constructor - http://git.io/z4McvQ | 20:42 |
blackburn1 | okay seems almost no leaks at all now? | 20:42 |
-!- blackburn1 [~blackburn@188.168.14.135] has quit [Quit: Leaving.] | 20:42 | |
@sonney2k | blackburn, blackburn2 or whoever - that kind of change is dangerous... | 20:47 |
@sonney2k | in particular since SGVector etc all don't do it by default too (and I assume that this is used in the code) | 20:49 |
blackburn2 | ooops | 20:54 |
-!- blackburn2 is now known as gsomix | 20:54 | |
@sonney2k | wtf? | 20:56 |
gsomix | sonney2k, good evening. how are you? :) | 20:57 |
@sonney2k | gsomix, fighting with the webserver | 21:12 |
gsomix | sonney2k, for new site? | 21:14 |
gsomix | ok, good luck with it :) | 21:27 |
@sonney2k | yes | 21:30 |
-!- vaio [~vaio@117.32.153.153] has joined #shogun | 22:27 | |
vaio | Is there anyone here? | 22:27 |
vaio | here ? | 22:27 |
vaio | I am doing some exps using shogun-toolbox | 22:28 |
-!- vaio [~vaio@117.32.153.153] has quit [Client Quit] | 22:29 | |
n4nd0 | hello vaio | 22:32 |
n4nd0 | maybe next time... | 22:32 |
-!- n4nd0 [~nando@s83-179-44-135.cust.tele2.se] has quit [Quit: leaving] | 23:03 | |
-!- hoijui [~hoijui@dslb-092-078-182-230.pools.arcor-ip.net] has quit [Quit: Leaving] | 23:15 | |
CIA-52 | shogun: Sergey Lisitsyn master * r25ba47c / (2 files in 2 dirs): Another way to avoid memleak with dense features serialization - http://git.io/lEHERg | 23:39 |
--- Log closed Fri Aug 31 00:00:17 2012 |
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!