Open in new window / Try shogun cloud
--- Log opened Mon Jul 24 00:00:08 2017
-!- olinguyen [81615ad9@gateway/web/freenode/ip.129.97.90.217] has joined #shogun00:01
-!- olinguyen [81615ad9@gateway/web/freenode/ip.129.97.90.217] has quit [Ping timeout: 260 seconds]00:07
-!- olinguyen [81615ad9@gateway/web/freenode/ip.129.97.90.217] has joined #shogun01:05
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3859 synchronized by MikeLing04:40
-!- olinguyen_ [ae73da84@gateway/web/freenode/ip.174.115.218.132] has joined #shogun05:45
-!- olinguyen_ [ae73da84@gateway/web/freenode/ip.174.115.218.132] has quit [Client Quit]05:46
-!- olinguyen_ [ae73da84@gateway/web/freenode/ip.174.115.218.132] has joined #shogun05:46
olinguyen_wiking: do you have a moment?05:46
@wikingolinguyenhi07:13
@wikingsorry just got around07:13
@wikingyes07:13
olinguyen_I'm starting to get involved more with the actual shogun code base. I'm having trouble compiling some stuff07:14
olinguyen_i'm running a docker instance as shown in the readmes07:15
olinguyen_when trying to compile 'make random_forest' i get the following error07:15
olinguyen_Translation of dependencies failed! Could not translate file multiclass_classifier/multiclass_logisticregression.sg to .cpp.07:15
olinguyen_Any idea what i could be doing wrong07:15
olinguyen_wiking, i'm on the develop branch, fyi07:16
@wikingmmm07:17
@wikingyou try to compile a meta example?07:17
olinguyen_I'm trying to compile the random forest target07:18
olinguyen_seems like the meta examples are part of the targets (or i might not be fully understanding whats happening)07:18
olinguyen_longer error message, could helpful: [  3%] Generating example multiclass_classifier-multiclass_logisticregression Translation of dependencies failed! Could not translate file multiclass_classifier/multiclass_logisticregression.sg to .cpp. Traceback (most recent call last):   File "/home/gsoc2017-shogun-dataproject/shogun/examples/meta/generator/generate.py", line 158, in <module>     generatedFilesOutputDir=args.parser_files_di07:20
olinguyen_Am I doing the right thing here? Since I'll be making changes on random forest, I only want to build the random_forest target. Am I missing a step or something?07:24
@wikingolinguyen     but that just an example08:02
@wikinghow to use random forest08:02
@wikingif you do changes in random forest08:02
@wikingnamely in the library08:02
@wikingyou will stll need to ocmpile shogun itself08:02
-!- mikeling [uid89706@gateway/web/irccloud.com/x-vnjdedvjrqtnzqrm] has joined #shogun08:06
olinguyen_wiking, my bad on that part08:11
olinguyen_When trying to make all, I get this error however08:11
olinguyen_fatal error: shogun/mathematics/JacobiEllipticFunctions.h: No such file or directory  #include <shogun/mathematics/JacobiEllipticFunctions.h>08:11
olinguyen_that files missing in the develop branch08:12
@wikingmmm08:13
@wikingdid u do git pull and git submodule update --init ?08:13
@wikingas some stuff08:14
@wikingmoved from the main repo08:14
@wikingto the gpl part08:14
@wikinginclind JacobiEllipticFunctions.h08:15
olinguyen_i forgot to run the submodule update again. I think it's fine now, thanks!08:19
-!- geektoni [~geektoni@93-34-87-36.ip49.fastwebnet.it] has joined #shogun09:12
-!- iglesiasg [~iglesiasg@217.119.234.214] has joined #shogun09:51
-!- mode/#shogun [+o iglesiasg] by ChanServ09:51
olinguyen_wiking, so even when compiling all i end up with the same error as before in the end09:51
olinguyen_[ 91%] Generating example multiclass_classifier-multiclass_logisticregression Translation of dependencies failed! Could not translate file multiclass_classifier/multiclass_logisticregression.sg to .cpp.09:52
olinguyen_should I just turn off the build w/ examples?09:52
olinguyen_turned off the examples, worked. sorry for bugging you :P, it is late here and haven't had much sleep09:54
-!- olinguyen_ [ae73da84@gateway/web/freenode/ip.174.115.218.132] has quit [Quit: Page closed]10:03
@sukey[https://github.com/shogun-toolbox/shogun] Issue https://github.com/shogun-toolbox/shogun/issues/3950 opened by Galaxy-Fish10:50
Trixiswiking: im testing the library you linked right now, fingers crossed12:31
@wikingfingerz x-ed12:32
-!- mikeling [uid89706@gateway/web/irccloud.com/x-vnjdedvjrqtnzqrm] has quit [Quit: Connection closed for inactivity]12:35
-!- geektoni [~geektoni@93-34-87-36.ip49.fastwebnet.it] has quit [Remote host closed the connection]12:38
Trixiswiking: hm it loaded the library just fine, but then i got unsatisfied link error when init() gets called... might be just because i have an older jar in classpath, ill try rebuilding with the one you supplied. also, "ldd: warning: you do not have execution permission for `./libshogun.so'" not sure if thats an issue or not12:56
@wikingthere's the jar12:56
@wikingno?12:56
Trixiswiking: my .jar contains the older (6.0.0 i think) shogun.jar12:58
Trixiswiking: ooh did  modshogun get renamed?13:05
@wikingi did it13:05
@wikingmanuall to shogun.so13:05
@wiking*manually13:05
Trixiswiking: no i mean in the sources, did the class get renamed13:06
@wikingyes13:06
Trixisbecause it's not in the jar anymore13:06
Trixisahh13:06
Trixiswhat's it called now?13:06
@wikingshogun13:06
Trixiskk13:06
Trixisthat explains the link error lol13:06
@wikingorg/shogun/IntronList.class13:07
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/3942 synchronized by micmn13:08
@sukey[https://github.com/shogun-toolbox/shogun] Pull Request https://github.com/shogun-toolbox/shogun/pull/394213:08
Trixiswell, it's in the queue13:13
Trixisfingers crossed13:13
Trixisnow it's running13:15
Trixisno crash so far13:15
Trixislooks promising13:15
Trixiswiking: thanks, it's working!13:23
@wikinglol cool13:24
@wiking:)13:24
@wikingit was easy13:24
@wiking:>13:24
Trixishaha13:24
Trixisi mean the job crashed because of a different error, but this seems to be an issue on my side... or the way you construct binary labels changed between versions, lol13:25
Trixiswiking: how bad are these warnings? "libgomp: Thread creation failed: Resource temporarily unavailable" ?13:43
@wikingoh13:44
@wikingwtf13:44
Trixisare they only going to slow performance13:44
Trixisor are they going to influence the integrity of the run?13:44
Trixisoh they crashed it13:45
Trixislovely13:45
Trixislo13:45
Trixisl13:45
Trixiswiking: seems to be a cluster issue i guess13:46
@wikingmmm13:49
@wikingi never ever seen these errors13:49
@wikinghttps://github.com/sequencing/isaac_aligner/issues/1013:49
@wikingi guess you ahve some hard limits on the cluster13:49
@wikingyou can play with the number of threads13:50
@wikingwith the env var SHOGUN_NUM_THREADS=<N>13:50
@wikingbut i think this is more about runtime speed13:50
@wikingnothing else13:50
Trixisye apparently it happens if the machine allocated has the number of cores requested but the job gets assigned only a limited amount13:50
Trixisoshit segfault13:52
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has joined #shogun14:01
-!- mode/#shogun [+o HeikoS] by ChanServ14:01
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has quit [Quit: Leaving.]14:24
@wikingTrixis, does it work or not really ? :)14:38
Trixisnot really14:38
Trixisi fixed the segfaults due to the omp error14:38
Trixisby setting the number of threads to match requested cores14:38
Trixisbut14:38
Trixisnow im getting label errors - apparently they're invalid due to only containing one class. Except, they are initialised with two classes and this error didnt occur when i tested it locally on a small dataset :\14:40
@wiking:P14:44
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has joined #shogun14:44
-!- mode/#shogun [+o HeikoS] by ChanServ14:44
@wikingok but this is more like error with the logic14:44
@wikingbut shogun itself work14:45
@wikings14:45
@wikingso that is a great news :)14:45
Trixisye14:46
Trixishaha14:46
Trixisi hope it's an error with logic, perhaps i changed something unwittingly14:46
Trixisill retest it locally once shogun finishes rebuilding (snappy got updated, and shogun was linked against prev version -.-)14:48
-!- geektoni [~geektoni@93-34-128-39.ip49.fastwebnet.it] has joined #shogun14:54
geektoniping HeikoS14:55
Trixiswiking: seems to be an issue with the new version of shogun, do you know where i can find what changes have been done to labels/binary labels?15:10
@wikingTrixis, u can simply take that verison15:11
@wikingand change only 1 line in cmake15:11
Trixishmm15:11
@wikingto generate the same thing15:11
@wikingthat i gave you15:11
@wikingso then you dont have problems with the labels etc15:12
Trixisi also need the centos utils, though15:12
@wikingheheeh yeah but you can do all that15:12
@wikingon your machine + centos docker image15:12
Trixistrue15:12
@wikingin the docker image you build the thing15:12
@wikingand then take it15:12
@wikingthat's what i did15:13
@wikingthis is the diff for the cmake https://pastebin.com/q4M2xFeK15:13
Trixisactually, i might run down to the IT and get them to install the devtools15:13
@wikingsimple stuff15:13
Trixisyah i just request the install15:34
Trixisnow i just have to wait a bit15:34
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has quit [Ping timeout: 260 seconds]15:50
-!- geektoni [~geektoni@93-34-128-39.ip49.fastwebnet.it] has quit [Quit: Leaving.]15:52
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has joined #shogun15:59
-!- mode/#shogun [+o HeikoS] by ChanServ15:59
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has quit [Client Quit]16:03
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has joined #shogun16:03
-!- mode/#shogun [+o HeikoS] by ChanServ16:03
Trixiswiking: what exactly do the devtools-2 do in the context of the static linking?16:38
@HeikoSwiking: you around?16:48
-!- HeikoS [~heiko@eduroam-int-pat-8-214.ucl.ac.uk] has quit [Quit: Leaving.]16:51
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun17:16
-!- mode/#shogun [+o HeikoS] by ChanServ17:16
-!- iglesiasg [~iglesiasg@217.119.234.214] has quit [Quit: leaving]17:28
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 260 seconds]18:13
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has joined #shogun18:19
-!- geektoni [~geektoni@93-34-234-212.ip52.fastwebnet.it] has quit [Remote host closed the connection]19:13
@wikingTrixis, allows you to compile shogun on rhel620:44
Trixiswiking: ah so its not just the gcc thats important inside20:45
@wikingas otherwise i dont know how else u could compile c++11 in rhel20:45
@wikingnono it's only that20:45
@wikingthat you have a gcc that is newer20:45
@wikingso u can compile20:45
Trixishuh. because i tried compiling it with the static flag / edited cmake, + i used module load gcc 4.9.120:45
Trixisbut20:45
Trixisit doesnt seem to be linked correctly20:46
Trixisbecause running ldd shows a lot of libraries and its throwing an error that it cant find suitable version of libompg ... except its searching at the site of the old gcc20:46
Trixisnot the 4.9.1 its linked against ( as confirmed by gcc)20:47
@wikingTrixis, you ran ldd on?21:48
Trixiswiking: src/interfaces/libmodshogun.so21:48
Trixisthe one i compiled with the static flag21:48
@wikingwhat was the ldd output of that?21:49
Trixiswiking: https://pastebin.com/SSU5yBET21:50
@wikingTrixis, ok but first of all ldd will take what your ldcache gives21:52
@wikingldd is looking for libgomp.so.121:52
@wikingso if /sw/opt/gcc-4.9.3/lib64/ is not in ldpath21:52
@wikingthen it wont find it21:52
@wikingand will default to something else21:52
Trixisit was in ld_library_path21:52
@wikingthats why ldd uses taht21:53
Trixishm21:54
@wikingbut if you put this .so21:54
@wikingto a machine where tehre's no  /sw/opt/gcc-4.9.3/lib621:54
@wikingthen it'll use soemthing else21:54
@wikingif libgomp.so.1 is available21:54
Trixiswiking: hence it should find it when i run it, no?21:54
@wikingyes21:55
@wikingusing your env setup21:55
@wikingthose libs21:55
@wikingare loaded during runtime21:55
Trixisexcept it picks up the gcc in usr/lib21:55
Trixis"|21:55
@wikingif avilable21:55
Trixis*:|21:55
@wikingyeah but that's what it should do21:55
@wikingno?21:55
Trixisshouldnt it use the /sw/opt/ gcc?21:56
@wikingi mean it's requiring21:56
@wikinglibgomp.so.121:56
@wikingwhich is a verison of libgomp21:56
@wikingif /usr/lib has that21:56
@wikingit should be fine21:56
Trixisyeah except the one there is outdated as fuck21:57
Trixisso it throws an error that minimum req. version couldnt be found21:57
Trixisalso if i static linked it21:57
Trixisshould it still be doing this?21:57
@wikingwell21:58
@wikingbut yeah21:59
@wikinggood question21:59
@wikingwhy gomp is still .so there21:59
Trixiswiking: "[ 77%] Linking CXX shared module libmodshogun.so"22:00
Trixiscompare to22:01
@wikingok i guess22:01
Trixis[ 77%] Linking CXX static library libshogun.a22:01
@wikingcmake flags for static linking22:01
@wikingis not yet 100%22:01
@wikingcan u run ldd on22:02
@wikingthe so i have given u?22:02
Trixiscmake -DCMAKE_INSTALL_PREFIX=$HOME -DJavaModular=ON -DBUILD_META_EXAMPLES=OFF -DENABLE_HDF5=OFF -DLIBSHOGUN_BUILD_STATIC=ON ..  was the cmake command22:02
Trixiswiking: i did that and the output was like 4 lines22:02
Trixissec if i can find it in terminal history22:03
Trixishm cant find it, the point is however that libompg wasnt there definitely22:05
Trixisin fact22:05
Trixismost of those werent there22:05
@wikingok22:08
@wikingthen i guess in this caeeeeee22:08
@wikingcase22:08
@wikinggomp is pulled in by openmp22:08
@wikingi mean22:08
@wikingby openblass22:08
@wiking*openblas22:08
@wikingi need to sleep22:09
@wikingits 4am22:09
@wikingsorry man22:09
@wikingyou should minimize your external dependencies in shogun22:09
@wikingturn off blas etc22:09
Trixisits ok22:12
Trixiskk22:12
Trixisi will22:12
Trixisthanks a lot22:12
Trixissleep well22:13
--- Log closed Tue Jul 25 00:00:10 2017