[01:54:38] scfc_de: Coren how do I find out how much memory my job used before it was killed? [01:54:54] Coren: also, are subprocesses counted? :) [02:11:45] YuviPanda: qacct => max_vmem. [02:24:33] scfc_de: cool, found my issue [02:25:03] scfc_de: which was that I was doing 'jsub script -mem 4G' instead of 'jsub -mem 4G script' [03:37:44] YuviPanda: And yes, subprocesses are counted. [14:12:11] Coren: cool [14:28:33] valhallasw: around and want to help me test something? [14:28:38] YuviPanda: sure [14:29:01] valhallasw: step 1. clone https://github.com/yuvipanda/labsbooks, 2. install paramoko, 3. run start.py [14:29:37] paramiko [14:29:50] pycryyyyyyyyyyypto [14:29:55] ok [14:29:58] vim start.py :-p [14:30:06] valhallasw: you can do that first too, yeah [14:30:20] ssh.connect('tools-dev.wmflabs.org', username=labs_config['user'], key_filename=labs_config['identityfile']) [14:30:23] KeyError: 'user' [14:30:24] valhallasw: this enables easy (and secure) usage of ipython notebooks on toollabs :) I'm working on makign t hem easily publishable [14:30:45] valhallasw: bah, you don't have username set in your .ssh/config? :) [14:30:55] no, because it's the same as on my local VM :P [14:31:03] valhallasw: can you just replace username== your actual name and see if it runs? :) [14:31:05] *and* I don't have tools-dev [14:31:19] valhallasw: that's fine, that should be handled by SSHConfig [14:31:33] do I need to set IdentityFile? [14:31:43] it seems I do [14:31:45] I have an agent ;p [14:32:09] valhallasw: hmm, so you don't actually have the key in the VM at all [14:32:16] * YuviPanda ponders how to deal with *that* [14:32:27] I do have it in the VM [14:32:32] job_num, job_name = JSUB_OUTPUT_REGEX.match(jsub_output).groups() [14:32:32] AttributeError: 'NoneType' object has no attribute 'groups' [14:32:47] just don't pass anything, I guess? [14:33:01] valhallasw: that doesn't quite work well, I think. I shall figure it out [14:33:11] YuviPanda: it does [14:33:16] valhallasw: oh! [14:33:18] sh.connect('tools-dev.wmflabs.org') [14:33:19] heh, ok! [14:33:48] valhallasw: heh, not for me! (authentication fails), but I'll debug [14:34:01] valhallasw: hmm, so you're connected but jsub_output regex fails... [14:34:20] valhallasw: do you quiet jsub by default with any means? [14:34:38] valhallasw: also can you print value of jsub_output? [14:34:46] (Pdb) _.read() [14:34:47] '/home/valhallasw/run.bash: No such file or directory\n' [14:34:48] :-p [14:34:55] valhallasw: aah, pull again :) [14:34:55] *maybe* you should also read stderr [14:36:04] valhallasw: pfft, my code never fails :P [14:36:34] valhallasw: if start.py seems to work alright, go to localhost:9500 [14:36:56] ERROR:paramiko.transport:Exception: Error reading SSH protocol banner [14:37:03] in the second ssh [14:37:46] hmm, bah [14:37:56] not sure how to debug that... [14:38:27] valhallasw: that just means that an ssh connection to tools-exec-whatever.eqiad.wmflabs is failing [14:38:30] despite the proxycommand [14:39:00] The RSA host key for tools-dev.wmflabs.org has changed, [14:39:02] ... [14:39:13] then how did the first connection work O_o [14:39:36] valhallasw: auto_add_policy [14:39:41] ? [14:40:19] YuviPanda: even if it got added automatically in the first connection, why doesn't the second connection work? The key wouldn't have changed [14:40:25] valhallasw: it ignores hostkey warnings for now :) I'll hard code in the RSA value. [14:40:37] valhallasw: ah, because the second one relies on your local 'ssh' command as well as paramiko's ssh [14:40:43] ohhh [14:40:51] ok second connection now works [14:40:58] valhallasw: w00t. go to localhost:9500 [14:41:30] sloooooooooow [14:41:42] an ancient ipython notebook appears!@ [14:41:49] yeah, the proxying leaves a lot to be desired [14:41:52] valhallasw: 'ancient'? [14:42:04] also yay, an ipython notebook appears :) [14:42:09] YuviPanda: yes, ancient. this is like pre-1.0 [14:42:16] valhallasw: what, no. this is 2.2 [14:42:29] also I'm in '/home/yuvipanda/notebooks' [14:42:36] what lol [14:42:49] you should've been in your own account /notbooks [14:42:53] and I can WRITE there, apparently [14:43:03] 'notebook saved' [14:43:09] did I connect to your notebook? :'D [14:43:41] valhallasw: hahaha you did [14:43:44] wtf [14:43:55] so much for 'secure' [14:44:15] YuviPanda: hrm. So I guess I'm the one with an ancient ipython notebook :-p [14:44:29] which is super weird, becasue that means anaconda is out of date more than ubuntu [14:44:46] valhallasw: has your instance stopped now? [14:45:13] well, I don't see a notebook anymore [14:45:21] and paramiko is trying to reconnect frantically [14:45:34] valhallasw: right [14:45:38] valhallasw: run start.py again now [14:45:43] as in, close and run [14:45:44] again [14:46:14] valhallasw: I haven't implemented port randomizing yet, and 'by chance' both our things were scheduled in exec-09, and yours kinda failed and you connected to mine :D [14:46:21] I shall set passwords and randomize the ports [14:46:34] :-p [14:47:48] YuviPanda: doesn't work? [14:47:57] valhallasw: what does it say? [14:48:12] Re-using existing kernel [14:48:16] ssh -e none tools-dev.wmflabs.org exec nc -w 3600 tools-exec-10.eqiad.wmflabs 22 [14:48:20] yeah [14:48:25] that's fine [14:48:31] oh, there's a job in qstat now [14:48:32] valhallasw: and when you go to localhost:9500? [14:48:38] and now I can connect [14:48:39] good [14:48:44] cool :) [14:48:47] I was probably too impatient [14:48:50] valhallasw: is this also 'ancient'? [14:48:58] valhallasw: yeah, I need to print more informative messages [14:49:01] yeah, it's also the 2.2 [14:49:04] this has had like 3h of work put into it [14:49:13] valhallasw: yeah, that's the last released version :D no 3.x release yet [14:49:26] yeah [14:49:42] I'm confused :P because as mentioned, anaconda apparently still has 1.x [14:50:12] valhallasw: right, so things I need to fix is the sshconfig issue, and potential hostkey problems for the second tunnel, port randomizing, and a password [14:50:45] valhallasw: ah, right :D I guess anaconda hasn't upgraded [14:51:30] but even with ipnb 1.1, anaconda >>>>>> whatever ubuntu does [14:51:40] because anaconda actually has the packages I want :-p [14:51:59] valhallasw: ah, right. what packages should I install? [14:52:14] oh, I doubt any of them would be relevant for TL :-p [14:52:16] but er [14:52:54] numpy is there, I'll add scipy. installing matplotlib now [14:53:01] I think ubuntu doesn't have pytables, or at least not the version I needed [14:53:10] but that's not something one would typically need on TL [14:53:12] valhallasw: this is on pip :) [14:53:25] HAHAHAHA yeah good luck with that [14:53:27] valhallasw: everything runs from a virtualenv [14:53:29] valhallasw: why? [14:53:36] you want to compile numpy yourself? [14:53:46] do you like FORTRAN? [14:53:59] do you like linking C, Python and FORTRAN together? [14:54:08] then please don't compile numpy yourself :-p [14:54:20] pytables is the same mess, but with hdf5 stuff, which is marginally better [14:55:07] although I think it's only the linear algebra stuff that's hopeless [14:55:16] so maybe pip just skips that, or installs non-optimized versions [14:55:24] valhallasw: :D I am trying to install the whole deal [14:55:45] instead of getting whatever version of BLAS to work [14:56:24] valhallasw: numpy is compiling now [14:56:27] let's see if it fails [14:56:51] YuviPanda: it also takes a gazillion years to compile, but that's OK, I guess :-p [14:56:58] :D [14:57:10] (but I have to admit -- last time I compiled it was 1.5 years ago, so maybe the situation has improved) [14:58:24] valhallasw: numpy is installed, can you see if anything obvious is missing? [14:58:40] YuviPanda: errr [14:58:48] am doing scipy now [14:58:54] YuviPanda: not sure what to test for [14:59:01] if it compiles, it will at least work [14:59:16] \o/ [14:59:20] now scipy and pandas [14:59:23] will do pytable after [15:03:34] valhallasw: scipy does take forever to build [15:03:43] ohh, wait [15:03:49] maybe it's scipy that has the linear algebra stuff [15:04:18] valhallasw: it is [15:04:24] YuviPanda: anyway, I'll push the notebook that generates the graphs for my paper through at some point [15:04:28] valhallasw: numpy has minimal portran [15:04:28] see if it works [15:04:30] valhallasw: \o/ [15:04:50] YuviPanda: hmm, is LaTeX installed? :-p [15:04:51] valhallasw: you can just put them in your home ~/notebooks folder and that should be picked up, I think [15:04:55] valhallasw: it can be ;) [15:05:17] http://arxiv.org/pdf/1404.3156v2.pdf < graphs <3 [15:05:23] \o/ [15:05:51] valhallasw: I read the abstract, couldn't parse :) [15:05:55] but yeah, they need LaTeX and the txfonts package [15:06:10] YuviPanda: yeah. Sounds about right :-p [15:06:58] YuviPanda: fig 1(a) and (b) are the clearest, I think. We shear a 'soft harmonic particle packing' = basically 2d system of foam bubbles and we try to find when two bubbles make or break a contact [15:07:07] aha! [15:07:15] the physics of 2D bubble wrap?! [15:07:20] LOL [15:07:29] pop-pop! [15:07:36] :D [15:07:37] [/community reference] [15:07:46] the bubbles need to be able to move, though [15:07:55] 2D liquid bubble wrap! [15:08:04] well, 2D fluid bubble wrap! [15:08:10] but yeah, my sister writes much more readable papers [15:08:38] "We performed a cross-sectional study analyzing U.S. health care utilization in 964,633 patients with IBD between 2010 and 2012 using insurance claims data, including pharmacy and medical claims." [15:09:03] which is awesome, as it's about 60% of all IBD (=Inflammatory Bowel Diseases) patients in the US [15:10:16] :D [15:10:17] niice [15:10:26] she knows more SQL than I do :P [15:11:02] :D [15:11:08] * YuviPanda knows almost no SQL [15:12:55] valhallasw: I guess installing texlive package should be enough [15:12:58] or even texlive-full [15:13:33] yeah [15:13:43] * YuviPanda makes patch [15:15:13] valhallasw: scipy installed! [15:15:29] :O [15:16:12] valhallasw: ;) [15:18:05] well, then the world has improved [15:18:08] that's good! [15:18:16] valhallasw: :D [15:21:50] valhallasw: pytables installed! [15:22:38] valhallasw: wanna put the notebook with your paper's graphs on and see what fails? [15:23:00] YuviPanda: I have to git clone some private repos anyway, and I have to scp a data set I don't have at home [15:23:15] valhallasw: aaah, ok. [15:23:18] I shall wait [15:23:23] and add security things in the meantime [15:24:44] valhallasw: can you stop your kernel by doing a 'jstop start'? :) [15:24:48] I should code that in somewhere [15:24:50] unsure where [15:24:52] *where [15:27:40] YuviPanda: yeah, that's what I did [15:27:46] YuviPanda: but you don't want that [15:27:50] that breaks cleanup routines [15:28:11] you want to send SIGINT, not SIGKILL [15:28:28] valhallasw: yeah, but am unsure how to setup the kill. [15:28:43] valhallasw: since you'd want your kernel to be persistent and not die when your tunnel dies [15:28:46] YuviPanda: ssh kill SIGKILL pid? [15:28:51] right [15:28:57] right [15:29:02] valhallasw: but *when* is the question [15:29:03] just have a stop.py to stop? [15:29:10] yeah [15:29:12] might do that [15:29:19] I need to cleanup this code anyway [15:29:21] probably [15:29:24] labsbooks start [15:29:26] labsbooks stop [15:29:42] EC2 is 5 cents per CPU per hour... now I just need to know how much it does per hour :-p [15:30:04] valhallasw: I have data on 'edits per country' for all projects for 30days. I want to anonymize it enough, then put it onlabsdb, and then use ipython to generate graphs :) [15:30:10] ah nice [15:30:31] I've never tried to use mysql in numpy [15:30:38] with* [15:30:47] I hopefully don't have to use numpy [15:31:07] http://stackoverflow.com/a/18258173/118671 looks good [15:31:16] you don't want to use list comprehensions [15:31:39] well, depends on the data maybe [15:31:47] but most things are more easily expressed in numpy operators [15:32:59] valhallasw: yeah [15:33:04] valhallasw: I haven't used numpy for realz anywhere [15:33:21] valhallasw: although there are only about 200 countries anyway [18:18:51] Hi helderwiki. I noticed you have a local copy of TemplateScript on ptwikibooks: https://pt.wikibooks.org/wiki/Utilizador:Helder.wiki/Tools/Pathoschild/TemplateScript.js. Seems to be the same code (just a bit outdated). Was that just to avoid loading it from GitHub? [18:19:10] yep! [18:19:19] it was causing warnings [18:19:39] and was a little slow at some point (it got worse when it was migrated to labs) [18:19:46] so I just restored my fork [18:20:46] Hmm. It shouldn't be slow on Labs; if it still is we should look into why, for everyone else using it. :) [18:21:12] What I use the most is the regex tool, and it was taking too long to load [18:22:31] It seems to be properly cached on Tool Labs, so there should be almost zero load time. Maybe it's faster than it was when you last tried it? [18:26:06] You can maintain a local copy if you prefer, but you (and others using it) won't get any of the updates. :) [18:29:51] of course, the advantage is Pathoschild cannot take over your account ;p [18:31:27] valhallasw: my plans are foiled. ;) [18:44:32] Pathoschild: I'll probably try the labs version again once I migrate my (global) scripts to meta, and if I don't forget I'll tell you what happens... [18:44:39] Pathoschild: it does make me wonder what kind of user jses WMF staff have :-p [18:50:19] valhallasw: I haven't seen many WMF staff with user scripts. :p [18:50:46] Pathoschild: https://meta.wikimedia.org/w/index.php?title=Special%3ASearch&profile=default&search=User%3A*%28WMF%29%2F*.js&fulltext=Search [18:50:51] meta disagrees :-p [18:52:38] Aww, and none of them use my scripts. :p [19:10:32] Pathoschild: if you responded, I missed it as my internet died [19:11:36] "Aww, and none of them use my scripts." :p [19:11:40] :D [19:12:33] no staff accounts for you! [19:18:46] valhallasw: I'm trying to setup a python fcgi server following gerrit-patch-uploader [19:18:57] valhallasw: and I keep getting 404s :( any idea how to figure out wtf is happening? [19:22:17] YuviPanda: what does error.log say? did you turn on extended logging? [19:22:25] YuviPanda: also which project :P [19:22:32] valhallasw: 'notebooks' :) [19:22:37] valhallasw: error log has nothing useful [19:22:48] valhallasw: how do I turn on extended debugging? :) [19:22:53] YuviPanda: ...where's your .lighttpd.conf? [19:22:58] oh [19:23:02] name it DOT lighttpd.conf [19:23:05] not lighttpd.conf :P [19:23:17] valhallasw: https://github.com/yuvipanda/labsbooks/blob/test/lighttpd.conf [19:23:30] yeah that looks OK [19:23:36] just mv lighttpd.conf .lighttpd.conf [19:23:43] * YuviPanda facepalkms [19:23:54] * valhallasw hands YuviPanda a palm to face in [19:25:53] valhallasw: hmm, so now I get debugging, but it just seems to look in public_html [19:32:45] YuviPanda: mmh. it seems gerrit-patch-uploaders log has the same :/ [19:33:22] it should show something like [19:33:23] 2014-08-24 19:32:58: (response.c.350) URI-path : /gerrit-patch-uploader/login [19:33:26] 2014-08-24 19:32:58: (mod_access.c.135) -- mod_access_uri_handler called [19:33:29] 2014-08-24 19:32:58: (mod_fastcgi.c.3636) handling it in mod_fastcgi [19:35:22] YuviPanda: env/error.log does get the requests [19:35:27] so they are handled by flask [20:29:11] valhallasw`cloud: hmm, was away [20:29:23] valhallasw: it does seem to, except still http://tools.wmflabs.org/notebooks/yuvipanda/Untitled0 [20:33:16] YuviPanda: what's wrong with that? [20:33:33] valhallasw: nevermind, it just worked. I was being stupid in routing [20:33:38] :D [20:39:30] valhallasw:YUSSSS http://tools.wmflabs.org/notebooks/yuvipanda/Untitled0 [20:48:40] YuviPanda: fancy [20:48:53] can I take over ~notebooks with that? >:) [20:53:35] valhallasw: it's just nbconvert, I don't think you can run code from there [20:54:07] YuviPanda: mmm, probably [20:54:45] ok, time for bed [20:54:47] good night [20:55:19] valhallasw: night