[06:46:49] I am trying to use connect to labs sql replicas using python over ssh [06:48:10] in python i have to specify the host, which because this is a tunnel i believe should be 127.0.0.1 [06:48:50] but i must also specify the path to the mysql db relative to the host. how do i append that to the host string [06:54:16] OperationalError: (2005, "Unknown MySQL server host 'enwiki.labsdb' (2)") [16:40:53] I'm having trouble connecting to one of my instances. "Connection to UNKNOWN timed out while waiting to read". Connections to bastion and other instances work great. Can anyone help? [16:42:34] Could this be related to the migration? These vms are on pmtpa. [16:45:43] meh. Nevermind. I was still setting up the instance anyway, so I decided to just start over with a new instance. [16:55:48] Fun story, fresh VM has the same issue. Still can log into other project instances and bastion. [16:57:00] halfak: The migration won't be for ... weeks probably :-). What does "ssh -v" say? [16:57:13] Good question. Sec. [16:58:23] (If it's a timeout, this might also be one of those Gluster hiccups. You would then need to ping one of the Labs admins to fix that.) [16:59:56] http://pastebin.com/xjsmAKhS [17:00:06] ^ The next line will be the timeout in a couple of minutes. [17:02:12] scfc_de: ^ see pastebin above. Any ideas? [17:04:18] could be NFS issues -- sshd tries to read your public key list and that read hangs [17:04:19] halfak: I *think* sshd is stuck trying to verify your key against /public/keys/..., but you need some Labs admin to confirm that. andrewbogott_afk, Ryan_Lane: ^ [17:04:35] Thanks :) [17:04:54] valhallasw: I think /public/keys is actually GlusterFS (apart from Tools, I believe). [17:09:47] Any hypotheses about me being able to connect to other instances in the same project? [17:11:58] halfak: No, outside my scope. Just curious: What does "mount | fgrep public" say on them? [17:13:07] uh oh. Same problem on my other instances now. [17:13:27] * halfak should have left himself logged in. [17:13:35] :-) [17:14:50] Keys is nfs backed by gluster iirc [18:16:10] Hey guys. Sorry I got booted due to a random power outage. [18:16:19] Any updates on potential NFS issues? [18:16:34] NFS on labs, halfak? [18:16:39] Yup. [18:16:51] I was asking about issues earlier today. [18:17:27] Here's a paste of "ssh -v " http://pastebin.com/xjsmAKhS [18:17:43] After that last line, I'll time out. [18:18:13] scfc_de was speculating that it could be an NFS issue related to reading keys. [18:19:33] halfak: I had other weird issues [18:20:36] Gotcha. Any chance we'll see someone who has the power to check things out online this weekend? I usually do my work on labs during the weekdays. [18:21:02] halfak: probably not, sadly :( coren and andrewbogot are the only folks doing labs stuff these days... [18:21:39] Gotcha. That's a bummer. [18:21:45] halfak: yeah [18:22:07] halfak: I should setup a vagrant role that lets you access labsdb, I think. should make development easier [18:24:13] What's in labsdb? [18:25:33] halfak: hmm? [18:25:37] halfak: readonly mysql replicas? [18:25:48] Oh! Yes. That sounds like a good idea. [18:25:51] :) [18:25:56] halfak: yeah, and also not too hard... [18:26:01] I know nettrom would like it too. [18:26:33] halfak: yeah. I've done it manually before [18:26:37] maybe a blog post wou7ld be nice as well [18:29:59] FWIW, I'll have a use-case for that immediately. [19:37:11] anyone tools person around to add tlevine to the project? [19:37:13] https://wikitech.wikimedia.org/wiki/Special:UserRights/Tlevine [19:37:27] jeremyb: sure [19:38:37] jeremyb: done and done [19:38:43] danke**2 [19:38:53] yw [21:48:35] YuviPanda: Ping [21:48:45] TParis: pong [22:24:27] Does anyone here know about the relations between wsgi and Labs? [22:27:14] There isn't any really - you can run apps under lighty on tools if you want [22:29:04] Documentation on the wiki appears to be lacking [22:31:00] You've got a whole lighttpd server to do anything you want with, read their documentation [22:35:03] H [22:35:06] ah [22:35:16] Damianz: there are docs under the new web section [22:35:20] er, SigmaWP [22:35:35] https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help/NewWeb#Example_FCGI_configuration [22:35:53] and then you use flup.server.fcgi as WSGI handler [22:35:58] * Damianz pats valhallasw [22:36:12] Interesting [22:36:14] Thanks [22:37:09] SigmaWP, thank you. [22:37:27] Cyberpower678: You're welcome [22:37:32] What did I do? [22:37:53] Lifted a job off my immediate todo list. [22:38:11] Ah [22:42:43] Ryan_Lane: Are you available for a minute? [22:46:18] TParis: sup? [22:48:01] nevermind, I figured it out, thanks for responding [22:48:23] was trying to connect to SSH through the HTTPS proxy and I realized that I needed the public IP [22:52:50] ah [22:52:50] heh [23:24:35] Damn, I broke something. [23:26:25] SigmaWP: check your ~/error.log? ;-) [23:26:53] valhallasw: webservice stop, and now I have 503 errors :L [23:27:02] http://tools.wmflabs.org/sigma/foo.py?u=1 [23:27:17] Hmm [23:27:24] Maybe I should make it use cgi first >_> [23:27:55] SigmaWP: let me guess: it's not chmod +x? :-) [23:28:15] although I'm not sure why you would want to stop using lighttpd [23:28:35] It looks pretty annoying :\ [23:29:22] "REQUEST_METHOD=get ./foo.py" works fine [23:29:26] Damn. [23:30:30] lighttpd is far less annoying than apache, but still I'd rather use nginx+gunicorn [23:33:38] Meh [23:33:52] Hey, I broke my entire public_html! [23:44:19] I have been forced to revert to lighttpd >_> [23:44:26] * SigmaWP thanks valhallasw and Damianz [23:45:03] You could just fix your public_html for apache, but lightty is better for wsgi [23:52:50] Meh