[00:18:59] 07/20/2012 - 00:18:59 - User jdlrobson may have been modified in LDAP or locally, updating key in project(s): mobile,bastion [00:19:21] 07/20/2012 - 00:19:21 - Updating keys for jdlrobson at /export/keys/jdlrobson [05:49:58] 07/20/2012 - 05:49:58 - User ori may have been modified in LDAP or locally, updating key in project(s): bastion,editor-engagement,visualeditor [05:50:20] 07/20/2012 - 05:50:20 - Updating keys for ori at /export/keys/ori [08:10:41] !log deployment-prep rebooting deployment-cache-upload03, stalled because of the recent metadata server issue [08:11:59] puppet not configured to start, please edit /etc/default/puppet to enable [08:12:01] bahhaha [08:23:37] oh? [08:27:32] paravoid: that on the first run [08:27:42] but then puppet is forced to start :) [08:27:44] false alarm! [08:28:22] paravoid: also on a fresh instance, the first manual run still install some changes. [08:29:26] paravoid: http://imgur.com/DOv64 [08:29:35] that is a manual run done just after the first one [16:32:51] !log deployment-prep running upgrade on upload-cache03 [16:33:39] !log deployment-prep and running dist-ugprade + reboot [16:33:44] bah the bot is dead again [16:33:51] I give up :-D [16:33:54] byebye! [17:28:09] can someone allocate editor engagement a public ip? ryan did it yesterday but the server 500'd when i tried to allocate it to an instance and now it's lost. [17:36:38] ori-l: what's the project and instance name? [17:39:39] actually, ori-I and/or ori-l, when I ping ee-prototype.wmflabs.org it resolves to 208.80.153.212 so you should be in business. [17:46:49] ori-l: I need to step away; check in with me when I'm back from lunch if you still have questions. [18:01:13] !log taking down gerrit [18:08:01] \o/ [18:08:17] gonna replace it with something more user friendly? ;-) [18:08:52] Making it fasterish [18:08:55] <^demon> No, just give it a better database server to live on :) [18:09:15] andrewbogott: ee is editor engagement, we're editor engagement experimentations (yes, it's silly). [18:10:03] andrewbogott: ryan bundled us together because creating projects on labs was complicated [18:10:17] well, it's not complicated [18:10:28] it's just that it makes sense to bundle the ee people together [18:10:32] you're all in the same group [18:11:34] <^demon> Who could be surprised that we'd name something so poorly? [18:11:48] <^demon> We did create Wikipedia/Wikimedia/MediaWiki. [18:12:27] WP::WM::MW :D [18:12:44] heya Ryan [18:13:30] paravoid: howdy [18:38:41] Ryan_Lane: does "Failed to allocate new public IP address. " mean we don't have one allocated, or does it indicate some other failure? [18:40:08] likely that your quota has been reached [18:40:11] which by default is 0 [18:40:14] you need to request one [18:41:16] Ryan_Lane: i did, and you allotted one, but i couldn't allocate it because the server 500'd [18:41:21] Ryan_Lane: and now it appears to have vanished [18:42:59] it 500'd? [18:43:29] when did this happen? [18:43:39] yesterday. i have witnesses :) [18:43:43] ori-l: you should definitely let me know if that happens [18:43:47] ori-l: Did you catch the bit where I said that ee-prototype.wmflabs.org resolves to a proper ip? [18:44:01] andrewbogott: yes, it's a different machine, used by ee for different purposes. [18:44:07] Oh, ok. [18:44:41] this is the ee project? [18:44:53] Ryan_Lane: on labs, yes [18:45:00] it says floating_ips 0 [18:45:08] the exact name is ee? [18:45:19] Ryan_Lane: dunno, let me check [18:45:37] Ryan_Lane: "Editor-engagement" [18:45:48] ok. 1 assigned [18:46:12] there's still 3 ips available in the created ips [18:46:31] hm [18:46:32] Ryan_Lane: much obliged. and duly noted re reporting the error -- i meant to but we were in the midst of a lot of things [18:46:39] I'm getting a 500 right now on addresses [18:47:27] describe instances is erroring out [18:47:46] we really need to upgrade nova [18:47:55] paravoid: so, bad news [18:47:57] Ryan_Lane: it's consistent with the foundation-wide inability to describe E3 [18:47:59] they deleted the ppa we were using [18:48:06] ori-l: heh [18:48:55] this describe instances bug was fixed in a newer version of diablo [18:49:00] which wasn't in the ppa [18:49:05] * Ryan_Lane sighs [18:49:30] lol [18:49:46] Sorta why ppas are useless for anyone but yourself [18:50:13] well, it would have been fine if they didn't decide in the middle of a release to stop using the ppa [18:50:46] I can't even fucking load the addresses page [18:51:25] ori-l: so [18:51:31] you have an address allocated and associated [18:51:38] you only have 1 address in your quota [18:51:43] ori-l: did you need another one? [18:51:57] Ryan_Lane: no -- 1 is exactly right. thanks a bunch [18:52:10] it's associated with ee-prototype.wmflabs.org [18:52:34] * andrewbogott waits for punchline [18:52:40] * ori-l cries. [18:52:45] so, 1 IP doesn't seem accurate [18:52:48] when you said "you have an address allocated and associated" [18:52:51] you need another one, correct? [18:52:55] i thought you meant you had just assigned and allocated one [18:53:12] yes, i should have been clearer. one *more*. [18:53:13] no [18:53:55] ori-l: ok, you can allocate and associate it now [18:54:03] Ryan_Lane: sweet, thanks [18:54:06] keep trying if you get 500 errors [18:54:11] it's due to a bug in nova [18:54:28] I don't know if I should track it down and fix it or concentrate on upgrading :( [18:54:45] paravoid: sooooooo, upgrade is now the priority [18:54:46] Ryan_Lane: can you also set up a dns record for e3.wmflabs or add me to the cloud admin group? [18:55:00] seems we're now hitting the bugs I've been told about from the nova people [18:55:01] I:/ [18:55:16] Ryan_Lane: If you can survive Diablo for another 60 days then we can leapfrog to folsom… [18:55:23] andrewbogott: cant :( [18:55:44] Yeah, diablo seems to be crumbling rapidly. [18:55:50] paravoid: when I told the nova people we were using diablo from the stable ppa, they said "and it's working for you?" [18:56:08] lol [18:56:09] What can I do to help prepare for upgrade? [18:56:11] they didn't release fixes to the ppa, instead they deprecated it [18:56:19] andrewbogott: I need to get keystone working [18:56:28] and I need openstackmanager to use it [18:57:16] paravoid: at this point I'm willing to take downtime for the upgrade [18:57:30] as much as I'd prefer to avoid it [18:57:30] Hm… I think that keystone was drastically revised in essex, so getting it to work in diablo might be wasted effort. Maybe that's not what you meant anyway. [18:57:50] the ldap backend is the same [18:57:59] the api is different, but not much so for our purposes [18:58:09] ok [18:58:13] code modifications won't be horrible between them, I think [18:58:27] ooohhhhh [18:58:29] sorry [18:58:35] andrewbogott: I'm talking about keystone in essex [19:00:50] When you said 'I need to get keystone working' did you mean that that's the only blocker and hence nothing much I can do to help? Or should I help with that? [19:01:42] Ryan_Lane: didn't you say that you fixed the bug already? [19:01:47] or is this a different bug? [19:01:51] different one [19:02:09] andrewbogott: if you'd like to help with that, that would be goo [19:02:10] *good [19:02:17] I'm working on nova-precise1 [19:03:58] maybe I can patch this [19:04:06] I think the patchset is pretty lengthy, though [19:04:18] :/ [19:04:27] let me check it's in the repo [19:04:34] maybe I can cherry-pick it [19:04:48] I hate being rushed on upgrades [19:05:38] I hate saying this, but you'll have to forgive me [19:05:40] it's friday 10pm and I've been sitting here more or less working since 7:30am [19:05:52] paravoid: no worries [19:05:56] this won't get fixed today [19:05:58] and my hand is *killing* me [19:06:17] and it's a degredation, not something that totally interrupts service [19:06:54] everything still works, people might have to retry an action on occasion [19:07:41] how often does it happen? could we finish the migrations first? [19:07:53] or is it that urgent? [19:08:08] we can finish migrations first [19:08:16] it usually happens after an instance is created [19:08:29] it's a race condition [19:08:35] okay [19:11:15] paravoid: so, have a good night. talk to you on monday [19:11:24] yeah, sorry... [19:11:24] I'll respond back to your email :) [19:11:29] heh, thanks a lot [19:11:36] no worries. get some rest for your hands [19:12:01] hand, fortunately :) [20:19:00] Ryan_Lane: busy? [20:19:28] responding to a giant email :) [20:19:57] hehe, if you can, poke me when you get time, no worry. [20:20:34] by the way, is it to Sumana's mail? [20:20:35] Change on 12mediawiki a page Wikimedia Labs was modified, changed by 188.102.18.49 link https://www.mediawiki.org/w/index.php?diff=563776 edit summary: typo, +r [20:21:59] yeah [21:01:55] <^demon> Ryan_Lane: I poked gitlist a bit more. It requires some manual rewrite rule setup, but otherwise looks good. Also supports repo exclusion, and since we only have < 5 hidden repos, that's not difficult to maintain. [21:03:24] ^demon: Does it support pattern-based branch exclusion within a repo? [21:03:28] (i.e. drafts) [21:03:42] <^demon> No. I don't know of any tool that will do that properly. [21:03:49] <^demon> Although it's php, so we might be able to hack it in. [21:08:55] <^demon> RoanKattouw: We could fork it maybe. But the code is ugly :\ [21:09:54] So, to recap the thread on #mediawiki: I'm writing a new extension designed for eventual Wikimedia deployment. In particular, it is reliant on librsvg, which I cannot install on my own personal shared hosting. I am therefore struggling to find a public wiki where users can at some point test my (by then beta quality) extension. Labs may or may not be good for this (are there enough public... [21:09:56] ...IPs to go round?), but I'm officially asking here anyway :) [21:10:51] jarry1250: What you can do is set it up in labs and worry about a public IP later [21:11:15] In the meantime you can use something like FoxyProxy to access your labs instance [21:11:34] Roan: Cool, sounds reasonable. *searches out getting started with labs page* [21:12:00] You'll have to ask Ryan_Lane to create a project, I think [21:12:18] You should already have a labs account because you have a Gerrit account [21:13:42] jarry1250: I've also just added you to the bastion project, so you should be able to log into bastion.wmflabs.org in a few minutes [21:14:59] 07/20/2012 - 21:14:59 - Created a home directory for jarry1250 in project(s): bastion [21:15:58] 07/20/2012 - 21:15:58 - User jarry1250 may have been modified in LDAP or locally, updating key in project(s): bastion [21:16:36] jarry1250: I can create a project for you if you like. I'm always interested in new recruits to test my mediawiki deployment code. [21:17:17] andrewbogott: If that's allowed then yes, that would be great (sounds slightly ominous though! :P ) [21:17:30] jarry1250_ A public IP is probably also possible, but you may as well get things up and running before crossing that bridge. [21:17:42] What name would you like for the project? [21:17:49] translatesvg [21:17:54] Would be good :) [21:18:03] Oh, what's the caps convention? [21:19:17] That's a good question… I don't know if there is one. [21:19:28] Hmm, the "List projects" page would suggest... [21:19:30] Translatesvg [21:19:35] Is the way to go. [21:20:21] Actually, I don't see mixed case in any of the project names (possibly because everything is automatically lowercased.) Lots of hyphenations though. [21:20:35] in general all lowercase [21:20:37] so, translatesvg or translate-svg seem like good candidates. [21:20:54] case is possible, but it makes things confusing [21:21:41] translatesvg then please :0 [21:21:44] * :) [21:22:59] 07/20/2012 - 21:22:59 - Creating a project directory for translatesvg [21:22:59] 07/20/2012 - 21:22:59 - Created a home directory for jarry1250 in project(s): translatesvg [21:23:50] jarry1250: This page should get you started on your server. Please let me know how it goes and if you have doc suggestions. https://labsconsole.wikimedia.org/wiki/Help:InstanceConfigMediawiki [21:23:59] 07/20/2012 - 21:23:59 - User jarry1250 may have been modified in LDAP or locally, updating key in project(s): translatesvg [21:24:20] make sure to read the security group and instance docs [21:24:22] !instances [21:24:22] need help? -> https://labsconsole.wikimedia.org/wiki/Help:Instances want to manage? -> https://labsconsole.wikimedia.org/wiki/Special:NovaInstance want resources? use !resource [21:24:23] err [21:24:28] ah. yeah. that's right [21:24:30] !security [21:24:30] https://labsconsole.wikimedia.org/wiki/Help:Security_Groups [21:24:48] Oh, good point, you'll need a web-enabled security group before you create an instance. [21:24:53] * andrewbogott adds that to the docs [21:25:38] <^demon> Ryan_Lane: gitlist code sucks. it's just php doing a bunch of exec()s to cli git. i mean sure, there's no git interface for php...but ugggghhhh [21:25:52] <^demon> And basically anything that's expensive gets lazy cached in a "cache" directory. [21:26:20] * jarry1250 starts reading [21:28:06] Ryan_Lane: That security page could use some examples. Do we have those someplace already, or should I add them? [21:28:15] adding them would be good :) [21:28:19] ok [21:31:50] Ryan_Lane: Does every new project automatically have a 'default' security group? [21:31:56] yes [21:32:00] but no default sudo group [21:32:05] err [21:32:07] sudo policy [21:32:51] does 'no sudo policy' mean that everyone gets it, or no one? [21:42:21] Hmm, so what's a good web security group to copy? [21:49:28] jarry1250: Awkwardly, it's hard for you to see existing security groups in other projects because you have to be netadmin to see them. [21:49:34] That's why I'm trying to add examples to the docs :) [21:49:56] andrewbogott: Well, I just picked one that sounded acceptable. [21:50:11] I'm afraid " Optional: Set the 'labs_mediawiki_hostname' to the fully qualified hostname of your instance." has lost me though [21:50:20] Basically you just want to open ports 80 and 25 to everyone. [21:51:27] jarry1250: That'll just be .pmtpa.wmflabs [21:51:44] Okay, that makes sense. But how/where do I set that? [21:53:05] Hm… you want to wait and make sure your instance is up and running first. Then you go to the 'configure' page for the instance, and there's a list of puppet classes and variables to set. [21:53:20] Where a 'puppet class' is a server configuration rule. [21:54:04] Ryan_Lane: Any idea what template is used to display the security groups in the wiki? I can't view the source of those pages :( [21:55:01] andrew: Aha! Found it now. Was expecting the list to be in alphabetical order. [21:55:30] andrewbogott: template? [21:55:36] oh [21:55:39] those are special pages [21:55:41] it's code doing it [21:55:46] Maybe its just formatted in code. [21:55:47] OK. [21:55:51] I'll just use a table then. [21:55:58] * Ryan_Lane nods [22:04:54] andrewbogott: Okay, I'm up to "sudo puppetd -tv" [22:05:00] So I logged into Bastion... [22:05:21] that's a good start :) [22:05:30] But where should I be? [22:05:43] (Before I run that) [22:06:21] On your instance [22:06:26] So on bastion, run ssh yourinstancename [22:07:21] Okay, so my instance is named translatesvg (same as the project, or is that a mistake?) [22:07:27] No that's fine [22:07:33] You can have multiple instances per project though [22:07:37] But ssh translatesvg doesn't work? [22:07:42] What does it say? [22:07:55] Name or service not known? [22:08:02] Did you actually create an instance yet? [22:08:49] jarry1250: Mind if I add myself to your project so I can poke about? [22:08:50] Yes. [22:08:55] andrew: Sure. [22:09:04] Yeah it has no instance [22:09:06] s [22:09:09] You didn't actually create the instance yet [22:09:20] Go to Manage instances, then next to your project click Create Instance [22:09:37] I did that? [22:09:54] It doesn't look like your instance creation actually went through [22:10:01] I now have a table row, finishing with Actions: Delete, configure, reboot [22:10:08] Oh, OK [22:10:11] , get console output. [22:10:14] Right [22:10:15] So it is there [22:10:21] Maybe it hasn't finished building yet [22:10:27] That usually takes 5 minutes or so [22:10:36] Yeah, it flashed through real quick [22:11:58] Get console output doesn't suggest "In process" though [22:12:00] 07/20/2012 - 22:12:00 - Created a home directory for andrew in project(s): translatesvg [22:12:50] actually… RoanKattouw, Jarry1250, I can access that instance just fine as long as I don't call it by name. I can't guess why that would be. [22:12:58] Hmm weird [22:12:59] 07/20/2012 - 22:12:59 - User andrew may have been modified in LDAP or locally, updating key in project(s): translatesvg [22:13:09] Ryan_Lane, any idea? Naming an instance after a project should be kosher shouldn't it? [22:13:14] Can you access it by its i-NNNNN name? [22:13:24] yes. [22:13:38] Maybe we're just being impatient, but I would've thought that both names would make it through DNS at the same time. [22:13:49] I find it, but I get access denied. [22:14:02] jarry1250: Did you forward your key when you logged into bastion? [22:14:11] Does ssh-add -l list your key? [22:14:26] Roan: Given that I have no idea how to do that, I suspect not :P [22:14:48] jarry1250: OK, lemme walk you through it [22:14:50] Log out of bastion [22:15:00] k [22:15:02] On your machine, run eval `ssh-agent` to start an agent [22:15:25] Then ssh-add path/to/your/keyfile to add your key (if your key is ~/.ssh/id_rsa you may omit the filename) [22:15:32] It'll prompt you for the passphrase if the key has one [22:16:01] Then, log into bastion using ssh -A [22:16:06] Hmm, I'm on Windows. [22:16:13] * RoanKattouw sighs [22:16:44] There must be a way to do agent forwarding on Windows but I have no idea how [22:17:21] I'll google it. [22:18:46] Okay, got it I think *tests* [22:20:26] Hmm, I've ticked "Allow agent forwarding" in PuTTY but to no avail - more searching I think. [22:21:43] jarry1250: I know there are other folks in this channel who use PuTTY to do what you're doing. I am not one of them, though. [22:21:57] Just saying a) it's possible b) someone might come to your rescue at any minute [22:22:30] Hi All. Hi jarry1250. I'm new to this community and I was just going through your conversations. I think what you are lokking for is Pagent [22:22:39] http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html [22:23:00] Yeah I suppose if you run Pageant and then enable forwarding in PuTTY somehow, that should work [22:23:02] It's a PuTTY implementation of openssh's ssh-agent [22:23:32] Ah yes, I have that installed, I just never worked out what it was for. Now I discover :) [22:24:17] You just have to add your private key and you are good to go!! It'll be at the notification area. [22:25:03] Yes! :) [22:25:17] Thanks adminxor :) [22:25:34] Anything to help! :D [22:25:41] andrewbogott: it won't cause any issues [22:27:55] Do you know why translatesvg doesn't have a dns entry then? [22:28:28] hm [22:29:55] it's missing an arecord [22:30:39] strange [22:31:26] ugh. I wonder if this is due to the stupid describe instances bug [22:33:19] rebooting, back in a flash [22:33:31] ok. it's good now [22:33:42] I manually fixed it [22:33:45] Ta muchly :) [22:38:26] jarry1250: Do you have ssh working now? [22:38:36] andrewbogott: yes [22:38:47] SO now I'm trying to run "sudo puppetd -tv" on my project. I wasn't able to initially, so I followed the instruction and added myself to a sudo policy. Now, it gives a different message, but I'm still prevented from running it. [22:39:00] Oh, I see. [22:39:01] what message? [22:39:11] Sorry, I'm being silly. I left commands blank. [22:39:26] Put "ALL" [22:40:16] Roan: Ah, top tip, thanks :) [22:50:21] 07/20/2012 - 22:50:20 - Updating keys for adminxor at /export/keys/adminxor [22:51:20] 07/20/2012 - 22:51:20 - Updating keys for alchimista at /export/keys/alchimista [22:52:46] brb, restarting [22:55:00] 07/20/2012 - 22:55:00 - Created a home directory for adminxor in project(s): bastion [22:55:20] adminxor: ^ [22:55:45] Reedy: Thank you! [22:56:00] 07/20/2012 - 22:56:00 - User adminxor may have been modified in LDAP or locally, updating key in project(s): bastion [22:59:31] Ryan_Lane: simple one for you https://gerrit.wikimedia.org/r/16195 [23:02:34] Reedy: projeects? [23:02:49] and manage vs Manage? [23:02:58] yeah [23:03:04] it's used inline in a string usually [23:03:08] ah [23:03:13] why projeects, though? :) [23:03:28] and OpernStack? [23:03:29] heh [23:03:47] Okay, still got to fix my proxy, but that can wait until tomorrow. [23:03:53] Thanks everyone for their help [23:04:01] Very useful [23:04:08] opernstack was already there for rights-manageproject [23:04:37] ugh [23:04:41] damn typos [23:04:46] fixed them all [23:05:07] cool. thanks [23:19:21] 07/20/2012 - 23:19:20 - Updating keys for alchimista at /export/keys/alchimista [23:24:59] 07/20/2012 - 23:24:58 - User adminxor may have been modified in LDAP or locally, updating key in project(s): bastion [23:25:21] 07/20/2012 - 23:25:20 - Updating keys for adminxor at /export/keys/adminxor [23:34:59] 07/20/2012 - 23:34:59 - User adminxor may have been modified in LDAP or locally, updating key in project(s): bastion [23:35:21] 07/20/2012 - 23:35:21 - Updating keys for adminxor at /export/keys/adminxor [23:35:59] 07/20/2012 - 23:35:59 - User adminxor may have been modified in LDAP or locally, updating key in project(s): bastion [23:36:21] 07/20/2012 - 23:36:21 - Updating keys for adminxor at /export/keys/adminxor