[13:23:51] hi [16:14:22] @commands [16:14:22] Commands: channellist, trusted, trustadd, trustdel, infobot-off, refresh, infobot-on, drop, whoami, add, reload, RC-, recentchanges-on, recentchanges-off, logon, logoff, recentchanges-, recentchanges+, RC+ [16:16:00] @trusted [16:16:00] I trust: petan!.*@wikimedia/Petrb, .*@wikimedia/.*, .*@mediawiki/.*, .*@wikimedia/Ryan-lane, [16:18:11] @recentchanges-on [16:18:11] Feed is enabled [16:19:21] @RC+ mediawiki Talk:Wikimedia_Labs [16:19:21] Inserted new item to feed of changes [16:20:45] hmm [16:21:13] I've just spammed that page [16:21:35] weird [16:21:47] @RC+ mediawiki Talk:Wikimedia_Labs [16:21:47] There is already this string in a list of watched items [16:22:03] @RC+ mediawiki User:Petrb/spam [16:22:03] Inserted new item to feed of changes [16:22:25] Change on 12mediawiki1 a page User:Petrb/spam was modified, summary: changed by Petrb link https://www.mediawiki.org/w/index.php?diff=474116 [16:22:39] urlencoding? [16:22:43] it can only watch the edit changes :o [16:23:03] what's wrong? [16:23:25] Change on 12mediawiki1 a page User:Petrb/spam was modified, summary: changed by Petrb link https://www.mediawiki.org/w/index.php?diff=474117 [16:24:02] damn I need to implement liquid threads [16:29:20] yuvipanda: inserted new feature to a bot [16:29:23] !wm-bot [16:29:32] bah [16:29:34] @commands [16:29:34] Commands: channellist, trusted, trustadd, trustdel, infobot-off, refresh, infobot-on, drop, whoami, add, reload, RC-, recentchanges-on, recentchanges-off, logon, logoff, recentchanges-, recentchanges+, RC+ [16:29:40] @regsearch . [16:29:40] No results found! :| [16:29:47] yay [16:30:02] !test is tset [16:30:02] Key was added! [16:30:03] !test [16:30:03] tset [16:34:51] @reload [16:34:51] Channel config was reloaded [16:34:58] !ping [16:35:02] !wm-bot [16:35:02] http://meta.wikimedia.org/wiki/WM-Bot [16:35:04] better [16:35:30] !wm-bot | yuvipanda [16:35:30] yuvipanda: http://meta.wikimedia.org/wiki/WM-Bot [16:35:33] check it out :P [16:38:31] !git [16:38:31] for more information about git on labs see https://labsconsole.wikimedia.org/wiki/Git [16:39:07] @recentchanges-on [16:39:07] Channel had already feed enabled [16:39:18] @RC+ mediawiki User:Petrb/spam [16:39:19] Inserted new item to feed of changes [16:39:20] @RC+ mediawiki User:Petrb/spam [16:39:20] There is already this string in a list of watched items [16:39:37] Change on 12mediawiki1 a page User:Petrb/spam was modified, summary: test changed by Petrb link https://www.mediawiki.org/w/index.php?diff=474120 [16:43:37] @RC+ mediawiki Talk:Wikimedia_Labs [16:43:37] Inserted new item to feed of changes [16:43:38] ah [16:43:38] @RC+ mediawiki User:Petrb/spam [16:43:38] Inserted new item to feed of changes [16:53:43] petan, have you got a Properties/AssemblyInfo.cs file in your working copy? [16:53:50] If so, can you svn add the Properties folder pleease [16:54:14] sure [16:57:57] @RC+ mediawiki Huggle [16:57:57] Inserted new item to feed of changes [16:58:27] Change on 12mediawiki1 a page Huggle was modified, summary: testing bot on i ... changed by Petrb link https://www.mediawiki.org/w/index.php?diff=474125 [17:00:51] @RC- mediawiki Huggle [17:00:51] Deleted item from feed [17:05:03] hello everyone [17:06:04] hey [17:08:00] sorry i've been hibernating for the past [17:08:13] what's been happening? [17:15:54] !sal [17:16:32] !sal is https://labsconsole.wikimedia.org/w/index.php?title=SAL [17:16:32] Key was added! [17:16:43] !sal del [17:16:43] Successfully removed sal [17:17:11] !sal is https://labsconsole.wikimedia.org/wiki/Server_Admin_Log see it and you will know all you need [17:17:11] Key was added! [17:30:12] hi RoanKattouw I implemented rc feed to bot if you think it could have use in some channels, let me know and I set it up [17:30:51] but watching lqt doesn't work yet, so support board can't be watched for instance [17:30:58] Oh, nice [17:34:05] Reedy: chance that feed bot would be reporting changes to lqt? [17:34:23] because now there is no message when someone post a thread [17:34:28] on pmtpa [17:34:53] I guess LQT doesn't have the UDP RC stuff built in [17:35:26] it should have some antivandalism tools would be unable to watch talk vandalism if it was deployed further [17:35:41] Does it appear in RC? [17:35:45] yes [17:35:49] but not in irc feed [17:35:55] https://www.mediawiki.org/wiki/Special:RecentChanges [17:36:05] User talk:Petrb; 17:33 . . Petrb (Talk | contribs | block) posted a reply to "test"‎ (test) [17:41:55] Needs RecentChanges::sendToUDP() adding into LQT on edit/save/delete [17:42:48] essentially [17:45:35] Hmm, that's strange though [17:45:38] Shouldn't RC do that itself? [17:45:46] I wonder if this has been refactored properly in trunk/1.19 [17:50:37] Wonder if it is a regression [18:22:03] Ryan_Lane: I am getting pretty close to wanting to start writing the ldap DNS driver, for which I'll need a test system. [18:22:11] cool [18:22:24] andrewbogott: let me add you to the openstack project [18:22:35] If I make a test system on labs, will it automatically have ldap access? [18:22:48] Or should I set up my own ldap system? [18:22:59] there's a set of puppet classes for ldap [18:23:17] and powerdns [18:23:22] so, it'll be all set up for you :) [18:23:28] fair enough :) [18:23:40] lemme see if I know how to do this... [18:23:50] Ok, I added you to the projects [18:24:01] *project [18:24:05] 12/27/2011 - 18:24:04 - Creating a home directory for andrew at /export/home/openstack/andrew [18:24:40] doh. seems the ldap classes are mingled in with the openstack ones [18:24:45] lemme fix that [18:25:05] 12/27/2011 - 18:25:05 - Updating keys for andrew [18:25:53] ah [18:25:54] openstack::ldap-server [18:25:59] I can just add that [18:26:53] I am slightly menaced by the current openstack instance having 'production' in its name. I can make another instance in that project without messing with anyone, right? [18:27:04] yeah [18:27:09] nova-dev1 or something [18:27:27] I'm working on openstackmanager in nova-production1 [18:27:52] 'k [18:27:53] so, I think you only need to add the openstack::ldap-server class [18:28:06] do it after the instance builds, via configure, though [18:28:16] I don't know if the system will build otherwise. heh [18:30:20] What do I need to do to set up ssh for a new instance? [18:35:24] it's enabled automatically [18:35:29] it takes a little bit to finish bulding [18:35:58] shit [18:35:58] Could not connect to brewster.wikimedia.org:8080 (208.80.152.171). - connect (111: Connection refused) [18:36:15] that's not good [18:36:21] c.c [18:36:45] instances can't build without that [18:37:02] 12/27/2011 - 18:37:02 - Updating keys for andrew [18:37:05] 12/27/2011 - 18:37:05 - Updating keys for andrew [18:37:10] 12/27/2011 - 18:37:09 - Updating keys for andrew [18:38:03] seems its dead [18:38:30] hmm [18:38:32] no apache installed [18:38:38] I wonder what's running the apt proxy [18:40:02] 12/27/2011 - 18:40:01 - Updating keys for andrew [18:40:08] 12/27/2011 - 18:40:07 - Updating keys for andrew [18:40:09] squid wasn't running on it [18:40:12] 12/27/2011 - 18:40:11 - Updating keys for andrew [18:40:25] ok, now I think my keys are actually updated :) [18:40:29] disk is full [18:42:37] ok... [18:42:43] brewster should work again [18:42:54] andrewbogott: you'll need to delete and re-create that instance [18:43:00] ok [18:48:32] PROBLEM host: nova-dev1 is DOWN address: nova-dev1 CRITICAL - Host Unreachable (nova-dev1) [18:49:55] :( [18:50:00] ? [18:51:09] heh [18:51:12] dns cache [18:51:52] So I should be able to ssh from bastion1 to the new instance, right? [18:52:08] yeah, but when you delete an instance and recreate it [18:52:09] ... [18:52:14] the dns cache screws you [18:52:16] lemme clear it [18:52:33] Well, if I connect by ip it prompts me for a password. [18:52:39] oh [18:52:41] weird [18:52:43] it shouldn't [18:53:09] actually, it's maybe just login name... I seem to be 'andrew' on bastion1... am I also 'andrew' on the new instance? [18:53:19] yep [18:53:47] working for me... [18:54:11] did you forward your key? [18:54:13] YOU WILL DO IT NOW FOR GREAT JUSTICE [18:54:15] sorry...wrong channel [18:54:16] err [18:54:18] agent [18:54:21] hyperon: :D [18:54:23] Ryan_Lane: me? [18:54:27] * andrewbogott DOES IT NOW FOR GREAT JUSTICE [18:54:36] hyperon: nah. I was just laughing at your mischannel :) [18:54:42] andrewbogott: did you forward your agent? [18:54:52] RECOVERY host: nova-dev1 is UP address: nova-dev1 PING OK - Packet loss = 0%, RTA = 0.56 ms [18:54:57] I don't think I know what that is. [18:55:04] ah [18:55:17] * andrewbogott digs toe nervously in the dust [18:55:25] you should start an ssh agent on your laptop, then forward it; here's how... [18:55:31] eval `ssh-agent` [18:55:41] ssh-add .ssh/ [18:55:47] i assume that someone else puppetized morebots while i was away being uncomputerey [18:55:47] ssh -A bastion.wmflabs.org [18:55:54] hyperon: nop [18:55:56] *nope [18:56:05] awwwww now i feel really bad for not doing it [18:56:16] it's no biggie [18:56:22] we know how to restart it :) [18:56:31] if it dies, for some reason [18:56:41] it died? [18:56:45] puppetization is good to do for when we move it [18:56:49] no. I mean if it was to die [18:56:52] we know how to deal with it [18:56:57] oh [18:57:05] ok, that makes plenty o' sense [18:57:36] Ryan_Lane: works, thanks. [18:57:40] cool [18:57:45] mhm, how can I set up external access to my instance? [18:57:53] you have sudo via your wiki password [18:57:59] MaxSem: depends on what you need [18:58:02] !socks-proxy [18:58:02] see https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances ssh @bastion.wmflabs.org -D [18:58:12] PROBLEM dpkg-check is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [18:58:17] for most stuff a socks proxy is appropriate [18:58:32] Ryan_Lane: Can you point me to a slice of your existing code that writes dns entries to ldap? [18:58:43] MaxSem: what kind of external access do you need? [18:58:45] andrewbogott: sure [18:58:58] I must warn you, my code may want to make you punch kittens [18:59:32] http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/OpenStackManager/OpenStackNovaDomain.php?view=markup [18:59:38] http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/OpenStackManager/OpenStackNovaHost.php?view=markup [19:00:23] thanks. (And, it clearly works, so I'm unlikely to complain.) [19:00:36] heh [19:00:37] so what? do you think it lacks a couple more abstraction layers? [19:01:02] I think some parts are hacky and perform terribly [19:01:07] the abstraction is actually fine [19:01:53] PROBLEM Current Load is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [19:02:03] PROBLEM Free ram is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [19:02:13] PROBLEM Disk Space is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [19:02:14] I wonder why nrpe isn't working on that node [19:02:23] PROBLEM Total Processes is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [19:02:33] PROBLEM Current Users is now: CRITICAL on nova-dev1 nova-dev1 output: CHECK_NRPE: Error - Could not complete SSL handshake. [19:02:34] * Ryan_Lane checks [19:04:21] ah. also need dns::auth-server-ldap [19:04:26] * Ryan_Lane adds that to puppet conf [19:05:56] ok. I think all variables and classes you need are available [19:06:24] andrewbogott: in the puppet repo, if you look at the virt1 node definition in manifests/site.pp, you can see how its configured [19:13:31] Ryan_Lane: can you update my ssh key?? [19:13:39] OrenBochman: where? [19:13:41] hm... actually, do you want me to write a fresh Python driver? If the php/dns code is going to endure, it might be better to have openstack just use a think python wrapper around it. [19:13:59] andrewbogott: I want to kill off the php code :) [19:13:59] I'm noticing that your code does a lot more than the nova driver would need to, which makes me think it's getting used for other stuff. [19:14:11] it does puppet stuff too [19:14:29] I guess it also creates domains... [19:14:36] well, and domain manipulation which is well outside the scope of the nova blueprint. [19:14:42] yeah. [19:14:49] OrenBochman: no, I mean where do you need your key updated? [19:14:56] OrenBochman: if in labs, then you can do it yourself [19:15:02] andrewbogott: ah. [19:15:06] no subversion [19:15:10] andrewbogott: we should add it in the next version [19:15:19] Sure, ok. [19:16:09] also I'd like you to set me up with labs access [19:16:15] managing puppet should also be next on our radar [19:16:23] OrenBochman: can do [19:16:28] great [19:16:34] gimme a sec [19:16:41] no problem [19:17:26] Ryan_Lane: Do you want to write a blueprint (or a draft of a blueprint) about puppet in nova? [19:17:42] sure, that would be cool [19:18:18] basically, right now it's the ability to provide the information to nova on instance creation, and afterward, to have the instance reconfigure itself [19:18:49] and it's just puppet classes and variables [19:19:01] I think that's all puppet supports in external nodes [19:20:09] I'm not clear on whether the openstack folks aspire to having a Horizon panel for every extension. For puppet, the GUI seems like the hard part. (Or at least, the hard part for me.) [19:20:24] Maybe we can just ignore that and hope that someone else comes along and implements it for us. [19:20:56] well, if we switch to horizon, I'll add the gui part [19:21:48] I don't have any short-term plans to move to horizon [19:22:14] Sure, I just worry that a feature in nova w/out a GUI will bitrot. [19:22:19] although, not if we are using it I suppose :) [19:22:35] What are the chances of me getting sudo on my new instance? [19:22:36] I want people to be able to use the euca and nova commandline tools, which is the main motivation for adding these features :) [19:22:40] andrewbogott: you have it [19:22:49] it's your wiki password [19:22:52] Ah, ok. [19:23:09] hey, there it is! [19:23:13] <- mad with power [19:23:17] so, we can add the puppet stuff to the nova command [19:23:26] the euca tools can go screw themselves :) [19:24:06] openstack seems to look like they are dropping ec2 api support at some point anyway [19:24:22] So... I need to add 'nova commandline tools for dns' to my list of things to do. Or are those somehow implicit in the interface I've already written? [19:24:41] instance dns is implicit [19:24:46] public DNS isn't [19:24:50] may need to add that [19:24:56] ok [19:25:17] that should be much easier than adding it to horizon, thankfully [19:25:43] I'm sure someone will add DNS support to horizon [19:25:56] the puppet people may add the puppet support to horizon, if we make the api [19:26:06] they have a vested interest in that :) [19:26:17] I have a roughed-in implementation that I've been using for testing. I'm hoping someone else will adopt it because I don't plan on making it look right. [19:26:25] um... of dns/horizon I mean. [19:26:27] * Ryan_Lane nods [19:26:43] PROBLEM Disk Space is now: WARNING on puppet-lucid puppet-lucid output: DISK WARNING - free space: / 78 MB (5% inode=35%): [19:26:57] Can I nudge my instance to do a puppet refresh? [19:27:01] yep [19:27:03] puppetd -tv [19:27:21] it may throw errors [19:27:56] I haven't tried installing those openstack classes without the rest of them [19:28:48] sudo puppetd -tv? [19:29:01] yeah [19:29:13] I usually sudo to root :) [19:29:59] err: Could not retrieve catalog from remote server: Error 400 on SERVER: Parametmer not defined! at /etc/puppet/manifests/dns.pp:26 on node i-000000c4.pmtpa.wmflabs [19:30:07] heh [19:30:34] probably missing a varaible [19:30:43] yep [19:30:55] you need to set dns_auth_ipaddress [19:31:05] and dns_auth_soa_name [19:31:13] the former is the instance's IP [19:31:23] the latter is pmtpa.wmflabs [19:31:36] wait [19:31:37] no [19:31:42] it's the instance's DNS name [19:31:47] Oh yeah, I see those now. (puppet page should validate!) [19:31:57] it's hard for the puppet page to validate [19:32:10] it would have to know what's required [19:32:38] and what is required depends on what classes are used [19:32:50] I guess I could add these dependencies via the interface [19:33:02] Isn't it just the case that if there are variables under a package, they should be non-empty? [19:33:11] not always [19:33:25] Ah, ok, not straightforward then. I retract my scold. [19:33:29] heh [19:33:33] you are right, though [19:33:41] and I think I can add dependencies, actually [19:33:53] RECOVERY Disk Space is now: OK on nova-production1 nova-production1 output: DISK OK [19:34:10] in the interface, when classes and variables are added, it could show a list of other classes and variables, to select as "required" [19:34:17] Getting further now, it says: err: Failed to apply catalog: Could not find dependency File[/etc/ssl/private/.key] for Exec[.opendj_create_pkcs12] at /etc/puppet/manifests/certs.pp:21 [19:34:25] it'll be difficult to implement, but it would be nice :) [19:35:06] ldap_certificate, ldap_first_master, and ldap_server_bind_ips needs to be set [19:35:13] and actually, ldap_server_bind_ips is optional [19:35:19] and you probably shouldn't set it [19:35:30] no setting there means it binds to 0.0.0.0 [19:37:55] forgive me: ldap_certificate, ldap_first_master need to be set... to what? [19:38:11] ah [19:38:18] sorry [19:38:24] ldap_first_master = true [19:38:32] ldap_certificate..... [19:39:14] star.wmflabs [19:39:22] you also need to include the certificates::star_wmflabs class [19:40:02] I know a feature I should add to the interface :) [19:40:10] templates [19:40:22] puppet likes me now! Thanks. [19:40:27] pre-configured instances [19:40:31] andrewbogott: yw [19:40:52] we need to ensure instances can build with out puppet configuration, though, and that's a *ton* of work [19:40:56] *our [19:44:15] You mean, as opposed to starting up and then having a config applied once they're up? [19:44:31] partially, yeah [19:44:49] but even after the instance fully builds, the puppet config needs to apply cleanly [19:45:02] a lot of ours doesn't [19:45:10] sometimes it takes 2-3 runs to work [19:45:33] Is that because of ordering, or race conditions? Or... [19:45:44] Does puppet even allow you to specify ordering? [19:45:50] ordering, yeah [19:45:52] it does [19:46:10] So, fixable. But work. [19:46:14] yep [19:46:28] especially with some of the more complex stuff [20:09:02] 12/27/2011 - 20:09:02 - Updating keys for py [20:09:09] 12/27/2011 - 20:09:09 - Updating keys for py [20:13:03] 12/27/2011 - 20:13:02 - Updating keys for py [20:13:10] 12/27/2011 - 20:13:09 - Updating keys for py [20:55:49] Hmmm time to work out puppet before I do server migrations :D [20:57:23] heh [22:28:14] I'm trying to think of a way to better arrange the OSM code files from the directory root [22:29:19] yeah. I'm not a big fan of all the model classes sitting in the root [22:29:33] nearly all of the classes in there are for the model [22:29:59] well, I guess it's model and controller [22:30:11] in the case of the ldap backed classes [22:32:06] It's not really helped by us not having a "mediawiki way" to order things [22:32:10] so people do what they see fit [22:32:11] yeah [22:32:14] yep [22:32:16] not that that is wrong [22:32:21] Just inconsistent [22:32:26] I've been mostly looking at what other larger extensions do [22:32:36] I didn't originally intend to have this much code written :D [22:33:08] Aaron has massively refactored FR into frontend/backend [22:33:11] I've going to move some of the logic into /api [22:33:19] *I'm [22:33:40] then I'm going to change the special pages to use the api functions [22:33:46] then I'm going to ajaxify the interface [22:34:22] well, all of my frontend and backend code is separate [22:34:29] all frontend code is in /specials [22:50:21] Ryan_Lane: It looks to me like you're binding to server with username . Is that right? Do I not need to worry about authentication because this will run as root? [22:50:45] I'm binding with configurable settings [22:50:59] it should be a user/pass combo [22:51:13] look at OpenStackNovaLdapConnection.php [22:51:19] I connect and bind there [22:51:27] $wgAuth->bindAs( $wgOpenStackManagerLDAPUser, $wgOpenStackManagerLDAPUserPassword ); [22:52:01] Sure, just, those vars are initialized as "". I must be missing the place where they are set to something real. [22:52:16] they are set in mediawiki's LocalSettings.php [22:52:52] basically, you make a user in LDAP with rights to modify/add/delete the right entries [22:53:02] Hm... I don't suppose they're also in the environment? [22:53:03] then you use that user's userdn and password [22:53:33] if you check out /srv/org/wikimedia/controller/wikis/1.18wmf1/LocalSettings.php on nova-production1, you'll see this [22:54:22] I also set the SOA record, and primary/secondary DNS servers there [22:54:44] since the SOA record must be updated every single change [22:56:08] andrewbogott: see /root/dirmananagers.ldif for an example of how I set up the user [22:56:11] on nova-production1 [22:56:19] ok, reading... [22:56:52] I set up the file that way so that I can just keep running it with ldap-modify -c [22:57:07] also see: /root/admin-privileges.ldif [22:57:17] though you won't really need that [22:58:02] in many LDAP servers, ACIs are used to manage rights. they can be set at any level of a tree, and they'll apply to everything below them [22:58:14] *everything at or below them [22:58:42] though, you can change the ACI to only apply to the one entry, one level below the entry, or all levels below it [22:59:09] the ACI can be targeted at specific entries or attributes on entries [23:01:18] see: http://opendj.forgerock.org/doc/admin-guide/OpenDJ-Admin-Guide/chap-privileges-acis.html [23:03:02] I think I understand, but maybe not. Will my driver need to extract user and password from Localsettings.php, or does that not matter since we're moving dns control out of mediawiki code anyway? [23:03:19] nah. you'll have that info in nova's config [23:03:56] basically, we're going to get rid of all the DNS code in mediawiki, and move it to nova [23:03:57]