[08:25:00] Change on 12mediawiki a page Developer access was modified, changed by Solon link https://www.mediawiki.org/w/index.php?diff=574995 edit summary: [09:12:35] !test [09:12:35] !ping [09:12:35] $1 [09:12:36] pong [09:12:39] there had to be an outage yesterday or something [09:12:41] the connectivity with freenode didn't work [09:12:42] from labs [14:22:09] I have a development wikipedia on Labs with content I don't want to lose. Do I need a backup strategy in case the instance becomes corrupted and I need to rebuild? [14:22:22] yes [14:22:34] there is no backup provided right now [14:22:41] you have to perform backups on your own [14:23:42] Thanks, that's what I thought. A simple database export should suffice. Along with any back end configuration I'll need. [14:24:22] you can just make a dump of db and download it using ftp [14:24:29] or you can use project storage for that [14:24:38] it's unlike both storages get broken in same time [14:24:57] good to know [14:25:13] they are physically different machines [14:25:17] AFAIK [14:26:30] !logs [14:26:38] !search logs [14:26:42] @search logs [14:26:42] Results (Found 1): hyperon, [14:26:59] !logs is http://bots.wmflabs.org/~wm-bot/logs [14:26:59] Key was added [14:28:55] How can I access project storage? [14:29:32] slevinski /data/project [14:29:47] !search is http://bots.wmflabs.org/~wm-bot/searchlog/index.php?action=search&channel=%23wikimedia-labs [14:29:47] Key was added [14:30:06] @search log [14:30:06] Results (Found 14): morebots, labs-morebots, credentials, terminology, newgrp, requests, hyperon, hashar, Thehelpfulone, blehlogging, initial-login, beta, logs, search, [14:30:14] !blehlogging [14:30:14] To log a message, use the following format: !log [14:30:20] :O [14:30:27] aha [14:51:42] Hi, I'd like to make a public demo and protopying of a simple feature [14:52:03] Could anyone help me getting a publicly accessible instance? [14:53:02] hi whym [14:53:19] petan: can you help whym with this? [14:53:24] hi [14:53:24] This is the feature https://gist.github.com/3337993 which is to add an option to AllPages [14:53:34] sure [14:53:49] by the way I already have an account for analytics [14:54:04] whym what project you talk about? is it a new project? [14:54:38] also can you describe that little feature? [14:54:43] * simple [14:54:48] Probably a new project if there is no group for 'others' [14:54:50] is it mediawiki feature? or extension [14:54:59] there is somewhat a group for others [14:55:07] but for that I need to know what it is :D [14:55:20] some extensions can be tested on deployment project [14:55:29] This is a feature to enlist all pages oredered by sortkeys [14:55:38] is it an extension? [14:55:42] or core [14:55:46] Currently implemented as a moficifatioin to the core [14:55:53] ok then you need a new project [14:56:04] unfortunatelly I don't have powers to create it [14:56:12] you need to ask someone who can do that [14:56:26] ok [14:56:45] I think paravoid can [14:56:56] I don't know if there is a process for this yet [14:57:07] so far we always pinged people who can create new projects [14:57:42] that's understandable assuming it's not very frequent [14:57:53] it's going to be frequent [14:57:57] so we should have some [14:58:21] I think my project is primarily for Japanese wikis [14:58:35] what is your linux knowledge? do you need any assistance setting the servers up? [14:58:49] because that's probably the part I could help you with [14:59:00] petan, whym: I can create a project. What project name & members would you like? [14:59:09] petan: whym - I think the process is "email labs-l" [14:59:16] And, thanks for offering to do system support, petan, I'm pretty swamped today. [14:59:35] :) [14:59:53] petan: i have basic knowledge of daily use of it, but maybe I ask you later for labs-specific things [15:00:00] ok [15:01:04] andrewbogott: could it be a project for development for Japanese wikis with the name "jawiki"? [15:01:19] * andrewbogott nods [15:01:28] andrewbogott: if there is other naming conventions i'd happy to follow [15:01:37] *i'd be [15:01:43] Project names are pretty arbitrary so far. [15:01:58] whym there are rather naming conventions for instances, but no one follows them [15:02:01] My only request is that you notify me (or ryan or paravoid) when your demo is over so we can delete the project and clean up. [15:02:17] whym it's a good idea to prefix name of your instance with project name [15:02:18] and currently the only member is me [15:02:23] because they are all in one network [15:02:26] whym: You already have ssh access to labs worked out? Bastion account and so forth? [15:02:39] so common names like "web" or "db" is a bad hostname [15:02:46] andrewbogott: I have access to bastion and another project for analytics [15:02:48] jawiki-db is a better one [15:02:54] great. [15:03:44] whym: I'm think you're all set… go ahead and create an instance and see how it goes. [15:04:37] andrewbogott: thanks, I'll try [15:05:41] 08/23/2012 - 15:05:41 - Creating a project directory for jawiki [15:05:41] 08/23/2012 - 15:05:41 - Created a home directory for whym in project(s): jawiki [15:07:14] petan: does that apply to non publicly accessible instances? because I remember I didn't prefix them before [15:07:41] yes it applies to all instances :) [15:08:09] they all are in same subnet, if you create 2 instances with same name you will have troubles [15:08:32] and if you won't prefix the names, there is a big chance you will conflict with someone else [15:09:08] petan: good to know, I'll keep in mind [15:09:14] :) [15:10:40] 08/23/2012 - 15:10:39 - User whym may have been modified in LDAP or locally, updating key in project(s): jawiki [15:28:36] sorry for bothering again but how can I get a public IP for an instance? [15:28:58] Special:NovaAddress doesn't seem to work for me [15:41:16] ^demon: could you perhaps help me getting a public ip to my instance? [15:47:14] whym: I think I can do it, lemme see... [15:47:34] whym: Public IPs are in short supply, so the thing I said earlier about reminding me to clean up when you're done goes double. [15:49:36] andrewbogott: ok, how long would it usually take to turn it on? [15:49:58] The IP? Exactly as long as it takes me to remember how to do it. [15:50:27] andrewbogott: if that's really in short, I think I can ask just after everything ready [15:50:48] whym: That's maybe easier, especially if you wind up throwing out your instance and needing to start over :) [15:51:11] In theory we give you a quota and then you can allocate it yourself when the time comes. I'm sure I /can/ change the quota, but not sure I know how. [15:51:52] andrewbogott: but if it's acceptable to reserve it for a couple of days before it's ready, that would make it easier for me [15:52:01] * andrewbogott nods [15:52:15] <^demon> whym: For future reference, I don't have access to that :) [15:52:17] considering the time difference between most of you and me (in UTC+0) [15:52:21] UTC+9 [15:52:52] paravoid, do you know if there's a labs-console interface for IP quotas, or if I need to do it on the commandline? [15:53:09] I'm afraid I don't [15:53:25] labs-console is not that big, I'd probably look at every page [15:53:53] It's bigger when two out of every three page loads error out :( [16:10:05] whym: Well, I raised your IP quota but labsconsole still isn't showing you as having any. So we'll need to wait for Ryan_Lane to show up to sort this out. [16:10:16] If you need to go to sleep, I can email him about it. [16:11:04] andrewbogott: ok, thank you so much for your help [16:12:23] ^demon: before I investigate… does 'Error 400 on SERVER: Duplicate definition: Class[Role::Ldap::Client::Labs] is already defined' mean anything to you? [16:12:40] andrewbogott: could you ping me here when you get it? i'll probably continue to be online here while sleeping [16:12:51] <^demon> andrewbogott: Lemme login and see if I can replicate. Haven't hit that one yet. [16:13:04] whym: PM me your email address and I'll just move this to an asych channel [16:13:36] ^demon: It's what I got the first time I tried to apply your class on my new instance. You don't have any other local changes that aren't part of your patch? [16:13:57] <^demon> Not right now, everything I've done has been committed. [16:14:23] hm [16:14:30] OK, I'll actually debug in that case. [16:15:15] <^demon> Right now I'm hitting "err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not parse for environment production: No file(s) found for import of '../private/manifests/passwords.pp' at /etc/puppet/manifests/base.pp:10 on node i-0000037a.pmtpa.wmflabs" [16:17:32] <^demon> LG is releasing an 84in HDTV in September. In case anyone's looking for Xmas gifts for me ;-) [16:28:06] ^demon: Yeah, my short-term goal is to figure out how to produce the same failure as you. [16:28:33] And, you should hold out for retina-projection display… that screen will be as big as the whole world! [16:28:40] <^demon> Granted, I've run puppetd -tv on this host about 2 dozen times, so might be worth me firing up a fresh instance and trying that :) [17:02:27] <^demon> Ewww, stacktrace from labsconsole http://p.defau.lt/?4bSEoTJVhprw_mjyZU7_QQ [17:02:37] <^demon> After trying to "Return to instance list" after making a new instance. [17:05:56] ^demon: Labsconsole will be doing things like that more-or-less constantly for the next little while. It seems to be a coin-toss. [17:06:07] If you retry a few times you will eventually get a success message. [17:06:10] F5 ftw [17:07:54] * andrewbogott goes to lunch [17:08:38] <^demon> Hmm, new instance doesn't have my keys :\ [17:09:22] <^demon> Oh nvm, puppet's still running first time. [17:28:51] hi all [17:29:06] i'm trying to follow the instructions on labsconsole to set up a local apt repo [17:29:09] https://labsconsole.wikimedia.org/wiki/Help:Using_debs_in_labs [17:29:36] I've got the /etc/apt/sources.list.d/labdebsrepo.list file [17:29:46] and my .deb in /data/project/repo/ [17:29:55] apt-get update doesn't seem to find it though [17:30:07] Ign file:/data/project/repo/ Translation-en_US [17:38:05] ottomata: you're on precise? [17:39:01] no, lucid [17:39:27] local repo debs only work on precise. [17:40:02] I though that disclaimer got into that wiki page, [17:40:05] but it appears it's not there. [17:40:05] ahh [17:42:02] https://labsconsole.wikimedia.org/w/index.php?title=Help%3AUsing_debs_in_labs&diff=5581&oldid=5580 [17:42:22] danke! [17:42:37] i actually remembered that I had set up an http apt repo already on the machine I was using a long time go [17:42:37] ago [17:42:39] with reprepro [17:42:43] so i'm just using that right now [17:55:05] <^demon> andrewbogott: I think I might've tracked down the duplicate Ldap::client::utils. I believe puppetmaster::self already installs this, so it's conflicting with the gerrit manifest. [18:05:12] <^demon> andrewbogott: Ok, on a fresh instance I managed to make the password error go away (must've been my mistake somewhere). Installing puppetmaster::self worked fine. After I added role::gerrit::labs and pulled the changes, I replicated your error about duplicate ldap classes. [18:05:19] <^demon> So, that's the problem :) [18:25:13] <^demon> andrewbogott: Got pass the ldap client and one other problem with patch set 24: https://gerrit.wikimedia.org/r/#/c/13484/23..24/manifests/gerrit.pp [18:28:27] ^demon: ok! I will catch up. [18:29:15] Hmm [18:30:52] Is it actually possible in gerrit to clone out a patch set? Like it's really a branch, but it's not a branch. Normally you could clone out the working branch then eventually merge it in but gerrit does weird management/merging stuff. [18:31:25] <^demon> You can make a branch then fetch the changes in. [18:31:32] <^demon> That's what git review -d does. [18:32:07] Ah, I suppose that works. Doesn't seem as clean though for some things creating/merging/deleting a branch isn't so clean without scripts to tidy up later. [18:58:11] We really need to use modules, trying to make ops/puppet work outside of labs is near impossible. [19:03:53] ^demon, now you're seeing "Failed to parse template gerrit/gerrit.config.erb…" ? [19:04:05] <^demon> Yep, that's where I'm stuck right now [19:04:16] glad we're seeing the same thing [19:25:46] <^demon> Ryan_Lane: That cron for logs worked, gerrit's log retention is much more sane now :) [19:25:57] cool :) [19:26:21] Damianz: we're working towards that goal [19:26:46] :d [19:27:02] Hmm, what version of puppet runs on labs? Default out of ubuntu repos, right? [19:27:15] for precise, yes [19:27:17] for lucid, no [19:27:37] Ah, lucid custom debs in the repo? [19:27:48] same version as precise [19:27:49] Could explain why I'm just getting syntax errors now I figured out the last bit [19:27:52] it's just a backport, I think [19:28:48] I totally could just make a pacakaging instance in labs rather than using a spare vm which is easier to re-install. [19:32:18] Ryan nearly made be choke on the turkey :( [19:32:24] ? [19:32:46] Thought your last tweet said arse.wikipedia not ase [19:33:05] hahaha [19:33:06] shit [19:33:15] I just realize I tweeted that and this wiki allows anon editing [19:33:25] that's going to end poorly [19:33:29] lol [19:33:42] Maybe we need to train cluebot ng in sign language [19:34:18] Though I do plan to roll in multiple wiki support when I re-write the review interface, get requests for smaller wikis all the time :( [19:34:24] ^demon: OK, I'm going to ask some very dumb, basic puppet questions now. Ready? [19:34:53] <^demon> Ok. [19:35:39] When you define something with 'define'… That is essentially a type declaration, right? [19:35:49] Or does it also implicitly create an instance of that type? [19:36:12] <^demon> I'm pretty sure it's the former. [19:36:16] (Feel free to tell me that my question doesn't even make sense as a question.) [19:36:48] OK. So, in that case, how can a template have something like this? scope.lookupvar('gerrit::instance::ldap_host') [19:37:07] gerrit::instance is a type, not an object. So, does it make sense to ask for the value of a member of a type? [19:37:59] <^demon> Hmmm, I'm thinking too procedurally I think. [19:38:23] http://ase.wikipedia.wmflabs.org/wiki/Wikimedia_Labs [19:38:25] \o/ [19:38:30] labs in ase! [19:39:03] Ryan_Lane: That's super cool! And, I look forward to internet vandals insulting your parentage in sign! [19:39:06] Now we just need internet relay video not irc to practice :D [19:39:13] heh [19:39:24] Hell signtext looks confusing [19:39:36] Which, btw, what does 'ase' stand for? I take it it's different from asl [19:40:12] I think it's american sign language [19:40:25] That's usually ASL though =/ [19:40:32] Hell japan wiki isn't named right either though [19:41:15] Ah, the wiki code for ASL is 'ase' for some reason. [19:41:17] ja? [19:41:21] it's the language code [19:42:06] ^demon: Does 'I'm thinking too procedurally' mean "You're right, I need to instantiate that object someplace"? [19:42:25] <^demon> Well, I thought I was instantiating it in the role when I called it. [19:42:25] Hmm jp redirects to ja, swear that never use to :P [19:43:32] ^demon: Oh, yeah. I expect you are instantiating it. But then the template probably needs to refer to that one in particular. Somehow. [19:44:07] ^demon: btw, I'm not being socratic, I have no idea what the solution is [19:44:24] <^demon> Nor do I, at the moment. [19:55:08] Change on 12mediawiki a page Developer access was modified, changed by Euloiix link https://www.mediawiki.org/w/index.php?diff=575154 edit summary: /* Requested developer access to publish an update of the TagAsCategory extension. */ new section [19:56:36] Change on 12mediawiki a page Developer access was modified, changed by Euloiix link https://www.mediawiki.org/w/index.php?diff=575155 edit summary: Took out an unwanted headline [20:06:03] <^demon> Ryan_Lane: manganese needs a release-upgrade to precise at some point :\ [20:08:54] <^demon> We'll probably have to wait on gitblit until the plugin's ready :\ [20:09:07] <^demon> I don't see any way to make it listen to gerrit's acl's out of the box. [20:45:35] ^demon: I don't at all understand why simply including gerrit::jetty causes that class to instantiate. Again confused about type vs. object :( [20:45:50] 08/23/2012 - 20:45:50 - Updating keys for drtrigon at /export/keys/drtrigon [20:50:50] 08/23/2012 - 20:50:50 - Updating keys for drtrigon at /export/keys/drtrigon [21:02:30] Was it paravoid that did the puppetmaster::self stuff? [21:03:42] yep [21:05:02] Cool, I'll annoy him tomorrow then :D [21:10:51] 08/23/2012 - 21:10:51 - Updating keys for drtrigon at /export/keys/drtrigon [22:06:09] A new version of git-review is availble on PyPI. < I only just installed it! [22:13:34] Bleh, don't understand this manifest [22:14:20] * Damianz wonders if he makes a vm in zone pmtpa but dns domain eqiad does the world explode [22:14:36] you won't be able to [22:14:51] the new interface removes that option [22:15:04] dns domain is linked with zone now [22:15:10] *region [22:15:17] Yeah, currently it has the option though :P [22:15:25] on virt1000? [22:15:33] nah, live [22:15:38] Fuuuuuu stacktrace [22:15:41] heh [22:15:57] I only have one more change needed to be ready for the upgrade [22:16:15] I could make this on virt1000 but last time I tried it never build as it couldn't get to the proxy :o [22:16:36] things build properly in eqiad now [22:16:42] but eqiad isn't a real region yet [22:16:42] Awesome [22:16:50] Yeah it's like test sorta place [22:16:52] and you can't connect to its instances ;) [22:17:01] I made sure of that [22:17:27] I could write a puppet class to connect out then use that connection to connect in, though that would be too much effort and reviewing :P [22:17:45] you'd need for it to be approved ;) [22:17:51] I wonder if I'm suppose to use bots-dev for development, considering I want to trash the instance probably best not [22:18:27] Let's see if I can figure out how to change $gitdir in puppetmaster::self so I can store stuff in project store rather than /var/lib, would make it more useful imo [22:19:15] The last Puppet run was at Wed Aug 15 22:24:07 UTC 2012 (11507 minutes ago). < might want to fix puppet on bastion [22:21:56] I wonder why its broken [22:22:37] Dunno but we really should fix nagios [22:23:02] * Damianz waits for puppet to finish [22:26:06] Are all bastion hosts in the same project? [22:28:01] Actually nvm, seens you can't disable the banner on the client side meh [22:28:49] you can disable the banner on the client side [22:29:05] for ssh, that is [22:29:07] Really? I can only see the motd option in the docs, though I could be looking at the wrong page [22:33:33] * Damianz shrugs [22:49:00] Ryan_Lane: Are puppet classes in labsconsole pulled from git or added manually? [22:49:12] added manually [22:50:10] Ah, hmm might need to get creative here then =/ [22:50:55] hm [22:50:58] I wonder why nagios is down [22:51:06] was it one of the corrupted instances? [22:51:12] yeah [22:51:17] it was back though [22:51:27] well, if it's corrupted, then it won't reboot [22:51:32] it needs to be recreated [22:51:39] I thought it was hmm [22:51:50] oh [22:51:54] I know petan wanted some files off the corrupt instance but I swear nagios has been around after then [22:51:55] maybe it's another instance [22:51:57] Under a different box name [22:52:12] * Damianz isn't actually in that project or would poke it with fire [22:52:50] well, the webserver is there [22:53:01] it's the "it works" page, though [22:53:15] !nagios [22:53:15] http://208.80.153.210/nagios3 http://nagios.wmflabs.org/nagios3 [22:53:29] that works [22:53:36] maybe just missing the redirect [22:53:40] ah [22:53:41] likely [22:54:22] down 48days heh, I know this is labs but we probably should clean monitoring up so it's useful [22:54:42] what's down 48 days? [22:54:44] * Damianz wonders if petan's thing is simple enough to just filter by stopped hosts [22:55:06] deployment backup, deployment cache upload, nova daas 1 [22:55:06] nope [22:55:10] it has no idea if they are stopped [22:55:42] nrpe is still broken anyway [22:56:48] interesting.... [22:57:04] I don't see how this is working [22:57:10] I don't see the nagios software installed [22:57:15] lol [22:57:29] I assume it's not on 10.4.0.120 anymore [22:57:38] it's installed.. [22:57:38] seen as nrpe is just refusing connections on everything [22:57:43] I don't see how nagios is serving i [22:57:43] it [22:57:46] err [22:57:47] apache [22:58:11] did someone like symlink it into default rather than setup a proper config? [22:58:33] ah [22:58:33] actually they can't, it's cgi shizzle [22:58:36] it's in the cgi-bin [22:58:58] what's the internal ip? [22:59:08] I see how it works [23:00:05] 10.4.0.34 I'd assume [23:00:48] there we go [23:00:54] added the redirect [23:01:12] ah. right. need to update nrpe [23:01:15] Puppet is right ip wise, some hosts seem to need a force puppet run [23:01:28] weird [23:02:09] some hosts probably have broken puppet [23:02:13] or puppetmaster::self [23:02:31] lots and lots of hosts, it seems [23:02:38] I can manually fix them, I guess [23:02:42] nrpe_allowed_hosts ; "labs" => "10.4.0.34" but a brand new instance I *just* built 10.4.0.120 [23:02:45] wth [23:03:25] Ah fuck [23:03:32] Why is my clone on the test branch still [23:03:59] Yeah [23:04:01] prod branch is wrong [23:04:06] so just change that and wait a while [23:06:25] fixing them manually [23:06:33] some may re-break themselves with old puppetmaster::self repos [23:06:43] this is one of the crappy things about puppetmaster::self [23:07:02] yeah [23:07:20] It's more the lack of merging from remote though in general :( [23:07:30] It would be nice to say like prod apart from THIS file [23:07:54] yep [23:07:58] not possible, though [23:08:13] Sadly not [23:09:21] Review in ops when bored [23:09:59] I really suck at a 'short overview' 'long description' in commits [23:10:40] Thinking about puppetmaster::self, it's more annoying because there's at least 2 repos. [23:11:11] well, eventually we'll just use environments from a single puppet master [23:11:19] Permission denied (publickey). ah freaking hell, I can't pull because I'm su'd to root [23:11:31] heh [23:11:35] yeah… it sucks [23:11:50] someone did a chown, pull, chown the other day [23:11:53] Frick it, chowning to me can't be that bad [23:12:07] It can stay chown'd to me for now :P [23:12:51] Oh, I totally don't have my agent forwarded either [23:13:21] Considering labs private has the key in there, we could easily fix this by cloning over http. it would suck for merging back for review but then doing that as root is bad anyway [23:14:47] Ah I see [23:14:49] GIT_SSH=/var/lib/git/ssh git pull [23:15:01] bash wrapper passing in the ssh key [23:15:08] * Damianz yays at working for puppet as well as private [23:15:36] Weird security model that anyone can install something that lets them access 'private' yet we can't see it in gerrit :P [23:15:41] Damianz: http://i.imgur.com/mnj9j.jpg?1?4359 [23:15:59] that's because I broke it in gerrit [23:16:08] I need to do some dev to fix it again [23:16:31] rofl [23:16:42] I'd happily hack on it but it's written in bloody java [23:16:55] nah, I broke it in ldap/labsconsole [23:17:00] Ah [23:17:29] It's in php, same feels to less of an extend :) [23:17:35] s/extend/extenet/ [23:17:42] :D [23:21:06] Can I just make up a 'namespace' in puppet and hope the ops gods love me later? [23:21:54] Gerrit-Reviewer: gerrit2 < it's good to know *someone* reviews my shit [23:22:15] heh [23:22:19] make up a namespace? [23:24:17] Like if I want to puppetize a bot while rewriting part of it, it will end up in misc but will have to get added to labsconsole etc eventually. assume misc::bots::cbng or suck would be acceptable as I'm not aware of anything like misc::randomdudes::damian::bots::cbng being a standard for 3rd party stuff being in prod [23:24:52] make it a module [23:25:03] I could do that actually [23:25:13] and then add a role to configure it [23:25:21] then you add the role to labscosnole [23:25:24] labsconsole [23:26:02] role == group? [23:26:10] no, like a role class [23:26:16] role::ldap::client::labs [23:26:22] in manifests/roles/ldap.pp [23:26:24] for instance [23:26:49] then in the node you include role::ldap::client::labs [23:27:08] So module generic stuff, manifest lab/prod specific config and load the module [23:27:42] module generic stuff, then have roles for non-generic stuff [23:27:54] the roles load the modules [23:28:02] the modules should ideally have paramaterized classes [23:28:10] makes sense [23:28:33] In that sense anything private would just be passed into the class as a var (like db logins/wikipedia password etc) [23:29:01] yep [23:29:27] Cool, totally doing puppet first as it seems easier than building debs :P [23:30:43] heh [23:30:59] debs would be easier if we had per-project repos [23:31:03] want to work on that next? :) [23:31:16] Thinking about it, would you have one module to include others or specify multiple in the role class? (for example if multiple required the same details). [23:31:31] multiple in the role class [23:31:37] I'd love nothing more than to push a deb source to gerrit, have it built automaigcally then on merge pushed to mirror. [23:31:40] that way the modules don't have many dependencies [23:31:44] that would be ideal [23:31:47] jenkins could do it [23:32:13] I was thinking of the security implications of having a script to call to upload the stuff, seeing as anyone has the option to add jobs. [23:32:53] Also I'd rather not have it manual, but no idea if we could just say anything matcing /ops/debs/* build as a deb, or have an option in gerrit (plugin?) totally more ^demon|away's area of know how ;P [23:38:12] well, the source should be included [23:42:42] source? [23:46:42] err: Could not retrieve catalog from remote server: Error 400 on SERVER: Cannot allocate memory - fork(2) at /etc/puppet/manifests/ssh.pp:59 on node i-000003cd.pmtpa.wmflabs < maybe puppet at tiny instances don't get on so well. [23:49:45] tiny doesn't work very well [23:49:52] I'm thinking of deleting that flavor type [23:50:03] Might rebuild this then :P [23:51:15] * Damianz pokes paravoid and wonders if he's around or just visiting [23:51:33] hi Damianz [23:51:42] Hi [23:51:48] well, I'm not here for long, it's almost 3am [23:51:51] but if it's something quick, shoot [23:51:59] Ouch, it's only like 1 here heh [23:52:18] <^demon|away> <- 8pm. Dinner time :) [23:52:51] I was just wondering if it would be possible/easy to make $gitdir on puppetmaster::self configurable. Ideally if I could move /var/lib/git to project storage so it persisted accross machine rebuilds and could in theory be used for testing 'cluster' stuff. [23:53:22] Clearly it would need to be different per project/person/task but useful in some cases I think [23:54:04] hmm, indeed, doesn't sound like a bad idea [23:56:23] I'm fairly busy these days with other stuff [23:56:59] so you could either wait or prepare something and I'll review/push it ;-) [23:57:33] Thought so heh, I was going to take a bash at it but not quite sure how the 'sub class' gets called and how to lookup the value out the parent. If I get anywhere I'll give you a poke :) [23:58:22] feel free to ask too, I guess I could help there too [23:58:33] well, not right now :), but since we're in close timezones [23:58:54] where are you based off if I may ask? the UK? [23:58:54] Yeah I don't have to wait until afternoon to talk to you lol [23:59:05] Yep middle of the uk