[00:34:11] 03/03/2012 - 00:34:11 - Creating a home directory for thehelpfulone at /export/home/bastion/thehelpfulone [00:35:13] 03/03/2012 - 00:35:13 - Updating keys for thehelpfulone [00:41:59] andrewbogott_: Can you add me to the bugtracker project? [00:42:11] I think so... let's see :) [00:44:07] johnduhart: How's that? Better? [00:44:10] 03/03/2012 - 00:44:10 - Creating a home directory for johnduhart at /export/home/bugtracker/johnduhart [00:44:19] that's encouraging... [00:44:48] Great, can you also make me a sysadmin so I can create instances? [00:45:10] 03/03/2012 - 00:45:10 - Updating keys for johnduhart [00:47:07] Dantman, are you happy for johnduhart to join you as sysadmin in bugtracker? [00:47:23] sure [00:47:24] * andrewbogott_ doesn't know what the policies are about this, but looks lie Dantman is the only one invested so far. [00:47:29] cool, thanks. [00:48:00] I was interested... but I never got around to it, I didn't know how to use Nova, and I didn't have a bugzilla dump I could import [00:48:15] Dantman: I'm going to send an email to wikitech-l about getting ahold of a dump of bugzilla and what trackers we should test [00:48:21] Sure [00:48:32] The big focus is Redmine/ChilliProject [00:48:33] What software did you have in mind? [00:48:38] Ah [00:49:05] One of the old WMF people looked over it and said it had some project management things that WMF could use [00:49:18] It also has a good repo viewer that includes git support [00:50:03] We may be able to integrate some form of external authentication into it to use either MW.org or Labs accounts [00:50:21] I'm looking at testing JIRA, YouTrack, The Bug Genie [00:50:31] And I'd like to see if we can find some way to alter it to make fixing our spam issue easier [00:50:44] It also has good activity feeds [00:50:53] So you can actually watch changes with RSS [00:51:12] johnduhart: I'm about to vanish... are you all set? [00:51:26] andrewbogott_: Yep, thanks for your help [00:51:29] np [00:55:22] !log bugtracker Creating bugtracker-proxy instance to act as a reverse proxy instance [00:55:28] Great [00:55:44] petan|wk: morebots died [01:01:36] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 3.04, 4.12, 5.28 [01:03:26] is the WMF labs server open SSH? [01:03:32] Yes [01:04:22] okay, I'm trying to get it set up on putty on Windows but I'm not quite sure what I'm doing wrong [01:04:36] What is happening? [01:05:08] I get a "No supported authenications method available" error message [01:05:22] Thehelpfulone: You need a SSH key to log in [01:05:26] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [01:05:26] I loaded my private key in Pageant [01:05:31] Hm [01:05:38] and loaded my public key onto the Special:NovaKey [01:05:52] it's a SSH-DSA key - would that break it? [01:06:00] Don't think so [01:06:24] Dantman: I need you to add 2 security rules, if you could [01:06:42] johnduhart: this is my first time using SSH, so presume that nothing's perfect, any ideas on where to troubleshoot? [01:07:22] johnduhart: If we have the same rights could you? [01:07:28] Thehelpfulone: Something isn't working with paegent and putty, I'm not sure where to look [01:07:30] johnduhart: I've never actually used anything on nova [01:07:35] Dantman: You have netadmin, I don't [01:07:41] Oh... ok [01:07:51] Dantman: It's easy, I'll just give you some values and you submit a form ;) [01:07:59] the hostname is bastion.wmflabs.org right? [01:08:03] yup [01:08:16] Dantman: Once you're signed in https://labsconsole.wikimedia.org/w/index.php?title=Special:NovaSecurityGroup&action=addrule&project=bugtracker&groupname=default [01:09:13] Form port: 80 To Port: 80 protocol: tcp CIDR Ranges 0.0.0.0/0 [01:09:26] :/ stupid LastPass bug [01:09:32] That allows us to access HTTP externally [01:10:30] I'll re-add the key johnduhart although I doubt that's it [01:11:13] 03/03/2012 - 01:11:13 - Updating keys for thehelpfulone [01:11:15] I'm trying to use the agent forwarding method, so I've checked "allow agent forwarding" and "allow authentication through pageant" was already checekd [01:11:46] johnduhart: Group rule bugtracker/default? [01:12:04] Dantman: Don't touch group rule [01:12:12] my username should be thehelpfulone too - is there a difference between Thehelpfulone and thehelpfulone? [01:12:17] "Note: Individual rules and group rules are mutually exclusive. Choose one or the other, not both." [01:12:20] I know it's confusing [01:12:41] I messed that up when I was working on phabricator :p [01:13:15] Thehelpfulone: Not sure, I would make sure it's lowercase though [01:13:51] done [01:13:59] No irc update? [01:14:08] Dantman: Thanks [01:14:14] * johnduhart proceeds to take over the world [01:14:17] muhaha [01:14:22] Dantman: Guess not [01:14:35] We get an update when someone updates their own key, but nothing when we add a new hole into the firewall? [01:14:53] nope [01:24:22] Great, nrpe is broken [01:26:36] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 3.38, 4.01, 4.83 [01:32:12] 03/03/2012 - 01:32:12 - Updating keys for thehelpfulone [01:32:56] damn. any puttyers about? [01:35:56] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [01:39:36] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 7.39, 6.39, 5.39 [01:41:56] johnduhart: how do you check that you actually have access to bastion? [01:42:06] PROBLEM Free ram is now: WARNING on bots-cb bots-cb output: Warning: 10% free memory [01:42:25] Thehelpfulone: https://labsconsole.wikimedia.org/wiki/Special:NovaProject [01:42:31] You're on the list for bastion [01:43:19] then I have no idea what's going wrong [01:43:46] I created a new SSH key, re-uploaded it - deleted all configuration settings for putty.. [01:47:06] PROBLEM Free ram is now: CRITICAL on bots-cb bots-cb output: Critical: 5% free memory [01:52:06] RECOVERY Free ram is now: OK on bots-cb bots-cb output: OK: 82% free memory [02:05:56] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [02:35:56] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [02:41:46] RECOVERY Free ram is now: OK on puppet-lucid puppet-lucid output: OK: 20% free memory [03:06:06] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [03:36:06] PROBLEM host: bugtracker-proxy is DOWN address: bugtracker-proxy PING CRITICAL - Packet loss = 100% [03:40:37] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 2.92, 4.56, 5.26 [03:45:47] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [04:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [04:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [05:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [05:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [05:55:37] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 3.38, 3.21, 4.49 [06:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [06:33:37] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 7.27, 7.53, 6.22 [06:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [07:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [07:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [07:53:37] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 2.75, 3.16, 4.51 [08:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [08:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [08:46:37] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 4.92, 5.84, 5.34 [09:01:37] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 2.60, 3.87, 4.75 [09:12:23] could someone fix gadgets at http://hi.wikipedia.beta.wmflabs.org now that the update is done? :) [09:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [09:24:49] anyone here? [09:27:09] anyone here? [09:27:33] could someone fix gadgets at http://hi.wikipedia.beta.wmflabs.org now that the update is done? :) [09:28:22] its missing the gadgets tab in prefs and the gadgets dont load. This has happened twice before. [09:28:46] oh fcuk [09:29:11] thanks, now I can feel this bug with my own hands:)) [09:29:25] nobody touch anything! [09:29:31] MaxSem: yeah, its been there since 29th. [09:29:43] Reedy purged it [09:32:02] seems its still there [09:34:03] blargh, where's the deployed copy of MW located? [09:34:11] MaxSem:The problem is definitely still there [09:37:35] > grrr [09:37:35] PHP Notice: Use of undefined constant grrr - assumed 'grrr' in /usr/local/apache/common-local/live-hom/maintenance/eval.php(72) : eval()'d code on line 1 [09:45:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [09:59:34] damn, my sudo is restricted [10:01:46] I'll poke someone who knows how it's supposed to work when they get up [10:09:37] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 5.61, 6.95, 5.59 [10:15:57] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [10:46:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [10:50:23] petan:u here? [10:51:38] petan:could u fix gadgets at http://hi.wikipedia.beta.wmflabs.org now that the update is done? :) [10:52:20] PROBLEM Current Load is now: WARNING on bots-sql3 bots-sql3 output: WARNING - load average: 5.61, 6.63, 5.45 [10:54:13] MaxSem:How long before people get up? [10:57:20] RECOVERY Current Load is now: OK on bots-sql3 bots-sql3 output: OK - load average: 1.26, 3.48, 4.43 [11:01:40] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 2.93, 3.85, 4.72 [11:16:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [11:40:24] could someone fix gadgets at http://hi.wikipedia.beta.wmflabs.org now that the update is done? [11:46:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [12:16:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [12:22:45] hmm [12:22:47] I got to https://labsconsole.wikimedia.org/wiki/Special:NovaKey [12:22:53] and I get "No Nova credentials found for your account." [12:23:14] There were no Nova credentials found for your user account. Please ask a Nova administrator to create credentials for you. [12:23:26] it worked yesterday? [12:46:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [13:07:41] Thehelpfulone: Logout and log back in again [13:14:21] Damianz: ah that worked [13:14:30] now I'm going to try putty again [13:15:48] Damianz: do you have any experience with setting up PuTTY on windows? I keep getting a "no supported authentication methods available" error [13:16:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [13:18:54] Thehelpfulone, it usually works out of the box [13:19:11] did you mess with the options for authentication? [13:19:21] or the allowed ciphers? [13:19:26] Platonides: that's what I thought, nope I just checked allow agent forwarding [13:19:41] I tried using pageant too [13:19:47] bastion.wmflabs.org [13:19:53] username: thehelpfulone [13:19:56] port 22 (default) [13:20:13] at the end of that message it says (server sent: publickey) [13:20:19] do you have the private key loaded? [13:20:45] yes [13:21:03] in pageant, but not in the configuration of putty [13:21:07] should I do it for both? [13:21:24] maybe your putty is not communication right with pageant? [13:21:35] close pageant and try with just putty first [13:22:20] okay, I did that [13:22:24] it says same error message [13:22:33] but it does say in the console "Server refused our key" [13:22:42] is it the right key? [13:22:50] it should be - I've only created one key [13:22:52] in which format did you provide your public key¿ [13:23:01] Thehelpfulone: Did you add your key in the openssh format in the wiki? [13:23:09] It currently doesn't convert the format properly. [13:23:09] I tried SSH2 DSA first [13:23:15] then I tried SSH2 RSA [13:23:19] what's the openSSH format? [13:23:33] mine starts off with ---- BEGIN SSH2 PUBLIC KEY ---- [13:23:35] ssh- on one line [13:23:43] wrong [13:23:51] you need to export in openssh format [13:23:57] ah [13:24:10] how do I do that? [13:24:19] it's in a menu in puttygen [13:24:25] Load the key in puttys keygen thing, unlock it then click export. [13:24:30] you can convert the one you have [13:25:12] got it [13:26:39] hmm, it only allows me to export the private key into SSH? [13:26:47] Public key [13:27:17] yeah it says "load an existing private key" [13:27:57] oh found it [13:28:06] does my private key also need to be in the OpenSSH format? [13:28:12] Nah [13:29:12] 03/03/2012 - 13:29:11 - Updating keys for thehelpfulone [13:29:52] great it works [13:31:46] :) [13:32:02] Feel free to submit a patch to close the bug about converting keys into the right format :P [13:41:16] Damianz: if I want to run a bot, I need access to the bots group? [13:41:35] What bot do you want to run? [13:41:50] pywikipedia scripts [13:42:10] 03/03/2012 - 13:42:10 - Creating a home directory for thehelpfulone at /export/home/bots/thehelpfulone [13:42:14] redirect.py and noreferences.py [13:42:46] bots-3 should do for you [13:43:05] or bots-4 I guess, didn't know we where on 4 [13:43:09] 03/03/2012 - 13:43:09 - Updating keys for thehelpfulone [13:44:35] hmm I didn't tell it to do that [13:44:40] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 6.66, 6.55, 5.52 [13:44:50] the updating keys bit [13:45:01] I added you to the bots project so it updated your keys on the bots servers. [13:45:01] or is that added to the bots project [13:45:19] ah okay [13:46:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [13:50:15] Damianz: a n00b question, how do I get to bots3? [13:50:55] oh it's bots-3 [14:16:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [14:46:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [14:54:40] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 3.44, 3.50, 4.83 [15:16:40] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [15:31:07] Damianz: Do you know anything about why nrpe would be causing instance creation to fail? [15:33:26] Thehelpfulone: What's up? [15:34:25] hey, I'm wondering how do I get to pywikipedia on bots-3? [15:34:37] or rather, how do I put it on bots-3 [15:35:24] Should just be able to checkout the code to your home folder and use it like any pywikipedia install [15:37:13] just a simple "svn checkout http://svn.wikimedia.org/svnroot/pywikipedia/trunk/pywikipedia/ pywikipedia" ? [15:38:06] yup [15:39:39] okay, there's a couple of changes that I made to the original pywikipedia scripts, how do I edit the scripts or upload my ones? [15:40:57] I would use an SFTP client to upload them or upload them somewhere web accessible and then download them [15:41:35] I have FileZilla, how do I SFTP? [15:43:43] Thing is that's a bit tricky since you need to go through bastion. Hmmm. [15:45:05] PROBLEM host: bugtracker-jira is DOWN address: bugtracker-jira PING CRITICAL - Packet loss = 100% [15:46:45] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [15:47:24] how do you connect to bastion to start off with? [15:47:45] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 6.00, 5.82, 5.20 [15:48:17] Thehelpfulone: Well maybe you can upload them to bastion first. [15:50:29] I'm not quite sure how I connect to bastion johnduhart [15:51:06] You should just be able to put in the hostname for it in FileZilla and choose the SFTP option [15:51:50] oh it works now, I chose the interactive mode [15:51:57] does it not require my SSH key file? [15:52:19] I guess pageant is doing that for you [15:52:35] ah yes [15:55:34] johnduhart: so I can just put them in /home/thehelpfulone ? [15:55:39] Yeah [15:56:17] okay, it was only one file at the moment [15:56:25] how do I get that into bots-3? [15:57:37] Thehelpfulone: Look up scp to copy the file from bastion to bots-3 [15:58:21] look up scp? is there a page that tells you how to do that? [15:59:11] http://www.basicconfig.com/linux/linux_scp_command_example [15:59:26] Skip the install part [16:02:12] so "scp -r thehelpfulone@bastion : /home/thehelpfulone/noreferences.py" ? [16:02:45] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 2.90, 4.09, 4.88 [16:03:20] Thehelpfulone: Yes, except without the spaces around the colon [16:04:13] okay, I did that and it gave me examples of usage? [16:04:39] scp thehelpfulone@bastion1:/home/thehelpfulone/noreferences.py . [16:04:43] Try that [16:04:45] Include the period [16:05:06] that gets me somewhere [16:05:11] but it says permission denied [16:05:29] permission denied (public key) [16:05:38] Okay, when you SSH'd to bots-3, did you use the -A parameter? [16:05:45] ssh -A bots-3 [16:05:53] That passes your keys over [16:06:06] ah no - I did tick agent forwarding though [16:06:16] in putty [16:06:41] That only goes one level [16:07:15] okay so do I need to restart putty? [16:07:20] no [16:08:00] what do I type? :) [16:08:14] When you ssh to bots-3: [16:08:17] ssh -A bots-3 [16:08:27] Then the scp command should work [16:10:01] if putty is stuck on a green rectangle does that mean it's loading? [16:10:12] Huh? [16:10:38] I put that ssh -A bots-3 command on the autorun command [16:10:43] I got up to Authenticating with public key "wmf-labs" from agent [16:10:48] but now it's not doing anything [16:10:49] No don't do that [16:10:56] Just run the command yourself [16:10:59] ok [16:11:30] okay it worked [16:12:44] johnduhart: now how do I move those into the pywikipedia directory? [16:12:51] I probably should have cd'ed to it first [16:12:59] use the mv command [16:13:50] how can I view the directory at the moment? [16:13:55] ls [16:15:15] PROBLEM host: bugtracker-jira is DOWN address: bugtracker-jira PING CRITICAL - Packet loss = 100% [16:16:11] okay, so in pywikipedia/ [16:16:21] I moved noreferences.py into it [16:16:28] but in my local folder, there's also a file called noreferences.pyc [16:16:41] which doesn't seem to exist on bots-3 [16:16:45] PROBLEM host: bugtracker-web is DOWN address: bugtracker-web PING CRITICAL - Packet loss = 100% [16:16:45] I didn't create it [16:16:49] Don't copy it [16:16:56] ok [16:17:29] how do I run login.py? [16:17:41] python login.py [16:18:41] now the command I usually run is [16:18:42] noreferences.py -cat:"Pages with missing references list" -namespace:0 -always -family:wikipedia -lang:en [16:18:58] how do you set that to run every x hours, is it called a crontab? [16:19:10] Yes, google how to setup crons [16:26:36] Grrr, this broken nrpe bullshit is really annoying [16:32:00] johnduhart: /bin/nano is the easiest editor? [16:32:04] yes [16:33:26] so to run a task, every 6 hours for example, would it be this: [16:34:06] * */4 * * * /home/thehelpfulone/pywikipedia/python noreferences.py [16:36:49] no [16:37:12] The python executable isn't in that folder [16:37:19] plus you didn't pass any parameters [16:37:35] yeah that was without the parameters [16:37:42] where is the python executable then? [16:38:20] * johnduhart shurgs [16:38:23] Run which python [16:38:41] the python that makes pywikipedia work :) [16:38:50] which python [16:38:52] That's a command [16:39:05] oh [16:40:38] that gives /usr/bin/python [16:41:14] * */4 * * * /home/thehelpfulone/pywikipedia/python noreferences.py -namespace:0 -always -family:wikipedia -lang:en [16:41:22] is what I am up to at the moment [16:42:24] * */4 * * * /usr/bin/python /home/thehelpfulone/pywikipedia/noreferences.py -namespace:0 -always -family:wikipedia -lang:en [16:44:00] ok [16:47:24] how do I save it? [16:51:28] Look at crontab guides on Google, I'm not an expert on crons [16:56:04] okay I think it worked [16:56:12] now we just need to wait and see :) [18:20:16] johnduhart: it's working thanks, are there any restrictions as to how often tasks should be run? [18:37:14] Not that I know of [18:39:11] robla: Are you good with puppet, or do you know of an op that I can talk to about an issue? Instance creation is broken atm [18:39:42] johnduhart: sorry, I'm not good w/Puppet [18:40:22] ...and I'm pretty sure most folks would defer to Ryan_Lane about that type of thing [18:40:37] He's camping atm :/ [18:53:49] mark: Are you able to help with a puppet issue? [19:13:44] PROBLEM Current Load is now: WARNING on dumps-nfs1 dumps-nfs1 output: WARNING - load average: 6.56, 6.77, 5.49 [19:28:44] RECOVERY Current Load is now: OK on dumps-nfs1 dumps-nfs1 output: OK - load average: 4.07, 4.44, 4.91 [19:49:45] Could someone fix gadgets at http://hi.wikipedia.beta.wmflabs.org ? [19:51:08] johnduhart: There is sometimes issues with the apt mirror which causes breakages, usually that's around puppet and it failing the first run though. [19:57:14] could someone fix the gadgets? There's no gadgets tab in the prefs panel and the scripts dont load [19:58:08] I could but I don't have access to the server. Might want to email the mailing list and someone will do it. [19:58:49] Damianz:I would but its not a long process to fix it. Its been fixed through this chan twice before [19:59:31] Damianz: Plus, it just recurs; and this time i've been waiting since 29th (since back then devs were busy with the deployment) [20:01:02] MaxSem or Reedy or petan : Could any of u guys fix this pleeeeease? [20:01:34] anyone who knows how to do live hacks around? [20:08:57] I've no idea how labs is setup.... [20:09:39] Reedy: If I remember correctly, you were the one to fix it the second time (I may be wrong) [20:09:51] I fixed it both times [20:09:57] sudo neither barfs nor helps [20:10:15] now we've got an empty array in cache [20:10:25] Reedy:Then can u pleeeeeeease fix it :) [20:10:33] use eval.php and purge the key? [20:10:38] whaddaya need it for? [20:11:11] I need to repro. for repro I need my debugging code to be in place when I reset everything [20:12:00] How do you push code out to labs? [20:20:25] Reedy:So, can this be fixed? [20:20:42] Yes, but Max is wanting to try and debug it to stop it happening [20:20:58] ohkay :) [20:21:24] Go MaxSem! :D Go Reedy! :D [20:21:33] which NovaProject is *.beta.wmflabs.org under ? [20:22:25] * Damianz paws Krinkly [20:22:44] deployment-prep [20:23:45] MaxSem: and what role do you need to have on there to add extensions for example? [20:23:56] no idea [20:24:16] I just need to introduce local hacks [20:24:51] Krinkle is listed as a sys-admin there - Krinkle what role do you need to have to be able to add extensions on beta.wmflabs.org (deployment-prep) [20:25:01] None [20:25:15] well, minimal [20:25:19] You should just need shell access to the project [20:25:46] Reedy: okay, there's a request on http://labs.wikimedia.beta.wmflabs.org/wiki/Global_Requests#Extensions [20:25:55] [20:25:57] requesting that the translate extension be enabled on wikimedia-incubator [20:29:00] could you do it? [20:29:41] I've not configured mediawiki before but if it's as simple as adding one line to localsettings.php as per http://www.mediawiki.org/wiki/Manual:Extensions#Installing_an_extension -> if you give me access, I could have a go [20:48:42] Needs configuring and database tables creating [20:52:46] MaxSem:Any luck? [21:49:21] Sid-G: hi [21:50:33] Thehelpfulone: no it's not so easy [21:50:54] petan: ah okay, it says that it was enabled on live but not beta [21:50:55] there is no local settings ;) [21:51:01] let me check [21:51:45] petan, how do I make local hacks, sudo doesn't help [21:51:56] local hacks to what [21:52:47] to deployed code on deployment-web [21:53:21] Sid-G: no idea to me gadgets work [21:53:37] deployed code is updated from svn trunk [21:53:55] so if u want to change commit the change to svn, then svn up [21:54:24] eh, there should always be the possibility to do local debugging [21:55:24] hm, you can of course make changes to files too, if you are member or depops group, but the changes will be overwriten [21:55:38] site is for testing software before deployment [21:55:53] labs are going to be for development, these are still being prepared [21:56:21] Hmm I'm lacking in food. [21:56:31] * petan feeds Damianz [22:04:31] Damianz: Nah, apt-get runs fine [22:04:42] petan: Did you see my labs-l email? [22:07:13] * Damianz eats petan, mmm cookies [22:08:11] petan: are there any restrictions on wmflabs as to how often you can run your pywikipedia scripts? [22:08:21] I've set it hourly at present but I can reduce that if needs be [22:08:52] currently it's only for redirect.py and noreferences.py - but I'd like to use interwiki.py if that's not too much of a resources hog [23:59:15] if you have a task running in a session [23:59:25] can you leave the session but leave the task running?