[00:13:54] For some reason, whenever XFS craps out many of AnomieBOT's tasks just die instead of hanging waiting for IO. But since IO is b0rked it doesn't log why exactly they die. Does XFS breaking screw up either mysql or redis? [00:26:52] is it safe to submit jobs again on tools albs? [00:31:51] Coren: whats the eta on conversion to the new DC? [00:53:02] Betacommand: Couple of weeks. We have running openstack right now and are working on the services and testing. [21:25:12] http://tools.wmflabs.org/ takes a while to load [21:25:15] Should it be cached? [21:27:22] huh: The content is mostly dynamic. [21:27:56] Coren: can it be cached or made more efficient? [21:28:57] i get some email from cron. is the grid somehow broken? [21:29:05] +s [21:29:45] It doesn't seem to be. What do the email say? [21:30:31] segfault, oom, no such file in spool [21:32:58] tools-login seems to be clogged. [21:33:46] Coren: Wanted to have a go at https://bugzilla.wikimedia.org/show_bug.cgi?id=18831, would it be possible to do that in the Bugzilla labs project? Does it have wikibugs etc. installed? [21:34:20] What an asshole. I had disabled his crontab with a note what he needs to do to submit jobs to the grid. *Argl* [21:34:51] You disabled it and he reenabled it? [21:35:10] Coren: Yes, and removed my comments. [21:35:18] scfc_de: Gimme account name please. [21:35:30] local-shuaib-bot [21:35:37] Howdy team - I'm assuming you guys are aware of issues? Just woke up to a flood of cron mail ;) [21:36:47] Krenair: I think both Bugzilla in production and in Labs is "special"; you probably need to query Andre directly about that. He's andre__, I believe (or three underscores?). [21:36:59] Hello all! Where do I have to apply in order to get an maillist on lists.wikimedia.org for my bot project on labs? [21:37:30] scfc_de, yeah, he's not online at the moment [21:37:57] DrTrigon, I think those go through the mailing lists component of Wikimedia on BZ [21:38:35] scfc_de: Removed shell access. Feel free to comment out the crontab again, ima give him a talk page message. [21:39:00] DrTrigon: https://meta.wikimedia.org/wiki/Mailing_lists#Create_a_new_list [21:39:06] wow, drastic [21:39:15] Coren: Already done that & pkilled -HUP. [21:40:43] Thanks guys!!! [21:40:49] *bye* [21:40:53] bye [21:41:06] giftpflanze: One only gets so many free passes. When you reenable a crontab when it has been commented out with instructions from the sysadmin, you need to acknowledge before you can go on. [21:41:26] i guess so [21:42:26] scfc_de: https://wikitech.wikimedia.org/wiki/User_talk:%D9%85%D8%AD%D9%85%D8%AF_%D8%B4%D8%B9%DB%8C%D8%A8 if you want to use as a template. [21:43:00] Coren: Bookmarked, thanks. [21:44:18] scfc_de: that mailing list wiki page is maybe technically true, but I requested an email list *ages* ago, and it still hasn't happened. [21:47:40] valhallasw: DrTrigon asked where to apply, not where to get one :-). nlwikibots? [21:47:50] yeah, that one [21:48:09] (I just bumped the bug, that will probably help.) [21:48:42] Thehelpfulone is even in this channel :-). [21:52:21] scfc_de, so what do you mean by 'special'? [21:52:46] Is Thehelpfulone ever around? [21:52:56] I haven't seen them for weeks if not months. [21:56:51] Krenair: I think it's mostly, but not completely puppetized; *but* wikibugs isn't part of that at all, but sitting as a script on the mail server (mchenry). I think there's a bug where ops still hasn't disclosed what version of wikibugs is currently deployed, because another bug has been fixed in the repo, but isn't live. [21:58:32] Krenair: (https://bugzilla.wikimedia.org/show_bug.cgi?id=42774) [22:00:25] Now if we just had mail on tool labs, we could use that as wikibugs hook ;-) [22:00:55] (although that would lag a bit due to the mailing list, but it always worked without issues for the pywikipedia bugs mailing list) [22:04:02] I looked around a bit recently, and I think a better way would be a Bugzilla extension like Push that relays Bugzilla's "recent changes" to a message queue to which an IRC bot can then subscribe. But (obviously) a lot more work than fixing up a regex in wikibugs :-). [22:06:03] Ah, that would be even cooler. [22:07:36] (also a lot more stuff that can break) [22:09:32] I think the current setup (Bugzilla sends mail -> mail server -> local delivery -> wikibugs) has a lot of moving parts as well :-). [22:10:27] Yes, but bugzilla mail (and mailing lists, etc), have been there for ages, are unlikely to break, and if they break, it's high-priority to fix it. [22:11:52] And how long have these bugs been open? :-) [22:17:40] scfc_de: my point exactly. We should get bugzilla data over well-supported gateways to a community-maintained place :-)