[02:15:24] It seems that https://tools.wmflabs.org/templatecount/ 's webservice is stopped. Is it possible for server admins to start it? [03:08:59] (03PS4) 10Awight: Write the "blacklist" config variable as an array [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/119924 [05:44:56] Oh [06:50:38] PROBLEM - Puppet failure on tools-exec-06 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:54:34] PROBLEM - Puppet failure on tools-master is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:15:38] RECOVERY - Puppet failure on tools-exec-06 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:34] RECOVERY - Puppet failure on tools-master is OK: OK: Less than 1.00% above the threshold [0.0] [09:52:05] 10Tool-Labs: Error mails from SGE are encoded as application/octet-stream - https://phabricator.wikimedia.org/T63160#1201622 (10scfc) The debugging showed that it was indeed called (only) on `tools-exec-*` and `tools-master`. So generalize those to "all hosts that execute jobs" and "all hosts that master the gr... [12:00:28] 10Wikimedia-Labs-wikitech-interface: Possible to register mail addresses with trailing newlines - https://phabricator.wikimedia.org/T73692#1201743 (10scfc) [12:31:21] bearND: Hallo! [15:51:02] !xtools is You might want to check for an xTools maintainer in #xTools. [15:51:02] Key was added [17:12:58] 6Labs, 6operations: One instance hammering on NFS should not make it unavailable to everyone else - https://phabricator.wikimedia.org/T95766#1201894 (10JanZerebecki) At the time this happened I looked at https://grafana.wikimedia.org/#/dashboard/db/labs-monitoring and the inbound traffic on labstore1001 nearly... [18:31:18] 10Tool-Labs: Fix htmlpurifier deployment - https://phabricator.wikimedia.org/T95815#1201040 (10scfc) [18:36:07] 10Tool-Labs: Fix htmlpurifier deployment - https://phabricator.wikimedia.org/T95815#1201914 (10scfc) [19:24:49] * valhallasw`cloud prods grrrit-wm [19:37:13] oh, it's in puppet, of course [19:37:54] so grrrit-wm should report it but only if modules/toollabs/* is changed. Beh. [19:39:47] valhallasw`cloud: yeah, that’s kind of hard [19:39:56] rewrite it in python! [19:39:57] ;-D [19:41:18] valhallasw`cloud: :D You want to hit the gerrit API to fetch modified files every time? :) [19:41:24] you probably can do that tho [19:41:31] GPU does that [19:41:53] although maybe it parses the email [19:41:57] but I think it just asks gerrit [19:42:21] eh, GRB [19:42:42] biggest hack ever [19:42:45] heh yeah [19:43:02] yeah, it fetches the changeset from gerrit [19:43:10] but only for newchange messages [19:43:35] right [19:43:49] outside of that, our webservice infrastructure still needs an overhaul [19:44:31] yeah [19:44:38] I'm ok with having the 30s method as temp fix [19:46:29] valhallasw`cloud: yeah [19:46:36] valhallasw`cloud: I didn’t know qsub returned jobid [19:47:08] valhallasw`cloud: wanna +1? :) [20:07:43] YuviPanda|zzz: still not sure about the exit code [20:07:59] valhallasw`cloud: ah, hmm [20:08:55] valhallasw`cloud: 1 sounds better yeah [20:09:18] -1 is wrong in any case, should be 255 if anything [20:09:58] also why xrange O_o [20:10:23] oh well, it's not py3 compatible anyway [20:10:26] I guess [20:11:03] yeah [21:42:09] YuviPanda|zzz: it looks like wikiviewstats needs new creds, where do I look to see what xTools needs to do to get that? [22:50:27] T13|naps: how are you at writing bots? [22:50:35] or javascript? [22:50:38] or templates? [22:57:34] JavaScript and templates pretty good [22:57:47] Bots, still just learning [22:59:56] harej: ^