[03:03:33] !log ores deployed ores-wmflabs-deploy:022fb7d [03:03:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL, Master [03:19:05] !log tools.stashbot Restarted to pick up phabricator configuration change (https://github.com/bd808/tools-stashbot/issues/14) [03:19:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL, Master [03:21:12] !log tools.stashbot Testing for https://github.com/bd808/tools-stashbot/issues/14 -- T108720 [03:21:13] T108720: pick up ticket mentions from !log lines - https://phabricator.wikimedia.org/T108720 [03:21:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL, Master [04:50:37] !log tools.stashbot Deployed phab echo rate lmiiting (4b81ded) [04:50:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL, Master [07:29:29] 06Labs, 10Wikimedia-Extension-setup, 10wikitech.wikimedia.org, 07I18n, and 2 others: Install Translate extension on wikitech - https://phabricator.wikimedia.org/T100313#1310188 (10Nemo_bis) > What level of translation services do we want on wikitech? What the user asked, i.e. Translate. [08:10:48] so commons is lagged on one of the replica hosts for maintenance; if you need an up to date slave, use the other one for a day or so [08:17:33] 10Tool-Labs-tools-stewardbots: Automatically start the irc bots - https://phabricator.wikimedia.org/T144461#2631966 (10MarcoAurelio) @valhallasw So, it was something "external" to our tools? I mean, is the bigbrother file of tools.stewardbots correctly written? [10:09:44] I want to track them [10:37:08] 06Labs, 10Tool-Labs: jsub should respect .sge_request - https://phabricator.wikimedia.org/T145269#2632296 (10whym) I have tried switching to qsub, but it is harder than I imagined to stop using jsub entirely, particularly for cron jobs. For example, I would have to modify my scripts to emulate the "-once" beha... [12:41:55] PROBLEM - SSH on tools-webgrid-lighttpd-1210 is CRITICAL: Server answer [15:27:39] 06Labs, 10Tool-Labs: "exim paniclog on tools-cron-01.tools.eqiad.wmflabs has non-zero size" - https://phabricator.wikimedia.org/T145524#2633198 (10AlexMonk-WMF) [15:56:15] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dmw was created, changed by Dmw link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Dmw edit summary: Created page with "{{Tools Access Request |Justification=Work on BotBracket |Completed=false |User Name=Dmw }}" [16:14:21] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Track labs instances hanging - https://phabricator.wikimedia.org/T141673#2633432 (10bd808) google-api-proxy-02.google-api-proxy.eqiad.wmflabs noticed: 2016-09-13T16:00 ssh: no io wait: no io spike: no graphite: no nfs: no ``` [8765040.476194] INFO: task jbd... [16:14:57] !log google-api-proxy Rebooting google-api-proxy-02.google-api-proxy.eqiad.wmflabs for T141673 [16:14:58] T141673: Track labs instances hanging - https://phabricator.wikimedia.org/T141673 [16:15:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Google-api-proxy/SAL, Master [17:06:41] Just a heads up that the User:Dmw request is a legitimate one from a friend who will help fix some of my bots. [17:08:10] a930913: I'll get them an approval then :) [17:08:46] bd808: Thanks :) [17:10:55] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dmw was modified, changed by BryanDavis link https://wikitech.wikimedia.org/w/index.php?diff=832596 edit summary: [19:26:44] 06Labs, 10Tool-Labs: templatetiger is using 613G in Tools out of 8T - https://phabricator.wikimedia.org/T136192#2634408 (10Kolossos) @yuvipanda : Do you know why templatetiger doesn't work and why I can't bring it back by restarting webserver? I get an "502 Bad Gateway" . Thanks BTW: How useful do you think t... [19:37:19] PROBLEM - Puppet run on tools-docker-builder-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:42:25] (03CR) 10Lokal Profil: [C: 032] "> Hmmm, I do agree with your comment in 309842 ; however I meant this" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/309841 (owner: 10Jean-Frédéric) [20:44:10] (03Merged) 10jenkins-bot: Add unit tests for getCommonscatTemplates method [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/309841 (owner: 10Jean-Frédéric) [20:44:51] !log tools Disabling puppet across proxy hosts [20:44:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [20:45:09] (03CR) 10Lokal Profil: "recheck" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/309842 (owner: 10Jean-Frédéric) [20:45:41] 06Labs, 10Tool-Labs: jsub should respect .sge_request - https://phabricator.wikimedia.org/T145269#2634679 (10yuvipanda) 05Open>03declined We don't really want to add more gridengine specific things to jsub. As for -l release=trusty, that'll become the default soon! [20:47:39] !log deployment-prep Created SRV record _etcd._tcp.beta.wmflabs.org for etcd/confd [20:47:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL, Master [20:52:06] (03CR) 10Lokal Profil: "Follow up now tracked in https://phabricator.wikimedia.org/T145574" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/309841 (owner: 10Jean-Frédéric) [21:07:45] bd808, do we have standardized tool labs banners somewhere? [21:07:59] !log tools Reenabled puppet across proxy hosts [21:08:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [21:09:48] !log tools Bumped proxy nginx worker_connections limit T143637 [21:09:49] T143637: Tune nginx config parameters for tools / labs proxies - https://phabricator.wikimedia.org/T143637 [21:09:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master [21:10:32] 06Labs, 10Tool-Labs, 13Patch-For-Review, 07Wikimedia-Incident: Tune nginx config parameters for tools / labs proxies - https://phabricator.wikimedia.org/T143637#2634787 (10madhuvishy) 05Open>03Resolved This is done now [21:20:06] MaxSem: banners? Like the github clone ribbon or something? There is https://wikitech.wikimedia.org/wiki/Template:Tool but that may not be at all what you are looking for. [21:22:08] bd808, I meant HTML to slap in my tool [21:22:22] so like https://github.com/blog/273-github-ribbons yeah [21:22:51] hmmm... Not any that I know of. [21:23:43] 06Labs, 10Tool-Labs: templatetiger is using 613G in Tools out of 8T - https://phabricator.wikimedia.org/T136192#2634825 (10yuvipanda) The error.log file was owned by root, probaly as a result of it having grown super large in the past and someone truncating it. I've removed that now, and the webservice seems b... [21:26:07] MaxSem: there is https://tools-static.wmflabs.org/static/logos/powered-by-tool-labs.png [21:28:01] ah, thanks [21:54:50] Is it https://wikitech.wikimedia.org/w/index.php?title=Special:NovaServiceGroup&action=managemembers&projectname=tools&servicegroupname=tools.bracketbot where I add a user to a tool? [22:04:30] yes a930913 [22:05:09] Krenair: It seems to only show me as a maintainer, when I'm sure there's more. [22:05:43] nope that appears to be it [22:05:50] krenair@tools-bastion-03:~$ ldapsearch -x cn=tools.bracketbot | grep member: [22:05:51] member: uid=a930913,ou=people,dc=wikimedia,dc=org [22:05:51] krenair@tools-bastion-03:~$ [22:06:12] obviously roots, tools.admin members, etc. can also become that tool [22:06:22] Hmm, maybe it was just cluestuff that had others. [22:07:02] Yes, that must be it. [22:07:27] This is what happens when you leave your tools in disrepair for too long :( [22:07:39] Krenair: Danke. [22:09:14] a930913: you can check maintainership in a couple of places: https://toolsadmin.wikimedia.org/tools/id/bracketbot -- https://tools.wmflabs.org/?tool=bracketbot [23:28:30] 10Tool-Labs-tools-Pageviews, 06Community-Tech: Restrict Topviews to showing data only for individual days or months - https://phabricator.wikimedia.org/T142403#2635348 (10DannyH)