[03:51:26] 10Tool-Labs-tools-Erwin's-tools: erwin's tools doesn't show a skin - https://phabricator.wikimedia.org/T146569#2666071 (10Peachey88) [06:43:34] PROBLEM - Puppet run on tools-webgrid-lighttpd-1412 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [07:28:33] RECOVERY - Puppet run on tools-webgrid-lighttpd-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [07:45:52] PROBLEM - Puppet staleness on tools-k8s-master-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [08:37:38] 06Labs: Ignored file /etc/apt/apt.conf.d/20auto-upgrades.ucf-dist - https://phabricator.wikimedia.org/T110055#2666334 (10hashar) 05stalled>03Resolved a:03hashar New labs images have been build and they no more shipp the 20auto-upgrades.ucf-dist file in my experience. [09:11:29] 10Tool-Labs-tools-Erwin's-tools: For related changes make category changes switchable. - https://phabricator.wikimedia.org/T146594#2666390 (10Aklapper) @Akoopal: What are steps to reproduce this? URL welcome to the tool on Tool Labs that you added as project; but your description sounds more like being about Spe... [09:48:21] yuvipanda: did you see the puppet staleness alert on the k8s master? Grafana says 2 weeks [09:53:19] madhuvishy: yeah, that's me. [09:53:24] madhuvishy: I put up a patch to fix it [09:53:24] ah okay [09:53:26] cool [10:12:45] 06Labs, 10Labs-Infrastructure, 10Beta-Cluster-Infrastructure, 07Tracking: Log files on labs instance fill up disk (/var is only 2GB) (tracking) - https://phabricator.wikimedia.org/T71601#2666479 (10hashar) [10:22:10] 06Labs, 10Labs-Infrastructure, 07Puppet: Investigate usage of hiera_hash in our puppet repo - https://phabricator.wikimedia.org/T146621#2666501 (10Andrew) [10:32:22] 10Tool-Labs-tools-Erwin's-tools: For related changes make category changes switchable. - https://phabricator.wikimedia.org/T146594#2666551 (10Akoopal) A URL where you see this is: https://tools.wmflabs.org/erwin85/relatedchanges.php?lang=nl&family=wikipedia&category=Molen&d=10&submit=Submit&ignore= This is not... [10:41:08] 06Labs, 10Labs-Infrastructure, 05Continuous-Integration-Scaling, 13Patch-For-Review: Bump quota of Nodepool instances (contintcloud tenant) - https://phabricator.wikimedia.org/T133911#2666606 (10hashar) [10:56:46] 06Labs, 10Beta-Cluster-Infrastructure, 06Operations, 07Puppet: Implement role based hiera lookups for labs - https://phabricator.wikimedia.org/T120165#2666657 (10hashar) [13:20:59] hello wikipedia people.. I was wondering, how can i protect a page i wrote? i mean what stops someone from coming and just deleting the entire page? is there a way to lock my content? [13:23:34] anyone? [13:24:02] Jangan322: asking in the #wikipedia-en channel might be more useful [13:24:39] yuvipanda: tyvm [14:18:10] 10Labs-project-Wikipedia-Requests, 10WikiProject-X, 10Wikimedia-Site-requests: Ada Lovelave Day Edit-a-thon - https://phabricator.wikimedia.org/T146654#2667301 (10CaitlinLombardo) [14:21:21] 10Labs-project-Wikipedia-Requests, 10WikiProject-X, 10Wikimedia-Site-requests: Ada Lovelave Day Edit-a-thon - https://phabricator.wikimedia.org/T146654#2667322 (10Urbanecm) a:05CaitlinLombardo>03None [14:48:55] 06Labs, 10Beta-Cluster-Infrastructure, 06Operations, 07Puppet: Implement role based hiera lookups for labs - https://phabricator.wikimedia.org/T120165#2667411 (10yuvipanda) I'm no longer convinced we have to do this. https://phabricator.wikimedia.org/T91990 should cover most of the things we want from this. [16:33:27] 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#2667746 (10Milimetric) @Quiddity, we're very close to allowing this kind of query in Hadoop, which you've worked with before, right? Grab me and I'll show you what we're working... [19:08:24] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Maxeeder was created, changed by Maxeeder link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Maxeeder edit summary: Created page with "{{Tools Access Request |Justification=i want to create article in azb and persian language |Completed=false |User Name=Maxeeder }}" [21:02:31] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Gabrieloli was created, changed by Gabrieloli link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Gabrieloli edit summary: Created page with "{{Tools Access Request |Justification=Develop an Self Repairing AI mainly for the wikitech base, later for others wikis. |Completed=false |User Name=Gabrieloli }}" [21:21:48] 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#2668912 (10yuvipanda) Hadoop isn't public though, so not the same thing :) You can do similar things via tool labs now too. [21:33:00] 10Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#1195035 (10matmarex) You can fairly easily turn a query for a single wiki into a query over all wikis. It's a bit of boring work though. The pattern goes like this: ```lang=sql s... [23:07:07] 10Labs-project-Wikistats: allthetropes is not updating on wikistats - https://phabricator.wikimedia.org/T146712#2669198 (10labster) [23:12:13] 06Labs, 10Labs-Infrastructure, 13Patch-For-Review: Track labs instances hanging - https://phabricator.wikimedia.org/T141673#2669244 (10AlexMonk-WMF) fqdn: deployment-aqs01.deployment-prep.eqiad.wmflabs virt host: labvirt1005.eqiad.wmnet noticed: 2016-09-26T23:44, shinken check for puppet staleness broken sin... [23:13:57] !log deployment-prep Rebooting deployment-aqs01 for T141673 [23:13:58] T141673: Track labs instances hanging - https://phabricator.wikimedia.org/T141673 [23:14:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL, Master [23:18:32] very interesting... [23:18:48] After I rebooted it, /var/log/syslog has lost all the crap it had between roughly the time of it getting stuck, and the restart [23:19:16] Sep 21 17:25:01 deployment-aqs01 CRON[27843]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) [23:19:16] Sep 26 23:17:31 deployment-aqs01 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="593" x-info="http://www.rsyslog.com"] start [23:19:25] hm, I should put this on the task