[00:10:35] <grrrit-wm>	 (03CR) 10Legoktm: "Slightly concerned about how many jobs we're going to end up with like this...can we at least merge the 'repo-api' jobs with 'repo'?" [integration/config] - 10https://gerrit.wikimedia.org/r/202289 (https://phabricator.wikimedia.org/T95230) (owner: 10Legoktm)
[00:28:22] <grrrit-wm>	 (03CR) 10Mattflaschen: "Erik said he would review, merge, and deploy this, unless there was a reason to hold off." [integration/config] - 10https://gerrit.wikimedia.org/r/194749 (https://phabricator.wikimedia.org/T91220) (owner: 10Mattflaschen)
[00:28:38] <Krenair>	 bd808, you need to ask outside the engineering list too
[00:28:54] <Krenair>	 it's wmf only, IIRC
[01:15:51] <grrrit-wm>	 (03PS1) 10Krinkle: Publish code coverage for VisualEditor [integration/config] - 10https://gerrit.wikimedia.org/r/202300 
[01:16:57] <Krinkle>	 legoktm: So I originally suspected there'd be many jslint jobs for repos without jshintrc config
[01:17:08] <Krinkle>	 while that it true, I did not expect that even mroe of them didn't even have any *.js files
[01:19:06] <Krinkle>	 legoktm: 1835 jobs o/
[01:21:46] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] "Deployed https://integration.wikimedia.org/ci/job/visualeditor-publish/" [integration/config] - 10https://gerrit.wikimedia.org/r/202300 (owner: 10Krinkle)
[01:25:19] <grrrit-wm>	 (03Merged) 10jenkins-bot: Publish code coverage for VisualEditor [integration/config] - 10https://gerrit.wikimedia.org/r/202300 (owner: 10Krinkle)
[01:25:47] <Krinkle>	 !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/202300 
[01:25:50] <qa-morebots>	 Logged the message, Master
[02:02:33] <grrrit-wm>	 (03CR) 10Krinkle: "https://integration.wikimedia.org/cover/visualeditor/" [integration/config] - 10https://gerrit.wikimedia.org/r/202300 (owner: 10Krinkle)
[02:29:48] <bd808>	 Krenair: I thought everyone with shell was on engineering-l? I can check with aude and hoo for sure. Not sure who else would have access to logs on fluorine.
[02:30:01] <Krenair>	 no engineering is wmf-only
[02:30:11] <bd808>	 huh. ok
[02:30:16] <Krenair>	 https://lists.wikimedia.org/mailman/listinfo/engineering
[02:30:19] <Krenair>	 "WMF Engineering staff and contractors (only!) list."
[02:30:26] <Krenair>	 wikitech is the public one
[02:30:49] <Krenair>	 I only happen to be there because I currently do some contract work
[02:31:13] <bd808>	 wikitech is any random folks. The change I wrote about is WMF prod cluster only
[02:31:42] <bd808>	 but that does include folks who aren't WMF staff
[02:33:09] <Krenair>	 Ryan Lane and Domas Mituzas have root so could probably look at the archives regardless of not being subscribed, I don't know if they care about these logs
[02:34:35] <Krenair>	 I'm not aware of any other volunteer roots/deployers...
[02:35:24] <Krenair>	 avar does not have any ssh keys in currently, but is in the restricted group.
[02:35:37] <Krenair>	 diederik has shell keys and is in the restricted group
[02:36:33] <Krenair>	 I should probably add restricted group people to https://meta.wikimedia.org/wiki/System_administrators and list affiliations
[04:42:11] <wikibugs>	 10Continuous-Integration, 10Tool-Labs: labs-toollabs-debian-glue fails apparently with a timeout - https://phabricator.wikimedia.org/T91247#1184622 (10scfc) p:5Triage>3Low
[05:02:40] <wmf-insecte>	 Project browsertests-VisualEditor-production-linux-firefox-sauce build #56: FAILURE in 2 min 40 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-production-linux-firefox-sauce/56/
[05:23:57] <shinken-wm>	 PROBLEM - Content Translation Server on deployment-cxserver03 is CRITICAL: Connection refused  
[05:26:55] <legoktm>	 Krinkle|detached: probably because all repos have .json i18n files, and they were using mwext-*-jslint for its jsonlint only
[05:29:16] <legoktm>	 bd808: emailing both ops-l and engineering should cover everyone with shell
[05:32:56] <greg-g>	 we add aude and hoo to ops for a reason
[05:38:57] <shinken-wm>	 RECOVERY - Content Translation Server on deployment-cxserver03 is OK: HTTP OK: HTTP/1.1 200 OK - 1103 bytes in 0.034 second response time  
[08:13:18] <grrrit-wm>	 (03CR) 10Hashar: "Looks good https://doc.wikimedia.org/mw-tools-scap/ . Thanks!" [tools/scap] - 10https://gerrit.wikimedia.org/r/202240 (owner: 10Hashar)
[08:52:50] <grrrit-wm>	 (03PS8) 10Hashar: Pass MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD to GettingStarted browser test [integration/config] - 10https://gerrit.wikimedia.org/r/194749 (https://phabricator.wikimedia.org/T91220) (owner: 10Mattflaschen)
[08:54:17] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] "I forgot to +2 the patch sorry. I have rebased it and refreshed all jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/194749 (https://phabricator.wikimedia.org/T91220) (owner: 10Mattflaschen)
[08:58:27] <grrrit-wm>	 (03Merged) 10jenkins-bot: Pass MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD to GettingStarted browser test [integration/config] - 10https://gerrit.wikimedia.org/r/194749 (https://phabricator.wikimedia.org/T91220) (owner: 10Mattflaschen)
[09:01:41] <wikibugs>	 10Continuous-Integration, 10Browser-Tests, 6Collaboration-Team, 10MediaWiki-extensions-GettingStarted, and 2 others: Pass MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD in on Jenkins so GettingStarted browser tests pass - https://phabricator.wikimedia.org/T91220#1184983 (10hashar) I forgot to +2 the patch https://gerri...
[09:37:47] <hashar>	 !log integration project has been switched to a new labs DNS resolver ( https://lists.wikimedia.org/pipermail/labs-l/2015-April/003585.html ) . It is missing the dnsmasq hack to resolve beta cluster URls to the instance IP instead of the public IP.  Causes a wild range of jobs to fail.
[09:37:51] <qa-morebots>	 Logged the message, Master
[09:39:11] <hashar>	 !sal
[09:39:11] <wm-bot>	 https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:44:23] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185067 (10hashar) 3NEW
[10:01:23] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185081 (10hashar) That is caused by puppet change https://gerrit.wikimedia.org/r/#/c/202278/ which renames the variable from `use_dnsmasq` to `use_dnsmas...
[10:15:01] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1002 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[10:17:04] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1404 is CRITICAL: CRITICAL: 57.14% of data above the critical threshold [0.0]  
[10:20:56] <shinken-wm>	 PROBLEM - Puppet failure on integration-labsvagrant is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0]  
[10:23:31] <shinken-wm>	 PROBLEM - Puppet failure on integration-zuul-packaged is CRITICAL: CRITICAL: 57.14% of data above the critical threshold [0.0]  
[10:25:08] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185128 (10hashar) So the problem is VERY nasty. All instances have been switched to the new DNS resolver which causes puppet client to fail.  The puppetm...
[10:25:14] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1405 is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0]  
[10:25:53] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1004 is CRITICAL: CRITICAL: 42.86% of data above the critical threshold [0.0]  
[10:26:27] <shinken-wm>	 PROBLEM - Puppet failure on integration-publisher is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[10:26:53] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[10:27:08] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1005 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[10:27:31] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[10:27:59] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1003 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[10:29:07] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave1403 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[10:29:15] <hashar>	 stupid puppet certs
[10:29:45] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[10:31:48] <shinken-wm>	 PROBLEM - Puppet failure on integration-dev is CRITICAL: CRITICAL: 57.14% of data above the critical threshold [0.0]  
[10:35:11] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185151 (10scfc) I think that certificate signing nowadays is handled automatically, so you only need two Puppet runs patience?  Also, `/etc/resolv.conf`...
[10:40:04] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[10:49:24] <hashar>	 !log manually hacking integration-puppetmaster /etc/puppet/puppet.conf config file which is missing the [master] section
[10:49:27] <qa-morebots>	 Logged the message, Master
[10:52:10] <hashar>	 !log made puppetmaster certname = integration-puppetmaster.eqiad.wmflabs instead of the ec2 id :(
[10:52:13] <qa-morebots>	 Logged the message, Master
[11:08:20] <hashar>	 !log dishing out puppet SSL configuration on all integratio nodes. Can't figure out so lets restart from scratch
[11:08:23] <qa-morebots>	 Logged the message, Master
[11:28:05] <hashar>	 !log restored /etc/puppet/fileserver.conf
[11:28:10] <qa-morebots>	 Logged the message, Master
[11:31:56] <hashar>	 !log integration-puppetmaster is back and operational with local puppet client working properly.
[11:31:59] <qa-morebots>	 Logged the message, Master
[11:49:32] <wmf-insecte>	 Yippee, build fixed!
[11:49:33] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-10-sauce build #210: FIXED in 1 min 31 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-10-sauce/210/
[12:00:16] <wmf-insecte>	 Yippee, build fixed!
[12:00:16] <wmf-insecte>	 Project browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #71: FIXED in 3 min 15 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/71/
[12:00:41] <hashar>	 !log running puppet on all integration machines and resigning puppet client certs
[12:00:44] <qa-morebots>	 Logged the message, Master
[12:01:46] <zeljkof>	 aharoni: coming to the meeting?
[12:03:12] <hashar>	 !log Browser tests against beta cluster were all failing due to an improper DNS resolver being applied on CI labs instances {{bug|T95273}}. Should be fixed now.
[12:03:15] <qa-morebots>	 Logged the message, Master
[12:03:25] <hashar>	 zeljkof-meeting: I have wasted my whole morning fixing browser tests :D
[12:03:39] <zeljkof-meeting>	 hashar: that is not time wasted ;)
[12:03:39] <hashar>	 the browser tests hitting beta.wmflabs.org were all failing. Should be fixed now
[12:03:43] <hashar>	 it is
[12:03:52] <hashar>	 someone pushed a bad change in puppet :(
[12:05:02] <wikibugs>	 10Continuous-Integration, 10Browser-Tests, 6Collaboration-Team, 10MediaWiki-extensions-GettingStarted, and 2 others: Pass MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD in on Jenkins so GettingStarted browser tests pass - https://phabricator.wikimedia.org/T91220#1185271 (10hashar) The DNS failure (T95273) has been fixe...
[12:05:53] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave1004 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:06:05] <shinken-wm>	 RECOVERY - Puppet failure on integration-labsvagrant is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:07:06] <grrrit-wm>	 (03CR) 10JanZerebecki: [C: 031] "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/202289 (https://phabricator.wikimedia.org/T95230) (owner: 10Legoktm)
[12:07:23] <hashar>	 !log Puppet being fixed, it is finishing the installation of integration-slave-trusty-*** hosts
[12:07:26] <qa-morebots>	 Logged the message, Master
[12:08:05] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave1003 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:10:17] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave1405 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:10:35] <wmf-insecte>	 Yippee, build fixed!
[12:10:36] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #214: FIXED in 1 min 10 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/214/
[12:10:38] <wmf-insecte>	 Yippee, build fixed!
[12:10:39] <wmf-insecte>	 Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #506: FIXED in 1 min 9 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/506/
[12:10:43] <shinken-wm>	 RECOVERY - Puppet failure on integration-zuul-server is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:11:19] <wmf-insecte>	 Yippee, build fixed!
[12:11:20] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-os_x_10.9-chrome-sauce build #13: FIXED in 1 min 13 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-os_x_10.9-chrome-sauce/13/
[12:11:49] <hashar>	 !log retriggering a bunch of browser tests hitting beta.wmflabs.org
[12:11:52] <shinken-wm>	 RECOVERY - Puppet failure on integration-dev is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:11:52] <qa-morebots>	 Logged the message, Master
[12:11:53] <wmf-insecte>	 Yippee, build fixed!
[12:11:54] <wmf-insecte>	 Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #501: FIXED in 1 min 37 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/501/
[12:13:12] <wmf-insecte>	 Yippee, build fixed!
[12:13:13] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #277: FIXED in 1 min 1 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/277/
[12:13:32] <wmf-insecte>	 Yippee, build fixed!
[12:13:33] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #278: FIXED in 1 min 24 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/278/
[12:14:06] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave1403 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:17:08] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave1404 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:18:32] <shinken-wm>	 RECOVERY - Puppet failure on integration-zuul-packaged is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:18:59] <wmf-insecte>	 Yippee, build fixed!
[12:18:59] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-firefox-sauce build #13: FIXED in 2 min 15 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-firefox-sauce/13/
[12:19:27] <wmf-insecte>	 Yippee, build fixed!
[12:19:27] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-chrome-sauce build #13: FIXED in 2 min 39 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-chrome-sauce/13/
[12:19:43] <wmf-insecte>	 Yippee, build fixed!
[12:19:44] <wmf-insecte>	 Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #215: FIXED in 2 min 54 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/215/
[12:20:09] <wmf-insecte>	 Yippee, build fixed!
[12:20:10] <wmf-insecte>	 Project browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-os_x_10.10-iphone-sauce build #30: FIXED in 3 min 13 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-os_x_10.10-iphone-sauce/30/
[12:20:38] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185320 (10hashar) 5Open>3Resolved a:3hashar So I have fixed it eventually and that was nasty.  On integration-puppetmaster I have: * manually rewro...
[12:20:48] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185323 (10hashar) p:5Triage>3Unbreak!
[12:22:13] <wmf-insecte>	 Yippee, build fixed!
[12:22:14] <wmf-insecte>	 Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #464: FIXED in 12 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/464/
[12:22:53] <wikibugs>	 10Continuous-Integration, 10RESTBase, 7Documentation: Publish RESTBase documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T87702#1185336 (10Qgil) Resolved? Where is the RESTBase documentation at https://doc.wikimedia.org/ ?
[12:23:38] <wmf-insecte>	 Yippee, build fixed!
[12:23:38] <wmf-insecte>	 Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #498: FIXED in 1 min 52 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/498/
[12:26:54] <hashar>	 !log deleting integration-slave-trusty-1005 has been provisioned with role::ci::website instead of role::ci::slave::labs
[12:27:00] <qa-morebots>	 Logged the message, Master
[12:27:43] <shinken-wm>	 PROBLEM - Host integration-slave-trusty-1005 is DOWN: CRITICAL - Host Unreachable (10.68.18.2)  
[12:30:59] <wmf-insecte>	 Yippee, build fixed!
[12:31:00] <wmf-insecte>	 Project browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #78: FIXED in 14 min: https://integration.wikimedia.org/ci/job/browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/78/
[12:32:29] <wikibugs>	 10Continuous-Integration, 10RESTBase, 7Documentation: Publish RESTBase documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T87702#1185345 (10hashar) 5Resolved>3Open
[12:33:41] <wmf-insecte>	 Yippee, build fixed!
[12:33:42] <wmf-insecte>	 Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #544: FIXED in 3 min 2 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/544/
[12:40:49] <wmf-insecte>	 Yippee, build fixed!
[12:40:50] <wmf-insecte>	 Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #495: FIXED in 1 min 2 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/495/
[12:42:10] <wikibugs>	 10Continuous-Integration: Downgrade intergration-puppetmaster back to Ubuntu Precise (re-create instance) - https://phabricator.wikimedia.org/T94927#1185377 (10hashar) 5Open>3Resolved It was surely passing on all available instances when I closed the bug. puppet is passing on integration-dev.eqiad.wmflabs ri...
[12:42:11] <wikibugs>	 10Continuous-Integration: Re-create ci slaves (April 2015) - https://phabricator.wikimedia.org/T94916#1185379 (10hashar)
[12:44:30] <wikibugs>	 10Continuous-Integration: Re-create ci slaves (April 2015) - https://phabricator.wikimedia.org/T94916#1185390 (10hashar) The certificates errors reported above are due to a faulty puppet configuration change which switched the integration labs instance to a new DNS resolver.  That one introduces in the fully qua...
[12:45:18] <hashar>	 !log recreating integration-slave-trusty-1005
[12:45:21] <qa-morebots>	 Logged the message, Master
[12:46:44] <wmf-insecte>	 Yippee, build fixed!
[12:46:45] <wmf-insecte>	 Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #1747: FIXED in 43 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/1747/
[12:49:09] <wikibugs>	 10Continuous-Integration: Create CI slaves using Debian Jessie (tracking) - https://phabricator.wikimedia.org/T94836#1185399 (10hashar)
[12:50:10] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185402 (10Andrew) I don't think this was actually caused by that patch; I suspect that it's another example of https://phabricator.wikimedia.org/T95240 w...
[12:55:13] <hashar>	 going to skip today CI triage I guess
[12:55:17] <hashar>	 forgot to announce it properly
[12:56:55] <jzerebecki>	 hashar: ok. it would be nice to get https://gerrit.wikimedia.org/r/#/c/180418/ merged and deployed.
[12:57:06] <hashar>	 jzerebecki: hey :) 
[12:57:13] <jzerebecki>	 hey
[12:57:14] <hashar>	 jzerebecki: I guess I will start the meeting so :)
[12:57:44] <hashar>	 !log running apt-get upgrade on integration-slave-trusty* hosts
[12:57:47] <qa-morebots>	 Logged the message, Master
[12:58:20] <hashar>	 going to #wikimedia-office for the CI weekly meeting
[12:58:41] <jzerebecki>	 Krinkle: ^^ perhaps you can look at it again as your comment was adressed
[12:58:50] <Krinkle>	 hashar: o/
[13:01:09] <shinken-wm>	 PROBLEM - Puppet failure on integration-slave-trusty-1005 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[13:02:36] <^d>	 Clearly nobody runs unit tests on their local machine anymore.
[13:03:03] * ^d sighs at everyone
[13:03:44] <wikibugs>	 10Continuous-Integration: Provide a way to have a demo directory alongside the documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T62143#1185430 (10Krinkle) a:3Krinkle
[13:04:23] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185433 (10thcipriani) @hashar we had the same problem yesterday in the staging project. The hiera hack should have been a no-op on every environment unle...
[13:04:44] <wikibugs>	 10Continuous-Integration: Provide a way to have a demo directory alongside the documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T62143#1185435 (10hashar) Timo worked on generic templates to publish /cover/ directories. It can be reused now to publish the /demos/ directories.
[13:31:51] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave-trusty-1001 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:32:29] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave-trusty-1004 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:33:24] <aharoni>	 hashar: Bonjour. Any idea why does jshint fail here? - https://gerrit.wikimedia.org/r/#/c/202385/
[13:33:34] <aharoni>	 Looks like a Jenkins problem.
[13:35:24] <wikibugs>	 10Continuous-Integration: Garbage collect orphan jobs workspaces on slaves - https://phabricator.wikimedia.org/T94408#1185516 (10Krinkle) p:5Triage>3Low
[13:36:07] <wikibugs>	 10Continuous-Integration, 10MediaWiki-extensions-ContentTranslation: JSHint fails with ContentTranslation - https://phabricator.wikimedia.org/T95283#1185523 (10Amire80) 3NEW
[13:37:36] <Krinkle>	 aharoni: no
[13:37:37] <Krinkle>	 aharoni: 13:23:14 modules/dashboard/ext.cx.magnuslink.js: line 15, col 14, 'getMagnusToolLink' is defined but never used. 
[13:37:52] <aharoni>	 aha! thanks.
[13:37:53] <aharoni>	 missed that.
[13:38:21] <aharoni>	 however,
[13:38:59] <aharoni>	 Krinkle: the reason I thought it's a Jenkins problem is the Java errors at the top
[13:39:03] <aharoni>	 doesn't look right
[13:39:07] <aharoni>	 java.lang.IndexOutOfBoundsException etc.
[13:39:50] <wikibugs>	 10Continuous-Integration, 10MediaWiki-extensions-ContentTranslation: JSHint fails with ContentTranslation - https://phabricator.wikimedia.org/T95283#1185534 (10Amire80) Krinkle aharoni: no Krinkle aharoni: 13:23:14 modules/dashboard/ext.cx.magnuslink.js: line 15, col 14, 'getMagnusToolLink' is defined but neve...
[13:39:56] <Krinkle>	 aharoni: not important
[13:40:04] <Krinkle>	 aharoni: Happens on most jobs, is just a version conflict. Does nothign.
[13:40:05] <aharoni>	 OK.
[13:40:22] <Krinkle>	 Requires mega work load to upgrade zuul. Nothig we can do about it now
[13:40:30] <Krinkle>	 aharoni: I filed a bug. Thanks
[13:40:32] <wikibugs>	 10Continuous-Integration, 10MediaWiki-extensions-ContentTranslation: JSHint fails with ContentTranslation - https://phabricator.wikimedia.org/T95283#1185536 (10Amire80) 5Open>3Invalid a:3Amire80 It actually is a jshint problem.
[13:43:58] <wmf-insecte>	 Yippee, build fixed!
[13:43:58] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #562: FIXED in 1 hr 13 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/562/
[13:44:06] <wikibugs>	 10Continuous-Integration, 5Patch-For-Review: Have jenkins jobs logrotate their build history - https://phabricator.wikimedia.org/T91396#1185543 (10hashar) p:5Normal>3Low The bulk of the work has been done though it is not fully completed yet. Keeping this task open as a reminder to complete the work.
[13:59:15] <wikibugs>	 10Continuous-Integration, 10MediaWiki-extensions-ContentTranslation: JSHint fails with ContentTranslation - https://phabricator.wikimedia.org/T95283#1185585 (10Krinkle) >>! In T95283#1185534, @Amire80 wrote: > <aharoni>: java.lang.IndexOutOfBoundsException etc.  Filed as T93321. It's a harmless error caused by...
[14:01:43] <wikibugs>	 10Continuous-Integration: Upgrade Zuul server to latest upstream - https://phabricator.wikimedia.org/T94409#1185600 (10Krinkle) p:5Triage>3Normal
[14:05:19] <wikibugs>	 10Continuous-Integration, 6Security: Jenkins run 'rake validate' on any ops/puppet change - https://phabricator.wikimedia.org/T44929#1185647 (10Krinkle) p:5Triage>3Unbreak!
[14:06:53] <wmf-insecte>	 Yippee, build fixed!
[14:06:53] <wmf-insecte>	 Project browsertests-Wikidata-PerformanceTests-linux-firefox-sauce build #209: FIXED in 52 sec: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-PerformanceTests-linux-firefox-sauce/209/
[14:07:37] <wikibugs>	 10Continuous-Integration, 6Release-Engineering, 5Patch-For-Review: Zuul-cloner forgets to clear workspace - https://phabricator.wikimedia.org/T76304#1185660 (10Krinkle) p:5High>3Lowest
[14:07:57] <wikibugs>	 10Continuous-Integration, 6Release-Engineering, 5Patch-For-Review: Zuul-cloner forgets to clear workspace - https://phabricator.wikimedia.org/T76304#795567 (10Krinkle) p:5Lowest>3High
[14:22:02] <wikibugs>	 10Continuous-Integration, 6Labs: integration labs project DNS resolver improperly switched to openstack-designate - https://phabricator.wikimedia.org/T95273#1185732 (10hashar) I have filled {T95288}  Poked @andrew about the issue on https://lists.wikimedia.org/pipermail/labs-l/2015-April/003592.html
[14:22:36] <jzerebecki>	 hashar, Krinkle: will one of you take a stab at https://gerrit.wikimedia.org/r/#/c/180418/ ?
[14:23:30] <hashar>	 jzerebecki: gotta write the meeting notes then prepare for my weekly team meeting and quaterly review :/
[14:24:19] <hashar>	 jzerebecki: the WikibaseJavaScriptApi thing, I have no idea how to fix it beside having the dependencie data-values shipped in either mediawiki/vendor or as a mediawiki extension (the later sounds lame)
[14:25:44] <wikibugs>	 10Continuous-Integration, 7Jenkins: Jenkins: "Warning: Some projects have builds whose timestamps are inconsistent." - https://phabricator.wikimedia.org/T94963#1185765 (10Krinkle) p:5Triage>3Lowest
[14:26:02] <wikibugs>	 10Continuous-Integration: Find a way to deduplicate post-merge builds like mediawiki-core-doxygen-publish - https://phabricator.wikimedia.org/T94715#1185771 (10Krinkle) p:5Triage>3Normal
[14:26:02] <jzerebecki>	 hashar: so you are opposed to using composer to pull it in?
[14:27:38] <legoktm>	 jzerebecki: if you're going to create a special one-off job for it to use composer, I think it would be fine if it was in wikidata.yaml
[14:28:58] <jzerebecki>	 legoktm: that is what the patch does except that it is in jjb/mediawiki-extensions.yaml
[14:29:38] <wikibugs>	 10Continuous-Integration: Remove pywiki* and integration* from 'mediawiki' gate queue. - https://phabricator.wikimedia.org/T93304#1185806 (10Krinkle)
[14:29:39] <wikibugs>	 10Continuous-Integration: Re-evaluate use of "Dependent Pipeline" in Zuul for gate-and-submit in the short term - https://phabricator.wikimedia.org/T94322#1185807 (10Krinkle)
[14:29:43] <wikibugs>	 10Continuous-Integration: Remove pywiki* and integration* from 'mediawiki' gate queue. - https://phabricator.wikimedia.org/T93304#1185808 (10Krinkle) p:5Triage>3Low
[14:30:29] <wikibugs>	 10Continuous-Integration, 7Jenkins: Jobs stuck indefinitely after they are finished - https://phabricator.wikimedia.org/T91430#1185815 (10Krinkle) p:5Triage>3Lowest
[14:35:00] <wikibugs>	 10Continuous-Integration: Fix "Entry point ('console_scripts', 'tox') not found" on new slaves running Ubuntu Precise - https://phabricator.wikimedia.org/T91526#1089265 (10Krinkle)
[14:35:01] <wikibugs>	 10Continuous-Integration: Re-create ci slaves (March 2015) - https://phabricator.wikimedia.org/T91524#1185836 (10Krinkle)
[14:35:32] <wikibugs>	 10Continuous-Integration, 6ArchCom, 10MediaWiki-RfCs: RFC: Extensions continuous integration - https://phabricator.wikimedia.org/T1350#1185838 (10Krinkle) p:5Triage>3Normal
[14:35:52] <wikibugs>	 10Continuous-Integration, 6Labs: Continuous integration should not depend on labs NFS - https://phabricator.wikimedia.org/T90610#1185847 (10Krinkle) p:5Triage>3Low
[14:40:02] <wikibugs>	 10Continuous-Integration: Re-create ci slaves (April 2015) - https://phabricator.wikimedia.org/T94916#1185879 (10hashar) I have deleted and recreated `integration-slave-trusty-1005` because it has been provisioned with role::ci::website instead of role::ci::slave::labs
[14:42:42] <wikibugs>	 10Continuous-Integration: Replace project-specific "{name}-thing" jobs with generic "thing" ones - https://phabricator.wikimedia.org/T91997#1185891 (10Krinkle)
[14:42:42] <wikibugs>	 10Continuous-Integration: Migrate jsduck-publish to use a single job and variable destination path - https://phabricator.wikimedia.org/T91854#1185889 (10Krinkle) 5Open>3Resolved Obsolete with npm-run-doc and doc-publish macros.
[14:48:07] <wmf-insecte>	 Yippee, build fixed!
[14:48:08] <wmf-insecte>	 Project browsertests-Wikidata-SmokeTests-linux-firefox-sauce build #210: FIXED in 31 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce/210/
[14:48:36] <wikibugs>	 10Continuous-Integration, 10RESTBase, 7Documentation: Publish RESTBase documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T87702#1185932 (10Krinkle) Following recent developments, the publish job is now a lot simpler.  It basically needs only these macros:  ``` - npm-install - npm-run-doc...
[14:51:16] <shinken-wm>	 RECOVERY - Puppet failure on integration-slave-trusty-1005 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:02:20] <jzerebecki>	 legoktm: so will you merge https://gerrit.wikimedia.org/r/#/c/180418/ ?
[15:02:40] <legoktm>	 jzerebecki: yeah, I'll look at in a few hours
[15:02:44] <jzerebecki>	 thx
[15:18:58] <wikibugs>	 10Continuous-Integration, 10RESTBase, 7Documentation: Publish RESTBase documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T87702#1186073 (10hashar) Excellent! That is a great example of tightly coupling the code and the documentation, I love it :)  Much better than a static doc on doc.wik...
[15:39:31] <grrrit-wm>	 (03PS1) 10Krinkle: doc: Add redirect for old mediawiki-core/master/php/html/ [integration/docroot] - 10https://gerrit.wikimedia.org/r/202413 (https://phabricator.wikimedia.org/T73060) 
[15:42:42] <grrrit-wm>	 (03PS1) 10Krinkle: mwcore-docgen: Remove dangerous TARGET_BASEDIR default value [integration/jenkins] - 10https://gerrit.wikimedia.org/r/202414 
[15:43:08] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] mwcore-docgen: Remove dangerous TARGET_BASEDIR default value [integration/jenkins] - 10https://gerrit.wikimedia.org/r/202414 (owner: 10Krinkle)
[15:46:29] <grrrit-wm>	 (03PS1) 10Krinkle: doc: Add link to RESTBase documentation [integration/docroot] - 10https://gerrit.wikimedia.org/r/202418 (https://phabricator.wikimedia.org/T87702) 
[16:00:04] <grrrit-wm>	 (03Merged) 10jenkins-bot: mwcore-docgen: Remove dangerous TARGET_BASEDIR default value [integration/jenkins] - 10https://gerrit.wikimedia.org/r/202414 (owner: 10Krinkle)
[16:00:27] <wikibugs>	 6Release-Engineering, 6Phabricator, 10Phabricator-Sprint-Extension, 5Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#1186233 (10Aklapper)
[16:00:54] <greg-g>	 meeting time :)
[16:01:23] <greg-g>	 twentyafterfour: ^d marx<tab>
[16:01:46] <^d>	 Yes joining
[16:09:16] <zeljkof-meeting>	 twentyafterfour: beret vs. mullet https://phabricator.wikimedia.org/T94288
[16:22:47] <wikibugs>	 7Blocked-on-RelEng, 10Browser-Tests, 10Continuous-Integration, 6Collaboration-Team, and 2 others: Pass MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD in on Jenkins so GettingStarted browser tests pass - https://phabricator.wikimedia.org/T91220#1186346 (10greg)
[16:22:50] <wikibugs>	 10Staging, 6Labs, 5Patch-For-Review: Labs puppet ENC scrambles booleans - https://phabricator.wikimedia.org/T95240#1186348 (10mmodell) fixed?
[16:23:34] <wikibugs>	 10Staging, 6Labs, 5Patch-For-Review: Labs puppet ENC scrambles booleans - https://phabricator.wikimedia.org/T95240#1186351 (10yuvipanda) Nope but dnsmasq errors fixed for now. I'll take a look at the underlying issue today.
[16:24:44] <wikibugs>	 6Release-Engineering: Delete #Blocked-on-Continuous-Integration tag in favor of using #Blocked-On-Releng - https://phabricator.wikimedia.org/T95310#1186357 (10hashar) 3NEW
[16:26:40] <Krinkle>	 !log git-deploy: Deploying integration/slave-scripts 4c6f541
[16:26:43] <qa-morebots>	 Logged the message, Master
[16:28:35] <twentyafterfour>	 ^d: what's the deal with 1.25 branching?  does that have any impact on what I'm doing with production branches?
[16:29:03] <^d>	 Ideally I've done by branching before you do so you can create 1.26wmf1 from 1.26alpha
[16:29:09] <^d>	 s/by/my/
[16:31:13] <grrrit-wm>	 (03PS1) 10Krinkle: mediawiki-core-doxygen-publish: Drop /html/ subdir from url [integration/config] - 10https://gerrit.wikimedia.org/r/202429 (https://phabricator.wikimedia.org/T73060) 
[16:37:37] <wikibugs>	 10Browser-Tests, 6Collaboration-Team, 10Echo, 10Flow, 6Mobile-Web: Move user agent assignment to Ruby gem - https://phabricator.wikimedia.org/T73531#1186439 (10dduvall) We should try to name the step to be clearer to its intention. What about "my browser doesn't support JavaScript", or "I have JavaScript...
[16:38:13] <shinken-wm>	 PROBLEM - Citoid on deployment-sca01 is CRITICAL: Connection refused  
[16:39:45] <shinken-wm>	 PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds  
[16:49:38] <shinken-wm>	 RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 27930 bytes in 0.809 second response time  
[17:00:23] <wikibugs>	 10Browser-Tests, 10Gather, 3Gather Sprint Enwiki, 7Jenkins: Browser tests failing regularly due to styles not alway loading - https://phabricator.wikimedia.org/T94891#1186563 (10Jdlrobson) @dduval thanks for the explanation!  So has the Gather job been switched to Chrome version 41 now? Can I call this clo...
[17:06:11] <wikibugs>	 3Continuous-Integration-Isolation, 6operations: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1186590 (10RobH)
[17:27:05] <grrrit-wm>	 (03PS1) 10Krinkle: Delete old jshint jobs from repos without any *.js files (A-B) [integration/config] - 10https://gerrit.wikimedia.org/r/202445 
[17:29:13] <wikibugs>	 3Continuous-Integration-Isolation, 6operations, 5Patch-For-Review: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1186679 (10RobH)
[17:31:58] <wikibugs>	 3Continuous-Integration-Isolation, 6operations, 5Patch-For-Review: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1186702 (10RobH)
[17:32:01] <wikibugs>	 10Browser-Tests, 10Gather, 3Gather Sprint Enwiki, 7Jenkins: Browser tests failing regularly due to styles not alway loading - https://phabricator.wikimedia.org/T94891#1186703 (10dduvall) 5Open>3Resolved a:3dduvall Right, it looks like the builder has since been refactored to not explicitly specify th...
[17:33:59] <Krinkle>	 legoktm: Wow, it's depressing how many useless jslint jobs ethere are
[17:34:01] <Krinkle>	 way more than I thought
[17:34:07] <Krinkle>	 This is going to take for ever xD
[17:34:10] <Krinkle>	 I'll do one a way
[17:34:12] <Krinkle>	 one a day
[17:35:13] <wikibugs>	 10Browser-Tests, 5Patch-For-Review: Support MEDIAWIKI_PROXY_URL for browser tests - https://phabricator.wikimedia.org/T71725#1186734 (10dduvall) Actually, any `environments.yml` configuration can be overridden by an environment variable, so `BROWSER_HTTP_PROXY=proxy.example:8080 bundle exec cucumber ...` would...
[17:35:40] <grrrit-wm>	 (03PS2) 10Krinkle: Delete old jshint jobs from repos without any *.js files (A-B) [integration/config] - 10https://gerrit.wikimedia.org/r/202445 
[17:38:36] <wikibugs>	 10Continuous-Integration, 10RESTBase, 7Documentation, 5Patch-For-Review: Publish RESTBase documentation on doc.wikimedia.org - https://phabricator.wikimedia.org/T87702#1186738 (10Qgil) 5Resolved>3declined The this task ("Publish RESTBase documentation on doc.wikimedia.org") is declined, because of the...
[17:40:13] <wikibugs>	 3Continuous-Integration-Isolation, 6operations: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1186746 (10RobH)
[17:42:01] <wikibugs>	 3Continuous-Integration-Isolation, 6operations: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1186777 (10RobH)
[17:49:45] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] doc: Add link to RESTBase documentation [integration/docroot] - 10https://gerrit.wikimedia.org/r/202418 (https://phabricator.wikimedia.org/T87702) (owner: 10Krinkle)
[17:49:47] <grrrit-wm>	 (03Merged) 10jenkins-bot: doc: Add link to RESTBase documentation [integration/docroot] - 10https://gerrit.wikimedia.org/r/202418 (https://phabricator.wikimedia.org/T87702) (owner: 10Krinkle)
[17:51:26] <grrrit-wm>	 (03PS1) 10Legoktm: Generalize test_mw_repos_have_composer_validate_job for any job [integration/config] - 10https://gerrit.wikimedia.org/r/202451 
[17:51:28] <grrrit-wm>	 (03PS1) 10Legoktm: Test that all mediawiki repos have phplint jobs [integration/config] - 10https://gerrit.wikimedia.org/r/202452 
[17:51:30] <grrrit-wm>	 (03PS2) 10Krinkle: mediawiki-core-doxygen-publish: Drop /html/ subdir from url [integration/config] - 10https://gerrit.wikimedia.org/r/202429 (https://phabricator.wikimedia.org/T73060) 
[17:52:45] <grrrit-wm>	 (03CR) 10Krinkle: Test that all mediawiki repos have phplint jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/202452 (owner: 10Legoktm)
[17:52:57] <Krinkle>	 legoktm: Where's that purge script again?
[17:53:39] <grrrit-wm>	 (03CR) 10Legoktm: Test that all mediawiki repos have phplint jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/202452 (owner: 10Legoktm)
[17:53:49] <legoktm>	 Krinkle: https://phabricator.wikimedia.org/T94408#1170948 ?
[17:54:13] <Krinkle>	 legoktm: Aye. Thanks
[17:54:26] <Krinkle>	 Thinking of a way to make that delete jobs in Jenkins
[17:54:37] <Krinkle>	 Though come to think of it, that'd be silly.
[17:54:41] <Krinkle>	 Jenkinsjob builder has that built in
[17:54:44] <Krinkle>	 and we can't run thatbecause of legacy jobs
[18:04:53] <legoktm>	 Krinkle: I just use jjb-diff to generate a list of removed jobs and then create a really long bash command to run
[18:31:39] <grrrit-wm>	 (03CR) 10Krinkle: Test that all mediawiki repos have phplint jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/202452 (owner: 10Legoktm)
[19:06:14] <wikibugs>	 3Continuous-Integration-Isolation, 6operations: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1187062 (10RobH)
[19:06:46] <wikibugs>	 3Continuous-Integration-Isolation, 6operations: install/deploy labnodepool1001 - https://phabricator.wikimedia.org/T95045#1179075 (10RobH) network switch port setup has an issue, described in sub-task T95048, once resolved installation can continue.
[19:21:02] <grrrit-wm>	 (03CR) 10GWicke: "Thank you, Timo!" [integration/docroot] - 10https://gerrit.wikimedia.org/r/202418 (https://phabricator.wikimedia.org/T87702) (owner: 10Krinkle)
[19:35:18] <greg-g>	 marxarelli: ^d hashar how'd the rest of that meeting go?
[19:35:49] <^d>	 long
[19:36:04] <marxarelli>	 what he said :)
[19:36:13] <^d>	 also yellow is no more.
[19:36:17] <^d>	 everything is green or red
[19:36:20] <marxarelli>	 red is the new yellow
[19:36:27] <^d>	 (sorry if you're colorblind)
[19:36:54] <hashar>	 greg-g: I have left just after our part
[19:36:56] <greg-g>	 haha
[19:36:58] <hashar>	 sorry folks :/
[19:37:02] <greg-g>	 hashar: no worries!
[19:37:02] <hashar>	 2nd kid is sick
[19:37:05] <hashar>	 1st was tired
[19:37:05] <greg-g>	 :(
[19:37:17] <hashar>	 I am taking time off tonight
[19:37:31] <greg-g>	 really I just wanted you there 1) if all goes well, to just see how it'd go next week and 2) if I needed backup :)
[19:37:34] <hashar>	 trying to figure out how to switch the audio output when I plug headphones in my comp
[19:37:40] <hashar>	 WHICH JUST WORKS ON MAC !!!!!!!!!!!!!
[19:37:48] <greg-g>	 heh
[19:37:51] <greg-g>	 works for me :)
[19:37:55] <hashar>	 terry had a bunch of points
[19:38:00] <hashar>	 probably intended for the general audience
[19:38:10] <hashar>	 I could barely understand him due to the bad mic in the room
[19:38:30] <^d>	 Well standing 6+ feet away never helps
[19:38:31] <hashar>	 seems next quarter we will want to be more specific about "things we learned"
[19:38:46] <^d>	 (people who want to speak should sit at the table nearer to the mic)
[19:39:04] <hashar>	 or everyone has its personal mic, but that is rather expensive :)
[19:39:23] <greg-g>	 ^d: yep
[19:39:37] <greg-g>	 hashar: yeah, agree re "things we learned"
[19:39:47] <greg-g>	 I just got back from an appt, catching up
[19:40:16] <hashar>	 I think he made the point for each of the teams
[19:40:22] <greg-g>	 cool
[19:40:25] <hashar>	 not necessarly blaming us
[19:40:27] <YuviPanda>	 is this the releng qr?
[19:40:33] <hashar>	 !google define qr
[19:40:37] <YuviPanda>	 or something more generic?
[19:40:40] <YuviPanda>	 Quarterly Review :)
[19:40:45] <greg-g>	 YuviPanda: releng+ops+labs+ect+parsoid+services+thekitchensink
[19:40:52] <YuviPanda>	 oh, I see :)
[19:40:57] <hashar>	 in just an hour
[19:40:59] <^d>	 Review all the quarters
[19:41:05] <hashar>	 to cover work of like 40 peoples? 
[19:41:09] <greg-g>	 yep!
[19:41:26] <greg-g>	 brb, getting mifi (coffee wifi sucks)
[19:41:34] <YuviPanda>	 oh, this is the ‘new quarterly review’ thing isn’t it?
[19:41:39] <YuviPanda>	 everrrytyhingg is in 40mins
[19:41:41] <YuviPanda>	 or something
[19:43:56] <greg-g>	 YuviPanda: yeppp
[19:45:19] <YuviPanda>	 ah, heh
[19:45:19] <YuviPanda>	 ok
[20:00:44] <wikibugs>	 7Blocked-on-RelEng, 10Continuous-Integration, 6Release-Engineering, 10Fundraising Tech Backlog, and 2 others: Configure Jenkins to run CiviCRM builds on Fundraising CI slave instance - https://phabricator.wikimedia.org/T89895#1187285 (10greg)
[20:00:46] <wikibugs>	 7Blocked-on-RelEng, 10Continuous-Integration, 6Release-Engineering, 10Fundraising Tech Backlog, and 4 others: Deploy CiviCRM integration job to WMF integration server - https://phabricator.wikimedia.org/T86374#1187282 (10greg)
[20:00:49] <wikibugs>	 7Blocked-on-RelEng, 10Continuous-Integration, 6Release-Engineering, 10Fundraising Tech Backlog, and 2 others: Create and provision CI slave instance for CiviCRM testing - https://phabricator.wikimedia.org/T89894#1187284 (10greg)
[20:02:22] <wikibugs>	 6Release-Engineering: Delete #Blocked-on-Continuous-Integration tag in favor of using #Blocked-On-Releng - https://phabricator.wikimedia.org/T95310#1187291 (10greg) 5Open>3Resolved a:3greg Added # blocked-on-releng (purposefully didn't link it here) to all of the blocked-on-ci tasks and archived blocked-on...
[20:09:31] <wmf-insecte>	 Yippee, build fixed!
[20:09:31] <wmf-insecte>	 Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #615: FIXED in 22 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/615/
[20:12:19] <wikibugs>	 10Staging, 6Labs, 5Patch-For-Review: Labs puppet ENC scrambles booleans - https://phabricator.wikimedia.org/T95240#1187339 (10thcipriani) FWIW puppet's internal ldap isn't that sophisticated:  Grabs all `puppetvar` values and splits them on '=' (just like the python enc): https://github.com/puppetlabs/puppet...
[20:54:42] <legoktm>	 greg-g: can I and MatmaRex get a deployment window on Thursday at 1700 UTC for the SandboxLink deployment?
[21:05:36] <stephanebisson>	 hi
[21:05:53] <legoktm>	 o/ hello!
[21:07:23] <stephanebisson>	 trying to run browser tests with vagrant (host is mac)
[21:07:46] <stephanebisson>	 the error is "unable to obtain stable firefox connection in 60 seconds", probably because firefox can't launch
[21:24:14] <^d>	 twentyafterfour: Master's now 1.26alpha for your 1.26wmf1 tomorrow
[21:30:53] <stephanebisson>	 so, is running MW-Vagrant on mac a valid config? Am I supposed to be able to run the browser tests?
[21:31:58] <^d>	 I run vagrant on a mac every day. No clue about browser tests though
[21:36:09] <greg-g>	 legoktm: yessir
[21:36:17] <greg-g>	 legoktm: go ahead and add it, plz, kthx
[21:37:04] <legoktm>	 will do
[21:37:09] <legoktm>	 thanks
[21:42:50] <grrrit-wm>	 (03PS27) 10Legoktm: Fix WikibaseJavaScriptApi tests [integration/config] - 10https://gerrit.wikimedia.org/r/180418 (https://phabricator.wikimedia.org/T86176) (owner: 10Adrian Lang)
[21:52:50] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/202429 (https://phabricator.wikimedia.org/T73060) (owner: 10Krinkle)
[21:53:00] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] doc: Add redirect for old mediawiki-core/master/php/html/ [integration/docroot] - 10https://gerrit.wikimedia.org/r/202413 (https://phabricator.wikimedia.org/T73060) (owner: 10Krinkle)
[21:54:20] <wikibugs>	 10Continuous-Integration: Jenkins: Doxygen job shouldn't have extra "html/" subdirectory and publish error as artefact - https://phabricator.wikimedia.org/T73060#1187706 (10Krinkle) 5Open>3Resolved a:3Krinkle
[21:55:44] <grrrit-wm>	 (03PS28) 10Legoktm: Fix WikibaseJavaScriptApi tests [integration/config] - 10https://gerrit.wikimedia.org/r/180418 (https://phabricator.wikimedia.org/T86176) (owner: 10Adrian Lang)
[21:56:33] <grrrit-wm>	 (03Merged) 10jenkins-bot: mediawiki-core-doxygen-publish: Drop /html/ subdir from url [integration/config] - 10https://gerrit.wikimedia.org/r/202429 (https://phabricator.wikimedia.org/T73060) (owner: 10Krinkle)
[21:59:03] <grrrit-wm>	 (03CR) 10Legoktm: "PS28: Moved custom job template to wikidata.yaml, and re-used wd-mw-composer-install-ext builder instead of creating a new one. Also switc" [integration/config] - 10https://gerrit.wikimedia.org/r/180418 (https://phabricator.wikimedia.org/T86176) (owner: 10Adrian Lang)
[22:05:05] <grrrit-wm>	 (03CR) 10Legoktm: "(I know I said I'd deploy and merge this today, but I'd prefer either Adrian or Jan to review my changes first.)" [integration/config] - 10https://gerrit.wikimedia.org/r/180418 (https://phabricator.wikimedia.org/T86176) (owner: 10Adrian Lang)
[23:05:59] <wmf-insecte>	 Project browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #79: FAILURE in 8 min 58 sec: https://integration.wikimedia.org/ci/job/browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/79/
[23:23:02] <wikibugs>	 10Staging, 6Labs, 5Patch-For-Review: Labs puppet ENC scrambles booleans - https://phabricator.wikimedia.org/T95240#1187958 (10yuvipanda) 5Open>3Resolved ```root@staging-palladium:/var/lib/git/operations/puppet# /usr/local/bin/ldap-yaml-enc.py $(facter -p ec2id).$(facter -p domain) classes: [base, 'role::...