[01:22:24] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10Mholloway) The fix for T259126 is backported to wmf/1.36.0-wmf.2 and the train sho... [02:00:49] !log Cherry-picking https://gerrit.wikimedia.org/r/c/performance/arc-lamp/+/617275 in beta, should only affect deployment-webperf12. [02:00:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:05:52] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.62 ms [02:10:50] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [02:15:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.83 ms [02:20:51] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [02:25:03] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: CRITICAL: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found)deployment-prep.deployment-snapshot01.diskspace.root.byte_percentfree (<10.00%) [02:35:00] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [02:45:55] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 2.84 ms [02:55:53] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [05:06:43] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Parsoid, 10MW-1.36-notes (1.36.0-wmf.3; 2020-08-04), and 2 others: Parsoid CI is failing when parser tests are run - https://phabricator.wikimedia.org/T259186 (10cscott) 05Open→03Resolved a:03cscott [05:40:54] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 3.65 ms [05:50:53] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [07:05:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.07 ms [07:10:50] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [07:34:21] Yippee, build fixed! [07:34:22] Project mwcore-phpunit-coverage-master build #817: 09FIXED in 4 hr 34 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/817/ [08:28:27] PROBLEM - Host deployment-changeprop is DOWN: CRITICAL - Host Unreachable (172.16.5.21) [08:28:28] PROBLEM - Host deployment-cpjobqueue is DOWN: CRITICAL - Host Unreachable (172.16.4.124) [08:28:28] PROBLEM - Host deployment-chromium02 is DOWN: CRITICAL - Host Unreachable (172.16.4.14) [08:57:17] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10Epic, 10Testing-Initiative: Follow up workshop & brown bag ideas from Testing: Where does it hurt? (tracking) - https://phabricator.wikimedia.org/T108122 (10Aklapper) [09:44:00] (03PS3) 10Hashar: Reuse previous artifacts as a wheel cache [integration/zuul/deploy] - 10https://gerrit.wikimedia.org/r/606242 [09:44:02] (03PS1) 10Hashar: Upgrade gear from 0.7.0 to 1.15.1+wmf1 [integration/zuul/deploy] - 10https://gerrit.wikimedia.org/r/617404 (https://phabricator.wikimedia.org/T258630) [09:49:13] (03CR) 10Hashar: "Looks like I forgot to add reviewers :) The intent is to avoid rebuilding wheels." [integration/zuul/deploy] - 10https://gerrit.wikimedia.org/r/606242 (owner: 10Hashar) [09:54:38] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10User-DannyS712: Some extensions' code coverage reports fail... - https://phabricator.wikimedia.org/T259109 [09:55:59] (03CR) 10Hashar: [C: 03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/617097 (https://phabricator.wikimedia.org/T259109) (owner: 10Hashar) [09:56:57] (03Merged) 10jenkins-bot: jjb: enhance rsync output in publish-to-doc1001 [integration/config] - 10https://gerrit.wikimedia.org/r/617097 (https://phabricator.wikimedia.org/T259109) (owner: 10Hashar) [10:00:54] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 3.78 ms [10:02:26] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10User-DannyS712: Some extensions' code coverage reports fail... - https://phabricator.wikimedia.org/T259109 [10:04:43] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE), 10Patch-For-Review, 10User-Vidhi-Mody, 10User-zeljkofilipin: Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10zeljkofilipin) [10:05:01] 10Phabricator, 10Quality-and-Test-Engineering-Team (QTE), 10Patch-For-Review, 10User-Vidhi-Mody, 10User-zeljkofilipin: Upgrade WebdriverIO in the phab-deployment repository - https://phabricator.wikimedia.org/T255471 (10zeljkofilipin) a:05Vidhi-Mody→03None [10:05:50] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [10:08:59] (03PS1) 10Elukey: Add settings for operations/software/pywmflib [integration/config] - 10https://gerrit.wikimedia.org/r/617408 (https://phabricator.wikimedia.org/T257905) [10:11:04] (03CR) 10Volans: [C: 03+1] "Thanks Luca, looks sane to me, but I'll leave it to RelEng for a final sanity check." [integration/config] - 10https://gerrit.wikimedia.org/r/617408 (https://phabricator.wikimedia.org/T257905) (owner: 10Elukey) [10:14:17] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-DannyS712: Some extensions' code coverage reports fail with `Empty directory... - https://phabricator.wikimedia.org/T259109 [10:31:07] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-DannyS712: Some extensions' code coverage reports fail with `Empty directory... - https://phabricator.wikimedia.org/T259109 [10:32:39] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-DannyS712: mwext-codehealth-master-non-voting erase doc.wikimedia.org covera... - https://phabricator.wikimedia.org/T259109 [10:51:41] Urbanecm: does https://phabricator.wikimedia.org/p/Jstone6336/ want disabling? [10:51:57] * RhinosF1 can [10:52:07] * Urbanecm can too, but is not sure [10:52:26] Urbanecm: I'll keep my finger on the button and watch them [10:52:26] I'm more inclined to monitor them, and let's see what happens [10:52:41] (y) [10:52:47] I was about to ask about that [10:53:16] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Promoted back to group1, out of train window. Wanted to do this ear... [10:54:45] Both my laptop and phone have it ready [11:00:36] PROBLEM - Host deployment-mcs01 is DOWN: CRITICAL - Host Unreachable (172.16.5.64) [11:01:02] 10Continuous-Integration-Infrastructure, 10Operations, 10Product-Infrastructure-Team-Backlog, 10Push-Notification-Service, 10serviceops: puppet errors on contint servers related to helmfiles for push-notifications - https://phabricator.wikimedia.org/T259152 (10akosiaris) 05Open→03Resolved a:03akosia... [11:09:45] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) [11:10:10] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) [11:16:29] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10LarsWirzenius) Reported {T259247} but not as a blocker at this time. [11:33:48] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10Urbanecm) [11:40:53] (03PS1) 10Dr0ptp4kt: Edit Repo Config [extensions/WikiLambda] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/617426 [11:43:55] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-DannyS712: mwext-codehealth-master-non-voting erase doc.wikimedia.org covera... - https://phabricator.wikimedia.org/T259109 [12:19:39] 10Continuous-Integration-Config, 10Code-Health-Metrics, 10Patch-For-Review: Enable codehealth pipeline for more extensions - https://phabricator.wikimedia.org/T224990 (10hashar) [12:19:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-DannyS712: mwext-codehealth-master-non-voting erase doc.wikimedia.org covera... - https://phabricator.wikimedia.org/T259109 [12:46:06] (03PS1) 10Hashar: Avoid duplicate codecoverage report [integration/config] - 10https://gerrit.wikimedia.org/r/617453 (https://phabricator.wikimedia.org/T242444) [12:49:35] (03PS2) 10Hashar: Avoid duplicate codecoverage report [integration/config] - 10https://gerrit.wikimedia.org/r/617453 (https://phabricator.wikimedia.org/T242444) [12:54:15] (03CR) 10Hashar: "So apparently the codehealth job is supposed to craft a HTML report and to replace the extension-coverage template, that was done by 55043" [integration/config] - 10https://gerrit.wikimedia.org/r/617453 (https://phabricator.wikimedia.org/T242444) (owner: 10Hashar) [13:26:28] (03PS1) 10Giuseppe Lavagetto: Add envoy to the helm-linter image [integration/config] - 10https://gerrit.wikimedia.org/r/617459 [13:48:45] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10hashar) >>! In T257879#6346810, @Tgr wrote: > Per the [[https://www.mediawiki.org/wiki/Support_policy_for_PHP|po... [13:54:31] (03CR) 10Alexandros Kosiaris: [C: 04-1] "The changelog says it adds envoy, but I don't see Dockefile.template being indeed updated" [integration/config] - 10https://gerrit.wikimedia.org/r/617459 (owner: 10Giuseppe Lavagetto) [14:21:21] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Tgr) [14:21:42] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Tgr) >>! In T257879#6348477, @hashar wrote: > Hope that clarify? It does, thanks! [14:33:17] (03PS1) 10Hashar: jjb: debian-glue should not checkout to a branch [integration/config] - 10https://gerrit.wikimedia.org/r/617476 (https://phabricator.wikimedia.org/T141607) [14:36:40] (03CR) 10Hashar: [C: 03+2] "I rebuild the faulty acme-chief build and it passed just fine once this got deployed ;) Thank you for the notification!" [integration/config] - 10https://gerrit.wikimedia.org/r/617476 (https://phabricator.wikimedia.org/T141607) (owner: 10Hashar) [14:37:35] (03Merged) 10jenkins-bot: jjb: debian-glue should not checkout to a branch [integration/config] - 10https://gerrit.wikimedia.org/r/617476 (https://phabricator.wikimedia.org/T141607) (owner: 10Hashar) [15:13:24] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10hashar) >>! In T257879#6348477, @hashar wrote: > ... I think we had asked for a rebuild of php7.2 for Buster al... [15:15:39] (03CR) 10Giuseppe Lavagetto: "> Patch Set 1: Code-Review-1" [integration/config] - 10https://gerrit.wikimedia.org/r/617459 (owner: 10Giuseppe Lavagetto) [15:16:47] (03PS2) 10Giuseppe Lavagetto: Add envoy to the helm-linter image [integration/config] - 10https://gerrit.wikimedia.org/r/617459 [15:18:23] (03PS3) 10Giuseppe Lavagetto: Add envoy to the helm-linter image [integration/config] - 10https://gerrit.wikimedia.org/r/617459 [15:21:59] PROBLEM - Parsoid on deployment-parsoid11 is CRITICAL: connect to address 172.16.1.115 and port 8000: Connection refused [15:36:46] 10Phabricator: Spaces request for ...Tech Sr Leadership - https://phabricator.wikimedia.org/T259156 (10Aklapper) @debt: If I get it right, it sounds like we should archive one Space (which one?) and one project (which one?) in any case? >>! In T259156#6346645, @debt wrote: > Would it be easier to close those tw... [15:36:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Selenium quibble jobs have a huge cache overloading the CI ag... - https://phabricator.wikimedia.org/T258972 [15:45:52] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.76 ms [15:53:15] !log Marked integration-agent-docker-1005 offline in Jenkins to poke at a Puppet failure related to Docker upgrade there [15:53:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:53:36] * bd808 waits for jobs to drain from integration-agent-docker-1005 [15:55:53] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [16:05:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.74 ms [16:12:43] !log Rebooting integration-agent-docker-1005 [16:12:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:14:02] PROBLEM - SSH on integration-agent-docker-1005 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:18:53] RECOVERY - SSH on integration-agent-docker-1005 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u7 (protocol 2.0) [16:19:45] !log Marked integration-agent-docker-1005 as online after reboot [16:19:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:29:20] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy: Selenium quibble jobs have a huge cache overloading the CI ag... - https://phabricator.wikimedia.org/T258972 [16:41:28] 10Gerrit, 10Repository-Admins: Decide whether requests for new Gerrit repos are handled on mediawiki.org or in Phabricator (currently it's both) - https://phabricator.wikimedia.org/T259147 (10MarcoAurelio) This discussion is happening already at https://www.mediawiki.org/wiki/Topic:Uqfdno10snrrfmc0 (started 29... [16:45:21] 10Gerrit, 10Patch-For-Review: Use WMF Gerrit's logo as favicon - https://phabricator.wikimedia.org/T257218 (10hashar) I too, I definitely like that new comer {F31952330} I would definitely have voted for it in V21 if it was there ;) [16:50:09] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10bd808) [16:51:41] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10Dzahn) This comes from T254310#6347281. [16:54:43] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10MediaWiki Train Development Environment: Create static host keys for train dev vm - https://phabricator.wikimedia.org/T259279 (10thcipriani) [16:54:56] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10MediaWiki Train Development Environment: Create static host keys for train dev vm - https://phabricator.wikimedia.org/T259279 (10thcipriani) p:05Triage→03Medium [16:55:40] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10greg) Hey @dpifke, pinging you since I saw xhgui above :) [16:55:59] bah, mutante beat me and phab didn't notify me :) [16:56:03] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10MediaWiki Train Development Environment: Move user public keys for train dev into user-data for cloudinit - https://phabricator.wikimedia.org/T259280 (10thcipriani) [16:56:16] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10MediaWiki Train Development Environment: Move user public keys for train dev into user-data for cloudinit - https://phabricator.wikimedia.org/T259280 (10thcipriani) p:05Triage→03Medium [17:01:31] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10User-DannyS712: mwext-codehealth-master-non-voting erase do... - https://phabricator.wikimedia.org/T259109 [17:24:54] No worries. :) [17:45:29] maintenance-disconnect-full-disks build 198764 integration-agent-docker-1016 (/: 17%, /srv: 97%, /var/lib/docker: 44%): OFFLINE due to disk space [17:50:28] maintenance-disconnect-full-disks build 198765 integration-agent-docker-1016 (/: 17%, /srv: 47%, /var/lib/docker: 44%): RECOVERY disk space OK [17:50:42] PROBLEM - Free space - all mounts on integration-agent-docker-1016 is CRITICAL: CRITICAL: integration.integration-agent-docker-1016.diskspace._srv.byte_percentfree (<11.11%) [17:54:19] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10Dzahn) It was introduced by https://gerrit.wikimedia.org/r/c/operations/puppet/+/610446 Working on providing the php-... [17:54:29] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10User-dancy, 10User-zeljkofilipin: Selenium quibble jobs have a huge cac... - https://phabricator.wikimedia.org/T258972 [17:55:25] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10Dzahn) [17:55:44] RECOVERY - Free space - all mounts on integration-agent-docker-1016 is OK: OK: All targets OK [18:09:43] PROBLEM - Free space - all mounts on integration-agent-docker-1003 is CRITICAL: CRITICAL: integration.integration-agent-docker-1003.diskspace._srv.byte_percentfree (<11.11%) [18:11:17] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Akuckartz) Maybe a compromise is possible: # No official support for 7.2 # Continue supporting 7.2 as long as i... [18:14:46] RECOVERY - Free space - all mounts on integration-agent-docker-1003 is OK: OK: All targets OK [18:49:30] 10phan, 10phan-taint-check-plugin, 10Patch-For-Review: Upgrade to phan/phan 3.1.0 or later in mediawiki/tools/phan - https://phabricator.wikimedia.org/T250170 (10Umherirrender) [19:20:06] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) [19:26:08] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Patch-For-Review, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10Dzahn) a:03Dzahn [19:31:26] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) a:05LarsWirzenius→03brennen [20:11:06] how did the deployment-prep puppetmaster get synced from prod puppetmaster [20:45:11] !log Fixed merge conflicts in beta puppetmaster. [20:45:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:48:37] 10Beta-Cluster-Infrastructure, 10Performance-Team, 10Puppet: Puppet failing due to dependency cycle on deployment-xhgui03 - https://phabricator.wikimedia.org/T259278 (10Dzahn) 05Open→03Resolved The issue is gone now with the change above which stopped using require_package to install xhgui. (and after D... [21:17:18] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Graphoid, 10Operations, and 2 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10kaldari) I'd like to propose that we close this ticket, since we've decided we are no longer going to... [21:21:42] 10Release-Engineering-Team, 10MediaWiki-General, 10MediaWiki-Stakeholders-Group, 10serviceops, and 3 others: Drop PHP 7.2 support in MediaWiki 1.35 - https://phabricator.wikimedia.org/T257879 (10Krinkle) @Akuckartz As I understand it, yes, that is and has been the proposal from the start of this task. [21:24:45] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Graphoid, 10Operations, and 2 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10Krinkle) So who has taken stewardship over the service for its remaining operation and to do the work... [21:31:26] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Platform Engineering, 10Quibble, and 3 others: Jenkins no longer fails build if mw-error.log or mw-dberror.log are non-empty - https://phabricator.wikimedia.org/T246358 (10Krin... [21:31:28] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Developer Productivity, 10Regression, 10Sustainability (Incident Followup): Jenkins no longer fails build if mw-error.log or mw-dbe... - https://phabricator.wikimedia.org/T246358 [21:31:32] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Performance-Team, 10Developer Productivity, and 2 others: Jenkins no longer fails build if mw-error.log or mw-dberror.log are non-emp... - https://phabricator.wikimedia.org/T246358 [21:46:25] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Graphoid, 10Operations, and 2 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10kaldari) The undeployment/client-side enabling is being handled by @Jseddon. For supporting Graphoid... [22:23:23] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) [22:32:19] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) a:05brennen→03LarsWirzenius Rolled out with fix for T258609 at 21:52... [22:36:26] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) [22:55:22] (03PS3) 10Dduvall: Convert kask ci to use pipelinelib [integration/config] - 10https://gerrit.wikimedia.org/r/616177 (https://phabricator.wikimedia.org/T224041) (owner: 10Jeena Huneidi) [23:02:04] !log created 6 new jenkins jobs to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/616177 [23:02:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:02:11] (03CR) 10Dduvall: [C: 03+2] "INFO:jenkins_jobs.builder:Number of jobs generated: 6" [integration/config] - 10https://gerrit.wikimedia.org/r/616177 (https://phabricator.wikimedia.org/T224041) (owner: 10Jeena Huneidi) [23:02:23] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10cscott) It's the end of my working day Eastern Time Thursday. From the above, I'm... [23:03:17] (03Merged) 10jenkins-bot: Convert kask ci to use pipelinelib [integration/config] - 10https://gerrit.wikimedia.org/r/616177 (https://phabricator.wikimedia.org/T224041) (owner: 10Jeena Huneidi) [23:05:02] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) @cscott sounds good - thanks. [23:07:20] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/616177 [23:07:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:13:16] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release, 10Train Deployments, 10User-brennen: 1.36.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T257970 (10brennen) (For clarity: No need to accelerate.) [23:45:04] 10Release-Engineering-Team (Pipeline), 10Release Pipeline: PipelineLib fails on non-string values for deploy.overrides - https://phabricator.wikimedia.org/T259318 (10dduvall) [23:45:31] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release Pipeline: PipelineLib fails on non-string values for deploy.overrides - https://phabricator.wikimedia.org/T259318 (10dduvall) [23:45:49] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Release Pipeline: PipelineLib fails on non-string values for deploy.overrides - https://phabricator.wikimedia.org/T259318 (10dduvall) [23:54:20] PROBLEM - Host deployment-sentry01 is DOWN: CRITICAL - Host Unreachable (172.16.5.16)