[01:23:14] PROBLEM - Free space - all mounts on deployment-deploy02 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy02.diskspace._srv.byte_percentfree (<11.11%) [01:28:13] RECOVERY - Free space - all mounts on deployment-deploy02 is OK: OK: All targets OK [01:28:49] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace._srv.byte_percentfree (<10.00%) [01:43:51] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [04:02:20] 10Phabricator, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Core Platform Team: Make Milestone columns moveable like regular columns - https://phabricator.wikimedia.org/T147690 (10mmodell) 05Open→03Resolved [04:03:30] 10Phabricator, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Core Platform Team: Make Milestone columns moveable like regular columns - https://phabricator.wikimedia.org/T147690 (10mmodell) @WDoranWMF: As requested, you should now be able to reorder milestones just like any other column. [05:58:59] (03CR) 10DannyS712: [C: 03+1] "LGTM" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592746 (owner: 10Umherirrender) [06:00:01] (03CR) 10DannyS712: [C: 03+1] "LGTM" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592751 (https://phabricator.wikimedia.org/T232265) (owner: 10Umherirrender) [06:00:31] (03PS9) 10DannyS712: Move `SuperfluousVariadicArgComment` out of the FunctionCommentSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592286 (https://phabricator.wikimedia.org/T250958) [08:40:43] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Push-Notification-Service, 10Product-Infrastructure-Team-Backlog (Kanban): Add PushNotifications extension to release branch cut - https://phabricator.wikimedia.org/T251442 (10Aklapper) [10:07:51] (03CR) 10Umherirrender: [C: 03+2] "This is refactoring, lets improve it in a separate patch set" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592286 (https://phabricator.wikimedia.org/T250958) (owner: 10DannyS712) [10:08:31] (03Merged) 10jenkins-bot: Move `SuperfluousVariadicArgComment` out of the FunctionCommentSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592286 (https://phabricator.wikimedia.org/T250958) (owner: 10DannyS712) [10:10:33] 10Diffusion, 10Release-Engineering-Team, 10Projects-Cleanup, 10Wikimedia-GitHub, 10User-MarcoAurelio: Delete/Archive Github and Diffusion mirrors of Gerrit repos related wikiba.se - https://phabricator.wikimedia.org/T239429 (10MarcoAurelio) a:03MarcoAurelio [10:13:53] 10Diffusion, 10Release-Engineering-Team, 10Projects-Cleanup, 10Wikimedia-GitHub, 10User-MarcoAurelio: Delete/Archive Github and Diffusion mirrors of Gerrit repos related wikiba.se - https://phabricator.wikimedia.org/T239429 (10MarcoAurelio) @WMDE-leszek Could you please activate those projects again on G... [10:27:01] (03PS1) 10Umherirrender: Check index in VariadicArgumentSniff and avoid some local vars [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593732 [11:11:58] (03PS1) 10Umherirrender: Code cleanup: Remove extra php7.4 code from UnsortedUseStatement [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593737 [11:56:10] (03CR) 10Umherirrender: Full rewrite of the UnusedUseStatementSniff, 4x faster (039 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/592415 (owner: 10Thiemo Kreuz (WMDE)) [14:56:54] (03CR) 10Thcipriani: "Nice :)" [blubber] - 10https://gerrit.wikimedia.org/r/591530 (https://phabricator.wikimedia.org/T250764) (owner: 10Jeena Huneidi) [15:19:29] (03PS1) 10DannyS712: Move FunctionCommentSniff annotation checks to FunctionAnnotationSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593746 (https://phabricator.wikimedia.org/T250958) [15:22:06] (03PS2) 10DannyS712: Move FunctionCommentSniff annotation checks to FunctionAnnotationSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593746 (https://phabricator.wikimedia.org/T250958) [15:25:34] (03PS3) 10DannyS712: Move FunctionCommentSniff annotation checks to FunctionAnnotationSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593746 (https://phabricator.wikimedia.org/T250958) [15:27:15] (03PS4) 10DannyS712: Move FunctionCommentSniff annotation checks to FunctionAnnotationSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/593746 (https://phabricator.wikimedia.org/T250958) [15:38:40] 10Gerrit, 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10ci-test-error (WMF-deployed Build Failure): Jenkins job failing intermittently due to Gerrit HTTP 502 errors when interacting with repos - https://phabricator.wikimedia.org/T246763 (10thcipriani) 05Open→03Resolved... [16:08:16] PROBLEM - Host integration-slave-jessie-1002 is DOWN: CRITICAL - Host Unreachable (172.16.1.99) [16:13:00] PROBLEM - Host integration-trigger-01 is DOWN: CRITICAL - Host Unreachable (172.16.6.6) [16:25:27] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Parsoid, 10Patch-For-Review: Undeploy ParsoidBatchAPI from the Wikimedia cluster - https://phabricator.wikimedia.org/T242430 (10cscott) [16:55:14] (03PS10) 10Jeena Huneidi: node.go: Add npm ci to build instructions [blubber] - 10https://gerrit.wikimedia.org/r/591530 (https://phabricator.wikimedia.org/T250764) [17:09:15] (03PS1) 10Jeena Huneidi: PythonConfig: Change UseSystemFlag to Flag [blubber] - 10https://gerrit.wikimedia.org/r/593771 [17:12:28] (03CR) 10Jforrester: [C: 03+2] "…" [tools/release] - 10https://gerrit.wikimedia.org/r/593346 (https://phabricator.wikimedia.org/T91649) (owner: 10Jforrester) [17:13:02] (03Merged) 10jenkins-bot: Stop branching the Sentry extension for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/593346 (https://phabricator.wikimedia.org/T91649) (owner: 10Jforrester) [17:21:45] 10Release-Engineering-Team-TODO, 10Scap, 10MediaWiki-Internationalization, 10Performance-Team, 10Patch-For-Review: Use static php array files for l10n cache at WMF (instead of CDB) - https://phabricator.wikimedia.org/T99740 (10ori) @Joe, I appreciate the effort you put in to evaluating this change! If yo... [17:27:56] (03PS5) 10Thcipriani: Add the very dangerous --delete argument to mass delete branches [tools/release] - 10https://gerrit.wikimedia.org/r/575571 (owner: 1020after4) [17:30:47] (03CR) 10Thcipriani: [C: 03+2] Add the very dangerous --delete argument to mass delete branches [tools/release] - 10https://gerrit.wikimedia.org/r/575571 (owner: 1020after4) [17:31:29] (03Merged) 10jenkins-bot: Add the very dangerous --delete argument to mass delete branches [tools/release] - 10https://gerrit.wikimedia.org/r/575571 (owner: 1020after4) [18:06:33] (03Abandoned) 10Brennen Bearnes: (DNM) delete-wmf-branches: tag at branchpoints [tools/release] - 10https://gerrit.wikimedia.org/r/588467 (https://phabricator.wikimedia.org/T250095) (owner: 10Brennen Bearnes) [18:15:38] (03CR) 10Thcipriani: [C: 03+2] node.go: Add npm ci to build instructions [blubber] - 10https://gerrit.wikimedia.org/r/591530 (https://phabricator.wikimedia.org/T250764) (owner: 10Jeena Huneidi) [18:17:57] (03Merged) 10jenkins-bot: node.go: Add npm ci to build instructions [blubber] - 10https://gerrit.wikimedia.org/r/591530 (https://phabricator.wikimedia.org/T250764) (owner: 10Jeena Huneidi) [18:21:24] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10MediaWiki-Release-Tools, 10User-brennen: branch.py: Convert deleted wmf branches to tags - https://phabricator.wikimedia.org/T250095 (10brennen) [18:41:50] !log Deleting integration-slave-jessie-1002 and integration-slave-jessie-1004 instances from WMCS for T236576 [18:41:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:41:53] T236576: "integration" Cloud VPS project jessie deprecation - https://phabricator.wikimedia.org/T236576 [18:42:58] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Cloud-VPS (Debian Jessie Deprecation): "integration" Cloud VPS project jessie deprecation - https://phabricator.wikimedia.org/T236576 (... [18:43:21] !log Deleting integration-trigger-01 from WMCS for T247109 [18:43:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:43:24] T247109: Replace integration-trigger-01 with running the dummy job on contint1001 agent - https://phabricator.wikimedia.org/T247109 [18:43:55] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Replace integration-trigger-01 with running the dummy job on contint1001 agent - https://phabricator.wikimedia.org/T247109 (10Jdforrester-W... [18:44:00] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Cloud-VPS (Debian Jessie Deprecation): "integration" Cloud VPS project jessie deprecation - https://phabricator.wikimedia.org/T236576 (... [18:50:01] longma: Are you using integration-slave-test-jeena? We're out of VCPUs and I need to create the non-jessie replacement for integration-agent-puppet-docker-1001… [18:50:22] James_F: no, I'll delete it [18:51:04] Thanks! [18:51:19] Once we've migrated puppet, that leaves just cumin, and then we're done. [18:51:40] And then there'll be spare VCPUs for re-create test-jeena if you want it. [18:52:16] James_F: done! [18:55:09] Thanks! [18:56:27] !log Created integration-agent-puppet-docker-1002 based on stretch for T250502 [18:56:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:56:29] T250502: Rebuild integration-agent-puppet-docker-1001 to use Stretch - https://phabricator.wikimedia.org/T250502 [18:58:14] PROBLEM - Host integration-slave-test-jeena is DOWN: CRITICAL - Host Unreachable (172.16.0.234) [19:20:19] !log Added self to integration sudoers policy [19:20:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:40:01] !log jjb: Dropping operations-puppet-tests-stretch-docker, not triggered for over a month [19:40:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:42:21] (03PS1) 10Jforrester: jjb: Create operations-puppet-tests-buster-docker-test [integration/config] - 10https://gerrit.wikimedia.org/r/593802 (https://phabricator.wikimedia.org/T250502) [19:42:23] (03PS1) 10Jforrester: layout: Add operations-puppet-tests-buster-docker-test experimental [integration/config] - 10https://gerrit.wikimedia.org/r/593803 (https://phabricator.wikimedia.org/T250502) [19:42:39] (03CR) 10Jforrester: [C: 03+2] jjb: Create operations-puppet-tests-buster-docker-test [integration/config] - 10https://gerrit.wikimedia.org/r/593802 (https://phabricator.wikimedia.org/T250502) (owner: 10Jforrester) [19:42:46] (03CR) 10Jforrester: [C: 03+2] layout: Add operations-puppet-tests-buster-docker-test experimental [integration/config] - 10https://gerrit.wikimedia.org/r/593803 (https://phabricator.wikimedia.org/T250502) (owner: 10Jforrester) [19:43:35] (03Merged) 10jenkins-bot: jjb: Create operations-puppet-tests-buster-docker-test [integration/config] - 10https://gerrit.wikimedia.org/r/593802 (https://phabricator.wikimedia.org/T250502) (owner: 10Jforrester) [19:43:40] (03Merged) 10jenkins-bot: layout: Add operations-puppet-tests-buster-docker-test experimental [integration/config] - 10https://gerrit.wikimedia.org/r/593803 (https://phabricator.wikimedia.org/T250502) (owner: 10Jforrester) [19:44:17] !log Zuul: [operations/puppet] Add operations-puppet-tests-buster-docker-test as experimental T250502 [19:44:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:44:22] T250502: Rebuild integration-agent-puppet-docker-1001 to use Stretch - https://phabricator.wikimedia.org/T250502 [19:49:27] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review: Rebuild integration-agent-puppet-docker-1001 to use Stretch - https://phabricator.wikimedia.org/T250502 (10Jdforreste... [20:01:22] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Parsoid, 10Patch-For-Review: Undeploy ParsoidBatchAPI from the Wikimedia cluster - https://phabricator.wikimedia.org/T242430 (10cscott) [21:16:42] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Operations: Assess whether we should still disable seccomp in Docker for CI - https://phabricator.wikimedia.org/T249729 (10hashar) >>! In T249729#6092568, @MoritzMuehlenhoff wrote... [21:35:24] 10Release-Engineering-Team-TODO, 10Scap, 10MediaWiki-Internationalization, 10Performance-Team, 10Patch-For-Review: Use static php array files for l10n cache at WMF (instead of CDB) - https://phabricator.wikimedia.org/T99740 (10Nemo_bis) > This is a valid point! I'll repeat the test on a special page. I f... [21:50:11] (03PS1) 10Paladox: Merge branch 'stable-2.16' into wmf/stable-2.16 [software/gerrit] (wmf/stable-2.16) - 10https://gerrit.wikimedia.org/r/593836 [21:51:41] (03CR) 10jerkins-bot: [V: 04-1] Merge branch 'stable-2.16' into wmf/stable-2.16 [software/gerrit] (wmf/stable-2.16) - 10https://gerrit.wikimedia.org/r/593836 (owner: 10Paladox) [22:05:05] (03PS2) 10Jeena Huneidi: PythonConfig: Change UseSystemFlag to Flag [blubber] - 10https://gerrit.wikimedia.org/r/593771 [22:06:57] (03PS1) 10Paladox: Remove 4 plugins [software/gerrit] (wmf/stable-2.16) - 10https://gerrit.wikimedia.org/r/593852 [22:07:15] (03PS3) 10Jeena Huneidi: PythonConfig: Change UseSystemFlag to Flag [blubber] - 10https://gerrit.wikimedia.org/r/593771 [22:29:13] 10Release-Engineering-Team, 10Developer-Advocacy, 10Documentation: Document venues for hosting new code repositories and recommend when to use what - https://phabricator.wikimedia.org/T251651 (10srishakatux) [22:47:14] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:51] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:56] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:00:08] 10Phabricator (Upstream), 10Upstream: Unknown search function "XXX". Supported functions are: all, title, body, core, comment. - https://phabricator.wikimedia.org/T243483 (10epriestley) This error got a little stricter recently so it might crop up more often after the next upgrade, and it's normally hard to hi... [23:04:42] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 93570 bytes in 1.094 second response time [23:04:47] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 93265 bytes in 0.984 second response time [23:07:06] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 52097 bytes in 1.108 second response time [23:30:56] 10Continuous-Integration-Config, 10Parsoid: Set up extension tests for Parsoid repo - https://phabricator.wikimedia.org/T227352 (10cscott) [23:31:24] (03PS3) 10C. Scott Ananian: jjb: [parsoidsvc-parsertests-docker] Update path to parsoid [integration/config] - 10https://gerrit.wikimedia.org/r/588026 (https://phabricator.wikimedia.org/T248726) [23:50:52] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:50:57] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:53:14] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds