[03:03:34] PROBLEM - PHD should be supervising processes on phab1001 is CRITICAL: PROCS CRITICAL: 2 processes with UID = 497 (phd) https://wikitech.wikimedia.org/wiki/Phabricator [03:06:02] RECOVERY - PHD should be supervising processes on phab1001 is OK: PROCS OK: 4 processes with UID = 497 (phd) https://wikitech.wikimedia.org/wiki/Phabricator [06:40:51] (03CR) 10Thiemo Kreuz (WMDE): "I introduced a few of these customizations. I tried my best to document them directly in place, in the code. Maybe it's better to just poi" (034 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/682272 (https://phabricator.wikimedia.org/T274016) (owner: 10Umherirrender) [07:17:02] 10Release-Engineering-Team, 10MW-on-K8s, 10serviceops: Generate non-restricted versions of the mediawiki-webserver image - https://phabricator.wikimedia.org/T281079 (10Joe) [07:27:26] (03CR) 10Hashar: [C: 03+2] Archive the SmoothGallery extension [integration/config] - 10https://gerrit.wikimedia.org/r/679090 (https://phabricator.wikimedia.org/T252273) (owner: 10Zoranzoki21) [07:27:31] (03CR) 10Hashar: [C: 03+2] Archive the DumpHTML extension [integration/config] - 10https://gerrit.wikimedia.org/r/679443 (https://phabricator.wikimedia.org/T280185) (owner: 10Zabe) [07:28:51] (03Merged) 10jenkins-bot: Archive the DumpHTML extension [integration/config] - 10https://gerrit.wikimedia.org/r/679443 (https://phabricator.wikimedia.org/T280185) (owner: 10Zabe) [07:30:32] (03CR) 10Hashar: [C: 03+2] zuul/layout.yaml: Add Alex Mashin in the &email_allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/679079 (owner: 10Zoranzoki21) [07:31:53] (03Merged) 10jenkins-bot: zuul/layout.yaml: Add Alex Mashin in the &email_allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/679079 (owner: 10Zoranzoki21) [07:32:10] (03Merged) 10jenkins-bot: Archive the SmoothGallery extension [integration/config] - 10https://gerrit.wikimedia.org/r/679090 (https://phabricator.wikimedia.org/T252273) (owner: 10Zoranzoki21) [08:43:24] 10Gerrit, 10Release-Engineering-Team (Doing), 10Patch-For-Review: Gerrit Apache out of workers - https://phabricator.wikimedia.org/T277127 (10hashar) 05Resolved→03Open I have looked at the Apache scoreboard and Gerrit thread pool and they seem fine now. Gerrit is at less than 40 threads out of a total po... [09:02:46] 10Continuous-Integration-Infrastructure: Jenkins: Investigate using concurrent builds of the same job - https://phabricator.wikimedia.org/T49063 (10hashar) [10:08:14] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [10:11:40] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [10:27:46] CI is rather clogged up [10:41:18] 10Beta-Cluster-Infrastructure, 10ContentTranslation, 10Wikimedia-Site-requests, 10Language-Team (Language-2021-April-June): Beta cluster Special:ContentTranslation Error: Unsupported operand types - https://phabricator.wikimedia.org/T276656 (10Pginer-WMF) p:05Triage→03Medium a:03Nikerabbit [10:46:48] 10phan-taint-check-plugin: Correctly process argument links when backpropagating EXEC taintedness - https://phabricator.wikimedia.org/T274780 (10Daimona) Note (mostly to self): this can be easily resolved once [[https://gerrit.wikimedia.org/r/c/mediawiki/tools/phan/SecurityCheckPlugin/+/681443/7 | r681443 ]] is... [10:55:43] (03PS1) 10Lars Wirzenius: fix: avoid duplicate plugin directories [tools/scap] - 10https://gerrit.wikimedia.org/r/682593 [13:04:14] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Doing): Upgrade Jenkins to 2.277.x - https://phabricator.wikimedia.org/T279033 (10hashar) Neat! Looks like that requires //Winstone-Jetty is configured to handle SSL/TLS connections// but on our setup we use plain HTTP (and env... [15:13:11] (03CR) 10Ahmon Dancy: [C: 03+2] fix: avoid duplicate plugin directories [tools/scap] - 10https://gerrit.wikimedia.org/r/682593 (owner: 10Lars Wirzenius) [15:13:39] (03CR) 10Ottomata: [C: 03+1] Enable sonar analysis post merge for java projects. [integration/config] - 10https://gerrit.wikimedia.org/r/681988 (owner: 10Gehel) [15:14:07] (03Merged) 10jenkins-bot: fix: avoid duplicate plugin directories [tools/scap] - 10https://gerrit.wikimedia.org/r/682593 (owner: 10Lars Wirzenius) [15:19:00] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281145 (10thcipriani) p:05Triage→03Medium a:03brennen [15:19:49] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T281146 (10thcipriani) p:05Triage→03Medium a:03dancy [15:23:03] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T281147 (10thcipriani) p:05Triage→03Medium a:03hashar [15:23:44] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T281148 (10thcipriani) p:05Triage→03Medium a:03mmodell [16:20:19] 10Phabricator: Spaces request for Technical-Program-Management - https://phabricator.wikimedia.org/T277107 (10MBinder_WMF) Thanks, @Aklapper . I have created some templates to assist with this. The final step is user habit! :) [16:25:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Wikidata, and 5 others: Create docker based CI job to build the wikidata-query-gui - https://phabricator.wikimedia.org/T210286 (10Lucas_Werkmeister_WMDE) Random note: merged build... [16:38:21] (03CR) 10Jforrester: "Thanks. :-)" [integration/docroot] - 10https://gerrit.wikimedia.org/r/664608 (owner: 10Jforrester) [16:44:15] (03PS2) 10Jforrester: Zuul: Add AnjaliKumari to the CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/682318 (owner: 10Anjali041) [16:46:35] (03CR) 10Jforrester: [C: 03+2] Zuul: Add AnjaliKumari to the CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/682318 (owner: 10Anjali041) [16:48:47] (03Merged) 10jenkins-bot: Zuul: Add AnjaliKumari to the CI allow list [integration/config] - 10https://gerrit.wikimedia.org/r/682318 (owner: 10Anjali041) [16:53:21] !log Zuul: Add AnjaliKumari to the CI allow list [16:53:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:12:43] 10Gerrit, 10Release-Engineering-Team (Doing): Gerrit Apache out of workers - https://phabricator.wikimedia.org/T277127 (10Dzahn) hard restarted apache after deploy [17:37:18] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog (Tracking): MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10Jdlrobson) We're looking to migrate these to Node.js. According to @Edtadros we should be... [17:55:20] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T278347 (10matmarex) [18:06:56] (03PS2) 10Umherirrender: Add Customization section to README [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/682272 (https://phabricator.wikimedia.org/T274016) [18:07:44] (03CR) 10Umherirrender: "> Patch Set 1:" (034 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/682272 (https://phabricator.wikimedia.org/T274016) (owner: 10Umherirrender) [18:18:40] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog (Tracking): MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10Jdforrester-WMF) This is a critical path to dropping stretch support from CI. The choice... [18:19:44] anyone know anything about "memory compaction" on CI / jenkins docker nodes? [18:39:17] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T281145 (10brennen) a:05brennen→03dancy Assigning to @dancy for coverage while I'm out on the 4th; will pick back up as primary after that. [18:52:45] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog (Tracking): MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10Jdlrobson) I think it's fine to stop running these in CI. I've taken a look at the 7 test... [19:51:36] 10MediaWiki-Codesniffer, 10User-DannyS712: Add a warning when MediaWikiServices is used instead of dependency injection - https://phabricator.wikimedia.org/T281190 (10DannyS712) [19:53:55] (03CR) 10DannyS712: "This change is ready for review." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/682713 (https://phabricator.wikimedia.org/T281190) (owner: 10DannyS712) [19:58:13] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog (Tracking): MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10hashar) [20:03:20] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Readers-Web-Backlog (Tracking): MinervaNeue legacy ruby browser tests need to be updated to work on buster/ruby2.5 - https://phabricator.wikimedia.org/T280901 (10hashar) That is because the job Docker image moved from Stretch to Buster and thus it upg... [20:06:37] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T278347 (10Majavah) [20:07:43] 10Gerrit, 10Release-Engineering-Team (Doing): Gerrit Apache out of workers - https://phabricator.wikimedia.org/T277127 (10hashar) 05Open→03Resolved Perfect, thank you @Dzahn and @Elukey ;) [21:07:15] 10Release-Engineering-Team (Doing): Add link to risky change template to deployment blockers tasks - https://phabricator.wikimedia.org/T263547 (10thcipriani) 05Open→03Resolved [21:15:55] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T278347 (10matmarex)