[00:03:31] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-docker-mobileapps02 on project deployment-prep - https://phabricator.wikimedia.org/T409979#11369010 (10bd808) https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/8a1d22ecbf504f8b4fd1c6fd89847e70bab009fa%5E%21/#F0 `... [00:04:14] !log Reboot deployment-docker-mobileapps02 (T409979) [00:04:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:04:17] T409979: No Puppet resources found on instance deployment-docker-mobileapps02 on project deployment-prep - https://phabricator.wikimedia.org/T409979 [00:07:01] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-docker-mobileapps02 on project deployment-prep - https://phabricator.wikimedia.org/T409979#11369035 (10bd808) The `2` in the max-file data needs to be quoted: ` Nov 13 00:04:27 deployment-docker-mobileapps02 dockerd[581]: unable to c... [00:08:37] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-docker-mobileapps02 on project deployment-prep - https://phabricator.wikimedia.org/T409979#11369040 (10bd808) https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/d0d61dc325831793a2234b586d4d53556b410675%5E%21/#F0 `... [00:09:23] 10Beta-Cluster-Infrastructure: No Puppet resources found on instance deployment-docker-mobileapps02 on project deployment-prep - https://phabricator.wikimedia.org/T409979#11369044 (10bd808) 05In progressβ†’03Resolved [00:12:52] 10Beta-Cluster-Infrastructure: Add global profile::docker::engine::settings config to rotate docker logs in deployment-prep - https://phabricator.wikimedia.org/T409985 (10bd808) 03NEW [00:15:42] 10Beta-Cluster-Infrastructure: Add global profile::docker::engine::settings config to rotate docker logs in deployment-prep - https://phabricator.wikimedia.org/T409985#11369065 (10bd808) 05Openβ†’03In progress p:05Triageβ†’03Medium a:03bd808 https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-pup... [00:19:15] 10Beta-Cluster-Infrastructure: Add global profile::docker::engine::settings config to rotate docker logs in deployment-prep - https://phabricator.wikimedia.org/T409985#11369071 (10bd808) `lang=shell-session bd808@mbp03:~/projects/wmf/cloud/instance-puppet/deployment-prep$ git grep profile::docker::engine::settin... [00:43:13] 10Beta-Cluster-Infrastructure: Add global profile::docker::engine::settings config to rotate docker logs in deployment-prep - https://phabricator.wikimedia.org/T409985#11369158 (10bd808) Drop the prefix from T409985#11369065: https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/16c7a72e8dfda519... [00:46:26] (03PS1) 10Pwangai: jjb: Add JS coverage collection to mwcore-codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204711 [00:47:57] (03CR) 10CI reject: [V:04-1] jjb: Add JS coverage collection to mwcore-codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204711 (owner: 10Pwangai) [00:54:15] (03PS2) 10Pwangai: jjb: Add JS coverage collection to mwcore-codehealth jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204711 [00:59:09] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Add global profile::docker::engine::settings config to rotate docker logs in deployment-prep - https://phabricator.wikimedia.org/T409985#11369196 (10bd808) 05In progressβ†’03Resolved Those orphan settings for deployment-eventgate-1 and deployment-eventga... [01:03:50] 10Beta-Cluster-Infrastructure: Delete orphaned Puppet ENC data for deployment-eventgate-1 & deployment-eventgate-2 - https://phabricator.wikimedia.org/T409989 (10bd808) 03NEW [07:04:27] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Request for WMDE Board Statistics bot account - https://phabricator.wikimedia.org/T409809#11369469 (10ArthurTaylor) @bd808 if you were going to build a Phabricator HTML scraper bot *after 7. November 2025* :) Thanks for the tip - I'll take a look! [07:38:39] 10GitLab, 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 10wikimedia.biterg.io: Bitergia crawler bot runs into errors collection GitLab data - https://phabricator.wikimedia.org/T409903#11369538 (10Izubiaurre) > How do you get the list of all repos you are crawling to begin with?... [08:02:39] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Request for WMDE Board Statistics bot account - https://phabricator.wikimedia.org/T409809#11369593 (10Peachey88) (Also if you think the phab/phorge APIs are lacking anywhere where other phorge users might also like to use them, don't hesitate to... [08:03:09] 10Continuous-Integration-Config, 10MediaWiki-libs-IPUtils, 06MediaWiki-Platform-Team, 10Projects-Cleanup, 05MW-1.39-notes: Archive wikimedia/ip-set library - https://phabricator.wikimedia.org/T401155#11369596 (10malberts) This seems to be causing a problem when installing jobrunner: https://phabricat... [08:15:41] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Request for WMDE Board Statistics bot account - https://phabricator.wikimedia.org/T409809#11369634 (10ArthurTaylor) @bd808 That works better now - thanks so much! [08:18:08] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Request for WMDE Board Statistics bot account - https://phabricator.wikimedia.org/T409809#11369642 (10ArthurTaylor) @Peachey88 yeah - looking at the docs it seems like there might be ways to get project / board information over conduit. I must h... [08:19:29] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎): Request for WMDE Board Statistics bot account - https://phabricator.wikimedia.org/T409809#11369645 (10ArthurTaylor) 05Openβ†’03Resolved [09:17:23] 09:10:47 [mediawiki-publish-83] received unexpected HTTP status: 500 Internal Server Error [09:17:26] subprocess.CalledProcessError: Command '['sudo', '/usr/local/bin/docker-pusher', '-q', 'docker-registry.discovery.wmnet/restricted/mediawiki-multiversion-debug:2025-11-13-090954-publish-83']' returned non-zero exit status 1. [09:17:33] Yay. So much for the train today. [09:18:14] jnuche, hashar: I think I'd appreciate your input ^ [09:18:54] sounds like an issue with the Docker registry :-( [09:19:22] you should raise it to #wikimedia-sre [09:19:25] :) [09:19:30] Interestingly, if I run "scap train" again, it thinks that group2 _is_ already on wmf.2, while https://versions.toolforge.org/ correctly disagrees [09:19:37] yeah, makes sense. thanks. [09:26:20] 10Phabricator (2025-10-28), 06Release-Engineering-Team (Radar), 06collaboration-services, 10VPS-project-Phabricator: 'Fulltext' searches fail on test Phab instance due to ElasticSearch default config (PhutilAggregateException: All Fulltext Search hosts fai... - https://phabricator.wikimedia.org/T403948#11369888 [09:33:41] 06Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T408272#11369913 (10Aklapper) `scap train` failed: ` 09:10:47 [mediawiki-publish-83] received unexpected HTTP status: 500 Internal Server Error [...]... [09:36:13] 10Diffusion, 10Phabricator, 06collaboration-services, 10Wikidata, and 3 others: WikibaseLexeme submodule is not available - https://phabricator.wikimedia.org/T409519#11369920 (10dena) [09:51:25] andre: has it been sorted out? [09:51:36] (I have finished my meeting=) [09:51:44] hashar, nope, no replies yet [09:51:59] aohh [09:52:19] cpoy paste your last comment to a standalone UBN task [09:52:30] and we can raise it I guess [09:52:45] alternatively, you can try again [09:52:53] maybe the registry had a transient error [09:53:22] ^ worth trying (also πŸ‘‹) [09:53:31] hi jnuche ! [09:53:55] looking at /home/aklapper/scap-image-build-and-push-log-20251113-fail the exception says: "app.ThreadedAppJoinException: One or more tasks raised an exception" [09:54:49] 10Diffusion, 14RelEng-Archive-FY201718-Q1, 10Cloud-Services: create conduit method for the creation of phabricator policy objects - https://phabricator.wikimedia.org/T135249#11370001 (10Aklapper) For the papertrail, the related commit is https://gitlab.wikimedia.org/repos/phabricator/extensions/-/commit/... [09:55:22] * hashar notes the build reinstall a bunch of debian packages from scratch ... :b [09:55:33] byt yteah that is the push that failed due to a 500 Internal Server Error :/ [09:56:12] hashar: I am tempted to try "scap train" again, but scap thinks that group2 is already on wmf.2 (which is not the case). I mean, I could tell it, and see what happens [09:56:27] +1 [09:56:55] my guess is scap think that because it reads the local /srv/mediawiki-staging/wikiversions.json [09:57:04] okay, running scap train again now, let's see [09:57:12] yeah, it should still try to perform the actual rollout [09:57:40] looking at the registry dashboard, there was a single (PUT, 500) https://grafana.wikimedia.org/d/StcefURWz/docker-registry?orgId=1&from=now-6h&to=now&timezone=utc&var-datasource=000000005&var-instance=$__all [09:58:33] (it is not like it receives a steady rate of PUT commands though, so maybe it is really broken) [09:59:30] and I don't know how https://versions.toolforge.org/ retrieves its data :/ [10:00:02] html-scraping Special:Version :D (jk) [10:02:37] you are not that far! [10:02:50] the neat PHP script is at https://gitlab.wikimedia.org/toolforge-repos/versions/-/blob/main/index.php?ref_type=heads [10:03:02] it retrieves group0 / group1 / group2 from https://noc.wikimedia.org/conf/#dblist [10:04:07] and the versions from https://noc.wikimedia.org/conf/wikiversions.json [10:04:11] Deployment success on second try. [10:04:12] (all at wmf.2 currently) [10:04:15] Yay. <3 [10:04:18] congratulations! [10:04:26] πŸŽ‰ [10:04:29] eh, a few more grey hairs [10:04:39] Thanks for being around, really. :) [10:04:42] :-b [10:05:22] I missed the original ping, it didn't pop up in my desktop... sorry about that... again [10:05:45] 06Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T408272#11370040 (10Aklapper) Deployment worked on second attempt trying to run `scap train`. >>! In T408272#11369913, @Aklapper wrote: > Interesting... [10:06:04] I am quite happy there are 3 of us in the same timezone :-] [10:06:11] np, day is still long. :P [10:06:41] I hear ya [10:06:47] alright, time to file followup tickets and watch Logstash [10:06:54] it is 11am and I already feel like I had a full day of work :) [10:09:29] should also cut a few hours today or tomorrow, yesterday was long :-/ [10:14:14] My flatmate's company wiki is also down but with a 502. Heh. [10:20:20] obviously the conjunction of celestial spheres is bad for wikis today [10:20:52] remember to throw salt over your shoulders everyone [10:23:37] Thanks, I'll share that advice [10:33:44] 10Scap: Initial group versions displayed by "scap train" can be wrong if previous scap attempt failed - https://phabricator.wikimedia.org/T410018 (10Aklapper) 03NEW p:05Triageβ†’03Low [11:00:28] 06Release-Engineering-Team (Doing 😎), 07Essential-Work, 05Release, 05Train Deployments: 1.46.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T408272#11370213 (10Aklapper) 05Openβ†’03Resolved Declaring victory. See you again next week, little locomotive. {F70171773} [11:26:27] (03update) 10aklapper: Merge upstream master from Phorge as of 2025-11-12 (rP8b70728e412d5b13a3d1372ac65b09545b57b713) [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/108 (https://phabricator.wikimedia.org/T409947) (owner: 10brennen) [11:27:21] (03update) 10aklapper: Merge upstream master from Arcanist as of 2025-11-12 (rARCa83f646561bff9a20411952bf37ef93790d77e54) [repos/phabricator/arcanist] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/arcanist/-/merge_requests/8 (https://phabricator.wikimedia.org/T409947) (owner: 10brennen) [11:36:48] 10Phabricator (Upstream), 07Upstream: Phabricator doesn't fetch/display a linked comment when too much activity has occurred in the task since it was made - https://phabricator.wikimedia.org/T409907#11370301 (10A_smart_kitten) [11:56:18] 10Phabricator (Upstream), 07CSS, 07Upstream: When a workboard has a long sidebar, scrolling down that sidebar scrolls down the entire page (and scrolls away the workboard columns) - https://phabricator.wikimedia.org/T409705#11370336 (10A_smart_kitten) [13:30:58] 10Scap, 07Security: Document correct way to deploy security patches - https://phabricator.wikimedia.org/T409855#11370703 (10Lucas_Werkmeister_WMDE) >>! In T409855#11368736, @sbassett wrote: > Just noting again that `deploy_security.py` isn't really officially sanctioned or maintained by anyone on the #security... [14:48:14] 10Continuous-Integration-Config, 10WMF-JobQueue, 07ci-test-error: mediawiki/services/jobrunner fails to publish coverage (phpunit-coverage-php81-publish) - https://phabricator.wikimedia.org/T410042 (10hashar) 03NEW [14:48:50] (03open) 10jnuche: make Scap operations resilient to transient failures in GitLab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1040 (https://phabricator.wikimedia.org/T358876) [14:52:22] (03update) 10jnuche: make Scap operations resilient to transient failures in GitLab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1040 (https://phabricator.wikimedia.org/T358876) [14:52:45] 10Continuous-Integration-Config, 10WMF-JobQueue, 07ci-test-error: mediawiki/services/jobrunner fails to publish coverage (phpunit-coverage-php81-publish) - https://phabricator.wikimedia.org/T410042#11370916 (10hashar) 05Openβ†’03Declined The repository has no tests written and hence no PHPUnit. The pro... [15:04:55] (03PS1) 10Hashar: Zuul: add quibble --requires-only to production extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1204895 (https://phabricator.wikimedia.org/T407797) [15:13:05] (03PS10) 10Hashar: Jobs running tests solely with required extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) [15:15:51] (03CR) 10Hashar: "I gave it a go from contint1002 using:" [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [15:30:58] 10GitLab (CI & Job Runners), 06Release-Engineering-Team, 07Essential-Work: Buildkit v0.26.0 released - https://phabricator.wikimedia.org/T410049 (10dancy) 03NEW [15:42:56] (03PS11) 10Hashar: Jobs running tests solely with required extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) [15:44:59] (03CR) 10Hashar: "PS11: I have fixed an issue that caused `--requires-only`Β to still run the selenium/phpunit jobs with dependencies processed from the layo" [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [15:59:06] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/ParserMigration] Drop comment, back in prod [integration/config] - 10https://gerrit.wikimedia.org/r/1204905 [15:59:06] (03PS1) 10Jforrester: Zuul: Update comment about supported PHP versions [integration/config] - 10https://gerrit.wikimedia.org/r/1204906 [15:59:06] (03PS1) 10Jforrester: jjb: Provide mediawiki-core-php83-phan [integration/config] - 10https://gerrit.wikimedia.org/r/1204907 [15:59:07] (03PS1) 10Jforrester: Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204908 [15:59:08] (03PS1) 10Jforrester: Zuul: Drop PHP 8.1 testing from REL1_45, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204909 [15:59:09] (03PS1) 10Jforrester: Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 [15:59:13] (03PS1) 10Jforrester: Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 [15:59:17] (03PS1) 10Jforrester: Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 [16:01:00] (03CR) 10CI reject: [V:04-1] Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204908 (owner: 10Jforrester) [16:01:01] (03CR) 10CI reject: [V:04-1] Zuul: Drop PHP 8.1 testing from REL1_45, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204909 (owner: 10Jforrester) [16:01:03] (03CR) 10CI reject: [V:04-1] Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 (owner: 10Jforrester) [16:01:23] (03CR) 10CI reject: [V:04-1] Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 (owner: 10Jforrester) [16:01:30] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:01:50] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:02:05] (03CR) 10CI reject: [V:04-1] Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 (owner: 10Jforrester) [16:04:24] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/ParserMigration] Drop comment, back in prod [integration/config] - 10https://gerrit.wikimedia.org/r/1204905 (owner: 10Jforrester) [16:04:59] (03CR) 10Jforrester: [C:03+2] Zuul: Update comment about supported PHP versions [integration/config] - 10https://gerrit.wikimedia.org/r/1204906 (owner: 10Jforrester) [16:05:28] (03CR) 10Jforrester: [C:03+2] jjb: Provide mediawiki-core-php83-phan [integration/config] - 10https://gerrit.wikimedia.org/r/1204907 (owner: 10Jforrester) [16:06:00] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ParserMigration] Drop comment, back in prod [integration/config] - 10https://gerrit.wikimedia.org/r/1204905 (owner: 10Jforrester) [16:06:24] (03Merged) 10jenkins-bot: Zuul: Update comment about supported PHP versions [integration/config] - 10https://gerrit.wikimedia.org/r/1204906 (owner: 10Jforrester) [16:06:55] (03Merged) 10jenkins-bot: jjb: Provide mediawiki-core-php83-phan [integration/config] - 10https://gerrit.wikimedia.org/r/1204907 (owner: 10Jforrester) [16:13:53] (03PS2) 10Jforrester: Zuul: Drop PHP 8.1 testing from REL1_45, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204909 [16:13:53] (03PS2) 10Jforrester: Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 [16:13:53] (03PS2) 10Jforrester: Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 [16:13:54] (03PS2) 10Jforrester: Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 [16:13:55] (03PS1) 10Jforrester: jjb: Provide quibble-with-gated-extensions-vendor-mysql-php83 [integration/config] - 10https://gerrit.wikimedia.org/r/1204917 [16:13:57] (03PS1) 10Jforrester: jjb: Provide api-testing-*-php83 [integration/config] - 10https://gerrit.wikimedia.org/r/1204918 [16:15:35] (03CR) 10CI reject: [V:04-1] Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 (owner: 10Jforrester) [16:15:42] (03CR) 10CI reject: [V:04-1] Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 (owner: 10Jforrester) [16:15:56] (03CR) 10CI reject: [V:04-1] Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 (owner: 10Jforrester) [16:16:08] (03CR) 10Jforrester: [C:03+2] jjb: Provide quibble-with-gated-extensions-vendor-mysql-php83 [integration/config] - 10https://gerrit.wikimedia.org/r/1204917 (owner: 10Jforrester) [16:16:25] 10Phabricator (2025-10-28), 06Release-Engineering-Team (Radar), 06collaboration-services, 10VPS-project-Phabricator: 'Fulltext' searches fail on test Phab instance due to ElasticSearch default config (PhutilAggregateException: All Fulltext Search hosts fa... - https://phabricator.wikimedia.org/T403948#11371383 [16:16:28] 10Phabricator (2025-10-28), 06Release-Engineering-Team (Radar), 06collaboration-services, 10VPS-project-Phabricator: 'Fulltext' searches fail on test Phab instance due to ElasticSearch default config (PhutilAggregateException: All Fulltext Search hosts fa... - https://phabricator.wikimedia.org/T403948#11371384 [16:17:53] (03Merged) 10jenkins-bot: jjb: Provide quibble-with-gated-extensions-vendor-mysql-php83 [integration/config] - 10https://gerrit.wikimedia.org/r/1204917 (owner: 10Jforrester) [16:18:16] (03PS3) 10Jforrester: Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 [16:18:16] (03PS3) 10Jforrester: Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 [16:18:16] (03PS3) 10Jforrester: Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 [16:19:46] (03CR) 10CI reject: [V:04-1] Zuul: Drop PHP 8.1 testing from MW ecosystem master, switch default to 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204910 (owner: 10Jforrester) [16:19:55] (03CR) 10CI reject: [V:04-1] Zuul: [wikipeg] Drop PHP 8.1 CI testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204911 (owner: 10Jforrester) [16:19:56] (03CR) 10CI reject: [V:04-1] Zuul: [oojs/ui] Drop PHP 8.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/1204912 (owner: 10Jforrester) [16:19:59] (03CR) 10Hashar: [C:03+2] "There are 44 projects failing so far. I will file some as sub tasks of T407797, some others are due to Resource Loader dependencies hard w" [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [16:21:26] (03Merged) 10jenkins-bot: Jobs running tests solely with required extensions [integration/config] - 10https://gerrit.wikimedia.org/r/1197340 (https://phabricator.wikimedia.org/T407797) (owner: 10Hashar) [16:21:32] 10Continuous-Integration-Config, 13Patch-For-Review: Create a CI job to enforce tests to pass with solely required extensions - https://phabricator.wikimedia.org/T407797#11371424 (10hashar) For the projects that are on WMF production, 44 projects failing so far. I have filed some as subtasks, some others are d... [16:22:54] hashar: Isn't life fun? [16:24:25] yeah [16:24:34] I don't know how to tackle that one :b [16:24:49] but I guess I have a good enough knowledge of the situation that I can start exposing it to others [16:24:51] with some guidelines [16:24:53] and questions [16:25:03] I'll work on that tomorrow and send smoe announce next week I guess [16:25:24] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:25:25] and will surely look at enforcing tests to pass with solely the extensions that are marked as required in extension.json [16:25:44] (wish me good luck with CommunityConfiguration which is all other the place) [16:26:35] ugh meant to go to train log triage, was going to be 5 min early, then wound up 15 min late, now no one there.... [16:29:02] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:29:28] Oh, it's the time of the month where thcipriani triages dozens of production issue tasks and Phab e-mails me about each one, yay. ;-) [16:29:36] (03PS1) 10Jforrester: jjb: Rename phan jobs for better greppability [integration/config] - 10https://gerrit.wikimedia.org/r/1204922 [16:29:36] (03PS1) 10Jforrester: Zuul: Switch over to new phan job names [integration/config] - 10https://gerrit.wikimedia.org/r/1204923 [16:29:36] (03PS1) 10Jforrester: jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 [16:31:41] (03CR) 10CI reject: [V:04-1] Zuul: Switch over to new phan job names [integration/config] - 10https://gerrit.wikimedia.org/r/1204923 (owner: 10Jforrester) [16:31:57] (03CR) 10CI reject: [V:04-1] jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 (owner: 10Jforrester) [16:32:11] James_F: https://we.phorge.it/book/phorge/article/mail_rules/ :P [16:32:34] thcipriani: Yes, but I actually *want* to know about disposals around all prod issues. :-) [16:32:39] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:32:45] (03update) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:33:17] James_F: thanks and sorry <3 [16:33:26] :-) [16:34:22] well hmm [16:34:57] I am wrapping up early, I need to bicycle to the harbor and then back to go bicycle in the countryyard (yes by night cause it is fun) [16:35:04] hashar: Fancy. [16:35:23] I'll see whether I start enforcing those `quibble --requires-only` run [16:35:50] or maybe come back to the chang eI have made to `zuul/parameters_functions.py`Β  to disable recursiveness of a dependencies on a per project basis [16:35:52] or [16:36:01] I can just win the lottery! [16:36:35] Or we could just enforce immediately and let maintainers fix it, right? [16:36:37] ;-) [16:36:49] you know [16:37:00] when we start writing a movie about our story [16:37:13] you would be the gentle vilain James :] [16:37:15] (03approved) 10jnuche: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) (owner: 10dancy) [16:37:19] always pushing forward! (which is great) [16:37:24] na seriously [16:37:31] <3 [16:37:35] https://phabricator.wikimedia.org/T407797#11371424 [16:37:45] PHP 8.4 voting coming soon. But first let's drop 8.1 [16:37:47] there are a few of them that are broken due to some oddities in our stack [16:38:02] eg Wikihiero has a resource loader module requiring VE [16:38:07] (03merge) 10dancy: backport.py: Revise backport vote rule 3b [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1039 (https://phabricator.wikimedia.org/T362987) [16:38:15] Yup. [16:38:16] and the reason for the failure is in core rather than in Wikihiero [16:38:24] so it feels unfair to break those maintainers [16:38:32] Bien sΓ»r. [16:38:45] (it is not like Wikihiero has any maintainer or anyone care about it beside us and a few hieroglyphs writers) [16:38:52] anyway [16:38:58] yeah 8.1 going away is GREAT NEWS [16:39:10] * hashar refrains from paraphrasing POTUS [16:39:20] (03open) 10dancy: Release 4.226.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1041 [16:39:37] 10Scap, 07Security: Document correct way to deploy security patches - https://phabricator.wikimedia.org/T409855#11371578 (10sbassett) >>! In T409855#11370703, @Lucas_Werkmeister_WMDE wrote: > So what //is// the officially sanctioned or maintained way to deploy security patches, then? I don't think there is on... [16:39:53] thx for phasing out 8.1 and moving us to 8.3! :party: [16:40:30] 06Release-Engineering-Team (Doing 😎), 10Scap, 07Essential-Work, 13Patch-For-Review: Proposal for refactoring of backport dependency handling - https://phabricator.wikimedia.org/T362987#11371590 (10dancy) Final rule 3b: * Master branch MW code dependencies MUST be included in all live branches (error if not... [16:41:47] (03merge) 10dancy: Release 4.226.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1041 [16:43:28] * hasharAway πŸš΄β€β™‚οΈ πŸͺΎ πŸŒƒ [16:44:25] 06Release-Engineering-Team (Doing 😎), 10Scap, 07Essential-Work, 13Patch-For-Review: Proposal for refactoring of backport dependency handling - https://phabricator.wikimedia.org/T362987#11371605 (10jnuche) >>! In T362987#11371590, @dancy wrote: > Final rule 3b: > * Master branch MW code dependencies MUST be... [16:45:35] 06Release-Engineering-Team (Doing 😎), 10Scap, 07Essential-Work, 13Patch-For-Review: Proposal for refactoring of backport dependency handling - https://phabricator.wikimedia.org/T362987#11371610 (10dancy) p:05Triageβ†’03Medium [16:46:23] hashar: lol [16:47:11] (03PS2) 10Jforrester: Zuul: Switch over to new phan job names [integration/config] - 10https://gerrit.wikimedia.org/r/1204923 [16:47:11] (03PS2) 10Jforrester: jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 [16:49:36] (03CR) 10CI reject: [V:04-1] jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 (owner: 10Jforrester) [16:50:17] 06Release-Engineering-Team (Doing 😎), 10Scap, 07Essential-Work, 13Patch-For-Review: Proposal for refactoring of backport dependency handling - https://phabricator.wikimedia.org/T362987#11371623 (10dancy) 05Openβ†’03Resolved [16:51:32] 06Release-Engineering-Team (Doing 😎), 10Scap, 07Essential-Work: scap not complaining about dependencies only partially deployed with the train - https://phabricator.wikimedia.org/T397931#11371628 (10dancy) >>! In T397931#11344804, @Lucas_Werkmeister_WMDE wrote: > Actually, that SpiderPig deployment is st... [16:52:21] (03update) 10dancy: scap remove-patch: support removing multiple patches [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1012 (https://phabricator.wikimedia.org/T404593) [16:53:44] (03merge) 10dancy: scap remove-patch: support removing multiple patches [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1012 (https://phabricator.wikimedia.org/T404593) [16:55:19] (03update) 10dancy: Release 4.227.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1042 [16:55:23] (03open) 10dancy: Release 4.227.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1042 [16:55:59] (03PS3) 10Jforrester: Zuul: Switch over to new phan job names [integration/config] - 10https://gerrit.wikimedia.org/r/1204923 [16:55:59] (03PS3) 10Jforrester: jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 [16:56:47] (03merge) 10dancy: Release 4.227.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1042 [17:06:02] 10Scap, 13Patch-For-Review: scap remove-patch: support removing multiple patches - https://phabricator.wikimedia.org/T404593#11371715 (10dancy) 05Openβ†’03Resolved a:03dancy Fix deployed in scap 4.227.0. [17:06:54] 10GitLab, 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 10wikimedia.biterg.io: Bitergia crawler bot runs into errors collection GitLab data - https://phabricator.wikimedia.org/T409903#11371726 (10Dzahn) @Izubiaurre I see! I think it might be best if we replace a manual list wit... [17:09:11] 10Scap, 07Security: Document correct way to deploy security patches - https://phabricator.wikimedia.org/T409855#11371744 (10Dzahn) > how do we best keep this group of people informed How about putting the manual method you described on the docs page? That would have worked and it seems that's all this ticke... [17:17:48] taavi: Sorry, I intentionally didn't merge the VE patch as the new failing rules were unexpected; we'll probably ship a new version of eslint-config-wikimedia once we understand what's going on. [17:28:59] (03update) 10jnuche: make Scap operations resilient to transient failures in GitLab [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1040 (https://phabricator.wikimedia.org/T358876) [17:33:06] (03open) 10dancy: cli.py: prompt_for_approval_or_exit: Add exit_code parameter [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1043 [17:33:11] (03update) 10dancy: cli.py: prompt_for_approval_or_exit: Add exit_code parameter [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1043 [17:35:31] (03merge) 10dancy: cli.py: prompt_for_approval_or_exit: Add exit_code parameter [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1043 [18:29:13] (03PS2) 10Jforrester: jjb: Rename phan jobs for better greppability [integration/config] - 10https://gerrit.wikimedia.org/r/1204922 [18:29:13] (03PS4) 10Jforrester: Zuul: Switch over to new phan job names [integration/config] - 10https://gerrit.wikimedia.org/r/1204923 [18:29:13] (03PS4) 10Jforrester: jjb: Drop old name for phan jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1204924 [18:29:24] (03PS4) 10Jforrester: Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204908 (https://phabricator.wikimedia.org/T360995) [18:29:28] (03CR) 10Jforrester: [C:03+2] Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204908 (https://phabricator.wikimedia.org/T360995) (owner: 10Jforrester) [18:30:33] 10GitLab, 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 10wikimedia.biterg.io: Bitergia crawler bot runs into errors collection GitLab data - https://phabricator.wikimedia.org/T409903#11372128 (10Izubiaurre) > @Izubiaurre I see! I think it might be best if we replace a manual l... [18:31:47] (03Merged) 10jenkins-bot: Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3 [integration/config] - 10https://gerrit.wikimedia.org/r/1204908 (https://phabricator.wikimedia.org/T360995) (owner: 10Jforrester) [18:32:05] !log Zuul: Switch wmf/* PHP testing from PHP 8.1 to PHP 8.3, for T360995 [18:32:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:32:10] T360995: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995 [21:42:39] 10GitLab, 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 10wikimedia.biterg.io: Bitergia crawler bot runs into errors collection GitLab data - https://phabricator.wikimedia.org/T409903#11372778 (10Dzahn) Oh, I see. Thank you for the extra details. Let me try to sync with people... [22:11:02] 10GitLab, 06Release-Engineering-Team (Priority Backlog πŸ“₯), 06collaboration-services, 10wikimedia.biterg.io: Bitergia crawler bot runs into errors collection GitLab data - https://phabricator.wikimedia.org/T409903#11372849 (10brennen) See {T385529} for some related background. [22:58:36] (03open) 10jforrester: release: Bump eslint-config-wikimedia to 0.32.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/99 [22:58:49] (03merge) 10jforrester: release: Bump eslint-config-wikimedia to 0.32.1 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/99