[03:51:45] Project mediawiki-core-phpmetrics-docker build #1555: 04FAILURE in 2 min 44 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-phpmetrics-docker/1555/ [05:40:48] works never stop pfff [05:41:49] 00:02:43.894 docker: Error response from daemon: No such container: 02d5a936cf6fb71cb71a4dfde8fb9613f29e664d9020ba6f45abff6569f1095b. [05:41:52] lewl [05:44:15] cause surely Fri 2023-03-10 03:51:44 UTC 1h 51min ago docker-system-prune-dangling.timer [05:44:37] and the build died at 03:51:44 [05:45:25] so either: there is a race condition with docker run asking a container and given immediately a checksum [05:45:38] or what not [05:46:17] maybe docker system prune just deletes everything blindly [05:51:21] Yippee, build fixed! [05:51:21] Project mediawiki-core-phpmetrics-docker build #1556: 09FIXED in 4 min 30 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-phpmetrics-docker/1556/ [07:00:02] 10Project-Admins, 10WMCH-Infrastructure, 10Wikimini.org: Create a #WMCH-Wikimini Tag - https://phabricator.wikimedia.org/T275536 (10valerio.bozzolan) p:05Triage→03Medium [07:10:02] (03Abandoned) 10Hashar: jjb: add coverage jobs for MediaWiki services [integration/config] - 10https://gerrit.wikimedia.org/r/884843 (https://phabricator.wikimedia.org/T321837) (owner: 10Hashar) [07:16:29] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10SRE, 10Patch-For-Review: git: detected dubious ownership in repository at '/srv/mediawiki-staging' - https://phabricator.wikimedia.org/T325128 (10hashar) On `deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud`: ` Notice: /Stage[ma... [07:29:03] 10Phabricator, 10good first task: Replace redirect links in Phabricator footer with direct links - https://phabricator.wikimedia.org/T331694 (10Aklapper) 05Open→03Stalled p:05Triage→03Lowest [07:29:14] 10Phabricator, 10good first task: Replace redirect links in Phabricator footer with direct links - https://phabricator.wikimedia.org/T331694 (10Aklapper) [07:29:18] 10Phabricator, 10Release-Engineering-Team, 10User-brennen: Remove unused(?) fixed_settings.yaml from Phabricator puppet config - https://phabricator.wikimedia.org/T331673 (10Aklapper) [10:24:43] (03PS1) 10Majavah: zuul: Remove some Renameuser dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/896311 (https://phabricator.wikimedia.org/T27482) [10:41:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scap, 10Patch-For-Review: l10nupdate is still using HHVM - https://phabricator.wikimedia.org/T205313 (10taavi) [10:41:24] 10Release-Engineering-Team (Radar), 10MediaWiki-extensions-LocalisationUpdate, 10Security: Reenable l10update in production - https://phabricator.wikimedia.org/T220481 (10taavi) 05Open→03Declined Per T158360. [10:43:03] (03CR) 10Samtar: [C: 03+1] zuul: Remove some Renameuser dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/896311 (https://phabricator.wikimedia.org/T27482) (owner: 10Majavah) [13:19:32] 10Phabricator, 10serviceops-collab: create aphlict2001 (Phabricator realtime notifications codfw) - https://phabricator.wikimedia.org/T322369 (10eoghan) I've tried deploying phabricator::config with an empty config to aphlict2001 and running scap after that, but scap needs some credentials in there. At a minim... [14:28:25] !log (deployment-prep) `[samtar@deployment-deploy03 ~]$ scap lock --all --verbose Debugging issue with extension maintenance script, T324233` [14:28:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:28:29] T324233: Create maintenance script to count/delete orphaned Phonos files - https://phabricator.wikimedia.org/T324233 [14:36:49] (03CR) 10Hashar: "RenameUser remains a dependency for older release branches isn't it? That would potentially breaks CI when backporting changes for those " [integration/config] - 10https://gerrit.wikimedia.org/r/896311 (https://phabricator.wikimedia.org/T27482) (owner: 10Majavah) [14:37:40] !log (deployment-prep) clear lock [14:37:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:39:29] (03CR) 10Krinkle: "Does quibble tolerate an empty extension repo? if so, we can keep it listed until the last LTS with Renameuser ext is gone." [integration/config] - 10https://gerrit.wikimedia.org/r/896311 (https://phabricator.wikimedia.org/T27482) (owner: 10Majavah) [15:03:01] Project beta-code-update-eqiad build #434344: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/434344/ [15:07:48] oh yeah [15:07:50] !log (deployment-prep) `[samtar@deployment-deploy03 ~]$ scap lock --all --verbose Debugging issue with extension maintenance script, T324233 [15:07:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:07:54] T324233: Create maintenance script to count/delete orphaned Phonos files - https://phabricator.wikimedia.org/T324233 [15:13:02] Project beta-code-update-eqiad build #434345: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/434345/ [15:15:32] Yippee, build fixed! [15:15:33] Project beta-code-update-eqiad build #434346: 09FIXED in 2 min 29 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/434346/ [15:24:59] Project beta-scap-sync-world build #93799: 15ABORTED in 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/93799/ [15:25:14] (sorry for the noise) [15:25:44] At least it’s meant to be broken TheresNoTime [15:34:18] 10Release-Engineering-Team (GitLab V: Event Horizon 🌄), 10Patch-For-Review: Reggie raises frequent "sqlite3.OperationalError: database is locked" under high load - https://phabricator.wikimedia.org/T330239 (10dancy) 05Open→03Resolved The newest code handles many concurrent image uploads with no problem. [15:41:49] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), and 2 others: `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/vendor - https://phabricator.wikimedia.org/T287419 (10cscott) I'm just going... [15:43:01] Project beta-code-update-eqiad build #434348: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/434348/ [15:50:27] dancy: are scap updates on deployment-prep automatic these days or do they happen manually? [15:51:21] You mean making and deploying a new release of scap? That require's an explicit step. [15:52:13] yeah, ok, thanks [15:52:28] Since you only removed code, i'm not inclined to do that until a functional change comes along [15:53:57] no worries, I was just curious if I get to test the puppet change removing l10nupdate yet or if that needs to wait [15:54:07] (I'm totally fine with waiting!) [15:54:12] oh, gotcha.. [15:54:12] Yippee, build fixed! [15:54:13] Project beta-code-update-eqiad build #434349: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/434349/ [15:54:42] I'll schedule a deployment on Monday. [16:08:39] (AAAAAH) okay so `MediaWiki\MediaWikiServices::getInstance()->getSiteStore()->getSites()`, ran on the beta cluster, is still returning `deploymentwiki` in its list, yet we got rid of that in T198673, right? [16:08:40] T198673: Remove deployment.wikimedia.beta.wmflabs.org wiki (deploymentwiki) - https://phabricator.wikimedia.org/T198673 [16:12:42] (ah its still in the `sites` table) [16:16:16] can I just delete that row...? [16:21:25] 10Beta-Cluster-Infrastructure: deploymentwiki site still present in sites table - https://phabricator.wikimedia.org/T331739 (10TheresNoTime) [16:21:59] TheresNoTime: probably [16:22:33] "probably" and "deleting rows" makes me nervous :p [16:24:36] TheresNoTime: I [16:24:42] Check betawikidata [16:25:00] If that uses any links to deploymentwiki [16:25:16] Does the database still exist for deploymentwiki TheresNoTime [16:26:05] Nope [16:26:32] TheresNoTime: then I’m more confident it’s fine [16:32:52] 10Beta-Cluster-Infrastructure: deploymentwiki site still present in sites table - https://phabricator.wikimedia.org/T331739 (10TheresNoTime) I have deleted the row in beta's `metawiki` database `lang=sql root@BETA[metawiki]> select site_id, site_global_key from sites where site_global_key = 'deploymentwiki'; +-... [16:34:50] oh for goodness sake.. the `sites` table contains more sites which no longer/don't exist (: [16:36:25] TheresNoTime: you should check other wikis too [16:37:40] !log `[samtar@deployment-deploy03 ~]$ scap lock --all --verbose Debugging issue with extension maintenance script, T324233` [16:37:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:37:44] T324233: Create maintenance script to count/delete orphaned Phonos files - https://phabricator.wikimedia.org/T324233 [16:40:15] TheresNoTime: question [16:40:29] Why not just foreachwiki if it runs on each wiki [16:40:36] Why do weird stuff with sites table [16:40:54] Or is there not a way to check the dblist [16:42:31] Not my script ^^' I think there was a concern about memory usage or something... [16:42:49] "we can't use `foreachwiki` because it needs to keep things in memory across each call" [16:42:57] TheresNoTime: no problem, my brain went to maybe I should wonder why we do this [16:43:09] Then realised it’s nearly 5pm on Friday [16:43:20] Because normally when I ask why the answer is just because [16:44:35] :p [16:44:43] TheresNoTime: that does not sound right, foreachwiki just runs mwscript on a loop so php memory is reset between each wiki [16:45:07] oh sorry, misread, 'needs' is the exact opposite of what I thought [16:58:20] 10Phabricator (Upstream), 10Patch-For-Review, 10Upstream: Phabricator "reconcile timezone" dialog options are confusing - https://phabricator.wikimedia.org/T158177 (10Aklapper) [16:58:22] 10Phabricator, 10Release-Engineering-Team, 10User-brennen: Remove unused(?) fixed_settings.yaml from Phabricator puppet config - https://phabricator.wikimedia.org/T331673 (10Aklapper) [17:21:49] 10GitLab (Administration, Settings & Policy), 10Release-Engineering-Team (Priority Backlog 📥), 10Privacy Engineering, 10Product-Analytics, and 2 others: Request for Private repos to be enabled - https://phabricator.wikimedia.org/T305082 (10mpopov) @thcipriani: Emil mentioned checking in with you on this in... [17:37:33] 10Phabricator, 10MediaWiki-Internationalization, 10MediaWiki-extensions-General, 10WMF-General-or-Unknown, and 3 others: Update footer links - https://phabricator.wikimedia.org/T331680 (10Dzahn) [17:55:28] 10Phabricator, 10good first task: Replace redirect links in Phabricator footer with direct links - https://phabricator.wikimedia.org/T331694 (10Aklapper) [18:22:20] 10Phabricator (Upstream), 10Upstream: Phabricator dashboard tab panel loads two panels' content - https://phabricator.wikimedia.org/T328200 (10Aklapper) I cannot reproduce this anymore on https://phabricator.wikimedia.org/ . Can anyone else? [18:26:54] 10Phabricator (Upstream), 10Upstream: Phabricator dashboard tab panel loads two panels' content - https://phabricator.wikimedia.org/T328200 (10Dzahn) No, I don't see "Good first newcomer task" link at all. (and separately have doubts that those lists are accurate) [18:27:44] 10Phabricator (Upstream), 10Upstream: Phabricator dashboard tab panel loads two panels' content - https://phabricator.wikimedia.org/T328200 (10Dzahn) Also if it's going to be fixed in Phorge per comment above then it's likely also going to be fixed for us in the future, even if it was still a bug right now. [18:30:50] 10Phabricator (Upstream), 10Upstream: Phabricator dashboard tab panel loads two panels' content - https://phabricator.wikimedia.org/T328200 (10Reedy) {F36905466 size=full} [19:57:14] 10Scap: helm env vars not set when running sudo -u mwpresync scap stage-train - https://phabricator.wikimedia.org/T331479 (10dancy) p:05Triage→03Medium a:03dancy [20:23:21] (03PS1) 10Krinkle: zuul: Add mediawiki-i18n-check job to intuition repos [integration/config] - 10https://gerrit.wikimedia.org/r/896427 [20:27:01] (03CR) 10Krinkle: [C: 03+2] zuul: Add mediawiki-i18n-check job to intuition repos [integration/config] - 10https://gerrit.wikimedia.org/r/896427 (owner: 10Krinkle) [20:28:44] (03Merged) 10jenkins-bot: zuul: Add mediawiki-i18n-check job to intuition repos [integration/config] - 10https://gerrit.wikimedia.org/r/896427 (owner: 10Krinkle) [21:07:08] (03PS1) 10Krinkle: zuul: Add CI jobs for tools/orphantalk.git [integration/config] - 10https://gerrit.wikimedia.org/r/896431 [21:07:21] (03CR) 10Krinkle: [C: 03+2] zuul: Add CI jobs for tools/orphantalk.git [integration/config] - 10https://gerrit.wikimedia.org/r/896431 (owner: 10Krinkle) [21:08:28] (03Merged) 10jenkins-bot: zuul: Add CI jobs for tools/orphantalk.git [integration/config] - 10https://gerrit.wikimedia.org/r/896431 (owner: 10Krinkle) [21:08:58] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/896431 [21:09:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:08:32] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.40.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T330204 (10thcipriani) 05Open→03Resolved