[12:19:00] <wikibugs>	 06serviceops, 06Commons, 10Shellbox, 10TimedMediaHandler-Transcode, and 2 others: Videos intermittently failing to transcode with error "Exception: Shellbox server returned status code 503" - https://phabricator.wikimedia.org/T385365#10516498 (10jijiki) p:05Triage→03Medium While I have not gone back on...
[12:40:44] <wikibugs>	 06serviceops: Mercurius does not retry failed transcodes beyond 15m - https://phabricator.wikimedia.org/T385225#10516552 (10jijiki)
[12:40:46] <wikibugs>	 06serviceops, 06Commons, 10Shellbox, 10TimedMediaHandler-Transcode, and 2 others: Videos intermittently failing to transcode with error "Exception: Shellbox server returned status code 503" - https://phabricator.wikimedia.org/T385365#10516551 (10jijiki)
[12:41:03] <wikibugs>	 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Add mwcron functionality to mediawiki chart - https://phabricator.wikimedia.org/T377961#10516553 (10Clement_Goubert) 05In progress→03Resolved
[12:41:13] <wikibugs>	 06serviceops, 10MW-on-K8s, 07Datacenter-Switchover, 13Patch-For-Review: Control mw-on-k8s periodic maintenance jobs with an etcd value - https://phabricator.wikimedia.org/T367118#10516555 (10Clement_Goubert) 05In progress→03Resolved
[13:10:57] <wikibugs>	 06serviceops, 10Thumbor: Add file size metric to Thumbor metrics - https://phabricator.wikimedia.org/T385292#10516764 (10jijiki) p:05Triage→03Low
[13:24:03] <wikibugs>	 06serviceops, 10MediaWiki-Platform-Team (Radar): Please review MediaWiki Apache config changes adding new docroot for the auth domain - https://phabricator.wikimedia.org/T385228#10516855 (10jijiki) p:05Triage→03Medium
[13:25:41] <wikibugs>	 06serviceops, 07Datacenter-Switchover: SRE comms for March 2025 Datacentre switchover - https://phabricator.wikimedia.org/T385157#10516873 (10jijiki) p:05Triage→03Medium
[13:25:44] <wikibugs>	 06serviceops, 07Datacenter-Switchover: 🧭 Northward Datacentre Switchover (March 2025) - https://phabricator.wikimedia.org/T385155#10516874 (10jijiki) p:05Triage→03Medium
[13:28:01] <wikibugs>	 06serviceops, 10MW-on-K8s, 06SRE: mwgrep cannot be used from a deployment host - https://phabricator.wikimedia.org/T384764#10516923 (10jijiki) p:05Triage→03Medium
[13:30:52] <wikibugs>	 06serviceops, 10MW-on-K8s, 10TimedMediaHandler, 13Patch-For-Review, 07Video: Clean up remaining videoscaling and jobrunner components - https://phabricator.wikimedia.org/T383317#10516961 (10jijiki) p:05Triage→03Low
[13:33:32] <wikibugs>	 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 07Kubernetes: Update wikikube-staging-codfw to kubernetes 1.31 - https://phabricator.wikimedia.org/T384450#10517003 (10jijiki) p:05Triage→03Medium
[13:34:37] <wikibugs>	 06serviceops: Improve / extend prometheus metrics exported by mercurius - https://phabricator.wikimedia.org/T383641#10517017 (10jijiki) p:05Triage→03Medium
[14:55:31] <wikibugs>	 06serviceops: mc-gp100[4-6] implementation tracking - https://phabricator.wikimedia.org/T377033#10517425 (10jijiki) 05Open→03Resolved
[14:55:42] <wikibugs>	 06serviceops: mc-gp200[4-6] implementation tracking - https://phabricator.wikimedia.org/T376969#10517431 (10jijiki) 05Open→03Resolved Resolved with T377033
[14:56:58] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org, 07User-notice: Communication for Wikitech/Wikimedia Developer Account migration - https://phabricator.wikimedia.org/T373615#10517436 (10jijiki) 05Open→03Resolved a:03jijiki I think this task has served its purpose, closing
[14:58:00] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: Cleanup: Wikitech code leftovers - https://phabricator.wikimedia.org/T371378#10517446 (10jijiki)
[14:58:10] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: ☂ Migrate Wikitech to Kubernetes - https://phabricator.wikimedia.org/T292707#10517447 (10jijiki)
[14:58:29] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: Cleanup: Wikitech code leftovers - https://phabricator.wikimedia.org/T371378#10517461 (10jijiki)
[14:58:40] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: ☂ Migrate Wikitech to Kubernetes - https://phabricator.wikimedia.org/T292707#10517462 (10jijiki)
[15:00:26] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: ☂ Migrate Wikitech to Kubernetes - https://phabricator.wikimedia.org/T292707#10517468 (10jijiki)
[15:02:01] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: ☂ Migrate Wikitech to Kubernetes - https://phabricator.wikimedia.org/T292707#10517475 (10jijiki) 05In progress→03Resolved I think we can close this, kudos to everyone who worked on making this happen!
[15:36:15] <wikibugs>	 06serviceops, 10MW-on-K8s, 13Patch-For-Review: Turn up mwcron releases - https://phabricator.wikimedia.org/T377962#10517628 (10Clement_Goubert) 05In progress→03Resolved ` $ kubectl logs mediawiki-main-test-version-28976610-55kv9 mediawiki-main-app  testcommonswiki:  MediaWiki version: 1.44.0-wmf.14 (...
[15:52:04] <wikibugs>	 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org: Cleanup: Wikitech code leftovers - https://phabricator.wikimedia.org/T371378#10517702 (10Aklapper)
[19:15:14] <wikibugs>	 06serviceops, 13Patch-For-Review: MediaWiki on PHP 8.1 production traffic ramp-up - https://phabricator.wikimedia.org/T383845#10518729 (10Scott_French) About 40 minutes into the initial pilot of 1% of mw-api-int traffic on PHP 8.1, no issues encountered so far. Service health (errors, latency) looks good and /...
[22:19:21] <wikibugs>	 06serviceops, 06Commons, 10Shellbox, 10TimedMediaHandler-Transcode, and 2 others: Videos intermittently failing to transcode with error "Exception: Shellbox server returned status code 503" - https://phabricator.wikimedia.org/T385365#10519214 (10A_smart_kitten) Noting {T376914} which has a similar error de...
[22:58:41] <wikibugs>	 06serviceops, 10Citoid, 06Editing QA, 06Editing-team, and 3 others: Switchover plan from restbase to api gateway for Citoid - https://phabricator.wikimedia.org/T361576#10519300 (10Ryasmeen) >>! In T361576#10515182, @Mvolz wrote: > This is now available to test on test Wikipedia: >  > https://test.wikipedia...
[23:23:34] <wikibugs>	 06serviceops, 06Infrastructure-Foundations: Image publishing via docker-pkg on build2001 repeatedly failing - https://phabricator.wikimedia.org/T385531 (10Scott_French) 03NEW
[23:25:18] <wikibugs>	 06serviceops, 13Patch-For-Review: Mercurius does not retry failed transcodes beyond 15m - https://phabricator.wikimedia.org/T385225#10519343 (10Scott_French) I was hopeful that I would get the fix for this live today, however that appears to be blocked on T385531, which is blocking php8.1 image builds from bei...