[01:07:17] 06serviceops, 06Content-Transform-Team, 10Wikifeeds, 06Wikipedia-Android-App-Backlog: Significant increase in wikifeeds latency and mobileapps error rate since 2025/11/13 - https://phabricator.wikimedia.org/T410296#11394696 (10Scott_French) Roughly 11 hours have passed since the revert to `2025-10-13-12243... [11:39:09] 06serviceops, 06Growth-Team, 10PageViewInfo, 06Content-Transform-Team (Work In Progress), and 3 others: Determine the source of internal requests going through the API gateway. - https://phabricator.wikimedia.org/T410198#11395336 (10OWresch-WMF) [11:40:23] 06serviceops, 06MW-Interfaces-Team, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work: api-gateway chart: support secondary rate limit - https://phabricator.wikimedia.org/T408130#11395348 (10OWresch-WMF) [11:40:31] 06serviceops, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work, 13Patch-For-Review: api rate limiting: Create a helper service for exposing per-user (per-bucket) metrics - https://phabricator.wikimedia.org/T407999#11395347 (10OWresch-WMF) [11:40:45] 06serviceops, 06MW-Interfaces-Team, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work, 13Patch-For-Review: api-gateway chart: support rate limits for multiple time units - https://phabricator.wikimedia.org/T408132#11395349 (10OWresch-WMF) [11:41:05] 06serviceops, 06Traffic, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work: api-gateway helm chart: rest routes should return retry-after when a rate limit applies. - https://phabricator.wikimedia.org/T405636#11395355 (10OWresch-WMF) [11:43:24] 06serviceops, 10MediaWiki-extensions-OAuth, 06MediaWiki-Platform-Team, 07OKR-Work, and 2 others: Migrate OAuth extension back from wikimedia/oauth2-server fork to upstream - https://phabricator.wikimedia.org/T261462#11395386 (10OWresch-WMF) [11:44:21] 06serviceops, 07Epic, 06MediaWiki-Platform-Team (Kanban Board): Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11395395 (10OWresch-WMF) [11:44:55] 06serviceops, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work, 13Patch-For-Review: api rate limiting: Assign ratelimit class based on IP range - https://phabricator.wikimedia.org/T410273#11395408 (10OWresch-WMF) [11:47:06] 06serviceops, 10MediaWiki-extensions-OAuth, 06MediaWiki-Platform-Team (Kanban Board), 07OKR-Work, and 2 others: Migrate OAuth extension back from wikimedia/oauth2-server fork to upstream - https://phabricator.wikimedia.org/T261462#11395464 (10OWresch-WMF) [16:02:52] 06serviceops, 06Content-Transform-Team, 10Wikifeeds, 06Wikipedia-Android-App-Backlog: Significant increase in wikifeeds latency and mobileapps error rate since 2025/11/13 - https://phabricator.wikimedia.org/T410296#11396503 (10Dbrant) Yep, the upgrade from 18 to 22 definitely looks like the culprit. I'm gu... [16:28:57] 06serviceops, 06Connection-Team: MediaWiki periodic job campaignevents-aggregateanswers-metawiki failed - https://phabricator.wikimedia.org/T410748#11396609 (10Daimona) Hmmmm I can see that the job skipped the last execution at 15:00Z, but I don't know where to run the command above, and logstash has a lot of... [17:54:13] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: eqiad row C/D Service Ops host migrations - https://phabricator.wikimedia.org/T405950#11396798 (10RobH) Can we also set a date/time for moving the other three hosts remaining? wikikube-ctrl1003 kafka-main1008 kafka-main1009 [18:17:10] 06serviceops, 07Software-Licensing: repos/sre/sophroid is missing a software license - https://phabricator.wikimedia.org/T410767 (10taavi) 03NEW [19:39:50] 06serviceops, 13Patch-For-Review, 07Software-Licensing: repos/sre/sophroid is missing a software license - https://phabricator.wikimedia.org/T410767#11397173 (10RLazarus) a:03RLazarus Thanks @taavi. [23:52:11] 06serviceops, 13Patch-For-Review, 07Software-Licensing: repos/sre/sophroid is missing a software license - https://phabricator.wikimedia.org/T410767#11397982 (10RLazarus) 05Open→03Resolved