[02:26:30] 06serviceops, 10MW-Interfaces-Team (MWI-Sprint-17 (2025-08-26 to 2025-09-09)), 07OKR-Work: [SPIKE] Test plan for rest.php routes in REST gateway - https://phabricator.wikimedia.org/T400152#11146380 (10aaron) I think something "good enough" would be: * Use test2.wikipedia.org, and assume some certain content... [08:50:12] 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q1:rack/setup/install wikikube-ctrl2006 - https://phabricator.wikimedia.org/T400661#11146872 (10Clement_Goubert) >>! In T400661#11143881, @Jhancock.wm wrote: > @jasmine_ how do you feel about the server going in row D? doesn't look like we have one in that row.... [11:08:30] 06serviceops, 07sre-alert-triage: Alert in need of triage: KubernetesWorkerUnschedulable - https://phabricator.wikimedia.org/T400969#11147326 (10LSobanski) Just a heads up that the alert fired again, can it be silenced for another month? [14:10:25] 06serviceops, 13Patch-For-Review: Migrate production Shellbox services to PHP 8.3 - https://phabricator.wikimedia.org/T403284#11148084 (10Scott_French) After a bit of discussion, it seems wikipeg 6.0.0 primarily brings performance improvements to the base parser, which necessitate code regeneration (thus the m... [14:22:59] 06serviceops, 10CXServer, 06Language and Product Localization, 10Content-Transform-Team (Work In Progress), 07Essential-Work: Decommission cxserver endpoints from RESTBase - https://phabricator.wikimedia.org/T372753#11148165 (10MSantos) [14:26:54] 06serviceops, 10CXServer, 06Language and Product Localization, 10RESTBase Sunsetting, 07Essential-Work: Decommission cxserver endpoints from RESTBase - https://phabricator.wikimedia.org/T372753#11148175 (10MSantos) [14:28:03] 06serviceops, 10RESTBase, 10RESTBase Sunsetting, 06Traffic, and 2 others: Block external traffic to RESTBase /page/data-parsoid endpoint and investigate internal usage - https://phabricator.wikimedia.org/T393557#11148179 (10MSantos) >>! In T393557#10918979, @hnowlan wrote: > There is a larger chunk of work... [14:28:10] 06serviceops, 10RESTBase, 10RESTBase Sunsetting, 06Traffic, and 2 others: Block external traffic to RESTBase /page/data-parsoid endpoint and investigate internal usage - https://phabricator.wikimedia.org/T393557#11148180 (10MSantos) 05Open→03Resolved a:03MSantos [14:35:38] 06serviceops, 06collaboration-services, 10Prod-Kubernetes, 07Kubernetes: Fix alternatives entries in helm and kubernetes-client packages - https://phabricator.wikimedia.org/T387548#11148233 (10Gehel) [14:57:52] 06serviceops, 10envoy, 06SRE, 10Data-Platform-SRE (2025.08.16 - 2025.09.05): Apply Envoy updates to wcqs and wdqs hosts - https://phabricator.wikimedia.org/T403738 (10bking) 03NEW [16:15:32] 06serviceops, 06Data-Platform-SRE, 10Dumps-Generation, 10MW-on-K8s: Investigate memcache errors during wikidata and commons dumps runs - https://phabricator.wikimedia.org/T401425#11148769 (10Scott_French) This appears to be happening again as we speak ([[ https://logstash.wikimedia.org/goto/8d44cfb44f7ccfa... [17:03:23] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11149137 (10Krinkle) Nothing new in `channel:exception` or `channel:error` on "mwmaint" and "jobrunner" hosts in Beta Cluster Logstash after running PHP 8.3 for tw... [17:07:16] 06serviceops: Migrate production Shellbox services to PHP 8.3 - https://phabricator.wikimedia.org/T403284#11149151 (10Scott_French) We're now serving ~ 8% of traffic on PHP 8.3 in shellbox-syntaxhighlight in both DCs. No issues observed so far: * General service health (errors, latency) looks good ([[ https://gr... [17:08:11] 06serviceops, 10envoy, 06SRE: Envoy config updates from v1.26 - https://phabricator.wikimedia.org/T403101#11149158 (10RLazarus) [17:18:31] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11149197 (10Krinkle) [17:20:14] 06serviceops, 06MediaWiki-Platform-Team, 07Epic: Migrate Wikimedia production from PHP 8.1 to PHP 8.3 - https://phabricator.wikimedia.org/T360995#11149200 (10Krinkle) [17:32:46] 06serviceops, 06Data-Platform-SRE, 10Dumps-Generation, 10MW-on-K8s, 13Patch-For-Review: Investigate memcache errors during wikidata and commons dumps runs - https://phabricator.wikimedia.org/T401425#11149286 (10jijiki) In wikikube we run a `mw-mcrouter` daemonset to interact with the memcached cluster. T... [17:59:49] 06serviceops, 10envoy, 06SRE, 10Data-Platform-SRE (2025.08.16 - 2025.09.05): Apply Envoy updates to wcqs and wdqs hosts - https://phabricator.wikimedia.org/T403738#11149376 (10RKemper) Current status: [x] WCQS [x] WDQS main [x] WDQS scholarly [] WDQS public [18:06:29] 06serviceops, 10envoy, 06SRE, 10Data-Platform-SRE (2025.08.16 - 2025.09.05): Apply Envoy updates to wcqs and wdqs hosts - https://phabricator.wikimedia.org/T403738#11149401 (10RKemper) a:03RKemper [18:19:01] 06serviceops, 06Data-Platform-SRE, 10Dumps-Generation, 10MW-on-K8s, 13Patch-For-Review: Investigate memcache errors during wikidata and commons dumps runs - https://phabricator.wikimedia.org/T401425#11149464 (10Scott_French) Many thanks for investigating and posting a fix, @jijiki. While that's in progre... [18:55:43] 06serviceops, 06Content-Transform-Team: Migrate parsoidtest1001 to PHP 8.3 - https://phabricator.wikimedia.org/T403772 (10Scott_French) 03NEW [19:17:13] 06serviceops, 06Content-Transform-Team, 13Patch-For-Review: Migrate parsoidtest1001 to PHP 8.3 - https://phabricator.wikimedia.org/T403772#11149713 (10Scott_French) [19:26:12] 06serviceops, 13Patch-For-Review: Configure mw-next-routing for the PHP 8.3 migration - https://phabricator.wikimedia.org/T403655#11149747 (10Scott_French) [19:26:23] 06serviceops, 13Patch-For-Review: Configure mw-next-routing for the PHP 8.3 migration - https://phabricator.wikimedia.org/T403655#11149748 (10Scott_French) 05Open→03In progress p:05Triage→03Medium