[00:41:31] 06serviceops, 07Datacenter-Switchover: Verify our current wikikube capacity (in both DCs) can handle all our traffic - https://phabricator.wikimedia.org/T371273#10155654 (10Scott_French) Alright, following up on this: I'd propose that we this on Thursday of this week during the MediaWiki infrastructure (UTC l... [07:24:06] 06serviceops, 10CirrusSearch, 03Discovery-Search (Current work), 10MediaWiki-Platform-Team (Radar): PHP web requests running for multiple hours - https://phabricator.wikimedia.org/T374662#10155933 (10Joe) >>! In T374662#10154543, @Krinkle wrote: > When the excimer timeout exception is thrown, MediaWiki rep... [07:42:02] 06serviceops, 06SRE, 13Patch-For-Review: Migrate chartmuseum to Bookworm - https://phabricator.wikimedia.org/T331969#10155980 (10elukey) 05Open→03Resolved [07:44:39] folks rdb2010 is in codfw rack D5, scheduled for switch migration later on (T373104) [07:44:57] do we need to do anything special in advance of that to ensure it goes smoothly? [07:48:16] topranks: o/ from https://wikitech.wikimedia.org/wiki/Redis#Cluster_redis_misc it seems a simple replica, so in theory if the network downtime is not severe it should be fine [07:50:04] elukey: ah some good sleuthing, indeed yes sounds like the cluster should be able to tolerate a brief interruption to one of them :) [07:50:11] ty! [07:50:42] <3 [08:26:17] 06serviceops, 10LPL Essential, 10MinT, 10Community Wishlist (Translations), 10Community-Tech (Island Fox (Sept 9 - 20)): Caching service request for MinT - https://phabricator.wikimedia.org/T370755#10156131 (10santhosh) Hi @jijiki , any updates on this request? thanks. [08:32:34] 06serviceops, 10conftool: Requestctl sync writes unchanged objects - https://phabricator.wikimedia.org/T375059 (10Joe) 03NEW [08:33:12] 06serviceops, 10conftool: Requestctl sync writes unchanged objects - https://phabricator.wikimedia.org/T375059#10156153 (10Joe) [08:33:51] 06serviceops, 06collaboration-services, 06Infrastructure-Foundations, 10Puppet-Core, and 4 others: Migrate roles to puppet7 - https://phabricator.wikimedia.org/T349619#10156154 (10MoritzMuehlenhoff) [08:34:12] 06serviceops, 10conftool: Requestctl sync writes unchanged objects - https://phabricator.wikimedia.org/T375059#10156155 (10Joe) p:05Triage→03High Setting to high and not UBN because we don't have this version installed anywhere. [08:44:12] 06serviceops, 10conftool: Requestctl sync writes unchanged objects - https://phabricator.wikimedia.org/T375059#10156200 (10Joe) a:03Joe [10:36:27] 06serviceops, 10MW-on-K8s, 06Traffic: Some sites try and fail to serve favicon.ico - https://phabricator.wikimedia.org/T374997#10156467 (10Vgutierrez) a:05Vgutierrez→03None Provided URLs are currently handled by mw-web: ` vgutierrez@carrot:/tmp$ ./T374997.sh https://donate.wikimedia.org/favicon.ico < se... [13:57:21] 06serviceops, 13Patch-For-Review: Migrate poolcounter hosts to bookworm - https://phabricator.wikimedia.org/T332015#10157161 (10elukey) All poolcounter IPs for MediaWIki/Thumbor are now on Bookworm! Next steps: 1) Check for stale connections in all old VMs during the next couple of days, restart poolcounterd... [14:20:20] swfrench-wmf: sorry to pick on you, might you be able to assist with depool's for today's set of k8's hosts? [14:20:30] (network maintenance - https://phabricator.wikimedia.org/T373104) [14:21:26] topranks: sure, can do - still targeting 16:00? [14:21:40] (ah, nevermind - says so on the task) [14:21:53] yep, so just over an hour and a half from now [14:30:09] great, thanks! confirmed these are all in-service and indeed are k8s workers [14:31:20] I'll back-time into completing by 15:45 given the time it took yesterday (similar number of hosts) [14:50:39] 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org, 13Patch-For-Review: MVP: Privately serve wikitech via mwdebug1001 - https://phabricator.wikimedia.org/T371537#10157459 (10jijiki) >>! In T371537#10154673, @dancy wrote: > The updated Firefox add-on is available at https://addons.mozilla.org/en-US/fir... [15:35:18] topranks: k8s workers in D5/6 are depooled [16:22:41] starting to repool k8s workers after all-clear from t.opranks [16:35:06] done [16:38:43] 06serviceops, 10MW-on-K8s, 10wikitech.wikimedia.org, 13Patch-For-Review: MVP: Privately serve wikitech via mwdebug1001 - https://phabricator.wikimedia.org/T371537#10158128 (10dancy) https://chromewebstore.google.com/detail/wikimediadebug/binmakecefompkjggiklgjenddjoifbb?hl=en-US&pli=1 is showing versio... [16:42:15] 06serviceops, 10MW-on-K8s, 10Abstract Wikipedia team (25Q1 (Jul–Sep)): Some wikifunctions calls end up served by mw-web - https://phabricator.wikimedia.org/T374556#10158166 (10Sharvaniharan) p:05Triage→03Medium [16:43:40] 06serviceops, 10MW-on-K8s, 10Abstract Wikipedia team (25Q1 (Jul–Sep)): Some wikifunctions calls end up served by mw-web - https://phabricator.wikimedia.org/T374556#10158168 (10Sharvaniharan)