[07:01:12] o/ [07:01:15] Going to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1338937 [07:01:19] I'll report bac [07:01:21] *back [07:10:55] gdnsd rightfully hates me gdnsd[2119199]: Name 'pki.discovery.wmnet.': resolver plugin 'geoip' rejected resource name 'disc-pki' [07:11:05] so I guess I need to remove that from the dns repo [07:11:43] so maybe the safe thing is to revert, run puppet on dns1004/4004, patch the dns repo, deploy and then retry [07:11:58] (continuing on #sre) [07:59:42] 06Traffic: Upgrade HAProxy to 3.2.24 on cp hosts - https://phabricator.wikimedia.org/T439667 (10Fabfur) 03NEW [08:01:49] 06Traffic: Upgrade HAProxy to 3.2.25 on cp hosts - https://phabricator.wikimedia.org/T439667#12377992 (10Fabfur) [09:54:31] 06Traffic, 10Liberica: Support nft enabled realservers using IPIP encapsulation - https://phabricator.wikimedia.org/T414666#12378534 (10Vgutierrez) implementation happening on T433601 [10:08:18] 06Traffic, 10Liberica, 10Prod-Kubernetes, 06ServiceOps, and 2 others: Migrate Wikikube k8s apiserver and services to IPIP - https://phabricator.wikimedia.org/T420436#12378620 (10Jelto) [12:27:55] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE: Track down memory leak with URL downloaders in Trixie - https://phabricator.wikimedia.org/T439712 (10MoritzMuehlenhoff) 03NEW [12:28:11] 06Traffic, 06Infrastructure-Foundations, 06SRE: Track down memory leak with URL downloaders in Trixie - https://phabricator.wikimedia.org/T439712#12379180 (10MoritzMuehlenhoff) p:05Triage→03Medium [12:34:46] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE: Remove the legacy CNAME endpoints for the URL downloaders - https://phabricator.wikimedia.org/T439713 (10MoritzMuehlenhoff) 03NEW [12:47:05] 06Traffic: Upgrade HAProxy to 3.2.25 on cp hosts - https://phabricator.wikimedia.org/T439667#12379315 (10dkertesz) 05Open→03In progress Changelog from: https://www.mail-archive.com/haproxy@formilux.org/msg47533.html > Amaury Denoyelle (1): > BUG/MAJOR: mux_quic: fix potential crash on RESET_STREAM rec... [13:06:13] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE, 13Patch-For-Review: Scaling urldownloaders by adding redundancy and load balancing - https://phabricator.wikimedia.org/T429175#12379456 (10MoritzMuehlenhoff) I created two followups for the open followup work: - https://phabricator.wikimedia.... [13:09:10] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE, 06tools-platform-team: Remove the legacy CNAME endpoints for the URL downloaders - https://phabricator.wikimedia.org/T439713#12379479 (10fnegri) [13:10:04] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE, 06tools-platform-team: Remove the legacy CNAME endpoints for the URL downloaders - https://phabricator.wikimedia.org/T439713#12379487 (10MoritzMuehlenhoff) [13:20:07] 06Traffic, 06Infrastructure-Foundations, 06ServiceOps, 06SRE, 13Patch-For-Review: Scaling urldownloaders by adding redundancy and load balancing - https://phabricator.wikimedia.org/T429175#12379542 (10ssingh) 05Open→03Resolved >>! In T429175#12379456, @MoritzMuehlenhoff wrote: > I created two fol... [13:22:36] 06Traffic: Migrated busted container image to Trixie - https://phabricator.wikimedia.org/T437442#12379554 (10dkertesz) @MoritzMuehlenhoff unless there's anything else that I may be missing, I think we can close this ticket; let me know what you think, thanks! [13:23:57] the good thing is that gdnsd does not replace the daemon if there is a failure so there isn't anything to worry about as such [13:24:07] (conf check failure that is) [13:24:47] 06Traffic, 06Collaboration-Services, 06Infrastructure-Foundations, 06ServiceOps, and 2 others: Remove the legacy CNAME endpoints for the URL downloaders - https://phabricator.wikimedia.org/T439713#12379559 (10Jelto) Adding #collaboration-services tag for wmf-navigator [13:28:43] 10netops, 06cloud-services-team, 10Horizon, 06Infrastructure-Foundations, and 3 others: Move cloudweb hosts to cloud racks? - https://phabricator.wikimedia.org/T411783#12379579 (10taavi) 05Open→03Declined Per the network sync meeting, we will be doing {T392478} instead. [14:05:12] 06Traffic, 06Infrastructure-Foundations, 06SRE, 07LDAP: Add a new LVS ldap-ro-next service - https://phabricator.wikimedia.org/T439730 (10MoritzMuehlenhoff) 03NEW [14:18:11] 06Traffic, 06Collaboration-Services, 06Infrastructure-Foundations, 06ServiceOps, and 2 others: Remove the legacy CNAME endpoints for the URL downloaders - https://phabricator.wikimedia.org/T439713#12379989 (10aputhin) Separating the Toolhub work onto its own subtask: T439732 [14:39:43] 06Traffic, 06MediaWiki-API-Platform-Team, 10Page Content Service: Purged fails to parse purge URLs for /api/rest_v1/page/mobile-html - https://phabricator.wikimedia.org/T439533#12380136 (10ArielGlenn) After checking in with Yiannis: the issue is how these urls are constructed in mobileapps, and the Content T... [14:47:11] 06Traffic, 06MediaWiki-API-Platform-Team, 10Page Content Service: Purged fails to parse purge URLs for /api/rest_v1/page/mobile-html - https://phabricator.wikimedia.org/T439533#12380172 (10Jgiannelos) The root cause is that PCS doesn't ensure that the URL is valid, just sends a purge event for the incoming U... [14:52:39] 06Traffic: Upgrade HAProxy to 3.2.25 on cp hosts - https://phabricator.wikimedia.org/T439667#12380193 (10dkertesz) [14:53:18] 06Traffic, 06Infrastructure-Foundations, 06SRE, 07LDAP: Add a new LVS ldap-ro-next service - https://phabricator.wikimedia.org/T439730#12380195 (10ssingh) Let us know how/if we can help :) [14:53:48] hello - basic question: if a WMF project wants to buy a new domain for a project, is there a process to follow? [14:54:19] hnowlan: brett can help you with that [15:14:30] 06Traffic: Migrated busted container image to Trixie - https://phabricator.wikimedia.org/T437442#12380362 (10MoritzMuehlenhoff) Looks all done, nice work! [15:18:00] 06Traffic: Migrated busted container image to Trixie - https://phabricator.wikimedia.org/T437442#12380385 (10dkertesz) 05In progress→03Resolved [16:06:47] hnowlan: there were attempts made long ago for that but they fell through. Reach out to legal to get a new domain registered, that's still their wheelhouse. The actual *management* has shifted hands a little [16:10:11] brett: ah okay, thanks! [21:12:03] 10netops, 06Infrastructure-Foundations: codfw: upgrade routers (2026) - https://phabricator.wikimedia.org/T417871#12382618 (10Papaul) I spoke with @ssingh and @BCornwall to see if we can schedule the core routers Junos upgrade next week Wednesday at 9am CT, @ssingh will get back in touche with me on the confir... [21:24:04] 06Traffic: Upgrade dnsdist to v1.9.16-0+deb13u1 - https://phabricator.wikimedia.org/T439034#12382691 (10CDobbins) [21:34:02] 06Traffic: Upgrade dnsdist to v1.9.16-0+deb13u1 - https://phabricator.wikimedia.org/T439034#12382753 (10CDobbins) [21:37:07] 06Traffic: Upgrade dnsdist to v1.9.16-0+deb13u1 - https://phabricator.wikimedia.org/T439034#12382767 (10CDobbins)