[05:12:12] Switching s6 codfw master [06:08:08] Switching es7 primary [07:50:08] marostegui: can I start updating s1 in eqiad? [07:53:29] yes [07:59:30] starting, thanks [09:14:01] morning folks, ms-fe2015 has been misbehaving since 2025-06-06, I've filled T396573 but given that Emp.eror is out this week could somebody else take a look? [09:14:01] T396573: ms-fe2015 is suffering intermittent errors on port 80 - https://phabricator.wikimedia.org/T396573 [09:45:56] marostegui: db2230 is in read-only moed [09:47:31] federico3: use its master, as that's a slave [09:47:41] In any case you can set it yo off, as it's a test host [09:47:52] ah do we still have both? ok [09:48:21] there you go db1176 [09:48:28] Yeah [09:48:43] can I create the user / db there? [09:53:03] vgutierrez: Matthew is out until next week [09:53:25] Amir1: yeah, that's why I'm asking here :) [09:53:43] let me see if I can find something obvious on the host [10:01:39] marostegui: I'm confused, db2230 is part of m3 and db1176 is just the DC master [10:02:30] it's just a testing part, don't worry, you can do db1176 [10:02:35] marostegui: let's just create this stuff on db1215 using the dedicated zarcillo_preprod db, we know the netmask syntax works [10:02:46] sobanski: any ETA on when we can get back the m3 test part? [10:03:02] federico3: ok up to you [10:03:16] thanks [10:06:40] vgutierrez: just to double check, is "depool" on the host the right way to depool them from lvs? See the last command in https://wikitech.wikimedia.org/wiki/Service_restarts#Swift [10:07:13] yes, that works [10:07:33] (I want to use confctl but not sure it would work with new load balancer) [10:07:36] thanks! [10:07:36] you could also use confctl from a puppetserver [10:07:59] swift is still using pybal as a load balancer BTW [10:08:22] but with both pybal or liberica etcd is still the source of truth for realserver state and everything should work in the same way [10:11:55] ah thanks! [10:14:45] hey folks, I have a thanos-swift-capacity question for anybody that feels up for it: https://phabricator.wikimedia.org/T396584 [10:14:59] I know that Empe*ror is out, so in case it is fine to wait [10:37:08] marostegui: so.banski is ooo. perhaps ask Mutan.te later? [10:49:44] kwakuofori: I'll ping them in the task [10:49:47] Thanks [11:23:43] marostegui: I have something working using db1215 as storage: https://zarcillo.wikimedia.org/kernel-updates/6.1.140 [11:26:29] it also outputs the checklist at https://zarcillo.wikimedia.org/api/v1/show_kernel_updates/6.1.140 [12:52:34] a small funny diff between MariaDB 10.6 and 10.11, courtesy of a wiki replicas user whose query started to fail: https://phabricator.wikimedia.org/T394372#10904329 [14:28:50] marostegui: thanks for your response to the alert-routing question! one follow-up when you get a chance: when triaging tasks, would it be an acceptable constraint if you needed to _keep_ #data-persistence on it when also adding #DBA? [14:28:50] the context is that if the "original" project is removed, a new task will be opened (similar to if you re-title the task) [14:31:59] speaking of which, I think it could make sense to have a MariaDB-specific tag in alertmanager and IRC alerts so we can set a highlight/ding on it [14:48:26] * swfrench-wmf wonders if this might be possible to do in the AM receiver vs. annotating alert summary strings in operations/alerts [14:57:50] marostegui: can I start the schema change on s6 codfw previous master? https://phabricator.wikimedia.org/T391056#10902652 [19:39:14] federico3: yes