[06:44:08] godog: https://phabricator.wikimedia.org/T393874 is awesome, thx! (cc jhathaway, elukey) [07:04:41] Hello, we are working on deploying the dse-k8s-codfw cluster and had a question, do we mark the dse-k8s-ctrl.svc.codfw.wmnet reserved VIP as active before or after merging 1171592: dns: Add a VIP for dse-k8s-ctrl.svc.codfw.wmnet | https://gerrit.wikimedia.org/r/c/operations/dns/+/1171592 and 1171584: dse-k8s: deploy etcd service | https://gerrit.wikimedia.org/r/c/operations/puppet/+/1171584 [07:24:48] stevemunene: It doesn't matter [07:24:53] :) [07:25:32] stevemunene: long version, is that Netbox doesn't drive the .svc. IPs (yet), only one day with https://phabricator.wikimedia.org/T263429 [07:25:48] so it's still done in the DNS plain text files: https://github.com/wikimedia/operations-dns/blob/c1aebf8e287c72fcd946345f5ab55a4207a40e1c/templates/wmnet#L580 [07:27:13] In Netbox they're there to document the .svc. IPs, and to help ensure that nobody uses the other sites' twin IP (so if you use .2.73 in eqiad, you mark the .2.73 as reserved in codfw so nobody takes it for something else) [07:27:40] now that you're well into the deployment of that new cluster you can set the IP to active in Netbox so you don't forget to do it later on [07:40:11] XioNoX: yeah no problem! I'm glad it can be of help [07:45:35] Ack, thanks XioNoX :) [09:56:46] I think the image used for debci building (bullseye) is still affected by the bullseye-backports issue [09:56:50] https://www.irccloud.com/pastebin/QBUgW0UN/ [09:57:09] it should be auto-regenerated periodically? [10:44:47] that seems plausible, check with collab-services? [10:44:56] (I assume they own the underlying image, IYSWIM) [10:45:06] ok thanks [11:14:49] if there are issues, maybe it would be good to reopen https://phabricator.wikimedia.org/T383557, as we would like to write down as many things as possible regarding actions need to be taken in such cases [11:17:55] thanks effie, I'll drop a comment on that task [12:25:58] hey hey, we have a small issue related to the bullseye-backports deprecation, not sure how best to solve it, but it doesn't seem too difficult. [12:25:58] TL;DR (I think) is how do we force a rebuild of https://github.com/wikimedia/operations-docker-images-production-images/tree/master/images/python-build ? [12:25:58] As (I think) the current version still have the calls to the backports. and we need the updated version to run that workflow: https://github.com/wikimedia/operations-software-netbox-deploy/blob/master/Makefile#L13 [12:29:06] any idea who would be the proper person to contact? hasharAway seems away, maybe jnuche ? [13:01:03] XioNoX: I wrote in -sre-collab for a similar issue [13:02:24] fabfur: thx, I'm not in there, was there any answer? [13:03:01] XioNoX: I'm not really sure what's needed there. Are those images to be rebuilt from the config repo? maybe @James_F or @dancy have more insight, they seemed to be able to help here https://phabricator.wikimedia.org/T383557#11029224 [13:03:38] jnuche: yeah I pinged you are you're on our timezone :) [13:03:41] as* [13:03:45] We re-did a re-build of the CI images, yes. [13:04:08] But I don't think I have access to the SRE prod images. I really hope I don't have it. [13:04:34] James_F: any idea who owns Imageupdatebot ? [13:04:46] Err. Moritz maybe? Or Alex? [13:05:01] But it should Just Work™ with the weekly re-build? [13:05:28] James_F: that's my guess, but it's blocking a deployment we needed to do today [13:05:33] https://github.com/wikimedia/operations-docker-images-production-images/blob/master/images/python-build/bullseye/changelog -> the bot re-builds each Sunday. The one in two days' time will pick up the new base image. [13:05:42] Ah. Presumably it's a cron/timer somewhere? [13:06:07] https://gerrit.wikimedia.org/g/operations/puppet/+/e5b5103d7c0fb13fb1a0067bb42d25437ff66d3e/modules/profile/files/docker/update-production-images.sh [13:06:29] ah, _joe_ ^ ? :) [13:06:33] So any root(?) can "just" trigger the weekly-update.sh file manually? [13:06:42] your name is all over git blame :) [13:06:59] I mean if you want to rebuild a single image, you can just add a new changelog entry and build it "normally" [13:07:06] James_F: yeah, I'd prefer to not touch it as it's not my area of expertise [13:07:32] If it's just https://wikitech.wikimedia.org/wiki/Kubernetes/Images#Production_images that's required, I have done this before [13:10:09] taavi: Yeah, not sure what special rights/locations are needed though. [13:10:26] James_F: you want the python-build image rebuilt? I'll do so [13:10:47] Emperor: XioNoX does, don't drag me into this. ;-) [13:10:58] Emperor: yeah that's topranks and I that needed it [13:11:07] thx! [13:11:25] XioNoX: is there a change to the image that's been made already? [13:11:55] it's to pickup the latest latest of https://github.com/wikimedia/operations-docker-images-production-images/blob/master/images/python-build/bullseye/Dockerfile.template#L1 [13:12:00] Should just work with a changelog bump. It implicitly was updated as it's using :latest on the main bullseye image. [13:12:06] Yeah. [13:22:52] darn it, I can't get --select to work [13:24:07] Emperor: I'll do it, no worries [13:25:33] claime: ah, I'd just about got there [13:25:41] as you wish :) [13:25:50] I think the changelog needs a bump [13:25:51] XioNoX fabfur just a question, do you remember if there is a specific package you need from backports, just in case :) [13:26:22] effie: in theory no, it's just to build a python package [13:26:25] (as taavi and James_F already noticed, didn't backlog well enough) [13:26:27] grand [13:26:35] generate the wheels more exactly [13:26:49] I mainly used to install go1.19 but now I prefer to install golang manually so there's no need anymore for bullseye-backports (at least for me) [13:26:54] claime: oh, no, I spoke too soon, --select '*python3-build-*' still only tries to build the buster image, please go ahead and then tell me what --select run I should have used? Sorry :( [13:27:03] that leads me to another question but I can work on that later :) [13:27:48] Emperor: The select is fine, but we need to bump the changelog in git [13:28:57] claime: ah, or use --nightly (at least "build --nightly images --select '*python3-build-bullseye*'" in my local checkout looks to be doing the right thing now) ? [13:29:07] Yeah or that [13:29:12] Forgot that option [13:29:33] let me check something because I don't think it will update the changelog though [13:30:27] No, it looks not to have, just to have built 0.1.0-20250720-20250725 [13:32:01] (so yeah, bump changelog, merge, pull, rebuild will work, but I'll stop getting in the way now) [13:32:21] Emperor: CR incoming [13:32:29] https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/1172631 [13:32:38] 👀 [13:34:28] Merged, pulled, rebuild should work now [13:35:21] claime: I don't think your changelog entry is syntactically valid, which may cause issues [13:35:35] *grmbls* [13:36:08] yeah it's not building, fofc [13:36:09] ofc* [13:41:04] Fixed, building [13:43:38] Successfully published image docker-registry.discovery.wmnet/python3-build-bullseye:0.1.0-20250725 [13:44:22] nice, thx! [13:44:27] topranks: ^ [13:45:31] Emperor, claime: thank you! [16:27:37] inflatador: want me to merge dse-k8s: "Add dse-k8s-codfw k8s configuration" ? [16:28:20] andrewbogott by all mean [16:28:21] s [16:28:32] done [16:28:58] thanks!