[08:00:27] Cteam: welcome to today 🦄! Don’t forget to post your update in thread. [08:00:27] Feel free to include: [08:00:27] 1. 🕫 Anything you'd like to share about your work [08:00:27] 2. ☏ Anything you'd like to get help with [08:00:27] 3. ⚠ Anything you're currently blocked on [08:00:27] (this message is from a toolforge job under the admin project) [08:40:24] today: [08:40:24] * will work on T346441 in preparation for T346439 [08:40:25] T346439: cloudlb: eqiad1: change openstack.eqiad1.wikimediacloud.org endpoint to point to cloudlb - https://phabricator.wikimedia.org/T346439 [08:40:25] T346441: openstack eqiad1: verify new control plane works with cloudlb - https://phabricator.wikimedia.org/T346441 [09:04:05] Done: [09:04:05] * Deployed the maintain-kubeusers changes to increase secrets quota (T339916) [09:04:05] Doing: [09:04:05] * Still some reboots of k8s workers due to the NFS hiccup a couple days ago (T346123) [09:04:05] * Patching our openstack cli to have the fixes for using clouds.yaml and overriding the project in the file with cli flags [09:04:05] * Once that's in, I can continue with ceph cookbooks to drain nodes (T346427) [09:04:05] Blockers: [09:04:05] T339916: [envvars-api] Review and potentially increase the quota for secrets - https://phabricator.wikimedia.org/T339916 [09:04:06] T346123: [Ceph] [alert] toolschecker: NFS read/writeable on Toolforge - https://phabricator.wikimedia.org/T346123 [09:04:06] * The builds-builder patches are getting old, reviews are highly appreciated there, ping me if you want any help https://gitlab.wikimedia.org/groups/repos/cloud/-/merge_requests?scope=all&state=opened&label_name[]=Needs%20review [09:04:06] T346427: [wmcs-cookbooks] changes to openstack cli / auth things broke several cookbooks - https://phabricator.wikimedia.org/T346427 [11:07:07] Done: [11:07:17] * restarted a stuck tool (T346126) [11:07:17] T346126: Some of my tools (eg wikidata-todo) just start throwing 504 errors - https://phabricator.wikimedia.org/T346126 [11:07:40] * created a patch to improve the parsing of cookbook logs (https://gerrit.wikimedia.org/r/c/labs/tools/stashbot/+/958439) [11:08:26] Working on: [11:08:33] * T345810 [11:08:33] T345810: [openstack] Upgrade codfw hosts to bookworm - https://phabricator.wikimedia.org/T345810 [11:08:48] * T344717 [11:08:49] T344717: [toolsdb] test creating a new replica host - https://phabricator.wikimedia.org/T344717 [13:32:09] Today: [13:32:09] * Probably cleaning up DNS chaos like every day [13:32:09] • Hopefully spending some time on my much-delayed k8s training course