[08:00:36] Cteam: welcome to today 🦄! Don’t forget to post your update in thread. [08:00:36] Feel free to include: [08:00:36] 1. 🕫 Anything you'd like to share about your work [08:00:36] 2. ☏ Anything you'd like to get help with [08:00:36] 3. ⚠ Anything you're currently blocked on [08:00:36] (this message is from a toolforge job under the admin project) [11:41:19] Done: [11:41:21] * fixed PAWS and updated docs [11:41:23] Working on: [11:41:25] * T396724 trove disk full issue [11:41:27] * clinic duties [11:41:29] ** T399746 object storage quotas not working as expected [11:41:31] * reviewing toolforge UI designs [17:53:34] Done: [17:53:34] * [toolforge-ui] did a 'live' session with sarai in barcelona, focused on defining the data that is needed from the API to show the UI flows that we are designing [17:53:34] * [toolforge-cd] fixed a bug in the CI setting for push-to-deploy, and added the option to specify the name of your tool (useful if it does not match the repo name), waiting for user feedback [17:53:34] Doing: [17:53:34] * [jobs-api] started reviewing a user contribution (yay!) adding a TcpHealthCheck to the jobs-api, and allowing eventually to specify udp ports for jobs [17:53:34] * [paws] some troubleshooting of the latest issues, switched the web proxies to different nodes, so if one fails, we can try to pin-point which service is causing it (kinda) and also one node being down won't bring grafana/prometheus down (so we can debug better) [17:53:34] * [ceph] some troubleshooting also of the current issues, mostly just checking stuff, found nothing interesting [17:53:34] Blocker: [17:53:34] * [jobs-api] Tested the MR repos/cloud/toolforge/jobs-api!182, to allow doing the job diff at the model level, Raymond_Ndibe please test and merge into yours if you don't find any issues [19:05:48] Today/Weekend: [19:05:48] * Cleaned up many unused pre-g4 instance flavors in codfw1dev and eqiad1 [19:05:48] * Did a total rebuild (including OSDs) of cloudcephos1006 to see if that finally makes ceph happy [19:05:48] * Moved a few OSD nodes in codfw1dev back to bullseye so we can test the upgrade process anew [19:05:48] * Reset many puppet certs in project-proxy; built a new acme-chief node; cleared various alerts in that project