[07:00:40] greetings [07:06:35] prepping webservice-cli for the memory request work in ~1h https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/merge_requests/107 [07:58:40] mmhh waiting for the bump-version pipeline to finish, it hung in the same spot on its previous run too https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/jobs/829846 [08:01:43] not sure what's the best action in this case tbh [08:15:00] waiting actually "worked" [08:15:44] 2026-05-20T07:54:20.389364Z 01O fakeroot debian/rules clean [08:15:44] 2026-05-20T08:07:14.892427Z 01O dh clean --with python3 --buildsystem=pybuild [08:16:10] now waiting some more for this [08:16:11] 2026-05-20T08:07:17.395999Z 01O fakeroot debian/rules binary [08:22:49] actually yes, webservice-cli deb jobs went from taking 9m to 26m [08:22:50] ... [08:23:02] 6d ago https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/jobs/822575 [08:23:11] 8h ago https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/jobs/829555 [08:23:22] I'll go ahead with the deploy [08:43:45] ok looks like the current limitrange in place at least for toolsbeta.test is 100Mi memory request thus I can't request 64Mi right away [08:44:18] I'll revert my change back to 128Mi instead [08:50:05] what is the correct/recommended way to rollback in this case? I have already pushed the revert and closed the bump version MR [08:53:40] of course now toolsbeta has webservice-cli 0.103.23 [08:54:55] ok so new bump_version mr I suppose, wait 26 minutes, re-launch the cookbook which will re-install "new" 0.103.23 ? [08:55:28] maybe there was a way to rollback to a prev version, but I don't remember it right now :D [08:55:47] if it's toolsbeta only, I think rolling forward is ok [08:56:55] ok thank you [09:00:15] will go with https://gitlab.wikimedia.org/repos/cloud/toolforge/webservice-cli/-/merge_requests/109 [09:09:00] and the task for slow ci T426827 [09:09:00] T426827: webservice-cli package deb gitlab CI job went from 9 minutes to 27 minutes - https://phabricator.wikimedia.org/T426827 [09:27:27] heh of course now aptly really doesn't like another .23 version [09:37:02] ok unfortunately I have to run to an appt, I'll resume fixing the mess I made after lunch [10:47:39] rebooting all the cloudlbs [11:02:23] when clouddumps100[12] are reboot, could you also use the opportunity to upgrade nfs-kernel-server (and nfs-common)? that got updated in the last Bullseye point release, but was never updated so far [11:08:55] does someone understand why prospector is not happy with how https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1289911 uses a decorator? [11:20:09] taavi: pylint shouldn't complains, it's says it doesn't see a function passed, that's weird. Why using wmflib's @retry and not the spicerack-specialized one? (it calls the other but has some spicerack-specific quirk) [11:22:05] huh. spicerack's @retry passes, at least locally?? [12:26:29] moritzm: sure [12:45:10] taavi: re: anycast-healthchecker it did ring a bell, https://phabricator.wikimedia.org/T314457 [12:45:16] so I'm +1 of course [12:46:48] and I think at the time that was a side quest maybe two or three layers deep [13:07:54] I'm out today, but I have a screen session rebooting cloudvirts which should finish with the last one (1076) in the next 20 minutes or so. [13:09:26] andrewbogott: ack, thank you [13:15:54] The other loose end I've left is https://phabricator.wikimedia.org/T425892 -- a project request that needs a postgresql db. That means either fulfilling the request, or getting them a trove-only project and then coaching them extensively about how to get their stuff built on toolforge. Just approving the request as-is is the easiest path for us but goes against the 'move everything to toolforge' goal. [13:16:21] I guess that can be in taavi's hands since he's on clinic duty. Sorry taavi :) [13:16:26] * andrewbogott really going now