[07:13:43] greetings [07:13:47] o/ [07:47:26] /topic ☁️ https://etherpad.wikimedia.org/p/WMCS-2025-09-04 | Channel is logged at https://wm-bot.wmcloud.org/logs/%23wikimedia-cloud-admin/ | ping cteam | clinic duty: godog [07:47:35] I would do that ^ though I can't get op here [07:48:12] can someone grant me op please or get op and run the above ? [08:13:23] hmm I think that's now managed via https://gitlab.wikimedia.org/toolforge-repos/ircservserv-config/ [08:13:25] one second [08:19:49] ack, thank you [08:25:10] !issync [08:25:11] Syncing #wikimedia-cloud-admin (requested by Majavah) [08:25:13] Set /cs flags #wikimedia-cloud-admin godog +Afiortv [08:25:54] neato [08:42:10] morning [12:05:35] https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/jobs/599913 has someone manually resized the volume? [12:20:17] not me :/ [12:20:53] https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/72 [12:21:15] +1d [12:21:53] and now I get to deal with gitlab's wonderful "You should run a new pipeline, because the target branch has changed for this merge request." [12:22:05] taavi: as a gerritlab user, you might have opinions on https://github.com/yaoyuannnn/gerritlab/pull/85 [12:23:48] hmm [12:25:06] dhinus: I think `git remote show` makes a request to the remote to get that info, I wonder whether it should use the tracking branch if that is set and only make that slow request if it's not there [12:25:30] the problem is that the tracking branch might be different from the default branch [12:25:44] and I don't think there's a way to find the default one without a network call [12:26:48] is there a case in which the tracking branch is not the branch you want to be the target branch? [12:26:59] I put two examples in the commit message :) [12:27:03] * taavi reads [12:28:59] you could solve the second problem by stripping out the change ID suffix from the branch name, like how wikibugs does it https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/commit/64a8e04d58f16a96571ee570f0872f4dcb8e5793 [12:29:51] yep, there are other possible solutions. I guess it depends if the first example is something that other users might actually like (e.g. use something like "stable" as the target branch) [12:30:01] and for the first one, I tend to either push branches as a whole, or with gerritlab, but at least I don't tend to combine those two [12:30:35] the thing is that gerritlab is already annoyingly slow, so I'd rather not make it even more slower :-) [12:30:37] as I'm still experimenting, I first pushed without (and created a tracking branch), then tried to use gerritlab and that messed things up [12:30:57] happy to discuss in the PR itself, the other solution I see is to change the README to specify the expected behavior :) [12:31:06] also the FIXME should be kept I think [12:31:27] I think that feature was added in the linked commit? [12:31:37] (specifying a branch on the command line) [12:33:08] hrm, indeed, this is just some global setup code [12:33:17] thanks for the feedback, I might have a go at implementing the "suffix stripping" you suggested [12:33:38] so now it could pull the default branch info from the remote, and then just ignore it one is passed on the command line? :/ [12:33:48] please leave a comment in the PR as well, I pinged dancy and tyler on Slack so I will wait for their opinions too [12:33:50] * taavi remembers not liking how the gerritlab codebase used globals [12:34:19] taavi: yes that's suboptimal, but my thinking was the opposite: it's already so slow it won't make much difference adding 2 more secs :D [12:35:45] now I'm just imagining someone else getting annoyed about it as well, going around and finding this, fixing it, and then claiming victory even through we're back at the status quo [12:38:25] "lol" [12:39:56] side-question, I'm looking at this libup task for pywikibot https://phabricator.wikimedia.org/T401076 and from https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Pywikibot_image did I get it right that we basically want to merge gerrit/stable into gitlab/stable, then gitlab/stable into gerrit/toolforge then kick off a build ? [12:45:09] note that T401076 and T401077 are different tasks :-) [12:45:10] T401076: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T401076 [12:45:10] T401077: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T401077 [12:45:43] the toolforge branch lives in our gitlab repo, not in gerrit, but otherwise yes [12:46:00] and for paws there's a version number somewhere in the paws repo (which unfortunately still lives in github) that needs to be bumped [12:46:59] doh yes, of course gitlab/toolforge! ok thank you I'll try the toolforge thing first [12:50:46] toolforge thing == updating pywikibot on toolforge [12:51:57] ... or not, I can't push to toolforge-repos/ atm [12:53:21] hmm I don't remember how permissions are managed in gitlab [12:55:14] unfortunately the answer is "manually" [12:55:36] you should have an invite now [12:56:11] yes I do, thank you taavi [12:56:37] separate issue: the wiki says to pick the "merge branch master" commit from gerrit, but it looks like that commit is missing for the latest 2 releases (10.3.1 and 10.3.2). [12:58:24] yeah I'm going to get to the end, assuming I'm successful, and update the instructions [13:00:35] taavi: re:permissions, can you see who are the owners of the toolforge group? I think I cannot invite other people to the group [13:00:48] which group? [13:00:51] github? [13:00:54] gitlab [13:01:08] I didn't notice it was gitHUB in this case [13:01:16] paws is on github [13:01:18] so I was trying to add godog to gitLAB [13:01:28] and can't find a way [13:01:30] but the toolforge thing is on gitlab [13:01:43] so on gitlab we have both https://gitlab.wikimedia.org/groups/toolforge-repos/-/group_members where I do see you as an owner [13:01:56] right, why I couldn't find that link? :D [13:01:58] and https://gitlab.wikimedia.org/groups/repos/cloud/toolforge/-/group_members where you're also an owner [13:02:09] it's quite well hidden, manage -> members in the sidebar [13:02:15] gotcha thanks [13:02:47] the mind boggles [13:02:47] I tried "settings" but not "manage" :P [13:03:45] why make things simple and efficient when you can sell a support contract for people who can't find it :D [13:03:50] please bear with me folks, 'become pywikibot' says 'You are not a member of the group tools.pywikibot.' I thought I had superpowers but clearly not [13:03:52] LOL [13:03:57] try `sudo become` [13:04:13] hah, thank you taavi that works [13:04:16] taavi clearly you lost your chance to reply with https://xkcd.com/149/ [13:04:34] or just `sudo -iu tools.pywikibot`, `become` is just a sudo wrapper that does its own group check first [13:05:13] got it, makes sense [13:05:36] [step-results] 2025-08-29T13:05:22.173990432Z Built image tools-harbor.wmcloud.org/tool-pywikibot/pywikibot-scripts-stable:latest@sha256:7d78c8ab800cf57a76405f2627e8c9e3f95747789a6e740acbcf2d252ea8a77c [13:08:26] I'll follow this next https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Pywikibot_image#Doing_a_test [13:08:53] https://miro.medium.com/v2/resize:fit:1400/format:webp/1*7N_pMAomi-v06ZW4-NAqNA.jpeg [13:11:48] looks like it worked to the extent that exit code is 0 [13:11:58] taavi: toolforge jobs logs -f is awesome btw [13:15:39] ok I got this going, does it make sense ? https://wikitech.wikimedia.org/w/index.php?title=Portal%3AToolforge%2FAdmin%2FPywikibot_image&diff=2337185&oldid=2315115 [13:49:32] godog: I checked the history of that page, we used to do "git merge gerrit/stable", but taavi changed it to use a commit hash instead https://wikitech.wikimedia.org/w/index.php?title=Portal:Toolforge/Admin/Pywikibot_image&diff=prev&oldid=2158262 [13:49:53] taavi do you remember what was the reason for that? [13:50:36] i think the point was to get the actual tagged release, and not any of the commits staged on that branch after it [13:52:14] I see, those commits should only be pulled after a future release [13:53:16] not sure why they commit those to "stable" and not to master [13:54:47] I'm also not sure why we should pick the "merge" commit, and not the commit with the version tag [13:55:52] (I'm only ruminating this because there are no "merge" commits for 10.3.2 so I'm not sure what to pick) [13:58:42] mmhh yes that'd be my other question, why not merge the tag ? [14:00:32] tbh now that there's no separate merge commit, merging the tag is probably fine [14:01:04] don't you risk pulling extraneous commits if you include the merge (when that's available)? [14:01:17] actually, stupid question: do all changes to `stable` make it to `master` first? [14:01:27] no idea... [14:51:15] godog: did you want me to investigate/clean up those cursed VMs with broken IPs/DNS? Or did you already take care of that? [14:52:12] andrewbogott: I did look into it briefly and didn't get anywhere, if you have some time now to look together that'd be nice, and next week is fine too [14:52:41] there's mediawiki2latex.collection-alt-renderer.eqiad1.wikimedia.cloud with the mismatch, the rest I "fixed" with a reboot [14:52:55] ok, so that's the only remaining oddball? [14:53:58] correct, yes [14:54:19] the VM ip and nova's idea match, DNS doesn't [14:55:30] OK. So DNS thinks it's 172.16.2.213 but nova/neutron says 172.16.0.22 [14:55:55] but there's a ptr record for 172.16.0.22 that is correct, so it seems like there's just the one record to fix. [14:58:53] hah, not sure if that's good news or bad news? i.e. that A and PTR got out of sync ? [14:59:50] it's weird! [15:05:19] https://www.irccloud.com/pastebin/ByBJlQwh/ [15:06:21] godog: three things about that paste: 1) --all-projects is just a cheat to save me keeping track of what project owns the zone. 2) openstack designate always calls dns domains 'zones' because the word 'domain' is overloaded and used elsewhere 3) don't be fooled by that paste into thinking I got those commands right the first time [15:06:55] andrewbogott: are you still using abogott-nstesting.tools.eqiad1.wikimedia.cloud? [15:07:11] taavi: nope, delete it [15:07:24] andrewbogott: hah! ok thank you for the additional context and the fix, quite useful [15:09:36] * taavi does [15:20:09] earlier godog found this bit of our clinic duties documentation: "Requests that represent an increase of more than double the quota or more than 300GB of storage should also be reviewed at the weekly meeting." [15:20:45] I think this was not strictly followed until now... what if we replaced it with a requirement for an extra +1? [15:21:43] requests like T403165 are technically "more than double" but they're relatively small [15:21:44] T403165: Quota increase for Wiki Loves Monuments app - https://phabricator.wikimedia.org/T403165 [15:22:22] indeed that's the task I had in mind [15:22:27] dhinus: I agree that it shouldn't have to wait for a meeting, extra +1 sounds good to me [15:22:53] (even though I probably wrote that rule originally) [15:25:33] I will boldly edit the wiki page, and send an email to cloud-admin to notify everyone [15:25:54] SGTM [15:34:40] done! [15:38:23] btw dhinus, cloudcephosd1013 is now decom'd so you don't need to think about that broken drive anymore [15:38:46] nice! yeah i think it was pulled out by dcops anyway [15:38:53] so I had already stopped worrying :)