[06:56:43] greetings [08:18:36] morning [08:57:35] mystery of the day: T426584 [08:57:36] T426584: [toolsbeta] probe flapping on ipv6 only - https://phabricator.wikimedia.org/T426584 [10:06:31] taavi: FYI (as clinic duty) lingualibre replied to T419525, the next step should be equivalent to the one I described in my last comment, but ping me if needed ofc [10:06:31] T419525: Configure vanity domain for lingualibre - https://phabricator.wikimedia.org/T419525 [10:07:25] volans: thanks, looking [10:19:55] accidentally found T426594 at the same time :D [10:19:55] T426594: acme-chief http-01 backend requests failing on ipv6 - https://phabricator.wikimedia.org/T426594 [10:27:32] :/ [10:27:34] nice catch [10:47:44] fix is https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/312 [10:56:31] why the removed lines? (I might be missing some context) [11:18:38] I just discovered I no longer have the permission create a patch on gerrit. Not sure when this happened, perhaps when I switched to yubikey? Anyway anyone know how I can quickly resolve this? I already tried looking at wikitech and couldn’t find something useful fast enough [11:19:23] Raymond_Ndibe: what exact error are you receiving? [11:20:06] `remote rejected…prohibited by Gerrit: not permitted: create` [11:20:29] It’s explicitly said `You need 'Create' rights to create new references` [11:22:50] can you post the full error, not just a shortening of it? that is missing some detail, namely what it's pushing things to [11:22:53] 'creating new references' is not the same as pushing a patch for review [11:25:24] https://www.irccloud.com/pastebin/vrW12Bll/gerrit+push+error [11:25:49] The the text snippet above taavi [11:26:39] you don't push directly to the production branch in gerrit, either use git review or push to the correct refs that creates a CR [11:27:16] https://www.mediawiki.org/wiki/Gerrit/Tutorial#Push_your_commit_to_Gerrit [11:27:42] where you have 'refs/production', should be 'refs/for/production' (or use git-review as volans is saying) [11:30:04] Ooo yeaa, thanks Taavi. I think it’s the `for`. Volans I wasn’t pushing directly to production, since HEAD is on the branch I’m working on [11:31:05] This worked `git push origin HEAD:refs/for/production`, thanks Taavi [11:34:24] volans: the removed lines are stuff that does nothing (import_id is not wired up anywhere?? and even if it was, that resource was already imported) or the defaults that aren't re-stated anywhere else in that file [11:56:14] ack [13:07:49] hashar: do you have time to supervise if I merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1282006/8 and https://gerrit.wikimedia.org/r/c/operations/puppet/+/1282007/7 ? Or do you want to do more local testing first? [15:00:26] taavi, the two servers godog asked about in the meeting are these: [15:00:34] https://www.irccloud.com/pastebin/CXxXFzgt/ [15:00:44] correct [15:01:21] 'OVS experiments' sounds to me like something we're done with. Should we throw those servers back, or migrate to them because they're newer? Or something else? [15:01:38] yeah, not needed for OVS use anymore? [15:01:50] s/?/./ [15:02:03] * andrewbogott checks their age [15:02:42] iirc these were ordered for one of the openstack rebuild projects some time ago (either the "let's build a second production region in codfw" or the openstack-in-k8s one, can't remembered) which then got cancelled [15:03:21] yep [15:05:07] 2005 and 2006 are from march of 2022, so close to EOL. 200[78] are from July of 2023. So they are /slightly/ newer... [15:05:18] how much effort is it to move from old cloudnet to new? [15:11:41] ack andrewbogott thank you for checking [15:11:49] to your question, I don't know [15:12:02] not impossible but not trivial either [15:12:07] I suspect that it requires some back-and-forth with netops, might or might not be worth it to save the money on amortization [15:12:17] but 14 months is not a lot [15:12:25] not with netops [15:12:59] Oh, it would be strictly a taavi operation? Then it's up to you to decide it it's worth it to delay replacement by a year :) [15:17:07] I gotta go, will catch up on backlog later/tomorrow [15:18:52] i don't think i have an opinion in either direction [15:18:54] this is somewhat tracked oon T377568 [15:18:55] T377568: wmcs codfw hardware changes proposal - https://phabricator.wikimedia.org/T377568 [16:40:00] andrewbogott: I am back from vacations today, I'll try them tomorrow ;) [16:40:08] I have loaded both changes to add them to my backlog [16:40:13] works for me! [16:40:14] thx