[11:21:53] T427905 turned out way easier than I expected, since apparently last time I had to deal with a similar thing I wrote a script to automate it :D [11:21:53] T427905: Renew metricsinfra CA certificate - https://phabricator.wikimedia.org/T427905 [13:14:18] any objections to me upgrading toolsbeta to 1.32 tomorrow? [14:03:25] not from me! [14:04:14] taavi, would you advise using istio in paws and quarry for the sake of consistency? I don't know enough to know if it's the right tool for the job. [14:05:37] andrewbogott: in quarry we can probably get away with just exposing the service directly as a NodePort and not having anything in the middle for simplicity. maybe in paws as well, if not I would go with Istio just because we know it works now [14:06:23] ok! we will see if I know enough to properly experiment. [14:07:05] well, I just noticed I need to upgrade Kyverno first to stay in support so nevermind on that tomorrow upgrade :D T427928 [14:07:06] T427928: Upgrade Kyverno to 1.16 series - https://phabricator.wikimedia.org/T427928 [14:41:14] hrm, the WMCS meeting etherpad hasn't been rotated yet? [14:43:59] oh, I think that's on Raymond_Ndibe but he was (is?) ooo [14:44:27] I have not yet used the rotation-robot so I will take this opportunity to give it a try. [14:47:46] huh, does everyone on the team have their own individual exec environ and bot account? [15:18:49] andrewbogott: I have finally tested the lvm/ephemeral fix you did on https://gerrit.wikimedia.org/r/c/operations/puppet/+/1282006/ [15:19:00] I have validated them, that passes 🎉 [15:19:09] great! thank you [15:19:21] the last couple weeks have been a bit denses sorry :-\ [15:19:50] No worries. The only rush was 'get this fixed before hashar builds his next fleet of CI workers' [15:20:31] I did a similar change in the shell script but it looks slightly less easy to follow compared to you ( using sed and old versions of lsblk might not support regex matching). Thus I have abandoned it ( was: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1268941/1/modules/labs_lvm/files/make-instance-vg-ephem.sh ) [15:20:54] ;) [15:21:17] thus I guess you can puppet-merge both change and mark T422258 resolved [15:21:18] T422258: Puppet fail to create volume group for ephemeral disk space when it is sda (instead of sdb) - https://phabricator.wikimedia.org/T422258 [15:21:44] (it was not even a blocker, the manual fix is easy, but I felt like we could fix it [15:25:10] not quite a blocker but very annoying