[07:52:53] <moritzm>	 headsup: bast2002 will be reimaged in ~10m
[14:20:04] <cdanis>	 moritzm: jbond42: have either of you done any thinking about a SSH CA for host keys?
[14:20:07] <cdanis>	 just curious :)
[14:22:17] <jbond42>	 cdanis: not really no
[14:23:51] <moritzm>	 yeah, I think we already have a task for this as well, but realistically there's many other things with higher precedence before that happens
[14:25:11] <cdanis>	 sure, makes sense
[14:25:12] <cdanis>	 thanks
[14:32:33] <paravoid>	 yeah we've talked about it years in the past, I'm sure there are Tasks
[14:32:39] <paravoid>	 but what moritz said :)
[14:34:11] <paravoid>	 I remember also talking at some point about... ephemeral keys even, for things like cumin
[14:34:20] <paravoid>	 or short-lived ones rather
[14:39:27] <cdanis>	 I stumbled across smallstep this weekend (https://smallstep.com/blog/diy-single-sign-on-for-ssh/ https://github.com/smallstep/certificates https://github.com/smallstep/cli) and it looks like it would be able to issue short-lived SSH client certificates after you auth against our CAS install (via OIDC)
[14:39:59] <kormat>	 https://github.com/nsheridan/cashier is in the same general direction
[14:41:45] <cdanis>	 anyway it seems like an interesting way to solve a few problems: 2FA for SSH, short-lived client keys, and also, you could have CNAMEs for bastions instead of physical machine hostnames
[14:41:56] <cdanis>	 but ofc, just idle thoughts :)
[15:50:15] <andrewbogott>	 gehel: wcqs-beta-01 is about to run out of disk space — is that somehow on purpose?
[15:50:33] <andrewbogott>	 It may fill up its hypervisor slightly before it fills up its /srv volume which may produce weird behavior
[15:51:07] <gehel>	 andrewbogott: already? We reloaded the data last week to clean it up
[15:51:45] <andrewbogott>	 hm...
[15:51:54] <andrewbogott>	 I guess because of thin provisioning the actual image didn't shrink
[15:52:04] <andrewbogott>	 so maybe I should just turn off the alert on the hypervisor
[15:52:09] <gehel>	 andrewbogott: looks like we have plenty of space inside that VM, you might be looking at it from the outside and our cleanup did not shrink the allocation
[15:52:30] <andrewbogott>	 yeah, I think that's right.  I'll just silence the disk checks on those hypervisors
[15:52:32] <andrewbogott>	 so, nevermind :)
[15:53:20] <gehel>	 I think ryankemper talked to you (or someone on your team) about it
[15:53:40] <gehel>	 is there a way to shrink that disk allocation? Or should we just kill and re-create the VM ?
[15:54:18] <andrewbogott>	 you could kill and re-create but I don't think it matters
[15:54:39] <andrewbogott>	 having the physical drive 'full' with an empty VM is just working as designed.
[15:54:59] <gehel>	 :)
[15:55:09] <andrewbogott>	 I downtimed the physical disk alerts until 2031; I'll let y'all worry about disk space yourselves
[15:55:34] <gehel>	 the data reload was tracked as T273636, but we should have circled by to you with an update
[15:55:35] <stashbot>	 T273636: Blazegraph journal for wcqs is too big - https://phabricator.wikimedia.org/T273636
[16:01:04] <gehel>	 andrewbogott: I'll add a note to T273579, ping me (or ryankemper) if there is something more to do on our side
[16:01:05] <stashbot>	 T273579: cloudvirt-wdqs1001 getting out of space due to huge VM - https://phabricator.wikimedia.org/T273579
[16:03:06] <andrewbogott>	 sounds good
[16:46:50] <cdanis>	 mutante: smh, can't believe you linked placeholder.com when the strictly-better http://placekitten.com/ exists ;)
[16:51:07] <_joe_>	 ^^
[16:51:21] <_joe_>	 placekitten is amazing
[16:51:49] <_joe_>	 I think cdanis showed it to me, in fact
[16:54:01] <_joe_>	 bd808: I'm merging your two python images patches and build the new image
[17:29:40] <bd808>	 _joe_: thank you :)
[17:29:58] <_joe_>	 the image should be available
[17:30:25] <_joe_>	 I wanted to get fancy and add a test, but then I realized I forgot to add http proxy support to the verification step in docker-pkg :/
[17:32:21] <bd808>	 I just purged my locally built image and fetched the one from the real reop. All looks good on my end.
[17:32:48] <bd808>	 Now I just need to sweet talk the Blubber folks into merging the tweak I need there as well :)
[17:32:49] <_joe_>	 great
[17:32:54] <_joe_>	 ahah
[17:33:19] * bd808 will shill for +2 with sticker bribes
[17:36:33] <Majavah>	 stickers? where?
[17:37:30] <bd808>	 Majavah: piles and piles of them all over my office. I have a sticker hoarding problem. :)
[17:38:04] <Majavah>	 bd808: I do too, and I want more :D
[17:41:42] <_joe_>	 Majavah: bd808 is the official sticker dealer of the WMF tech community
[18:14:33] <mutante>	 cdanis: lol, nice! add it :)
[18:29:18] <apergos>	 got any more goats??
[18:41:06] <tabbycat>	 apergos: we have cattorneys too https://www.youtube.com/watch?v=lGOofzZOyl8
[19:08:36] <apergos>	 seen it (of course, haven't we all by now? 😸)
[19:10:14] <tabbycat>	 and of course, catdevelopers too https://www.mediawiki.org/wiki/File:Im_in_ur_patchez_lolcat.jpg
[19:10:39] <tabbycat>	 I should remove it from my UP though as I no longer have +2
[19:10:44] <tabbycat>	 meh
[19:10:47] <tabbycat>	 too lazy
[19:11:45] <apergos>	 lol
[21:34:31] <apergos>	 congrats mutante on putting the last nail in hiera()'s coffin!
[21:41:42] <mutante>	 apergos: lookup('thank_you::message')
[21:41:50] <apergos>	 :-)
[21:42:17] <chaomodus>	 :D
[21:45:05] <mutante>	 mwdebug1002 (VM) not coming back from reboot.. just a goner.. nothing on console.. uhm.. the ones in codfw had no problem. Guess I will just delete it and make a new one with the same name, not really a difference for the user. VM was created back in 2016
[21:46:12] <apergos>	 long as it comes back in some form
[23:16:26] <mutante>	 so that VM is broken and if I manually delete it there will be issues with icinga/puppet/netbox
[23:16:58] <mutante>	 and if i try to use the decom script it fails with exceptions in generate_DNS_snippets
[23:17:40] <mutante>	 so much about the "why make new ones"
[23:25:09] <bd808>	 cattle not pets :)
[23:26:02] <mutante>	 yea, was my plan. people did not agree