[08:31:14] greetings [09:38:13] morning! [09:48:47] * volans errand to run, will bbiab [09:53:29] morning [11:00:19] * dcaro lunch [11:34:55] turns out the fix to cloudidp-dev CAS complaining about a key being too short was to replace that key with an another one of the exact same length :/ [11:35:10] * taavi tries to remember why he wanted to log in to labtesthorizon in the first place [11:35:29] lolsob [11:48:52] and looks like I broke puppet on cloudwebs as I tried to understand where those keys came from, https://gerrit.wikimedia.org/r/c/operations/puppet/+/1240251 fixes that [12:15:11] the prize of knowledge xd [12:15:23] *price :facepalm: [12:38:34] is there an easy way to hit something hosted by the toolsbeta services vm? [12:38:47] sorry I meant static [12:38:47] toolsbeta-static-2 [12:41:19] https://toolsbeta-static.wmcloud.org/taavi-test/foo.txt [12:54:35] thx [12:55:17] ok to update Bird on the codfw1dev instances of cloudlb and cloudservices to 2.18 now? [12:57:54] sure [13:06:04] k, will do so in a bit [14:42:36] does anyone have an idea why the Hosts header https://gerrit.wikimedia.org/r/c/operations/puppet/+/1240278 doesn't match the current PCC instances we run in cloud VPS? [14:48:37] moritzm: probably due to this? [14:48:37] Feb 17 23:46:39 cloudinfra-cloudvps-puppetserver-1 puppet-facts-upload[548930]: 413 Request Entity Too Large</tit> [14:51:07] <moritzm> ah, interesting [14:51:20] <taavi> that error seems to be coming from the nginx running on pcc-db1002 [14:56:18] <taavi> moritzm: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1055499 says that file was 27M in 2024, and now it's 147M, so something seems to have grown in numbers :/ [14:57:44] <taavi> shouldn't there be something cleaning up old files from /var/lib/puppetserver/server_data/facts? [15:30:09] <taavi> T417795 [15:30:10] <stashbot> T417795: Prune old reports from /var/lib/puppetserver/server_data/facts on Cloud VPS puppet servers - https://phabricator.wikimedia.org/T417795 [17:04:29] * dcaro gtg do some paperwork, be back in a bit [20:24:22] * dcaro off [20:24:24] <dcaro> cya tomorrow