[08:07:40] https://wikitech.wikimedia.org/wiki/APT_repository#Security seems to be out-dated. the key being used for buster-wikimedia is a different gpg key [08:42:52] kormat: if the procedure is still right I think it is only a matter of adding a new Buster key page, should be ok [08:43:12] elukey: afaict the key being used is _much_ older [08:43:16] like.. 2006 older [08:44:46] ah I see in list-keys [08:44:58] there are two [09:47:46] tomorrow morning I'll upgrade Druid (the cluster that is used by Turnilo/Superset/etc..) to a new version, we don't expect any impact [10:51:25] jbond42: out of curiosity, do you know what the value of `$role` would have been before this fix? https://gerrit.wikimedia.org/r/c/operations/puppet/+/618022/ [10:58:10] John's on vacation for the next two weeks [11:00:46] ah, good for him :) [14:01:22] btw I've again edited the post-commit hook on /srv/private on the puppetmasters, I'm pretty sure I got it right this time though [14:02:08] heads up: I'll be upgrading librenms to 1.66 shortly, no impact expected [16:38:07] cdanis, XioNoX - o/ as FYI tomorrow morning I'll upgrade Druid, and the netflow realtime ingestion will be stopped for a bit [16:38:21] cool! [16:38:34] elukey: ping me right before you start just in case [16:38:35] ok! thanks for the heads up [16:41:54] XioNoX: I will yes, and the realtime data "lost" during the upgrade will be filled by the subsequent batch job [17:25:48] XioNoX: I'm still seeing networking issues with three of the new cloudvirts (1037, 1038, 1039). Was the change you made earlier per-host? [17:30:12] andrewbogott: nah per switches [17:30:16] and I did it on both [17:30:44] are those the only three in the other rack? [17:31:03] 4 [17:31:07] 6/7/8/9 [17:32:01] looks like there is a typo in their interface name, fixing [17:32:23] I was just going to say: VMs on 1036 work right [17:32:26] so it's not a per-switch thing [17:32:39] andrewbogott: should be good now [17:32:59] thanks, testing now [17:43:33] XioNoX: seems better, thanks [17:46:38] cool! [23:26:58] oh man, so an Apache httpd config has a line with "IncludeOptional" so you would think the thing that gets included is well.. optional.. right? [23:27:34] but here i have one where the whole server just dies on restart / reload fails just because that optional thing does not exist [23:27:58] now docs say "will be silently ignored (instead of causing an error) if wildcards are used and they do not match any file or directory or if a file path does not exist on the file system." [23:28:26] but the Debian package for mod_security2 installs a file with: [23:28:28] IncludeOptional /usr/share/modsecurity-crs/owasp-crs.load [23:28:59] the intention is made clear in a comment "Include OWASP ModSecurity CRS rules if installed" [23:29:14] but...there are no "wildcards used" here.. so surprise.. this is not actually optional [23:29:52] so .. ."owasp-crs.load: No such file or directory" and that leads to a nice "Syntax error on line 12" yay [23:45:55] in addition.. an appserver like mwdebug1001 _does_ have that missing file.. but it is not installed by any package. even though there would be a package for that [23:48:56] modsecurity-crs would install this.. but the package is not installed. so why is it even there. since only the existence of that file keeps the appservers from all having that syntax error and die on restart