[08:05:48] 10Traffic, 10Operations: Update certspotter - https://phabricator.wikimedia.org/T204993 (10MoritzMuehlenhoff) The Icinga servers in production are now running 0.9-1~bpo9+1, but the Cron job still needs to be re-instated. [10:29:40] moritzm: this is a bit of a moving target... there's a 0.9-2 [10:29:52] with a backported fix [13:38:18] 10Traffic, 10Operations, 10Continuous-Integration-Infrastructure (Slipway): CI jobs for authdns linting need to run on Stretch - https://phabricator.wikimedia.org/T205439 (10hashar) Part of the project to migrate all CI jobs to Docker containers (#ci-slipway ) [14:39:57] certcentral so far hasn't failed a single certificate issuance O:) [14:40:05] yay [14:50:33] nice work! At some point we can also migrate archiva to it? [14:51:16] sure :) [14:51:19] we're taking care of "private" services first [14:51:37] sure sure :) [14:52:46] elukey, fwiw the list is at https://phabricator.wikimedia.org/T207050 [14:52:52] I dunno if we're necessarily doing it in order but [14:53:19] tendril is mid-process, then there's apt and archiva [14:53:59] I've actually never interacted with archiva to my knowledge, the others are more scary to muck with [14:54:09] might be worth doing archiva first of the public ones elukey vgutierrez ? [14:54:24] sure, as elukey is volunteering for that [14:54:28] O:) [14:55:01] haha maybe lets get that confirmed from elu.key first [14:55:26] also I see the old foundation logo on that site [14:55:43] I can definitely work with you guys for archiva [14:58:39] I also dunno how this all comes out in terms of scheduling [14:59:02] seeing as its a friday and a US holiday etc. [15:00:30] yup... if elukey doesn't mind, let's wait till Monday [15:00:36] it should be pretty low risk but I don't want to mess with something public on a Friday afternoon [15:01:31] Krenair: BTW; we should revisit at some point librenms and netbox and get rid of the apache2 config handling http-01 challenges there [15:01:44] I just saw that piece of config in tendril while submitting my last commit [15:02:33] vgutierrez, it'll just be deploying the standard templates for http-01 handling right? [15:02:38] we can turn that off in our setup [15:03:07] we don't need http-01 config at all there cause we are using it dns-01 [15:03:32] +1 for next week :) [15:03:52] I'm talking about https://github.com/wikimedia/puppet/blob/production/modules/tendril/templates/apache/tendril.wikimedia.org.erb#L39-L49 [15:04:01] it's a leftover from the old LE puppetization [15:04:14] I guess that something similar it's there in librenms and netbox templates [15:08:21] oh that [15:08:23] sure [15:14:44] 10Traffic, 10Operations: ATS path normalization - https://phabricator.wikimedia.org/T210295 (10ema) p:05Triage>03Normal [15:19:34] 10Traffic, 10Operations: ATS path normalization - https://phabricator.wikimedia.org/T210295 (10ema) [15:26:31] Krenair: so https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/475482/ and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/475483/ [15:27:14] vgutierrez, did you also get rid of the letsencrypt::cert::integrated resources? [15:28:36] yes.. that's already merged in https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/475335/ and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/474747/ (this one is yours) [15:29:01] we only forgot about the http-01 challenge config [15:30:13] hah forgot I did that [15:30:14] okay then [15:32:20] take a look to https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/475481/ as well please :) [15:35:34] thx [15:40:56] 10Traffic, 10Operations, 10Patch-For-Review: Migrate most standard public TLS certificates to CertCentral issuance - https://phabricator.wikimedia.org/T207050 (10Vgutierrez) [16:16:44] 10Traffic, 10Operations: ATS path normalization - https://phabricator.wikimedia.org/T210295 (10ema) I now see that @BBlack prepped https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/407643/ to bring reality closer to theory by adding the missing characters to `{mediawiki,restbase}_encode` -- with a caveat... [16:18:35] 10Traffic, 10Operations, 10Patch-For-Review: ATS backend-side request-mangling - https://phabricator.wikimedia.org/T209021 (10ema) [16:26:24] 10Traffic, 10Operations: Renew Digicert Unified in 2019 - https://phabricator.wikimedia.org/T209515 (10BBlack) Downtimes set, we shouldn't get cert alerts in icinga [16:28:34] 10Traffic, 10Beta-Cluster-Infrastructure, 10DNS, 10Operations, and 4 others: Ferm's upstream Net::DNS Perl library questionable handling of NOERROR responses without records causing puppet errors when we try to @resolve AAAA in labs - https://phabricator.wikimedia.org/T153468 (10MoritzMuehlenhoff) Status u... [16:32:29] 10Traffic, 10Beta-Cluster-Infrastructure, 10DNS, 10Operations, and 4 others: Ferm's upstream Net::DNS Perl library questionable handling of NOERROR responses without records causing puppet errors when we try to @resolve AAAA in labs - https://phabricator.wikimedia.org/T153468 (10Krenair) This was already a...