[15:19:44] 07Puppet, 06cloud-services-team, 10Toolforge, 13Patch-For-Review: Too many puppet facts on toolforge k8s workers - https://phabricator.wikimedia.org/T381293#10406389 (10Andrew) 05Open→03Resolved This warning is no longer displayed, and having lots of facts doesn't seem to actually break anything. [15:27:39] 10Mail, 06Infrastructure-Foundations, 06SRE: Log tls cipher information - https://phabricator.wikimedia.org/T381927#10406412 (10jhathaway) p:05Triage→03Low [16:10:48] FIRING: PuppetZeroResources: Puppet has failed generate resources on bast4005:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetZeroResources [16:22:55] FIRING: MaxConntrack: Max conntrack at 82.64% on krb1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [16:27:55] RESOLVED: MaxConntrack: Max conntrack at 82.64% on krb1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [16:46:49] 10Mail, 06collaboration-services, 06Infrastructure-Foundations, 06SRE, and 2 others: VRTS e-mail address unreachable / e-mail routing issue - https://phabricator.wikimedia.org/T380009#10406698 (10Arnoldokoth) 05Open→03Resolved [16:47:22] 10Mail, 06collaboration-services, 06Infrastructure-Foundations, 10vrts, 10Znuny: spamassassin broken for VRTS - https://phabricator.wikimedia.org/T380396#10406707 (10LSobanski) 05Open→03Resolved a:03LSobanski Since the change improved the situation we will not be subscribing to the Validity RBL... [16:55:48] RESOLVED: PuppetZeroResources: Puppet has failed generate resources on bast4005:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetZeroResources [17:06:32] 10netops, 10HIDDENPARMA, 06Infrastructure-Foundations: Reduce the steps needed to deploy hiddenparma - https://phabricator.wikimedia.org/T382268 (10Joe) 03NEW [17:12:16] 10netops, 10HIDDENPARMA, 06Infrastructure-Foundations, 10Prod-Kubernetes, 07Kubernetes: Allow reaching services on the aux k8s cluster bypassing the CDN - https://phabricator.wikimedia.org/T382269 (10Joe) 03NEW [17:56:56] FIRING: MaxConntrack: Max conntrack at 81.77% on krb1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [18:01:55] RESOLVED: MaxConntrack: Max conntrack at 81.77% on krb1001:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack