[01:39:51] <jinxer-wm>	 FIRING: FermMSS: Unexpected MSS value on 10.2.2.27:80 @ ms-fe1010 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS
[01:44:51] <jinxer-wm>	 RESOLVED: FermMSS: Unexpected MSS value on 10.2.2.27:80 @ ms-fe1010 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS
[07:13:32] <wikibugs>	 10netops, 06Infrastructure-Foundations, 10ops-drmrs: cr1-drmrs to asw1-b12-drmrs link down - https://phabricator.wikimedia.org/T389071#10655910 (10ayounsi) @RobH make sure to link the inbound shipment to the existing ticket, so remote hands can set it up directly.  Let's also use the initial positions : port...
[07:53:51] <jinxer-wm>	 FIRING: FermMSS: Unexpected MSS value on 10.2.2.27:80 @ ms-fe1010 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS
[07:58:51] <jinxer-wm>	 RESOLVED: FermMSS: Unexpected MSS value on 10.2.2.27:80 @ ms-fe1010 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS
[10:32:09] <wikibugs>	 06Traffic: Provide cookbook(s) to operate liberica - https://phabricator.wikimedia.org/T388369#10656404 (10Vgutierrez) 05In progress→03Resolved
[16:33:15] <wikibugs>	 06Traffic, 10Liberica: Alert on deployed config not being used - https://phabricator.wikimedia.org/T389175#10657806 (10Vgutierrez) 05Open→03Resolved
[16:59:53] <wikibugs>	 06Traffic, 13Patch-For-Review: Upgrade Varnish from 6.0 to 7.1 - https://phabricator.wikimedia.org/T378737#10657971 (10BCornwall)
[17:31:13] <wikibugs>	 06Traffic, 13Patch-For-Review: Upgrade Varnish from 6.0 to 7.1 - https://phabricator.wikimedia.org/T378737#10658157 (10BCornwall)
[17:48:13] <wikibugs>	 06Traffic, 10Data-Engineering (Q3 2025 January 1st - March 31th), 10DPE HAProxy Migration, 13Patch-For-Review: Make webrequest_frontend being ingested using the in-data `dt` field - https://phabricator.wikimedia.org/T388397#10658363 (10JAllemandou)
[17:52:42] <wikibugs>	 06Traffic, 10Data-Engineering (Q3 2025 January 1st - March 31th), 10DPE HAProxy Migration: Update HAProxyKafka  kafka-timestamp type - https://phabricator.wikimedia.org/T389521 (10JAllemandou) 03NEW
[19:04:35] <jinxer-wm>	 FIRING: SystemdUnitFailed: haproxykafka.service on cp1105:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[19:04:44] <sukhe>	 huh
[19:05:33] <sukhe>	 that's weird it looks healthy 
[19:05:41] <sukhe>	 ah ok
[19:05:55] <sukhe>	 fabfur: fyi ^
[19:05:56] <sukhe>	 Mar 20 18:58:22 cp1105 systemd[2250]: haproxykafka.service: Failed to set up mount namespacing: /run/systemd/unit-root/run/haproxykaf>
[19:05:59] <sukhe>	 Mar 20 18:58:22 cp1105 systemd[2250]: haproxykafka.service: Failed at step NAMESPACE spawning /usr/bin/haproxykafka: No such file or >
[19:06:32] <sukhe>	 Restart=always kicked in but yeah
[19:07:32] <jinxer-wm>	 FIRING: SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:07:33] <jinxer-wm>	 FIRING: SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:08:58] <jinxer-wm>	 FIRING: SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:09:35] <jinxer-wm>	 RESOLVED: SystemdUnitFailed: haproxykafka.service on cp1105:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[19:12:32] <jinxer-wm>	 RESOLVED: SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:12:33] <jinxer-wm>	 FIRING: [2x] SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:13:58] <jinxer-wm>	 RESOLVED: SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[19:17:33] <jinxer-wm>	 RESOLVED: [2x] SLOMetricAbsent: <no value>   - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent
[20:22:00] <fabfur>	 strange, let me check 
[20:22:07] <fabfur>	 that's the only host?
[20:25:42] <sukhe>	 yeah 
[20:26:20] <fabfur>	 apparently 
[21:48:08] <wikibugs>	 06Traffic, 13Patch-For-Review: Upgrade Varnish from 6.0 to 7.1 - https://phabricator.wikimedia.org/T378737#10659607 (10BCornwall)
[22:06:31] <wikibugs>	 06Traffic, 13Patch-For-Review: Allow acmecerts to deploy certificates in tmpfs storage - https://phabricator.wikimedia.org/T384227#10659780 (10Fabfur)
[22:26:25] <wikibugs>	 06Traffic, 06SRE, 13Patch-For-Review: Incorrect X-Cache-Status reported by deployment-prep caches - https://phabricator.wikimedia.org/T269825#10659837 (10Aklapper) @BBlack, @Vgutierrez: Could you please answer the last comment? Thanks in advance!