[03:51:02] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, 06SRE: ULSFO:Switch refresh diagram - https://phabricator.wikimedia.org/T408511#11637925 (10Papaul) Initial configuration done on both switches. what left on the switches : - user homer password - sre.network.tls cookbook I will work on this... [05:50:04] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad A/B switch cabling documentation - https://phabricator.wikimedia.org/T418018#11637958 (10Papaul) @robh please see below for the diagram requested. Note: On site need to provide the fiber length needed for each connection. I have... [09:42:14] 06Traffic, 10API Platform, 10MediaWiki-User-login-and-signup, 06MediaWiki-Platform-Team (Q3 Kanban Board), 13Patch-For-Review: Login with `action=login` and bot password does not create a JWT session cookie - https://phabricator.wikimedia.org/T415007#11638082 (10daniel) [09:49:43] FIRING: HaproxyKafkaSocketDroppedMessages: Sustained high rate of dropped messages from HaproxyKafka - https://wikitech.wikimedia.org/wiki/HAProxyKafka#HaproxyKafkaSocketDroppedMessages - https://grafana.wikimedia.org/d/d3e4e37c-c1d9-47af-9aad-a08dae2b3fd5/haproxykafka?orgId=1&var-site=codfw&var-instance=cp2029&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DHaproxyKafkaSocketDroppedMessages [09:59:43] RESOLVED: HaproxyKafkaSocketDroppedMessages: Sustained high rate of dropped messages from HaproxyKafka - https://wikitech.wikimedia.org/wiki/HAProxyKafka#HaproxyKafkaSocketDroppedMessages - https://grafana.wikimedia.org/d/d3e4e37c-c1d9-47af-9aad-a08dae2b3fd5/haproxykafka?orgId=1&var-site=codfw&var-instance=cp2029&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DHaproxyKafkaSocketDroppedMessages [10:56:20] 06Traffic, 06collaboration-services, 10Continuous-Integration-Infrastructure, 10Gerrit, and 2 others: Gerrit events not received by Zuul due to TCP Proxy timeout (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11638102 (10IKhitron) Sure it's resolved? Because now it's... [14:25:49] 06Traffic, 06collaboration-services, 10Continuous-Integration-Infrastructure, 10Gerrit, and 2 others: Gerrit events not received by Zuul due to TCP Proxy timeout (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11638293 (10hashar) @IKhitron yes it is solved. #### stre... [14:31:09] 06Traffic, 06collaboration-services, 10Continuous-Integration-Infrastructure, 10Gerrit, and 2 others: Gerrit events not received by Zuul due to TCP Proxy timeout (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11638306 (10IKhitron) I see, thanks. In this case, it shou... [15:19:36] 06Traffic, 06collaboration-services, 10Continuous-Integration-Infrastructure, 10Gerrit, and 2 others: Gerrit events not received by Zuul due to TCP Proxy timeout (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11638365 (10hashar) >>! In T417497#11638306, @IKhitron wro... [15:23:41] 06Traffic, 06collaboration-services, 10Continuous-Integration-Infrastructure, 10Gerrit, and 2 others: Gerrit events not received by Zuul due to TCP Proxy timeout (CI is not triggered for some patches) - https://phabricator.wikimedia.org/T417497#11638366 (10IKhitron) No problem. [19:35:56] 06Traffic: wikimediastatus.net has expired certificate - https://phabricator.wikimedia.org/T418063 (10SunAfterRain) 03NEW