[00:10:21] 06Traffic, 06collaboration-services, 10Gerrit: ATS/Gerrit: validate TLS hosts for gerrit (revert workaround that skips validation) - https://phabricator.wikimedia.org/T411904#11453382 (10Dzahn) 05Open→03Stalled There is still debate whether https://gerrit.wikimedia.org/r/c/operations/puppet/+/1215684 is... [07:18:44] 10netops, 06Infrastructure-Foundations, 13Patch-For-Review: Enable gNMI on SRX devices and fasw - https://phabricator.wikimedia.org/T390052#11453884 (10ayounsi) 05Stalled→03Resolved Everything that can be done has been done. We can revisit it the day the management switches or routers support gNMI.... [07:25:27] 10netops, 06Infrastructure-Foundations, 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11453887 (10ayounsi) I removed the "Traffic direction" option from my previous comment, as Nokia replied saying that the... [07:28:56] 10netops, 06Infrastructure-Foundations, 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11453895 (10ayounsi) @xcollazo what's the reason for the failure ? No disk space ? [12:35:08] 10netops, 06Infrastructure-Foundations, 06SRE: No byte counters for interfaces on cr2-codfw PIC 0/0 (MPC10E QSFP28 card) - https://phabricator.wikimedia.org/T412513 (10cmooney) 03NEW p:05Triage→03High [12:40:29] 10netops, 06Infrastructure-Foundations, 06SRE: No byte counters for interfaces on cr2-codfw PIC 0/0 (MPC10E QSFP28 card) - https://phabricator.wikimedia.org/T412513#11454417 (10cmooney) Hmm so this problem is worse than I thought at first. It is not just affecting the gnmic stats, but also the SNMP counters... [12:58:29] 10netops, 06Infrastructure-Foundations, 06SRE: No byte counters for interfaces on cr2-codfw PIC 0/0 (MPC10E QSFP28 card) - https://phabricator.wikimedia.org/T412513#11454493 (10cmooney) 05Open→03Resolved a:03cmooney So it seems this is a known problem, we actually hit it before on another card. To... [14:01:21] 10netops, 06Infrastructure-Foundations, 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Handle `network_flows_internal` data growth - https://phabricator.wikimedia.org/T412443#11454673 (10xcollazo) >>! In T412443#11453895, @ayounsi wrote: > @xcollazo what's the reason for the failure ? No disk s... [14:51:33] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: lvs1018: remove cross-rack links to rows A, C and D - https://phabricator.wikimedia.org/T411781#11454759 (10Jclark-ctr) 05Open→03Resolved a:03Jclark-ctr cables have been disconnected and deleted from netbox [15:03:41] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad: rows C/D Upgrade Decom Asw Switches in Rows C & D - https://phabricator.wikimedia.org/T412525 (10Jclark-ctr) 03NEW [15:04:03] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad: rows C/D Upgrade Decom Asw Switches in Rows C & D - https://phabricator.wikimedia.org/T412525#11454818 (10Jclark-ctr) [15:04:05] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: eqiad: rows C/D Upgrade Tracking - https://phabricator.wikimedia.org/T404609#11454819 (10Jclark-ctr)