[08:22:55] 10netops, 10Operations: mr1-esams i2c syslog flood - https://phabricator.wikimedia.org/T242097 (10ayounsi) {F31686555} Problem solved after remote hands replaced the issue. [08:23:07] 10netops, 10Operations: mr1-esams i2c syslog flood - https://phabricator.wikimedia.org/T242097 (10ayounsi) 05Open→03Resolved [17:33:01] 10netops, 10Analytics, 10DC-Ops, 10Operations: kafka-jumbo1006 and stat1005 network issues - https://phabricator.wikimedia.org/T247561 (10elukey) 05Open→03Resolved a:03elukey stat1005 is back, John and Papaul switched it to port /43. [17:47:53] 10netops, 10Operations, 10ops-eqiad: Three ports on asw2-d-eqiad are not working as expected - https://phabricator.wikimedia.org/T247881 (10elukey) [17:49:04] 10netops, 10Operations, 10ops-eqiad: Three ports on asw2-d-eqiad are not working as expected - https://phabricator.wikimedia.org/T247881 (10elukey) [18:43:51] 10Traffic, 10Commons, 10MediaWiki-File-management, 10Multimedia, and 8 others: Picture from Commons not found from Singapore - https://phabricator.wikimedia.org/T231086 (10JJMC89) [20:15:01] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) 05Resolved→03Open This went down again today at 20:12 UTC per Icinga [20:16:08] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) 20:12 <+icinga-wm> PROBLEM - Router interfaces on cr2-eqord is CRITICAL: CRITICAL: host 208.80.154.198, interfaces up: 62, down: 1, dormant: 0, excluded: 0, unused: 0: htt... [20:27:02] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) mailed Telia about it [21:02:21] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) Telia: We have opened case 01134611 and are currently investigating. [23:32:12] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) "Kindly be informed that your circuit involved in major outage in our network , suspecting faulty card , investigation is ongoing. Major case 01134704."