[14:33:42] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2735948 (10BBlack) [14:40:07] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2735964 (10BBlack) Interface on asw-c-eqiad says down as well: ``` bblack@asw-c-eqiad> show interfaces xe-8/0/7 Physical interface: xe-8/0/7, Enabled, Physical link is Down Interfac... [14:44:55] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2735972 (10BBlack) Tried resetting the interface on the switch side with `test interface xe-8/0/7 restart-auto-negotiation` as well as commits of disable then re-enable of the interfac... [14:52:13] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2735980 (10BBlack) Tried re-setting from host-side software, and seems to have worked! In response to `ifconfig eth0 down`, dmesg had new output: ``` [Sat Oct 22 14:46:17 2016] failed... [15:05:17] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2735991 (10BBlack) 05Open>03Resolved a:03BBlack Seems ok post-reboot, repooled. [16:06:39] 10Traffic, 06Operations, 10ops-eqiad: cp1052 ethernet link down 2016-10-22 14:11 - https://phabricator.wikimedia.org/T148891#2736011 (10BBlack) 05Resolved>03Open It failed again: ``` 15:32 < icinga-wm> PROBLEM - Host cp1052 is DOWN: PING CRITICAL - Packet loss = 100% ``` @Joe depooled again ~15:37. Lea... [17:12:21] 10Traffic, 06Operations: nginx SSL_do_handshake spam filling disks - https://phabricator.wikimedia.org/T148893#2736029 (10BBlack)