[00:29:51] 10Traffic, 10Gerrit, 10Operations, 10Release-Engineering-Team-TODO, 10Release-Engineering-Team (Development services): Enable avatars in gerrit - https://phabricator.wikimedia.org/T191183 (10Aklapper) [02:39:56] 10Traffic, 10Operations, 10Performance Issue, 10Performance-Team (Radar), 10User-notice: Sometimes pages load slowly for users routed to the Amsterdam data center (due to some factor outside of Wikimedia cluster) - https://phabricator.wikimedia.org/T226048 (10MBH) 05Open→03Resolved Looks like fixed m... [11:33:46] 10netops, 10Operations: fastnetmon spamming /var/log on netflow hosts leading to disk saturation - https://phabricator.wikimedia.org/T240658 (10ayounsi) This issue is fixed in Fastnetmon 1.1.4: https://github.com/pavel-odintsov/fastnetmon/releases/tag/v1.1.4 > Suppressed excessive logging about missing IPFIX o... [11:58:46] 10netops, 10Operations, 10cloud-services-team: Return traffic to eqiad WMCS triggering FNM - https://phabricator.wikimedia.org/T240789 (10ayounsi) p:05Triage→03Low [13:34:23] 10netops, 10Operations: BGP peering sessions with corp partially down in ulsfo - https://phabricator.wikimedia.org/T239893 (10ayounsi) One of the 2 office routers had a hardware failure. I have an OIT ticket open. No ETA on resolution yet. [13:40:51] 10netops, 10Operations: Facebook BGP peering links down in ulsfo - https://phabricator.wikimedia.org/T239896 (10ayounsi) Yep, good for me! Thanks! [13:55:00] 10netops, 10Operations: Network issues reaching phabricator on IPv6 (Comcast/Portland OR) - https://phabricator.wikimedia.org/T240488 (10ayounsi) Thanks Chris for looking into it, and Brion for providing the outputs. All those mtr look fine though (the destination doesn't have any packet loss). Is the issue wi... [14:40:49] 10Traffic, 10Operations: API Querying for XML/JSON, you might get the Browser Connection Security warning HTML page (which is invalid XML) - https://phabricator.wikimedia.org/T240497 (10DavidBrooks) 05Open→03Invalid Closed as suggested. Sorry about the delay; I'll open another ticket although it's rapidly... [15:19:18] 10netops, 10Operations, 10Patch-For-Review: Add cloudmetrics1002 to network devices ACL - https://phabricator.wikimedia.org/T240456 (10ayounsi) 10.64.4.15 is already present in the router ACLs. I opened the CR bellow to rename its description. https://gerrit.wikimedia.org/r/c/operations/homer/public/+/557565 [16:35:13] 10Traffic, 10Operations: /sec-warning page: please add a helpful XML comment explaining why it's being delivered. - https://phabricator.wikimedia.org/T240794 (10Aklapper) [20:19:31] 10netops, 10Operations, 10cloud-services-team: Return traffic to eqiad WMCS triggering FNM - https://phabricator.wikimedia.org/T240789 (10Krinkle) [20:28:42] 10Traffic, 10Gerrit, 10Operations, 10Release-Engineering-Team-TODO, 10Release-Engineering-Team (Development services): Enable avatars in gerrit - https://phabricator.wikimedia.org/T191183 (10Krinkle) I've come around to agreeing with @Tgr. Gravatar seems like something we could support in good conscience...