[02:50:02] 10Traffic, 10Commons, 10MediaWiki-File-management, 10Multimedia, and 9 others: Picture from Commons not found from Singapore - https://phabricator.wikimedia.org/T231086 (10Krinkle) [10:44:53] 10Traffic, 10Analytics, 10Operations, 10Research, and 2 others: Enable layered data-access and sharing for a new form of collaboration - https://phabricator.wikimedia.org/T245833 (10Miriam) [11:01:39] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: enable BGP in the neutron transport network - https://phabricator.wikimedia.org/T245606 (10ayounsi) BGP and firewall filter config removed from codfw's router. [11:09:57] 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): CloudVPS: introduce filtering for neutron BGP addresses - https://phabricator.wikimedia.org/T246887 (10aborrero) 05Open→03Resolved We decided to drop the BGP setup for now. [11:10:01] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: enable BGP in the neutron transport network - https://phabricator.wikimedia.org/T245606 (10aborrero) [11:11:23] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: enable BGP in the neutron transport network - https://phabricator.wikimedia.org/T245606 (10aborrero) 05Open→03Declined We decided to drop the BGP project for now. We collected valuable information about the setup, how it works and what we... [11:23:56] 10netops, 10Operations: Fix LibreNMS alert "CDR bills over 75% used" - https://phabricator.wikimedia.org/T247949 (10ayounsi) p:05Triage→03Medium [11:40:29] 10netops, 10Operations, 10cloud-services-team (Kanban): New network request for CloudVPS CODFW instances transport - https://phabricator.wikimedia.org/T247633 (10aborrero) 05Open→03Declined hey @JHedden, in conversation with @ayounsi on IRC today, he asked me to prioritize this task vs {T245495}. He don'... [12:40:06] 10netops, 10Operations, 10observability, 10Epic: Migrate role::netmon to Buster - https://phabricator.wikimedia.org/T247967 (10fgiunchedi) [14:17:18] 10Traffic, 10Operations, 10observability, 10Patch-For-Review: some Prometheis not scraping the full set of targets - https://phabricator.wikimedia.org/T246860 (10CDanis) 05Open→03Resolved No current differences in targets scraped between the proms in each cluster, and no hits for `too many open files`... [14:25:12] * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): [14:25:14] * Connection state changed (MAX_CONCURRENT_STREAMS == 100)! [14:25:16] < HTTP/2 301 [14:25:18] vgutierrez: \o/ awesome [14:25:36] cdanis: more or less, yeah [14:25:41] I'm finding some issues [14:26:00] ahah, all I've tried is a simple GET against upload.wikimedia.org/ [14:27:08] hmm I'm assuming upload-lb.ulsfo [14:27:16] that's the only place where is enabled :) [14:27:31] yeah [14:27:43] I have a heinous shell function that makes such things easy [14:27:58] I'm pretty sure is shorter than your prompt function [14:27:59] ;P [14:28:11] it is [14:28:33] https://github.com/cdanis/dotfiles/blob/master/zsh/.zshrc#L59 [14:28:35] ;) [14:29:36] cdanis: BTW, why not dig +short text-lb... [14:29:45] instead of the python inline script? [14:31:12] you need to parse the URL to not repeat yourself [14:31:31] since curl needs the hostname from the URL twice, once in the URL and once as part of --resolve [14:31:44] so since I already needed to parse a URL, might as well... [14:32:07] oh cool [14:55:06] 10Traffic, 10Analytics, 10Operations, 10Patch-For-Review: Test atskafka deployment - https://phabricator.wikimedia.org/T247497 (10elukey) ` elukey@kafka-jumbo1001:~$ kafka acls --add --allow-principal User:CN=varnishkafka --producer --topic atskafka_test_webrequest_text kafka-acls --authorizer-properties z... [14:58:35] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: enable BGP in the neutron transport network - https://phabricator.wikimedia.org/T245606 (10faidon) 05Declined→03Open Reopening this per IRC, and given this is a prod/WMCS task affecting prod in major ways. First of all, it'd be great to h... [15:52:20] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: enable BGP in the neutron transport network - https://phabricator.wikimedia.org/T245606 (10ayounsi) * Neutron BGP is outbound only, so we would still need to keep the VRRP VIP between cr1 and cr2 and a static route from cloud -> core * Neutron... [16:42:29] 10Traffic, 10Analytics, 10Operations: Publishing project anomaly data for censorship researchers. Evaluate privacy threats - https://phabricator.wikimedia.org/T183990 (10Nuria) [18:16:18] nuria: oh wow, I just noticed that this task is from 2018 [18:25:03] 10HTTPS, 10Traffic, 10Operations: store.wikimedia.org HTTPS issues - https://phabricator.wikimedia.org/T128559 (10BBlack) Update: sometime since I last checked, they've changed the header to: `strict-transport-security: max-age=31557600` (~1 year, vs ~90 days before). Still missing the other attributes (`pr... [18:26:22] 10HTTPS, 10Traffic, 10Operations: store.wikimedia.org HTTPS issues - https://phabricator.wikimedia.org/T128559 (10BBlack) [18:28:28] 10HTTPS, 10Traffic, 10Operations: Enable HSTS on store.wikimedia.org for HTTPS - https://phabricator.wikimedia.org/T128559 (10Krinkle) [18:28:37] 10HTTPS, 10Traffic, 10Operations: Enable HSTS on store.wikimedia.org for HTTPS - https://phabricator.wikimedia.org/T128559 (10Krinkle) [18:31:33] 10Traffic, 10Operations, 10Reading-Admin: TEST: redirect small portion of unauthenticated desktop users to mobile web - https://phabricator.wikimedia.org/T117826 (10dr0ptp4kt) 05Open→03Declined Not planning to do this. [18:32:19] 10Traffic, 10Multimedia, 10Operations, 10RESTBase-API, and 2 others: Thumb API: Varnish / CDN questions - https://phabricator.wikimedia.org/T150673 (10dr0ptp4kt) [18:34:42] 10HTTPS, 10Traffic, 10Operations: Enable HSTS on store.wikimedia.org for HTTPS - https://phabricator.wikimedia.org/T128559 (10BBlack) [18:44:33] 10Traffic, 10Operations: Switch port 80 to nginx on primary clusters - https://phabricator.wikimedia.org/T107236 (10BBlack) 05Open→03Declined We're not using nginx software for this functionality anymore, and everything else related to these parts of the software stack have changed and are still evolving,... [18:44:36] 10Traffic, 10Operations, 10Patch-For-Review: Investigate TCP Fast Open for tlsproxy - https://phabricator.wikimedia.org/T108827 (10BBlack) [18:47:41] 10Traffic, 10Operations: Switch port 80 to nginx on primary clusters - https://phabricator.wikimedia.org/T107236 (10BBlack) a:05BBlack→03None [18:54:13] 10HTTPS, 10Traffic, 10Operations, 10Tracking-Neverending: HTTPS Plans (tracking / high-level info) - https://phabricator.wikimedia.org/T104681 (10BBlack) 05Open→03Resolved Resolving this, since it has become an undead tracker for too long. There are still two trailing issues, but having this over-arch... [21:05:58] 10HTTPS, 10Traffic, 10Operations: Enable HSTS on store.wikimedia.org for HTTPS - https://phabricator.wikimedia.org/T128559 (10Krenair) Do we have a document somewhere describing the requirements of hosts pointed to by records under the wikimedia.org zone? If not should one be made and a compliance requiremen... [22:39:40] 10Traffic, 10Operations, 10SRE-tools, 10Goal, and 3 others: Automate generation of Management DNS records from Netbox - https://phabricator.wikimedia.org/T233183 (10Volans) @BBlack @crusnov This is the script I use to compare the results P10716 both ways. These is the output checking that all ops/dns repo... [22:40:00] bblack, chaomodus ^^^ direct link is: https://phabricator.wikimedia.org/T233183#5982268 [22:40:14] script and output to compare the records