[00:11:22] 10netops, 10Operations, 10Operations-Software-Development, 10netbox, 10User-crusnov: Netbox report to validate network equipment data - https://phabricator.wikimedia.org/T221507 (10ayounsi) >>! In T221507#5178669, @crusnov wrote: > [ ] test_nb_device_in_librenms: every Staged,Active asw `Device` in Netbo... [00:41:37] 10Traffic, 10Operations, 10Wikidata, 10Wikidata-Query-Service, and 2 others: Reduce / remove the aggessive cache busting behaviour of wdqs-updater - https://phabricator.wikimedia.org/T217897 (10Addshore) So, I did some really crappy analysis of the hit rate in varnish before and after this change, looking... [00:58:26] "Chosen-prefix collisions for SHA-1 hash function are now [00:58:27] practical (cost lower than 100K $), all hope is lost." [01:06:08] I saw that earlier and posted to -operations [01:07:26] ssl_ciphersuite 'strong' removes ECDHE-ECDSA-AES128-SHA, ECDHE-RSA-AES128-SHA, and DHE-RSA-AES128-SHA that are present in 'mid' [01:10:27] there is a pending patch to switch Gerrit to 'strong' [01:10:29] I assume that is all SHA-1, the remaining SHA stuff is SHA-256 and SHA-384 [01:11:46] https://mailarchive.ietf.org/arch/msg/cfrg/NhiGvOFzcEw108YLwF_ndyfB1k4?fbclid=IwAR2NiuZLlbKK3xu5Vg1EysyZ2Dab7N9mgGYQNPC0p5tGPZOwuJBCQ7R7XQY says TLS 1.2 is broken [01:12:04] ugh [01:12:12] is that just referring to the ciphers listed above or is there something more fundamental [01:12:13] ? [01:48:01] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) 05Resolved→03Open This is down again since about 5 hours. Is it ok to reuse the ticket? Same circuit IC-313592, same interfaces. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?t... [01:48:31] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) a:05Dzahn→03ayounsi [01:52:17] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10Dzahn) 5 hours ago: "We regret to inform you that we are facing a cable Between Denver and Strasburg in US. We will investigate and update you accordingly" 4 hours ago: "Our provider confirmed th... [10:07:57] 10Traffic, 10Operations, 10Patch-For-Review: Replace Varnish backends with ATS on cache upload nodes in esams - https://phabricator.wikimedia.org/T222937 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by ema on cumin1001.eqiad.wmnet for hosts: ` ['cp3036.esams.wmnet'] ` The log can be found in `... [10:49:30] 10Traffic, 10Operations, 10Patch-For-Review: Replace Varnish backends with ATS on cache upload nodes in esams - https://phabricator.wikimedia.org/T222937 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['cp3036.esams.wmnet'] ` and were **ALL** successful. [11:07:09] 10netops, 10DC-Ops, 10Operations: Juniper network device audit - all sites - https://phabricator.wikimedia.org/T213843 (10faidon) Update from IRC: Juniper's install base is actually missing a whole lot of our devices (e.g. only lists 9 EX4300s, out of... 52). @ayounsi is asking them, but this clearly needs m... [12:13:28] 10Domains, 10Traffic, 10Operations, 10WMF-Legal, 10Patch-For-Review: Move wikimedia.ee under WM-EE - https://phabricator.wikimedia.org/T204056 (10tramm) @Dzahn Just to make everything clear, we are going to use virtual hosting of our service provider on IP aadresss 185.7.252.114 to run our Wordpress home... [13:26:48] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10aborrero) [13:48:21] 10Domains, 10Traffic, 10Operations, 10WMF-Legal, 10Patch-For-Review: Move wikimedia.ee under WM-EE - https://phabricator.wikimedia.org/T204056 (10Dzahn) Hi, @tramm thanks for the update. I'll assign this back to @CRoslof for doing this at the Zone.ee level. For us in SRE the initial comment still stan... [13:48:49] 10Domains, 10Traffic, 10Operations, 10WMF-Legal, 10Patch-For-Review: Move wikimedia.ee under WM-EE - https://phabricator.wikimedia.org/T204056 (10Dzahn) a:05tramm→03CRoslof [14:23:53] 10Domains, 10Traffic, 10Operations, 10WMF-Legal, 10Patch-For-Review: Move wikimedia.ee under WM-EE - https://phabricator.wikimedia.org/T204056 (10tramm) @Dzahn I try to express myself clearly. Wikimedia Estonia is okey with taking responsibility of the whole domain and AFAIK you transfer the domain at:... [14:56:23] 10Traffic, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar): Refactor public-facing DYNA scheme for primary project hostnames in our DNS - https://phabricator.wikimedia.org/T208263 (10kostajh) > Our analytics seems to indicate the changes above had the intended effect in restoring normal levels... [15:15:49] 10netops, 10DC-Ops, 10Operations: Juniper network device audit - all sites - https://phabricator.wikimedia.org/T213843 (10ayounsi) From Juniper: > I am still in the process of changing the installed base address of the serial numbers given. > Furthermore, for the serial numbers that are not showing in MyJuni... [15:21:35] 10netops, 10Operations: eqord - ulsfo Telia link down - IC-313592 - https://phabricator.wikimedia.org/T221259 (10ayounsi) 05Open→03Resolved Thanks! Yes it's fine to reuse the ticket. Link is back up now. [15:42:11] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10aborrero) A quick iperf3 test shows that I'm wrong and this is working actually: ` aborrero@cloudvirt1018:~ 1 $ iperf3 -c cloudvirt1024.eqiad.wm... [15:43:21] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10Bstorm) Is the Neutron hardware on 10G? [15:44:03] 10Traffic, 10Operations, 10Wikidata, 10serviceops, and 4 others: [Task] move wikiba.se webhosting to wikimedia cluster - https://phabricator.wikimedia.org/T99531 (10WMDE-leszek) @BBlack @Dzahn: I have passed the topic of domain ownership transfer to the C-level ranks here at WMDE, and I have to inform that... [15:44:12] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10Bstorm) Not that it should matter to the virts themselves, but it would matter to the VMs. Just curious. [15:49:10] "WMDE is not going to transfer the ownership of wikiba.se domain to WMF." "this basically invalidates the whole idea of moving the hosting of the said website to WMF " [15:49:22] i don't understand that statement.. [15:49:34] how is "moving it to WMF" invalidating "moving it to WMF" [15:49:42] but oh well.. i dont have to [15:55:21] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10aborrero) >>! In T223272#5181553, @Bstorm wrote: > Is the Neutron hardware on 10G? yes! And they passed the 1G boundary several times already, s... [15:55:56] mutante: at this point I'd avoid responding on the ticket, as things probably need to move further up the chain with politics or whatever [15:59:55] bblack: fair enough. ok, i was about how i _personally_ dont get it but i won't [16:40:31] 10Traffic, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar): Refactor public-facing DYNA scheme for primary project hostnames in our DNS - https://phabricator.wikimedia.org/T208263 (10BBlack) @kostajh - The OONI article you linked ( https://ooni.torproject.org/post/2019-china-wikipedia-blocking... [16:41:44] 10Traffic, 10Operations, 10Patch-For-Review, 10Performance-Team (Radar): Refactor public-facing DYNA scheme for primary project hostnames in our DNS - https://phabricator.wikimedia.org/T208263 (10kostajh) Got it, thanks so much for this clarification, it's a very helpful summary of events. [16:59:45] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: evaluate if 10G is working correctly in cloudvirts - https://phabricator.wikimedia.org/T223272 (10Andrew) drive-by-comment: I've also been disappointed at transfer speeds when migrating to/from 10G systems but never followed up to figure out... [17:39:53] https://blog.cloudflare.com/better-http-2-prioritization-for-a-faster-web/ [18:18:36] It would be nice for us to have that kind of control... [18:18:48] currently nginx just does whatever it wants [21:53:23] k8s sneaked into that dashboard https://grafana.wikimedia.org/d/000000343/load-balancers?orgId=1 [22:15:32] 10netops, 10Operations, 10Operations-Software-Development, 10netbox, and 2 others: Netbox report to validate network equipment data - https://phabricator.wikimedia.org/T221507 (10crusnov) The >>! In T221507#5182977, @gerritbot wrote: > Change 510256 had a related patch set uploaded (by CRusnov; owner: CR... [22:24:53] 10netops, 10Operations, 10Operations-Software-Development, 10netbox, and 2 others: Netbox report to validate network equipment data - https://phabricator.wikimedia.org/T221507 (10crusnov) It was pointed out to me that the vendor name in entPhysical is there, so we could hypothetically check that (for inven... [22:40:01] 10Traffic, 10Operations, 10observability, 10PHP 7.2 support, 10Performance-Team (Radar): [Regression] Varnish is replacing the detailed HTTP 500 page from PHP 7 with "503 Service Temporarily Unavailable" - https://phabricator.wikimedia.org/T223336 (10Krinkle) [22:42:10] 10Traffic, 10Operations, 10observability, 10PHP 7.2 support, and 2 others: [Regression] Varnish is replacing the detailed HTTP 500 page from PHP 7 with "503 Service Temporarily Unavailable" - https://phabricator.wikimedia.org/T223336 (10jijiki)