[00:01:57] 10netops, 10Operations: Netbox has incorrect email address for GTT - https://phabricator.wikimedia.org/T246564 (10faidon) We have one global account, migrated from a previous system. I wasn't able to find how to create individual accounts, so that will do I guess :) I've added it to the pwstore, so @ayounsi s... [00:32:08] 10netops, 10Operations, 10Wikimedia-Incident: Add graceful-restart to cr2-esams - https://phabricator.wikimedia.org/T246338 (10CDanis) Committed configuration at 00:21 UTC. Took a few minutes for all BGP sessions to be recreated but eventually wound up with this: `cdanis@re0.cr2-esams> show bgp summary | m... [00:35:41] 10Traffic, 10Operations, 10Performance-Team, 10Patch-For-Review, 10Wikimedia-Incident: 15% response start regression as of 2019-11-11 (Varnish->ATS) - https://phabricator.wikimedia.org/T238494 (10Krinkle) [02:15:43] 10netops, 10Operations, 10Wikimedia-Incident: Add graceful-restart to cr2-esams - https://phabricator.wikimedia.org/T246338 (10CDanis) `cdanis@re0.cr2-esams> show bgp summary | match "(Active|Idle|Connect)" Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/... [10:36:43] 10Traffic, 10Operations, 10Wikimedia-Logstash, 10observability, and 3 others: Changing Kibana filters is ridiculously slow - https://phabricator.wikimedia.org/T189333 (10fgiunchedi) >>! In T189333#5945454, @Krinkle wrote: > This is still an issue. > > Editing Kibana dashboards: > * In Safari, crashes the... [11:06:19] 10Traffic, 10MediaWiki-ResourceLoader, 10Operations, 10Performance-Team, 10Wikimedia-Incident: load.php?modules=startup miss rate trippled on 2020-02-05 - https://phabricator.wikimedia.org/T247020 (10akosiaris) p:05Triage→03High [11:08:50] akosiaris: apologize me for volansing [11:08:53] err wow [11:08:59] I apologize for volansing [11:09:08] 10Traffic, 10MediaWiki-ResourceLoader, 10Operations, 10Performance-Team, 10Wikimedia-Incident: load.php?modules=startup miss rate tripled on 2020-02-05 - https://phabricator.wikimedia.org/T247020 (10CDanis) [11:09:35] ahahahaha [11:10:56] I think _joe_ and volans are getting to me; "remember me to ask you" is a common form of Englitalian [11:11:19] yup [11:11:38] cdanis: since when I'm a verb? [11:11:46] :-P [11:11:51] volans: some time after you became a meme [11:11:58] lol [11:11:58] <_joe_> cdanis: remember/remind is very hard for us [11:12:15] _joe_: I now make that mistake sometime, and also, I omit articles occasionally [11:12:18] remember me to remind you :D [11:12:19] and btw, verbs are first class citizen in almost every language [11:12:26] <_joe_> cdanis: oh noes [11:12:27] so you got a promotion volans! [11:12:32] 🎉 [11:12:34] <_joe_> well articles are hard in other languages [11:12:58] <_joe_> we err on the side of removal in english, Italian uses articles so much we always feel we're adding them [11:13:44] interesting [11:14:33] We don't really use pronouns (I, you, she/he) for what is worth. The verb suffix is enough [11:15:04] articles... what do you mean you remove articles? [11:15:08] example? [11:16:42] <_joe_> In Italian you'd say 'the suffix of the verb' [11:17:18] in English you could say that, or "the verb's suffix", or you could even say "the verb suffix", taking the whole thing to be a noun phrase, I think any of those three are totally acceptable [11:18:08] <_joe_> Also, we would say 'the Italian uses so many articles' [11:18:16] lol [11:18:27] ok, we do that too [11:18:48] <_joe_> English uses articles sparsely [11:19:02] yup... same with spaniards :) [11:19:08] I tend to love in English that I can say "the verb suffix" and not "the suffix of the verb" [11:19:12] we tend to overcomplicate stuff in English [11:20:03] <_joe_> Or at least it sounds like it does. But I learned English listening to the Ramones, and it shows ofc [11:49:05] 10netops, 10Operations: Configure management-instance on router with Junos > 17.3 - https://phabricator.wikimedia.org/T247073 (10ayounsi) p:05Triage→03Low [11:49:18] 10netops, 10Operations: Configure management-instance on routers with Junos > 17.3 - https://phabricator.wikimedia.org/T247073 (10ayounsi) [11:52:04] 10netops, 10Operations: Configure management-instance on routers with Junos > 17.3 - https://phabricator.wikimedia.org/T247073 (10ayounsi) [11:52:07] 10netops, 10Operations: Upgrade routers - https://phabricator.wikimedia.org/T243080 (10ayounsi) [12:10:07] "people" being plural in English is another classic source of confusion for us [12:11:13] but then again the United States is singular, so what do I know [12:12:30] yep [12:12:39] "people" is a PITA [12:13:01] I usually think how is it in Spanish (singular) and then use it like a plural in English [12:13:19] luckily we have Jim Morrison reminding (ha!) us that people *are* strange [12:18:03] then there's "peoples" too, which obviously is the plural of a plural [12:23:13] 10netops, 10Operations, 10Wikimedia-Incident: Juniper HA audit - https://phabricator.wikimedia.org/T191667 (10ayounsi) Next step here as this is something that needs to be squared away. Decide what should be configured for which type of devices, respectively: * Dual-REs routers: GRES + GR or NSR ** The GRES... [12:33:08] 10Traffic, 10MediaWiki-ResourceLoader, 10Operations, 10Performance-Team, 10Wikimedia-Incident: load.php?modules=startup miss rate tripled on 2020-02-05 - https://phabricator.wikimedia.org/T247020 (10ema) I suspect this is due to the fact that we are unsetting `Accept-Encoding` in `do_global_send_request`... [12:44:17] there's lots of singular/plural confusion in English -- compare also referring to companies https://english.stackexchange.com/questions/1338/are-collective-nouns-always-plural-or-are-certain-ones-singular [12:48:12] after all this I'm currently having troubles writing a commit message :D [12:48:46] we're on RO Friday, don't commit! [12:58:13] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: introduce filtering for neutron BGP addresses - https://phabricator.wikimedia.org/T246887 (10aborrero) Ping. It would be nice to address this ASAP. [13:59:29] * Krinkle likes to use "persons" from time to time in a sentence [14:21:22] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: introduce filtering for neutron BGP addresses - https://phabricator.wikimedia.org/T246887 (10ayounsi) Will push the following to keep the previous behavior and be on a whitelist basis instead of blacklist. We can tune it later on if needed. `... [14:47:09] 10netops, 10Operations, 10cloud-services-team (Kanban): CloudVPS: introduce filtering for neutron BGP addresses - https://phabricator.wikimedia.org/T246887 (10ayounsi) Confirmed that `nc -zv 208.80.153.189 22` doesn't work anymore. While ping to 208.80.153.190 does. I'll send a Homer CR to make it generic a... [15:07:04] 10netops, 10Operations: Netbox has incorrect email address for GTT - https://phabricator.wikimedia.org/T246564 (10ayounsi) LGTM. The main advantage I see is that it reduces the amount of passwords we have to rotate when someone leaves. The portal doesn't have a NOC email but phone numbers. I added the 1st and... [15:50:22] 10netops, 10Operations, 10Wikimedia-Incident: Investigate Juniper storm control - https://phabricator.wikimedia.org/T245192 (10ayounsi) Looking good! So the `description storm_control_interface;` is not needed here as the individual interface descriptions have the priority. Now that we have the interface-ra... [17:10:10] bblack: do you have a preferred path in mind for the checkout of netbox-generated snippet repo on the authdns hosts? [17:13:18] and maybe a specific user too? [17:23:20] I'd guess its own user, and just make it world-readable for the other bits to pick up. [17:23:54] I don't have any strong prefs on path. The one for ops/dns is at /srv/authdns/git/ (which probably wasn't the best name, in retrospect) [17:24:03] /srv/something-else/git ? [17:29:25] ./srv/netbox/git [17:29:45] on the netbox hosts will be /srv/netbox-exports [17:29:52] sorry [17:29:55] /srv/netbox-exports/dns.git [17:29:58] that's the bare one [17:32:41] all in all it's clear on the authdns servers it should be /srv/git/ [20:34:25] 10netops, 10Operations, 10Wikimedia-Incident: Add graceful-restart to cr2-esams - https://phabricator.wikimedia.org/T246338 (10CDanis) 05Open→03Resolved Discussion of rolling out `graceful-restart` to other dual-RE routers is at T191667#5948038 [22:34:58] 10netops, 10Operations, 10Wikimedia-Incident: Investigate Juniper storm control - https://phabricator.wikimedia.org/T245192 (10Papaul) - Removed all interfaces xxx unit 0 family ethernet-switching of the interfaces covered by the two existing interface-range (yours and the existing disabled one) - Commit us... [23:37:35] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2005.codfw.wmnet - https://phabricator.wikimedia.org/T246666 (10Papaul) [23:38:56] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2004.codfw.wmnet - https://phabricator.wikimedia.org/T246669 (10Papaul) [23:40:06] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2006.codfw.wmnet - https://phabricator.wikimedia.org/T246329 (10Papaul)