[06:25:39] 10Traffic, 10Analytics-Radar, 10Operations: Spammy events coming our way for sites such us https://ru.wikipedia.kim - https://phabricator.wikimedia.org/T190843 (10JAllemandou) > I thought ru stand for Russia, this can just be a Russia version of wikipedia `ru.wikipedia.ORG` is official Russian wikipedia - `... [06:47:42] 10netops, 10Operations: No LACP info for cr2-esams:ae2 - https://phabricator.wikimedia.org/T253970 (10ayounsi) This didn't work. Configuring the link as active took it down. But did show statistics. ` cr2-esams> show lacp interfaces ae2 Aggregated interface: ae2 LACP state: Role Exp Def Dis... [07:04:16] 10netops, 10Operations: ulsfo - codfw Zayo link down - https://phabricator.wikimedia.org/T255393 (10ayounsi) 05Open→03Resolved Restored as of Tue Jun 16 22:11:00 GMT 2020 [10:23:41] 10Traffic, 10Operations: HTML Dumps 429 error on RESTBase endpoints - https://phabricator.wikimedia.org/T255524 (10Kelson) We get many HTTP 429 errors from the rest(base) API if we scrape with nodes outside the VPS cluster. Really a hassle to deal with. It seems to me we are impacted... But maybe I get somethi... [11:31:03] 10Traffic, 10Operations: noc.wikimedia.org consistently 503s in eqsin and sometimes 503s in esams - https://phabricator.wikimedia.org/T255368 (10ema) Alright I finally understood what's going on. The problem here is that (1) the origin server is adding `Transfer-Encoding: chunked` to conditional HEAD requests,... [12:56:05] 10Traffic, 10Operations, 10SRE-tools, 10Goal, and 3 others: Automate generation of Management DNS records from Netbox - https://phabricator.wikimedia.org/T233183 (10Volans) @ayounsi @crusnov I've created mini-one-time script to automatically reserve the first 5 IP addresses in all relevant prefixes. Did a... [13:19:39] ema: wow, nice find [13:30:05] 10Traffic, 10Operations, 10SRE-tools, 10Goal, and 3 others: Automate generation of Management DNS records from Netbox - https://phabricator.wikimedia.org/T233183 (10ayounsi) LGTM! Especially for a one-time job. [14:36:39] cdanis: it took a while but the facepalm moment was worth it! :) [15:25:49] 10Traffic, 10Operations, 10SRE-tools, 10Goal, and 3 others: Automate generation of Management DNS records from Netbox - https://phabricator.wikimedia.org/T233183 (10crusnov) Nice, this is what we pretty much had in mind, although in the future of course if we add more prefixes or change them we'll have to... [15:42:39] 10Traffic, 10Operations, 10Phabricator: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 (10sbassett) Removing #security-team as this is now managed by #operations. [15:49:04] 10Traffic, 10Operations, 10Phabricator: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 (10Dzahn) @sbassett Where can Operations find information on where and how these blocks are configured? (also: T229620#5386233 , T218589) [15:57:01] mutante: re ^ I believe jbond42 did that migration work [15:58:17] cdanis: ok, thanks. I think we need to share the knowledge somewhere (even if not public) [15:58:28] mutante: the short answer is the puppet private repo [15:59:13] mutante: my concerns on that ticket were more wrt: security helping us figure out what policy should be and/or handing off the 'whys' of past policy decisions [16:00:10] yea, that too [16:00:36] is this T254568? just about to go into a meeting however /srv/private/modules/secret/secrets/waf/admin{2,3} [16:00:36] T254568: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 [16:00:52] yeah [16:02:29] i have a ticket to move theses to the abuse_networks hira config however chase configuered theses originaly so need his input to know if they can go or how they should be migrated [16:05:57] 10Traffic, 10Operations, 10Phabricator: Accessing Phabricator from Tor - https://phabricator.wikimedia.org/T254568 (10sbassett) @Dzahn - @herron and @chasemp would have the most domain knowledge about this right now, as they initially worked on T218784. @JBennett should also be able provide any broad securi... [17:07:29] 10netops, 10Operations, 10ops-codfw: codfw: rack/setup new srx300 (mr1) - https://phabricator.wikimedia.org/T255577 (10Papaul) @ayounsi there is an IRB interface setup on the new SRX300 . Are we going to use it ? if yes what will be the unit to use and the inet default config below ~~~ irb { unit 0 {... [17:29:28] 10netops, 10Operations, 10ops-codfw: codfw: rack/setup new srx300 (mr1) - https://phabricator.wikimedia.org/T255577 (10ayounsi) IRBs are used when several L2 devices are connected to the router and they all share the same vlan. For example in POPs where all mgmt switches terminate. In core sites, only the ms... [17:41:23] 10Traffic, 10Operations, 10SRE-tools, 10Goal, and 3 others: Automate generation of Management DNS records from Netbox - https://phabricator.wikimedia.org/T233183 (10Volans) >>! In T233183#6231998, @crusnov wrote: > Nice, this is what we pretty much had in mind, although in the future of course if we add mo...