[15:12:42] 10Traffic, 10SRE, 10Patch-For-Review: cp_upload @ eqsin cascading failures, February 2021 - https://phabricator.wikimedia.org/T274888 (10BBlack) [16:02:04] 10Traffic, 10Varnish, 10Maps, 10SRE: Tilerator should purge Varnish cache - https://phabricator.wikimedia.org/T109776 (10MSantos) [16:15:45] 10Traffic, 10SRE, 10Patch-For-Review: cp_upload @ eqsin cascading failures, February 2021 - https://phabricator.wikimedia.org/T274888 (10BBlack) Updates on where we're at on some of the pain points above, in terms of solution analysis: 1. `large_objects_cutoff` and all related things - the key thing that ma... [16:58:02] 10Traffic, 10Analytics, 10Analytics-Kanban, 10SRE: Traffic anomalies: Factor out list of countries into a dedicated Hive table - https://phabricator.wikimedia.org/T272052 (10mforns) a:03mforns [17:53:22] 10Traffic, 10SRE: provision more machines for eqsin caches - https://phabricator.wikimedia.org/T275046 (10BBlack) [19:02:04] 10Traffic, 10SRE, 10Patch-For-Review: cp_upload @ eqsin cascading failures, February 2021 - https://phabricator.wikimedia.org/T274888 (10BBlack) [19:04:27] 10Traffic, 10SRE: cache_upload cache policy + large_objects_cutoff concerns - https://phabricator.wikimedia.org/T275809 (10BBlack) [19:04:35] 10Traffic, 10SRE, 10Patch-For-Review: cp_upload @ eqsin cascading failures, February 2021 - https://phabricator.wikimedia.org/T274888 (10CDanis) >>! In T274888#6861566, @BBlack wrote: > 4. `sh` hashing - I think @CDanis already worked on some patches to transition us to maglev hashing a quarter or two ago, b... [19:07:54] 10Traffic, 10SRE, 10Patch-For-Review: cp_upload @ eqsin cascading failures, February 2021 - https://phabricator.wikimedia.org/T274888 (10BBlack) I've spun out T275809 to go into some depth on the #1 part about `large_objects_cutoff` [19:36:20] 10Traffic, 10SRE: cache_upload cache policy + large_objects_cutoff concerns - https://phabricator.wikimedia.org/T275809 (10BBlack) So, to expand a little bit on the text quoted at the top with some initial insights about cutoff vs nuke-limit tradeoffs and some of my current thinking and/or assumptions: * Turn...