[00:06:22] 10Traffic, 10Operations, 10Performance-Team, 10SRE-swift-storage, and 2 others: Automatically clean up unused thumbnails in Swift - https://phabricator.wikimedia.org/T211661 (10dpifke) Yes, this is totally something I can tackle. It meshes well with the work I'm doing on converting ArcLamp to use Swift fo... [01:31:33] 10HTTPS, 10Traffic, 10DBA, 10Operations, and 4 others: dbtree loads third party resources (from google.com/jsapi) - https://phabricator.wikimedia.org/T96499 (10Krinkle) [11:04:03] 10Traffic, 10Operations, 10observability: prometheus2004 not scraping lvs2007 & lvs2008 - https://phabricator.wikimedia.org/T246860 (10elukey) The issue went away by itself after mentioning it to Chris and Filippo :) [11:06:01] 10Traffic, 10Operations, 10Performance-Team, 10SRE-swift-storage, and 2 others: Automatically clean up unused thumbnails in Swift - https://phabricator.wikimedia.org/T211661 (10Gilles) Note that we're not running the Swift daemon that consumes that header and deletes objects yet (swift-object-expirer), afa... [11:43:52] 10Traffic, 10Operations, 10observability: prometheus2004 not scraping lvs2007 & lvs2008 - https://phabricator.wikimedia.org/T246860 (10fgiunchedi) I took a look at this on both prometheus200[34] for `up{instance=~"elastic2055.*9108"}` and the metric appears yesterday on 2004 at 9:44 and 2003 at 17:48. Wherea... [13:46:22] 10Traffic, 10Operations, 10observability: some Prometheis not scraping the full set of targets - https://phabricator.wikimedia.org/T246860 (10CDanis) [13:46:56] 10Traffic, 10Operations, 10observability: some Prometheis not scraping the full set of targets - https://phabricator.wikimedia.org/T246860 (10CDanis) I think the following shell one-liner could be turned into an Icinga check for this condition pretty easily: {P10675} [15:24:31] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations: debian-glue-backports not enabling backports on buster - https://phabricator.wikimedia.org/T247316 (10ema) [15:29:57] 10Traffic, 10Continuous-Integration-Infrastructure, 10Operations, 10Patch-For-Review: debian-glue-backports not enabling backports on buster - https://phabricator.wikimedia.org/T247316 (10ema) p:05Triage→03Medium [16:07:42] 10Acme-chief, 10IPv6, 10Patch-For-Review, 10cloud-services-team (Kanban): tools-acme-chief-01 is attempting to validate DNS challenge against cloud authdns IPv6 addresses - https://phabricator.wikimedia.org/T245937 (10Andrew) p:05Triage→03Medium [18:39:31] 10Traffic, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install cp202[7-0], cp203[0-9], cp204[0-2] - https://phabricator.wikimedia.org/T247340 (10Papaul) [18:40:08] 10Traffic, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install cp202[7-0], cp203[0-9], cp204[0-2] - https://phabricator.wikimedia.org/T247340 (10Papaul) p:05Triage→03Medium [19:28:30] bblack: if/when you have a minute, chaomodus and I would like a second pair of eyes (or blessing ;) ) for 2 CRs: [19:28:33] - https://gerrit.wikimedia.org/r/c/operations/dns/+/569340 [19:28:37] - https://gerrit.wikimedia.org/r/c/operations/puppet/+/578506 [20:43:22] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2002.codfw.wmnet - https://phabricator.wikimedia.org/T246756 (10Papaul) ` [edit interfaces interface-range disabled] member xe-2/0/47 { ... } + member xe-2/0/46; [edit interfaces] - xe-2/0/46 { - description l... [20:43:52] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2002.codfw.wmnet - https://phabricator.wikimedia.org/T246756 (10Papaul) [20:45:01] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2003.codfw.wmnet - https://phabricator.wikimedia.org/T246334 (10Papaul) [20:51:30] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2004.codfw.wmnet - https://phabricator.wikimedia.org/T246669 (10Papaul) ` [edit interfaces interface-range disabled] member xe-2/0/46 { ... } + member xe-7/0/47; [edit interfaces] - xe-7/0/47 { - description l... [20:52:26] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2004.codfw.wmnet - https://phabricator.wikimedia.org/T246669 (10Papaul) [21:01:57] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2005.codfw.wmnet - https://phabricator.wikimedia.org/T246666 (10Papaul) ` [edit interfaces interface-range disabled] member xe-7/0/47 { ... } + member xe-7/0/46; [edit interfaces] - xe-7/0/46 { - description l... [21:02:16] 10Traffic, 10DC-Ops, 10Operations, 10decommission, 10ops-codfw: decommission lvs2005.codfw.wmnet - https://phabricator.wikimedia.org/T246666 (10Papaul)