[09:13:43] 10HTTPS, 10Traffic, 10SRE, 10Documentation, 10Performance-Team (Radar): TLS certificates renewal process - https://phabricator.wikimedia.org/T196248 (10Aklapper) [10:37:24] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, 10SRE-swift-storage: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10jcrespo) [10:38:06] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, 10SRE-swift-storage: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10jcrespo) Clarifying expected duration and method of depooling for next week. [10:38:46] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, 10SRE-swift-storage: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10jcrespo) [10:51:03] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, 10SRE-swift-storage: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10ayounsi) Note that one of the eqiad/codfw links is still down due to Texas weather issues. I hope it will be back up by the 22nd, but if it's not, we shoul... [11:04:15] XioNoX: so 1 out of 3 links still down right now? [11:04:39] well one of them is via chicago I suppose :P [11:05:58] technically we have 4 with GTT VPLS, but I hope we never get to use it :) [11:06:58] yeah and then also detours via ulsfo I suppose ;) [11:07:14] hahaha, the scenic route [11:07:31] I guess the last week was a testiment to how well our network redundancy works [11:07:59] we lost I think 2 links at the same time? [11:08:20] yeah, we lost ulsfo-codfw [11:08:39] and no impact indeed [13:20:36] 10Traffic, 10netops, 10Data-Persistence-Backup, 10SRE, 10SRE-swift-storage: Depool codfw swift cluster - https://phabricator.wikimedia.org/T267338 (10fgiunchedi) Reassessing the situation early next week sounds good to me -- we're not terribly in a rush to do this and might as well avoid unnecessary risks [15:44:35] 10Traffic: Wikipedia not opening images in any browser except Opera. - https://phabricator.wikimedia.org/T275211 (10Reedy) [16:57:45] 10Acme-chief, 10LDAP, 10cloud-services-team (Kanban): acme-chief ldap certs required chained (with intermediate CA) versions suddenly - https://phabricator.wikimedia.org/T271063 (10Majavah) 05Open→03Resolved a:03Andrew Nobody has disagreed. Boldly closing. [18:58:25] 10Traffic, 10netops: TATA SKY Broadband (AS134674) issues with connecting to upload.wikimedia.org - https://phabricator.wikimedia.org/T275234 (10CDanis) [18:58:54] 10Traffic, 10netops: TATA SKY Broadband (AS134674) issues with connecting to upload.wikimedia.org - https://phabricator.wikimedia.org/T275234 (10CDanis) [18:58:56] 10Traffic, 10SRE: Wikipedia not opening images in any browser except Opera. - https://phabricator.wikimedia.org/T275211 (10CDanis)