[05:17:40] 10DBA, 10Patch-For-Review: Decommission db2041 - https://phabricator.wikimedia.org/T223950 (10Marostegui) [05:21:29] 10DBA, 10Patch-For-Review: Decommission old coredb machines (<=db2042) - https://phabricator.wikimedia.org/T221533 (10Marostegui) [05:24:47] 10DBA, 10Operations, 10ops-eqiad, 10Goal, and 2 others: rack/setup/install db11[26-38].eqiad.wmnet - https://phabricator.wikimedia.org/T211613 (10Marostegui) [05:26:01] 10DBA, 10Goal, 10Patch-For-Review: Productionize db2[103-120] - https://phabricator.wikimedia.org/T222772 (10Marostegui) [05:26:59] 10DBA, 10Operations, 10ops-codfw, 10Goal, 10Patch-For-Review: rack/setup/install db2[103-120].codfw.wmnet (18 hosts) - https://phabricator.wikimedia.org/T221532 (10Marostegui) [05:27:04] 10DBA, 10Goal, 10Patch-For-Review: Productionize db2[103-120] - https://phabricator.wikimedia.org/T222772 (10Marostegui) 05Open→03Resolved All these hosts have been populated and pooled. Pending steps will be followed up at T220170: weights tackling, candidate masters conversions, moving old hosts around... [05:51:39] 10DBA, 10Operations, 10Performance-Team, 10serviceops, and 2 others: Increased instability in MediaWiki backends (according to load balancers) - https://phabricator.wikimedia.org/T223952 (10Marostegui) Update from the subtask related to the slow query detected (which might or might not be the cause for thi... [06:01:08] 10DBA: Decommission db2040 - https://phabricator.wikimedia.org/T224079 (10Marostegui) [06:01:19] 10DBA: Decommission db2040 - https://phabricator.wikimedia.org/T224079 (10Marostegui) p:05Triage→03Normal [06:04:47] 10DBA: Decommission db2040 - https://phabricator.wikimedia.org/T224079 (10Marostegui) [06:11:30] 10DBA, 10Operations, 10Performance-Team, 10serviceops, 10HHVM: Increased instability in MediaWiki backends (according to load balancers) - https://phabricator.wikimedia.org/T223952 (10Joe) [06:15:46] 10DBA, 10Patch-For-Review: BBU issues on codfw - https://phabricator.wikimedia.org/T214264 (10Marostegui) [06:16:40] 10DBA, 10Patch-For-Review: Decommission db2040 - https://phabricator.wikimedia.org/T224079 (10Marostegui) [09:12:01] 10DBA, 10Patch-For-Review: Decommission old coredb machines (<=db2042) - https://phabricator.wikimedia.org/T221533 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by marostegui on cumin1001.eqiad.wmnet for hosts: ` ['db2070.codfw.wmnet'] ` The log can be found in `/var/log/wmf-auto-reimage/20190522... [09:40:34] 10DBA, 10Patch-For-Review: Decommission old coredb machines (<=db2042) - https://phabricator.wikimedia.org/T221533 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['db2070.codfw.wmnet'] ` and were **ALL** successful. [11:11:46] Hey marostegui, There's a plan to deploy a new extension to production: https://phabricator.wikimedia.org/T216955 it would introduce a table for s8 and one table for s3 (testwikidatawiki) which is public and has one row only. Is it okay or should I make a ticket and discuss it further [11:19:18] Amir1: For new tables you can just add them [11:19:23] It's always been the case [11:21:25] Reedy: technically yes, but it needs to be signed off by DBAs becuase private daata, backup, size limit, etc. There's a note somewhere if I can find it [11:22:01] backup and size limits are issues that should have been addressed before the extension was even considered for deploy... [11:22:36] Similar for private data, at least, before the extension starts writing to the tables [11:23:55] yeah I know, just double checking [12:02:46] Amir1: Will get back to you in a bit [12:02:53] Sure [12:03:03] ah, you already said it is public [12:03:20] if you need a view on it, that requires a ticket to cloud [12:05:04] I don't think it should be replicated. It's public but useless for users [12:08:35] should not be replicated? meaning you want replication filter? [12:08:45] or you mean exposed with a view? [12:27:34] 10DBA, 10Core Platform Team, 10MediaWiki-Database, 10Patch-For-Review: Slow query ApiQueryRevisions on enwiki - https://phabricator.wikimedia.org/T224017 (10Marostegui) p:05Triage→03High a:03Marostegui [12:32:22] 10DBA, 10Core Platform Team, 10MediaWiki-Database, 10Patch-For-Review: Slow query ApiQueryRevisions on enwiki - https://phabricator.wikimedia.org/T224017 (10Marostegui) [13:12:06] marostegui: it's just one row, I don't think we need a view or any way of replication filter [13:12:26] Amir1: then go for it [13:12:37] Nice, thanks! [15:10:53] 10DBA, 10Core Platform Team, 10MediaWiki-Database: Slow query ApiQueryRevisions on enwiki - https://phabricator.wikimedia.org/T224017 (10Maintenance_bot) [15:13:14] 10DBA: Compress and defragment tables on labsdb hosts - https://phabricator.wikimedia.org/T222978 (10Maintenance_bot) [15:16:30] 10DBA: Create a recovery/provisioning script for database binary backups - https://phabricator.wikimedia.org/T219631 (10Maintenance_bot) [15:19:31] 10DBA, 10Operations, 10ops-eqiad, 10Goal, 10User-Marostegui: rack/setup/install db11[26-38].eqiad.wmnet - https://phabricator.wikimedia.org/T211613 (10Maintenance_bot) [15:24:13] 10DBA: Decommission db2040 - https://phabricator.wikimedia.org/T224079 (10Maintenance_bot) [15:26:48] 10DBA, 10Goal: Productionize db11[26-38] - https://phabricator.wikimedia.org/T222682 (10Maintenance_bot) [15:29:20] 10DBA: Decommission old coredb machines (<=db2042) - https://phabricator.wikimedia.org/T221533 (10Maintenance_bot) [15:32:39] 10DBA, 10Goal: Address Database infrastructure blockers on datacenter switchover & multi-dc deployment - https://phabricator.wikimedia.org/T220170 (10Maintenance_bot) [15:33:44] 10DBA, 10Goal: Decommission dbstore1001, dbstore2001, dbstore2002 - https://phabricator.wikimedia.org/T220002 (10Maintenance_bot) [22:20:21] 10DBA, 10Phabricator, 10Documentation, 10Release-Engineering-Team (Backlog), 10User-MModell: Prepare a disaster recovery plan for failing over from phab1001 to phab2001 (or phab2001 to 1001) - https://phabricator.wikimedia.org/T190572 (10greg)