[06:35:29] 10DBA, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Productionize clouddb10[13-20] - https://phabricator.wikimedia.org/T267090 (10Marostegui) For the record: notifications are disabled, and won't be enabled till the hosts are started to receive users. [06:35:32] 10DBA, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Productionize clouddb10[13-20] - https://phabricator.wikimedia.org/T267090 (10Marostegui) 05Open→03Resolved >>! In T267090#6691483, @Marostegui wrote: > All this is pretty much done. The last thing I am testing is that all the... [06:35:34] 10DBA: Ensure InnoDB is compressed on the new clouddb hosts - https://phabricator.wikimedia.org/T270473 (10Marostegui) [06:48:40] 10DBA, 10decommission-hardware, 10Patch-For-Review: decommission es1013.eqiad.wmnet - https://phabricator.wikimedia.org/T268436 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by marostegui@cumin1001 for hosts: `es1013.eqiad.wmnet` - es1013.eqiad.wmnet (**PASS**) - Downtimed host on Icinga... [07:01:16] 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) [07:01:35] 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) Replication flowing in s1 and s8 with no InnoDB errors for the last 2 days [07:57:20] 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) [08:02:58] 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1101 - https://phabricator.wikimedia.org/T270571 (10wiki_willy) a:03Jclark-ctr @Jclark-ctr or @Cmjohnson - do we have any decom'd servers onsite with this drive size? Thanks, Willy [08:09:27] 10DBA, 10Patch-For-Review: Productionize x2 databases - https://phabricator.wikimedia.org/T269324 (10Marostegui) @CDanis I was taking a look at the section values and I have seen: `flavour` which is sort of new to me and seems to accept `regular` or `external`. But I haven't been able to find what are each for... [08:32:45] 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) [10:39:37] 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10gmodena) Hey @Marostegui ; so that you know: we'll resume testing the ingestion process in the new year; we expect a new batch of data (similar size) at the beginning of January. I'll touch base before firing off a new ingesti... [10:42:26] 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10Marostegui) Sounds good, thank you [11:34:09] 10DBA, 10Community-Tech, 10Expiring-Watchlist-Items, 10Growth-Team, and 2 others: ClearUserWatchlistJob/WatchedItemStore::removeWatchBatchForUser bad database peformance on enwiki and others, causing database lag - https://phabricator.wikimedia.org/T270481 (10Marostegui) It would help if can come up with a... [11:50:19] Here's the John Travolta picture I meant: https://imgur.com/a/EGcrjdq [11:50:36] hahahahaha [11:54:33] rotfl [12:36:26] 10Blocked-on-schema-change, 10DBA: Schema change for renaming several indexes in logging table - https://phabricator.wikimedia.org/T270620 (10Ladsgroup) [12:38:29] 10Blocked-on-schema-change, 10DBA: Schema change for renaming several indexes in logging table - https://phabricator.wikimedia.org/T270620 (10LSobanski) p:05Triage→03Medium [12:39:47] 10Blocked-on-schema-change, 10DBA: Schema change for renaming several indexes in sites table - https://phabricator.wikimedia.org/T270621 (10Ladsgroup) [12:52:14] 10Blocked-on-schema-change, 10DBA: Schema change for renaming several indexes in sites table - https://phabricator.wikimedia.org/T270621 (10LSobanski) p:05Triage→03Medium [13:31:50] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for niawiki - https://phabricator.wikimedia.org/T270414 (10LSobanski) p:05Triage→03Medium Thanks, let us know when the database is created, so we can sanitize it. [13:32:09] 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for niawiktionary - https://phabricator.wikimedia.org/T270410 (10LSobanski) p:05Triage→03Medium Thanks, let us know when the database is created, so we can sanitize it. [14:04:51] After reading the details, I think https://phabricator.wikimedia.org/T258098 represents the "proactively delete bot watchlists" scenario [14:06:22] yes, I think for now it should only apply to bots [21:10:36] 10DBA, 10Patch-For-Review: Productionize x2 databases - https://phabricator.wikimedia.org/T269324 (10CDanis) >>! In T269324#6705379, @Marostegui wrote: > @CDanis I was taking a look at the section values and I have seen: `flavour` which is sort of new to me and seems to accept `regular` or `external`. But I ha...