[00:19:52] <wikibugs>	 10DBA, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Deploy labsdbuser and views to new clouddb hosts - https://phabricator.wikimedia.org/T268312 (10Bstorm)
[05:43:18] <wikibugs>	 10DBA: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) s8 has InnoDB errors.
[06:03:51] <wikibugs>	 10DBA: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui)
[06:13:21] <wikibugs>	 10DBA, 10decommission-hardware, 10Patch-For-Review: decommission es1019.eqiad.wmnet - https://phabricator.wikimedia.org/T270159 (10ops-monitoring-bot) cookbooks.sre.hosts.decommission executed by marostegui@cumin1001 for hosts: `es1019.eqiad.wmnet` - es1019.eqiad.wmnet (**FAIL**)   - Downtimed host on Icinga...
[06:18:21] <wikibugs>	 10DBA, 10decommission-hardware, 10Patch-For-Review: decommission es1019.eqiad.wmnet - https://phabricator.wikimedia.org/T270159 (10Marostegui) I have manually powered off the host as the idrac for this one has been broken for a long time.
[06:18:42] <wikibugs>	 10DBA, 10decommission-hardware, 10Patch-For-Review: decommission es1019.eqiad.wmnet - https://phabricator.wikimedia.org/T270159 (10Marostegui)
[06:26:11] <wikibugs>	 10DBA, 10decommission-hardware, 10Patch-For-Review: decommission es1013.eqiad.wmnet - https://phabricator.wikimedia.org/T268436 (10Marostegui)
[12:07:26] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) s5 has been built from sanitarium's master on db1154:3315. It has been sanitized and running check_private_data returned clean results. Let's see if InnoDB show up in the next...
[12:07:47] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui)
[12:55:42] <wikibugs>	 10DBA, 10Data-Services: Prepare and check storage layer for bclwiktionary - https://phabricator.wikimedia.org/T270280 (10LSobanski) p:05Triage→03Medium Thanks, let us know when the database is created, so we can sanitize it.
[12:56:00] <wikibugs>	 10DBA, 10Data-Services: Prepare and check storage layer for diqwiktionary - https://phabricator.wikimedia.org/T270276 (10LSobanski) p:05Triage→03Medium Thanks, let us know when the database is created, so we can sanitize it.
[14:02:00] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10hnowlan) Just fyi we're running our data ingestion at the moment - both just to note that the database will have data now, but also to ensure that our ingestion throttling is working acceptably. Let us know if anything stands...
[14:05:07] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10Marostegui) Thanks for the heads up, I am checking the graphs now - there is obviously an increase on pretty much all the graphs, but nothing to worrying as of now.
[14:18:04] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10jcrespo) @hnowlan thanlgs indeed. Once you have done a first complete pass, it would be nice, if possible, to aim for  a regular schedule for importing that works for you, dbas and backups, so we maximize available performance...
[14:48:41] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10Marostegui) @hnowlan how's the throttle being done at the moment, do you have some details about batch sizes, sleep times etc?
[14:53:45] <wikibugs>	 10DBA, 10Patch-For-Review: Test upgrading sanitarium hosts to Buster + 10.4 - https://phabricator.wikimedia.org/T268742 (10Marostegui) s1 and s8 are being sanitized on db1154, which I expect it to take a couple of days or so.
[15:04:40] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10hnowlan) >>! In T268505#6698493, @Marostegui wrote: > @hnowlan how's the throttle being done at the moment, do you have some details about batch sizes, sleep times etc?   Currently we're doing batches of 10000 entries with a 5...
[15:06:01] <wikibugs>	 10DBA: New database request: sockpuppet - https://phabricator.wikimedia.org/T268505 (10Marostegui) >>! In T268505#6698573, @hnowlan wrote: >>>! In T268505#6698493, @Marostegui wrote: >> @hnowlan how's the throttle being done at the moment, do you have some details about batch sizes, sleep times etc? >  >  > Curr...