[02:11:32] <wikibugs>	 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2679212 (10Mattflaschen-WMF)
[02:12:05] <wikibugs>	 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2054159 (10Mattflaschen-WMF)
[07:48:28] <marostegui>	 So if this gdb capture is truly representative of the state of the server during one of these stalls, I think you might need to go back to MariaDB folks to review the trace and see if they see anything that points to something in their server or TokuDB.
[07:48:32] <marostegui>	 :-)
[08:24:35] <wikibugs>	 10DBA, 13Patch-For-Review, 07Wikimedia-Incident: Improve db backup handling, specially of misc hosts - https://phabricator.wikimedia.org/T138562#2679301 (10jcrespo) p:05Triage>03Low
[09:35:50] <wikibugs>	 10DBA, 06Labs, 10Labs-Infrastructure: Implement a frontend failover solution for labsdb replicas - https://phabricator.wikimedia.org/T141097#2679433 (10Marostegui)
[09:39:40] <wikibugs>	 10DBA, 06Labs, 10Labs-Infrastructure: Provide at least 2 separate service endpoints: one for slow, long running queries; and another for quick, web requests - https://phabricator.wikimedia.org/T147051#2679435 (10Marostegui)
[09:40:40] <wikibugs>	 10DBA, 06Labs, 10Labs-Infrastructure: Provision with data the new labsdb servers and provide replica service with at least 1 shard from a sanitized copy from production - https://phabricator.wikimedia.org/T147052#2679448 (10Marostegui)
[11:36:08] <wikibugs>	 10DBA, 06Labs, 10Labs-Infrastructure: Provision with data the new labsdb servers and provide replica service with at least 1 shard from a sanitized copy from production - https://phabricator.wikimedia.org/T147052#2679448 (10Krenair) To actually have users connect to new labsdb servers we're going to need vie...
[14:40:39] <wikibugs>	 10DBA, 06Labs, 10Labs-Infrastructure, 06Operations: Migrate labsdb1005/1006/1007 to jessie - https://phabricator.wikimedia.org/T123731#2680059 (10fgiunchedi) 05Open>03stalled Setting as stalled, though next steps look like this:  [] Flip tools master from labsdb1005 to labsdb1004 [] Decommission labsdb...
[14:58:40] <wikibugs>	 10DBA, 06Operations: Add icinga check for all MySQL/MariaDB hosts to check they have the right read_only value - https://phabricator.wikimedia.org/T111766#1615926 (10fgiunchedi) out of curiosity I tried asking the following questions via prometheus for eqiad  ```name='mysql_global_variables_read_only{role="sla...
[17:35:03] <wikibugs>	 10DBA, 06Collaboration-Team-Triage, 06Community-Tech-Tool-Labs, 10Flow, and 5 others: Enable Flow on wikitech (labswiki and labtestwiki), then turn on for Tool talk namespace - https://phabricator.wikimedia.org/T127792#2680437 (10Mattflaschen-WMF) >>! In T127792#2618122, @Catrope wrote: > For consistency I...