[10:35:43] marostegui: do we really need to see the queries that are running when waiting for depool? i.e. use desired columns instead of SELECT * [10:36:10] i don't normally check, unless a host is taking a bit to get depooled and then I do check [10:36:20] that's my use case, not sure about what others do [10:38:06] I have a screen full of characters that I cannot read as part of the rolling restarts :) [10:38:13] XD [10:38:24] I usually check only if it's taking a long time and sometimes eyeball it if it prints out something unexpected [10:38:24] I guess we can just delete it [10:38:27] federico3: thoughts ^ [10:38:28] ah [10:38:47] should we maybe just print it after a few mins or attempts instead of all the time? [10:39:09] yes, works for me [10:39:31] also I configure byobu with a long backlog just in case [10:39:45] cezmunsta: want to give that patch a try? [10:40:50] To avoid the noise, sure :) [10:41:07] haha [10:42:30] "START - Cookbook sre.mysql.pool pool db1197: Maintenance" this is showing up as notifications disabled, yet seems to have been marked by the rolling restart for a repool - does that sounds correct? [10:44:26] https://phabricator.wikimedia.org/P95505 [10:48:32] https://gerrit.wikimedia.org/r/c/operations/puppet/+/1279290 [10:48:33] wow [10:48:37] Fixing [10:49:15] +1 [10:49:45] merging [10:50:00] how did this host go through in the last rolling restarts then? [10:50:35] I am using the cookbook for repool, which allows checking of Icinga. The restarts previously only checked replication lag [10:50:42] ah right [10:51:43] This is the only one that I have seen so far and most runs have used the cookbook. I will keep an eye out for any others though, should they appear [10:52:37] k thanks [10:53:16] I will wait to see how it is handled by the code, hopefully self-healing :) [11:16:51] It recovered itself and started repooling [12:54:58] cezmunsta: so this can be closed right? https://phabricator.wikimedia.org/T433493 [12:55:57] Yep, missed that one getting generated [12:56:03] no worries! [12:56:22] closing it [12:56:24] done [12:56:28] ah thanks!