[01:03:29] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2403739 (mmodell) @jcrespo: Thanks, sounds good to me! Is the 01:00 AM UTC time slot ok for you? It's evening in my time zone but I know that's super late for europe and I'm not sure... [08:24:08] DBA, Operations, Wikimedia-Etherpad, Patch-For-Review, User-notice: etherpad database issues - https://phabricator.wikimedia.org/T138516#2402605 (Nemo_bis) Thanks a lot for recovering the data, you're saving me some bitter teardrops. [08:31:58] DBA, Operations, Wikimedia-Etherpad: Correct m1 backups, that were being done supposing m1-master was still a slave - https://phabricator.wikimedia.org/T138559#2403895 (jcrespo) [09:00:24] DBA: Improve db backup handling, specially of misc hosts - https://phabricator.wikimedia.org/T138562#2403942 (jcrespo) [09:00:39] DBA: dbstore1001 low available space - https://phabricator.wikimedia.org/T131705#2403955 (jcrespo) [09:00:41] DBA: Improve db backup handling, specially of misc hosts - https://phabricator.wikimedia.org/T138562#2403954 (jcrespo) [09:11:08] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2403996 (jcrespo) @mmodel please let's use the more specific T138460 for coordinating this, so you do not get spamed with other server's activity. [09:13:54] DBA, Operations, Phabricator: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2404003 (jcrespo) @mmodell Maybe we can do the first part (depool db1048) ASAP, by removing all references to it, or to m3-slave (or just stop doing temporarily those long-running processes t... [10:06:44] DBA: Improve db backup handling, specially of misc hosts - https://phabricator.wikimedia.org/T138562#2404167 (jcrespo) The is to improve handling of cases such as T138516 [10:59:54] DBA, Operations, Wikimedia-Etherpad, Patch-For-Review, User-notice: etherpad database issues - https://phabricator.wikimedia.org/T138516#2404271 (jcrespo) [10:59:57] DBA, Operations, Wikimedia-Etherpad, Patch-For-Review: Correct m1 backups, that were being done supposing m1-master was still a slave - https://phabricator.wikimedia.org/T138559#2404269 (jcrespo) Open>Resolved a:jcrespo [11:15:12] DBA, Operations, Wikimedia-Etherpad, Patch-For-Review, User-notice: etherpad database issues - https://phabricator.wikimedia.org/T138516#2404306 (jcrespo) p:High>Low There is no pending recovery actions, just waiting to remove etherpad-restore and check backups continue as usual once... [11:38:55] DBA, Operations, Phabricator: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2404370 (jcrespo) Also, could this be part of your goal (maybe not technically, but give them equal importance as gallium)? I own these machines, but m3 only serves the Phabricator service, t... [12:18:15] DBA, MediaWiki-Special-pages, Patch-For-Review: Filtering Special:NewPages by page size - https://phabricator.wikimedia.org/T12817#2404490 (TTO) The patch requires attention from Jaime.