[09:12:57] <jynus>	 db1035 issues only transferred to db1044, including swapping :-(
[10:22:22] <springle>	 jynus: need anything with s4 alerts?
[10:22:44] <jynus>	 no, go back to sleep, its the schema change, worked on all dbs but on commons
[10:22:52] <jynus>	 (doh)
[10:22:52] <springle>	 ah ok :)
[10:23:12] <jynus>	 just long tail
[10:23:36] * springle away
[11:07:10] <jynus>	 db1068 failed nicely, db1056 and db1059 could keep up but db1064 is still recovering after an hour
[11:07:51] <jynus>	 I've depool it, will wait for connection saturation to finish
[18:21:33] <jynus>	 as pt-osc failed due to metadata locking, I'm using pt-osc still but in a rolling-way. Just updated db1064. Could be repooled at 100% but too late for me, Others still not altered. Context: T90300
[18:27:06] <jynus>	 i know you are advancing with db1047, but just watching that server is painful and is begging for mercy
[23:44:09] <springle>	 with mediawiki load balancing, metadata locking can be avoided with brief (~30s) sequence: stop slave -> connections move away -> start pt-osc -> start slave
[23:45:05] <springle>	 https://wikitech.wikimedia.org/wiki/MariaDB#Table_Metadata_Locking