[10:11:55] DBA, MediaWiki-Database, MediaWiki-Special-pages, Wikimedia-log-errors: Special:Contributions: "contributions page filtered for namespace or RevisionDeleted edits" error - https://phabricator.wikimedia.org/T108782#1530549 (Multichill) https://commons.wikimedia.org/w/index.php?title=Special:Contri... [10:17:20] DBA, MediaWiki-Database, MediaWiki-Special-pages, Wikimedia-log-errors: Special:Contributions: "contributions page filtered for namespace or RevisionDeleted edits" error - https://phabricator.wikimedia.org/T108782#1530549 (jcrespo) @Multichill I think in your case it was a 1 in a million error, I... [10:24:49] DBA, MediaWiki-Database, MediaWiki-Special-pages, Wikimedia-log-errors: Special:Contributions: "contributions page filtered for namespace or RevisionDeleted edits" error - https://phabricator.wikimedia.org/T108782#2398611 (jcrespo) The general issue is that there is literally (this is exact, we c... [10:29:18] DBA, Labs, Labs-Infrastructure, Operations: Migrate labsdb1005/1006/1007 to jessie - https://phabricator.wikimedia.org/T123731#1936600 (jcrespo) There is indeed a replacement for labsdb100[123] about to arrive. However, there are no short-term plans for these, as they have lower impact. labsdb10... [10:48:58] DBA, Beta-Cluster-Infrastructure, Flow, Collab-Team-2016-Apr-Jun-Q4: Run Flow External Store migration in dry-run mode on Beta - https://phabricator.wikimedia.org/T119567#2398659 (jmatazzoni) Open>Resolved [10:51:01] DBA, Beta-Cluster-Infrastructure, Flow, Collab-Team-2016-Apr-Jun-Q4: Run Flow External Store migration in dry-run mode on Beta - https://phabricator.wikimedia.org/T119567#2398664 (jmatazzoni) [10:51:48] DBA, Beta-Cluster-Infrastructure, Flow, Collab-Team-2016-Apr-Jun-Q4: Run Flow External Store migration in dry-run mode on Beta - https://phabricator.wikimedia.org/T119567#1829837 (jmatazzoni) [10:53:40] DBA, Collaboration-Team-Interested, Flow, Operations, WorkType-Maintenance: Setup separate logical External Store for Flow in production - https://phabricator.wikimedia.org/T107610#2398667 (jcrespo) I have not forgotten about this, it is on 'Next', blocked on me having proper time (there is n... [13:20:44] DBA, Datasets-General-or-Unknown, Patch-For-Review: Select of revisions for stub history files does not explicitly order revisions - https://phabricator.wikimedia.org/T29112#2398990 (ArielGlenn) The above patchset is now ready for review. Any takers? [13:58:52] DBA, Labs: write irc bot to report high replag of s{1,2,3}.labsdb on #wikimedia-labsdb - https://phabricator.wikimedia.org/T106151#1460476 (Boshomi) Please notice T138378 ; actually 73970 sec on s1 [17:53:43] DBA, Labs, Labs-Infrastructure, Operations, Blocked-on-Operations: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400045 (RobH) [18:19:21] DBA, Operations: m1-master switch from db1001 to db1016 - https://phabricator.wikimedia.org/T106312#2400123 (jcrespo) [18:19:23] DBA, Operations, Patch-For-Review: Upgrade m1 db servers - https://phabricator.wikimedia.org/T135973#2400122 (jcrespo) [18:20:46] DBA, Operations: m1-master switch from db1001 to db1016 - https://phabricator.wikimedia.org/T106312#1464530 (jcrespo) These are the notes from the migration, to be documented on wiki: * bacula ; sudo service bacula-director restart after the migration. I had already made sure no jobs were running with s... [18:51:35] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400172 (ori) Sample invocation: ```lang=bash $ DB_USER=root DB_PASS=password perl maintain-replicas.pl 2> T13... [19:42:23] DBA, Operations, Patch-For-Review: Upgrade m1 db servers - https://phabricator.wikimedia.org/T135973#2400248 (jcrespo) [19:42:25] DBA, Operations, Patch-For-Review: m1-master switch from db1001 to db1016 - https://phabricator.wikimedia.org/T106312#2400246 (jcrespo) Open>Resolved db1016 is the new master of m1. [19:48:26] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2400265 (jcrespo) After m1 failover, the only precise hosts left are: db1043.eqiad.wmnet: True db1048.eqiad.wmnet: True Which are m3 (phabricator) db hosts, and it requires #phabric... [19:50:34] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2400276 (jcrespo) >>! In T125028#2400265, @jcrespo wrote: > After m1 failover, the only precise hosts left are: > > db1043.eqiad.wmnet: True > db1048.eqiad.wmnet: True > > Which are... [22:28:54] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400629 (ori) I ran it against labsdb1001 and labsdb1003 and it seems to have done the trick. labsdb1002's MySQ... [22:34:42] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400632 (MaxSem) Confirmed working. Let's keep this bug open until labsdb1002 also gets fixed. [22:36:57] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400639 (Krenair) Thank you @ori! I believe the issues with labsdb1002 are {T126946} [23:23:49] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2400711 (mmodell) @jcrespo: The not-exactly-official list of phabricator admins would be myself, @demon and @Aklapper. I don't think there should be any issue with moving phabricator... [23:28:15] DBA, Labs, Labs-Infrastructure: maintain-replicas.pl unmaintained, unmaintainable - https://phabricator.wikimedia.org/T138450#2400728 (ori) [23:29:53] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400771 (ori) Filed {T138450} for tracking issues with the script. [23:30:28] DBA, Labs, Labs-Infrastructure: maintain-replicas.pl unmaintained, unmaintainable - https://phabricator.wikimedia.org/T138450#2400728 (ori) [23:30:30] DBA, Labs, Labs-Infrastructure, Operations, and 2 others: adywiki and jamwiki are missing the associated *_p databases with appropriate views - https://phabricator.wikimedia.org/T135029#2400777 (ori) [23:30:57] DBA, Labs, Labs-Infrastructure, Blocked-on-Operations: maintain-replicas.pl unmaintained, unmaintainable - https://phabricator.wikimedia.org/T138450#2400728 (ori) [23:43:45] DBA, Labs, Labs-Infrastructure, Blocked-on-Operations: maintain-replicas.pl unmaintained, unmaintainable - https://phabricator.wikimedia.org/T138450#2400728 (Krenair) I wouldn't call this strictly blocked-on-ops yet (since although ops would have to approve the code, the next step is writing it a...