[00:15:33] DBA, Community-Tech, MediaWiki-extensions-PageAssessments, Reports-bot, Schema-change: Create tables for PageAssessments in enwiki database - https://phabricator.wikimedia.org/T139552#2485966 (kaldari) According to Reedy, all I have to do is: 1. ssh to terbium 2. Run: ``` mwscript sql.php... [03:16:49] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486108 (jcrespo) Binary log at the time of the master change: ``` MariaDB db1043.eqiad.wmnet (none) > SHOW MASTER STATUS\G *************************** 1. row ********... [03:54:39] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486132 (jcrespo) db1048 is now the new m3 master, and it is being used though the proxy dbproxy1003. [05:56:17] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2486171 (jcrespo) [05:56:20] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486169 (jcrespo) Open>Resolved m3 servers are all now on jessie/10, and we are no longer in degraded/reduced redundancy mode. I've left a copy of the old maste... [05:59:55] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2486176 (jcrespo) T123525#2486173 [06:00:04] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2486178 (jcrespo) Open>Resolved [06:13:10] DBA, Operations, Patch-For-Review: reimage or decom db servers on precise - https://phabricator.wikimedia.org/T125028#2486189 (jcrespo) [06:13:14] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486187 (jcrespo) Resolved>Open I forgot we need to reenable slave jobs. [06:19:00] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486192 (mmodell) [07:49:10] DBA, Operations, Phabricator, Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2486271 (jcrespo) @mmodell @demon @dzahn I've lost track after so many enables and disables of jobs. I think the only pending thing to enable is: https://gerrit.wikimedi... [08:44:00] odd, for some reason I don't see wikibugs messages colored here, on other channels they appear with colors [08:47:33] maybe it is a configuration I didn't enable? [09:24:06] possible yeah, where is wikibugs configured from? [09:26:03] it could be channel configuration [10:28:43] it is channel config [10:35:32] [11:32:53] [ChanServ] Mode lock : +ntc-slk [10:35:40] jynus: I think you need to remove that +c [10:37:12] /mode #wikimedia-databases -c [13:21:54] 10DBA, 06Operations, 10ops-eqiad: dbstore1002 disk errors - https://phabricator.wikimedia.org/T140337#2486908 (10jcrespo) 05Open>03Resolved It finished, no media errors. [13:22:18] oh, colors, Reedy, so pretty [13:22:25] ^godog [13:27:52] heh :) [13:31:20] haha neat jynus [13:31:22] thanks Reedy [13:32:49] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Users can't run EXPLAIN queries to check the theoretical efficiency of their SQL - https://phabricator.wikimedia.org/T141095#2486960 (10chasemp) [13:33:10] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Users can't run EXPLAIN queries to check the theoretical efficiency of their SQL - https://phabricator.wikimedia.org/T141095#2486960 (10chasemp) p:05Triage>03Normal [13:36:16] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Having lots of accounts with separate grants makes auditing difficult. - https://phabricator.wikimedia.org/T141096#2486986 (10chasemp) [13:36:38] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Having lots of accounts with separate grants makes auditing difficult. - https://phabricator.wikimedia.org/T141096#2486986 (10chasemp) p:05Triage>03Normal [14:00:31] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: labsdb* has no HA solution - https://phabricator.wikimedia.org/T141097#2487045 (10chasemp) [14:02:48] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: labsdb* has no HA solution - https://phabricator.wikimedia.org/T141097#2487079 (10jcrespo) I think that, of all things to fix, this one is a hard blocker for T140452. [14:06:55] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: Users can't run EXPLAIN queries to check the theoretical efficiency of their SQL - https://phabricator.wikimedia.org/T141095#2486960 (10AlexMonk-WMF) Duplicate of {T50875}? [15:46:02] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: labsdb* has no High Availability solution - https://phabricator.wikimedia.org/T141097#2487451 (10bd808) [15:48:55] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: labsdb* has no High Availability solution - https://phabricator.wikimedia.org/T141097#2487492 (10chasemp) [16:41:57] 10DBA, 10MediaWiki-extensions-ContentTranslation, 07Performance, 07WorkType-Maintenance: Investigate increase in "readonly" saving errors in Content Translation - https://phabricator.wikimedia.org/T141090#2487770 (10aaron) [16:46:09] 10DBA, 06Operations, 10Phabricator, 13Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2487806 (10Dzahn) >>! In T138460#2486271, @jcrespo wrote: > Once that is deployed, we can set this as resolved. @jcrespo thank you! has been deployed, the cron has been c... [16:49:12] 10DBA, 06Operations, 10Phabricator, 13Patch-For-Review: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2487812 (10Dzahn) 05Open>03Resolved setting to resolved per Jaime's comment. (and since last phabricator upgrade this should not mean i automatically claim the task ) [16:49:47] 10DBA, 06Operations, 10Phabricator: Upgrade m3 (phabricator) db servers - https://phabricator.wikimedia.org/T138460#2487818 (10Dzahn) [18:25:21] 10DBA, 06Labs, 10Labs-Infrastructure, 07Epic, 07Tracking: labsdb* has no High Availability solution - https://phabricator.wikimedia.org/T141097#2488317 (10chasemp)