[07:16:47] <wikibugs>	 10DBA: db1130 BBU possible issues - https://phabricator.wikimedia.org/T240823 (10Marostegui) 05Open→03Resolved a:03Marostegui I am going to close this as resolved, as it seems it has stabilized (for now at least): `  root@db1130:~# megacli -AdpBbuCmd -a0  BBU status for Adapter: 0  BatteryType: BBU Voltage...
[07:23:07] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Extend flow_wiki_ref.ref_src_wiki - https://phabricator.wikimedia.org/T241387 (10Marostegui) List of wikis that have this table:  s1 - none s2 - none s3: ** officewiki ** techconductwiki s4; none  s5: none s6: none s7: none s8: none x1: `flowdb` s10: `labswiki`  @Andrew I gue...
[07:24:39] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Extend flow_wiki_ref.ref_src_wiki - https://phabricator.wikimedia.org/T241387 (10Marostegui)
[07:40:37] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1123 - https://phabricator.wikimedia.org/T240534 (10Marostegui) Thanks for catching that Jaime, I have created this: https://wikitech.wikimedia.org/w/index.php?title=MariaDB&type=revision&diff=1849295&oldid=1847866
[07:49:08] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[08:26:16] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[10:04:02] <wikibugs>	 10DBA, 10MediaWiki-General, 10Operations, 10Patch-For-Review: Evaluate and decide the future of relational datastore at WMF after the upgrade of MariaDB 10.1 is finished - https://phabricator.wikimedia.org/T193224 (10Marostegui) I have upgraded db1107 from 10.4.10 to the latest version: 10.4.11
[10:36:49] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[10:53:12] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[11:42:57] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[11:50:49] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Extend flow_wiki_ref.ref_src_wiki - https://phabricator.wikimedia.org/T241387 (10Andrew) That table isn't present on labtestwiki -- I must not have ever installed flow there.
[11:51:11] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Extend flow_wiki_ref.ref_src_wiki - https://phabricator.wikimedia.org/T241387 (10Andrew)
[11:51:13] <wikibugs>	 10Blocked-on-schema-change, 10DBA: Extend flow_wiki_ref.ref_src_wiki - https://phabricator.wikimedia.org/T241387 (10Marostegui) Cool, thanks for confirming!
[11:53:28] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[12:19:48] <wikibugs>	 10DBA: Re-build db2097 s1 and s6 - https://phabricator.wikimedia.org/T228613 (10Marostegui)
[12:20:04] <wikibugs>	 10DBA: Re-build db2097 s1 and s6 - https://phabricator.wikimedia.org/T228613 (10Marostegui) I have renamed this, and removed the 10.3 bits
[12:21:58] <wikibugs>	 10DBA: Re-build db2097 s1 and s6 - https://phabricator.wikimedia.org/T228613 (10Marostegui) db2097 is a backup source host, so probably best not to experiment with 10.4 there until we are sure it works fine. That 10.4 test will be a goal for the next Q, so for now we can just rebuilt s1 and s6 as it is, for now...
[12:57:36] <wikibugs>	 10DBA, 10Operations, 10Puppet, 10User-jbond: DB: perform rolling restart of mariadb deamons to pick up CA changes - https://phabricator.wikimedia.org/T239791 (10Marostegui)
[13:09:32] <wikibugs>	 10DBA, 10Operations: Upgrade BIOS and firmware on db2076 - https://phabricator.wikimedia.org/T241647 (10Marostegui)
[13:10:08] <wikibugs>	 10DBA, 10Operations, 10ops-codfw: Upgrade BIOS and firmware on db2076 - https://phabricator.wikimedia.org/T241647 (10Marostegui) p:05Triage→03Normal a:03Papaul
[13:11:53] <wikibugs>	 10DBA: Re-build db2097 s1 and s6 - https://phabricator.wikimedia.org/T228613 (10jcrespo) I agree we should keep the backup sources either the same version as the master or as >50% of the replicas (aka "upgrade it at the same time as the master"). However, if we decide to change a whole section (e.g. s1 and s6 on...