[05:11:12] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: Decommission db2060.codfw.wmnet - https://phabricator.wikimedia.org/T231625 (10Marostegui) p:05Triage→03Normal
[05:12:03] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: Decommission db2060.codfw.wmnet - https://phabricator.wikimedia.org/T231625 (10Marostegui)
[05:12:12] <wikibugs>	 10DBA, 10Operations: Decommission db2043-db2069 - https://phabricator.wikimedia.org/T228258 (10Marostegui)
[05:23:23] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: Decommission db2060.codfw.wmnet - https://phabricator.wikimedia.org/T231625 (10Marostegui)
[05:26:56] <wikibugs>	 10DBA, 10Operations: Decommission db2043-db2069 - https://phabricator.wikimedia.org/T228258 (10Marostegui)
[08:10:15] <wikibugs>	 10DBA, 10Operations, 10wikitech.wikimedia.org, 10Patch-For-Review, 10cloud-services-team (Kanban): Switchover m5 primary master: db1073 to db1133: Tuesday 3rd Sept at 13:00 UTC - https://phabricator.wikimedia.org/T229657 (10Marostegui) Window reserved on the Deployments page. It will happen at the same t...
[08:19:29] <wikibugs>	 10DBA, 10Operations, 10wikitech.wikimedia.org, 10Patch-For-Review, 10cloud-services-team (Kanban): Switchover m5 primary master: db1073 to db1133: Tuesday 3rd Sept at 13:00 UTC - https://phabricator.wikimedia.org/T229657 (10Marostegui)
[08:19:50] <wikibugs>	 10DBA, 10Operations, 10wikitech.wikimedia.org, 10Patch-For-Review, 10cloud-services-team (Kanban): Switchover m5 primary master: db1073 to db1133: Tuesday 3rd Sept at 13:00 UTC - https://phabricator.wikimedia.org/T229657 (10Marostegui)
[08:20:46] <wikibugs>	 10DBA, 10Operations, 10wikitech.wikimedia.org, 10Patch-For-Review, 10cloud-services-team (Kanban): Switchover m5 primary master: db1073 to db1133: Tuesday 3rd Sept at 13:00 UTC - https://phabricator.wikimedia.org/T229657 (10Marostegui)
[10:14:06] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Dzahn)
[10:18:09] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Dzahn)
[10:25:56] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Marostegui) [10:23:40]  <jynus> !log reseting db1074 from iLo
[10:26:09] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10jcrespo) On reboot: ` 313-HPE Smart Storage Battery 1 Failure - Battery Shutdown Event Code: 0x0400. Action: Restart system. Contact HPE support if condition persists.    Important information available or errors detected `
[10:30:04] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Marostegui) >>! In T231638#5453726, @jcrespo wrote: > On reboot: > ` > 313-HPE Smart Storage Battery 1 Failure - Battery Shutdown Event Code: 0x0400. > Action: Restart system. Contact HPE support if condition persists. >  >  >...
[10:31:32] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Marostegui)
[10:35:14] <wikibugs>	 10DBA, 10Operations: db1074 crashed - https://phabricator.wikimedia.org/T231638 (10Marostegui) BBU is broken: `     description=Smart Storage Battery failure (Battery 1, service information: 0x0A). Action: Gather AHS log and contact Support   Verbs     cd vers     description=POST Error: 313-HPE Smart Storage...
[10:36:06] <wikibugs>	 10DBA, 10Operations: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10Marostegui)
[10:53:11] <wikibugs>	 10DBA, 10Operations: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10Marostegui)
[15:42:09] <wikibugs>	 10DBA, 10conftool: #dbctl: manage 'externalLoads' data - https://phabricator.wikimedia.org/T229686 (10CDanis) In db-eqiad/codfw.php we currently provide IP addresses as the values of the keys in `externalLoads` instead of using `hostsByName` to translate hostnames to IP, e.g.: `lang=php 'externalLoads' => [  #...
[16:03:09] <wikibugs>	 10DBA, 10conftool, 10Performance-Team (Radar): #dbctl: manage 'externalLoads' data - https://phabricator.wikimedia.org/T229686 (10Krinkle)
[16:23:49] <wikibugs>	 10DBA, 10Operations, 10StructuredDiscussions, 10Growth-Team (Current Sprint), 10WorkType-Maintenance: Setup separate logical External Store for Flow in production - https://phabricator.wikimedia.org/T107610 (10Catrope) >>! In T107610#5443335, @Anomie wrote: > Or do we want one table that's shared by all...
[16:40:32] <wikibugs>	 10DBA, 10MediaWiki-Configuration, 10Operations, 10Patch-For-Review, and 3 others: Create tool to handle the state of database configuration in MediaWiki in etcd - https://phabricator.wikimedia.org/T197126 (10CDanis)
[17:24:38] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10wiki_willy) a:03Cmjohnson @Cmjohnson @Jclark-ctr - do you guys know offhand if we have a spare BBU lying around from a decom'd server by any chance?  If not, let me know and we'll order the part.  Th...
[17:25:59] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10Cmjohnson) @wiki_willy negative, we do not have any spare BBUs lying around.
[17:31:18] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: Request to Order HP BBU for HP ProLiant DL360 Gen9 (db1074) - https://phabricator.wikimedia.org/T231670 (10wiki_willy)
[17:31:56] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10wiki_willy) Thanks for confirming @Cmjohnson , subtask T231670 created for Rob to order the part.  Thanks, Willy
[17:34:07] <wikibugs>	 10DBA, 10Operations, 10ops-eqiad: db1074 crashed: Broken BBU - https://phabricator.wikimedia.org/T231638 (10Marostegui) >>! In T231638#5454666, @wiki_willy wrote: > Thanks for confirming @Cmjohnson , subtask T231670 created for Rob to order the part.  Thanks, Willy  Thank you sooo much :)
[17:40:20] <wikibugs>	 10DBA, 10Operations, 10procurement: Request to Order HP BBU for HP ProLiant DL360 Gen9 (db1074) - https://phabricator.wikimedia.org/T231670 (10wiki_willy)
[17:42:14] <wikibugs>	 10DBA, 10Operations, 10procurement: eqiad: Request to Order HP BBU for HP ProLiant DL360 Gen9 (db1074) - https://phabricator.wikimedia.org/T231670 (10wiki_willy)
[20:23:46] <wikibugs>	 10DBA, 10conftool, 10Performance-Team (Radar): #dbctl: manage 'externalLoads' data - https://phabricator.wikimedia.org/T229686 (10CDanis) Here's a mini-design proposal for the dbctl feature itself (@Volans and @Joe please review):  * Add a `flavor` enum to the section schema.  The default will be `regular`,...
[20:35:41] <wikibugs>	 10DBA, 10Operations, 10procurement: eqiad: order replacement bbu for db1074 (hp dl360 gen9) - https://phabricator.wikimedia.org/T231670 (10RobH)
[20:47:25] <wikibugs>	 10DBA, 10conftool, 10Performance-Team (Radar): #dbctl: manage 'externalLoads' data - https://phabricator.wikimedia.org/T229686 (10Volans) >>! In T229686#5455125, @CDanis wrote: > Here's a mini-design proposal for the dbctl feature itself (@Volans and @Joe please review): >  > * Add a `flavor` enum to the sec...