[18:27:23] <wikibugs>	 10DBA, 10Operations: db1082 crashed - https://phabricator.wikimedia.org/T258336 (10wkandek) Should dbtree reflect the replication lag on db1124?
[18:46:18] <wikibugs>	 10DBA, 10Operations: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10CDanis)
[18:47:15] <wikibugs>	 10DBA, 10Operations: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10CDanis) p:05Triage→03High
[18:50:15] <wikibugs>	 10DBA, 10Operations: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10Marostegui) BBU issues as expected. This host is also scheduled to be refreshed next Q: ` /system1/log1/record16   Targets   Properties     number=16     severity=Caution     date=07/19/2020     time=18:44     description=POST E...
[18:54:35] <wikibugs>	 10DBA, 10Operations: Refresh and decommission db1074-db1095 (22 servers) - https://phabricator.wikimedia.org/T258361 (10Marostegui)
[18:59:33] <wikibugs>	 10DBA, 10Operations: db1082 crashed - https://phabricator.wikimedia.org/T258336 (10Marostegui) I have rebooted the host, the BBU looks ok. MySQL started ok as well, did the recovery finely too. Replication is started.
[19:12:35] <wikibugs>	 10DBA, 10Operations: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10Marostegui) And the BBU is gone: ` root@db1085:~#  hpssacli controller all show detail | grep -i Battery    No-Battery Write Cache: Disabled    Battery/Capacitor Count: 0 `
[19:13:57] <wikibugs>	 10DBA, 10Operations: Refresh and decommission db1074-db1095 (22 servers) - https://phabricator.wikimedia.org/T258361 (10Marostegui) p:05Triage→03Medium
[19:17:00] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10Marostegui)
[19:20:28] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10Marostegui)
[19:26:46] <wikibugs>	 10DBA, 10Operations, 10Patch-For-Review: db1085 crashed - https://phabricator.wikimedia.org/T258360 (10Marostegui) Host upgraded and rebooted. MySQL looks ok, replication started
[21:57:58] <wikibugs>	 10DBA, 10Operations: db1082 crashed - https://phabricator.wikimedia.org/T258336 (10wkandek) PST         UTC Lag         Minutes Lag Minutes Lag Decay Rate/minute 12:30 19:30 17:43     12:56 19:56 12:44 0:26 299                 11.50 13:42 20:42 5:38 0:46 426                   9.26 13:52 20:52 4:31 0:10 67...