[00:43:56] 10DBA, 06Labs, 10Labs-Infrastructure: Data integrity issue with enwiki_p user_groups on Wikimedia Tool Labs (missing rows) - https://phabricator.wikimedia.org/T159493#3072494 (10MZMcBride) Thanks for the quick fix! I replied to the two users who were asking about the issue: 10DBA, 10MediaWiki-Database, 10MediaWiki-Logging, 06Performance-Team, and 2 others: Logging needs an index to optimize searching by log_title - https://phabricator.wikimedia.org/T68961#3072641 (10Huji) @Umherirrender thanks for clarifying. BloomCache is not an option so we are back to square one. [08:10:41] 10DBA, 13Patch-For-Review: Install and reimage dbstore1001 as jessie - https://phabricator.wikimedia.org/T153768#3072810 (10Marostegui) They are definitely not happening so we will need to run them manually on Monday: ``` root@helium:/var/log/bacula# echo "list jobs" | bconsole | grep dbstore1001 | grep 2017 |... [08:17:10] 10DBA, 06Operations, 10ops-eqiad: Degraded RAID on db1060 - https://phabricator.wikimedia.org/T158193#3072811 (10Marostegui) It finished its rebuilt - so we can go ahead and replace #7: ``` root@db1060:~# megacli -PDRbld -ShowProg -PhysDrv [32:4] -aALL Device(Encl-32 Slot-4) is not in rebuild process Exit... [15:41:22] 10DBA, 06Labs: labsdb1004 MySQL crash - https://phabricator.wikimedia.org/T159572#3073002 (10Marostegui) Looks like that database has been involved on a few things already: https://phabricator.wikimedia.org/T150759 https://phabricator.wikimedia.org/T131897 So maybe those privileges were there as a precaution...