[10:33:32] <wikibugs>	 10DBA, 10MediaWiki-extensions-WikibaseClient, 10Operations, 10Wikidata, and 5 others: Cache invalidations coming from the JobQueue are causing lag on several wikis - https://phabricator.wikimedia.org/T164173#3521978 (10Marostegui) @daniel @hoo and myself got together to talk about this yesterday and looks...
[11:52:00] <wikibugs>	 10DBA, 10Operations: Decommission db1015, db1035, db1044 and db1038 - https://phabricator.wikimedia.org/T148078#3521999 (10Marostegui) I will try to run pt-table-checksum on the most important tables on s3 next week as db1015 is quite low on disk space already :-( ```  root@db1015:~# df -hT /srv Filesystem...
[13:29:01] <wikibugs>	 10DBA, 10Data-Services: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3477438 (10chasemp) We are all either at wikimania or on vacation, I expect this will be addressed in the coming week. :)
[18:13:49] <wikibugs>	 10DBA, 10Operations, 10Wikidata: Wikidata.org currently very slow - https://phabricator.wikimedia.org/T173269#3522690 (10Legoktm)
[20:07:16] <wikibugs>	 10DBA, 10Operations, 10Wikidata: Wikidata.org currently very slow - https://phabricator.wikimedia.org/T173269#3522664 (10Marostegui) I don't see anything wrong with db1082 (or the s5 master). db1082 had some spikes but traffic is back to normal: https://grafana.wikimedia.org/dashboard/db/mysql?orgId=1&var-dc...
[20:13:34] <wikibugs>	 10DBA, 10Operations, 10Wikidata: Wikidata.org currently very slow - https://phabricator.wikimedia.org/T173269#3522952 (10Marostegui) I am seeing one disk on db1063 (the master) with errors and predictive failure, but it has been on predictive failure since 15th Jul, and the raid is OPTIMAL status.  As the tr...