[07:23:04] 10DBA: replication broken on db1124:3318 on wikidata.pagelinks - https://phabricator.wikimedia.org/T209521 (10Marostegui) Another breakage happened today: ` 05:28 <+icinga-wm> PROBLEM - MariaDB Slave SQL: s8 on db1124 is CRITICAL: CRITICAL slave_sql_state Slave_SQL_Running: No, Errno: 1032, Errmsg: Could not exe... [10:30:34] 10DBA: replication broken on db1124:3318 on wikidata.pagelinks - https://phabricator.wikimedia.org/T209521 (10Banyek) Maybe we should reimport the table in january? [12:03:09] 10DBA: replication broken on db1124:3318 on wikidata.pagelinks - https://phabricator.wikimedia.org/T209521 (10Marostegui) >>! In T209521#4843541, @Banyek wrote: > Maybe we should reimport the table in january? Yeah, check this https://phabricator.wikimedia.org/T212574#4842526 [15:19:31] 10DBA, 10Cloud-Services, 10Operations, 10cloud-services-team: Prepare and check storage layer for hywwiki - https://phabricator.wikimedia.org/T212625 (10MarcoAurelio) [15:28:23] 10DBA, 10Data-Services, 10Operations, 10cloud-services-team: Prepare and check storage layer for hywwiki - https://phabricator.wikimedia.org/T212625 (10Marostegui) p:05Triage→03Normal Let us know when this is created so we can sanitize it and give the green light to Cloud Team! Thanks [16:02:20] Feliz Navidad marostegui :) [16:02:26] Igualmente! [17:19:36] 10DBA: Recheck wikidatawiki.pagelinks table across all hosts in s8 - https://phabricator.wikimedia.org/T212574 (10Marostegui) I have proactively manually fixed a few chunks and ran the compare again between db1124:3318 and db1087: ` Execution ended, a total of 296 chunk(s) are different. `