[04:14:09] 10DBA, 10Data-Services: Table field templatelink.tl_from in database does not always match page.page_namespace - https://phabricator.wikimedia.org/T217647 (10bd808) This does not seem to be specific to the Wiki Replicas. It looks like this is some data corruption/drift in the production database tables: ` wiki... [05:07:21] 10DBA: Decommission old coredb machines (<=db2042) - https://phabricator.wikimedia.org/T221533 (10Marostegui) [05:14:23] 10DBA, 10Goal, 10Patch-For-Review: Productionize db11[26-38] - https://phabricator.wikimedia.org/T222682 (10Marostegui) [05:41:03] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) [05:41:31] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) [05:41:36] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) p:05Triage→03Normal [05:42:20] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) [05:43:04] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) codfw hosts have been all upgraded to 10.1.39 [05:47:52] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) eqiad hosts that need upgrade [] db1081 [] db1084 [] db1091 [] db1097 [] db1103 [] db1121 [] db1125 [] labsdb1012 [] dbstore1004 [05:50:39] 10DBA, 10Goal: Address Database infrastructure blockers on datacenter switchover & multi-dc deployment - https://phabricator.wikimedia.org/T220170 (10Marostegui) [05:50:41] 10DBA, 10Operations: Failover s4 primary master: db1068 to db1081 - https://phabricator.wikimedia.org/T224852 (10Marostegui) [07:30:16] 10DBA: Failover DB masters in row D - https://phabricator.wikimedia.org/T186188 (10jcrespo) [07:32:43] 10DBA, 10Operations: Decommission db1061-db1073 - https://phabricator.wikimedia.org/T217396 (10jcrespo) [07:52:01] 10DBA: Decommission db1064 - https://phabricator.wikimedia.org/T223217 (10Marostegui) [08:03:38] 10DBA, 10Patch-For-Review: Decommission db1064 - https://phabricator.wikimedia.org/T223217 (10Marostegui) [08:04:23] 10DBA, 10Operations: Decommission db1061-db1073 - https://phabricator.wikimedia.org/T217396 (10Marostegui) [09:56:48] 10DBA, 10Operations: Predictive failures on disk S.M.A.R.T. status - https://phabricator.wikimedia.org/T208323 (10Marostegui) [11:55:25] 10DBA, 10Tool-wikiloves: Some images have no `img_user` in the `image` table in Commons Wiki replicas - https://phabricator.wikimedia.org/T224862 (10JeanFred) p:05High→03Normal I played with and stripped down the query until isolating the issue, and the question boils down to: Why would `img_user` (and `... [13:55:02] marostegui and jynus: I have an especially unpopular bunch of replica view updates to launch today and tomorrow. Is the compression/maintenance work still going on? [13:55:19] no, I paused it on purpose [13:55:28] I upgraded the server version, though [13:55:31] nope, replicas are all yours :) [14:07:15] Cool :) [14:07:37] bstorm_: let me know when 1012 is fully done, as I am coordinating with luca to upgrade it too [14:07:46] Ok, I'll do that first [14:07:55] ta! [15:07:36] marostegui: Done with 1012 [15:07:54] bstorm_: thank you! [17:53:53] 10DBA, 10Wikimedia-Site-requests: Global renaming of B_dash - https://phabricator.wikimedia.org/T224916 (10mys_721tx)