[06:08:12] 10DBA, 10Orchestrator, 10Patch-For-Review, 10User-Kormat: Enable report_host for mariadb - https://phabricator.wikimedia.org/T266483 (10Marostegui) [06:25:44] 10DBA, 10Patch-For-Review: Productionize es20[26-34] and es10[26-34] - https://phabricator.wikimedia.org/T261717 (10Marostegui) On-going transfers: es1018 -> es1032 (es1) es1015 -> es1033 (es2) es1019 -> es1034 (es3) [08:05:39] 10Blocked-on-schema-change, 10DBA: Drop default of revactor_timestamp - https://phabricator.wikimedia.org/T267767 (10Marostegui) p:05Triage→03Medium @Ammarpad that link results in 404, would you mind checking it? (The gerrit patch URL also works for us!) [10:42:29] 10DBA, 10Operations, 10Patch-For-Review, 10User-Kormat, 10User-jbond: Refactor mariadb puppet code - https://phabricator.wikimedia.org/T256972 (10Kormat) [11:31:34] jynus, sobanski: `ssh -L3000:0:3000 dborch1001.eqiad.wmnet`, then point your browser at localhost:3000/ [11:32:57] * volans peekingtoo [11:33:26] * kormat puts up a "No Volanses" splashpage [11:37:41] the command line tools would work on dborch1001, right? [11:38:30] yes, but `orhcestrator` will ignore read-only, and allow you to break production [11:38:38] `orchestrator-client` however honors read-only [11:38:39] oh, ok, then I will not use it [11:38:45] ah, then I will use that one [11:38:51] thanks, kormat [11:38:56] yw [11:40:07] last question: what hint should I use for "orchestrator-client -c topology" for example? [11:40:29] -i pc1 ? [11:40:41] yeah, that worked [11:42:57] sobanski: re: INSERT I am personally satisfied with Krinkle's comments because the issue happened after refactoring, but that module will be moved to a separate cluster to "properly" fix it long term [12:25:56] 10Blocked-on-schema-change, 10DBA: Drop default of revactor_timestamp - https://phabricator.wikimedia.org/T267767 (10Ladsgroup) >>! In T267767#6623082, @Marostegui wrote: > @Ammarpad that link results in 404, would you mind checking it? (The gerrit patch URL also works for us!) We removed that file later. You... [13:09:53] 10Blocked-on-schema-change, 10DBA, 10Operations, 10User-Kormat: Schema change to make change_tag.ct_rc_id unsigned - https://phabricator.wikimedia.org/T259831 (10Kormat) [13:36:38] 10Blocked-on-schema-change, 10DBA: Drop default of revactor_timestamp - https://phabricator.wikimedia.org/T267767 (10Marostegui) Thanks - I am going to update the task with that link. [13:36:52] 10Blocked-on-schema-change, 10DBA: Drop default of revactor_timestamp - https://phabricator.wikimedia.org/T267767 (10Marostegui) [14:09:07] 10DBA, 10Orchestrator, 10Patch-For-Review, 10User-Kormat: Enable report_host for mariadb - https://phabricator.wikimedia.org/T266483 (10Marostegui) pc1 master done: ` root@pc1007:~# mysql -e "select @@report_host" +--------------------+ | @@report_host | +--------------------+ | pc1007.eqiad.wmnet | +... [14:09:24] 10DBA, 10Orchestrator, 10Patch-For-Review, 10User-Kormat: Enable report_host for mariadb - https://phabricator.wikimedia.org/T266483 (10Marostegui) [14:49:12] 10DBA, 10Orchestrator: Investigate hostname/fqdn handling in orchestrator - https://phabricator.wikimedia.org/T267929 (10Kormat) [14:49:58] 10DBA, 10Orchestrator: Investigate hostname/fqdn handling in orchestrator - https://phabricator.wikimedia.org/T267929 (10Kormat) [14:58:56] 10DBA, 10Orchestrator: Investigate hostname/fqdn handling in orchestrator - https://phabricator.wikimedia.org/T267929 (10Kormat) [14:59:33] 10DBA, 10Orchestrator: Investigate hostname/fqdn handling in orchestrator - https://phabricator.wikimedia.org/T267929 (10Marostegui) [14:59:35] 10DBA, 10Orchestrator: Orchestrator doesn't use FQDN when manipulating replicas - https://phabricator.wikimedia.org/T267389 (10Marostegui) [16:13:59] 10DBA, 10Beta-Cluster-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: Upgrade the MediaWiki servers to ICU 63 - https://phabricator.wikimedia.org/T264991 (10ArielGlenn) Note that I ran a little dumps test on a non-latin1 wiki in deployment-prep (ruwiki to be precise) and the results look... [17:45:11] 10DBA, 10Datasets-General-or-Unknown, 10Patch-For-Review, 10Sustainability (Incident Followup), 10WorkType-NewFunctionality: Automate the check and fix of object, schema and data drifts between mediawiki HEAD, production masters and slaves - https://phabricator.wikimedia.org/T104459 (10Ladsgroup) [18:06:29] 10DBA, 10Operations, 10ops-eqiad: db1139 memory errors on boot (issue continues after board change) 2020-08-27 - https://phabricator.wikimedia.org/T261405 (10wiki_willy) @Jclark-ctr - can you double-check the S/N for db1139. We're getting the following Netbox error: mismatched serials: MXQ91300JF (netbox) !... [18:13:50] 10DBA, 10Operations, 10ops-eqiad: db1139 memory errors on boot (issue continues after board change) 2020-08-27 - https://phabricator.wikimedia.org/T261405 (10Cmjohnson) @wiki_willy I do not know what the Q number would be, all of the HP servers start with MXQ and confirmed MXQ91300JF is correct. [23:31:20] 10DBA, 10Beta-Cluster-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: Upgrade the MediaWiki servers to ICU 63 - https://phabricator.wikimedia.org/T264991 (10jijiki) We upgraded to ICU 63: appservers, api, parsoid, jobrunners, mwmaint, and snapshot. What is left is deploy*. We are runnin... [23:32:24] 10DBA, 10Beta-Cluster-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: Upgrade the MediaWiki servers to ICU 63 - https://phabricator.wikimedia.org/T264991 (10jijiki)