[07:37:50] !log decommission analytics1033/34 from the Hadoop cluster [07:37:51] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:44:12] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Decommission old Hadoop worker nodes and add newer ones - https://phabricator.wikimedia.org/T209929 (10elukey) [11:23:40] PROBLEM - HDFS corrupt blocks on an-master1001 is CRITICAL: 6 ge 5 https://grafana.wikimedia.org/dashboard/db/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=39&fullscreen [11:24:39] this is probably temporary due to the decom, 6 blocks only [17:35:58] PROBLEM - HDFS corrupt blocks on an-master1001 is CRITICAL: 5 ge 5 https://grafana.wikimedia.org/dashboard/db/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=39&fullscreen [19:08:48] RECOVERY - HDFS corrupt blocks on an-master1001 is OK: (C)5 ge (W)2 ge 0 https://grafana.wikimedia.org/dashboard/db/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=39&fullscreen [20:21:14] gooood