[00:14:47] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 52037MiB (11% inode=99%); [00:20:51] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and dnsutils on changeprop151 [00:20:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:03] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on bots171 [00:21:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:05] RECOVERY - bots171 APT on bots171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:21:16] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and dnsutils on eventgate181 [00:21:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:27] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on jobchron171 [00:21:27] RECOVERY - jobchron171 APT on jobchron171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:21:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:38] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db171 [00:21:40] RECOVERY - changeprop151 APT on changeprop151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:21:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:50] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db172 [00:21:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:02] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mem151 [00:22:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:14] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db151 [00:22:14] RECOVERY - db151 APT on db151 is OK: APT OK: 42 packages available for upgrade (0 critical updates). [00:22:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:26] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db161 [00:22:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:27] RECOVERY - db161 APT on db161 is OK: APT OK: 41 packages available for upgrade (0 critical updates). [00:22:38] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cp37 [00:22:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:42] RECOVERY - db171 APT on db171 is OK: APT OK: 41 packages available for upgrade (0 critical updates). [00:22:44] RECOVERY - eventgate181 APT on eventgate181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:22:50] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and dnsutils on graphite151 [00:22:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:22:52] RECOVERY - graphite151 APT on graphite151 is OK: APT OK: 65 packages available for upgrade (0 critical updates). [00:23:02] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cp36 [00:23:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:23:15] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on graylog161 [00:23:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:23:27] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on kafka181 [00:23:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:23:35] RECOVERY - db172 APT on db172 is OK: APT OK: 41 packages available for upgrade (0 critical updates). [00:23:39] RECOVERY - mem151 APT on mem151 is OK: APT OK: 58 packages available for upgrade (0 critical updates). [00:23:39] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db182 [00:23:39] RECOVERY - cp36 APT on cp36 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:23:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:23:52] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on bast181 [00:23:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:23:55] RECOVERY - graylog161 APT on graylog161 is OK: APT OK: 62 packages available for upgrade (0 critical updates). [00:24:05] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cloud18 [00:24:07] RECOVERY - cp37 APT on cp37 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:24:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:21] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on bast161 [00:24:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:33] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on ldap171 [00:24:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:45] RECOVERY - bast181 APT on bast181 is OK: APT OK: 59 packages available for upgrade (0 critical updates). [00:24:45] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cloud17 [00:24:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:58] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cloud15 [00:24:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:59] RECOVERY - ldap171 APT on ldap171 is OK: APT OK: 58 packages available for upgrade (0 critical updates). [00:25:03] RECOVERY - kafka181 APT on kafka181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:25:10] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on db181 [00:25:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:20] RECOVERY - cloud17 APT on cloud17 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [00:25:20] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and dnsutils on mattermost1 [00:25:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:30] RECOVERY - cloud18 APT on cloud18 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [00:25:31] RECOVERY - db182 APT on db182 is OK: APT OK: 42 packages available for upgrade (0 critical updates). [00:25:35] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on matomo151 [00:25:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:41] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 12.41, 7.08, 4.02 [00:25:48] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on cloud16 [00:25:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:52] RECOVERY - mattermost1 APT on mattermost1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:26:00] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mem161 [00:26:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:26:06] RECOVERY - db181 APT on db181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:26:13] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mon181 [00:26:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:26:15] RECOVERY - bast161 APT on bast161 is OK: APT OK: 58 packages available for upgrade (0 critical updates). [00:26:27] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw151 [00:26:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:26:35] RECOVERY - cloud16 APT on cloud16 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [00:26:41] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw153 [00:26:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:26:42] RECOVERY - mem161 APT on mem161 is OK: APT OK: 58 packages available for upgrade (0 critical updates). [00:26:53] RECOVERY - cloud15 APT on cloud15 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [00:26:54] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw171 [00:26:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:26:59] RECOVERY - mon181 APT on mon181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:27:07] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw152 [00:27:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:27:21] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw161 [00:27:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:27:26] RECOVERY - mw151 APT on mw151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:27:28] RECOVERY - mw161 APT on mw161 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:27:35] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw162 [00:27:36] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 3.87, 5.87, 3.96 [00:27:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:27:39] RECOVERY - mw153 APT on mw153 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:27:49] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw174 [00:27:49] RECOVERY - mw152 APT on mw152 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:27:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:27:59] RECOVERY - mw174 APT on mw174 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:28:01] RECOVERY - mw162 APT on mw162 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:28:02] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw182 [00:28:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:28:08] RECOVERY - mw182 APT on mw182 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:28:16] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw173 [00:28:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:28:23] RECOVERY - mw171 APT on mw171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:28:30] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mwtask151 [00:28:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:28:44] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw164 [00:28:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:28:51] RECOVERY - mw173 APT on mw173 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:28:58] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw163 [00:29:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:29:09] RECOVERY - mw163 APT on mw163 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:29:12] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mwtask161 [00:29:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:29:26] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw172 [00:29:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:29:31] RECOVERY - mwtask151 APT on mwtask151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:29:40] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw181 [00:29:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:29:44] RECOVERY - mw172 APT on mw172 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:29:52] RECOVERY - mw164 APT on mw164 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:29:54] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mwtask171 [00:29:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:08] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mwtask181 [00:30:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:23] RECOVERY - mwtask161 APT on mwtask161 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:30:23] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw184 [00:30:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:27] RECOVERY - mw184 APT on mw184 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:30:32] RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:30:37] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw183 [00:30:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:46] RECOVERY - mw181 APT on mw181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:30:50] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on os151 [00:30:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:31:05] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on mw154 [00:31:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:31:10] RECOVERY - os151 APT on os151 is OK: APT OK: 60 packages available for upgrade (0 critical updates). [00:31:14] RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:31:18] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on os161 [00:31:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:31:32] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on os162 [00:31:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:31:39] RECOVERY - mw183 APT on mw183 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:31:51] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on ns1 [00:31:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:32:03] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on phorge171 [00:32:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:32:14] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on prometheus151 [00:32:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:32:25] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on rdb151 [00:32:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:32:36] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on puppet181 [00:32:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:32:48] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, and bind9-libs on reports171 [00:32:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:33:01] RECOVERY - mw154 APT on mw154 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:33:09] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and dnsutils on ns2 [00:33:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:33:10] RECOVERY - os161 APT on os161 is OK: APT OK: 59 packages available for upgrade (0 critical updates). [00:33:14] RECOVERY - os162 APT on os162 is OK: APT OK: 60 packages available for upgrade (0 critical updates). [00:33:16] RECOVERY - ns1 APT on ns1 is OK: APT OK: 58 packages available for upgrade (0 critical updates). [00:33:24] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on test151 [00:33:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:33:33] RECOVERY - phorge171 APT on phorge171 is OK: APT OK: 41 packages available for upgrade (0 critical updates). [00:33:38] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftac171 [00:33:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:33:52] RECOVERY - ns2 APT on ns2 is OK: APT OK: 59 packages available for upgrade (0 critical updates). [00:33:53] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftobject151 [00:33:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:02] RECOVERY - reports171 APT on reports171 is OK: APT OK: 41 packages available for upgrade (0 critical updates). [00:34:07] RECOVERY - rdb151 APT on rdb151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:34:08] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftobject161 [00:34:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:11] RECOVERY - prometheus151 APT on prometheus151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:34:23] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftobject171 [00:34:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:29] RECOVERY - swiftobject161 APT on swiftobject161 is OK: APT OK: 62 packages available for upgrade (0 critical updates). [00:34:35] RECOVERY - swiftobject151 APT on swiftobject151 is OK: APT OK: 62 packages available for upgrade (0 critical updates). [00:34:39] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftobject181 [00:34:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:52] RECOVERY - test151 APT on test151 is OK: APT OK: 69 packages available for upgrade (0 critical updates). [00:34:53] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftproxy161 [00:34:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:35:08] RECOVERY - swiftobject171 APT on swiftobject171 is OK: APT OK: 62 packages available for upgrade (0 critical updates). [00:35:09] !log [void@puppet181] Upgraded packages bind9-host, bind9-dnsutils, bind9-libs, and libopenjp2-7 on swiftproxy171 [00:35:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:35:12] RECOVERY - swiftac171 APT on swiftac171 is OK: APT OK: 64 packages available for upgrade (0 critical updates). [00:35:39] RECOVERY - swiftobject181 APT on swiftobject181 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [00:35:45] RECOVERY - swiftproxy171 APT on swiftproxy171 is OK: APT OK: 65 packages available for upgrade (0 critical updates). [00:36:18] RECOVERY - swiftproxy161 APT on swiftproxy161 is OK: APT OK: 64 packages available for upgrade (0 critical updates). [01:15:51] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importDump.php --wiki=mythcommunitywiki robloxianmythhunters949_pages_full.xml --no-updates (END - exit=0) [01:15:52] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/rebuildall.php --wiki=mythcommunitywiki (START) [01:15:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:15:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:31] RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [01:22:52] !log [void@puppet181] apply bind9 upgrades to puppet181 and matomo151 [01:22:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:32] RECOVERY - puppet181 APT on puppet181 is OK: APT OK: 63 packages available for upgrade (0 critical updates). [01:36:47] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49810MiB (10% inode=99%); [02:09:31] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/rebuildall.php --wiki=mythcommunitywiki (END - exit=0) [02:09:32] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/initSiteStats.php --wiki=mythcommunitywiki --update (END - exit=0) [02:09:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:09:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:48:44] !log [void@cloud17.wikitide.net] clear ipmi sel [02:48:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:23] RECOVERY - cloud17 IPMI Sensors on cloud17 is OK: IPMI Status: OK [03:32:26] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/refreshLinks.php --wiki=holidayswiki (END - exit=0) [03:32:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:37:39] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/cbaad8a0867f218257b688b9ec3af46e855bb24e [03:37:39] 02ssl/03master 07MacFan4000 03cbaad8a add wiki.deepwoken.net [05:30:23] PROBLEM - wiki.msnld.com - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [10:49:56] [02mw-config] 07BlankEclair opened pull request #5801: T13147: Set $wgLogRestrictions for fischwiki (07miraheze:03master...07BlankEclair:03T13147) 13https://github.com/miraheze/mw-config/pull/5801 [10:50:49] miraheze/mw-config - BlankEclair the build passed. [10:54:32] [02RemovePII] 07BlankEclair approved pull request #104 13https://github.com/miraheze/RemovePII/pull/104#pullrequestreview-2583564633 [12:57:16] [02RemovePII] 07paladox merged 07dependabot[bot]'s pull request #105: Update mediawiki/mediawiki-codesniffer requirement from 45.0.0 to 46.0.0 (03master...03dependabot/composer/mediawiki/mediawiki-codesniffer-46.0.0) 13https://github.com/miraheze/RemovePII/pull/105 [12:57:17] [02RemovePII] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/RemovePII/commit/07be62bb5dcd3f4222849915cbd6a0246510f18c [12:57:17] 02RemovePII/03master 07dependabot[bot] 0307be62b Update mediawiki/mediawiki-codesniffer requirement from 45.0.0 to 46.0.0 (#105)… [12:57:18] [02RemovePII] 07paladox 04deleted 03dependabot/composer/mediawiki/mediawiki-codesniffer-46.0.0 at 036f92cdd 13https://github.com/miraheze/RemovePII/commit/6f92cdd [12:57:31] [02RemovePII] 07paladox 04force-pushed 03paladox-patch-2 from 0337800f1 to 035161126 13https://github.com/miraheze/RemovePII/commits/paladox-patch-2 [12:57:31] 02RemovePII/03paladox-patch-2 07paladox 035161126 Remove experimental from REL1_43 [13:00:08] [02RemovePII] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/RemovePII/commit/d4b0acebfa87 [13:00:08] 02RemovePII/03paladox-patch-1 07paladox 03d4b0ace Fix support for MW 1.43 [13:00:12] [02RemovePII] 07paladox opened pull request #106: Fix support for MW 1.43 (03master...03paladox-patch-1) 13https://github.com/miraheze/RemovePII/pull/106 [13:00:20] [02RemovePII] 07coderabbitai[bot] commented on pull request #106: --- […] 13https://github.com/miraheze/RemovePII/pull/106#issuecomment-2624452974 [13:00:44] miraheze/RemovePII - paladox the build has errored. [13:02:13] miraheze/RemovePII - paladox the build passed. [13:02:51] miraheze/RemovePII - paladox the build has errored. [13:03:02] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/b2a7c32eced89e7ebfc78e0a2248c255ea726b97 [13:03:03] 02RemovePII/03paladox-patch-1 07paladox 03b2a7c32 Update SpecialRemovePII.php [13:05:40] miraheze/RemovePII - paladox the build has errored. [13:07:43] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/5598db67444f7b402ebcb5a2a6fa2ef8d393cf46 [13:07:43] 02RemovePII/03paladox-patch-1 07paladox 035598db6 Update RemovePIIJob.php [13:08:50] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/a695252f43c52712184db2ef20ab1000355a0441 [13:08:50] 02RemovePII/03paladox-patch-1 07paladox 03a695252 Update RemovePIIJob.php [13:11:14] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/2864a864510f1362ec21567911d3337e1ceee751 [13:11:14] 02RemovePII/03paladox-patch-1 07paladox 032864a86 Update RemovePIIJob.php [13:11:28] miraheze/RemovePII - paladox the build has errored. [13:16:00] miraheze/RemovePII - paladox the build passed. [13:16:47] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/1d49ad59471f2f259e540a5c19d2d8df78756229 [13:16:47] 02RemovePII/03paladox-patch-1 07paladox 031d49ad5 Update RemovePIIJob.php [13:17:25] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/e7690bada52d1cdc0ef63d038777551603c2b6d6 [13:17:25] 02RemovePII/03paladox-patch-1 07paladox 03e7690ba Update config.php [13:21:53] miraheze/RemovePII - paladox the build passed. [13:29:22] [02RemovePII] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/RemovePII/commit/f72ee0c24a71579fc4336b22e1b5d3e169707926 [13:29:22] 02RemovePII/03paladox-patch-1 07paladox 03f72ee0c Update config.php [13:34:03] miraheze/RemovePII - paladox the build passed. [13:34:13] [02RemovePII] 07paladox merged pull request #106: Fix support for MW 1.43 (03master...03paladox-patch-1) 13https://github.com/miraheze/RemovePII/pull/106 [13:34:13] [02RemovePII] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/RemovePII/commit/c02be9ce8bd813eae88953ef8f895d826f49bfdb [13:34:13] 02RemovePII/03master 07paladox 03c02be9c Fix support for MW 1.43 (#106) [13:34:14] [02RemovePII] 07paladox 04deleted 03paladox-patch-1 at 03f72ee0c 13https://github.com/miraheze/RemovePII/commit/f72ee0c [13:34:30] [02RemovePII] 07paladox 04force-pushed 03paladox-patch-2 from 035161126 to 030d2e548 13https://github.com/miraheze/RemovePII/commits/paladox-patch-2 [13:34:30] 02RemovePII/03paladox-patch-2 07paladox 030d2e548 Remove experimental from REL1_43 [13:38:43] miraheze/RemovePII - paladox the build passed. [13:38:54] miraheze/RemovePII - paladox the build passed. [13:39:04] [02RemovePII] 07paladox merged pull request #104: Remove experimental from REL1_43 (03master...03paladox-patch-2) 13https://github.com/miraheze/RemovePII/pull/104 [13:39:04] [02RemovePII] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/RemovePII/commit/bdbd339f7f9a7270aa1997e0fb270c8cef40873b [13:39:05] 02RemovePII/03master 07paladox 03bdbd339 Remove experimental from REL1_43 (#104) [13:39:07] [02RemovePII] 07paladox 04deleted 03paladox-patch-2 at 030d2e548 13https://github.com/miraheze/RemovePII/commit/0d2e548 [13:43:32] miraheze/RemovePII - paladox the build passed. [14:38:33] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/migrateBlocks.php --wiki=whitesideswiki (END - exit=65280) [14:38:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:38:45] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/migrateBlocks.php --wiki=whitesidewiki (END - exit=0) [14:38:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:47:27] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/d8f88ad3c6b8ff03cd85a787d35610c50d688e76 [15:47:27] 02ssl/03master 07MacFan4000 03d8f88ad add wiki.breadcraft.me [16:04:19] PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 27212MiB (5% inode=99%); [19:09:54] PROBLEM - jobchron171 APT on jobchron171 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [19:23:35] PROBLEM - changeprop151 APT on changeprop151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [19:25:25] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [19:26:49] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [19:30:21] PROBLEM - rdb151 APT on rdb151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [22:28:36] PROBLEM - matomo151 HTTPS on matomo151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to matomo151.wikitide.net port 443 after 0 ms: Couldn't connect to server [22:49:58] PROBLEM - matomo151 NTP time on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:50:00] PROBLEM - matomo151 Puppet on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:50:27] PROBLEM - matomo151 PowerDNS Recursor on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:50:33] PROBLEM - matomo151 Redis Process on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:50:56] PROBLEM - matomo151 ferm_active on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:51:07] PROBLEM - matomo151 SSH on matomo151 is CRITICAL: connect to address 10.0.15.112 and port 22: Connection refused [22:51:17] PROBLEM - matomo151 conntrack_table_size on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:51:28] PROBLEM - matomo151 php-fpm on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:51:36] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:51:52] PROBLEM - matomo151 Disk Space on matomo151 is CRITICAL: connect to address 10.0.15.112 port 5666: Connection refusedconnect to host 10.0.15.112 port 5666: Connection refused [22:55:28] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is WARNING: WARNING - NGINX Error Rate is 40% [22:57:23] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 26% [22:58:56] RECOVERY - matomo151 ferm_active on matomo151 is OK: OK ferm input default policy is set [22:59:07] RECOVERY - matomo151 SSH on matomo151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u4 (protocol 2.0) [22:59:17] RECOVERY - matomo151 conntrack_table_size on matomo151 is OK: OK: nf_conntrack is 0 % full [22:59:28] RECOVERY - matomo151 php-fpm on matomo151 is OK: PROCS OK: 37 processes with command name 'php-fpm8.2' [22:59:37] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 0.77, 0.21, 0.07 [22:59:48] [02CreateWiki] 07AgentIsai closed pull request #639: Sort queryOpenAI parameters (03master...03sortfunction) 13https://github.com/miraheze/CreateWiki/pull/639 [22:59:49] [02CreateWiki] 07AgentIsai commented on pull request #639: Superseded 13https://github.com/miraheze/CreateWiki/pull/639#issuecomment-2625790768 [22:59:52] RECOVERY - matomo151 Disk Space on matomo151 is OK: DISK OK - free space: / 3270MiB (18% inode=90%); [23:00:00] PROBLEM - matomo151 NTP time on matomo151 is WARNING: NTP WARNING: Offset -0.2920042574 secs [23:00:01] [02RemovePII] 07AgentIsai merged pull request #100: T12077, T12814: Remove user from groups upon vanish and change their password (03master...03groupsandpass) 13https://github.com/miraheze/RemovePII/pull/100 [23:00:01] [02RemovePII] 07AgentIsai pushed 1 new commit to 03master 13https://github.com/miraheze/RemovePII/commit/43a8297af9b58960b6294813cb93b41c87a1cb3e [23:00:01] 02RemovePII/03master 07Agent Isai 0343a8297 T12077, T12814: Remove user from groups upon vanish and change their password (#100)… [23:00:02] [02RemovePII] 07AgentIsai 04deleted 03groupsandpass at 037b2dc3b 13https://github.com/miraheze/RemovePII/commit/7b2dc3b [23:00:03] RECOVERY - matomo151 Puppet on matomo151 is OK: OK: Puppet is currently enabled, last run 35 minutes ago with 0 failures [23:00:27] RECOVERY - matomo151 PowerDNS Recursor on matomo151 is OK: DNS OK: 0.042 seconds response time. wikitide.net returns 2602:294:0:b13::110,38.46.223.205 [23:00:40] RECOVERY - matomo151 HTTPS on matomo151 is OK: HTTP OK: HTTP/2 200 - 553 bytes in 0.208 second response time [23:00:47] RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [23:02:05] RECOVERY - matomo151 NTP time on matomo151 is OK: NTP OK: Offset -6.487965584e-05 secs [23:02:33] RECOVERY - matomo151 Redis Process on matomo151 is OK: PROCS OK: 1 process with args 'redis-server' [23:04:43] miraheze/RemovePII - AgentIsai the build passed. [23:27:07] [02ssl] 07MacFan4000 created 03MacFan4000-patch-1 (+1 new commit) 13https://github.com/miraheze/ssl/commit/45793e047837 [23:27:07] 02ssl/03MacFan4000-patch-1 07MacFan4000 0345793e0 Update certs.yaml [23:27:19] [02ssl] 07MacFan4000 opened pull request #821: rm exordium.wiki (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/821 [23:27:55] [02ssl] 07MacFan4000 pushed 1 new commit to 03MacFan4000-patch-1 13https://github.com/miraheze/ssl/commit/b02fd81dca26e81dff04e5d73cb2e782b3e9732e [23:27:57] 02ssl/03MacFan4000-patch-1 07MacFan4000 03b02fd81 Delete certificates/exordium.wiki.crt [23:28:29] [02ssl] 07coderabbitai[bot] commented on pull request #821: --- […] 13https://github.com/miraheze/ssl/pull/821#issuecomment-2625925882 [23:29:33] [02ssl] 07MacFan4000 merged pull request #821: rm exordium.wiki (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/821 [23:29:33] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/5b6faaa414b30f6bed4f9fa089e46f4654b34157 [23:29:33] 02ssl/03master 07MacFan4000 035b6faaa rm exordium.wiki (#821) [23:29:35] [02ssl] 07MacFan4000 04deleted 03MacFan4000-patch-1 at 03b02fd81 13https://github.com/miraheze/ssl/commit/b02fd81