[00:06:39] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [00:08:41] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.56 ms [00:14:32] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 51956MiB (11% inode=99%); [00:15:10] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 31.52 ms [00:19:19] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [00:33:59] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 90%, RTA = 31.55 ms [00:36:03] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.59 ms [00:52:51] PROBLEM - wiki.hitscan.org - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [00:55:51] [02puppet] 07paladox created 03paladox-patch-4 (+1 new commit) 13https://github.com/miraheze/puppet/commit/ea311344b98d [00:55:51] 02puppet/03paladox-patch-4 07paladox 03ea31134 Nginx: Change default for nginx::keepalive_timeout to 60 [00:55:59] [02puppet] 07paladox opened pull request #4179: Nginx: Change default for nginx::keepalive_timeout to 60 (03master...03paladox-patch-4) 13https://github.com/miraheze/puppet/pull/4179 [00:56:07] [02puppet] 07coderabbitai[bot] commented on pull request #4179: --- […] 13https://github.com/miraheze/puppet/pull/4179#issuecomment-2644392381 [00:56:36] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/7a20deb624266d1e5ecaa574ce7f349428ed47a9 [00:56:36] 02puppet/03master 07paladox 037a20deb Nginx: Change default for nginx::keepalive_timeout to 60 (#4179) [00:56:36] [02puppet] 07paladox merged pull request #4179: Nginx: Change default for nginx::keepalive_timeout to 60 (03master...03paladox-patch-4) 13https://github.com/miraheze/puppet/pull/4179 [00:56:38] [02puppet] 07paladox 04deleted 03paladox-patch-4 at 03ea31134 13https://github.com/miraheze/puppet/commit/ea31134 [00:57:36] [02puppet] 07paladox created 03paladox-patch-4 (+1 new commit) 13https://github.com/miraheze/puppet/commit/b7adce96f892 [00:57:36] 02puppet/03paladox-patch-4 07paladox 03b7adce9 mediawiki: change nginx::keepalive_timeout to 60 for now [00:58:22] [02puppet] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/puppet/commit/8ac742a4b8da [00:58:22] 02puppet/03paladox-patch-5 07paladox 038ac742a mediawiki: change nginx::keepalive_timeout to 60 for now [00:58:41] [02puppet] 07paladox opened pull request #4180: mediawiki: change nginx::keepalive_timeout to 60 for now (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4180 [00:58:54] [02puppet] 07coderabbitai[bot] commented on pull request #4180: --- […] 13https://github.com/miraheze/puppet/pull/4180#issuecomment-2644394017 [00:59:08] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-5 13https://github.com/miraheze/puppet/commit/8a54d28f546b9cdb3af7ad8f9bfd65807f29d3d9 [00:59:08] 02puppet/03paladox-patch-5 07paladox 038a54d28 Update mediawiki_beta.yaml [00:59:34] [02puppet] 07paladox merged pull request #4180: mediawiki: change nginx::keepalive_timeout to 60 for now (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4180 [00:59:34] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/ce51c6925d93e92547f00f8a4b1b1d07e487f51f [00:59:34] 02puppet/03master 07paladox 03ce51c69 mediawiki: change nginx::keepalive_timeout to 60 for now (#4180) [00:59:35] [02puppet] 07paladox 04deleted 03paladox-patch-5 at 038a54d28 13https://github.com/miraheze/puppet/commit/8a54d28 [01:20:32] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49783MiB (10% inode=99%); [01:29:47] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 31.60 ms [01:31:49] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.59 ms [01:38:15] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 90%, RTA = 31.67 ms [01:42:27] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [02:21:35] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [02:21:51] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importDump.php --wiki=talesofwiki detalesof233_pages_full.xml --no-updates (START) [02:21:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:25:47] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.61 ms [03:26:57] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importDump.php --wiki=talesofwiki detalesof233_pages_full.xml --no-updates (END - exit=0) [03:26:58] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/rebuildall.php --wiki=talesofwiki (START) [03:26:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:27:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:51:42] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/rebuildall.php --wiki=talesofwiki (END - exit=0) [03:51:43] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/initSiteStats.php --wiki=talesofwiki --update (END - exit=0) [03:51:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:51:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:34:38] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [04:36:40] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.56 ms [04:41:06] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [04:41:36] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/9a029fb8bcd6713279f437e0afcd73531439b25c [04:41:37] 02ssl/03master 07MacFan4000 039a029fb add www.backroomszh.wiki [04:43:08] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.59 ms [04:45:49] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importImages.php --wiki=talesofwiki images --search-recursively (START) [04:45:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:59:25] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/importImages.php --wiki=talesofwiki images --search-recursively (END - exit=0) [04:59:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:01:54] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [05:03:57] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.59 ms [05:20:37] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [05:22:36] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.61 ms [05:54:01] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [05:58:06] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.54 ms [06:16:25] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [06:22:35] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.57 ms [06:54:50] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [06:56:50] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.74 ms [07:12:18] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 31.56 ms [07:14:17] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.61 ms [07:43:43] PROBLEM - elephants.ggreig.com - Cloudflare on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [07:55:01] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [07:57:01] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.58 ms [08:05:32] PROBLEM - matomo151 HTTPS on matomo151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to matomo151.wikitide.net port 443 after 0 ms: Couldn't connect to server [08:13:37] RECOVERY - elephants.ggreig.com - Cloudflare on sslhost is OK: OK - Certificate 'elephants.ggreig.com' will expire on Wed 26 Mar 2025 04:59:02 PM GMT +0000. [08:25:34] RECOVERY - matomo151 HTTPS on matomo151 is OK: HTTP OK: HTTP/2 200 - 553 bytes in 1.389 second response time [08:45:31] PROBLEM - sytbay.site - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [08:49:23] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 31.59 ms [08:55:33] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.61 ms [08:59:55] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:04:00] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.62 ms [09:32:15] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:34:15] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [10:13:04] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 90%, RTA = 31.62 ms [10:19:14] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.60 ms [10:57:46] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [11:01:51] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.58 ms [11:30:25] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [11:38:40] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [12:30:58] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [12:31:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:13] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 15.64, 10.12, 7.36 [12:35:50] !log [paladox@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [12:35:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:56] PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 37.88, 18.97, 8.98 [12:37:56] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 7.18, 13.62, 8.24 [12:38:02] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [12:38:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:39:13] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 4.04, 9.58, 8.52 [12:39:56] !log [paladox@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [12:40:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:41:42] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [12:41:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:43:32] !log [paladox@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [12:43:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:43:56] [02puppet] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/puppet/commit/e3dfdb9865d4 [12:43:56] 02puppet/03paladox-patch-5 07paladox 03e3dfdb9 mediawiki_task: unset memory_limit… [12:44:00] [02puppet] 07paladox opened pull request #4181: mediawiki_task: unset memory_limit (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4181 [12:44:08] [02puppet] 07coderabbitai[bot] commented on pull request #4181: --- […] 13https://github.com/miraheze/puppet/pull/4181#issuecomment-2645407586 [12:44:09] [02puppet] 07paladox merged pull request #4181: mediawiki_task: unset memory_limit (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4181 [12:44:09] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/efb346a42c4eaa0f9b6bdf0722d5c614cb1599b0 [12:44:10] 02puppet/03master 07paladox 03efb346a mediawiki_task: unset memory_limit (#4181)… [12:44:12] [02puppet] 07paladox 04deleted 03paladox-patch-5 at 03e3dfdb9 13https://github.com/miraheze/puppet/commit/e3dfdb9 [12:47:56] [02puppet] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/puppet/commit/af334dff9682 [12:47:56] 02puppet/03paladox-patch-5 07paladox 03af334df mediawiki: Set memory_limit to default to 500M… [12:48:01] [02puppet] 07paladox opened pull request #4182: mediawiki: Set memory_limit to default to 500M (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4182 [12:48:07] [02puppet] 07coderabbitai[bot] commented on pull request #4182: --- […] 13https://github.com/miraheze/puppet/pull/4182#issuecomment-2645430039 [12:48:13] [02puppet] 07paladox merged pull request #4182: mediawiki: Set memory_limit to default to 500M (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4182 [12:48:13] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/210049e9bf26291e06222c028a540523e0f4a38b [12:48:13] 02puppet/03master 07paladox 03210049e mediawiki: Set memory_limit to default to 500M (#4182)… [12:48:14] [02puppet] 07paladox 04deleted 03paladox-patch-5 at 03af334df 13https://github.com/miraheze/puppet/commit/af334df [12:48:27] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [12:48:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:51:39] [02mw-config] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/797f988b7bed6ec0e6ccd36993f73178e8f56437 [12:51:39] 02mw-config/03master 07paladox 03797f988 Lower memory_limit to 3000 mib [12:51:51] !log [paladox@mwtask181] finished deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all - SUCCESS in 203s [12:51:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:52:40] miraheze/mw-config - paladox the build passed. [12:52:44] !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [12:52:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:53:06] !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 21s [12:53:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:42] [02puppet] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/puppet/commit/768f51ee495b [12:55:42] 02puppet/03paladox-patch-5 07paladox 03768f51e mediawiki: remove unused variable [12:55:49] [02puppet] 07paladox opened pull request #4183: mediawiki: remove unused variable (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4183 [12:55:55] [02puppet] 07coderabbitai[bot] commented on pull request #4183: --- […] 13https://github.com/miraheze/puppet/pull/4183#issuecomment-2645484608 [12:56:57] [02puppet] 07paladox closed pull request #4183: mediawiki: remove unused variable (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4183 [12:57:00] [02puppet] 07paladox reopened pull request #4183: mediawiki: remove unused variable (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4183 [12:57:08] [02puppet] 07paladox merged pull request #4183: mediawiki: remove unused variable (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4183 [12:57:10] [02puppet] 07paladox 04deleted 03paladox-patch-5 at 03cf1106f 13https://github.com/miraheze/puppet/commit/cf1106f [12:57:41] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/80866f7fdf944fb9d35ca0c5bafd85fdbfcee9cc [12:57:41] 02puppet/03master 07paladox 0380866f7 mediawiki: remove unused variable (#4183) [13:06:28] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-5 13https://github.com/miraheze/puppet/commit/cf1106f02085a40b1f2a2cf28607c5b0dd39d981 [13:06:28] 02puppet/03paladox-patch-5 07paladox 03cf1106f Update mediawiki_beta.yaml [13:28:09] PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 27211MiB (5% inode=99%); [13:59:00] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [14:01:00] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.66 ms [14:09:34] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [14:09:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:12:18] !log [paladox@mwtask181] finished deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all - SUCCESS in 164s [14:12:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:05:50] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:05:54] PROBLEM - mem151 APT on mem151 is CRITICAL: APT CRITICAL: 58 packages available for upgrade (1 critical updates). [15:06:00] PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:06:20] PROBLEM - mw154 APT on mw154 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:06:29] PROBLEM - mw164 APT on mw164 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:08:54] PROBLEM - mw184 APT on mw184 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:09:27] PROBLEM - jobchron171 APT on jobchron171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:09:34] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:09:35] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:10:03] PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 58 packages available for upgrade (1 critical updates). [15:10:05] PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates). [15:10:06] PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:12:08] PROBLEM - cloud18 APT on cloud18 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:12:10] PROBLEM - os162 APT on os162 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (1 critical updates). [15:13:45] PROBLEM - kafka181 APT on kafka181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:14:15] PROBLEM - mw173 APT on mw173 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:14:33] PROBLEM - mattermost1 APT on mattermost1 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:14:49] PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (1 critical updates). [15:15:29] PROBLEM - mw174 APT on mw174 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:16:11] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:16:15] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:16:15] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 59 packages available for upgrade (1 critical updates). [15:17:21] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:17:26] PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:17:41] PROBLEM - db151 APT on db151 is CRITICAL: APT CRITICAL: 51 packages available for upgrade (1 critical updates). [15:17:58] PROBLEM - mw153 APT on mw153 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:19:19] PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (1 critical updates). [15:19:20] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 12 packages available for upgrade (1 critical updates). [15:19:36] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:19:56] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:20:50] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:21:48] PROBLEM - cp36 APT on cp36 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:23:10] PROBLEM - changeprop151 APT on changeprop151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:25:07] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (1 critical updates). [15:25:21] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 58 packages available for upgrade (1 critical updates). [15:26:02] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:26:29] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:26:30] PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 63 packages available for upgrade (1 critical updates). [15:26:54] PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:27:25] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (1 critical updates). [15:27:40] PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 59 packages available for upgrade (1 critical updates). [15:27:57] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:28:05] PROBLEM - graphite151 APT on graphite151 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:28:09] PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates). [15:28:18] PROBLEM - swiftac171 APT on swiftac171 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates). [15:28:44] PROBLEM - db161 APT on db161 is CRITICAL: APT CRITICAL: 50 packages available for upgrade (1 critical updates). [15:29:00] PROBLEM - db172 APT on db172 is CRITICAL: APT CRITICAL: 50 packages available for upgrade (1 critical updates). [15:29:04] PROBLEM - db171 APT on db171 is CRITICAL: APT CRITICAL: 50 packages available for upgrade (1 critical updates). [15:29:14] PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 63 packages available for upgrade (1 critical updates). [15:29:33] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:30:04] PROBLEM - rdb151 APT on rdb151 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:30:18] PROBLEM - mem161 APT on mem161 is CRITICAL: APT CRITICAL: 58 packages available for upgrade (1 critical updates). [15:30:26] PROBLEM - os161 APT on os161 is CRITICAL: APT CRITICAL: 59 packages available for upgrade (1 critical updates). [15:31:12] PROBLEM - bast161 APT on bast161 is CRITICAL: APT CRITICAL: 58 packages available for upgrade (1 critical updates). [15:31:35] PROBLEM - db181 APT on db181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:31:51] PROBLEM - eventgate181 APT on eventgate181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:32:00] PROBLEM - cp37 APT on cp37 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:32:00] PROBLEM - graylog161 APT on graylog161 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (1 critical updates). [15:32:22] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:32:26] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:33:07] PROBLEM - db182 APT on db182 is CRITICAL: APT CRITICAL: 51 packages available for upgrade (1 critical updates). [15:33:52] PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (1 critical updates). [15:33:57] [02mw-config] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/d0cf92c81c857721f8bf555af83d1625538080d8 [15:33:57] 02mw-config/03master 07paladox 03d0cf92c Fix name of maintenance scripts for DynamicPageList3 [15:34:02] PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates). [15:34:06] !log [paladox@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [15:34:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:34:23] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates). [15:34:28] !log [paladox@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [15:34:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:34:53] miraheze/mw-config - paladox the build passed. [15:34:59] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 1 packages available for upgrade (1 critical updates). [15:45:24] !log [paladox@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [15:45:24] !log [paladox@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s [15:45:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:45:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:30] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/fb949152b4e85b3d33270637dd29d5492f3e9c38 [16:20:30] 02ssl/03master 07MacFan4000 03fb94915 Update certs.yaml [16:23:53] PROBLEM - hardcore.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'hardcore.wiki' expires in 14 day(s) (Sun 23 Feb 2025 04:02:49 PM GMT +0000). [17:08:31] !log [paladox@mwtask181] starting deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all [17:08:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:11:09] !log [paladox@mwtask181] finished deploy of {'world': True, 'l10n': True, 'versions': '1.43'} to all - SUCCESS in 158s [17:11:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:47:37] PROBLEM - cp37 HTTPS on cp37 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [17:49:32] RECOVERY - cp37 HTTPS on cp37 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4107 bytes in 0.070 second response time [17:50:01] PROBLEM - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is CRITICAL: connect to address localhost and port 443: Connection refused [17:50:19] PROBLEM - swiftproxy161 HTTPS on swiftproxy161 is CRITICAL: connect to address 10.0.16.135 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [17:50:44] PROBLEM - swiftproxy171 HTTPS on swiftproxy171 is CRITICAL: connect to address 10.0.17.108 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [17:52:56] PROBLEM - swiftproxy171 Swift NGINX SSL check on swiftproxy171 is CRITICAL: connect to address localhost and port 443: Connection refused [17:57:26] RECOVERY - swiftproxy171 Swift NGINX SSL check on swiftproxy171 is OK: OK - Certificate 'wikitide.net' will expire on Fri Mar 28 03:25:53 2025 +0000.TCP OK - 0.046 second response time on localhost port 443 [17:57:49] RECOVERY - swiftproxy171 HTTPS on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.630 second response time [17:58:24] RECOVERY - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is OK: OK - Certificate 'wikitide.net' will expire on Fri Mar 28 03:25:53 2025 +0000.TCP OK - 0.010 second response time on localhost port 443 [17:58:26] RECOVERY - swiftproxy161 HTTPS on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.027 second response time [17:59:01] !log restart nginx on swiftproxy* [17:59:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:11:11] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [18:13:11] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [18:17:34] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [18:23:45] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.57 ms [19:34:31] PROBLEM - blackmagic.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'blackmagic.wiki' expires in 14 day(s) (Sun 23 Feb 2025 07:21:15 PM GMT +0000). [19:34:43] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/9257c37a96b4b5d2bf0414b33edcf87e45fcd26f [19:34:43] 02ssl/03master 07WikiTideSSLBot 039257c37 Bot: Update SSL cert for blackmagic.wiki [19:39:40] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 31.54 ms [19:43:45] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.57 ms [20:03:54] RECOVERY - blackmagic.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'blackmagic.wiki' will expire on Fri 09 May 2025 06:36:06 PM GMT +0000. [20:18:15] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:20:15] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [20:37:13] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/4bc73d500dbff85e5eca0d8bceed6af6094b1314 [20:37:13] 02ssl/03master 07WikiTideSSLBot 034bc73d5 Bot: Update SSL cert for hardcore.wiki [20:40:04] [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/6446956b7c68d70d88a6031e168823315e152104 [20:40:04] 02ssl/03master 07MacFan4000 036446956 add zh.wiki.animalroyale.com [20:46:32] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:48:32] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.65 ms [21:09:40] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/82ad70acb33f52fb35193a7f2622f2003b7adfe2 [21:09:40] 02puppet/03master 07paladox 0382ad70a varnish: set first_byte_timeout to 65s [21:13:55] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 32.03 ms [21:15:22] RECOVERY - hardcore.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hardcore.wiki' will expire on Fri 09 May 2025 07:38:37 PM GMT +0000. [21:15:55] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.55 ms [21:23:15] [02puppet] 07paladox created 03paladox-patch-5 (+1 new commit) 13https://github.com/miraheze/puppet/commit/a99a229a62f9 [21:23:15] 02puppet/03paladox-patch-5 07paladox 03a99a229 swift: don't look for x-client-ip… [21:23:19] [02puppet] 07paladox opened pull request #4184: swift: don't look for x-client-ip (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4184 [21:23:28] [02puppet] 07coderabbitai[bot] commented on pull request #4184: --- […] 13https://github.com/miraheze/puppet/pull/4184#issuecomment-2645947620 [21:24:40] miraheze/puppet - paladox the build passed. [21:25:11] [02puppet] 07paladox closed pull request #4184: swift: don't look for x-client-ip (03master...03paladox-patch-5) 13https://github.com/miraheze/puppet/pull/4184 [21:25:15] [02puppet] 07paladox 04deleted 03paladox-patch-5 at 03a99a229 13https://github.com/miraheze/puppet/commit/a99a229 [21:41:19] [02CreateWiki] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/CreateWiki/commit/e71e68d097d5cafd95f63c657f03cbf34ca7bda6 [21:41:19] 02CreateWiki/03master 07CosmicAlpha 03e71e68d RemoteWikiFactory: fix overriding assignment (#630)… [21:41:19] [02CreateWiki] 07Universal-Omega merged pull request #630: RemoteWikiFactory: fix overriding assignment (03master...03rw-assignment) 13https://github.com/miraheze/CreateWiki/pull/630 [21:41:20] [02CreateWiki] 07Universal-Omega 04deleted 03rw-assignment at 034acfe26 13https://github.com/miraheze/CreateWiki/commit/4acfe26 [21:53:27] miraheze/CreateWiki - Universal-Omega the build passed. [21:58:47] PROBLEM - matomo151 HTTPS on matomo151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to matomo151.wikitide.net port 443 after 0 ms: Couldn't connect to server [22:13:33] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [22:21:45] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.51 ms [22:24:48] RECOVERY - matomo151 HTTPS on matomo151 is OK: HTTP OK: HTTP/2 200 - 553 bytes in 1.562 second response time