[00:01:01] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/4f7dba71f752fee5cb57f0b966714adcc7536d78 [00:01:01] 02statichelp/03main 07WikiTideBot 034f7dba7 Bot: Auto-update Tech namespace pages 2026-07-11 00:00:58 [00:14:37] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 0.46, 1.68, 10.88 [00:16:37] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 0.57, 1.34, 9.63 [00:21:19] !log [petramagna@test151] starting deploy of {'versions': ['1.45', '1.46'], 'upgrade_extensions': 'DataTransfer'} to test151 [00:21:21] !log [petramagna@test151] finished deploy of {'versions': ['1.45', '1.46'], 'upgrade_extensions': 'DataTransfer'} to test151 - SUCCESS in 2s [00:21:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:21:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:07:59] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 17.33, 12.73, 9.29 [01:12:03] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 3 backends are down. mw152 mw201 mw203 [01:13:50] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 4 backends are down. mw161 mw162 mw172 mw181 [01:15:41] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 8 backends are down. mw162 mw181 mw153 mw183 mw191 mw201 mw202 mw203 [01:16:14] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 9 backends are down. mw152 mw162 mw171 mw182 mw153 mw173 mw192 mw201 mw203 [01:17:45] @posix_memalign howdy. Fyi see above if not aware 🥰 [01:18:52] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [01:20:47] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.079 second response time [01:21:36] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [01:21:51] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [01:22:03] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [01:22:14] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [01:31:59] Depooling db161 [01:32:03] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 11 backends are down. mw151 mw152 mw162 mw171 mw172 mw153 mw173 mw191 mw201 mw202 mw203 [01:32:14] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 4 backends are down. mw162 mw172 mw173 mw191 [01:32:25] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw162 [01:32:39] !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all [01:32:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:32:59] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 20s [01:33:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:34:03] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [01:34:14] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [01:34:23] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [01:36:18] Caused by excessive scraping on shintowiki. Repooling. [01:36:33] !log [petramagna@mwtask181] starting deploy of {'config': True} to all [01:36:34] !log [petramagna@mwtask181] DEPLOY ABORTED: Canary check failed for publictestwiki.com@localhost [01:36:36] !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all [01:36:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:53] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 18s [01:36:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:59] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 1.37, 2.18, 11.19 [01:53:59] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 1.29, 1.87, 9.98 [07:23:15] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 42% [07:25:15] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 4% [12:08:52] PROBLEM - test151 Check unit status of mediawiki_job_manage-inactive-wikis on test151 is CRITICAL: CRITICAL: Status of the systemd unit mediawiki_job_manage-inactive-wikis [16:14:43] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 15.64, 13.30, 11.06 [16:18:36] PROBLEM - matomo151 Current Load on matomo151 is CRITICAL: LOAD CRITICAL - total load average: 16.20, 14.87, 12.21 [16:22:27] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [16:22:29] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:22:46] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw153 [16:22:52] PROBLEM - mw153 Current Load on mw153 is CRITICAL: LOAD CRITICAL - total load average: 58.19, 31.06, 16.90 [16:23:17] PROBLEM - mw153 SSH on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:23:42] PROBLEM - mw153 php-fpm on mw153 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [16:23:50] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 1 backends are down. mw153 [16:24:03] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 1 backends are down. mw153 [16:24:14] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 1 backends are down. mw153 [16:24:25] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 15.84, 15.81, 13.50 [16:24:26] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.355 second response time [16:24:27] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.727 second response time [16:25:15] RECOVERY - mw153 SSH on mw153 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [16:25:38] RECOVERY - mw153 php-fpm on mw153 is OK: PROCS OK: 25 processes with command name 'php-fpm8.4' [16:25:50] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [16:26:03] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [16:26:14] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [16:26:46] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [16:28:41] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 13.49, 22.81, 18.52 [16:30:38] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 9.73, 18.27, 17.37 [16:47:51] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 12.25, 13.06, 13.54 [18:29:06] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 8.71, 12.29, 6.92 [18:31:06] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.82, 8.82, 6.29 [23:24:16] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 15.82, 10.64, 6.75 [23:26:15] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 8.11, 10.67, 7.32 [23:28:14] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 3.67, 8.00, 6.74 [23:28:29] daily heart attack