[00:00:59] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/0ba1dbdca43e4911641131f8fd9bd4f416a310ea [00:00:59] 02statichelp/03main 07WikiTideBot 030ba1dbd Bot: Auto-update Tech namespace pages 2026-05-18 00:00:57 [01:03:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 40% [01:08:06] RECOVERY - db172 Disk Space on db172 is OK: DISK OK - free space: / 2983MiB (10.1% inode=95%); [01:12:06] PROBLEM - db172 Disk Space on db172 is WARNING: DISK WARNING - free space: / 2892MiB (9.8% inode=95%); [01:15:13] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 37% [01:59:54] !log [petramagna@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php Bucket:CreateInitialSchemaForBucketIssues --wiki=sagan4alphawiki (END - exit=0) [01:59:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:38:06] RECOVERY - db172 Disk Space on db172 is OK: DISK OK - free space: / 2982MiB (10.1% inode=95%); [03:42:06] PROBLEM - db172 Disk Space on db172 is WARNING: DISK WARNING - free space: / 2890MiB (9.8% inode=95%); [04:06:17] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 38.46.223.205/cpweb [04:07:03] PROBLEM - mwtask181 Puppet on mwtask181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png] [04:08:17] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:35:03] RECOVERY - mwtask181 Puppet on mwtask181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:55:43] PROBLEM - mw171 Puppet on mw171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.45] [05:06:40] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 38.46.223.205/cpweb [05:08:38] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:16:13] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png] [05:16:58] PROBLEM - mwtask161 Puppet on mwtask161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid] [05:21:40] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:23:13] PROBLEM - puppet181 Check unit status of git_pull_ssl on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_ssl [05:29:13] RECOVERY - puppet181 Check unit status of git_pull_ssl on puppet181 is OK: OK: Status of the systemd unit git_pull_ssl [05:42:24] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:42:57] RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:13:13] PROBLEM - puppet181 Check unit status of git_pull_ssl on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_ssl [06:13:48] PROBLEM - mw202 Puppet on mw202 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png] [06:19:13] RECOVERY - puppet181 Check unit status of git_pull_ssl on puppet181 is OK: OK: Status of the systemd unit git_pull_ssl [06:39:46] RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [09:12:06] PROBLEM - puppet181 Check unit status of git_pull_puppet on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_puppet [09:20:06] RECOVERY - puppet181 Check unit status of git_pull_puppet on puppet181 is OK: OK: Status of the systemd unit git_pull_puppet [10:02:06] PROBLEM - puppet181 Check unit status of git_pull_puppet on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_puppet [10:03:13] PROBLEM - puppet181 Check unit status of git_pull_ssl on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_ssl [10:09:13] RECOVERY - puppet181 Check unit status of git_pull_ssl on puppet181 is OK: OK: Status of the systemd unit git_pull_ssl [10:10:06] RECOVERY - puppet181 Check unit status of git_pull_puppet on puppet181 is OK: OK: Status of the systemd unit git_pull_puppet [10:31:09] PROBLEM - mw151 Puppet on mw151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.45] [10:59:09] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:10:29] PROBLEM - swiftproxy161 Current Load on swiftproxy161 is CRITICAL: LOAD CRITICAL - total load average: 10.27, 5.19, 3.11 [12:12:29] RECOVERY - swiftproxy161 Current Load on swiftproxy161 is OK: LOAD OK - total load average: 4.55, 5.44, 3.51 [12:43:50] miraheze/landing - translatewiki the build passed. [12:43:56] miraheze/MatomoAnalytics - translatewiki the build passed. [12:44:00] miraheze/IncidentReporting - translatewiki the build passed. [12:44:12] miraheze/GlobalNewFiles - translatewiki the build passed. [12:44:34] miraheze/ImportDump - translatewiki the build passed. [12:46:20] miraheze/MirahezeMagic - translatewiki the build passed. [12:46:44] miraheze/CreateWiki - translatewiki the build passed. [12:47:08] miraheze/RemovePII - translatewiki the build passed. [12:47:40] miraheze/PDFEmbed - translatewiki the build passed. [12:48:09] miraheze/ManageWiki - translatewiki the build passed. [17:00:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/fd8f430a0a40c62780a9830838f2cce1c317415f [17:00:22] 02ssl/03main 07WikiTideBot 03fd8f430 Bot: Auto-update domain lists [17:34:29] !log doing a deploy [17:34:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:23:40] [02mw-config] 07YTFGolf opened pull request #6413: T15397: Add wgBucketMaxDataPerPage + configure for sagan4alphawiki (07miraheze:03main...07YTFGolf:03patch-bucket-T15397) 13https://github.com/miraheze/mw-config/pull/6413 [18:24:51] miraheze/mw-config - YTFGolf the build passed. [20:50:23] [02mw-config] 07Disgustedorite commented on pull request #6413: As noted in T15397 the limit is arbitrary and the performance is not expected to be too bad, it mostly affects how quickly the page can be saved tmu. I'm successfully writing thousands of rows on some pages already and it's not affecting performance […] 13https://github.com/miraheze/mw-config/pull/6413#issuecomment-4482061477 [21:38:06] RECOVERY - db172 Disk Space on db172 is OK: DISK OK - free space: / 2977MiB (10.1% inode=95%); [21:42:06] PROBLEM - db172 Disk Space on db172 is WARNING: DISK WARNING - free space: / 2885MiB (9.8% inode=95%); [22:36:54] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.009 second response time [22:37:01] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:37:07] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:37:13] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:37:19] PROBLEM - db151 Current Load on db151 is CRITICAL: LOAD CRITICAL - total load average: 71.86, 34.90, 14.02 [22:37:41] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:37:46] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:37:49] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:37:54] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [22:38:04] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:04] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 2 backends are down. mw172 mw163 [22:38:18] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 3 backends are down. mw172 mw163 mw173 [22:38:19] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:38:20] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:22] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:25] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:25] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:26] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:29] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:35] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [22:38:42] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:38:42] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:50] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 2 backends are down. mw172 mw173 [22:38:52] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 2 backends are down. mw172 mw173 [22:38:53] PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:38:54] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:38:59] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:39:04] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [22:39:05] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:39:11] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:39:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 79% [22:39:16] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [22:39:18] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:39:31] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [22:39:31] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [22:39:33] Ermagerd [22:39:50] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [22:39:50] Well that doesn't look good. [22:39:51] PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:40:28] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 9.206 second response time [22:40:35] https://cdn.discordapp.com/attachments/808001911868489748/1506063983419850762/image0.gif?ex=6a0ce6e2&is=6a0b9562&hm=727aaf14b36adf1f7ecee00ed04259b8acaf37fac6f345d313132def007bc7d9& [22:40:37] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.222 second response time [22:40:59] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.710 second response time [22:41:02] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.185 second response time [22:41:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 50% [22:41:16] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.708 second response time [22:41:21] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.882 second response time [22:41:24] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 8.116 second response time [22:41:32] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 4.607 second response time [22:41:34] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.073 second response time [22:41:35] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 7.025 second response time [22:41:46] RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.243 second response time [22:41:47] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.836 second response time [22:41:48] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 3.818 second response time [22:41:54] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.173 second response time [22:42:18] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [22:42:26] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.209 second response time [22:43:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 82% [22:44:20] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.352 second response time [22:44:27] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.223 second response time [22:44:29] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.078 second response time [22:44:29] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 5.518 second response time [22:44:30] RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.102 second response time [22:44:31] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 3.987 second response time [22:44:37] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.074 second response time [22:44:54] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.064 second response time [22:45:01] RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.217 second response time [22:45:03] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.190 second response time [22:45:09] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.080 second response time [22:45:12] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.198 second response time [22:45:13] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.203 second response time [22:45:13] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 10% [22:45:14] RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.262 second response time [22:45:55] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.081 second response time [22:46:03] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.078 second response time [22:46:04] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [22:46:50] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [22:46:52] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [22:48:28] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:34] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:48:36] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:48:37] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [22:48:43] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [22:48:44] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [22:48:45] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:48:47] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [22:48:59] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:49:02] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [22:49:07] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [22:49:15] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:16] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [22:49:16] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 1 backends are down. mw163 [22:49:18] PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:26] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:27] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:49:33] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [22:50:02] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [22:50:03] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:04] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 3 backends are down. mw163 mw173 mw201 [22:50:14] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:50:17] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:50:25] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:28] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:29] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:50:30] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:31] PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:50:31] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [22:50:33] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [22:50:35] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:50:50] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [22:50:52] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [22:51:12] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 7.222 second response time [22:51:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 81% [22:51:14] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [22:51:17] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 7.437 second response time [22:51:20] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:52:04] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [22:52:42] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [22:54:50] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [22:54:52] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [22:55:18] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [22:55:25] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [22:56:04] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 3 backends are down. mw172 mw173 mw201 [22:58:22] PROBLEM - db192 MariaDB Connections on db192 is WARNING: Warning connection usage: 80.9%Current connections: 809 [22:58:50] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 3 backends are down. mw172 mw173 mw201 [22:58:52] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 3 backends are down. mw172 mw173 mw201 [22:59:21] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [23:02:07] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 4.754 second response time [23:02:14] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 8.285 second response time [23:02:17] RECOVERY - db192 MariaDB Connections on db192 is OK: OK connection usage: 62.3%Current connections: 623 [23:02:23] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.063 second response time [23:02:24] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.064 second response time [23:02:26] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.477 second response time [23:02:26] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.205 second response time [23:02:26] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.203 second response time [23:02:26] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.262 second response time [23:02:26] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.090 second response time [23:02:27] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.071 second response time [23:02:32] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.069 second response time [23:02:36] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.083 second response time [23:02:38] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 7.373 second response time [23:02:41] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 1.234 second response time [23:02:43] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.208 second response time [23:02:45] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.074 second response time [23:02:50] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 6.717 second response time [23:02:51] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [23:02:52] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [23:03:00] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.103 second response time [23:03:05] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 6.879 second response time [23:03:13] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 26% [23:03:14] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [23:04:02] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [23:05:19] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:05:31] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [23:05:40] RECOVERY - mw201 MediaWiki Rendering on mw201 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.796 second response time [23:05:55] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:06:02] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:14] PROBLEM - mw152 HTTPS on mw152 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:23] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [23:06:24] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:25] PROBLEM - mw162 MediaWiki Rendering on mw162 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:06:27] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [23:06:27] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:06:27] PROBLEM - mw161 HTTPS on mw161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:28] PROBLEM - mw151 HTTPS on mw151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:29] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:30] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:35] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [23:06:38] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [23:06:40] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:40] PROBLEM - mw153 HTTPS on mw153 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:50] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:06:50] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 2 backends are down. mw163 mw201 [23:06:52] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 2 backends are down. mw163 mw201 [23:07:01] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:07:02] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:07:13] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 4 backends are down. mw172 mw163 mw173 mw201 [23:07:13] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 89% [23:09:08] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.228 second response time [23:09:24] RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.201 second response time [23:09:28] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 9.195 second response time [23:09:36] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.067 second response time [23:09:43] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 6.149 second response time [23:09:43] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.190 second response time [23:09:45] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 9.313 second response time [23:09:47] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.072 second response time [23:09:47] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.831 second response time [23:09:51] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.676 second response time [23:10:04] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [23:10:04] RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.235 second response time [23:10:08] RECOVERY - mw152 HTTPS on mw152 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.089 second response time [23:10:17] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.072 second response time [23:10:18] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.210 second response time [23:10:19] RECOVERY - mw151 HTTPS on mw151 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.057 second response time [23:10:20] RECOVERY - mw161 HTTPS on mw161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.070 second response time [23:10:24] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.067 second response time [23:10:25] RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.075 second response time [23:10:30] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.073 second response time [23:10:30] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.077 second response time [23:10:31] RECOVERY - mw162 MediaWiki Rendering on mw162 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.189 second response time [23:10:32] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.080 second response time [23:10:35] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.206 second response time [23:10:36] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.263 second response time [23:10:39] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.082 second response time [23:10:41] RECOVERY - mw153 HTTPS on mw153 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.066 second response time [23:10:42] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.069 second response time [23:10:50] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [23:10:52] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [23:11:00] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.052 second response time [23:11:01] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.203 second response time [23:11:10] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.055 second response time [23:11:12] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [23:11:12] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.210 second response time [23:11:13] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 8% [23:11:22] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.223 second response time [23:11:28] RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.206 second response time [23:11:49] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.078 second response time [23:14:55] PROBLEM - db192 MariaDB Connections on db192 is WARNING: Warning connection usage: 82.7%Current connections: 827 [23:15:02] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:15:09] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [23:15:19] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [23:15:26] PROBLEM - mw152 MediaWiki Rendering on mw152 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [23:15:26] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:15:34] PROBLEM - mw153 MediaWiki Rendering on mw153 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [23:15:39] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:16:04] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 1 backends are down. mw173 [23:16:53] RECOVERY - db192 MariaDB Connections on db192 is OK: OK connection usage: 4.3%Current connections: 43 [23:16:57] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.211 second response time [23:17:06] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.074 second response time [23:17:17] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.072 second response time [23:17:20] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.212 second response time [23:17:20] RECOVERY - mw152 MediaWiki Rendering on mw152 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.184 second response time [23:17:29] RECOVERY - mw153 MediaWiki Rendering on mw153 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.187 second response time [23:17:35] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.226 second response time [23:18:04] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [23:24:23] wtf happened here [23:24:37] idk db151 had a stroke [23:24:56] this looks funky [23:25:03] I can't believe that's good [23:25:07] is db172 test or core [23:25:11] iirc db192 is core [23:26:50] fairly sure it's always like that [23:26:57] 3GB available space is uh [23:27:00] interesting to say the least [23:27:24] I mean it's beta so who gives a shit but [23:27:25] even then [23:30:02] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [23:30:18] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/713af54e4c59bf04f57ee27d1d9ff11e5ed38133 [23:30:18] 02ssl/03main 07WikiTideBot 03713af54 Bot: Auto-update domain lists [23:37:22] yeah db172 has 30gb of disk space [23:38:25] and there's a 7G database dump from a year ago that never got cleaned up, I'll probably clear that up now [23:43:19] PROBLEM - db151 Current Load on db151 is WARNING: LOAD WARNING - total load average: 0.33, 1.61, 11.17 [23:45:19] RECOVERY - db151 Current Load on db151 is OK: LOAD OK - total load average: 0.43, 1.26, 9.88 [23:49:06] !log [void@db172] deleted mariadb full backup from June 2025 [23:49:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:50:06] RECOVERY - db172 Disk Space on db172 is OK: DISK OK - free space: / 10728MiB (36.4% inode=95%);