[00:00:58] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/2c9fb08f484e959c0af0e28adba83ce64f652b71 [00:00:58] 02statichelp/03main 07WikiTideBot 032c9fb08 Bot: Auto-update Tech namespace pages 2026-07-19 00:00:55 [00:38:53] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 44821MiB (9.9% inode=100%); [00:50:53] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 80848MiB (17.8% inode=100%); [01:52:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.25, 6.75, 5.49 [01:54:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.20, 5.44, 5.16 [02:00:15] [02mw-config] 07The-Voidwalker pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/f1fa0ff0b856c2bff613ef5278927fd7abd86748 [02:00:16] 02mw-config/03main 07The-Voidwalker 03f1fa0ff correct ext-3d -> ext-3D… [02:01:11] !log [void@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [02:01:27] miraheze/mw-config - The-Voidwalker the build passed. [02:02:24] fyi The web server reported a bad gateway error. [02:02:25] Ray ID: a1d623dccdbed916 Your IP address: Error reference number: 502 Cloudflare Location: Vancouver [02:02:56] !log [void@mwtask181] DEPLOY ABORTED: Canary check failed for publictestwiki.com@mw173 [02:03:32] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:03:45] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [02:03:45] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: CRITICAL - Plugin timed out while executing system call [02:03:59] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 13 backends are down. mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw183 mw191 mw192 mw193 mw201 mw203 [02:04:07] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [02:04:13] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 233.08, 120.32, 48.93 [02:04:23] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:04:29] !log [void@mwtask181] starting deploy of {'config': True, 'force': True} to all [02:04:38] The web server reported a bad gateway error. [02:04:39] Ray ID: a1d623dccdbed916 Your IP address: Error reference number: 502 Cloudflare Location: Vancouver [02:04:39] nope still getting error 502s [02:04:41] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:04:45] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 71% [02:04:47] !log [void@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 17s [02:04:50] ahh [02:05:20] !log [void@mwtask181] depool db161 (c2) [02:05:27] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.076 second response time [02:05:41] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.083 second response time [02:05:42] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 0.207 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [02:05:59] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [02:06:06] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.223 second response time [02:06:23] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.079 second response time [02:06:38] thanks :) [02:06:39] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.089 second response time [02:06:45] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 16% [02:07:31] PROBLEM - bots171 IRC Log Bot on bots171 is CRITICAL: PROCS CRITICAL: 0 processes with args 'adminlogbot.py' [02:09:44] !log [void@mwtask181] starting deploy of {'pull': 'config', 'config': True, 'force': True} to all [02:10:03] !log [void@mwtask181] finished deploy of {'pull': 'config', 'config': True, 'force': True} to all - SUCCESS in 18s [02:11:31] RECOVERY - bots171 IRC Log Bot on bots171 is OK: PROCS OK: 1 process with args 'adminlogbot.py' [02:11:38] !log [void@bots171] restart logbot [02:11:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:12:37] !log [void@mwtask181] repool db161 after recent outage [02:12:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:19:16] !log [void@test151] starting deploy of {'pull': 'config', 'config': True, 'force': True} to test151 [02:19:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:19:21] !log [void@test151] finished deploy of {'pull': 'config', 'config': True, 'force': True} to test151 - SUCCESS in 5s [02:19:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:22:17] seems to be recurring again, getting more 502s [02:22:59] miraheze/MirahezeMagic - dmehus the build passed. [02:23:23] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.013 second response time [02:23:25] PROBLEM - cp161 HTTPS on cp161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:23:29] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 6 backends are down. mw161 mw172 mw181 mw153 mw191 mw193 [02:23:37] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 6 backends are down. mw161 mw172 mw181 mw153 mw191 mw193 [02:23:41] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [02:23:45] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: CRITICAL - Plugin timed out while executing system call [02:23:59] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 13 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw182 mw153 mw163 mw183 mw191 mw201 mw203 [02:23:59] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.014 second response time [02:24:13] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:24:26] !log [void@mwtask181] starting deploy of {'config': True, 'force': True} to all [02:24:32] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [02:24:33] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [02:24:45] !log [void@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 18s [02:24:45] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [02:24:52] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [02:25:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:25:03] !log [void@mwtask181] depool db161 again [02:25:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:25:19] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.257 second response time [02:25:25] RECOVERY - cp161 HTTPS on cp161 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1518 bytes in 0.077 second response time [02:25:29] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [02:25:32] PROBLEM - db161 Puppet on db161 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:25:37] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.077 second response time [02:25:37] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [02:25:46] PROBLEM - db161 APT on db161 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [02:25:47] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 5.029 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [02:25:59] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [02:25:59] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.258 second response time [02:26:07] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.082 second response time [02:26:31] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.077 second response time [02:26:33] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.244 second response time [02:26:45] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 13% [02:26:51] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.083 second response time [02:27:30] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 12 minutes ago with 0 failures [02:27:48] RECOVERY - db161 APT on db161 is OK: APT OK: 108 packages available for upgrade (0 critical updates). [02:33:40] !log [void@mwtask181] starting deploy of {'config': True, 'force': True} to all [02:33:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:33:58] !log [void@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 18s [02:34:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:51:47] miraheze/MirahezeMagic - dmehus the build passed. [02:55:32] miraheze/MirahezeMagic - dmehus the build passed. [02:56:10] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 1.26, 1.45, 11.26 [02:58:10] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 0.95, 1.28, 10.01 [03:10:24] PROBLEM - db161 Check unit status of sql-backup on db161 is CRITICAL: CRITICAL: Status of the systemd unit sql-backup [03:15:39] miraheze/MirahezeMagic - dmehus the build passed. [03:15:50] miraheze/MirahezeMagic - dmehus the build passed. [03:20:49] hey, the dependabot updates aren't showing [03:20:58] which bot relays those to this channel? [03:22:06] MacFan4000: fyi MacFanBot seems to have stopped relaying dependabot updates [03:25:31] miraheze/MirahezeMagic - dmehus the build passed. [03:26:26] miraheze/MirahezeMagic - dmehus the build passed. [03:34:31] miraheze/MirahezeMagic - dmehus the build passed. [03:35:06] miraheze/MirahezeMagic - dmehus the build passed. [03:44:33] miraheze/MirahezeMagic - dmehus the build passed. [03:54:25] miraheze/MirahezeMagic - dmehus the build passed. [03:59:23] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [03:59:27] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [03:59:37] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 10 backends are down. mw151 mw161 mw162 mw171 mw181 mw183 mw191 mw193 mw201 mw203 [03:59:39] !log [petramagna@test151] starting deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 [03:59:40] !log [petramagna@test151] finished deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [03:59:41] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [03:59:45] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: CRITICAL - Plugin timed out while executing system call [03:59:56] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 12 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw182 mw153 mw163 mw183 mw191 mw203 [03:59:59] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 12 backends are down. mw151 mw152 mw161 mw162 mw171 mw182 mw153 mw163 mw183 mw191 mw201 mw203 [03:59:59] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.011 second response time [04:00:07] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 12 backends are down. mw151 mw152 mw162 mw171 mw172 mw181 mw182 mw153 mw163 mw183 mw191 mw193 [04:00:10] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 223.24, 130.79, 54.48 [04:00:13] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:00:13] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10006 milliseconds with 0 bytes received [04:00:33] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.010 second response time [04:00:45] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 44% [04:01:06] !log [petramagna@test151] starting deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 [04:01:07] !log [petramagna@test151] finished deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [04:01:32] PROBLEM - db161 Puppet on db161 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:01:41] miraheze/MirahezeMagic - dmehus the build passed. [04:01:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:01:45] PROBLEM - db161 APT on db161 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:02:20] PROBLEM - mw203 HTTPS on mw203 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:02:45] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [04:02:56] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [04:04:04] !log Depool db161 [04:04:12] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.077 second response time [04:04:28] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:33] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [04:04:46] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10005 milliseconds with 0 bytes received [04:04:59] !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all [04:05:18] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 19s [04:05:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:05:39] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.093 second response time [04:05:41] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 3.764 second response time [04:05:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:05:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:05:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:06:06] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [04:06:08] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.223 second response time [04:06:09] RECOVERY - mw203 HTTPS on mw203 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.084 second response time [04:06:10] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.082 second response time [04:06:28] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.079 second response time [04:06:33] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.236 second response time [04:06:33] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 1463 bytes in 0.296 second response time [04:06:45] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 340 bytes in 0.084 second response time [04:06:45] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 22% [04:07:19] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.244 second response time [04:07:20] miraheze/MirahezeMagic - dmehus the build passed. [04:07:37] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [04:07:50] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [04:07:59] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [04:09:56] [02MirahezeMagic] 07dmehus readied pull request #657: Sort and update MirahezeMagic interface messages (07miraheze:03main...07dmehus:03patch-2) 13https://github.com/miraheze/MirahezeMagic/pull/657 [04:11:09] RECOVERY - db161 Check unit status of sql-backup on db161 is OK: OK: Status of the systemd unit sql-backup [04:13:42] [02MirahezeMagic] 07dmehus commented on pull request #657: @Reception123 this should be ready to go. Per my DM with you, I confirmed with @Universal-Omega that autopatrolled, confirmed, and rollbacker groups are indeed _not_ part of MediaWiki core […] 13https://github.com/miraheze/MirahezeMagic/pull/657#issuecomment-5014210205 [04:14:07] ty MacFan4000 [04:15:24] PROBLEM - db161 Check unit status of sql-backup on db161 is CRITICAL: CRITICAL: Status of the systemd unit sql-backup [04:16:04] PROBLEM - db161 SSH on db161 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:20:10] RECOVERY - db161 SSH on db161 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [04:21:24] Um I WANT TO REMOVE MAINTANCE [04:23:04] hrm? [04:30:56] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [04:33:30] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 9.343 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [04:33:54] Tech Team: fyi re: On metawiki CreateWiki Extension CreateWiki Extension encountered an error while creating wiki request #86117; https://meta.miraheze.org/wiki/Special:Log/farmer [04:33:54] I've created a ticket for it #T15694 during your next work review [04:37:50] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:39:52] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 4.544 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [04:44:12] PROBLEM - db161 PowerDNS Recursor on db161 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:46:32] PROBLEM - db161 MariaDB on db161 is CRITICAL: Can't connect to server on 'db161.fsslc.wtnet' (115) [04:47:48] PROBLEM - db161 MariaDB Connections on db161 is UNKNOWN: PHP Fatal error: Uncaught mysqli_sql_exception: Connection refused in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db161.fsslc.wtn...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thrown in /usr/lib/nagios/plugins/check_mysql_connect [04:47:48] n line 66Fatal error: Uncaught mysqli_sql_exception: Connection refused in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db161.fsslc.wtn...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thrown in /usr/lib/nagios/plugins/check_mysql_connections.php on line 66 [02MirahezeMagic] 07lihaohong6 opened pull request #658: GenerateExtensionDatabaseList can exclude deleted wikis (07miraheze:03main...07lihaohong6:03exclude) 13https://github.com/miraheze/MirahezeMagic/pull/658 [04:58:10] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 0.65, 0.17, 0.06 [04:58:19] !log [universalomega@cloud16] Reboot db161 from host [04:58:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:58:32] RECOVERY - db161 MariaDB on db161 is OK: Uptime: 48 Threads: 1 Questions: 285 Slow queries: 0 Opens: 326 Open tables: 320 Queries per second avg: 5.937 [04:59:03] miraheze/MirahezeMagic - lihaohong6 the build passed. [04:59:17] RECOVERY - db161 Check unit status of sql-backup on db161 is OK: OK: Status of the systemd unit sql-backup [04:59:20] RECOVERY - db161 APT on db161 is OK: APT OK: 108 packages available for upgrade (0 critical updates). [04:59:38] RECOVERY - db161 PowerDNS Recursor on db161 is OK: DNS OK: 0.049 seconds response time. db161.fsslc.wtnet returns 10.0.16.128 [04:59:44] RECOVERY - db161 MariaDB Connections on db161 is OK: OK connection usage: 0.1%Current connections: 1 [05:01:13] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:06:20] !log [petramagna@mwtask181] starting deploy of {'config': True, 'force': True} to all [05:06:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:06:39] !log [petramagna@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 18s [05:06:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:55:15] [02mw-config] 07The-Voidwalker pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/87d709aa99130c83d0a2d201fc308a15be87baec [07:55:16] 02mw-config/03main 07The-Voidwalker 0387d709a the check for 3D is case sensitive… [07:55:40] !log [void@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [07:55:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:56:03] !log [void@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s [07:56:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:56:35] miraheze/mw-config - The-Voidwalker the build passed. [07:57:29] !log [void@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [07:57:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:57:38] !log [void@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 8s [07:57:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:01:28] [02mw-config] 07The-Voidwalker pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/6924e3f5ae92418276efc5face934fb65338a0d8 [08:01:29] 02mw-config/03main 07The-Voidwalker 036924e3f convert another 3d to 3D… [08:01:42] !log [void@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [08:01:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:02:04] !log [void@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [08:02:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:02:46] miraheze/mw-config - The-Voidwalker the build passed. [10:32:55] miraheze/mw-config - 90gq29 the build passed. [11:05:27] [02mw-config] 07joakim opened pull request #6467: T15698: Add rns:// and nomadnetwork:// to $wgUrlProtocols for reticulum (07miraheze:03main...07joakim:03patch-1) 13https://github.com/miraheze/mw-config/pull/6467 [11:16:31] miraheze/mw-config - joakim the build passed. [12:08:07] PROBLEM - test151 Check unit status of mediawiki_job_manage-inactive-wikis on test151 is CRITICAL: CRITICAL: Status of the systemd unit mediawiki_job_manage-inactive-wikis [12:18:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.87, 6.82, 5.70 [12:19:56] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.84, 6.94, 5.87 [12:20:24] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 9.32, 7.69, 6.15 [12:22:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.60, 7.49, 6.26 [12:24:24] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.28, 7.76, 6.51 [12:25:50] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.50, 6.76, 6.15 [12:29:43] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.50, 7.31, 6.53 [12:32:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.41, 7.86, 7.22 [12:39:32] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.51, 6.72, 6.61 [12:42:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.87, 6.22, 6.77 [12:52:16] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.87, 6.79, 6.46 [12:54:14] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.15, 6.26, 6.29 [12:58:07] PROBLEM - os202 Current Load on os202 is CRITICAL: LOAD CRITICAL - total load average: 8.12, 7.07, 6.60 [12:59:00] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.32, 7.15, 6.85 [13:00:05] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.07, 6.67, 6.51 [13:03:58] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.56, 7.12, 6.73 [13:05:56] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 5.62, 6.70, 6.64 [13:06:36] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.75, 6.64, 6.79 [13:28:42] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.91, 6.90, 6.37 [13:29:25] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.63, 6.60, 6.20 [13:30:36] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.21, 7.30, 6.58 [13:32:30] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.55, 7.38, 6.70 [13:33:21] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.65, 6.66, 6.32 [13:37:16] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.38, 7.16, 6.60 [13:43:16] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.10, 6.77, 6.64 [13:44:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.40, 6.38, 6.59 [13:45:12] [02CreateWiki] 07Reception123 created 03Reception123-patch-7 (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/e9267d4f71ca [13:45:13] 02CreateWiki/03Reception123-patch-7 07Reception123 03e9267d4 test branch [13:49:16] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.23, 7.01, 6.75 [13:50:17] !log [petramagna@test151] starting deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 [13:50:18] !log [petramagna@test151] finished deploy of {'folders': '1.46/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [13:50:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:50:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:50:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.13, 6.97, 6.84 [13:51:16] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.20, 6.73, 6.68 [13:52:33] miraheze/MirahezeMagic - lihaohong6 the build has errored. [13:56:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.49, 6.57, 6.76 [13:58:29] miraheze/MirahezeMagic - lihaohong6 the build has errored. [14:04:03] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 6.95, 6.66, 6.56 [14:04:45] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-7 13https://github.com/miraheze/CreateWiki/commit/b2e53951a0d24a1955ed928584cd2a4be3a481f8 [14:04:45] 02CreateWiki/03Reception123-patch-7 07Reception123 03b2e5395 changes [14:06:01] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 4.78, 5.93, 6.30 [14:10:24] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.48, 6.57, 6.44 [14:12:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.73, 6.19, 6.32 [14:14:53] [02CreateWiki] 07Reception123 04deleted 03Reception123-patch-7 at 03b2e5395 13https://api.github.com/repos/miraheze/CreateWiki/commit/b2e5395 [14:16:48] [02CreateWiki] 07Reception123 created 03Reception123-patch-8 (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/79e1ada74edb [14:16:50] 02CreateWiki/03Reception123-patch-8 07Reception123 0379e1ada popup when user abandons request; clarify that request is back in queue after NMD T15678 [14:17:29] [02CreateWiki] 07Reception123 opened pull request #831: popup after abandon; clarify that request is back in queue after NMD (03main...03Reception123-patch-8) 13https://github.com/miraheze/CreateWiki/pull/831 [14:19:22] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-8 13https://github.com/miraheze/CreateWiki/commit/dc332a641b45d2f54f2c632030fa1bdeb532276d [14:19:22] 02CreateWiki/03Reception123-patch-8 07Reception123 03dc332a6 minor changes [14:23:06] miraheze/CreateWiki - Reception123 the build has errored. [14:27:45] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-8 13https://github.com/miraheze/CreateWiki/commit/ee5bdd90fdaf65cdbb2cbfca197cf0dda9738f9a [14:27:46] 02CreateWiki/03Reception123-patch-8 07Reception123 03ee5bdd9 try to fix ci [14:32:02] miraheze/CreateWiki - Reception123 the build has errored. [15:27:24] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.89, 5.84, 4.37 [15:28:24] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.63, 6.31, 4.33 [15:30:24] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.93, 6.22, 4.55 [15:31:19] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 4.38, 5.56, 4.64 [15:53:59] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5857 bytes in 0.012 second response time [15:55:59] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.286 second response time [16:59:51] [02CreateWiki] 07originalauthority left a review on pull request #831 13https://github.com/miraheze/CreateWiki/pull/831#pullrequestreview-4731136147 [16:59:52] [02CreateWiki] 07originalauthority left a file comment in pull request #831 03ee5bdd9: can use the same user instance as above? 13https://github.com/miraheze/CreateWiki/pull/831#discussion_r3610973626 [17:19:02] [02CreateWiki] 07Reception123 pushed 1 new commit to 03Reception123-patch-8 13https://github.com/miraheze/CreateWiki/commit/5729793c19165ed39622385596fd2f0d6aa38c43 [17:19:02] 02CreateWiki/03Reception123-patch-8 07Reception123 035729793 use same user: instance per OA [17:19:17] [02CreateWiki] 07Reception123 left a file comment in pull request #831 03ee5bdd9: thanks, changed! 13https://github.com/miraheze/CreateWiki/pull/831#discussion_r3611006781 [17:23:22] miraheze/CreateWiki - Reception123 the build has errored. [17:27:29] [02CreateWiki] 07originalauthority commented on pull request #831: LGTM! 13https://github.com/miraheze/CreateWiki/pull/831#issuecomment-5016689733 [17:36:35] [02python-functions] 07renovate[bot] created 03renovate/all-deps (+1 new commit) 13https://github.com/miraheze/python-functions/commit/c93bce97a184 [17:36:35] 02python-functions/03renovate/all-deps 07renovate[bot] 03c93bce9 chore(deps): update pypa/gh-action-pypi-publish action to v1.14.1 [19:01:39] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (1 critical updates). [19:01:44] PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (1 critical updates). [19:01:52] PROBLEM - db181 APT on db181 is CRITICAL: APT CRITICAL: 110 packages available for upgrade (1 critical updates). [19:01:54] PROBLEM - cloud18 APT on cloud18 is CRITICAL: APT CRITICAL: 158 packages available for upgrade (1 critical updates). [19:02:40] PROBLEM - eventgate182 APT on eventgate182 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (1 critical updates). [19:02:52] PROBLEM - db151 APT on db151 is CRITICAL: APT CRITICAL: 110 packages available for upgrade (1 critical updates). [19:02:56] PROBLEM - os191 APT on os191 is CRITICAL: APT CRITICAL: 106 packages available for upgrade (1 critical updates). [19:03:07] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 168 packages available for upgrade (3 critical updates). [19:03:12] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:03:48] PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 116 packages available for upgrade (1 critical updates). [19:03:56] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 168 packages available for upgrade (3 critical updates). [19:04:18] PROBLEM - os161 APT on os161 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (1 critical updates). [19:04:20] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 116 packages available for upgrade (1 critical updates). [19:04:34] PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 99 packages available for upgrade (1 critical updates). [19:04:36] PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 108 packages available for upgrade (1 critical updates). [19:05:05] PROBLEM - mw201 APT on mw201 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:05:57] PROBLEM - db192 APT on db192 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:06:00] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 158 packages available for upgrade (1 critical updates). [19:06:02] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 68 packages available for upgrade (1 critical updates). [19:06:06] PROBLEM - cloud20 APT on cloud20 is CRITICAL: APT CRITICAL: 164 packages available for upgrade (1 critical updates). [19:06:07] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:06:15] PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 103 packages available for upgrade (1 critical updates). [19:06:16] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:06:21] PROBLEM - swiftobject211 APT on swiftobject211 is CRITICAL: APT CRITICAL: 106 packages available for upgrade (1 critical updates). [19:07:18] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:08:09] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 158 packages available for upgrade (1 critical updates). [19:08:41] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:08:52] PROBLEM - mem201 APT on mem201 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (1 critical updates). [19:09:03] PROBLEM - changeprop202 APT on changeprop202 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (1 critical updates). [19:09:04] PROBLEM - cp201 APT on cp201 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:09:35] PROBLEM - db172 APT on db172 is CRITICAL: APT CRITICAL: 110 packages available for upgrade (1 critical updates). [19:09:47] PROBLEM - mattermost2 APT on mattermost2 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:10:11] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 112 packages available for upgrade (1 critical updates). [19:10:14] PROBLEM - mw202 APT on mw202 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:11:05] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:11:28] PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 112 packages available for upgrade (1 critical updates). [19:11:39] PROBLEM - mw203 APT on mw203 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:11:59] PROBLEM - rdb191 APT on rdb191 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:12:02] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:12:11] PROBLEM - llm191 APT on llm191 is CRITICAL: APT CRITICAL: 113 packages available for upgrade (1 critical updates). [19:12:56] PROBLEM - bast161 APT on bast161 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (1 critical updates). [19:13:08] PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:13:25] PROBLEM - cp171 APT on cp171 is CRITICAL: APT CRITICAL: 102 packages available for upgrade (1 critical updates). [19:13:30] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 167 packages available for upgrade (3 critical updates). [19:13:50] PROBLEM - mw173 APT on mw173 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:14:06] PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 57 packages available for upgrade (1 critical updates). [19:14:09] PROBLEM - swiftobject191 APT on swiftobject191 is CRITICAL: APT CRITICAL: 117 packages available for upgrade (1 critical updates). [19:14:14] PROBLEM - swiftobject201 APT on swiftobject201 is CRITICAL: APT CRITICAL: 117 packages available for upgrade (1 critical updates). [19:14:34] PROBLEM - swiftac171 APT on swiftac171 is CRITICAL: APT CRITICAL: 112 packages available for upgrade (1 critical updates). [19:14:39] PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 102 packages available for upgrade (1 critical updates). [19:14:47] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 166 packages available for upgrade (3 critical updates). [19:14:47] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 166 packages available for upgrade (3 critical updates). [19:14:53] PROBLEM - mem161 APT on mem161 is CRITICAL: APT CRITICAL: 99 packages available for upgrade (1 critical updates). [19:15:14] PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 117 packages available for upgrade (1 critical updates). [19:16:47] PROBLEM - db161 APT on db161 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (1 critical updates). [19:17:42] PROBLEM - os201 APT on os201 is CRITICAL: APT CRITICAL: 106 packages available for upgrade (1 critical updates). [19:17:47] PROBLEM - mem151 APT on mem151 is CRITICAL: APT CRITICAL: 99 packages available for upgrade (1 critical updates). [19:18:25] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:18:55] PROBLEM - mw193 APT on mw193 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:19:02] PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (1 critical updates). [19:19:17] PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 164 packages available for upgrade (1 critical updates). [19:20:54] PROBLEM - db171 APT on db171 is CRITICAL: APT CRITICAL: 110 packages available for upgrade (1 critical updates). [19:21:11] PROBLEM - cloud21 APT on cloud21 is CRITICAL: APT CRITICAL: 156 packages available for upgrade (1 critical updates). [19:21:30] PROBLEM - db182 APT on db182 is CRITICAL: APT CRITICAL: 116 packages available for upgrade (1 critical updates). [19:22:00] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:22:03] PROBLEM - mw153 APT on mw153 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:22:22] PROBLEM - db201 APT on db201 is CRITICAL: APT CRITICAL: 121 packages available for upgrade (1 critical updates). [19:23:25] PROBLEM - mw191 APT on mw191 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:24:27] Sigh [19:24:32] I suppose I should look [19:24:41] PROBLEM - kafka182 APT on kafka182 is CRITICAL: APT CRITICAL: 112 packages available for upgrade (1 critical updates). [19:25:17] [02mw-config] 07dmehus opened pull request #6468: Add extendedconfirmed protection level and user group for otshestwiki (07miraheze:03main...07dmehus:03patch-17) 13https://github.com/miraheze/mw-config/pull/6468 [19:25:22] PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 114 packages available for upgrade (1 critical updates). [19:25:27] PROBLEM - cloud19 APT on cloud19 is CRITICAL: APT CRITICAL: 164 packages available for upgrade (1 critical updates). [19:25:59] PROBLEM - cp191 APT on cp191 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:26:28] miraheze/mw-config - dmehus the build has errored. [19:27:03] PROBLEM - cp161 APT on cp161 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates). [19:27:05] PROBLEM - mem191 APT on mem191 is CRITICAL: APT CRITICAL: 98 packages available for upgrade (1 critical updates). [19:27:12] !log [rhinos@puppet181] Upgraded packages on mem201: libtiff6 [19:27:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:27:26] PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 116 packages available for upgrade (1 critical updates). [19:27:29] !log [rhinos@puppet181] Upgraded packages on mem191: libtiff6 [19:27:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:17] PROBLEM - mw192 APT on mw192 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:28:31] !log [rhinos@puppet181] Upgraded packages on bast161: libtiff6 [19:28:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:41] PROBLEM - graylog161 APT on graylog161 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (1 critical updates). [19:28:45] !log [rhinos@puppet181] Upgraded packages on db172: libtiff6 [19:28:47] RECOVERY - mem201 APT on mem201 is OK: APT OK: 97 packages available for upgrade (0 critical updates). [19:28:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:59] !log [rhinos@puppet181] Upgraded packages on bots171: libtiff6 [19:29:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:07] RECOVERY - mem191 APT on mem191 is OK: APT OK: 97 packages available for upgrade (0 critical updates). [19:29:09] RECOVERY - bast161 APT on bast161 is OK: APT OK: 106 packages available for upgrade (0 critical updates). [19:29:12] !log [rhinos@puppet181] Upgraded packages on cloud21: libtiff6 [19:29:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:16] RECOVERY - cloud21 APT on cloud21 is OK: APT OK: 155 packages available for upgrade (0 critical updates). [19:29:25] !log [rhinos@puppet181] Upgraded packages on db151: libtiff6 [19:29:27] RECOVERY - bots171 APT on bots171 is OK: APT OK: 113 packages available for upgrade (0 critical updates). [19:29:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:34] RECOVERY - db172 APT on db172 is OK: APT OK: 109 packages available for upgrade (0 critical updates). [19:29:39] !log [rhinos@puppet181] Upgraded packages on db161: libtiff6 [19:29:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:43] PROBLEM - os162 APT on os162 is CRITICAL: APT CRITICAL: 107 packages available for upgrade (1 critical updates). [19:29:44] miraheze/mw-config - dmehus the build passed. [19:29:54] !log [rhinos@puppet181] Upgraded packages on cloud15: libtiff6 [19:29:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:00] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:30:06] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 161 packages available for upgrade (3 critical updates). [19:30:08]