[00:00:56] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/1ceda29c31f75c9bc4a40f4d188abbb888fe124b [00:00:56] 02statichelp/03main 07WikiTideBot 031ceda29 Bot: Auto-update Tech namespace pages 2026-02-24 00:00:54 [00:07:48] [02mediawiki-repos] 07Universal-Omega created 03private-submodule (+1 new commit) 13https://github.com/miraheze/mediawiki-repos/commit/b2dd5c6517e0 [00:07:48] 02mediawiki-repos/03private-submodule 07CosmicAlpha 03b2dd5c6 Add submodule for private patches [00:08:12] [02mw-config] 07SomeMWDev created 03excimer (+1 new commit) 13https://github.com/miraheze/mw-config/commit/7bbca331bfa7 [00:08:12] 02mw-config/03excimer 07SomeRandomDeveloper 037bbca33 T14537: Allow profiling requests using excimer… [00:08:14] [02mediawiki-repos] 07Universal-Omega opened pull request #124: Add submodule for private patches (03main...03private-submodule) 13https://github.com/miraheze/mediawiki-repos/pull/124 [00:08:57] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:08:58] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:08:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:09:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:12:08] [02puppet] 07Universal-Omega created 03private-patches (+1 new commit) 13https://github.com/miraheze/puppet/commit/4efaecfec852 [00:12:10] 02puppet/03private-patches 07CosmicAlpha 034efaecf Support new private repository for private patches [00:12:13] [02puppet] 07Universal-Omega opened pull request #4789: Support new private repository for private patches (03main...03private-patches) 13https://github.com/miraheze/puppet/pull/4789 [00:13:20] [02puppet] 07Universal-Omega pushed 1 new commit to 03private-patches 13https://github.com/miraheze/puppet/commit/b5ea857dc8752118a5fcf0bb9ca7362e784e3d52 [00:13:20] 02puppet/03private-patches 07CosmicAlpha 03b5ea857 Fix [00:14:39] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 77515MiB (17.1% inode=100%); [00:17:32] [02puppet] 07Universal-Omega pushed 1 new commit to 03private-patches 13https://github.com/miraheze/puppet/commit/a2deec8f0f953ee87933f237eaaebc3426e6f261 [00:17:32] 02puppet/03private-patches 07CosmicAlpha 03a2deec8 Use auth for cloning mediawiki-repos [00:17:40] [02mw-config] 07SomeMWDev pushed 1 new commit to 03excimer 13https://github.com/miraheze/mw-config/commit/227cef44c14cc47afda8445532da7bbd9ac1690c [00:17:40] 02mw-config/03excimer 07SomeRandomDeveloper 03227cef4 Use application/json script [00:18:00] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:18:01] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:18:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:18:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:20:24] [02mw-config] 07SomeMWDev pushed 1 new commit to 03excimer 13https://github.com/miraheze/mw-config/commit/c77ac7149861623bdbb3ffe85f62dcfc6b6d20a1 [00:20:24] 02mw-config/03excimer 07SomeRandomDeveloper 03c77ac71 try using deferred update [00:20:38] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:20:39] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:20:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:20:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:52] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 76732MiB (16.9% inode=100%); [00:25:24] [02mw-config] 07SomeMWDev pushed 1 new commit to 03excimer 13https://github.com/miraheze/mw-config/commit/afdccba0340e6c08c9ff7cd3b39e3ce078983d87 [00:25:24] 02mw-config/03excimer 07SomeRandomDeveloper 03afdccba use shutdown function again [00:25:31] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:25:32] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:25:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:25] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/24240e1f0a1b9275a59bf09909c2726be429ba94 [00:30:25] 02ssl/03main 07WikiTideBot 0324240e1 Bot: Auto-update domain lists [00:33:44] [02mw-config] 07SomeMWDev pushed 1 new commit to 03excimer 13https://github.com/miraheze/mw-config/commit/3c35557dfac10422cd4ec156e47e9ce442002806 [00:33:44] 02mw-config/03excimer 07SomeRandomDeveloper 033c35557 temporarily simplify if statement for testing [00:34:11] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:34:12] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:34:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:34:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:32] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:36:33] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:36:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:38:47] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:38:48] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:38:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:38:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:46:05] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:46:06] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:46:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:46:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:59:32] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 72322MiB (16.0% inode=100%); [01:55:17] [02mediawiki-repos] 07Universal-Omega created 03rm-patch (+1 new commit) 13https://github.com/miraheze/mediawiki-repos/commit/7f3a204678e5 [01:55:18] 02mediawiki-repos/03rm-patch 07CosmicAlpha 037f3a204 Remove DiscussionTools patch… [01:55:28] [02mediawiki-repos] 07Universal-Omega opened pull request #125: Remove DiscussionTools patch (03main...03rm-patch) 13https://github.com/miraheze/mediawiki-repos/pull/125 [01:55:47] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03rm-patch 13https://github.com/miraheze/mediawiki-repos/commit/55a51f2443bf72daa2d268e29be6fd53e9634954 [01:55:48] 02mediawiki-repos/03rm-patch 07CosmicAlpha 0355a51f2 - [01:56:48] [02mediawiki-repos] 07Universal-Omega merged pull request #125: Remove DiscussionTools patch (03main...03rm-patch) 13https://github.com/miraheze/mediawiki-repos/pull/125 [01:56:48] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/3c78f4bb46cea6f03d30d2f2c2005f81a0e37e92 [01:56:48] 02mediawiki-repos/03main 07CosmicAlpha 033c78f4b Remove DiscussionTools patch… [01:56:49] [02mediawiki-repos] 07Universal-Omega 04deleted 03rm-patch at 0355a51f2 13https://api.github.com/repos/miraheze/mediawiki-repos/commit/55a51f2 [02:02:01] !log [universalomega@mwtask181] starting deploy of {'versions': '1.45', 'upgrade_extensions': 'DiscussionTools'} to all [02:02:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:02:27] !log [universalomega@mwtask181] finished deploy of {'versions': '1.45', 'upgrade_extensions': 'DiscussionTools'} to all - SUCCESS in 26s [02:02:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:56:43] !log [petramagna@test151] starting deploy of {'l10n': True, 'versions': '1.45'} to test151 [02:56:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:00:28] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/b395116c5d81b5356f3b366f3f4d8362f3e8a47c [03:00:28] 02ssl/03main 07WikiTideBot 03b395116 Bot: Auto-update domain lists [03:02:37] !log [petramagna@test151] finished deploy of {'l10n': True, 'versions': '1.45'} to test151 - SUCCESS in 354s [03:02:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:42:39] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw172 [09:43:01] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 1 backends are down. mw172 [09:43:05] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 31.95, 25.89, 14.83 [09:44:39] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [09:45:00] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 11.98, 19.99, 13.94 [09:45:01] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [11:14:38] !log [blankeclair@mwtask171] runRefreshLinksChunked: [0/1] shintowiki exited with 137 [11:14:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:14:43] h [11:16:07] "Running final refreshLinks.php in case there are more pages beyond 97365" oops, oomed at 9600 [11:16:57] !log [blankeclair@mwtask171] runRefreshLinksChunked: [0/1] Handling shintowiki (start id: 106965) [11:16:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:27:27] !log [blankeclair@mwtask171] runRefreshLinksChunked: [1/1] Done \o/ [11:27:36] wee! [11:28:39] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 10 backends are down. mw151 mw161 mw171 mw182 mw153 mw163 mw173 mw193 mw201 mw203 [11:29:01] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 5 backends are down. mw151 mw172 mw181 mw201 mw202 [11:29:58] Hey BlankEclair [11:30:04] meow? [11:30:17] just saying hi [11:30:30] ah oki :3 [11:30:34] hello <3 [11:30:39] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [11:31:01] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [12:11:45] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.12, 16.43, 12.12 [12:13:45] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 12.27, 15.07, 12.18 [14:27:46] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 23.01, 19.33, 16.45 [14:29:46] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 12.46, 16.57, 15.79 [15:34:07] [02TSPortal] 07renovate[bot] created 03renovate/all-deps (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/bdbc44becbd3 [15:34:07] 02TSPortal/03renovate/all-deps 07renovate[bot] 03bdbc44b Update all dependencies [16:29:12] PROBLEM - swiftproxy171 HTTP on swiftproxy171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 503 Service Unavailable [16:30:20] PROBLEM - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift_dispersion_stats [16:31:12] RECOVERY - swiftproxy171 HTTP on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.017 second response time [16:44:20] RECOVERY - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is OK: OK: Status of the systemd unit swift_dispersion_stats [16:45:25] PROBLEM - ping6 on cloud20 is CRITICAL: PING CRITICAL - Packet loss = 50%, RTA = 15.87 ms [16:45:26] PROBLEM - ping6 on cloud21 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 0.36 ms [16:45:27] PROBLEM - ping6 on cloud15 is CRITICAL: PING CRITICAL - Packet loss = 44%, RTA = 0.36 ms [16:47:24] RECOVERY - ping6 on cloud20 is OK: PING OK - Packet loss = 0%, RTA = 0.31 ms [16:47:25] RECOVERY - ping6 on cloud21 is OK: PING OK - Packet loss = 0%, RTA = 0.32 ms [16:47:27] RECOVERY - ping6 on cloud15 is OK: PING OK - Packet loss = 0%, RTA = 0.34 ms [17:13:32] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45226MiB (10.0% inode=100%); [18:02:28] [02puppet] 07Universal-Omega pushed 1 new commit to 03private-patches 13https://github.com/miraheze/puppet/commit/426e6bb7d822c602acd7f267d822f9a28b3f2883 [18:02:28] 02puppet/03private-patches 07CosmicAlpha 03426e6bb Fix [18:03:04] [02mediawiki-repos] 07Universal-Omega closed pull request #124: Add submodule for private patches (03main...03private-submodule) 13https://github.com/miraheze/mediawiki-repos/pull/124 [18:03:06] [02puppet] 07github-actions[bot] pushed 1 new commit to 03private-patches 13https://github.com/miraheze/puppet/commit/39da469e0ffea754dc1cfe3dc3d441e474c865af [18:03:06] 02puppet/03private-patches 07github-actions 0339da469 CI: lint puppet code to standards… [18:03:09] [02mediawiki-repos] 07Universal-Omega 04deleted 03private-submodule at 03b2dd5c6 13https://api.github.com/repos/miraheze/mediawiki-repos/commit/b2dd5c6 [18:45:17] [02TSPortal] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/bdbc44becbd3a9ee9f6ca96697c4a0b2e454454f [18:45:17] 02TSPortal/03main 07renovate[bot] 03bdbc44b Update all dependencies [18:45:18] [02TSPortal] 07renovate[bot] 04deleted 03renovate/all-deps at 03bdbc44b 13https://api.github.com/repos/miraheze/TSPortal/commit/bdbc44b [18:49:41] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/b714c244fc3644526a3e3e4ba9006b925bab92d5 [18:49:42] 02puppet/03main 07CosmicAlpha 03b714c24 Use ImageMagick 7 now [19:00:52] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 45262MiB (10.0% inode=100%); [19:15:56] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 55081MiB (12.1% inode=100%); [19:16:02] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 51610MiB (11.4% inode=100%); [19:18:38] PROBLEM - swiftproxy161 HTTPS on swiftproxy161 is CRITICAL: connect to address 10.0.16.135 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [19:18:40] PROBLEM - swiftproxy171 Current Load on swiftproxy171 is CRITICAL: LOAD CRITICAL - total load average: 13.34, 8.13, 4.16 [19:19:36] PROBLEM - swiftproxy171 HTTP on swiftproxy171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:21:01] RECOVERY - swiftproxy171 HTTP on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 5.786 second response time [19:21:10] PROBLEM - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is CRITICAL: connect to address localhost and port 443: Connection refused [19:21:24] PROBLEM - swiftproxy171 HTTPS on swiftproxy171 is CRITICAL: connect to address 10.0.17.108 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [19:22:49] PROBLEM - swiftproxy171 Current Load on swiftproxy171 is WARNING: LOAD WARNING - total load average: 2.77, 7.45, 5.03 [19:22:54] PROBLEM - swiftproxy171 Swift NGINX SSL check on swiftproxy171 is CRITICAL: connect to address localhost and port 443: Connection refused [19:23:18] RECOVERY - swiftproxy171 Current Load on swiftproxy171 is OK: LOAD OK - total load average: 1.74, 6.75, 4.87 [19:24:21] RECOVERY - swiftproxy161 HTTPS on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 2.852 second response time [19:24:32] RECOVERY - swiftproxy171 HTTPS on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 8.844 second response time [19:24:36] RECOVERY - swiftproxy171 Swift NGINX SSL check on swiftproxy171 is OK: OK - Certificate 'wikitide.net' will expire on Fri Mar 13 15:30:27 2026 +0000.TCP OK - 0.013 second response time on localhost port 443 [19:24:39] RECOVERY - swiftproxy161 Swift NGINX SSL check on swiftproxy161 is OK: OK - Certificate 'wikitide.net' will expire on Fri Mar 13 15:30:27 2026 +0000.TCP OK - 0.010 second response time on localhost port 443 [19:27:11] PROBLEM - swiftproxy171 Current Load on swiftproxy171 is CRITICAL: LOAD CRITICAL - total load average: 12.64, 8.93, 6.05 [19:27:17] PROBLEM - swiftproxy161 Current Load on swiftproxy161 is CRITICAL: LOAD CRITICAL - total load average: 10.26, 6.62, 4.10 [19:28:20] PROBLEM - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is CRITICAL: CRITICAL: Status of the systemd unit swift_dispersion_stats [19:29:25] RECOVERY - swiftproxy161 Current Load on swiftproxy161 is OK: LOAD OK - total load average: 6.55, 6.64, 4.45 [19:30:41] excuse me [19:33:22] PROBLEM - swiftproxy161 Current Load on swiftproxy161 is WARNING: LOAD WARNING - total load average: 7.56, 7.24, 5.20 [19:35:21] RECOVERY - swiftproxy161 Current Load on swiftproxy161 is OK: LOAD OK - total load average: 3.80, 6.08, 5.04 [19:36:07] PROBLEM - swiftproxy171 Current Load on swiftproxy171 is WARNING: LOAD WARNING - total load average: 5.83, 7.76, 7.09 [19:38:37] RECOVERY - swiftproxy171 Current Load on swiftproxy171 is OK: LOAD OK - total load average: 5.30, 6.57, 6.72 [19:40:27] nginx OOMed it seems. [19:40:46] RECOVERY - swiftproxy171 Check unit status of swift_dispersion_stats on swiftproxy171 is OK: OK: Status of the systemd unit swift_dispersion_stats [19:47:51] [02puppet] 07paladox created 03paladox-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/06e8db112ad7 [19:47:51] 02puppet/03paladox-patch-1 07paladox 0306e8db1 ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte" [19:47:55] [02puppet] 07paladox opened pull request #4790: ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decod… (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4790 [19:48:00] [02puppet] 07paladox retitled PR #4790: "ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decod…" ➜ "ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decod…e byte"" 13https://github.com/miraheze/puppet/pull/4790 [19:48:05] [02puppet] 07paladox retitled PR #4790: "ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decod…e byte"" ➜ "ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte"" 13https://github.com/miraheze/puppet/pull/4790 [19:57:22] [02mediawiki-repos] 07YTFGolf opened pull request #126: T13977: install TableProgressTracking extension (07miraheze:03main...07YTFGolf:03TableProgressTracking) 13https://github.com/miraheze/mediawiki-repos/pull/126 [20:01:33] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-1 13https://github.com/miraheze/puppet/commit/cf3ad86f3c1bb005acf3123d51bd98126202416b [20:01:33] 02puppet/03paladox-patch-1 07paladox 03cf3ad86 Update lineReceived to decode with error handling… [20:01:57] [02puppet] 07paladox merged pull request #4790: ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte" (03main...03paladox-patch-1) 13https://github.com/miraheze/puppet/pull/4790 [20:01:57] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/ae584307d7e6afbb4dc07bab8236e8cd9720e139 [20:01:57] 02puppet/03main 07paladox 03ae58430 ircrcbot: Fix "builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte" (#4790) [20:01:59] [02puppet] 07paladox 04deleted 03paladox-patch-1 at 03cf3ad86 13https://api.github.com/repos/miraheze/puppet/commit/cf3ad86 [20:03:12] miraheze/puppet - paladox the build passed. [20:03:37] [02puppet] 07Universal-Omega 04force-pushed 03private-patches from 0339da469 to 0303912e3 13https://api.github.com/repos/miraheze/puppet/commits/private-patches [20:03:37] 02puppet/03private-patches 07CosmicAlpha 039faaf5b Support new private repository for private patches [20:03:37] 02puppet/03private-patches 07CosmicAlpha 03c954969 Fix [20:03:37] 02puppet/03private-patches 07CosmicAlpha 03eb5ea08 Use auth for cloning mediawiki-repos [20:03:38] 02puppet/03private-patches 07CosmicAlpha 030cae760 Fix [20:03:40] 02puppet/03private-patches 07github-actions 0303912e3 CI: lint puppet code to standards… [20:03:40] miraheze/puppet - paladox the build passed. [20:07:28] [02puppet] 07Universal-Omega merged pull request #4789: Support new private repository for private patches (03main...03private-patches) 13https://github.com/miraheze/puppet/pull/4789 [20:07:28] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/ac7b246b2314e16ce1c92d1d52c42d04f749ff97 [20:07:28] 02puppet/03main 07CosmicAlpha 03ac7b246 Support new private repository for private patches [20:07:30] [02puppet] 07Universal-Omega 04deleted 03private-patches at 0303912e3 13https://api.github.com/repos/miraheze/puppet/commit/03912e3 [20:12:18] [02mw-config] 07YTFGolf opened pull request #6329: T13977: install TableProgressTracking extension (07miraheze:03main...07YTFGolf:03patch-T13977) 13https://github.com/miraheze/mw-config/pull/6329 [20:13:38] miraheze/mw-config - YTFGolf the build passed. [20:16:00] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/b274c67b6b62c99a68f6be7b3e5ebed2072e5f81 [20:16:00] 02mediawiki-repos/03main 07CosmicAlpha 03b274c67 Always true for this one [20:34:04] PROBLEM - mw163 PowerDNS Recursor on mw163 is CRITICAL: CRITICAL - Plugin timed out while executing system call [20:34:14] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [20:34:39] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw163 [20:35:18] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 22.94, 23.24, 13.11 [20:36:00] RECOVERY - mw163 PowerDNS Recursor on mw163 is OK: DNS OK: 0.079 seconds response time. mw163.fsslc.wtnet returns 10.0.16.151 [20:36:10] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4288 bytes in 0.064 second response time [20:36:39] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [20:37:18] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 7.53, 17.29, 12.15 [20:40:24] PROBLEM - mw163 PowerDNS Recursor on mw163 is CRITICAL: CRITICAL - Plugin timed out while executing system call [20:40:32] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:40:36] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [20:40:39] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 1 backends are down. mw163 [20:40:45] PROBLEM - mw163 php-fpm on mw163 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [20:40:53] PROBLEM - mw163 SSH on mw163 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:41:01] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 1 backends are down. mw163 [20:41:26] PROBLEM - mw163 Current Load on mw163 is CRITICAL: LOAD CRITICAL - total load average: 56.91, 37.22, 21.31 [20:41:32] PROBLEM - cp161 Varnish Backends on cp161 is CRITICAL: 1 backends are down. mw163 [20:41:52] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 1 backends are down. mw163 [20:42:20] RECOVERY - mw163 PowerDNS Recursor on mw163 is OK: DNS OK: 0.054 seconds response time. mw163.fsslc.wtnet returns 10.0.16.151 [20:42:26] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.200 second response time [20:42:32] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4288 bytes in 0.076 second response time [20:42:39] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [20:42:43] RECOVERY - mw163 php-fpm on mw163 is OK: PROCS OK: 25 processes with command name 'php-fpm8.4' [20:42:49] RECOVERY - mw163 SSH on mw163 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [20:43:01] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [20:43:32] RECOVERY - cp161 Varnish Backends on cp161 is OK: All 31 backends are healthy [20:43:52] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [20:45:18] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 7.36, 20.86, 18.13 [20:47:18] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 4.94, 15.46, 16.47 [21:06:27] [02mediawiki-repos] 07lihaohong6 merged 07YTFGolf's pull request #126: T13977: install TableProgressTracking extension (07miraheze:03main...07YTFGolf:03TableProgressTracking) 13https://github.com/miraheze/mediawiki-repos/pull/126 [21:06:29] [02mediawiki-repos] 07lihaohong6 pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/45f50358db78b39117cd3511004d6a49220e6678 [21:06:30] 02mediawiki-repos/03main 07YTFGolf 0345f5035 T13977: install TableProgressTracking extension (#126) [21:11:35] @posix_memalign heads up: repo of that says it is incompatible with php 8.4 [21:27:42] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is WARNING: WARNING - NGINX Error Rate is 52% [21:29:42] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 15% [21:34:10] [1/3] That's unfortunate. It's not apparent to me why this is a problem for 8.4 since [21:34:10] [2/3] > Prior to PHP 8.4.0 this resulted in an unusable object. [21:34:10] [3/3] I guess I'll find out once the mw-config PR is deployed to test151. [22:18:18] !log [blankeclair@mwtask171] Starting import for animatorvsanimationwiki (XML: animatorvsanimation_pages_full.xml?dl=1&noscript=1&rlkey=2srmcxqq8utphzlhejttan4wn&st=gxhjxdyr; Images: None) (START) [22:18:19] !log [blankeclair@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=animatorvsanimationwiki --no-updates --username-prefix=fandom:animatorvsanimation -- 'animatorvsanimation_pages_full.xml?dl=1&noscript=1&rlkey=2srmcxqq8utphzlhejttan4wn&st=gxhjxdyr' (START) [22:18:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:18:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:18:37] why yes, i use wget, how did you know? [22:27:41] !log [blankeclair@mwtask151] Starting import for badorkbeewiki (XML: None; Images: all_images/) (START) [22:27:42] !log [blankeclair@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importImages --wiki=badorkbeewiki --sleep=1 '--comment=Importing images from https://badorkbee-games.fandom.com ([[phorge:T15000|T15000]])' -- all_images/ (START) [22:27:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:39:32] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45267MiB (10.0% inode=100%); [22:42:43] !log [blankeclair@mwtask161] Starting import for cavewiki (XML: None; Images: images/) (START) [22:42:44] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importImages --wiki=cavewiki --sleep=1 '--comment=Importing images from https://speleo.miraheze.org ([[phorge:T14886|T14886]])' -- images/ (START) [22:42:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:42:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:49] !log [blankeclair@mwtask161] Starting import for funorbwiki (XML: funorb_pages_full.xml; Images: None) (START) [22:44:50] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=funorbwiki --no-updates --username-prefix=fandom:funorb -- funorb_pages_full.xml (START) [22:44:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:55] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=funorbwiki --no-updates --username-prefix=fandom:funorb -- funorb_pages_full.xml (END - exit=1) [22:44:56] !log [blankeclair@mwtask161] Finished import for funorbwiki (XML: funorb_pages_full.xml; Images: None) (END - exit=1) [22:44:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:45:28] !log [petramagna@test151] starting deploy of {'l10n': True, 'extension_list': True, 'versions': '1.45', 'upgrade_extensions': 'TableProgressTracking'} to test151 [22:45:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:46:11] !log [blankeclair@mwtask161] Starting import for funorbwiki (XML: funorb_pages_full.xml; Images: None) (START) [22:46:12] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=funorbwiki --no-updates --username-prefix=fandom:funorb -- funorb_pages_full.xml (START) [22:46:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:46:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:50:47] !log [petramagna@test151] finished deploy of {'l10n': True, 'extension_list': True, 'versions': '1.45', 'upgrade_extensions': 'TableProgressTracking'} to test151 - SUCCESS in 319s [22:50:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:52:59] !log [petramagna@test151] starting deploy of {'config': True} to test151 [22:53:00] !log [petramagna@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [22:53:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:53:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:53:54] @thewwrnerdguy TableProgressTracking is available on Mirabeta now. [22:54:23] I split a single mwdeploy command into 2 so maybe it'll cause issues, but at least the extension showed up on ManageWiki [23:01:21] [02puppet] 07Universal-Omega created 03mwdeploy-non-git (+1 new commit) 13https://github.com/miraheze/puppet/commit/bd3dbe2109e4 [23:01:21] 02puppet/03mwdeploy-non-git 07CosmicAlpha 03bd3dbe2 mwdeploy: better support for non-git repos [23:01:30] [02puppet] 07Universal-Omega opened pull request #4791: mwdeploy: better support for non-git repos (03main...03mwdeploy-non-git) 13https://github.com/miraheze/puppet/pull/4791 [23:02:52] miraheze/puppet - Universal-Omega the build has errored. [23:07:59] [02puppet] 07Universal-Omega pushed 1 new commit to 03mwdeploy-non-git 13https://github.com/miraheze/puppet/commit/04e39a4180a8fd05761739b800a8dd6629e5749c [23:08:00] 02puppet/03mwdeploy-non-git 07CosmicAlpha 0304e39a4 Deploy directories [23:10:28] miraheze/puppet - Universal-Omega the build has errored. [23:11:09] [02puppet] 07Universal-Omega pushed 1 new commit to 03mwdeploy-non-git 13https://github.com/miraheze/puppet/commit/512d309e4afe08bdf6f4647da1b33325fab74fc8 [23:11:09] 02puppet/03mwdeploy-non-git 07CosmicAlpha 03512d309 Fix [23:12:39] miraheze/puppet - Universal-Omega the build passed. [23:15:51] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=funorbwiki --no-updates --username-prefix=fandom:funorb -- funorb_pages_full.xml (END - exit=0) [23:15:52] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=funorbwiki (START) [23:15:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:15:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:18:07] Seems to work fine although not sure what edge cases need to be checked [23:19:25] !log [blankeclair@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=animatorvsanimationwiki --no-updates --username-prefix=fandom:animatorvsanimation -- 'animatorvsanimation_pages_full.xml?dl=1&noscript=1&rlkey=2srmcxqq8utphzlhejttan4wn&st=gxhjxdyr' (END - exit=0) [23:19:26] !log [blankeclair@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=animatorvsanimationwiki (START) [23:19:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:19:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:52] [02puppet] 07Universal-Omega pushed 1 new commit to 03mwdeploy-non-git 13https://github.com/miraheze/puppet/commit/78b1661c4cbd3fdfc0002237ca3458495999fd5d [23:20:52] 02puppet/03mwdeploy-non-git 07CosmicAlpha 0378b1661 - [23:22:25] miraheze/puppet - Universal-Omega the build passed. [23:23:40] !log [blankeclair@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importImages --wiki=badorkbeewiki --sleep=1 '--comment=Importing images from https://badorkbee-games.fandom.com ([[phorge:T15000|T15000]])' -- all_images/ (END - exit=0) [23:23:41] !log [blankeclair@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=badorkbeewiki --update (START) [23:23:42] !log [blankeclair@mwtask151] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=badorkbeewiki --update (END - exit=0) [23:23:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:23:43] !log [blankeclair@mwtask151] Finished import for badorkbeewiki (XML: None; Images: all_images/) (END - exit=0) [23:23:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:23:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:23:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:31] [02puppet] 07Universal-Omega merged pull request #4791: mwdeploy: better support for non-git repos (03main...03mwdeploy-non-git) 13https://github.com/miraheze/puppet/pull/4791 [23:27:32] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/97106dc5c986748201332f7c8061a54d6aa81ef1 [23:27:32] 02puppet/03main 07CosmicAlpha 0397106dc mwdeploy: better support for non-git repos [23:27:33] [02puppet] 07Universal-Omega 04deleted 03mwdeploy-non-git at 0378b1661 13https://api.github.com/repos/miraheze/puppet/commit/78b1661 [23:29:04] miraheze/puppet - Universal-Omega the build passed. [23:30:47] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/281549f52e34fffe0374bea38e3d84687b0e93ff [23:30:47] 02puppet/03main 07CosmicAlpha 03281549f mwdeploy: don't attempt to automatically reverse patches if already applied [23:33:01] miraheze/puppet - Universal-Omega the build passed. [23:39:02] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=funorbwiki (END - exit=0) [23:39:03] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=funorbwiki (START) [23:39:04] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=funorbwiki (END - exit=0) [23:39:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:05] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=funorbwiki --update (START) [23:39:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:06] !log [blankeclair@mwtask161] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=funorbwiki --update (END - exit=0) [23:39:07] !log [blankeclair@mwtask161] Finished import for funorbwiki (XML: funorb_pages_full.xml; Images: None) (END - exit=0) [23:39:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:07] [02mediawiki-repos] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/b7b557084a2be6fd498240ea2b1894ac13ff07fb [23:53:07] 02mediawiki-repos/03main 07CosmicAlpha 03b7b5570 Use discard for Maps patcb