[00:00:58] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/3693d31993065d86788a152a19f8ab5393a15673 [00:00:58] 02statichelp/03main 07WikiTideBot 033693d31 Bot: Auto-update Tech namespace pages 2026-03-15 00:00:55 [01:22:53] [02TSPortal] 07renovate[bot] created 03renovate/lock-file-maintenance (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/f49018a0386a [01:22:55] 02TSPortal/03renovate/lock-file-maintenance 07renovate[bot] 03f49018a Lock file maintenance [01:44:39] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 11.04, 13.55, 6.41 [01:46:39] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 1.97, 9.25, 5.70 [05:15:37] [02TSPortal] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/f49018a0386ae7a42e4c8e18a1f5b8817aabe3f4 [05:15:37] 02TSPortal/03main 07renovate[bot] 03f49018a Lock file maintenance [05:15:37] [02TSPortal] 07renovate[bot] 04deleted 03renovate/lock-file-maintenance at 03f49018a 13https://api.github.com/repos/miraheze/TSPortal/commit/f49018a [06:38:35] [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553010000&orgId=1&to=1773556715089 [06:53:35] [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553010000&orgId=1&to=1773557615092[Grafana] FIRING: An unusually high number of threats are being reported by CloudFlare! https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553860000&orgId=1&to=1773557615092[Grafana] FIRING: A MediaWiki pool is sick according to CloudFlare https://grafana.wikitide.net/d/GtxbP1Xnk?from [06:53:35] 0000&orgId=1&to=1773557615092 [06:58:35] [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773554150000&orgId=1&to=1773557750000[Grafana] FIRING: An unusually high number of threats are being reported by CloudFlare! https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553860000&orgId=1&to=1773557915093[Grafana] FIRING: A MediaWiki pool is sick according to CloudFlare https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553860000&orgId=1&to=1 [06:58:35] 93 [07:33:35] [Grafana] RESOLVED: DatasourceError https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553860000&orgId=1&to=1773559860000[Grafana] RESOLVED: DatasourceError https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773553860000&orgId=1&to=1773559860000 [08:38:16] [1/4] https://cdn.discordapp.com/attachments/808001911868489748/1482659178882273371/image.png?ex=69b7c176&is=69b66ff6&hm=c07871585c9c62db6049aaef40a0011a6d913b2107d925a797761dda6b79f133& [08:38:16] [2/4] https://cdn.discordapp.com/attachments/808001911868489748/1482659180408995933/image.png?ex=69b7c177&is=69b66ff7&hm=aef2c6003b28ec9becdcc97f45ca823338560545b0b2098856a3237ec1eb8a01& [08:38:16] [3/4] https://cdn.discordapp.com/attachments/808001911868489748/1482659181004718101/image.png?ex=69b7c177&is=69b66ff7&hm=ea0c3cbdde8627f2fc626cfa7b16279e56ba2dd9047841b2a5b1254380b03387& [08:38:16] [4/4] https://cdn.discordapp.com/attachments/808001911868489748/1482659181726142615/image.png?ex=69b7c177&is=69b66ff7&hm=d807895d60c0023a7e60e01b56a7a1c5dbe738932cf4c6134e7e64cbffb3936f& [09:57:49] [02TSPortal] 07renovate[bot] created 03renovate/github-actions (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/12e128aa369c [09:57:49] 02TSPortal/03renovate/github-actions 07renovate[bot] 0312e128a Update shivammathur/setup-php digest to 4a2f84e [12:06:35] [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773572690000&orgId=1&to=1773576394924 [12:11:35] [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?from=1773572690000&orgId=1&to=1773576650000 [12:15:02] [02mw-config] 07YTFGolf left a file comment in pull request #6344 037305aa9: This isn't necessary 13https://github.com/miraheze/mw-config/pull/6344#discussion_r2936664839 [12:16:25] [02mw-config] 07YTFGolf left a file comment in pull request #6344 037305aa9: Formatting: array keys should all have their own line 13https://github.com/miraheze/mw-config/pull/6344#discussion_r2936666178 [12:16:38] [02mw-config] 07YTFGolf left a file comment in pull request #6344 037305aa9: Formatting: ditto above 13https://github.com/miraheze/mw-config/pull/6344#discussion_r2936666380 [12:24:38] [1/3] @pixldev idk if I have the authority to start a proper review so just left a few comments [12:24:38] [2/3] for the help section I'm not sure about the wording there, "removed from being show[n]" implies being hidden rather than being completely removed. plus https://discord.com/channels/407504499280707585/808001911868489748/1482509540653142156 [12:24:38] [3/3] Also Fandom haven't referred to themselves as FANDOM in years [12:27:08] and remember to `composer fix` [12:28:40] [02mw-config] 07YTFGolf left a file comment in pull request #6344 037305aa9: Formatting: indentation 13https://github.com/miraheze/mw-config/pull/6344#discussion_r2936678857 [13:05:27] Remember implies that I learned such in the first place [13:05:58] (Actually I may have in my brief extension dev stint) [13:27:10] [02TSPortal] 07renovate[bot] merged 03renovate/github-actions into 03main 13https://github.com/miraheze/TSPortal/compare/f49018a0386a...12e128aa369c [13:27:11] [02TSPortal] 07renovate[bot] 04deleted 03renovate/github-actions at 0312e128a 13https://api.github.com/repos/miraheze/TSPortal/commit/12e128a [14:03:51] exactly how I feel when Skye assumes I'm smart [14:04:36] twin [15:06:16] @thewwrnerdguy im trying to figure out what git thinks actually changed here https://github.com/miraheze/mw-config/pull/6344/changes/BASE..7305aa944f390a8a04c5938c93c2a0ff0e4b161b#r2936664839 [15:06:16] [GitHub] [miraheze/mw-config #6344] open PR by pixDeVl, created 2 days, 12 hours ago: Add wmgMobileFrontendAllowNavboxes | lets see if this works [15:17:29] you removed the newline [15:18:22] and by "you" I mean probably your editor [15:18:40] I remember neovim kept doing that to me so I had to steal some online code to stop it from doing that [15:19:56] [1/25] ```lua [15:19:56] [2/25] -- Most of this was clanker-generated [15:19:57] [3/25] local function strip_trailing_newlines() [15:19:57] [4/25] local current_buf = vim.api.nvim_get_current_buf() [15:19:57] [5/25] local lines = vim.api.nvim_buf_get_lines(current_buf, 0, -1, false) [15:19:57] [6/25] table.insert(lines, "") [15:19:58] [7/25] -- in case ending line needs to be inserted [15:19:58] [8/25] -- TODO research if this is needed [15:19:58] [9/25] vim.opt.fixeol = true [15:19:59] [10/25] -- ??? I hate neovim [15:19:59] [11/25] -- Start from the end and find the last non-empty line [15:20:00] [12/25] local last_non_empty = #lines [15:20:00] [13/25] while last_non_empty > 0 and vim.trim(lines[last_non_empty]) == "" do [15:20:01] [14/25] last_non_empty = last_non_empty - 1 [15:20:01] [15/25] end [15:20:02] [16/25] -- If there are empty lines to trim [15:20:02] [17/25] vim.api.nvim_buf_set_lines(current_buf, last_non_empty + 1, -1, false, {}) [15:20:03] [18/25] end [15:20:03] [19/25] -- Create an autocommand to run StripNewlines before saving [15:20:04] [20/25] vim.api.nvim_create_autocmd("BufWritePre", { [15:20:04] [21/25] callback = function() [15:20:05] [22/25] strip_trailing_newlines() [15:20:05] [23/25] end [15:20:06] [24/25] }) [15:20:06] [25/25] ``` [15:29:57] Huh [15:30:46] I have no idea how it works but it makes sure that each file has exactly one trailing newline when I press save [16:40:49] [02TSPortal] 07renovate[bot] created 03renovate/github-actions (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/495b0690958f [16:40:49] 02TSPortal/03renovate/github-actions 07renovate[bot] 03495b069 Update shivammathur/setup-php digest to 93cb314 [17:10:50] [02TSPortal] 07Universal-Omega created 03types from 03main (+0 new commit) 13https://github.com/miraheze/TSPortal/compare/types [17:11:01] !log upgrade phorge on phorge171 [17:11:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:12:29] PROBLEM - phorge171 issue-tracker.miraheze.org HTTPS on phorge171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 5.995 second response time [17:17:51] PROBLEM - phorge171 Current Load on phorge171 is CRITICAL: LOAD CRITICAL - total load average: 10.02, 7.65, 3.49 [17:19:51] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 2.81, 5.90, 3.37 [17:28:21] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:29:53] PROBLEM - phorge171 Current Load on phorge171 is CRITICAL: LOAD CRITICAL - total load average: 16.97, 11.21, 6.35 [17:32:22] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 1.356 second response time [17:36:42] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:00:08] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 9.928 second response time [18:06:00] PROBLEM - phorge171 PowerDNS Recursor on phorge171 is CRITICAL: CRITICAL - Plugin timed out while executing system call [18:07:59] RECOVERY - phorge171 PowerDNS Recursor on phorge171 is OK: DNS OK: 0.060 seconds response time. phorge171.fsslc.wtnet returns 10.0.17.124 [18:09:27] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:11:19] PROBLEM - phorge171 SSH on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:13:25] RECOVERY - phorge171 SSH on phorge171 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [18:22:01] PROBLEM - phorge171 PowerDNS Recursor on phorge171 is CRITICAL: CRITICAL - Plugin timed out while executing system call [18:36:28] PROBLEM - phorge171 SSH on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:38:59] RECOVERY - phorge171 PowerDNS Recursor on phorge171 is OK: DNS OK: 0.060 seconds response time. phorge171.fsslc.wtnet returns 10.0.17.124 [18:40:38] RECOVERY - phorge171 SSH on phorge171 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [18:41:35] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03types 13https://github.com/miraheze/TSPortal/commit/da5512c65d8f7004cce38b5aaef6d4e94cf24aa8 [18:41:35] 02TSPortal/03types 07CosmicAlpha 03da5512c Use PHP types and other cleanup [18:41:57] [02TSPortal] 07Universal-Omega opened pull request #71: Use PHP types and other cleanup (03main...03types) 13https://github.com/miraheze/TSPortal/pull/71 [18:47:41] PROBLEM - phorge171 PowerDNS Recursor on phorge171 is CRITICAL: CRITICAL - Plugin timed out while executing system call [18:53:36] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 1.537 second response time [18:56:12] RECOVERY - phorge171 PowerDNS Recursor on phorge171 is OK: DNS OK: 6.955 seconds response time. phorge171.fsslc.wtnet returns 10.0.17.124 [18:57:46] PROBLEM - phorge171 SSH on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:58:00] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:59:44] RECOVERY - phorge171 SSH on phorge171 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [19:06:21] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 0.066 second response time [19:11:22] PROBLEM - phorge171 SSH on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:15:29] RECOVERY - phorge171 SSH on phorge171 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [19:21:59] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:24:15] @paladox what happened here? [19:24:54] OOM it seems [19:25:48] !log reboot phorge171 [19:25:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:19] PROBLEM - phorge171 PowerDNS Recursor on phorge171 is CRITICAL: CRITICAL - Plugin timed out while executing system call [19:28:51] reboot didn't fix it [19:30:06] PROBLEM - phorge171 SSH on phorge171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:30:17] sorry i chewed on the servers [19:30:30] mistook them for candies [19:31:56] PROBLEM - phorge171 Current Load on phorge171 is WARNING: LOAD WARNING - total load average: 7.45, 2.45, 0.86 [19:32:05] RECOVERY - phorge171 SSH on phorge171 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [19:33:53] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 1.12, 0.27, 0.09 [19:35:23] RECOVERY - phorge171 PowerDNS Recursor on phorge171 is OK: DNS OK: 0.397 seconds response time. phorge171.fsslc.wtnet returns 10.0.17.124 [19:36:07] i'm not sure if it's the upgrade or if it's cloudflare [19:36:15] stopping php-fpm stops it [19:36:49] PROBLEM - phorge171 NTP time on phorge171 is CRITICAL: NTP CRITICAL: Offset 0.6904882789 secs [19:40:07] checking out ec7a82e58a5512b572753bfb24781b4c70a9807f fixed it [19:40:29] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 0.040 second response time [19:40:36] RECOVERY - phorge171 issue-tracker.miraheze.org HTTPS on phorge171 is OK: HTTP OK: HTTP/1.1 200 OK - 19375 bytes in 0.060 second response time [19:46:29] PROBLEM - phorge171 Puppet on phorge171 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 0 failures [19:49:11] i think 27616b7a8dff1707d61f4be49d43e90f80f45b38 might be the cause [19:52:35] RECOVERY - phorge171 Puppet on phorge171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:07:42] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03types 13https://github.com/miraheze/TSPortal/commit/8ceebf8d730bdb0fa1efa9b539fbff8b3c607f03 [20:07:42] 02TSPortal/03types 07CosmicAlpha 038ceebf8 return string [20:10:40] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/42f2f07fa336d0215e27f95606f5df5c25c40372 [20:10:40] 02TSPortal/03main 07CosmicAlpha 0342f2f07 Use PHP types and other cleanup [20:10:40] [02TSPortal] 07Universal-Omega merged pull request #71: Use PHP types and other cleanup (03main...03types) 13https://github.com/miraheze/TSPortal/pull/71 [20:10:41] [02TSPortal] 07Universal-Omega 04deleted 03types at 038ceebf8 13https://api.github.com/repos/miraheze/TSPortal/commit/8ceebf8 [20:10:58] [02TSPortal] 07renovate[bot] 04force-pushed 03renovate/github-actions from 03495b069 to 03cbcb632 13https://api.github.com/repos/miraheze/TSPortal/commits/renovate/github-actions [20:10:58] 02TSPortal/03renovate/github-actions 07renovate[bot] 03cbcb632 Update shivammathur/setup-php digest to 93cb314 [20:13:20] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03standing-good 13https://github.com/miraheze/TSPortal/commit/22f900043f05376ba47b770cc530a0303071b4a5 [20:13:20] 02TSPortal/03standing-good 07CosmicAlpha 0322f9000 Merge branch 'main' into standing-good [20:13:33] [02TSPortal] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/cbcb632ab26a12c5400b8be429460ed377d39d34 [20:13:33] 02TSPortal/03main 07renovate[bot] 03cbcb632 Update shivammathur/setup-php digest to 93cb314 [20:13:33] [02TSPortal] 07renovate[bot] 04deleted 03renovate/github-actions at 03cbcb632 13https://api.github.com/repos/miraheze/TSPortal/commit/cbcb632 [20:44:57] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03standing-good 13https://github.com/miraheze/TSPortal/commit/65486b6e3aef5e998884b1f7c00dcf00a52fca80 [20:44:58] 02TSPortal/03standing-good 07CosmicAlpha 0365486b6 Add user event [20:48:52] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03standing-good 13https://github.com/miraheze/TSPortal/commit/16555c5bd74debd684b6e15156a9b22e72683958 [20:48:52] 02TSPortal/03standing-good 07CosmicAlpha 0316555c5 Add event messages [20:49:35] [02TSPortal] 07Universal-Omega pushed 1 new commit to 03standing-good 13https://github.com/miraheze/TSPortal/commit/e9f520218a9c19ff16454b536df4a2518037d470 [20:49:35] 02TSPortal/03standing-good 07CosmicAlpha 03e9f5202 Update [21:52:30] phorge is down for me [21:52:39] https://cdn.discordapp.com/attachments/808001911868489748/1482859097584828467/image.png?ex=69b87ba7&is=69b72a27&hm=979abb288c332de1d8a3a35e3c91689ea6bf96991bb7086f0871e83118769c40& [21:53:15] just wanted to go to phorge to create a task about phorge being down lmao [21:53:38] PROBLEM - phorge171 php-fpm on phorge171 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm8.4' [21:53:40] PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway [21:55:38] RECOVERY - phorge171 php-fpm on phorge171 is OK: PROCS OK: 9 processes with command name 'php-fpm8.4' [21:55:40] RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17668 bytes in 0.032 second response time [21:56:37] seems to have recovered by itself [21:56:43] was me [21:56:45] oh [21:56:49] thx [21:56:53] i was trying out a fix [21:57:00] https://we.phorge.it/T16555 [21:57:10] seemed to have worked [21:57:38] nice [22:00:51] We should host a bugzilla as a backup [22:02:05] lol