[00:01:10] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/16c938a23580845839dd1b12bb91c3abeb08c7c2 [00:01:10] 02statichelp/03main 07WikiTideBot 0316c938a Bot: Auto-update Tech namespace pages 2026-06-04 00:01:05 [00:02:42] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importDump --wiki=evnovawiki --no-updates --username-prefix=fandom:evn -- evn_pages_full.xml (END - exit=0) [00:02:43] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importImages --wiki=evnovawiki --sleep=0 '--comment=Importing files from https://evn.fandom.com/ ([[phorge:T15480|T15480]])' -- images (START) [00:02:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:02:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:04] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php importImages --wiki=evnovawiki --sleep=0 '--comment=Importing files from https://evn.fandom.com/ ([[phorge:T15480|T15480]])' -- images (END - exit=0) [00:25:05] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=evnovawiki (START) [00:25:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:25:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:30:35] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/0ee51f48da27f95de2fa9d9343828b4fc184ee29 [00:30:35] 02ssl/03main 07WikiTideBot 030ee51f4 Bot: Auto-update domain lists [00:56:39] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php rebuildall --wiki=evnovawiki (END - exit=0) [00:56:40] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=evnovawiki (START) [00:56:41] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initEditCount --wiki=evnovawiki (END - exit=0) [00:56:42] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=evnovawiki --update (START) [00:56:43] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php initSiteStats --wiki=evnovawiki --update (END - exit=0) [00:56:44] !log [skye@mwtask171] Finished import for evnovawiki (XML: evn_pages_full.xml; Images: images) (END - exit=0) [00:56:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:56:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:56:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:57:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:57:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:57:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:20:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780521590000&orgId=1&to=1780539623392 [04:41:12] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [04:43:06] PROBLEM - puppet181 Check unit status of git_pull_ssl on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_ssl [04:43:20] PROBLEM - puppet181 Check unit status of git_pull_puppet on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_puppet [04:44:58] PROBLEM - mw203 Puppet on mw203 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.45] [04:46:15] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_mathoid] [04:49:06] RECOVERY - puppet181 Check unit status of git_pull_ssl on puppet181 is OK: OK: Status of the systemd unit git_pull_ssl [04:49:20] RECOVERY - puppet181 Check unit status of git_pull_puppet on puppet181 is OK: OK: Status of the systemd unit git_pull_puppet [05:02:14] PROBLEM - mw151 Puppet on mw151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.45] [05:03:06] PROBLEM - puppet181 Check unit status of git_pull_ssl on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_ssl [05:09:06] RECOVERY - puppet181 Check unit status of git_pull_ssl on puppet181 is OK: OK: Status of the systemd unit git_pull_ssl [05:12:16] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [05:12:55] RECOVERY - mw203 Puppet on mw203 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:26:24] PROBLEM - mw191 Puppet on mw191 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png] [05:30:14] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:44:39] PROBLEM - mw161 Puppet on mw161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_3d2png] [05:52:22] RECOVERY - mw191 Puppet on mw191 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [06:12:42] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:16:46] PROBLEM - puppet181 Puppet on puppet181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_puppet] [06:20:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780521590000&orgId=1&to=1780554023426 [06:27:52] PROBLEM - bots171 Puppet on bots171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_IRC-Discord-Relay-relaybot] [06:33:20] PROBLEM - puppet181 Check unit status of git_pull_puppet on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit git_pull_puppet [06:33:51] I guess GitHub is down again [06:35:39] puppet181 looks fine [06:36:47] RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:36:51] so is bots171 [06:36:58] GitHub being GitHub then [06:37:54] RECOVERY - bots171 Puppet on bots171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:39:12] [02TSPortal] 07renovate[bot] created 03renovate/dev-deps (+1 new commit) 13https://github.com/miraheze/TSPortal/commit/eab74b4c2515 [06:39:12] 02TSPortal/03renovate/dev-deps 07renovate[bot] 03eab74b4 Update dev dependencies to v13.1.14 [06:39:20] RECOVERY - puppet181 Check unit status of git_pull_puppet on puppet181 is OK: OK: Status of the systemd unit git_pull_puppet [07:30:44] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.63, 6.23, 5.02 [07:31:45] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.01, 6.74, 5.32 [07:32:44] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.72, 6.42, 5.24 [07:33:43] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.47, 6.88, 5.53 [07:35:42] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.78, 6.34, 5.52 [09:32:44] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.69, 6.25, 5.16 [09:33:06] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.28, 6.41, 5.35 [09:34:44] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 2.41, 4.86, 4.80 [09:35:04] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 2.37, 4.89, 4.92 [10:12:37] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.83, 7.36, 6.07 [10:12:44] PROBLEM - os202 Current Load on os202 is CRITICAL: LOAD CRITICAL - total load average: 8.53, 6.68, 5.68 [10:14:36] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 5.65, 6.86, 6.05 [10:14:44] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 5.89, 6.39, 5.71 [10:16:35] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.95, 6.15, 5.89 [10:20:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780521590000&orgId=1&to=1780568423468 [10:52:03] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.81, 6.92, 6.08 [10:54:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.20, 6.77, 6.14 [10:59:55] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.47, 7.02, 6.43 [11:05:50] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.99, 6.61, 6.49 [11:17:40] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.84, 6.52, 6.21 [11:17:50] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.88, 6.53, 6.12 [11:19:38] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.51, 6.62, 6.29 [11:19:46] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.50, 6.67, 6.24 [11:37:12] [02TSPortal] 07renovate[bot] pushed 1 new commit to 03main 13https://github.com/miraheze/TSPortal/commit/eab74b4c25151617c7f333d1ef10c165f62d8967 [11:37:12] 02TSPortal/03main 07renovate[bot] 03eab74b4 Update dev dependencies to v13.1.14 [11:37:13] [02TSPortal] 07renovate[bot] 04deleted 03renovate/dev-deps at 03eab74b4 13https://api.github.com/repos/miraheze/TSPortal/commit/eab74b4 [11:37:41] [02TSPortal] 07renovate[bot] 04force-pushed 03renovate/prod-deps from 03fbfa4e6 to 03939195f 13https://api.github.com/repos/miraheze/TSPortal/commits/renovate/prod-deps [11:37:43] 02TSPortal/03renovate/prod-deps 07renovate[bot] 03939195f Update prod dependencies [12:11:01] [02DiscordNotifications] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DiscordNotifications/commit/6ffd1c1f7d8dbd3dc1ac9b80563d6ed852573271 [12:11:01] 02DiscordNotifications/03main 07translatewiki.net 036ffd1c1 Localisation updates from https://translatewiki.net. [12:11:04] [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/44b0fc3bddc3a5a01d04afc23856b93293670307 [12:11:04] 02CreateWiki/03main 07translatewiki.net 0344b0fc3 Localisation updates from https://translatewiki.net. [12:11:06] [02ImportDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ImportDump/commit/c29b0860cd23b912b3cd72fc59149fa506187971 [12:11:07] 02ImportDump/03main 07translatewiki.net 03c29b086 Localisation updates from https://translatewiki.net. [12:11:09] [02landing] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/b8f9e14547cc5e45ca852a4d46ba78cabb83fc17 [12:11:11] [02DataDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DataDump/commit/0dd66d7efa3b1512f46cb22bd06e4bd6c585a4c7 [12:11:13] 02landing/03main 07translatewiki.net 03b8f9e14 Localisation updates from https://translatewiki.net. [12:11:15] [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/52ef58554703fc3723aaaaae3e124e01bc8bc78c [12:11:17] [02WikiDiscover] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/WikiDiscover/commit/816c0d22a8bdb2b7f53c7840aa3b2196c4566200 [12:11:18] 02DataDump/03main 07translatewiki.net 030dd66d7 Localisation updates from https://translatewiki.net. [12:11:20] 02MirahezeMagic/03main 07translatewiki.net 0352ef585 Localisation updates from https://translatewiki.net. [12:11:21] 02WikiDiscover/03main 07translatewiki.net 03816c0d2 Localisation updates from https://translatewiki.net. [12:12:05] miraheze/landing - translatewiki the build passed. [12:15:24] miraheze/DataDump - translatewiki the build passed. [12:15:43] miraheze/WikiDiscover - translatewiki the build passed. [12:16:10] miraheze/ImportDump - translatewiki the build passed. [12:16:51] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.80, 6.87, 5.78 [12:16:58] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.17, 6.56, 5.68 [12:17:20] miraheze/MirahezeMagic - translatewiki the build passed. [12:18:35] miraheze/CreateWiki - translatewiki the build passed. [12:18:50] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.62, 6.49, 5.79 [12:18:53] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 5.20, 6.16, 5.65 [12:19:19] miraheze/DiscordNotifications - translatewiki the build passed. [13:10:09] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.99, 5.14, 4.11 [13:12:07] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.45, 5.50, 4.39 [14:00:06] [1/4] https://cdn.discordapp.com/attachments/808001911868489748/1512093593186209912/image.jpg?ex=6a22d665&is=6a2184e5&hm=95cbf020510bcbba96110315c78b878e0bce33275a459ad56e3f75e1434b2cf3& [14:00:07] [2/4] https://cdn.discordapp.com/attachments/808001911868489748/1512093593647448235/image.jpg?ex=6a22d666&is=6a2184e6&hm=fa76cbed9fba44a657fcdca50e0ec9590e0bc7d15d2210f0534719f99310081f& [14:00:07] [3/4] https://cdn.discordapp.com/attachments/808001911868489748/1512093594092306513/image.jpg?ex=6a22d666&is=6a2184e6&hm=8a446ff055590022c71ec05ce84d1217b0735c6d22886613514c8531e1862ac3& [14:00:07] [4/4] https://cdn.discordapp.com/attachments/808001911868489748/1512093594574393475/image.jpg?ex=6a22d666&is=6a2184e6&hm=a358f583b332115f720e8497ec0d0c983747ef3b2bcd5b77a095af4cb4a43d79& [14:12:36] [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-13.1.14 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/34d6933a347c [14:12:36] 02mw-config/03dependabot/composer/phpunit/phpunit-13.1.14 07dependabot[bot] 0334d6933 Update phpunit/phpunit requirement from 13.1.13 to 13.1.14… [14:12:37] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6431 (Update phpunit/phpunit requirement from 13.1.13 to 13.1.14) 13https://github.com/miraheze/mw-config/pull/6431 [14:12:38] [02mw-config] 07dependabot[bot] opened pull request #6431: Update phpunit/phpunit requirement from 13.1.13 to 13.1.14 (03main...03dependabot/composer/phpunit/phpunit-13.1.14) 13https://github.com/miraheze/mw-config/pull/6431 [14:12:40] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6431 (Update phpunit/phpunit requirement from 13.1.13 to 13.1.14) 13https://github.com/miraheze/mw-config/pull/6431 [14:13:48] miraheze/mw-config - dependabot[bot] the build passed. [14:20:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780521590000&orgId=1&to=1780582823506 [16:20:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780586150000&orgId=1&to=1780589750000 [16:48:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780588070000&orgId=1&to=1780591703350 [16:58:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780588610000&orgId=1&to=1780592210000 [17:17:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780589810000&orgId=1&to=1780593443351 [17:25:23] oh my god it's STILL smw [17:37:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780590830000&orgId=1&to=1780594430000 [17:38:23] [02mw-config] 07pskyechology pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/45030ef0caca58f2caa968b52bbf96fc7b305afb [17:38:23] 02mw-config/03main 07Skye 0345030ef T15485: Set vecor-theme default user option for otherkinwiki [17:39:24] I love my vecor theme [17:39:28] miraheze/mw-config - pskyechology the build passed. [17:39:37] crying [17:40:14] hey excuse me where is my G logbot [17:40:20] !log test [17:40:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:41:50] @paladox you around to resurrect mirahezeLSbot? [17:42:00] sure [17:43:01] !log rebooted bots171 [17:43:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:43:08] !log [skye@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [17:43:09] !log [skye@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [17:43:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:43:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:43:17] !log [skye@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [17:43:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:43:27] thanks [17:44:08] !log [skye@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 51s [17:44:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:54:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780592030000&orgId=1&to=1780595663352 [18:14:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780593230000&orgId=1&to=1780596830000 [18:34:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780594430000&orgId=1&to=1780598063352 [18:49:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780595330000&orgId=1&to=1780598930000 [19:03:23] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=systemseveredunofficalwikiwiki --new=systemseveredwiki --user=Skye (END - exit=0) [19:03:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:04:40] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=systemseveredwiki (END - exit=0) [19:04:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:07:08] !log c1: DROP DATABASE systemseveredunofficalwikiwiki; [19:07:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:10:26] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php MirahezeMagic:RenameDatabase --wiki=loginwiki --rename --old=thesapphirinefictionaluniversewiki --new=sapphirinedatabasewiki --user=Skye (END - exit=0) [19:10:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:13:07] !log [skye@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CreateWiki:SetContainersAccess --wiki=sapphirinedatabasewiki (END - exit=0) [19:13:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:15:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780596890000&orgId=1&to=1780600523350 [19:20:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780597010000&orgId=1&to=1780600610000 [19:33:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780597970000&orgId=1&to=1780601603351 [19:38:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780598030000&orgId=1&to=1780601630000 [20:19:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780600730000&orgId=1&to=1780604363350 [20:31:56] Do you have sample log or the job type? I can follow up on that in SMW. There are a lot of work done in SMW 7 to significantly drive down the job load [20:35:11] it's always smw.changePropagationClassUpdate [20:35:25] Ah that one should be mostly resolved [20:49:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1780602530000&orgId=1&to=1780606130000 [22:15:38] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.17, 5.89, 4.14 [22:17:34] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.55, 5.94, 4.35 [22:21:28] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 6.96, 6.59, 4.98 [22:25:20] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.70, 6.58, 5.33 [22:39:54] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.28, 6.63, 5.71 [22:43:51] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.40, 7.48, 6.26 [22:45:50] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.59, 6.89, 6.17 [22:47:48] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.72, 5.72, 5.83 [22:55:34] [02TSPortal] 07renovate[bot] 04force-pushed 03renovate/prod-deps from 03939195f to 03ff0d2b7 13https://api.github.com/repos/miraheze/TSPortal/commits/renovate/prod-deps [22:55:34] 02TSPortal/03renovate/prod-deps 07renovate[bot] 03ff0d2b7 Update prod dependencies