[00:00:01] !log [www-data@mwtask181] Began automatic backing up [00:00:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:00:55] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/485c57dcdc57842460b3e49675a60a1930f9c41e [00:00:55] 02statichelp/03main 07WikiTideBot 03485c57d Bot: Auto-update Tech namespace pages 2026-06-01 00:00:52 [00:04:42] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.47, 6.73, 6.04 [00:05:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.35, 6.66, 5.91 [00:06:36] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 5.73, 6.41, 6.00 [00:09:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.92, 5.56, 5.67 [00:39:34] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 6.98, 6.97, 6.06 [00:41:28] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 3.99, 5.99, 5.82 [01:10:38] !log [petramagna@test151] starting deploy of {'world': True, 'versions': '1.46'} to test151 [01:10:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:10:41] !log [petramagna@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [01:10:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:11:24] !log [petramagna@test151] starting deploy of {'files': '1.46/includes/Block/AbstractBlock.php'} to test151 [01:11:25] !log [petramagna@test151] finished deploy of {'files': '1.46/includes/Block/AbstractBlock.php'} to test151 - SUCCESS in 0s [01:11:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:11:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:18:12] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.47, 6.10, 5.47 [01:20:12] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.01, 6.12, 5.57 [01:27:01] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.25, 6.89, 6.00 [01:29:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.51, 7.14, 6.21 [01:30:54] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.36, 6.88, 6.17 [01:32:48] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.59, 6.74, 6.21 [01:32:53] PROBLEM - phorge171 Disk Space on phorge171 is CRITICAL: DISK CRITICAL - free space: / 3553MiB (4.8% inode=96%); [01:33:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 4.79, 6.38, 6.16 [01:38:53] RECOVERY - phorge171 Disk Space on phorge171 is OK: DISK OK - free space: / 44835MiB (60.9% inode=96%); [01:40:54] PROBLEM - phorge171 Check unit status of phorge_db-backup on phorge171 is CRITICAL: CRITICAL: Status of the systemd unit phorge_db-backup [02:03:56] Does not lift the block. At this point I'm not even sure what is causing the block to occur. Might require multiple simultaneous changes. [02:12:17] !log [void@puppet181] Upgraded packages on matomo151: php-symfony-string, php-symfony-console, php-symfony-filesystem, php-symfony-finder, and php-symfony-process [02:12:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:12:31] !log [void@puppet181] Upgraded packages on reports171: php-symfony-string, php-symfony-console, php-symfony-filesystem, php-symfony-finder, and php-symfony-process [02:12:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:12:58] !log [void@puppet181] Upgraded packages on mwtask171: php-symfony-string, php-symfony-console, php-symfony-filesystem, php-symfony-finder, and php-symfony-process [02:13:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:13:25] RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 120 packages available for upgrade (0 critical updates). [02:13:30] !log [void@puppet181] Upgraded packages on swiftobject211: swift-object, swift, and python3-swift [02:13:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:13:41] RECOVERY - reports171 APT on reports171 is OK: APT OK: 82 packages available for upgrade (0 critical updates). [02:13:55] !log [void@puppet181] Upgraded packages on swiftobject151: swift-object, swift, and python3-swift [02:13:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:01] RECOVERY - swiftobject151 APT on swiftobject151 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:14:14] !log [void@puppet181] Upgraded packages on swiftproxy171: swift-proxy, swift, and python3-swift [02:14:16] RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 80 packages available for upgrade (0 critical updates). [02:14:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:34] !log [void@puppet181] Upgraded packages on swiftproxy161: swift-proxy, swift, and python3-swift [02:14:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:15:08] !log [void@puppet181] Upgraded packages on swiftac171: swift-account, swift-container, swift, and python3-swift [02:15:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:15:16] RECOVERY - swiftac171 APT on swiftac171 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:15:22] RECOVERY - swiftobject211 APT on swiftobject211 is OK: APT OK: 79 packages available for upgrade (0 critical updates). [02:15:25] !log [void@puppet181] Upgraded packages on test151: php-symfony-string, php-symfony-console, php-symfony-filesystem, php-symfony-finder, and php-symfony-process [02:15:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:15:47] RECOVERY - swiftproxy161 APT on swiftproxy161 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:15:51] !log [void@puppet181] Upgraded packages on swiftobject161: swift-object-expirer, swift-object, swift, and python3-swift [02:15:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:15:58] RECOVERY - swiftproxy171 APT on swiftproxy171 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:16:15] RECOVERY - swiftobject161 APT on swiftobject161 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:16:43] PROBLEM - swiftproxy171 Swift Proxy on swiftproxy171 is CRITICAL: connect to address 10.0.17.108 and port 80: Connection refused [02:16:50] PROBLEM - swiftproxy161 HTTPS on swiftproxy161 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway [02:16:58] !log [void@puppet181] Upgraded packages on swiftobject171: swift-object, swift, and python3-swift [02:16:59] RECOVERY - test151 APT on test151 is OK: APT OK: 122 packages available for upgrade (0 critical updates). [02:17:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:17:17] !log [void@puppet181] Upgraded packages on mwtask181: php-symfony-string, php-symfony-console, php-symfony-filesystem, php-symfony-finder, and php-symfony-process [02:17:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:17:25] PROBLEM - swiftproxy171 HTTP on swiftproxy171 is CRITICAL: connect to address 10.0.17.108 and port 80: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:17:34] PROBLEM - swiftproxy171 HTTPS on swiftproxy171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway [02:17:44] !log [void@puppet181] Upgraded packages on swiftobject191: swift-object, swift, and python3-swift [02:17:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:18:08] RECOVERY - swiftobject171 APT on swiftobject171 is OK: APT OK: 83 packages available for upgrade (0 critical updates). [02:18:08] PROBLEM - swiftproxy161 Swift Proxy on swiftproxy161 is CRITICAL: connect to address 10.0.16.135 and port 80: Connection refused [02:18:10] !log [void@puppet181] Upgraded packages on swiftobject181: swift-object, swift, and python3-swift [02:18:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:18:16] PROBLEM - swiftproxy161 HTTP on swiftproxy161 is CRITICAL: connect to address 10.0.16.135 and port 80: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:18:35] !log [void@puppet181] Upgraded packages on swiftobject201: swift-object, swift, and python3-swift [02:18:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:18:50] RECOVERY - swiftproxy161 HTTPS on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.032 second response time [02:18:52] RECOVERY - swiftobject191 APT on swiftobject191 is OK: APT OK: 84 packages available for upgrade (0 critical updates). [02:19:10] RECOVERY - swiftobject181 APT on swiftobject181 is OK: APT OK: 84 packages available for upgrade (0 critical updates). [02:19:15] RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 121 packages available for upgrade (0 critical updates). [02:20:02] RECOVERY - swiftobject201 APT on swiftobject201 is OK: APT OK: 84 packages available for upgrade (0 critical updates). [02:20:08] RECOVERY - swiftproxy161 Swift Proxy on swiftproxy161 is OK: TCP OK - 0.000 second response time on 10.0.16.135 port 80 [02:20:16] RECOVERY - swiftproxy161 HTTP on swiftproxy161 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.074 second response time [02:20:43] RECOVERY - swiftproxy171 Swift Proxy on swiftproxy171 is OK: TCP OK - 0.000 second response time on 10.0.17.108 port 80 [02:21:25] RECOVERY - swiftproxy171 HTTP on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.018 second response time [02:21:34] RECOVERY - swiftproxy171 HTTPS on swiftproxy171 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.028 second response time [02:25:12] PROBLEM - os162 Current Load on os162 is CRITICAL: LOAD CRITICAL - total load average: 8.22, 6.88, 6.06 [02:26:54] RECOVERY - phorge171 Check unit status of phorge_db-backup on phorge171 is OK: OK: Status of the systemd unit phorge_db-backup [02:28:12] PROBLEM - os202 Current Load on os202 is CRITICAL: LOAD CRITICAL - total load average: 8.08, 7.17, 6.27 [02:29:06] PROBLEM - matomo151 Check unit status of matomo-archiver-1 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-1 [02:30:12] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.77, 7.39, 6.46 [02:35:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 5.76, 7.23, 6.80 [02:36:12] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 4.44, 6.52, 6.47 [02:37:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.16, 6.30, 6.49 [02:53:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.60, 6.45, 6.10 [02:55:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 5.40, 5.78, 5.89 [03:01:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.41, 6.68, 6.24 [03:04:53] PROBLEM - phorge171 Disk Space on phorge171 is CRITICAL: DISK CRITICAL - free space: / 2569MiB (3.5% inode=96%); [03:06:30] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 6.67, 6.85, 6.55 [03:07:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.06, 6.78, 6.48 [03:08:24] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 4.33, 5.99, 6.27 [03:10:53] RECOVERY - phorge171 Disk Space on phorge171 is OK: DISK OK - free space: / 53395MiB (72.5% inode=96%); [03:12:54] PROBLEM - phorge171 Check unit status of phorge_db-backup on phorge171 is CRITICAL: CRITICAL: Status of the systemd unit phorge_db-backup [03:25:27] [02puppet] 07The-Voidwalker created 03The-Voidwalker-patch-1 (+1 new commit) 13https://github.com/miraheze/puppet/commit/179b39c3645b [03:25:27] 02puppet/03The-Voidwalker-patch-1 07The-Voidwalker 03179b39c move all systemd units to /etc/systemd/system… [03:25:37] [02puppet] 07The-Voidwalker opened pull request #4859: move all systemd units to /etc/systemd/system (03main...03The-Voidwalker-patch-1) 13https://github.com/miraheze/puppet/pull/4859 [03:26:14] [02puppet] 07The-Voidwalker commented on pull request #4859: Not willing to merge this myself just yet as it will trigger a restart of _all_ services. 13https://github.com/miraheze/puppet/pull/4859#issuecomment-4589290148 [03:30:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/62e9ed78de18e5cf8254801015ebc0c3771cd029 [03:30:23] 02ssl/03main 07WikiTideBot 0362e9ed7 Bot: Auto-update domain lists [03:59:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.33, 6.43, 5.35 [04:00:12] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.97, 6.82, 5.73 [04:01:01] RECOVERY - os162 Current Load on os162 is OK: LOAD OK - total load average: 6.54, 6.58, 5.55 [04:02:12] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 5.62, 6.49, 5.75 [05:00:20] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/f2eff2d58cc70806a4341cd1c4a74fb0ae53bffb [05:00:21] 02ssl/03main 07WikiTideBot 03f2eff2d Bot: Auto-update domain lists [08:01:06] RECOVERY - matomo151 Check unit status of matomo-archiver-1 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-1 [12:01:02] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 80.67, 40.18, 17.30 [12:17:58] [02DiscordNotifications] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DiscordNotifications/commit/fda9a752f0ca2f89975b8e5a2558395cb35ed380 [12:17:58] 02DiscordNotifications/03main 07translatewiki.net 03fda9a75 Localisation updates from https://translatewiki.net. [12:18:00] [02ErrorPages] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ErrorPages/commit/ca98ef5b4b9b7799b55cd350778d24614e9cb55f [12:18:01] 02ErrorPages/03main 07translatewiki.net 03ca98ef5 Localisation updates from https://translatewiki.net. [12:18:02] [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/422c9645f67b19a477aa6b8ee0bb080822b6ff35 [12:18:04] 02CreateWiki/03main 07translatewiki.net 03422c964 Localisation updates from https://translatewiki.net. [12:18:06] [02ImportDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ImportDump/commit/836663c395268d120415e3515a0564e4f59f85f0 [12:18:08] [02IncidentReporting] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/IncidentReporting/commit/5fe6a1241c89d5df380a4df2944479b43cfb42af [12:18:10] [02DataDump] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/DataDump/commit/6a6eaa4bed1822d9750b74f933fa9868da71c259 [12:18:11] 02ImportDump/03main 07translatewiki.net 03836663c Localisation updates from https://translatewiki.net. [12:18:13] [02landing] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/landing/commit/d9877fd57939bcb2fb9f83aafb0c48002053ab74 [12:18:15] 02IncidentReporting/03main 07translatewiki.net 035fe6a12 Localisation updates from https://translatewiki.net. [12:18:17] [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MatomoAnalytics/commit/7cc045296d1f627831f33577c34d45ea59f4e33b [12:18:19] [02RemovePII] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/RemovePII/commit/acf7c9cadec11a190d0a5c9f046ff6f40ce96e78 [12:18:20] 02DataDump/03main 07translatewiki.net 036a6eaa4 Localisation updates from https://translatewiki.net. [12:18:22] [02RequestCustomDomain] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/RequestCustomDomain/commit/f0f5f61bbf8dad7c137c5b8b6b7f12bf68f3caf5 [12:18:24] [02RottenLinks] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/RottenLinks/commit/d6f5de2fe67279736236cc58abad24f2464d7630 [12:18:26] [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/02dfb02e77e623c7fad6dcc32c41274665c91aff [12:18:28] [02ManageWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/ManageWiki/commit/c67b435052cc8629114f34d48aff1bdd0dd7b7b9 [12:18:30] 02landing/03main 07translatewiki.net 03d9877fd Localisation updates from https://translatewiki.net. [12:18:32] [02WikiDiscover] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/WikiDiscover/commit/d2107ad7b119ccbbff291561d0ec5e01bd2a96c4 [12:18:34] 02MatomoAnalytics/03main 07translatewiki.net 037cc0452 Localisation updates from https://translatewiki.net. [12:18:35] 02RemovePII/03main 07translatewiki.net 03acf7c9c Localisation updates from https://translatewiki.net. [12:18:37] 02RequestCustomDomain/03main 07translatewiki.net 03f0f5f61 Localisation updates from https://translatewiki.net. [12:18:38] 02RottenLinks/03main 07translatewiki.net 03d6f5de2 Localisation updates from https://translatewiki.net. [12:18:40] 02MirahezeMagic/03main 07translatewiki.net 0302dfb02 Localisation updates from https://translatewiki.net. [12:18:42] 02ManageWiki/03main 07translatewiki.net 03c67b435 Localisation updates from https://translatewiki.net. [12:18:43] 02WikiDiscover/03main 07translatewiki.net 03d2107ad Localisation updates from https://translatewiki.net. [12:19:02] miraheze/ErrorPages - translatewiki the build passed. [12:20:50] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 2.27, 4.10, 11.00 [12:22:49] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 2.40, 3.54, 9.95 [12:22:55] miraheze/landing - translatewiki the build passed. [12:22:58] miraheze/DataDump - translatewiki the build passed. [12:23:05] miraheze/IncidentReporting - translatewiki the build passed. [12:24:56] miraheze/ImportDump - translatewiki the build passed. [12:25:48] miraheze/MatomoAnalytics - translatewiki the build passed. [12:26:01] miraheze/RemovePII - translatewiki the build passed. [12:26:04] miraheze/RottenLinks - translatewiki the build passed. [12:26:12] miraheze/DiscordNotifications - translatewiki the build passed. [12:26:33] miraheze/CreateWiki - translatewiki the build passed. [12:26:43] miraheze/RequestCustomDomain - translatewiki the build passed. [12:27:40] miraheze/ManageWiki - translatewiki the build passed. [12:28:10] miraheze/WikiDiscover - translatewiki the build passed. [12:30:06] miraheze/MirahezeMagic - translatewiki the build passed. [15:30:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/da18fe0a8c22abbc6e3e08898489bfa562b1cacc [15:30:23] 02ssl/03main 07WikiTideBot 03da18fe0 Bot: Auto-update domain lists [16:09:55] !log destroyed F6727665 [16:09:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:00:25] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/983799c7fc024ac187ce9ae1023b54efbfee82c8 [17:00:25] 02ssl/03main 07WikiTideBot 03983799c Bot: Auto-update domain lists [17:39:52] PROBLEM - mwtask171 Current Load on mwtask171 is CRITICAL: LOAD CRITICAL - total load average: 27.02, 20.57, 15.20 [17:45:52] PROBLEM - mwtask171 Current Load on mwtask171 is WARNING: LOAD WARNING - total load average: 14.35, 20.52, 17.42 [17:47:52] RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 7.24, 16.10, 16.20 [18:57:37] [02mediawiki-repos] 07SomeMWDev created 03T15470 (+1 new commit) 13https://github.com/miraheze/mediawiki-repos/commit/7a78478657b6 [18:57:38] 02mediawiki-repos/03T15470 07SomeRandomDeveloper 037a78478 T15470: Pin TabberNeue… [18:58:01] [02mediawiki-repos] 07SomeMWDev opened pull request #147: T15470: Pin TabberNeue (03main...03T15470) 13https://github.com/miraheze/mediawiki-repos/pull/147 [18:58:16] [02mediawiki-repos] 07SomeMWDev merged pull request #147: T15470: Pin TabberNeue (03main...03T15470) 13https://github.com/miraheze/mediawiki-repos/pull/147 [18:58:18] [02mediawiki-repos] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mediawiki-repos/commit/70cff4d1b3a7c6c4d9633763caba881b69e1514e [18:58:19] 02mediawiki-repos/03main 07SomeRandomDeveloper 0370cff4d T15470: Pin TabberNeue (#147)… [18:58:21] [02mediawiki-repos] 07SomeMWDev 04deleted 03T15470 at 037a78478 13https://api.github.com/repos/miraheze/mediawiki-repos/commit/7a78478 [19:09:58] [02TSPortal] 07renovate[bot] 04force-pushed 03renovate/prod-deps from 03c0c0660 to 03363bbdc 13https://api.github.com/repos/miraheze/TSPortal/commits/renovate/prod-deps [19:09:58] 02TSPortal/03renovate/prod-deps 07renovate[bot] 03363bbdc Update prod dependencies [20:01:37] !log [wwr@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=reconvergentfateswiki --startOver (END - exit=256) [20:01:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:03:07] hmm [20:04:42] !log [wwr@mwtask171] curl -X DELETE https://opensearch-mw.wikitide.net/reconvergentfateswiki_content https://opensearch-mw.wikitide.net/reconvergentfateswiki_general [20:04:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:05:13] !log [wwr@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=reconvergentfateswiki --startOver (END - exit=0) [20:05:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:06:07] !log [wwr@mwtask171] sudo -u www-data php /srv/mediawiki/1.45/maintenance/run.php CirrusSearch:ForceSearchIndex --wiki=reconvergentfateswiki (END - exit=0) [20:06:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:35:42] !log [void@phorge171] ./bin/remove destroy PHID-FILE-k2tylsmhfn6x3kliwqzp -- 65GB dump, not used for anything [20:35:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:55:18] !log [void@phorge171] ./bin/remove destroy F6715656 -- 44GB dump, accessible in other ways, probably done being processed too [20:55:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:00:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/fb35e8b8da7de6d9346dde30b8871c5c685138a6 [21:00:22] 02ssl/03main 07WikiTideBot 03fb35e8b Bot: Auto-update domain lists [21:11:53] @orduin nope still processing [21:11:57] 500k is quite a few images [21:12:27] it was broken anyway so [21:16:03] !log [void@phorge171] ./bin/remove destroy F5468524 -- XML for completed import T14843 [21:16:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:16:40] oh, well, point is the phab file wasn't needed [21:16:46] yup [21:22:46] !log [void@phorge171] ./bin/remove destroy F1423152 -- old incomplete upload [21:22:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:10] !log [void@phorge171] ./bin/remove destroy F4059761 -- old unfinished upload [21:29:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:30:10] !log [void@phorge171] ./bin/remove destroy F4059762 -- old unfinished upload [21:30:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:31:47] !log [void@phorge171] ./bin/remove destroy F2122720 -- old unused upload [21:31:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:32:29] !log [void@phorge171] ./bin/remove destroy F4059763 -- old unused upload [21:32:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:36:47] !log [void@phorge171] ./bin/remove destroy F4051737 -- old unused upload [21:36:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:44:34] !log [void@db182] [phabricator_calendar]> UPDATE calendar_event SET editPolicy="PHID-PLCY-w6bznnja2gez3vq6ea57" WHERE name="Miraheze Creation Date (or Birthday)"; -- T13002 [21:44:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:46:46] !log [void@phorge171] restart phorge_db-backup service [21:46:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:46:54] RECOVERY - phorge171 Check unit status of phorge_db-backup on phorge171 is OK: OK: Status of the systemd unit phorge_db-backup [22:41:52] PROBLEM - mwtask171 Current Load on mwtask171 is WARNING: LOAD WARNING - total load average: 23.33, 18.39, 13.46 [22:43:52] RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 20.26, 19.10, 14.34 [23:00:21] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/ae7a0c4d9dab78d1a6bb29c42e9500000b8745e0 [23:00:21] 02ssl/03main 07WikiTideBot 03ae7a0c4 Bot: Auto-update domain lists [23:18:54] PROBLEM - os202 Current Load on os202 is WARNING: LOAD WARNING - total load average: 7.96, 7.05, 6.08 [23:20:48] RECOVERY - os202 Current Load on os202 is OK: LOAD OK - total load average: 6.36, 6.71, 6.07 [23:59:01] PROBLEM - os162 Current Load on os162 is WARNING: LOAD WARNING - total load average: 7.37, 6.49, 5.60