[00:02:58] <icinga-miraheze>	 PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 8769MiB (9% inode=98%);
[00:04:58] <icinga-miraheze>	 RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 20284MiB (22% inode=98%);
[00:15:25] <icinga-miraheze>	 RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 52258MiB (11% inode=99%);
[00:20:58] <icinga-miraheze>	 PROBLEM - mwtask161 Current Load on mwtask161 is WARNING: LOAD WARNING - total load average: 21.63, 14.24, 7.12
[00:22:57] <icinga-miraheze>	 RECOVERY - mwtask161 Current Load on mwtask161 is OK: LOAD OK - total load average: 9.15, 13.15, 7.67
[01:07:58] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/7fd6517df3d2dfc5de8eb99cec23d77a9c2a91f2
[01:07:58] <MacFanBot>	 02ssl/03master 07MacFan4000 037fd6517 add wiki.jsal.org
[01:16:32] <MirahezeLSBot>	 !log [oa@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/sql.php --wiki=loginwiki --wikidb mhglobal /srv/mediawiki/1.43/extensions/GlobalBlocking/sql/mysql/patch-global_block_whitelist-default-gbw_address.sql (END - exit=0)
[01:16:34] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:19:14] <MirahezeLSBot>	 !log [oa@mwtask181] starting deploy of {'world': True, 'versions': '1.43', 'upgrade_extensions': 'GlobalBlocking'} to all
[01:19:16] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:21:45] <MirahezeLSBot>	 !log [oa@mwtask181] finished deploy of {'world': True, 'versions': '1.43', 'upgrade_extensions': 'GlobalBlocking'} to all - SUCCESS in 151s
[01:21:47] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:22:50] <MirahezeLSBot>	 !log [oa@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php /srv/mediawiki/1.43/maintenance/sql.php --wiki=loginwiki --wikidb mhglobal /srv/mediawiki/1.43/extensions/GlobalBlocking/sql/mysql/patch-global_block_whitelist-default-gbw_address.sql (END - exit=0)
[01:22:52] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[01:34:57] <icinga-miraheze>	 PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49802MiB (10% inode=99%);
[01:49:48] <icinga-miraheze>	 PROBLEM - hellowhirled.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:49:49] <icinga-miraheze>	 PROBLEM - cnidaria.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:49:50] <icinga-miraheze>	 PROBLEM - lotr.su - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:49:52] <icinga-miraheze>	 PROBLEM - clubpenguinfanon.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:49:55] <icinga-miraheze>	 PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 100%
[01:50:00] <icinga-miraheze>	 PROBLEM - binrayarchives.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:50:01] <icinga-miraheze>	 PROBLEM - junxions.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[01:50:27] <icinga-miraheze>	 PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb
[01:50:33] <icinga-miraheze>	 PROBLEM - cp36 HTTPS on cp36 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 503
[01:51:04] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2602:294:0:b13::110/cpweb
[01:51:19] <icinga-miraheze>	 PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 7 backends are down. mw152 mw161 mw172 mw182 mw154 mw174 mw183
[01:51:53] <icinga-miraheze>	 RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 6%
[01:52:24] <icinga-miraheze>	 RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online
[01:52:27] <icinga-miraheze>	 RECOVERY - cp36 HTTPS on cp36 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4114 bytes in 0.058 second response time
[01:53:04] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[01:53:19] <icinga-miraheze>	 RECOVERY - cp36 Varnish Backends on cp36 is OK: All 29 backends are healthy
[02:19:01] <icinga-miraheze>	 RECOVERY - hellowhirled.org - LetsEncrypt on sslhost is OK: OK - Certificate 'hellowhirled.org' will expire on Sun 30 Mar 2025 04:36:12 PM GMT +0000.
[02:19:20] <icinga-miraheze>	 RECOVERY - lotr.su - LetsEncrypt on sslhost is OK: OK - Certificate 'lotr.su' will expire on Wed 12 Mar 2025 04:28:42 PM GMT +0000.
[02:19:33] <icinga-miraheze>	 RECOVERY - clubpenguinfanon.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'clubpenguinfanon.wiki' will expire on Tue 25 Mar 2025 11:01:22 PM GMT +0000.
[02:19:45] <icinga-miraheze>	 RECOVERY - junxions.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'junxions.wiki' will expire on Mon 19 May 2025 01:46:07 PM GMT +0000.
[02:19:46] <icinga-miraheze>	 RECOVERY - cnidaria.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'cnidaria.wiki' will expire on Sun 04 May 2025 05:19:11 PM GMT +0000.
[02:19:49] <icinga-miraheze>	 RECOVERY - binrayarchives.com - LetsEncrypt on sslhost is OK: OK - Certificate 'binrayarchives.com' will expire on Fri 25 Apr 2025 09:14:14 PM GMT +0000.
[04:08:25] <MacFanBot>	 [02mw-config] 07BlankEclair opened pull request #5832: T13250: Use Wikimedia Commons for JsonConfig on openfrontwiki (07miraheze:03master...07BlankEclair:03T13250) 13https://github.com/miraheze/mw-config/pull/5832
[04:09:19] <miraheze-github>	 miraheze/mw-config - BlankEclair the build passed.
[05:38:39] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.82, 16.50, 9.56
[05:44:39] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 24.31, 21.41, 14.02
[05:46:39] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 20.62, 20.83, 14.70
[05:50:39] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 15.02, 20.27, 16.04
[05:56:10] <icinga-miraheze>	 PROBLEM - mwtask151 Current Load on mwtask151 is WARNING: LOAD WARNING - total load average: 23.68, 16.29, 8.25
[06:00:10] <icinga-miraheze>	 RECOVERY - mwtask151 Current Load on mwtask151 is OK: LOAD OK - total load average: 18.50, 18.57, 11.09
[06:05:32] <MacFanBot>	 [02mw-config] 07Reception123 pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/f423748f293f329256cc2bdda47e38b6aada51b8
[06:05:34] <MacFanBot>	 02mw-config/03master 07Reception123 03f423748 globally disable SimpleBlogPage…
[06:05:52] <MirahezeLSBot>	 !log [reception@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all
[06:05:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[06:06:12] <MirahezeLSBot>	 !log [reception@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 19s
[06:06:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[06:06:34] <miraheze-github>	 miraheze/mw-config - Reception123 the build passed.
[06:30:34] <icinga-miraheze>	 PROBLEM - holostars.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:30:47] <icinga-miraheze>	 PROBLEM - archivesofhavnor.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:30:48] <icinga-miraheze>	 PROBLEM - cp36 HTTPS on cp36 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10002 milliseconds
[06:30:58] <icinga-miraheze>	 PROBLEM - pandorastale.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:31:01] <icinga-miraheze>	 PROBLEM - rippaverse.wikitide.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:31:04] <icinga-miraheze>	 PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb
[06:31:27] <icinga-miraheze>	 PROBLEM - yepedia.xyz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:31:30] <icinga-miraheze>	 PROBLEM - crocwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:31:40] <icinga-miraheze>	 PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 99%
[06:31:46] <icinga-miraheze>	 PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb
[06:31:55] <icinga-miraheze>	 PROBLEM - pso2ngs.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[06:32:42] <icinga-miraheze>	 RECOVERY - cp36 HTTPS on cp36 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4112 bytes in 0.128 second response time
[06:33:04] <icinga-miraheze>	 RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online
[06:33:40] <icinga-miraheze>	 RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 7%
[06:33:46] <icinga-miraheze>	 RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online
[06:51:54] <icinga-miraheze>	 PROBLEM - enc.for.uz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'enc.for.uz' expires in 14 day(s) (Fri 07 Mar 2025 06:34:10 AM GMT +0000).
[06:54:49] <icinga-miraheze>	 PROBLEM - mwtask171 Current Load on mwtask171 is WARNING: LOAD WARNING - total load average: 23.29, 14.05, 6.90
[06:59:31] <icinga-miraheze>	 RECOVERY - pandorastale.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pandorastale.wiki' will expire on Tue 01 Apr 2025 05:59:24 PM GMT +0000.
[06:59:34] <icinga-miraheze>	 RECOVERY - rippaverse.wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Sat 12 Apr 2025 07:36:30 PM GMT +0000.
[07:00:05] <icinga-miraheze>	 RECOVERY - holostars.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'holostars.wiki' will expire on Mon 17 Mar 2025 04:06:52 PM GMT +0000.
[07:00:29] <icinga-miraheze>	 RECOVERY - crocwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'crocwiki.com' will expire on Sat 08 Mar 2025 03:13:41 PM GMT +0000.
[07:00:30] <icinga-miraheze>	 RECOVERY - yepedia.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'yepedia.xyz' will expire on Thu 24 Apr 2025 02:10:34 PM GMT +0000.
[07:00:39] <icinga-miraheze>	 RECOVERY - archivesofhavnor.org - LetsEncrypt on sslhost is OK: OK - Certificate 'archivesofhavnor.org' will expire on Thu 24 Apr 2025 01:55:22 PM GMT +0000.
[07:00:53] <icinga-miraheze>	 RECOVERY - pso2ngs.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pso2ngs.wiki' will expire on Thu 03 Apr 2025 10:53:33 AM GMT +0000.
[07:02:59] <icinga-miraheze>	 PROBLEM - runzeppelin.ru - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'runzeppelin.ru' expires in 14 day(s) (Fri 07 Mar 2025 06:45:22 AM GMT +0000).
[07:03:12] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/984475ad8210a8283c71dda538ec25a028a79535
[07:03:12] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03984475a Bot: Update SSL cert for runzeppelin.ru
[07:03:13] <icinga-miraheze>	 PROBLEM - nonciclopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nonciclopedia.org' expires in 14 day(s) (Fri 07 Mar 2025 06:52:20 AM GMT +0000).
[07:03:24] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/e2f53d6a7de621780acaf8dac658729d20004cfe
[07:03:24] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03e2f53d6 Bot: Update SSL cert for nonciclopedia.org
[07:08:49] <icinga-miraheze>	 RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 4.38, 15.62, 13.94
[07:31:52] <icinga-miraheze>	 RECOVERY - nonciclopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nonciclopedia.org' will expire on Wed 21 May 2025 06:04:48 AM GMT +0000.
[07:32:24] <icinga-miraheze>	 RECOVERY - runzeppelin.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'runzeppelin.ru' will expire on Wed 21 May 2025 06:04:35 AM GMT +0000.
[07:43:55] <icinga-miraheze>	 PROBLEM - stablestate.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'stablestate.org' expires in 14 day(s) (Fri 07 Mar 2025 07:29:55 AM GMT +0000).
[07:44:07] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/63a7810b89c09fc3133f9eb5b63fec55f9ef1854
[07:44:07] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 0363a7810 Bot: Update SSL cert for stablestate.org
[07:51:33] <icinga-miraheze>	 PROBLEM - mh142.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mh142.com' expires in 14 day(s) (Fri 07 Mar 2025 07:39:12 AM GMT +0000).
[07:51:44] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/bf4768ebe9dd1bf2e1077e7c60f2103c3b3d753a
[07:51:45] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03bf4768e Bot: Update SSL cert for mh142.com
[08:13:18] <icinga-miraheze>	 RECOVERY - stablestate.org - LetsEncrypt on sslhost is OK: OK - Certificate 'stablestate.org' will expire on Wed 21 May 2025 06:45:31 AM GMT +0000.
[08:20:53] <icinga-miraheze>	 PROBLEM - kunwok.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'kunwok.org' expires in 14 day(s) (Fri 07 Mar 2025 08:15:03 AM GMT +0000).
[08:21:05] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/8d0ee871f39ccc1a8d451e39bf3be8f1627e8aa3
[08:21:05] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 038d0ee87 Bot: Update SSL cert for kunwok.org
[08:21:14] <icinga-miraheze>	 RECOVERY - mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Wed 21 May 2025 06:53:08 AM GMT +0000.
[08:50:46] <icinga-miraheze>	 RECOVERY - kunwok.org - LetsEncrypt on sslhost is OK: OK - Certificate 'kunwok.org' will expire on Wed 21 May 2025 07:22:29 AM GMT +0000.
[08:50:53] <icinga-miraheze>	 PROBLEM - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'encyclopediaofastrobiology.org' expires in 14 day(s) (Fri 07 Mar 2025 08:41:41 AM GMT +0000).
[08:51:04] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/506b05391ba787cabbfe44f2618905b709d71c22
[08:51:04] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03506b053 Bot: Update SSL cert for encyclopediaofastrobiology.org
[09:08:57] <icinga-miraheze>	 PROBLEM - monarchists.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'monarchists.wiki' expires in 14 day(s) (Fri 07 Mar 2025 08:41:30 AM GMT +0000).
[09:09:08] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/206d75df178273a8307cb3752f5fa0e8c39da2a1
[09:09:08] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03206d75d Bot: Update SSL cert for monarchists.wiki
[09:20:16] <icinga-miraheze>	 RECOVERY - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediaofastrobiology.org' will expire on Wed 21 May 2025 07:52:29 AM GMT +0000.
[09:20:22] <icinga-miraheze>	 PROBLEM - podpedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'podpedia.org' expires in 14 day(s) (Fri 07 Mar 2025 09:06:45 AM GMT +0000).
[10:07:16] <icinga-miraheze>	 RECOVERY - monarchists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'monarchists.wiki' will expire on Wed 21 May 2025 08:10:32 AM GMT +0000.
[10:16:17] <BlankEclair>	 tf is monarchists.wiki
[10:17:16] <BlankEclair>	 https://meta.miraheze.org/wiki/Special:RequestWikiQueue/6323v
[10:17:19] <BlankEclair>	 https://meta.miraheze.org/wiki/Special:RequestWikiQueue/6323
[10:17:20] <BlankEclair>	 v chill
[10:45:36] <icinga-miraheze>	 PROBLEM - bristolstudenthousingcoop.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'bristolstudenthousingcoop.org' expires in 14 day(s) (Fri 07 Mar 2025 10:39:11 AM GMT +0000).
[10:45:47] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/12ef911edb3e5e235d4190e54356ae2bb068a927
[10:45:48] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 0312ef911 Bot: Update SSL cert for bristolstudenthousingcoop.org
[11:09:35] <icinga-miraheze>	 PROBLEM - bebaskanpengetahuan.id - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'bebaskanpengetahuan.id' expires in 14 day(s) (Fri 07 Mar 2025 10:41:08 AM GMT +0000).
[11:09:49] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/134be87f7dfd981a3b328fad332e870a255e25d3
[11:09:49] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03134be87 Bot: Update SSL cert for bebaskanpengetahuan.id
[11:27:33] <icinga-miraheze>	 PROBLEM - matomo151 HTTPS on matomo151 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to matomo151.wikitide.net port 443 after 0 ms: Couldn't connect to server
[11:38:12] <icinga-miraheze>	 RECOVERY - bebaskanpengetahuan.id - LetsEncrypt on sslhost is OK: OK - Certificate 'bebaskanpengetahuan.id' will expire on Wed 21 May 2025 10:11:11 AM GMT +0000.
[11:43:36] <icinga-miraheze>	 RECOVERY - bristolstudenthousingcoop.org - LetsEncrypt on sslhost is OK: OK - Certificate 'bristolstudenthousingcoop.org' will expire on Wed 21 May 2025 09:47:11 AM GMT +0000.
[11:55:33] <icinga-miraheze>	 RECOVERY - matomo151 HTTPS on matomo151 is OK: HTTP OK: HTTP/2 200 - 553 bytes in 0.251 second response time
[12:12:15] <MacFanBot>	 [02CreateWiki] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/CreateWiki/commit/fd6bf420fccbed564af05c293c697608f9c6db0c
[12:12:15] <MacFanBot>	 02CreateWiki/03master 07translatewiki.net 03fd6bf42 Localisation updates from https://translatewiki.net.
[12:12:16] <MacFanBot>	 [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/MirahezeMagic/commit/e8e6246885d87cf0fa949ea417dd985d1ac03ad9
[12:12:18] <MacFanBot>	 02MirahezeMagic/03master 07translatewiki.net 03e8e6246 Localisation updates from https://translatewiki.net.
[12:12:20] <MacFanBot>	 [02landing] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/landing/commit/63ab712d7dfe226f07128d3f98d303724c08f792
[12:12:22] <MacFanBot>	 [02MatomoAnalytics] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/MatomoAnalytics/commit/ba7ad3fc4df9a60bff21b5212fcb99a477cd55c4
[12:12:24] <MacFanBot>	 [02DataDump] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/DataDump/commit/8f4929725ffb5b90a799a6dc4162b0590260c938
[12:12:25] <MacFanBot>	 02DataDump/03master 07translatewiki.net 038f49297 Localisation updates from https://translatewiki.net.
[12:12:27] <MacFanBot>	 [02RequestSSL] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/RequestSSL/commit/89313154c37de7f1fa2136ef9a479996d31f4e34
[12:12:29] <MacFanBot>	 02RequestSSL/03master 07translatewiki.net 038931315 Localisation updates from https://translatewiki.net.
[12:12:31] <MacFanBot>	 02MatomoAnalytics/03master 07translatewiki.net 03ba7ad3f Localisation updates from https://translatewiki.net.
[12:12:32] <MacFanBot>	 02landing/03master 07translatewiki.net 0363ab712 Localisation updates from https://translatewiki.net.
[12:12:34] <MacFanBot>	 [02WikiDiscover] 07translatewiki pushed 1 new commit to 03master 13https://github.com/miraheze/WikiDiscover/commit/da5355792c149e7c375d718bcfba7e963d821bad
[12:12:36] <MacFanBot>	 02WikiDiscover/03master 07translatewiki.net 03da53557 Localisation updates from https://translatewiki.net.
[12:13:10] <miraheze-github>	 miraheze/landing - translatewiki the build passed.
[12:15:42] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 11.27, 9.77, 5.61
[12:16:56] <miraheze-github>	 miraheze/MatomoAnalytics - translatewiki the build passed.
[12:17:03] <miraheze-github_>	 miraheze/WikiDiscover - translatewiki the build passed.
[12:17:06] <miraheze-github>	 miraheze/DataDump - translatewiki the build passed.
[12:17:42] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 21.01, 13.51, 7.46
[12:20:07] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.44, 19.92, 13.43
[12:22:02] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.26, 18.42, 13.64
[12:23:36] <icinga-miraheze>	 PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 23.09, 20.15, 14.70
[12:23:59] <miraheze-github>	 miraheze/MirahezeMagic - translatewiki the build passed.
[12:24:46] <miraheze-github>	 miraheze/RequestSSL - translatewiki the build passed.
[12:24:57] <miraheze-github>	 miraheze/CreateWiki - translatewiki the build passed.
[12:26:04] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.84, 19.45, 14.62
[12:27:25] <icinga-miraheze>	 RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.76, 19.49, 15.75
[12:29:42] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 4.46, 11.19, 10.93
[12:31:18] <icinga-miraheze>	 PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.45, 20.88, 17.10
[12:32:04] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 13.50, 19.31, 16.47
[12:33:13] <icinga-miraheze>	 RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 19.31, 19.34, 16.92
[12:33:23] <icinga-miraheze>	 PROBLEM - mw184 Current Load on mw184 is WARNING: LOAD WARNING - total load average: 23.35, 20.49, 15.97
[12:33:42] <icinga-miraheze>	 RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.65, 8.68, 9.91
[12:35:20] <icinga-miraheze>	 RECOVERY - mw184 Current Load on mw184 is OK: LOAD OK - total load average: 19.57, 20.00, 16.33
[12:36:00] <icinga-miraheze>	 [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[12:37:42] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 18.07, 12.27, 10.98
[12:38:04] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.01, 20.81, 18.00
[12:39:14] <icinga-miraheze>	 PROBLEM - mw174 Current Load on mw174 is WARNING: LOAD WARNING - total load average: 22.56, 20.38, 16.89
[12:41:09] <icinga-miraheze>	 RECOVERY - mw174 Current Load on mw174 is OK: LOAD OK - total load average: 15.39, 18.93, 16.81
[12:41:37] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.80, 20.01, 17.50
[12:43:37] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.68, 17.18, 16.74
[12:44:04] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 18.99, 19.56, 18.43
[12:46:33] <icinga-miraheze>	 PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.55, 21.06, 19.12
[12:46:58] <icinga-miraheze>	 PROBLEM - mw174 Current Load on mw174 is WARNING: LOAD WARNING - total load average: 20.64, 20.07, 17.99
[12:48:04] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.95, 21.05, 19.26
[12:48:27] <icinga-miraheze>	 RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 14.88, 18.91, 18.59
[12:48:53] <icinga-miraheze>	 RECOVERY - mw174 Current Load on mw174 is OK: LOAD OK - total load average: 19.22, 19.42, 17.98
[12:49:42] <icinga-miraheze>	 PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 3.69, 9.66, 11.11
[12:50:04] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 15.44, 20.05, 19.19
[12:51:42] <icinga-miraheze>	 RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 1.26, 6.73, 9.86
[12:56:00] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[13:22:10] <icinga-miraheze>	 PROBLEM - mwtask151 Current Load on mwtask151 is WARNING: LOAD WARNING - total load average: 21.94, 15.74, 8.02
[13:24:10] <icinga-miraheze>	 RECOVERY - mwtask151 Current Load on mwtask151 is OK: LOAD OK - total load average: 15.50, 16.40, 9.26
[14:50:11] <icinga-miraheze>	 PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 27154MiB (5% inode=99%);
[15:16:12] <icinga-miraheze>	 PROBLEM - infectowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'infectowiki.com' expires in 14 day(s) (Fri 07 Mar 2025 03:02:46 PM GMT +0000).
[15:16:29] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/29501138875679b0a91b81c297e2fe5e04d390fb
[15:16:29] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 032950113 Bot: Update SSL cert for infectowiki.com
[16:14:37] <icinga-miraheze>	 RECOVERY - infectowiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'infectowiki.com' will expire on Wed 21 May 2025 02:17:52 PM GMT +0000.
[18:02:10] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/5d022519391288f490f9c6cca04453a48c4b100d
[18:02:10] <MacFanBot>	 02puppet/03master 07CosmicAlpha 035d02251 Use namespaced TextContent class
[18:06:12] <MirahezeRelay>	 <rhinosf1> @cosmicalpha why is that in puppet and not an entry point in mw-config
[18:10:41] <MacFanBot>	 [02puppet] 07Universal-Omega created 03replace-wfExpandUrl (+1 new commit) 13https://github.com/miraheze/puppet/commit/29afab295f8a
[18:10:41] <MacFanBot>	 02puppet/03replace-wfExpandUrl 07CosmicAlpha 0329afab2 Replace deprecated wfExpandUrl
[18:10:48] <MacFanBot>	 [02puppet] 07Universal-Omega opened pull request #4190: Replace deprecated wfExpandUrl (03master...03replace-wfExpandUrl) 13https://github.com/miraheze/puppet/pull/4190
[18:11:06] <icinga-miraheze>	 PROBLEM - hololive.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'hololive.wiki' expires in 14 day(s) (Fri 07 Mar 2025 06:05:48 PM GMT +0000).
[18:11:18] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/20b71b1fa70d7e564d8479307f82c8b01df67c49
[18:11:19] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 0320b71b1 Bot: Update SSL cert for hololive.wiki
[18:14:28] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03replace-wfExpandUrl 13https://github.com/miraheze/puppet/commit/e50a763a26ee922ea82e7321fad004ab1d616202
[18:14:28] <MacFanBot>	 02puppet/03replace-wfExpandUrl 07CosmicAlpha 03e50a763 Replace
[18:14:36] <MacFanBot>	 [02puppet] 07coderabbitai[bot] commented on pull request #4190: --- […] 13https://github.com/miraheze/puppet/pull/4190#issuecomment-2672304215
[18:15:37] <MirahezeRelay>	 <cosmicalpha, replying to rhinosf1> Because it came before entrypoints/multiversion. I plan to move it eventually though.
[18:18:02] <MirahezeRelay>	 <rhinosf1, replying to cosmicalpha> Fair
[18:18:14] <MirahezeRelay>	 <rhinosf1> Did you see my DM too?
[18:18:30] <MirahezeRelay>	 <cosmicalpha> I did I'll respond in a moment.
[18:19:11] <MacFanBot>	 [02puppet] 07Universal-Omega merged pull request #4190: Replace deprecated wfExpandUrl (03master...03replace-wfExpandUrl) 13https://github.com/miraheze/puppet/pull/4190
[18:19:13] <MacFanBot>	 [02puppet] 07Universal-Omega pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/3f8968a6183fdaff77a7881de29a83e2a37ba43e
[18:19:14] <MacFanBot>	 02puppet/03master 07CosmicAlpha 033f8968a Replace deprecated wfExpandUrl (#4190)
[18:19:16] <MacFanBot>	 [02puppet] 07Universal-Omega 04deleted 03replace-wfExpandUrl at 03e50a763 13https://github.com/miraheze/puppet/commit/e50a763
[18:38:40] <icinga-miraheze>	 PROBLEM - n64brew.dev - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'n64brew.dev' expires in 14 day(s) (Fri 07 Mar 2025 06:15:58 PM GMT +0000).
[18:38:52] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/3c8976c86f4c98b01c99087771dbfe50ba0666ad
[18:38:52] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 033c8976c Bot: Update SSL cert for n64brew.dev
[19:07:49] <icinga-miraheze>	 RECOVERY - n64brew.dev - LetsEncrypt on sslhost is OK: OK - Certificate 'n64brew.dev' will expire on Wed 21 May 2025 05:40:16 PM GMT +0000.
[19:09:06] <icinga-miraheze>	 RECOVERY - hololive.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hololive.wiki' will expire on Wed 21 May 2025 05:12:42 PM GMT +0000.
[19:57:43] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/f0ad3099af5a24a5d50386d17673e5028c89db92
[19:57:43] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03f0ad309 Bot: Update SSL cert for podpedia.org
[19:58:22] <MacFanBot>	 [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/b16748e9c4f6c9879ce1b5d908e9d5fe54badf60
[19:58:22] <MacFanBot>	 02ssl/03master 07WikiTideSSLBot 03b16748e Bot: Update SSL cert for enc.for.uz
[20:02:21] <MacFanBot>	 [02ssl] 07MacFan4000 created 03MacFan4000-patch-1 (+1 new commit) 13https://github.com/miraheze/ssl/commit/015378b6b227
[20:02:21] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 03015378b remove expired/non-pointed domains
[20:02:28] <MacFanBot>	 [02ssl] 07MacFan4000 opened pull request #823: remove expired/non-pointed domains (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/823
[20:03:10] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03MacFan4000-patch-1 13https://github.com/miraheze/ssl/commit/597ce0598a7c94dda327f750e122cfbc6df85003
[20:03:10] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 03597ce05 Delete certificates/aryavartpedia.online.crt
[20:03:59] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03MacFan4000-patch-1 13https://github.com/miraheze/ssl/commit/b95513bf92a4e0e47899ee34ec2aa0f07f53cffe
[20:04:01] <MacFanBot>	 02ssl/03MacFan4000-patch-1 07MacFan4000 03b95513b Delete certificates/agesofconflict.wiki.crt
[20:04:21] <MacFanBot>	 [02ssl] 07coderabbitai[bot] commented on pull request #823: --- […] 13https://github.com/miraheze/ssl/pull/823#issuecomment-2672556106
[20:06:30] <MacFanBot>	 [02ssl] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/455c5a64266619b71fb8b40e7f1f7a96459cd147
[20:06:30] <MacFanBot>	 02ssl/03master 07MacFan4000 03455c5a6 remove expired/non-pointed domains (#823)
[20:06:30] <MacFanBot>	 [02ssl] 07MacFan4000 merged pull request #823: remove expired/non-pointed domains (03master...03MacFan4000-patch-1) 13https://github.com/miraheze/ssl/pull/823
[20:06:32] <MacFanBot>	 [02ssl] 07MacFan4000 04deleted 03MacFan4000-patch-1 at 03b95513b 13https://github.com/miraheze/ssl/commit/b95513b
[20:09:09] <MacFanBot>	 [02dns] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/dns/commit/e5eeb1aa4efcfa9359474bec134c28a1c4d139bd
[20:09:10] <MacFanBot>	 02dns/03master 07MacFan4000 03e5eeb1a Delete zones/agesofconflict.wiki
[20:09:28] <MacFanBot>	 [02dns] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/dns/commit/bd6b96566ebba8f93bf54ebccf3789779db9d4c4
[20:09:28] <MacFanBot>	 02dns/03master 07MacFan4000 03bd6b965 Delete zones/aryavartpedia.online
[20:10:14] <miraheze-github>	 miraheze/dns - MacFan4000 the build passed.
[20:10:29] <miraheze-github>	 miraheze/dns - MacFan4000 the build passed.
[20:32:31] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.30, 19.59, 15.20
[20:38:28] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is CRITICAL: LOAD CRITICAL - total load average: 26.33, 22.47, 17.80
[20:40:27] <icinga-miraheze>	 PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 23.35, 22.62, 18.40
[20:41:29] <icinga-miraheze>	 RECOVERY - enc.for.uz - LetsEncrypt on sslhost is OK: OK - Certificate 'enc.for.uz' will expire on Wed 21 May 2025 06:59:47 PM GMT +0000.
[20:43:39] <icinga-miraheze>	 RECOVERY - podpedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'podpedia.org' will expire on Wed 21 May 2025 06:59:07 PM GMT +0000.
[20:52:22] <icinga-miraheze>	 RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 19.68, 20.34, 19.69
[22:20:24] <icinga-miraheze>	 PROBLEM - mwtask171 Current Load on mwtask171 is CRITICAL: LOAD CRITICAL - total load average: 29.52, 15.23, 6.55
[23:26:25] <icinga-miraheze>	 PROBLEM - mwtask171 Current Load on mwtask171 is WARNING: LOAD WARNING - total load average: 0.50, 11.60, 21.23
[23:28:24] <icinga-miraheze>	 RECOVERY - mwtask171 Current Load on mwtask171 is OK: LOAD OK - total load average: 2.33, 9.12, 19.18
[23:56:01] <icinga-miraheze>	 PROBLEM - civwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'civwiki.org' expires in 14 day(s) (Fri 07 Mar 2025 11:39:55 PM GMT +0000).