[00:02:20] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.59, 3.70, 3.59
[00:03:47] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 27.42, 23.66, 20.78
[00:05:40] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.100 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[00:05:45] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 19.84, 22.43, 20.70
[00:09:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 25.61, 22.92, 21.21
[00:10:07] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[00:11:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.62, 22.22, 21.15
[00:12:06] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.068 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[00:15:30] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[00:18:32] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[00:22:40] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.063 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[00:41:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 25.08, 22.31, 21.57
[00:43:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 18.38, 21.22, 21.29
[00:47:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 25.13, 21.35, 21.16
[00:49:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 20.59, 20.21, 20.72
[00:50:30] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[00:53:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 24.33, 21.93, 21.30
[00:55:30] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[00:55:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 19.16, 20.61, 20.89
[01:03:41] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.30, 3.07, 3.90
[01:07:38] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 0.08, 1.44, 3.04
[01:07:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 24.45, 21.96, 21.23
[01:09:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 20.09, 21.12, 21.02
[01:19:40] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 20.28, 19.69, 20.31
[01:25:30] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[01:29:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 26.37, 23.09, 21.26
[01:33:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.71, 22.70, 21.53
[01:43:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 25.52, 22.81, 21.80
[01:45:40] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 18.07, 21.15, 21.33
[01:51:40] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 18.73, 18.98, 20.26
[02:18:47] <botifico>	 [02mw-config] 07Universal-Omega closed pull request 03#5634: T12443: Add $wgCitizenOverflowNowrapClasses - 13https://github.com/miraheze/mw-config/pull/5634
[02:18:50] <botifico>	 [02mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/e89395fcc7c0...1131adef04a6
[02:18:53] <botifico>	 [02mw-config] 07BlankEclair 031131ade - T12443: Add $wgCitizenOverflowNowrapClasses (#5634)
[02:19:17] <botifico>	 [02mw-config] 07Universal-Omega closed pull request 03#5632: T12437: Add $wgTabberNeueUseCodex - 13https://github.com/miraheze/mw-config/pull/5632
[02:19:20] <botifico>	 [02mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/1131adef04a6...a3596b946548
[02:19:21] <botifico>	 [02mw-config] 07BlankEclair 03a3596b9 - T12437: Add $wgTabberNeueUseCodex (#5632)
[02:19:28] <botifico>	 [02mw-config] 07Universal-Omega closed pull request 03#5626: Update justinrainbow/json-schema requirement from 5.3.0 to 6.0.0 - 13https://github.com/miraheze/mw-config/pull/5626
[02:19:30] <botifico>	 [02mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a3596b946548...049db98ce5c0
[02:19:32] <botifico>	 [02mw-config] 07dependabot[bot] 03049db98 - Update justinrainbow/json-schema requirement from 5.3.0 to 6.0.0 (#5626)
[02:19:33] <botifico>	 [02mw-config] 07Universal-Omega deleted branch 03dependabot/composer/justinrainbow/json-schema-6.0.0
[02:19:34] <botifico>	 [02mw-config] 07Universal-Omega deleted branch 03dependabot/composer/justinrainbow/json-schema-6.0.0 - 13https://github.com/miraheze/mw-config
[02:19:44] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[02:20:28] <miraheze-github>	 miraheze/mw-config - Universal-Omega the build passed.
[02:22:41] <botifico>	 [02ssl] 07Universal-Omega closed pull request 03#789: T12430: Redirect strinova.miraheze.org to strinova.org - 13https://github.com/miraheze/ssl/pull/789
[02:22:44] <botifico>	 [02ssl] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a5d71e37affe...016eee3e4d62
[02:22:46] <botifico>	 [02ssl] 07lihaohong6 03016eee3 - T12430: Redirect strinova.miraheze.org to strinova.org (#789)
[02:23:06] <MirahezeLSBot>	 !log [@test151] starting deploy of {'config': True} to test151
[02:23:07] <MirahezeLSBot>	 !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s
[02:23:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:23:17] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:25:51] <botifico>	 [02puppet] 07Universal-Omega reviewed pull request 03#3893 commit - 13https://github.com/miraheze/puppet/pull/3893#discussion_r1712491728
[02:32:48] <MirahezeLSBot>	 !log [@mwtask181] starting deploy of {'config': True} to all
[02:32:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:33:01] <MirahezeLSBot>	 !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 13s
[02:33:10] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:37:43] <MirahezeLSBot>	 !log [@mwtask171] starting deploy of {'config': True} to all
[02:37:49] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[02:37:57] <MirahezeLSBot>	 !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 14s
[02:38:03] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[03:10:23] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.17, 3.41, 1.99
[03:12:21] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.07, 3.40, 2.14
[03:15:51] <icinga-miraheze>	 PROBLEM - www.rosettacode.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rosettacode.org' expires in 15 day(s) (Mon 26 Aug 2024 03:10:07 AM GMT +0000).
[03:18:16] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.84, 3.27, 2.58
[03:26:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.18, 3.96, 3.09
[03:28:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.25, 4.53, 3.38
[03:34:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.08, 3.57, 3.33
[03:36:16] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.51, 3.02, 3.14
[03:36:46] <icinga-miraheze>	 PROBLEM - rosettacode.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rosettacode.org' expires in 15 day(s) (Mon 26 Aug 2024 03:10:07 AM GMT +0000).
[03:37:06] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/016eee3e4d62...f8abf4ab31c8
[03:37:08] <botifico>	 [02ssl] 07WikiTideSSLBot 03f8abf4a - Bot: Update SSL cert for rosettacode.org
[03:48:13] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 3.77, 4.29, 3.56
[03:58:18] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[03:59:20] <icinga-miraheze>	 [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:00:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 29.87, 23.95, 19.37
[04:00:16] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.072 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[04:04:01] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.96, 3.58, 3.84
[04:04:20] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:05:18] <icinga-miraheze>	 RECOVERY - rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Fri 08 Nov 2024 02:38:28 AM GMT +0000.
[04:07:59] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.50, 4.20, 3.94
[04:09:17] <icinga-miraheze>	 PROBLEM - mw162 Current Load on mw162 is CRITICAL: LOAD CRITICAL - total load average: 25.08, 22.35, 17.31
[04:09:52] <icinga-miraheze>	 PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 23.19, 19.33, 15.33
[04:09:57] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.10, 3.57, 3.77
[04:11:13] <icinga-miraheze>	 PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 23.73, 22.22, 17.82
[04:11:52] <icinga-miraheze>	 RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 17.25, 18.28, 15.43
[04:11:55] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.51, 4.52, 4.10
[04:12:12] <icinga-miraheze>	 [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:13:54] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.36, 3.75, 3.85
[04:15:06] <icinga-miraheze>	 RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 16.70, 20.05, 17.99
[04:15:08] <icinga-miraheze>	 RECOVERY - www.rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Fri 08 Nov 2024 02:38:28 AM GMT +0000.
[04:17:52] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.82, 4.19, 3.97
[04:19:50] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.60, 3.74, 3.82
[04:21:50] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.55, 3.84, 3.81
[04:22:19] <icinga-miraheze>	 PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 20.43, 19.89, 17.87
[04:24:07] <icinga-miraheze>	 PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 23.68, 21.53, 18.81
[04:24:15] <icinga-miraheze>	 RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 15.92, 18.48, 17.60
[04:26:07] <icinga-miraheze>	 RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 19.27, 20.24, 18.64
[04:28:06] <icinga-miraheze>	 PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 23.11, 20.26, 18.42
[04:30:02] <icinga-miraheze>	 RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 16.25, 18.77, 18.12
[04:30:07] <icinga-miraheze>	 PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 19.21, 20.44, 19.16
[04:32:02] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[04:32:07] <icinga-miraheze>	 RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 16.84, 19.07, 18.81
[04:32:12] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:34:01] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[04:37:12] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[04:39:36] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.04, 3.35, 3.96
[04:41:35] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.01, 4.03, 4.13
[04:43:34] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.59, 3.15, 3.81
[04:47:32] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.00, 4.20, 4.05
[04:57:06] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[05:01:18] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 4.270 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[05:02:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 18.58, 21.92, 23.98
[05:07:12] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[05:07:17] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 0.34, 2.90, 4.00
[05:11:13] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 0.09, 1.36, 3.12
[05:32:16] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 17.66, 18.58, 20.33
[05:46:29] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H <host> [-4|-6] [-w <warn>] [-c <crit>] [-v verbose] [-o <time offset>]
[05:50:32] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.001457750797 secs
[06:16:16] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[06:46:38] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[07:03:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.80, 22.20, 23.77
[07:06:17] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on bots171
[07:06:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:06:31] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on eventgate181
[07:06:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:06:43] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on jobchron171
[07:06:48] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:06:54] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp36
[07:07:03] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:07:06] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on bast161
[07:07:11] <icinga-miraheze>	 RECOVERY - jobchron171 APT on jobchron171 is OK: APT OK: 61 packages available for upgrade (0 critical updates).
[07:07:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:07:18] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cloud16
[07:07:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:07:28] <icinga-miraheze>	 RECOVERY - cp36 APT on cp36 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:07:30] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp37
[07:07:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:07:35] <icinga-miraheze>	 RECOVERY - bast161 APT on bast161 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:07:41] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on db161
[07:07:46] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:07:52] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on matomo151
[07:07:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:04] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on db151
[07:08:08] <icinga-miraheze>	 RECOVERY - bots171 APT on bots171 is OK: APT OK: 62 packages available for upgrade (0 critical updates).
[07:08:09] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:15] <icinga-miraheze>	 RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 62 packages available for upgrade (0 critical updates).
[07:08:15] <icinga-miraheze>	 RECOVERY - db161 APT on db161 is OK: APT OK: 58 packages available for upgrade (0 critical updates).
[07:08:16] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on db171
[07:08:21] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:21] <icinga-miraheze>	 RECOVERY - eventgate181 APT on eventgate181 is OK: APT OK: 44 packages available for upgrade (0 critical updates).
[07:08:26] <icinga-miraheze>	 RECOVERY - cp37 APT on cp37 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:08:27] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on graylog161
[07:08:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:39] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cloud15
[07:08:44] <icinga-miraheze>	 RECOVERY - cloud16 APT on cloud16 is OK: APT OK: 101 packages available for upgrade (0 critical updates).
[07:08:45] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:50] <icinga-miraheze>	 RECOVERY - graylog161 APT on graylog161 is OK: APT OK: 55 packages available for upgrade (0 critical updates).
[07:08:51] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cloud17
[07:08:52] <icinga-miraheze>	 RECOVERY - db171 APT on db171 is OK: APT OK: 58 packages available for upgrade (0 critical updates).
[07:08:56] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:08:58] <icinga-miraheze>	 RECOVERY - cloud15 APT on cloud15 is OK: APT OK: 109 packages available for upgrade (0 critical updates).
[07:09:03] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on graphite151
[07:09:09] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:09:16] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on db182
[07:09:21] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:09:27] <icinga-miraheze>	 RECOVERY - cloud17 APT on cloud17 is OK: APT OK: 101 packages available for upgrade (0 critical updates).
[07:09:28] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on ldap171
[07:09:33] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:09:34] <icinga-miraheze>	 RECOVERY - db151 APT on db151 is OK: APT OK: 59 packages available for upgrade (0 critical updates).
[07:09:40] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on db181
[07:09:45] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:09:53] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cloud18
[07:09:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:10:06] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on bast181
[07:10:07] <icinga-miraheze>	 RECOVERY - db182 APT on db182 is OK: APT OK: 59 packages available for upgrade (0 critical updates).
[07:10:12] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:10:18] <icinga-miraheze>	 RECOVERY - bast181 APT on bast181 is OK: APT OK: 52 packages available for upgrade (0 critical updates).
[07:10:19] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on kafka181
[07:10:24] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:10:28] <icinga-miraheze>	 RECOVERY - ldap171 APT on ldap171 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:10:32] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp27
[07:10:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:10:44] <icinga-miraheze>	 RECOVERY - graphite151 APT on graphite151 is OK: APT OK: 39 packages available for upgrade (0 critical updates).
[07:10:47] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp26
[07:10:53] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:10:57] <icinga-miraheze>	 RECOVERY - cloud18 APT on cloud18 is OK: APT OK: 102 packages available for upgrade (0 critical updates).
[07:11:00] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mem151
[07:11:00] <icinga-miraheze>	 RECOVERY - kafka181 APT on kafka181 is OK: APT OK: 37 packages available for upgrade (0 critical updates).
[07:11:05] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:11:05] <icinga-miraheze>	 RECOVERY - db181 APT on db181 is OK: APT OK: 59 packages available for upgrade (0 critical updates).
[07:11:08] <icinga-miraheze>	 RECOVERY - cp26 APT on cp26 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:11:13] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mem161
[07:11:19] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:11:25] <icinga-miraheze>	 RECOVERY - mem151 APT on mem151 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:11:26] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mon181
[07:11:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:11:38] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on os161
[07:11:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:11:52] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp41
[07:11:58] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:12:03] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on prometheus151
[07:12:10] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:12:17] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mwtask171
[07:12:20] <icinga-miraheze>	 RECOVERY - mon181 APT on mon181 is OK: APT OK: 57 packages available for upgrade (0 critical updates).
[07:12:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:12:23] <icinga-miraheze>	 RECOVERY - cp41 APT on cp41 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:12:27] <icinga-miraheze>	 RECOVERY - cp27 APT on cp27 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:12:29] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on phorge171
[07:12:35] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:12:41] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on os151
[07:12:46] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:12:47] <icinga-miraheze>	 RECOVERY - os161 APT on os161 is OK: APT OK: 52 packages available for upgrade (0 critical updates).
[07:12:53] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on os162
[07:12:53] <icinga-miraheze>	 RECOVERY - os151 APT on os151 is OK: APT OK: 53 packages available for upgrade (0 critical updates).
[07:12:56] <icinga-miraheze>	 RECOVERY - prometheus151 APT on prometheus151 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:12:57] <icinga-miraheze>	 RECOVERY - mem161 APT on mem161 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:12:58] <icinga-miraheze>	 RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 62 packages available for upgrade (0 critical updates).
[07:13:00] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:13:05] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw151
[07:13:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:13:18] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw161
[07:13:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:13:27] <icinga-miraheze>	 RECOVERY - phorge171 APT on phorge171 is OK: APT OK: 53 packages available for upgrade (0 critical updates).
[07:13:38] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5, postgresql-client-15, and postgresql-15 on puppet181
[07:13:43] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:13:51] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw152
[07:13:57] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:14:04] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw172
[07:14:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:14:10] <icinga-miraheze>	 RECOVERY - mw152 APT on mw152 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:14:17] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw162
[07:14:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:14:28] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on rdb151
[07:14:28] <icinga-miraheze>	 RECOVERY - puppet181 APT on puppet181 is OK: APT OK: 58 packages available for upgrade (0 critical updates).
[07:14:31] <icinga-miraheze>	 RECOVERY - mw151 APT on mw151 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:14:33] <icinga-miraheze>	 RECOVERY - os162 APT on os162 is OK: APT OK: 53 packages available for upgrade (0 critical updates).
[07:14:33] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:14:34] <icinga-miraheze>	 RECOVERY - mw162 APT on mw162 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:14:40] <icinga-miraheze>	 RECOVERY - mw172 APT on mw172 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:14:41] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mwtask181
[07:14:46] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:14:54] <icinga-miraheze>	 RECOVERY - mw161 APT on mw161 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:14:54] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw171
[07:15:00] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:15:07] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on ns1
[07:15:13] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:15:22] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw182
[07:15:27] <icinga-miraheze>	 RECOVERY - rdb151 APT on rdb151 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:15:27] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:15:32] <icinga-miraheze>	 RECOVERY - mw171 APT on mw171 is OK: APT OK: 64 packages available for upgrade (0 critical updates).
[07:15:35] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on reports171
[07:15:38] <icinga-miraheze>	 RECOVERY - ns1 APT on ns1 is OK: APT OK: 50 packages available for upgrade (0 critical updates).
[07:15:40] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:15:48] <icinga-miraheze>	 RECOVERY - mw182 APT on mw182 is OK: APT OK: 65 packages available for upgrade (0 critical updates).
[07:15:50] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on mw181
[07:15:55] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:16:05] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on cp51
[07:16:11] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:16:18] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on test151
[07:16:19] <icinga-miraheze>	 RECOVERY - mw181 APT on mw181 is OK: APT OK: 65 packages available for upgrade (0 critical updates).
[07:16:23] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:16:34] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftac171
[07:16:40] <icinga-miraheze>	 RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 62 packages available for upgrade (0 critical updates).
[07:16:42] <icinga-miraheze>	 RECOVERY - cp51 APT on cp51 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:16:44] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:16:48] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftobject151
[07:16:52] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:16:59] <icinga-miraheze>	 RECOVERY - reports171 APT on reports171 is OK: APT OK: 63 packages available for upgrade (0 critical updates).
[07:16:59] <icinga-miraheze>	 RECOVERY - test151 APT on test151 is OK: APT OK: 74 packages available for upgrade (0 critical updates).
[07:17:01] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftobject161
[07:17:07] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:17:44] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftproxy161
[07:17:48] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:17:52] <icinga-miraheze>	 RECOVERY - swiftobject151 APT on swiftobject151 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:17:57] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftobject171
[07:17:59] <icinga-miraheze>	 RECOVERY - swiftac171 APT on swiftac171 is OK: APT OK: 40 packages available for upgrade (0 critical updates).
[07:18:03] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:18:12] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftobject181
[07:18:18] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:18:26] <MirahezeLSBot>	 !log [void@puppet181] Upgraded packages libpq5 on swiftproxy171
[07:18:29] <icinga-miraheze>	 RECOVERY - swiftproxy171 APT on swiftproxy171 is OK: APT OK: 54 packages available for upgrade (0 critical updates).
[07:18:32] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[07:18:35] <icinga-miraheze>	 RECOVERY - swiftobject161 APT on swiftobject161 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:18:40] <icinga-miraheze>	 RECOVERY - swiftobject181 APT on swiftobject181 is OK: APT OK: 52 packages available for upgrade (0 critical updates).
[07:19:08] <icinga-miraheze>	 RECOVERY - swiftproxy161 APT on swiftproxy161 is OK: APT OK: 40 packages available for upgrade (0 critical updates).
[07:19:34] <icinga-miraheze>	 RECOVERY - swiftobject171 APT on swiftobject171 is OK: APT OK: 51 packages available for upgrade (0 critical updates).
[07:19:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.27, 20.68, 21.40
[07:21:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.30, 19.56, 20.93
[07:23:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.83, 21.38, 21.41
[07:31:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.79, 23.03, 22.51
[07:45:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.75, 19.01, 20.35
[07:49:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.20, 20.92, 20.82
[07:57:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.51, 18.86, 20.08
[08:04:56] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H <host> [-4|-6] [-w <warn>] [-c <crit>] [-v verbose] [-o <time offset>]
[08:08:59] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.0009034872055 secs
[08:13:30] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.07, 20.68, 19.98
[08:15:27] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.39, 20.34, 19.95
[08:21:14] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.71, 22.42, 20.93
[08:23:51] <botifico>	 [02mw-config] 07BlankEclair reviewed pull request 03#5633 commit - 13https://github.com/miraheze/mw-config/pull/5633#discussion_r1712604299
[08:23:52] <botifico>	 [02mw-config] 07BlankEclair reviewed pull request 03#5633 commit - 13https://github.com/miraheze/mw-config/pull/5633#discussion_r1712604416
[08:25:08] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.37, 22.79, 21.37
[08:27:05] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.56, 22.99, 21.63
[08:34:53] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.21, 22.25, 21.48
[08:36:50] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.69, 21.50, 21.32
[08:44:38] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.36, 23.03, 21.82
[08:45:55] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[08:48:31] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.29, 23.91, 22.46
[08:58:16] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 12.98, 17.14, 19.89
[09:14:20] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures
[09:17:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.42, 21.26, 19.71
[09:19:05] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 3.63, 4.18, 2.69
[09:19:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.43, 20.84, 19.77
[09:21:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.89, 18.94, 19.15
[09:23:02] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.16, 3.52, 2.76
[09:25:02] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 1.78, 3.00, 2.67
[09:25:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.38, 21.40, 20.17
[09:27:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.06, 18.76, 19.36
[09:28:21] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[09:28:59] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 8.23, 5.71, 3.84
[09:30:19] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.054 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[09:32:57] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.18, 3.66, 3.47
[09:37:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.62, 19.54, 19.19
[09:38:52] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.91, 3.74, 3.54
[09:39:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.00, 21.62, 19.95
[09:40:50] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.53, 3.84, 3.61
[09:42:48] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.16, 3.14, 3.38
[09:45:39] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[09:50:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 28.97, 23.41, 18.16
[09:52:35] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.27, 3.76, 3.55
[09:52:35] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 20.50, 18.77, 15.11
[09:54:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.51, 23.94, 19.61
[09:54:35] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 15.11, 17.00, 14.88
[09:58:29] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 3.20, 3.10, 3.33
[10:01:09] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[10:02:27] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 8.93, 6.30, 4.58
[10:03:08] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.060 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[10:08:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 25.70, 22.76, 20.80
[10:10:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.88, 22.89, 21.07
[10:12:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 24.02, 23.80, 21.66
[10:14:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 20.35, 22.00, 21.23
[10:16:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.15, 3.69, 4.00
[10:22:17] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.04, 2.61, 3.40
[10:26:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 26.22, 22.23, 21.47
[10:27:14] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: NXDOMAIN: The DNS query name does not exist: franchise.franchising.org.ua.
[10:28:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.81, 22.39, 21.62
[10:36:16] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 14.85, 17.70, 19.85
[10:37:30] <icinga-miraheze>	 [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[10:41:16] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - LetsEncrypt on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket
[10:43:58] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures
[10:45:58] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.41, 3.60, 3.25
[10:47:11] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[10:51:20] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[10:53:52] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.37, 3.77, 3.80
[10:57:30] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[10:59:46] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.00, 2.63, 3.30
[11:03:41] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.87, 4.39, 3.82
[11:04:37] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[11:06:39] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 3.427 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[11:07:20] <icinga-miraheze>	 [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[11:12:20] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[11:17:25] <icinga-miraheze>	 [Grafana] FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[11:17:38] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:27:25] <icinga-miraheze>	 [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1[Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[11:42:25] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[11:44:11] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures
[11:54:53] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - franchise.franchising.org.ua All nameservers failed to answer the query.
[12:06:33] <icinga-miraheze>	 PROBLEM - wiki.amplicity.fr - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket
[12:10:58] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.58, 3.03, 3.87
[12:12:16] <icinga-miraheze>	 PROBLEM - wiki.amplicity.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.amplicity.fr All nameservers failed to answer the query.
[12:12:56] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 7.50, 4.60, 4.33
[12:17:59] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[12:18:13] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[12:19:58] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[12:20:12] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.058 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[12:24:07] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for franchise.franchising.org.ua could not be found
[12:26:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 20.96, 19.79, 18.60
[12:28:16] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 18.99, 19.28, 18.55
[12:28:43] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.21, 3.00, 3.86
[12:30:53] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.07, 3.70, 3.97
[12:32:51] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.38, 3.01, 3.68
[12:34:49] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.07, 3.91, 3.94
[12:36:48] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.41, 3.33, 3.73
[12:37:30] <icinga-miraheze>	 [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[12:38:46] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.15, 4.37, 4.06
[12:40:45] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.27, 3.67, 3.85
[12:42:16] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.28, 19.70, 18.11
[12:42:44] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.86, 4.53, 4.14
[12:44:16] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.33, 19.68, 18.32
[12:48:38] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.33, 3.59, 3.90
[12:52:30] <icinga-miraheze>	 [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[12:54:34] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 1.61, 2.70, 3.39
[13:00:27] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.25, 4.11, 3.80
[13:00:34] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[13:02:22] <icinga-miraheze>	 [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[13:02:33] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.235 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[13:04:24] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.15, 3.13, 3.52
[13:06:22] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 0.29, 2.16, 3.11
[13:07:22] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[13:23:25] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[13:25:21] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset 2.533197403e-05 secs
[13:27:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.50, 22.45, 23.78
[13:37:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.34, 21.78, 22.26
[13:41:57] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H <host> [-4|-6] [-w <warn>] [-c <crit>] [-v verbose] [-o <time offset>]
[13:44:03] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[13:45:39] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[13:46:00] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset 7.635354996e-05 secs
[14:05:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.19, 21.24, 23.78
[14:15:41] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[14:17:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.90, 17.72, 20.38
[14:25:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.24, 21.38, 20.74
[14:29:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.60, 21.77, 21.12
[14:37:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.52, 18.94, 20.04
[14:39:55] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[14:41:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.61, 22.67, 21.37
[14:41:51] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.0003966987133 secs
[14:43:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 28.07, 24.69, 22.27
[14:49:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.24, 22.66, 22.28
[14:51:43] <icinga-miraheze>	 PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 59%
[14:53:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.62, 23.75, 22.70
[14:57:43] <icinga-miraheze>	 RECOVERY - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is OK: OK - NGINX Error Rate is 38%
[15:03:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 15.75, 21.61, 22.85
[15:06:17] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.14, 3.29, 1.42
[15:10:17] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.02, 3.70, 2.09
[15:11:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.95, 16.81, 19.95
[15:12:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.08, 5.11, 2.83
[15:17:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.85, 20.13, 20.29
[15:18:14] <botifico>	 [02puppet] 07redbluegreenhat pushed 031 commit to 03pywikibot-ua [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f91885c1e556...25957b32ff7c
[15:18:17] <botifico>	 [02puppet] 07redbluegreenhat 0325957b3 - use @wikitide.org address
[15:18:18] <botifico>	 [02puppet] 07redbluegreenhat synchronize pull request 03#3893: Pywikibot: Add user agent description - 13https://github.com/miraheze/puppet/pull/3893
[15:19:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.95, 18.88, 19.82
[15:19:52] <miraheze-github>	 miraheze/puppet - redbluegreenhat the build passed.
[15:20:18] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.14, 3.69, 3.08
[15:20:41] <botifico>	 [02puppet] 07redbluegreenhat closed pull request 03#3893: Pywikibot: Add user agent description - 13https://github.com/miraheze/puppet/pull/3893
[15:20:42] <botifico>	 [02puppet] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/4e879b564647...428f44da4aa1
[15:20:45] <botifico>	 [02puppet] 07redbluegreenhat 03428f44d - Pywikibot: Add user agent description (#3893)
[15:20:48] <botifico>	 [02puppet] 07redbluegreenhat deleted branch 03pywikibot-ua - 13https://github.com/miraheze/puppet
[15:20:51] <botifico>	 [02puppet] 07redbluegreenhat deleted branch 03pywikibot-ua
[15:22:08] <miraheze-github>	 miraheze/puppet - redbluegreenhat the build passed.
[15:22:16] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.23, 3.13, 2.95
[15:23:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.44, 21.01, 20.40
[15:24:02] <icinga-miraheze>	 PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 122.90 ms
[15:25:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.77, 19.82, 20.05
[15:26:06] <icinga-miraheze>	 RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 124.09 ms
[15:29:12] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.51, 3.96, 3.51
[15:33:08] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.29, 3.38, 3.40
[15:39:01] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.41, 3.75, 3.49
[15:39:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.15, 19.74, 19.18
[15:40:59] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.72, 3.35, 3.36
[15:41:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 20.01, 19.48, 19.14
[15:42:57] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 1.89, 3.14, 3.31
[15:43:22] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[15:45:18] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset 4.950165749e-05 secs
[15:45:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.40, 21.11, 19.90
[15:46:04] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[15:49:48] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.69, 3.97, 3.60
[15:51:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.32, 20.31, 20.07
[15:51:48] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.73, 3.24, 3.36
[16:03:28] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[16:03:36] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 9.93, 5.81, 4.27
[16:04:24] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[16:05:27] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.067 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[16:06:30] <icinga-miraheze>	 PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 117.61 ms
[16:08:33] <icinga-miraheze>	 RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 119.54 ms
[16:08:34] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[16:11:28] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.88, 3.47, 3.78
[16:13:31] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 7.00, 4.95, 4.30
[16:14:20] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures
[16:21:27] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.92, 3.38, 3.84
[16:23:26] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.18, 4.28, 4.12
[16:25:25] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.85, 3.50, 3.85
[16:29:21] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.42, 4.41, 4.11
[16:29:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.61, 18.90, 17.31
[16:30:56] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[16:31:22] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[16:31:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.22, 16.97, 16.78
[16:32:54] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.221 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[16:37:21] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[16:38:29] <icinga-miraheze>	 PROBLEM - prometheus151 Puppet on prometheus151 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[16:39:20] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.331 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[16:39:48] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[16:39:48] <icinga-miraheze>	 [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[16:40:29] <icinga-miraheze>	 RECOVERY - prometheus151 Puppet on prometheus151 is OK: OK: Puppet is currently enabled, last run 23 minutes ago with 0 failures
[16:44:48] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[16:52:34] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[16:54:30] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.001143574715 secs
[16:59:02] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.36, 2.94, 3.97
[17:04:58] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.00, 3.97, 3.99
[17:06:09] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[17:06:38] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[17:08:08] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.067 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[17:08:20] <icinga-miraheze>	 [Grafana] FIRING: There has been a rise in the MediaWiki exception rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[17:08:38] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[17:12:51] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.67, 3.23, 3.86
[17:13:20] <icinga-miraheze>	 [Grafana] RESOLVED: MediaWiki Exception Rate https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[17:16:48] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.06, 3.38, 3.72
[17:18:46] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.33, 3.09, 3.57
[17:20:45] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.17, 2.70, 3.36
[17:26:39] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.13, 3.59, 3.54
[17:28:37] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.10, 3.11, 3.37
[17:32:32] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.42, 3.73, 3.57
[17:34:31] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.19, 3.61, 3.52
[17:36:30] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 2.48, 3.04, 3.32
[17:46:10] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[17:48:09] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[17:48:24] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.81, 5.05, 3.87
[17:51:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.61, 19.83, 17.18
[17:52:17] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 1.065 second response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[17:53:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.01, 20.47, 17.74
[17:57:45] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.27, 19.07, 17.82
[17:58:42] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[18:00:45] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 3.984 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[18:06:36] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.43, 19.96, 18.73
[18:10:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.76, 3.29, 3.81
[18:10:30] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.47, 22.30, 19.86
[18:12:27] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.01, 22.63, 20.31
[18:14:24] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.73, 23.44, 20.88
[18:16:16] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 1.87, 2.74, 3.40
[18:16:21] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.31, 23.12, 21.10
[18:16:23] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[18:22:11] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.78, 23.60, 21.79
[18:24:18] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 3.82, 3.68, 3.49
[18:26:17] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.75, 3.72, 3.50
[18:28:29] <icinga-miraheze>	 PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.73, 20.28, 17.69
[18:28:39] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[18:30:26] <icinga-miraheze>	 RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.40, 20.30, 18.05
[18:30:37] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.063 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[18:35:03] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[18:35:04] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[18:35:10] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[18:37:02] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[18:37:02] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.061 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[18:37:06] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.000554472208 secs
[18:42:17] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.40, 3.45, 3.99
[18:44:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.18, 4.38, 4.28
[18:45:26] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[18:45:28] <icinga-miraheze>	 PROBLEM - prometheus151 SSH on prometheus151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[18:47:33] <icinga-miraheze>	 RECOVERY - prometheus151 SSH on prometheus151 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u3 (protocol 2.0)
[18:49:34] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 0.192 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[19:02:16] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.02, 3.09, 3.86
[19:03:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 18.88, 22.51, 23.89
[19:04:16] <icinga-miraheze>	 RECOVERY - prometheus151 Current Load on prometheus151 is OK: LOAD OK - total load average: 0.35, 2.12, 3.40
[19:05:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.84, 23.05, 23.87
[19:16:04] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[19:18:43] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - franchise.franchising.org.ua All nameservers failed to answer the query.
[19:47:38] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[20:17:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.20, 21.68, 23.77
[20:19:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.78, 23.36, 24.13
[20:21:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.77, 22.79, 23.83
[20:25:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 29.67, 25.82, 24.80
[20:34:32] <icinga-miraheze>	 RECOVERY - wiki.amplicity.fr - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.amplicity.fr' will expire on Sun 22 Sep 2024 04:39:26 PM GMT +0000.
[20:39:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.00, 22.62, 23.80
[20:41:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.51, 23.62, 24.01
[20:48:43] <icinga-miraheze>	 PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for franchise.franchising.org.ua could not be found
[21:16:38] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[21:40:16] <icinga-miraheze>	 RECOVERY - wiki.amplicity.fr - reverse DNS on sslhost is OK: SSL OK - wiki.amplicity.fr reverse DNS resolves to cp36.wikitide.net - CNAME OK
[21:44:17] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[21:53:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.46, 22.15, 23.95
[21:55:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 23.97, 23.39, 24.20
[21:59:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.44, 22.47, 23.68
[22:01:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.45, 23.73, 24.00
[22:46:01] <icinga-miraheze>	 PROBLEM - phorge171 php-fpm on phorge171 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm8.2'
[22:46:06] <icinga-miraheze>	 PROBLEM - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 502 Bad Gateway
[22:46:36] <icinga-miraheze>	 PROBLEM - phorge171 issue-tracker.miraheze.org HTTPS on phorge171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.014 second response time
[23:08:36] <icinga-miraheze>	 RECOVERY - phorge171 issue-tracker.miraheze.org HTTPS on phorge171 is OK: HTTP OK: HTTP/1.1 200 OK - 19644 bytes in 0.065 second response time
[23:10:01] <icinga-miraheze>	 RECOVERY - phorge171 php-fpm on phorge171 is OK: PROCS OK: 9 processes with command name 'php-fpm8.2'
[23:10:06] <icinga-miraheze>	 RECOVERY - phorge171 phorge-static.wikitide.net HTTPS on phorge171 is OK: HTTP OK: Status line output matched "HTTP/1.1 200" - 17718 bytes in 0.041 second response time
[23:25:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.13, 22.46, 23.76
[23:25:53] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.10, 3.99, 2.68
[23:35:44] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.84, 3.93, 3.38
[23:37:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 29.06, 23.90, 23.35
[23:39:40] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 6.32, 4.53, 3.71
[23:43:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.32, 22.22, 23.02
[23:45:35] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 1.73, 3.68, 3.71
[23:45:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.24, 23.25, 23.28
[23:47:33] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 5.43, 4.39, 3.96
[23:49:31] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is WARNING: LOAD WARNING - total load average: 2.30, 3.95, 3.88
[23:49:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.98, 23.86, 23.53
[23:53:28] <icinga-miraheze>	 PROBLEM - prometheus151 Current Load on prometheus151 is CRITICAL: LOAD CRITICAL - total load average: 4.82, 3.83, 3.81
[23:55:10] <icinga-miraheze>	 PROBLEM - prometheus151 PowerDNS Recursor on prometheus151 is CRITICAL: CRITICAL - Plugin timed out while executing system call
[23:55:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.83, 22.93, 23.03
[23:57:11] <icinga-miraheze>	 RECOVERY - prometheus151 PowerDNS Recursor on prometheus151 is OK: DNS OK: 2.015 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206
[23:59:45] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.34, 23.12, 23.18