[00:56:19] <wikitidebots>	 test
[01:06:45] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/19bc2ff03268...e0f1699c9cae
[01:06:48] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03e0f1699 - No IPv6
[01:08:14] <wikitide-github>	 WikiTideOrg/puppet - Universal-Omega the build passed.
[01:08:41] <icinga-wikitide>	 test
[03:43:30] <CosmicAlpha>	 !log added IRC_WIKITIDEBOTS secret to GitHub org
[03:43:30] <WikiTideBot>	 CosmicAlpha: Error: "log" is not a valid command.
[03:43:34] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[03:44:00] <CosmicAlpha>	 @Agent ^ seems a conflict
[03:46:29] <WikiTideRelay>	 <cosmicalpha> @agentisai ^
[03:52:02] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/e0f1699c9cae...41221808336d
[03:52:05] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 034122180 - Update permissions
[03:55:22] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/41221808336d...8df6922e6eb0
[03:55:23] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 038df6922 - Set max_files to 1500
[03:58:49] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/8df6922e6eb0...2a0ecc81a6e9
[03:58:50] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 032a0ecc8 - Use ensure =< installed
[04:16:42] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/2a0ecc81a6e9...38d998a1c3b9
[04:16:43] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 0338d998a - Use exec directly to avoid running every time
[04:22:48] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/38d998a1c3b9...aa4990559981
[04:22:51] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03aa49905 - matomo: use systemd::timer::job
[04:23:45] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/aa4990559981...be73236ddcb2
[04:23:47] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03be73236 - Fix
[04:24:18] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/WikiTideOrg/puppet/compare/be73236ddcb2...01ccc3de5b6d
[04:24:19] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 0301ccc3d - Remove runMatomoArchive
[05:22:48] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/01ccc3de5b6d...0a3fd0e122c8
[05:22:51] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 030a3fd0e - Add icinga-wikitide to relay ignore
[06:13:20] <icinga-wikitide>	 PROBLEM - mw2 NTP time on mw2 is WARNING: NTP WARNING: Offset 0.2620785534 secs
[06:15:20] <icinga-wikitide>	 RECOVERY - mw2 NTP time on mw2 is OK: NTP OK: Offset -0.01100856066 secs
[06:28:07] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/0a3fd0e122c8...fbbea9d3a7a9
[06:28:08] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03fbbea9d - Remove ignore for now
[06:53:51] <icinga-wikitide>	 RECOVERY - swiftac1 APT on swiftac1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[06:55:43] <icinga-wikitide>	 RECOVERY - mem1 APT on mem1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[06:57:35] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/fbbea9d3a7a9...165382310ed1
[06:57:36] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 031653823 - Use bots1 for logsalmsg
[06:59:04] <WikiTideLSBot>	 !log [universalomega@puppet1] test
[06:59:05] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[06:59:07] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[06:59:18] <icinga-wikitide>	 RECOVERY - jobrunner1 APT on jobrunner1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[06:59:28] <icinga-wikitide>	 RECOVERY - mw1 APT on mw1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:00:48] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on graylog1
[07:00:49] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:00:51] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:01:07] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on ldap1
[07:01:07] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:01:10] <icinga-wikitide>	 RECOVERY - mw2 APT on mw2 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:01:10] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:01:22] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on mail1
[07:01:22] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:01:23] <icinga-wikitide>	 RECOVERY - ldap1 APT on ldap1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:01:25] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:01:39] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libswscale6, libpostproc56, libswresample4, libavfilter8, libavformat59, libavcodec59, libavutil57, and libpq5 on cloud1
[07:01:40] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:01:42] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:01:58] <icinga-wikitide>	 RECOVERY - mail1 APT on mail1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:02:00] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on prometheus1
[07:02:00] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:02:03] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:02:08] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages Unmet on mon1
[07:02:08] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:02:11] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:02:14] <icinga-wikitide>	 RECOVERY - cloud1 APT on cloud1 is OK: APT OK: 24 packages available for upgrade (0 critical updates).
[07:02:15] <icinga-wikitide>	 RECOVERY - graylog1 APT on graylog1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:02:21] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on phorge1
[07:02:21] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:02:21] <icinga-wikitide>	 RECOVERY - os1 APT on os1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:02:24] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:02:31] <icinga-wikitide>	 RECOVERY - prometheus1 APT on prometheus1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:02:47] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libssl3, openjdk-17-jdk, openjdk-17-jdk-headless, openjdk-17-jre, openjdk-17-jre-headless, and openssl on puppet1
[07:02:47] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:02:51] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:03:10] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on cp2
[07:03:10] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:03:13] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:03:19] <icinga-wikitide>	 RECOVERY - phorge1 APT on phorge1 is OK: APT OK: 20 packages available for upgrade (0 critical updates).
[07:03:27] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on ns1
[07:03:27] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:03:30] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:03:40] <icinga-wikitide>	 RECOVERY - cp2 APT on cp2 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:03:58] <icinga-wikitide>	 RECOVERY - puppet1 APT on puppet1 is OK: APT OK: 12 packages available for upgrade (0 critical updates).
[07:03:59] <icinga-wikitide>	 RECOVERY - ns1 APT on ns1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:03:59] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages chromium, chromium-sandbox, chromium-common, and libpq5 on services1
[07:04:00] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:04:02] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:04:17] <icinga-wikitide>	 RECOVERY - services1 APT on services1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:04:19] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on cp4
[07:04:19] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:04:22] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:04:34] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on bast1
[07:04:35] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:04:37] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:04:55] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libswscale6, libavdevice59, libavformat59, libavfilter8, libavcodec59, libavutil57, libpostproc56, libswresample4, ffmpeg, and libpq5 on test1
[07:04:56] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:05:00] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:05:16] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on cp5
[07:05:16] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:05:19] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:05:33] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on swiftobject1
[07:05:33] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:05:33] <icinga-wikitide>	 RECOVERY - swiftobject1 APT on swiftobject1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:05:35] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:06:15] <icinga-wikitide>	 RECOVERY - cp4 APT on cp4 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:06:17] <icinga-wikitide>	 RECOVERY - bast1 APT on bast1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:06:17] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on cp6
[07:06:17] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:06:20] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:06:34] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on swiftproxy1
[07:06:34] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:06:37] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:06:39] <icinga-wikitide>	 RECOVERY - cp5 APT on cp5 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:06:46] <icinga-wikitide>	 RECOVERY - test1 APT on test1 is OK: APT OK: 33 packages available for upgrade (0 critical updates).
[07:06:57] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages libpq5 on cp3
[07:06:57] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:07:01] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:07:12] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages connect on cp1
[07:07:12] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:07:14] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:07:15] <icinga-wikitide>	 RECOVERY - cp6 APT on cp6 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:07:28] <icinga-wikitide>	 RECOVERY - cp3 APT on cp3 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:07:44] <icinga-wikitide>	 RECOVERY - swiftproxy1 APT on swiftproxy1 is OK: APT OK: 1 packages available for upgrade (0 critical updates).
[07:07:56] <icinga-wikitide>	 RECOVERY - cp1 APT on cp1 is OK: APT OK: 0 packages available for upgrade (0 critical updates).
[07:09:29] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages connect on mw1
[07:09:30] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:09:33] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[07:11:42] <WikiTideLSBot>	 !log [universalomega@puppet1] Upgraded packages connect on mw2
[07:11:42] <WikiTideBot>	 WikiTideLSBot: Error: "universalomega@puppet1" is not a valid command.
[07:11:45] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[11:31:13] <icinga-wikitide>	 RECOVERY - test1 JobChron Service on test1 is OK: PROCS OK: 1 process with args 'redisJobChronService'
[11:35:13] <icinga-wikitide>	 PROBLEM - test1 JobChron Service on test1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobChronService'
[13:51:01] <icinga-wikitide>	 PROBLEM - cp3 Disk Space on cp3 is CRITICAL: DISK CRITICAL - free space: / 1148MiB (5% inode=94%);
[16:40:47] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/174c6a5305aa...44b20feeac2d
[16:40:48] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 0344b20fe - Add challenge
[16:43:05] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/44b20feeac2d...ab82c1f67608
[16:43:06] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 03ab82c1f - Add challenge
[16:45:01] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/ab82c1f67608...d8c2ff008798
[16:45:03] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 03d8c2ff0 - Fix challenge value
[16:48:19] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/d8c2ff008798...19205651a512
[16:48:21] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 031920565 - Add challenge for cp6
[16:49:47] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/19205651a512...37b7e602c9f0
[16:49:49] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 0337b7e60 - Add cp4 challenge
[16:53:07] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/165382310ed1...224cf8038d5c
[16:53:09] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03224cf80 - Remove redundant map
[16:53:27] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/37b7e602c9f0...c4a8c70930ee
[16:53:30] <botifico>	 [02WikiTideOrg/dns] 07AgentIsai 03c4a8c70 - Add cp3 challenge
[16:55:17] <wikitide-github>	 WikiTideOrg/puppet - Universal-Omega the build passed.
[17:01:13] <icinga-wikitide>	 RECOVERY - test1 JobChron Service on test1 is OK: PROCS OK: 1 process with args 'redisJobChronService'
[17:05:13] <icinga-wikitide>	 PROBLEM - test1 JobChron Service on test1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobChronService'
[17:29:32] <icinga-wikitide>	 PROBLEM - wc.wikitide.org on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[17:31:39] <icinga-wikitide>	 PROBLEM - wc.wikitide.org on sslhost is WARNING: WARNING - Certificate 'wikitide.org' expires in 13 day(s) (Mon 04 Dec 2023 02:58:50 PM GMT +0000).
[17:37:58] <icinga-wikitide>	 PROBLEM - wc.wikitide.org on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[17:39:56] <icinga-wikitide>	 PROBLEM - wc.wikitide.org on sslhost is WARNING: WARNING - Certificate 'wikitide.org' expires in 13 day(s) (Mon 04 Dec 2023 02:58:50 PM GMT +0000).
[17:40:57] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±6] 13https://github.com/WikiTideOrg/puppet/compare/224cf8038d5c...b6f3343b399c
[17:41:00] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 03b6f3343 - irc: add monitoring for processes
[18:01:59] <icinga-wikitide>	 RECOVERY - test1 JobChron Service on test1 is OK: PROCS OK: 1 process with args 'redisJobChronService'
[18:03:56] <icinga-wikitide>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: Icinga configuration contains errors
[18:05:47] <icinga-wikitide>	 PROBLEM - test1 JobChron Service on test1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobChronService'
[18:22:35] <icinga-wikitide>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[18:23:11] <icinga-wikitide>	 PROBLEM - db1 MariaDB on db1 is CRITICAL: Access denied for user 'icinga'@'10.0.0.111' (using password: YES)
[18:23:27] <icinga-wikitide>	 PROBLEM - db1 MariaDB Connections on db1 is UNKNOWN: PHP Fatal error: Uncaught mysqli_sql_exception: Access denied for user 'icinga'@'10.0.0.111' (using password: YES) in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db1.wikitide.ne...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thrown i
[18:23:27] <icinga-wikitide>	 nagios/plugins/check_mysql_connections.php on line 66Fatal error: Uncaught mysqli_sql_exception: Access denied for user 'icinga'@'10.0.0.111' (using password: YES) in /usr/lib/nagios/plugins/check_mysql_connections.php:66Stack trace:#0 /usr/lib/nagios/plugins/check_mysql_connections.php(66): mysqli_real_connect(Object(mysqli), 'db1.wikitide.ne...', 'icinga', Object(SensitiveParameterValue), NULL, NULL, NULL, true)#1 {main} thrown in /u
[18:23:27] <icinga-wikitide>	 os/plugins/check_mysql_connections.php on line 66
[18:24:05] <icinga-wikitide>	 PROBLEM - db1 Backups SQL on db1 is WARNING: FILE_AGE WARNING: /var/log/backup-logs/sql-backup.log is 1001863 seconds old and 1980 bytes
[18:26:23] <icinga-wikitide>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: Icinga configuration contains errors
[19:09:33] <icinga-wikitide>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[19:22:24] <icinga-wikitide>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: Icinga configuration contains errors
[19:32:31] <icinga-wikitide>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[19:36:47] <icinga-wikitide>	 PROBLEM - mon1 Puppet on mon1 is UNKNOWN: NRPE: Unable to read output
[19:58:25] <icinga-wikitide>	 RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 29 minutes ago with 0 failures
[20:02:03] <icinga-wikitide>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: Icinga configuration contains errors
[20:05:21] <icinga-wikitide>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[20:07:45] <icinga-wikitide>	 PROBLEM - mon1 Puppet on mon1 is UNKNOWN: NRPE: Unable to read output
[20:09:58] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:09:59] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 0s
[20:10:00] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:10:01] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:10:04] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:10:07] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:11:07] <WikiTideLSBot>	 !log [universalomega@test1] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output.
[20:11:11] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:11:39] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'landing': True, 'errorpages': True, 'force': True, 'versions': ['1.40']} to [test1]
[20:11:40] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'landing': True, 'errorpages': True, 'force': True, 'versions': ['1.40']} to [test1] - SUCCESS in 0s
[20:11:42] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:11:54] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'config': True, 'force': True, 'versions': ['1.40']} to [test1]
[20:11:55] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'config': True, 'force': True, 'versions': ['1.40']} to [test1] - SUCCESS in 0s
[20:11:57] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:11:59] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:12:00] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:12:01] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 0s
[20:12:02] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:12:03] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:12:06] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:12:11] <icinga-wikitide>	 RECOVERY - test1 HTTPS on test1 is OK: HTTP OK: HTTP/2 200 - 359 bytes in 0.021 second response time
[20:12:12] <WikiTideLSBot>	 !log [universalomega@test1] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output.
[20:12:15] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:12:58] <icinga-wikitide>	 PROBLEM - test1 MediaWiki Rendering on test1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 2.035 second response time
[20:14:37] <WikiTideLSBot>	 !log [universalomega@test1] sudo -u www-data rsync --update -r --delete --exclude=.* /srv/mediawiki-staging/1.40/api.php /srv/mediawiki-staging/1.40/autoload.php /srv/mediawiki-staging/1.40/cache /srv/mediawiki-staging/1.40/CODE_OF_CONDUCT.md /srv/mediawiki-staging/1.40/composer.json /srv/mediawiki-staging/1.40/composer.local.json /srv/mediawiki-
[20:14:38] <WikiTideLSBot>	 staging/1.40/composer.local.json-sample /srv/mediawiki-staging/1.40/COPYING /srv/mediawiki-staging/1.40/CREDITS /srv/mediawiki-staging/1.40/DEVELOPERS.md /srv/mediawiki-staging/1.40/docker-compose.yml /srv/mediawiki-staging/1.40/docs /srv/mediawiki-staging/1.40/extensions /srv/mediawiki-staging/1.40/FAQ /srv/mediawiki-staging/1.40/Gruntfile.js /srv/mediawiki-
[20:14:39] <WikiTideLSBot>	 staging/1.40/HISTORY /srv/mediawiki-staging/1.40/images /srv/mediawiki-staging/1.40/img_auth.php /srv/mediawiki-staging/1.40/includes /srv/mediawiki-staging/1.40/index.php /srv/mediawiki-staging/1.40/INSTALL /srv/mediawiki-staging/1.40/jsduck.json /srv/mediawiki-staging/1.40/languages /srv/mediawiki-staging/1.40/load.php /srv/mediawiki-
[20:14:40] <WikiTideLSBot>	 staging/1.40/maintenance /srv/mediawiki-staging/1.40/mw-config /srv/mediawiki-staging/1.40/opensearch_desc.php /srv/mediawiki-staging/1.40/package.json /srv/mediawiki-staging/1.40/package-lock.json /srv/mediawiki-staging/1.40/phpunit.xml.dist /srv/mediawiki-staging/1.40/README.md /srv/mediawiki-staging/1.40/RELEASE-NOTES-1.40 /srv/mediawiki-
[20:14:40] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:14:41] <WikiTideLSBot>	 staging/1.40/resources /srv/mediawiki-staging/1.40/rest.php /srv/mediawiki-staging/1.40/SECURITY /srv/mediawiki-staging/1.40/skins /srv/mediawiki-staging/1.40/tests /srv/mediawiki-staging/1.40/thumb_handler.php /srv/mediawiki-staging/1.40/thumb.php /srv/mediawiki-staging/1.40/UPGRADE /srv/mediawiki-staging/1.40/vendor /srv/mediawiki/1.40/
[20:14:45] <WikiTideLSBot>	 !log [universalomega@test1] sudo -u www-data rsync --update -r --delete --exclude=.* /srv/mediawiki-staging/1.41/api.php /srv/mediawiki-staging/1.41/autoload.php /srv/mediawiki-staging/1.41/cache /srv/mediawiki-staging/1.41/CODE_OF_CONDUCT.md /srv/mediawiki-staging/1.41/composer.json /srv/mediawiki-staging/1.41/composer.local.json /srv/mediawiki-
[20:14:46] <WikiTideLSBot>	 staging/1.41/composer.local.json-sample /srv/mediawiki-staging/1.41/COPYING /srv/mediawiki-staging/1.41/CREDITS /srv/mediawiki-staging/1.41/DEVELOPERS.md /srv/mediawiki-staging/1.41/docker-compose.yml /srv/mediawiki-staging/1.41/docs /srv/mediawiki-staging/1.41/extensions /srv/mediawiki-staging/1.41/FAQ /srv/mediawiki-staging/1.41/Gruntfile.js /srv/mediawiki-
[20:14:47] <WikiTideLSBot>	 staging/1.41/HISTORY /srv/mediawiki-staging/1.41/images /srv/mediawiki-staging/1.41/img_auth.php /srv/mediawiki-staging/1.41/includes /srv/mediawiki-staging/1.41/index.php /srv/mediawiki-staging/1.41/INSTALL /srv/mediawiki-staging/1.41/jsduck.json /srv/mediawiki-staging/1.41/languages /srv/mediawiki-staging/1.41/load.php /srv/mediawiki-
[20:14:48] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:14:49] <WikiTideLSBot>	 staging/1.41/maintenance /srv/mediawiki-staging/1.41/mw-config /srv/mediawiki-staging/1.41/opensearch_desc.php /srv/mediawiki-staging/1.41/package.json /srv/mediawiki-staging/1.41/package-lock.json /srv/mediawiki-staging/1.41/phpunit.xml.dist /srv/mediawiki-staging/1.41/README.md /srv/mediawiki-staging/1.41/RELEASE-NOTES-1.41 /srv/mediawiki-
[20:14:49] <WikiTideLSBot>	 staging/1.41/RELEASE-NOTES-1.42 /srv/mediawiki-staging/1.41/resources /srv/mediawiki-staging/1.41/rest.php /srv/mediawiki-staging/1.41/SECURITY /srv/mediawiki-staging/1.41/skins /srv/mediawiki-staging/1.41/tests /srv/mediawiki-staging/1.41/thumb_handler.php /srv/mediawiki-staging/1.41/thumb.php /srv/mediawiki-staging/1.41/UPGRADE /srv/mediawiki-
[20:14:50] <WikiTideLSBot>	 staging/1.41/vendor /srv/mediawiki/1.41/
[20:14:58] <icinga-wikitide>	 PROBLEM - test1 MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 671 bytes in 0.603 second response time
[20:21:11] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/b6f3343b399c...4b2ffae80335
[20:21:14] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 034b2ffae - Use testwikitide
[20:22:56] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:22:57] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 0s
[20:22:58] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:22:59] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:02] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:05] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:13] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 16s
[20:23:14] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:23:16] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:18] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:35] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 38s
[20:23:36] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:23:38] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:23:42] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:24:02] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'world': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 65s
[20:24:06] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:24:45] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:24:46] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 0s
[20:24:47] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:24:50] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:24:53] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:26:32] <wikitide-github>	 WikiTideOrg/puppet - Universal-Omega the build passed.
[20:35:20] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 635s
[20:35:21] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:35:23] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:35:26] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:38:55] <icinga-wikitide>	 PROBLEM - test1 MediaWiki Rendering on test1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 2.647 second response time
[20:46:25] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 1300s
[20:46:26] <WikiTideLSBot>	 !log [universalomega@test1] starting deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1]
[20:46:28] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[20:58:08] <WikiTideLSBot>	 !log [universalomega@test1] finished deploy of {'l10n': True, 'force': True, 'versions': ['1.39', '1.40', '1.41']} to [test1] - SUCCESS in 2003s
[20:58:11] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[21:20:58] <WikiTideLSBot>	 !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/extensions/CreateWiki/maintenance/deleteWikis.php --wiki=metawikitide Agent Isai --delete (END - exit=0)
[21:21:01] <WikiTideLogbot>	 Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log
[22:56:24] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03phpldapadmin [+22/-0/±0] 13https://github.com/WikiTideOrg/puppet/commit/0b00b7a40702
[22:56:26] <botifico>	 [02WikiTideOrg/puppet] 07Universal-Omega 030b00b7a - Add phpldapadmin module
[22:56:27] <botifico>	 [02puppet] 07Universal-Omega created branch 03phpldapadmin - 13https://github.com/WikiTideOrg/puppet
[22:56:30] <botifico>	 [02puppet] 07Universal-Omega opened pull request 03#15: Add phpldapadmin module - 13https://github.com/WikiTideOrg/puppet/pull/15