[02:31:24] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega pushed 032 commits to 03cleanup [+0/-0/±4] 13https://github.com/WikiTideOrg/WikiTideDebug/compare/12599a41b6ca...f770058dd190 [02:31:26] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega 03f770058 - Merge branch 'master' into cleanup [02:31:27] [02WikiTideDebug] 07Universal-Omega synchronize pull request 03#1: Cleanup code - 13https://github.com/WikiTideOrg/WikiTideDebug/pull/1 [02:32:41] [02WikiTideDebug] 07Universal-Omega synchronize pull request 03#1: Cleanup code - 13https://github.com/WikiTideOrg/WikiTideDebug/pull/1 [02:32:42] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/WikiTideOrg/WikiTideDebug/compare/f770058dd190...978693c55044 [02:32:43] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega 03978693c - Fix [02:33:11] [02WikiTideDebug] 07Universal-Omega closed pull request 03#1: Cleanup code - 13https://github.com/WikiTideOrg/WikiTideDebug/pull/1 [02:33:12] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/WikiTideOrg/WikiTideDebug/compare/fd153c131ed3...7b2584ea3534 [02:33:13] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega 037b2584e - Cleanup code [02:33:15] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega deleted branch 03cleanup [02:33:16] [02WikiTideDebug] 07Universal-Omega deleted branch 03cleanup - 13https://github.com/WikiTideOrg/WikiTideDebug [02:33:47] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/WikiTideDebug/compare/7b2584ea3534...c2313aa06943 [02:33:50] [02WikiTideOrg/WikiTideDebug] 07Universal-Omega 03c2313aa - Update servers [02:57:17] PROBLEM - cp1 PowerDNS Recursor on cp1 is CRITICAL: Domain 'wikitide.org' was not found by the server [02:59:20] RECOVERY - cp1 PowerDNS Recursor on cp1 is OK: DNS OK: 4.838 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66 [03:02:29] PROBLEM - mon1 Backups Grafana on mon1 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864147 seconds old and 347 bytes [05:56:54] [02WikiTideOrg/mw-config] 07AgentIsai pushed 031 commit to 03main [+0/-0/±1] 13https://github.com/WikiTideOrg/mw-config/compare/be6935e1203a...192a8fb15733 [05:56:56] [02WikiTideOrg/mw-config] 07AgentIsai 03192a8fb - Disable RequestWiki [05:57:41] [02WikiTideOrg/mw-config] 07AgentIsai pushed 031 commit to 03main [+0/-0/±1] 13https://github.com/WikiTideOrg/mw-config/compare/192a8fb15733...d2af935b7ac1 [05:57:44] [02WikiTideOrg/mw-config] 07AgentIsai 03d2af935 - Remove requestwiki right from user [05:57:49] WikiTideOrg/mw-config - AgentIsai the build passed. [05:58:38] WikiTideOrg/mw-config - AgentIsai the build passed. [06:28:28] !log [agent@jobrunner1] starting deploy of {'pull': 'config', 'config': True, 'force': True, 'versions': ['1.40']} to [mw1, mw2, jobrunner1] [06:28:32] !log [agent@jobrunner1] finished deploy of {'pull': 'config', 'config': True, 'force': True, 'versions': ['1.40']} to [mw1, mw2, jobrunner1] - SUCCESS in 4s [06:28:35] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [06:28:42] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [14:44:44] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [15:14:43] PROBLEM - newcascadia.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - newcascadia.net All nameservers failed to answer the query. [22:20:07] PROBLEM - puppet1 NTP time on puppet1 is WARNING: NTP WARNING: Offset 0.1015821099 secs [22:34:07] RECOVERY - puppet1 NTP time on puppet1 is OK: NTP OK: Offset 0.06783810258 secs [22:44:07] PROBLEM - puppet1 NTP time on puppet1 is WARNING: NTP WARNING: Offset -0.1231761873 secs [22:52:07] RECOVERY - puppet1 NTP time on puppet1 is OK: NTP OK: Offset 0.00308072567 secs [23:10:07] PROBLEM - puppet1 NTP time on puppet1 is WARNING: NTP WARNING: Offset 0.1377582252 secs [23:16:07] RECOVERY - puppet1 NTP time on puppet1 is OK: NTP OK: Offset -0.05523210764 secs [23:38:19] PROBLEM - puppet1 NTP time on puppet1 is WARNING: NTP WARNING: Offset 0.1264110506 secs [23:52:07] RECOVERY - puppet1 NTP time on puppet1 is OK: NTP OK: Offset -0.03796190023 secs