[01:32:21] [02mw-config] 07Universal-Omega reviewed pull request 03#5643 commit - 13https://github.com/miraheze/mw-config/pull/5643#discussion_r1749439594 [03:07:20] [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [03:40:26] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 2 minutes ago with 5 failures. Failed resources (up to 3 shown): Package[nano],Package[postfix],Package[imagemagick],Package[nodejs] [03:46:26] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [03:57:31] [02puppet] 07AgentIsai pushed 031 commit to 03master [+10/-0/±1] 13https://github.com/miraheze/puppet/compare/6dde36814b73...b235f725e030 [03:57:32] [02puppet] 07AgentIsai 03b235f72 - T12563, T12564: Provision mwtask151/161, mw1x3 [04:17:45] [02mw-config] 07hurohukidaikon synchronize pull request 03#5643: Update UploadWizard settings for kagagawiki - 13https://github.com/miraheze/mw-config/pull/5643 [04:18:39] miraheze/mw-config - hurohukidaikon the build passed. [04:32:59] [02mw-config] 07Universal-Omega closed pull request 03#5643: Update UploadWizard settings for kagagawiki - 13https://github.com/miraheze/mw-config/pull/5643 [04:33:02] [02mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2bd200f6a181...3aa873fbb5dd [04:33:04] [02mw-config] 07hurohukidaikon 033aa873f - Update UploadWizard settings for kagagawiki (#5643) [04:33:55] miraheze/mw-config - Universal-Omega the build passed. [04:35:01] [02mw-config] 07Universal-Omega closed pull request 03#5653: T12556: Set some image thumbnail settings for dominionstrategywiki - 13https://github.com/miraheze/mw-config/pull/5653 [04:35:04] [02mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3aa873fbb5dd...f68babc74dcb [04:35:06] [02mw-config] 07BlankEclair 03f68babc - T12556: Set some image thumbnail settings for dominionstrategywiki (#5653) [04:35:53] miraheze/mw-config - Universal-Omega the build passed. [04:36:59] !log [@mwtask171] starting deploy of {'config': True} to all [04:37:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:37:10] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 11s [04:37:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:38:54] [02puppet] 07Universal-Omega synchronize pull request 03#3852: Rename shell user - 13https://github.com/miraheze/puppet/pull/3852 [04:38:55] [02puppet] 07Universal-Omega pushed 03119 commits to 03rename-shell [+519/-185/±193] 13https://github.com/miraheze/puppet/compare/34e86f9b6420...5f998fd2d670 [04:38:57] [02puppet] 07Universal-Omega 035f998fd - Merge branch 'master' into rename-shell [04:39:06] [02puppet] 07coderabbitai[bot] commented on pull request 03#3852: Rename shell user - 13https://github.com/miraheze/puppet/pull/3852#issuecomment-2337098504 [04:53:48] !log [@test151] starting deploy of {'config': True} to test151 [04:53:49] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [04:53:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:53:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:02:28] !log [@mwtask181] starting deploy of {'config': True} to all [05:02:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:02:41] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 13s [05:02:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:07:02] [02dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/cfb6613de57b...6d69726611f7 [05:07:05] [02dns] 07AgentIsai 036d69726 - T12563, T12564: Add mwtask1x1 and mw1*3/4 hostnames [05:18:15] PROBLEM - mw153 NTP time on mw153 is WARNING: Could not resolve hostname : Name or service not known [05:18:21] PROBLEM - Host mw153 is DOWN: check_ping: Invalid hostname/address - Usage:check_ping -H -w ,% -c ,% [-p packets] [-t timeout] [-4|-6] [05:18:22] PROBLEM - mw154 APT on mw154 is WARNING: Could not resolve hostname : Name or service not known [05:18:23] PROBLEM - mw154 MediaWiki Rendering on mw154 is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [05:18:29] PROBLEM - mw154 ferm_active on mw154 is WARNING: Could not resolve hostname : Name or service not known [05:18:30] PROBLEM - mw154 Current Load on mw154 is WARNING: Could not resolve hostname : Name or service not known [05:18:36] PROBLEM - mw154 Puppet on mw154 is WARNING: Could not resolve hostname : Name or service not known [05:18:41] PROBLEM - mw154 SSH on mw154 is UNKNOWN: Usage:check_ssh [-4|-6] [-t ] [-r ] [-p ] [05:18:43] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10003 milliseconds [05:18:57] PROBLEM - Host mw154 is DOWN: check_ping: Invalid hostname/address - Usage:check_ping -H -w ,% -c ,% [-p packets] [-t timeout] [-4|-6] [05:19:30] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:20:39] PROBLEM - mw163 ferm_active on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:42] PROBLEM - mw163 APT on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:43] PROBLEM - mw163 Current Load on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:46] PROBLEM - mw163 php-fpm on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:51] PROBLEM - mw163 NTP time on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:55] PROBLEM - mw163 PowerDNS Recursor on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:20:59] PROBLEM - mw163 Disk Space on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:21:04] PROBLEM - mw163 Puppet on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:21:05] PROBLEM - mw163 conntrack_table_size on mw163 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:22:34] PROBLEM - mw163 ferm_active on mw163 is UNKNOWN: NRPE: Unable to read output [05:22:38] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 0.20, 0.37, 0.25 [05:22:38] PROBLEM - mw163 APT on mw163 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [05:22:54] RECOVERY - mw163 PowerDNS Recursor on mw163 is OK: DNS OK: 0.053 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [05:22:55] RECOVERY - mw163 NTP time on mw163 is OK: NTP OK: Offset 0.01087486744 secs [05:22:58] RECOVERY - mw163 Disk Space on mw163 is OK: DISK OK - free space: / 49456MiB (95% inode=97%); [05:23:04] PROBLEM - mw163 Puppet on mw163 is UNKNOWN: NRPE: Unable to read output [05:23:05] PROBLEM - mw163 conntrack_table_size on mw163 is UNKNOWN: NRPE: Unable to read output [05:26:24] RECOVERY - mw163 ferm_active on mw163 is OK: OK ferm input default policy is set [05:27:03] RECOVERY - mw163 conntrack_table_size on mw163 is OK: OK: nf_conntrack is 0 % full [05:29:04] PROBLEM - mw163 Puppet on mw163 is CRITICAL: CRITICAL: Puppet has 94 failures. Last run 48 seconds ago with 94 failures. Failed resources (up to 3 shown): Package[mcrouter],Package[cgroup-tools],Package[imagemagick],Package[webp] [05:30:23] RECOVERY - mw163 APT on mw163 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:34:01] RECOVERY - mw163 php-fpm on mw163 is OK: PROCS OK: 3 processes with command name 'php-fpm8.2' [05:37:05] PROBLEM - mw184 php-fpm on mw184 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm8.2' [05:37:07] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: connect to address 2602:294:0:b23::115 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [05:37:08] PROBLEM - mw174 HTTPS on mw174 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to mw174.wikitide.net port 443 after 0 ms: Couldn't connect to server [05:37:08] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to mw173.wikitide.net port 443 after 0 ms: Couldn't connect to server [05:37:10] PROBLEM - mw173 APT on mw173 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [05:37:11] PROBLEM - mw174 ferm_active on mw174 is UNKNOWN: NRPE: Unable to read output [05:37:13] PROBLEM - mw173 conntrack_table_size on mw173 is UNKNOWN: NRPE: Unable to read output [05:37:14] PROBLEM - mw174 MediaWiki Rendering on mw174 is CRITICAL: connect to address 2602:294:0:b23::116 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [05:37:21] PROBLEM - mw174 Puppet on mw174 is UNKNOWN: NRPE: Unable to read output [05:37:23] PROBLEM - mw173 ferm_active on mw173 is UNKNOWN: NRPE: Unable to read output [05:37:27] PROBLEM - mw184 Puppet on mw184 is UNKNOWN: NRPE: Unable to read output [05:37:28] PROBLEM - mw164 Puppet on mw164 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 46 seconds ago with 3 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.42/skins/Femiwiki/node_modules],Service[cgroup],Service[nginx] [05:37:31] PROBLEM - mw173 Puppet on mw173 is UNKNOWN: NRPE: Unable to read output [05:37:32] PROBLEM - mw164 HTTPS on mw164 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to mw164.wikitide.net port 443 after 0 ms: Couldn't connect to server [05:37:34] PROBLEM - mw164 MediaWiki Rendering on mw164 is CRITICAL: connect to address 2602:294:0:b13::115 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [05:37:42] PROBLEM - mw184 HTTPS on mw184 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to mw184.wikitide.net port 443 after 0 ms: Couldn't connect to server [05:37:47] PROBLEM - mw184 ferm_active on mw184 is UNKNOWN: NRPE: Unable to read output [05:37:49] PROBLEM - mw173 php-fpm on mw173 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm8.2' [05:37:50] PROBLEM - mw184 MediaWiki Rendering on mw184 is CRITICAL: connect to address 2602:294:0:b12::114 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [05:37:59] PROBLEM - mw174 php-fpm on mw174 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'php-fpm8.2' [05:39:02] RECOVERY - mw184 php-fpm on mw184 is OK: PROCS OK: 3 processes with command name 'php-fpm8.2' [05:39:09] RECOVERY - mw174 ferm_active on mw174 is OK: OK ferm input default policy is set [05:39:42] RECOVERY - mw184 ferm_active on mw184 is OK: OK ferm input default policy is set [05:39:54] RECOVERY - mw174 php-fpm on mw174 is OK: PROCS OK: 3 processes with command name 'php-fpm8.2' [05:41:32] PROBLEM - mw174 Puppet on mw174 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.42/skins/Femiwiki/node_modules],Service[cgroup],Service[nginx] [05:41:55] PROBLEM - mw184 Puppet on mw184 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.42/skins/Femiwiki/node_modules],Service[cgroup],Service[nginx] [05:42:56] RECOVERY - mw173 conntrack_table_size on mw173 is OK: OK: nf_conntrack is 0 % full [05:43:19] RECOVERY - mw173 ferm_active on mw173 is OK: OK ferm input default policy is set [05:43:21] PROBLEM - mw173 Puppet on mw173 is CRITICAL: CRITICAL: Puppet has 95 failures. Last run 58 seconds ago with 95 failures. Failed resources (up to 3 shown): Package[mcrouter],Package[cgroup-tools],Package[imagemagick],Package[webp] [05:45:07] RECOVERY - mw173 APT on mw173 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:48:51] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10004 milliseconds [05:49:17] RECOVERY - mw173 php-fpm on mw173 is OK: PROCS OK: 3 processes with command name 'php-fpm8.2' [05:49:31] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:50:34] PROBLEM - mw183 ferm_active on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:50:39] PROBLEM - mw183 PowerDNS Recursor on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:50:41] PROBLEM - mw183 Current Load on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:50:45] PROBLEM - mw183 Disk Space on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:51:01] PROBLEM - mw183 conntrack_table_size on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:51:03] PROBLEM - mw183 Puppet on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:51:20] PROBLEM - mw183 php-fpm on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:51:20] PROBLEM - mw183 APT on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:51:23] PROBLEM - mw183 NTP time on mw183 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:55:30] RECOVERY - mw183 ferm_active on mw183 is OK: OK ferm input default policy is set [05:55:35] RECOVERY - mw183 PowerDNS Recursor on mw183 is OK: DNS OK: 0.035 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [05:55:39] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 0.12, 0.37, 0.42 [05:55:41] RECOVERY - mw183 Disk Space on mw183 is OK: DISK OK - free space: / 47064MiB (90% inode=96%); [05:55:51] RECOVERY - mw183 conntrack_table_size on mw183 is OK: OK: nf_conntrack is 0 % full [05:56:05] PROBLEM - mw183 Puppet on mw183 is UNKNOWN: NRPE: Unable to read output [05:56:14] RECOVERY - mw183 APT on mw183 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [05:56:16] RECOVERY - mw183 php-fpm on mw183 is OK: PROCS OK: 21 processes with command name 'php-fpm8.2' [05:56:19] RECOVERY - mw183 NTP time on mw183 is OK: NTP OK: Offset -2.813339233e-05 secs [05:56:54] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0e9073de25d9...9f5b27357e6b [05:56:56] [02ssl] 07AgentIsai 039f5b273 - Use miraheze-origin-cert for wiki.surf [05:57:55] [02WikiTideDebug] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiTideDebug/compare/e5e9fd52de23...ea0c47989249 [05:57:57] [02WikiTideDebug] 07Universal-Omega 03ea0c479 - Add new servers [06:02:21] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9f5b27357e6b...a7dbbbda6979 [06:02:24] [02ssl] 07AgentIsai 03a7dbbbd - miraheze.com: use origin cert [06:08:35] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a7dbbbda6979...d73b8a548a9e [06:08:38] [02ssl] 07AgentIsai 03d73b8a5 - thesimswiki: use origin cert [06:13:28] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d73b8a548a9e...c62046d3e931 [06:13:29] [02ssl] 07AgentIsai 03c62046d - Use origin cert for two more domains [06:14:22] miraheze/ssl - AgentIsai the build has errored. [06:18:03] PROBLEM - portalsofphereon.com - Cloudflare on sslhost is WARNING: WARNING - Certificate 'www.portalsofphereon.com' expires in 30 day(s) (Wed 09 Oct 2024 12:29:12 PM GMT +0000). [06:18:15] PROBLEM - thesimswiki.com - Cloudflare on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 18 day(s) (Fri 27 Sep 2024 11:58:22 PM GMT +0000). [06:18:21] PROBLEM - mw183 Puppet on mw183 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 50 seconds ago with 4 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.42/skins/Femiwiki/node_modules],Service[cgroup],Service[nginx],User[agent] [06:23:36] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c62046d3e931...540a6e9b2803 [06:23:37] [02ssl] 07AgentIsai 03540a6e9 - Move all redirects to use CF origin cert [06:24:37] miraheze/ssl - AgentIsai the build has errored. [06:26:02] [02ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/540a6e9b2803...0276ca9fe224 [06:26:05] [02ssl] 07AgentIsai 030276ca9 - Move orain.org redirects to CF [06:27:00] miraheze/ssl - AgentIsai the build has errored. [06:32:41] PROBLEM - mw163 MediaWiki Rendering on mw163 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:33:36] PROBLEM - mw174 MediaWiki Rendering on mw174 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:35:15] PROBLEM - mw183 MediaWiki Rendering on mw183 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:35:27] PROBLEM - mw173 MediaWiki Rendering on mw173 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:35:56] PROBLEM - mw164 MediaWiki Rendering on mw164 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:37:35] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:37:54] PROBLEM - mw184 MediaWiki Rendering on mw184 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:39:32] PROBLEM - mw173 MediaWiki Rendering on mw173 is UNKNOWN: HTTP UNKNOWN: Failed to unchunk message body [06:47:11] RECOVERY - portalsofphereon.com - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [06:47:36] PROBLEM - baharna.org - Cloudflare on sslhost is WARNING: WARNING - Certificate 'www.baharna.org' expires in 30 day(s) (Wed 09 Oct 2024 06:50:45 PM GMT +0000). [06:47:38] PROBLEM - rothwell-leeds.co.uk - Cloudflare on sslhost is WARNING: WARNING - Certificate 'www.rothwell-leeds.co.uk' expires in 30 day(s) (Wed 09 Oct 2024 11:31:31 PM GMT +0000). [06:47:57] RECOVERY - thesimswiki.com - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [06:48:15] PROBLEM - iceria.org - Cloudflare on sslhost is WARNING: WARNING - Certificate 'www.iceria.org' expires in 30 day(s) (Wed 09 Oct 2024 05:40:21 PM GMT +0000). [06:49:23] [02dns] 07Universal-Omega closed pull request 03#611: Delete zones/wiki.surf - 13https://github.com/miraheze/dns/pull/611 [06:49:26] [02dns] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/dns/compare/6d69726611f7...af0389ee3809 [06:49:28] [02dns] 07RhinosF1 03af0389e - Delete zones/wiki.surf (#611) [07:07:20] [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [07:16:20] RECOVERY - rothwell-leeds.co.uk - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [07:17:18] RECOVERY - iceria.org - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [07:17:35] RECOVERY - baharna.org - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [07:29:09] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/b235f725e030...b0f0d8571c18 [07:29:10] [02puppet] 07Universal-Omega 03b0f0d85 - Delete hieradata/hosts/cp26.yaml [07:29:18] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/b0f0d8571c18...6ef13f8bc94a [07:29:20] [02puppet] 07Universal-Omega 036ef13f8 - Delete hieradata/hosts/cp27.yaml [07:29:26] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/6ef13f8bc94a...31e2bed65ed4 [07:29:29] [02puppet] 07Universal-Omega 0331e2bed - Delete hieradata/hosts/cp28.yaml [07:29:34] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/31e2bed65ed4...9aaba4f17d66 [07:29:35] [02puppet] 07Universal-Omega 039aaba4f - Delete hieradata/hosts/cp41.yaml [07:29:44] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/9aaba4f17d66...c1cac7aaca67 [07:29:47] [02puppet] 07Universal-Omega 03c1cac7a - Delete hieradata/hosts/cp51.yaml [07:31:12] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c1cac7aaca67...c7905cbb9e8b [07:31:14] [02puppet] 07Universal-Omega 03c7905cb - Remove [07:31:54] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c7905cbb9e8b...7d029b48089c [07:31:57] [02puppet] 07Universal-Omega 037d029b4 - Remove [07:32:42] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7d029b48089c...c8cec5d81b8e [07:32:45] [02puppet] 07Universal-Omega 03c8cec5d - Remove [07:36:32] [02puppet] 07Universal-Omega pushed 031 commit to 03mwdeploy-pool-new-servers [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/50d4daea8cc4 [07:36:34] [02puppet] 07Universal-Omega 0350d4dae - mwdeploy: pool new servers [07:36:36] [02puppet] 07Universal-Omega created branch 03mwdeploy-pool-new-servers - 13https://github.com/miraheze/puppet [07:36:37] [02puppet] 07Universal-Omega opened pull request 03#3922: mwdeploy: pool new servers - 13https://github.com/miraheze/puppet/pull/3922 [07:36:44] [02puppet] 07coderabbitai[bot] commented on pull request 03#3922: mwdeploy: pool new servers - 13https://github.com/miraheze/puppet/pull/3922#issuecomment-2337353688 [07:38:03] [02puppet] 07Universal-Omega closed pull request 03#3922: mwdeploy: pool new servers - 13https://github.com/miraheze/puppet/pull/3922 [07:38:06] [02puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c8cec5d81b8e...107416c20e8a [07:38:07] [02puppet] 07Universal-Omega 03107416c - mwdeploy: pool new servers (#3922) [07:38:08] [02puppet] 07Universal-Omega deleted branch 03mwdeploy-pool-new-servers - 13https://github.com/miraheze/puppet [07:38:10] [02puppet] 07Universal-Omega deleted branch 03mwdeploy-pool-new-servers [07:38:14] miraheze/puppet - Universal-Omega the build passed. [07:39:16] miraheze/puppet - Universal-Omega the build passed. [09:31:16] PROBLEM - wiki.mahdiruiz.line.pm - Cloudflare on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [09:33:37] !log START DELETION (sudo -u www-data php /srv/mediawiki/1.42/extensions/CreateWiki/maintenance/deleteWikis.php --wiki=loginwiki --delete Reception123) [09:33:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:39:04] PROBLEM - sonicpedia.org - Cloudflare on sslhost is CRITICAL: No address associated with hostnameHTTP CRITICAL - Unable to open TCP socket [09:47:34] uhhh what [09:47:49] deleting loginwiki? or is it reception123wiki (if that's a thing) [10:00:44] RECOVERY - wiki.mahdiruiz.line.pm - Cloudflare on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 29 Nov 2024 07:59:27 PM GMT +0000. [10:01:17] I'm making the internal client cert ca right now [10:04:44] [02ssl] 07redbluegreenhat pushed 031 commit to 03master [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/0276ca9fe224...91b96b510add [10:04:44] [02ssl] 07redbluegreenhat 0391b96b5 - delete internal-client-certificate [10:05:41] miraheze/ssl - redbluegreenhat the build has errored. [10:05:50] how... [10:06:24] yes [10:06:36] [1/6] ``` [10:06:36] [2/6] ./redirects.yaml [10:06:37] [3/6] Error: 111:3 [key-duplicates] duplication of key "sslname" in mapping [10:06:37] [4/6] Error: 227:3 [key-duplicates] duplication of key "sslname" in mapping [10:06:37] [5/6] ``` [10:06:37] [6/6] not my fault at least [10:08:42] [02ssl] 07redbluegreenhat pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ssl/compare/91b96b510add...7d8c1f157779 [10:08:44] [02ssl] 07redbluegreenhat 037d8c1f1 - Add the Internal Client Certificate CA certificate [10:09:36] miraheze/ssl - redbluegreenhat the build has errored. [10:16:00] [02mw-config] 07BlankEclair opened pull request 03#5654: T12565: Add $wgEnableTranscode and .srt to ManageWiki - 13https://github.com/miraheze/mw-config/pull/5654 [10:16:57] miraheze/mw-config - BlankEclair the build passed. [10:33:04] [02ssl] 07redbluegreenhat pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ssl/compare/7d8c1f157779...7a51ef080c65 [10:33:07] [02ssl] 07redbluegreenhat 037a51ef0 - Add the mwdeploy client certificate [10:33:59] miraheze/ssl - redbluegreenhat the build has errored. [10:34:19] woah, miraheze-github logged in? [10:41:51] time to test this out [10:44:18] PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4|-6] [-w ] [-c ] [-v verbose] [-o