[00:02:25] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 19833MiB (22% inode=98%); [02:30:47] PROBLEM - cp37 Current Load on cp37 is CRITICAL: LOAD CRITICAL - total load average: 10.11, 6.57, 2.91 [02:30:47] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 38.46.223.205/cpweb, 38.46.223.206/cpweb, 2602:294:0:b13::110/cpweb, 2602:294:0:b23::112/cpweb [02:31:13] PROBLEM - cp36 Nginx Backend for mw171 on cp36 is CRITICAL: connect to address localhost and port 8117: Connection refused [02:31:14] PROBLEM - cp36 Nginx Backend for mw153 on cp36 is CRITICAL: connect to address localhost and port 8121: Connection refused [02:31:14] PROBLEM - cp36 Nginx Backend for mw161 on cp36 is CRITICAL: connect to address localhost and port 8115: Connection refused [02:31:20] PROBLEM - cp36 Nginx Backend for phorge171 on cp36 is CRITICAL: connect to address localhost and port 8202: Connection refused [02:31:21] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 17 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw154 mw163 mw164 mw173 mw174 mw183 mw184 mediawiki [02:31:23] PROBLEM - cp37 HTTPS on cp37 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10001 milliseconds [02:31:30] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 38.46.223.205/cpweb, 38.46.223.206/cpweb, 2602:294:0:b13::110/cpweb, 2602:294:0:b23::112/cpweb [02:31:34] PROBLEM - cp36 Nginx Backend for test151 on cp36 is CRITICAL: connect to address localhost and port 8181: Connection refused [02:31:38] PROBLEM - cp36 HTTPS on cp36 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to cp36.wikitide.net port 443 after 0 ms: Couldn't connect to server [02:31:41] PROBLEM - cp36 Nginx Backend for mw173 on cp36 is CRITICAL: connect to address localhost and port 8125: Connection refused [02:31:44] PROBLEM - cp36 Nginx Backend for puppet181 on cp36 is CRITICAL: connect to address localhost and port 8204: Connection refused [02:31:52] PROBLEM - cp36 Nginx Backend for matomo151 on cp36 is CRITICAL: connect to address localhost and port 8203: Connection refused [02:31:54] PROBLEM - cp36 Nginx Backend for mw164 on cp36 is CRITICAL: connect to address localhost and port 8124: Connection refused [02:31:54] PROBLEM - cp36 Nginx Backend for mw154 on cp36 is CRITICAL: connect to address localhost and port 8122: Connection refused [02:31:55] PROBLEM - cp36 Nginx Backend for mwtask171 on cp36 is CRITICAL: connect to address localhost and port 8161: Connection refused [02:31:59] PROBLEM - cp36 Nginx Backend for mw174 on cp36 is CRITICAL: connect to address localhost and port 8126: Connection refused [02:32:05] PROBLEM - cp36 Nginx Backend for mw182 on cp36 is CRITICAL: connect to address localhost and port 8120: Connection refused [02:32:05] PROBLEM - cp36 Nginx Backend for mw151 on cp36 is CRITICAL: connect to address localhost and port 8113: Connection refused [02:32:08] PROBLEM - cp36 Nginx Backend for swiftproxy171 on cp36 is CRITICAL: connect to address localhost and port 8207: Connection refused [02:32:19] PROBLEM - cp36 Nginx Backend for mw172 on cp36 is CRITICAL: connect to address localhost and port 8118: Connection refused [02:32:20] PROBLEM - cp36 Nginx Backend for mwtask151 on cp36 is CRITICAL: connect to address localhost and port 8162: Connection refused [02:32:23] PROBLEM - cp36 Nginx Backend for mwtask181 on cp36 is CRITICAL: connect to address localhost and port 8160: Connection refused [02:32:26] PROBLEM - cp36 Nginx Backend for mon181 on cp36 is CRITICAL: connect to address localhost and port 8201: Connection refused [02:32:26] PROBLEM - cp36 Nginx Backend for mw183 on cp36 is CRITICAL: connect to address localhost and port 8127: Connection refused [02:32:30] PROBLEM - cp36 Nginx Backend for reports171 on cp36 is CRITICAL: connect to address localhost and port 8205: Connection refused [02:32:31] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [02:32:34] PROBLEM - cp36 Nginx Backend for mw163 on cp36 is CRITICAL: connect to address localhost and port 8123: Connection refused [02:32:35] PROBLEM - puzzles.wiki - Cloudflare on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:35] PROBLEM - enc.for.uz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:35] PROBLEM - otherkin.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:37] PROBLEM - cp36 Nginx Backend for mw181 on cp36 is CRITICAL: connect to address localhost and port 8119: Connection refused [02:32:37] PROBLEM - cp36 Nginx Backend for mw162 on cp36 is CRITICAL: connect to address localhost and port 8116: Connection refused [02:32:38] PROBLEM - cp36 Nginx Backend for mwtask161 on cp36 is CRITICAL: connect to address localhost and port 8163: Connection refused [02:32:51] PROBLEM - cp37 Varnish Backends on cp37 is CRITICAL: 17 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw154 mw163 mw164 mw173 mw174 mw183 mw184 mediawiki [02:32:54] PROBLEM - cp36 Nginx Backend for mw152 on cp36 is CRITICAL: connect to address localhost and port 8114: Connection refused [02:32:55] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is WARNING: WARNING - NGINX Error Rate is 54% [02:32:58] PROBLEM - cp36 Nginx Backend for swiftproxy161 on cp36 is CRITICAL: connect to address localhost and port 8206: Connection refused [02:33:06] PROBLEM - cp36 Nginx Backend for mw184 on cp36 is CRITICAL: connect to address localhost and port 8128: Connection refused [02:33:30] PROBLEM - oxboxtra.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:33:55] PROBLEM - n64brew.dev - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:34:10] PROBLEM - podpedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:34:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 28% [02:34:44] PROBLEM - cp37 Current Load on cp37 is WARNING: LOAD WARNING - total load average: 7.81, 7.50, 4.21 [02:34:50] PROBLEM - smashbroswiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:06] PROBLEM - weavefarers.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:08] PROBLEM - segawiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:13] PROBLEM - lovebullet.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:14] PROBLEM - darkwaterswiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:35:19] PROBLEM - wonderfuleveryday.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:36:08] that no look good [02:36:10] PROBLEM - lotr.su - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:36:14] PROBLEM - caverncrusher.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:36:24] PROBLEM - poupedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:36:44] PROBLEM - cp37 Current Load on cp37 is CRITICAL: LOAD CRITICAL - total load average: 11.56, 8.55, 4.97 [02:36:56] PROBLEM - mh142.com - LetsEncrypt on sslhost is CRITICAL: connect to address mh142.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:37:22] PROBLEM - lostidols.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address lostidols.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:37:26] PROBLEM - holocron.net - LetsEncrypt on sslhost is CRITICAL: connect to address holocron.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:37:26] PROBLEM - royal-wiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address royal-wiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:37:26] PROBLEM - binrayarchives.com - LetsEncrypt on sslhost is CRITICAL: connect to address binrayarchives.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:37:34] PROBLEM - pyramidgames.wiki - Cloudflare on sslhost is CRITICAL: connect to address pyramidgames.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:09] PROBLEM - richterian.com - LetsEncrypt on sslhost is CRITICAL: connect to address richterian.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:11] PROBLEM - isocasg.org - LetsEncrypt on sslhost is CRITICAL: connect to address isocasg.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:19] PROBLEM - apeirology.com - LetsEncrypt on sslhost is CRITICAL: connect to address apeirology.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:22] PROBLEM - dariawiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address dariawiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:25] PROBLEM - worldsanskrit.net - LetsEncrypt on sslhost is CRITICAL: connect to address worldsanskrit.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:25] PROBLEM - cnidaria.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address cnidaria.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:34] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [02:38:44] RECOVERY - cp37 Current Load on cp37 is OK: LOAD OK - total load average: 1.63, 5.78, 4.39 [02:38:45] PROBLEM - allthetropes.org - LetsEncrypt on sslhost is CRITICAL: connect to address allthetropes.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:46] PROBLEM - mwcosmos.com - LetsEncrypt on sslhost is CRITICAL: connect to address mwcosmos.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:38:51] PROBLEM - cp37 Nginx Backend for mw152 on cp37 is CRITICAL: connect to address localhost and port 8114: Connection refused [02:38:55] PROBLEM - cp37 Nginx Backend for mw183 on cp37 is CRITICAL: connect to address localhost and port 8127: Connection refused [02:38:58] PROBLEM - cp37 Nginx Backend for reports171 on cp37 is CRITICAL: connect to address localhost and port 8205: Connection refused [02:38:59] PROBLEM - cp37 Nginx Backend for mw164 on cp37 is CRITICAL: connect to address localhost and port 8124: Connection refused [02:39:07] PROBLEM - cp37 Nginx Backend for mon181 on cp37 is CRITICAL: connect to address localhost and port 8201: Connection refused [02:39:08] PROBLEM - cp37 Nginx Backend for mw154 on cp37 is CRITICAL: connect to address localhost and port 8122: Connection refused [02:39:10] PROBLEM - cp37 Nginx Backend for mw174 on cp37 is CRITICAL: connect to address localhost and port 8126: Connection refused [02:39:11] PROBLEM - cp37 Nginx Backend for puppet181 on cp37 is CRITICAL: connect to address localhost and port 8204: Connection refused [02:39:11] PROBLEM - cp37 Nginx Backend for mw153 on cp37 is CRITICAL: connect to address localhost and port 8121: Connection refused [02:39:11] PROBLEM - cp37 Nginx Backend for swiftproxy161 on cp37 is CRITICAL: connect to address localhost and port 8206: Connection refused [02:39:16] PROBLEM - lotrmc.ru - LetsEncrypt on sslhost is CRITICAL: connect to address lotrmc.ru and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:17] PROBLEM - beyondtheveilrp.com - LetsEncrypt on sslhost is CRITICAL: connect to address beyondtheveilrp.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:21] PROBLEM - cp37 Nginx Backend for phorge171 on cp37 is CRITICAL: connect to address localhost and port 8202: Connection refused [02:39:23] PROBLEM - cp37 Nginx Backend for mw182 on cp37 is CRITICAL: connect to address localhost and port 8120: Connection refused [02:39:25] PROBLEM - vilexia.com - LetsEncrypt on sslhost is CRITICAL: connect to address vilexia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:31] PROBLEM - cp37 Nginx Backend for mw181 on cp37 is CRITICAL: connect to address localhost and port 8119: Connection refused PROBLEM - cp37 Nginx Backend for mw171 on cp37 is CRITICAL: connect to address localhost and port 8117: Connection refused [02:39:35] PROBLEM - spiceandwolf.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address spiceandwolf.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:36] PROBLEM - lads.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address lads.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:40] PROBLEM - runzeppelin.ru - LetsEncrypt on sslhost is CRITICAL: connect to address runzeppelin.ru and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:41] PROBLEM - cp37 Nginx Backend for mw151 on cp37 is CRITICAL: connect to address localhost and port 8113: Connection refused [02:39:43] PROBLEM - cp37 Nginx Backend for mw172 on cp37 is CRITICAL: connect to address localhost and port 8118: Connection refused [02:39:43] PROBLEM - cp37 Nginx Backend for mw184 on cp37 is CRITICAL: connect to address localhost and port 8128: Connection refused [02:39:43] PROBLEM - cp37 Nginx Backend for mwtask151 on cp37 is CRITICAL: connect to address localhost and port 8162: Connection refused [02:39:47] PROBLEM - cp37 Nginx Backend for matomo151 on cp37 is CRITICAL: connect to address localhost and port 8203: Connection refused [02:39:48] PROBLEM - cp37 Nginx Backend for mw161 on cp37 is CRITICAL: connect to address localhost and port 8115: Connection refused [02:39:48] PROBLEM - osdev.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address osdev.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:50] PROBLEM - cp37 Nginx Backend for swiftproxy171 on cp37 is CRITICAL: connect to address localhost and port 8207: Connection refused [02:39:52] PROBLEM - sdiy.info - LetsEncrypt on sslhost is CRITICAL: connect to address sdiy.info and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:53] PROBLEM - agesofconflict.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address agesofconflict.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:53] PROBLEM - rosettacode.org - LetsEncrypt on sslhost is CRITICAL: connect to address rosettacode.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:54] PROBLEM - edapedia.net - LetsEncrypt on sslhost is CRITICAL: connect to address edapedia.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:54] PROBLEM - bristolstudenthousingcoop.org - LetsEncrypt on sslhost is CRITICAL: connect to address bristolstudenthousingcoop.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:39:56] PROBLEM - cp37 Nginx Backend for mwtask181 on cp37 is CRITICAL: connect to address localhost and port 8160: Connection refused [02:39:58] PROBLEM - cp37 Nginx Backend for mwtask161 on cp37 is CRITICAL: connect to address localhost and port 8163: Connection refused [02:40:00] PROBLEM - cp37 Nginx Backend for mwtask171 on cp37 is CRITICAL: connect to address localhost and port 8161: Connection refused [02:40:13] PROBLEM - cp37 Nginx Backend for mw162 on cp37 is CRITICAL: connect to address localhost and port 8116: Connection refused [02:40:23] PROBLEM - creativecommons.id - LetsEncrypt on sslhost is CRITICAL: connect to address creativecommons.id and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:40:24] PROBLEM - beidipedia.com - LetsEncrypt on sslhost is CRITICAL: connect to address beidipedia.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:40:37] PROBLEM - cp37 Nginx Backend for test151 on cp37 is CRITICAL: connect to address localhost and port 8181: Connection refused [02:40:41] PROBLEM - cp37 Nginx Backend for mw163 on cp37 is CRITICAL: connect to address localhost and port 8123: Connection refused [02:40:45] PROBLEM - cp37 Nginx Backend for mw173 on cp37 is CRITICAL: connect to address localhost and port 8125: Connection refused [02:40:57] PROBLEM - dkwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address dkwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:41:09] PROBLEM - maxcapacity.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address maxcapacity.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:41:21] PROBLEM - antiguabarbudacalypso.com - LetsEncrypt on sslhost is CRITICAL: connect to address antiguabarbudacalypso.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:41:52] PROBLEM - mariopedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address mariopedia.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:28] PROBLEM - sptwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address sptwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:36] PROBLEM - exordium.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address exordium.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:48] PROBLEM - bobobay.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address bobobay.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:48] PROBLEM - wizardia.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wizardia.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:59] PROBLEM - permanentfuturelab.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address permanentfuturelab.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:42:59] PROBLEM - grayravens.com - LetsEncrypt on sslhost is CRITICAL: connect to address grayravens.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:43:23] PROBLEM - terraforming.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address terraforming.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:43:23] PROBLEM - farthestfrontier.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address farthestfrontier.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:43:28] PROBLEM - keitaiwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address keitaiwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:43:45] PROBLEM - yokaiwatchwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address yokaiwatchwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:43:50] PROBLEM - gielinorgames.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address gielinorgames.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:17] PROBLEM - thestarsareright.org - LetsEncrypt on sslhost is CRITICAL: connect to address thestarsareright.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:17] PROBLEM - aryavartpedia.online - LetsEncrypt on sslhost is CRITICAL: connect to address aryavartpedia.online and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:17] PROBLEM - thelonsdalebattalion.co.uk - LetsEncrypt on sslhost is CRITICAL: connect to address thelonsdalebattalion.co.uk and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:18] PROBLEM - electowiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address electowiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:24] PROBLEM - inourownwords.online - LetsEncrypt on sslhost is CRITICAL: connect to address inourownwords.online and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:24] PROBLEM - ncuwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address ncuwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:37] PROBLEM - journeytheword.wiki - Cloudflare on sslhost is CRITICAL: connect to address journeytheword.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:49] PROBLEM - christipedia.nl - LetsEncrypt on sslhost is CRITICAL: connect to address christipedia.nl and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:44:50] PROBLEM - crashspyrowiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address crashspyrowiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:45:33] PROBLEM - ourlifewiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address ourlifewiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:13] PROBLEM - kunwok.org - LetsEncrypt on sslhost is CRITICAL: connect to address kunwok.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:30] PROBLEM - androidwiki.info - LetsEncrypt on sslhost is CRITICAL: connect to address androidwiki.info and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:44] PROBLEM - vylet.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address vylet.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:46] PROBLEM - rodzinka.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address rodzinka.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:48] PROBLEM - comprehensibleinputwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address comprehensibleinputwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:46:51] PROBLEM - blackmagic.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address blackmagic.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:47:00] PROBLEM - ekumen.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address ekumen.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:47:11] PROBLEM - thegreatwar.uk - LetsEncrypt on sslhost is CRITICAL: connect to address thegreatwar.uk and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:47:13] PROBLEM - ff8.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address ff8.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:47:19] PROBLEM - landofliberos.com - LetsEncrypt on sslhost is CRITICAL: connect to address landofliberos.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:47:33] PROBLEM - roll-of-arms.com - LetsEncrypt on sslhost is CRITICAL: connect to address roll-of-arms.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:01] PROBLEM - squareenixwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address squareenixwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:04] PROBLEM - removededm.com - LetsEncrypt on sslhost is CRITICAL: connect to address removededm.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:17] PROBLEM - jackbox.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address jackbox.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:21] PROBLEM - acuralegendwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address acuralegendwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:43] PROBLEM - wikitide.net - LetsEncrypt on sslhost is CRITICAL: connect to address wikitide.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:44] PROBLEM - fischipedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address fischipedia.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:44] PROBLEM - dccomicswiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address dccomicswiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:51] PROBLEM - stablestate.org - LetsEncrypt on sslhost is CRITICAL: connect to address stablestate.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:48:51] PROBLEM - hoseki.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address hoseki.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:49:01] PROBLEM - gimkit.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address gimkit.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:49:13] RECOVERY - cp36 Nginx Backend for mw171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8117 [02:49:14] RECOVERY - cp36 Nginx Backend for mw153 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8121 [02:49:14] RECOVERY - cp36 Nginx Backend for mw161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8115 [02:49:20] RECOVERY - cp36 Nginx Backend for phorge171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8202 [02:49:34] RECOVERY - cp36 Nginx Backend for test151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8181 [02:49:41] RECOVERY - cp36 Nginx Backend for mw173 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8125 [02:49:44] RECOVERY - cp36 Nginx Backend for puppet181 on cp36 is OK: TCP OK - 0.001 second response time on localhost port 8204 [02:49:52] RECOVERY - cp36 Nginx Backend for matomo151 on cp36 is OK: TCP OK - 0.001 second response time on localhost port 8203 [02:49:54] RECOVERY - cp36 Nginx Backend for mw164 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8124 [02:49:55] RECOVERY - cp36 Nginx Backend for mw154 on cp36 is OK: TCP OK - 0.003 second response time on localhost port 8122 [02:49:55] RECOVERY - cp36 Nginx Backend for mwtask171 on cp36 is OK: TCP OK - 0.001 second response time on localhost port 8161 [02:49:59] RECOVERY - cp36 Nginx Backend for mw174 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8126 [02:50:05] RECOVERY - cp36 Nginx Backend for mw151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8113 [02:50:06] RECOVERY - cp36 Nginx Backend for mw182 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8120 [02:50:08] RECOVERY - cp36 Nginx Backend for swiftproxy171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8207 [02:50:19] RECOVERY - cp36 Nginx Backend for mw172 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8118 [02:50:20] RECOVERY - cp36 Nginx Backend for mwtask151 on cp36 is OK: TCP OK - 0.008 second response time on localhost port 8162 [02:50:23] RECOVERY - cp36 Nginx Backend for mwtask181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8160 [02:50:26] RECOVERY - cp36 Nginx Backend for mon181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8201 [02:50:26] RECOVERY - cp36 Nginx Backend for mw183 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8127 [02:50:30] RECOVERY - cp36 Nginx Backend for reports171 on cp36 is OK: TCP OK - 0.001 second response time on localhost port 8205 [02:50:34] RECOVERY - cp36 Nginx Backend for mw163 on cp36 is OK: TCP OK - 0.001 second response time on localhost port 8123 [02:50:37] RECOVERY - cp36 Nginx Backend for mw181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8119 [02:50:38] RECOVERY - cp36 Nginx Backend for mw162 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8116 [02:50:38] RECOVERY - cp36 Nginx Backend for mwtask161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8163 [02:50:56] RECOVERY - cp36 Nginx Backend for mw152 on cp36 is OK: TCP OK - 0.017 second response time on localhost port 8114 [02:51:03] RECOVERY - cp36 Nginx Backend for swiftproxy161 on cp36 is OK: TCP OK - 0.023 second response time on localhost port 8206 [02:51:05] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [02:51:06] RECOVERY - cp36 Nginx Backend for mw184 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8128 [02:51:21] PROBLEM - cp36 Varnish Backends on cp36 is WARNING: No backends detected. If this is an error, see readme.txt [02:53:06] PROBLEM - ssbuniverses.com - LetsEncrypt on sslhost is CRITICAL: connect to address ssbuniverses.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:53:08] PROBLEM - geeu.org - LetsEncrypt on sslhost is CRITICAL: connect to address geeu.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:53:19] PROBLEM - *.wikitide.org - LetsEncrypt on sslhost is CRITICAL: connect to address wikitide.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:53:21] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 17 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw154 mw163 mw164 mw173 mw174 mw183 mw184 mediawiki [02:54:13] PROBLEM - urbanshade.org - LetsEncrypt on sslhost is CRITICAL: connect to address urbanshade.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:19] PROBLEM - cp36 Nginx Backend for mw172 on cp36 is CRITICAL: connect to address localhost and port 8118: Connection refused [02:54:20] PROBLEM - cp36 Nginx Backend for mwtask151 on cp36 is CRITICAL: connect to address localhost and port 8162: Connection refused [02:54:23] PROBLEM - cp36 Nginx Backend for mwtask181 on cp36 is CRITICAL: connect to address localhost and port 8160: Connection refused [02:54:26] PROBLEM - rippaversewiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address rippaversewiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:26] PROBLEM - cp36 Nginx Backend for mon181 on cp36 is CRITICAL: connect to address localhost and port 8201: Connection refused [02:54:26] PROBLEM - cp36 Nginx Backend for mw183 on cp36 is CRITICAL: connect to address localhost and port 8127: Connection refused [02:54:27] PROBLEM - decimatedrive.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address decimatedrive.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:30] PROBLEM - cp36 Nginx Backend for reports171 on cp36 is CRITICAL: connect to address localhost and port 8205: Connection refused [02:54:31] PROBLEM - pokemonwiki.info - LetsEncrypt on sslhost is CRITICAL: connect to address pokemonwiki.info and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:31] PROBLEM - emmytherobot.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address emmytherobot.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:34] PROBLEM - cp36 Nginx Backend for mw163 on cp36 is CRITICAL: connect to address localhost and port 8123: Connection refused [02:54:37] PROBLEM - cp36 Nginx Backend for mw181 on cp36 is CRITICAL: connect to address localhost and port 8119: Connection refused [02:54:37] PROBLEM - cp36 Nginx Backend for mw162 on cp36 is CRITICAL: connect to address localhost and port 8116: Connection refused [02:54:38] PROBLEM - cp36 Nginx Backend for mwtask161 on cp36 is CRITICAL: connect to address localhost and port 8163: Connection refused [02:54:38] PROBLEM - wikimas.kr - LetsEncrypt on sslhost is CRITICAL: connect to address wikimas.kr and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:54] PROBLEM - cp36 Nginx Backend for mw152 on cp36 is CRITICAL: connect to address localhost and port 8114: Connection refused [02:54:55] PROBLEM - polcompball.net - LetsEncrypt on sslhost is CRITICAL: connect to address polcompball.net and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:54:58] PROBLEM - cp36 Nginx Backend for swiftproxy161 on cp36 is CRITICAL: connect to address localhost and port 8206: Connection refused [02:55:00] PROBLEM - wikappedia.cc - LetsEncrypt on sslhost is CRITICAL: connect to address wikappedia.cc and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:06] PROBLEM - cp36 Nginx Backend for mw184 on cp36 is CRITICAL: connect to address localhost and port 8128: Connection refused [02:55:06] PROBLEM - solaswiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address solaswiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:13] PROBLEM - cp36 Nginx Backend for mw171 on cp36 is CRITICAL: connect to address localhost and port 8117: Connection refused [02:55:14] PROBLEM - cp36 Nginx Backend for mw161 on cp36 is CRITICAL: connect to address localhost and port 8115: Connection refused [02:55:14] PROBLEM - cp36 Nginx Backend for mw153 on cp36 is CRITICAL: connect to address localhost and port 8121: Connection refused [02:55:19] PROBLEM - wikitide.org - LetsEncrypt on sslhost is CRITICAL: connect to address wikitide.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:20] PROBLEM - cp36 Nginx Backend for phorge171 on cp36 is CRITICAL: connect to address localhost and port 8202: Connection refused [02:55:20] PROBLEM - bonesword.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address bonesword.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:34] PROBLEM - cp36 Nginx Backend for test151 on cp36 is CRITICAL: connect to address localhost and port 8181: Connection refused [02:55:38] PROBLEM - hastursnotebook.org - LetsEncrypt on sslhost is CRITICAL: connect to address hastursnotebook.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:40] PROBLEM - pso2ngs.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address pso2ngs.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:55:41] PROBLEM - cp36 Nginx Backend for mw173 on cp36 is CRITICAL: connect to address localhost and port 8125: Connection refused [02:55:44] PROBLEM - cp36 Nginx Backend for puppet181 on cp36 is CRITICAL: connect to address localhost and port 8204: Connection refused [02:55:52] PROBLEM - cp36 Nginx Backend for matomo151 on cp36 is CRITICAL: connect to address localhost and port 8203: Connection refused [02:55:54] PROBLEM - cp36 Nginx Backend for mw164 on cp36 is CRITICAL: connect to address localhost and port 8124: Connection refused [02:55:54] PROBLEM - cp36 Nginx Backend for mw154 on cp36 is CRITICAL: connect to address localhost and port 8122: Connection refused [02:55:55] PROBLEM - cp36 Nginx Backend for mwtask171 on cp36 is CRITICAL: connect to address localhost and port 8161: Connection refused [02:55:59] PROBLEM - cp36 Nginx Backend for mw174 on cp36 is CRITICAL: connect to address localhost and port 8126: Connection refused [02:56:03] PROBLEM - monarchists.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address monarchists.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:03] PROBLEM - freedomplanetwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address freedomplanetwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:05] PROBLEM - cp36 Nginx Backend for mw151 on cp36 is CRITICAL: connect to address localhost and port 8113: Connection refused [02:56:05] PROBLEM - cp36 Nginx Backend for mw182 on cp36 is CRITICAL: connect to address localhost and port 8120: Connection refused [02:56:08] PROBLEM - cp36 Nginx Backend for swiftproxy171 on cp36 is CRITICAL: connect to address localhost and port 8207: Connection refused [02:56:09] PROBLEM - issue-tracker.wikitide.org - LetsEncrypt on sslhost is CRITICAL: connect to address issue-tracker.wikitide.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:10] PROBLEM - pizzatower.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address pizzatower.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:19] PROBLEM - wikigenius.org - LetsEncrypt on sslhost is CRITICAL: connect to address wikigenius.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:20] PROBLEM - icclopedia.org - LetsEncrypt on sslhost is CRITICAL: connect to address icclopedia.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:20] PROBLEM - speleo.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address speleo.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:56:27] PROBLEM - rainverse.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address rainverse.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:10] PROBLEM - wikimoma.art - LetsEncrypt on sslhost is CRITICAL: connect to address wikimoma.art and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:20] PROBLEM - junxions.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address junxions.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:38] PROBLEM - trollpasta.com - LetsEncrypt on sslhost is CRITICAL: connect to address trollpasta.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:38] PROBLEM - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is CRITICAL: connect to address encyclopediaofastrobiology.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:42] PROBLEM - civwiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address civwiki.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:48] PROBLEM - worldless.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address worldless.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:57:53] PROBLEM - fashiondreamer.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address fashiondreamer.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:58:20] PROBLEM - kingdomdeath.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address kingdomdeath.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:58:24] PROBLEM - cpmusicwiki.com - LetsEncrypt on sslhost is CRITICAL: connect to address cpmusicwiki.com and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:58:31] PROBLEM - vesc.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address vesc.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:08] PROBLEM - psycho.engineering - LetsEncrypt on sslhost is CRITICAL: connect to address psycho.engineering and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:08] PROBLEM - iol.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address iol.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:08] PROBLEM - warhammer40kturkiye.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address warhammer40kturkiye.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:08] PROBLEM - encyclopediarobotica.org - LetsEncrypt on sslhost is CRITICAL: connect to address encyclopediarobotica.org and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:08] PROBLEM - kingdomway.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address kingdomway.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:09] PROBLEM - burnout.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address burnout.wiki and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [02:59:11] RECOVERY - cp37 Nginx Backend for puppet181 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8204 [02:59:11] RECOVERY - cp37 Nginx Backend for mw153 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8121 [02:59:21] RECOVERY - cp37 Nginx Backend for phorge171 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8202 [02:59:23] RECOVERY - cp37 Nginx Backend for mw182 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8120 [02:59:31] RECOVERY - cp37 Nginx Backend for mw181 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8119 [02:59:31] RECOVERY - cp37 Nginx Backend for mw171 on cp37 is OK: TCP OK - 0.002 second response time on localhost port 8117 [02:59:38] PROBLEM - partopedia.ru - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:59:41] RECOVERY - cp37 Nginx Backend for mw151 on cp37 is OK: TCP OK - 0.003 second response time on localhost port 8113 [02:59:43] RECOVERY - cp37 Nginx Backend for mw172 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8118 [02:59:43] RECOVERY - cp37 Nginx Backend for mw184 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8128 [02:59:44] RECOVERY - cp37 Nginx Backend for mwtask151 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8162 [02:59:47] RECOVERY - cp37 Nginx Backend for mw161 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8115 [02:59:48] RECOVERY - cp37 Nginx Backend for matomo151 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8203 [02:59:50] RECOVERY - cp37 Nginx Backend for swiftproxy171 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8207 [02:59:56] RECOVERY - cp37 Nginx Backend for mwtask181 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8160 [03:00:03] RECOVERY - cp37 Nginx Backend for mwtask161 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8163 [03:00:03] RECOVERY - cp37 Nginx Backend for mwtask171 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8161 [03:00:03] PROBLEM - lostmediawiki.ru - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:13] RECOVERY - cp37 Nginx Backend for mw162 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:00:26] RECOVERY - cp37 Nginx Backend for swiftproxy161 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:00:31] RECOVERY - cp37 Nginx Backend for mw154 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8122 [03:00:35] RECOVERY - cp37 Nginx Backend for mw174 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8126 [03:00:37] RECOVERY - cp37 Nginx Backend for test151 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8181 [03:00:41] RECOVERY - cp37 Nginx Backend for mw163 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8123 [03:00:45] RECOVERY - cp37 Nginx Backend for mw173 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8125 [03:00:52] RECOVERY - cp37 Nginx Backend for mw152 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8114 [03:00:55] RECOVERY - cp37 Nginx Backend for mw183 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8127 [03:00:58] RECOVERY - cp37 Nginx Backend for reports171 on cp37 is OK: TCP OK - 0.002 second response time on localhost port 8205 [03:00:59] RECOVERY - cp37 Nginx Backend for mw164 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8124 [03:01:07] RECOVERY - cp37 Nginx Backend for mon181 on cp37 is OK: TCP OK - 0.000 second response time on localhost port 8201 [03:01:16] PROBLEM - sytbay.site - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:16] PROBLEM - fanonpedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:16] PROBLEM - worldtriggerwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:16] PROBLEM - opendatascot.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:17] PROBLEM - grayzonewarfare.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:26] RECOVERY - otherkin.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'otherkin.wiki' will expire on Wed 22 Jan 2025 02:41:57 PM GMT +0000. [03:01:28] RECOVERY - puzzles.wiki - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [03:01:28] RECOVERY - enc.for.uz - LetsEncrypt on sslhost is OK: OK - Certificate 'enc.for.uz' will expire on Fri 07 Mar 2025 06:34:10 AM GMT +0000. [03:02:01] [1/2] thats alot of problems... [03:02:01] [2/2] https://cdn.discordapp.com/attachments/808001911868489748/1318412860988063744/IMG_7602.png?ex=67623b28&is=6760e9a8&hm=3781f9f0190e9c5862c3cde372af48c48891a94a1be8c33165fd3d6c8520070e& [03:02:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 10% [03:02:51] RECOVERY - cp37 Varnish Backends on cp37 is OK: All 29 backends are healthy [03:03:02] RECOVERY - oxboxtra.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'oxboxtra.wiki' will expire on Wed 29 Jan 2025 06:47:21 AM GMT +0000. [03:03:03] RECOVERY - n64brew.dev - LetsEncrypt on sslhost is OK: OK - Certificate 'n64brew.dev' will expire on Fri 07 Mar 2025 06:15:58 PM GMT +0000. [03:03:03] RECOVERY - cp37 HTTPS on cp37 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4141 bytes in 0.056 second response time [03:03:44] RECOVERY - podpedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'podpedia.org' will expire on Fri 07 Mar 2025 09:06:45 AM GMT +0000. [03:03:57] RECOVERY - smashbroswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'smashbroswiki.com' will expire on Sat 08 Mar 2025 01:33:58 PM GMT +0000. [03:04:41] RECOVERY - weavefarers.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'weavefarers.wiki' will expire on Wed 19 Feb 2025 12:42:15 AM GMT +0000. [03:04:47] RECOVERY - lovebullet.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lovebullet.wiki' will expire on Sat 01 Mar 2025 11:30:48 PM GMT +0000. [03:05:02] RECOVERY - segawiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'segawiki.com' will expire on Sat 15 Mar 2025 06:21:40 PM GMT +0000. [03:05:02] RECOVERY - darkwaterswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'darkwaterswiki.com' will expire on Wed 08 Jan 2025 04:02:59 PM GMT +0000. [03:05:16] RECOVERY - wonderfuleveryday.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wonderfuleveryday.org' will expire on Sun 16 Feb 2025 01:46:05 AM GMT +0000. [03:05:36] RECOVERY - mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Fri 07 Mar 2025 07:39:12 AM GMT +0000. [03:05:50] RECOVERY - poupedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'poupedia.com' will expire on Wed 12 Feb 2025 02:03:07 PM GMT +0000. [03:05:51] RECOVERY - caverncrusher.com - LetsEncrypt on sslhost is OK: OK - Certificate 'caverncrusher.com' will expire on Sat 15 Mar 2025 02:06:28 PM GMT +0000. [03:05:54] RECOVERY - lotr.su - LetsEncrypt on sslhost is OK: OK - Certificate 'lotr.su' will expire on Wed 12 Mar 2025 04:28:42 PM GMT +0000. [03:06:27] RECOVERY - lostidols.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lostidols.wiki' will expire on Thu 02 Jan 2025 04:21:54 PM GMT +0000. [03:06:36] RECOVERY - binrayarchives.com - LetsEncrypt on sslhost is OK: OK - Certificate 'binrayarchives.com' will expire on Sun 09 Feb 2025 09:53:22 PM GMT +0000. [03:06:36] RECOVERY - royal-wiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'royal-wiki.org' will expire on Sun 09 Mar 2025 12:08:28 AM GMT +0000. [03:06:37] RECOVERY - holocron.net - LetsEncrypt on sslhost is OK: OK - Certificate 'holocron.net' will expire on Mon 24 Feb 2025 03:12:55 AM GMT +0000. [03:06:48] RECOVERY - isocasg.org - LetsEncrypt on sslhost is OK: OK - Certificate 'isocasg.org' will expire on Wed 29 Jan 2025 03:51:53 PM GMT +0000. [03:06:51] RECOVERY - pyramidgames.wiki - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [03:06:59] RECOVERY - apeirology.com - LetsEncrypt on sslhost is OK: OK - Certificate 'apeirology.com' will expire on Sun 09 Mar 2025 04:22:02 PM GMT +0000. [03:07:07] RECOVERY - dariawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dariawiki.org' will expire on Thu 02 Jan 2025 04:25:01 PM GMT +0000. [03:07:46] RECOVERY - richterian.com - LetsEncrypt on sslhost is OK: OK - Certificate 'richterian.com' will expire on Fri 07 Feb 2025 08:52:27 PM GMT +0000. [03:07:46] RECOVERY - cnidaria.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'cnidaria.wiki' will expire on Tue 18 Feb 2025 05:57:37 PM GMT +0000. [03:08:02] RECOVERY - mwcosmos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mwcosmos.com' will expire on Sun 02 Mar 2025 02:22:26 AM GMT +0000. [03:08:11] RECOVERY - lads.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lads.wiki' will expire on Thu 06 Mar 2025 04:34:20 PM GMT +0000. [03:08:19] RECOVERY - runzeppelin.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'runzeppelin.ru' will expire on Fri 07 Mar 2025 06:45:22 AM GMT +0000. [03:08:23] RECOVERY - worldsanskrit.net - LetsEncrypt on sslhost is OK: OK - Certificate 'worldsanskrit.net' will expire on Fri 14 Mar 2025 02:35:17 PM GMT +0000. [03:08:25] RECOVERY - allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Thu 06 Mar 2025 08:38:36 PM GMT +0000. [03:08:32] RECOVERY - lotrmc.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'lotrmc.ru' will expire on Sun 02 Feb 2025 04:49:27 PM GMT +0000. [03:08:34] RECOVERY - beyondtheveilrp.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beyondtheveilrp.com' will expire on Fri 21 Feb 2025 07:24:14 PM GMT +0000. [03:08:45] RECOVERY - rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Tue 21 Jan 2025 01:55:49 AM GMT +0000. [03:08:45] RECOVERY - agesofconflict.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'agesofconflict.wiki' will expire on Mon 10 Mar 2025 04:04:33 PM GMT +0000. [03:08:45] RECOVERY - sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Sun 02 Mar 2025 02:53:50 PM GMT +0000. [03:08:48] RECOVERY - edapedia.net - LetsEncrypt on sslhost is OK: OK - Certificate 'edapedia.net' will expire on Wed 22 Jan 2025 02:47:33 AM GMT +0000. [03:08:48] RECOVERY - bristolstudenthousingcoop.org - LetsEncrypt on sslhost is OK: OK - Certificate 'bristolstudenthousingcoop.org' will expire on Fri 07 Mar 2025 10:39:11 AM GMT +0000. [03:09:05] RECOVERY - vilexia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'vilexia.com' will expire on Sat 15 Feb 2025 05:40:59 PM GMT +0000. [03:09:21] RECOVERY - spiceandwolf.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'spiceandwolf.wiki' will expire on Wed 05 Feb 2025 03:24:43 AM GMT +0000. [03:09:32] RECOVERY - dkwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dkwiki.org' will expire on Sat 08 Mar 2025 03:26:18 PM GMT +0000. [03:09:46] RECOVERY - osdev.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'osdev.wiki' will expire on Fri 07 Feb 2025 09:54:48 PM GMT +0000. [03:09:48] RECOVERY - beidipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beidipedia.com' will expire on Wed 22 Jan 2025 02:50:31 PM GMT +0000. [03:09:56] RECOVERY - creativecommons.id - LetsEncrypt on sslhost is OK: OK - Certificate 'creativecommons.id' will expire on Sun 09 Mar 2025 06:07:09 PM GMT +0000. [03:10:30] RECOVERY - maxcapacity.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'maxcapacity.wiki' will expire on Tue 18 Feb 2025 03:36:47 PM GMT +0000. [03:10:57] RECOVERY - mariopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'mariopedia.org' will expire on Sat 08 Mar 2025 10:39:11 PM GMT +0000. [03:11:12] RECOVERY - exordium.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'exordium.wiki' will expire on Wed 12 Feb 2025 08:08:49 PM GMT +0000. [03:11:21] RECOVERY - antiguabarbudacalypso.com - LetsEncrypt on sslhost is OK: OK - Certificate 'antiguabarbudacalypso.com' will expire on Sat 08 Mar 2025 06:18:35 PM GMT +0000. [03:11:36] RECOVERY - bobobay.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'bobobay.wiki' will expire on Wed 12 Feb 2025 02:26:39 PM GMT +0000. [03:11:37] RECOVERY - wizardia.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'wizardia.wiki' will expire on Fri 03 Jan 2025 02:58:23 AM GMT +0000. [03:11:49] RECOVERY - sptwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'sptwiki.org' will expire on Sun 26 Jan 2025 06:17:34 PM GMT +0000. [03:11:58] RECOVERY - permanentfuturelab.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'permanentfuturelab.wiki' will expire on Thu 02 Jan 2025 04:22:43 PM GMT +0000. [03:11:58] RECOVERY - grayravens.com - LetsEncrypt on sslhost is OK: OK - Certificate 'grayravens.com' will expire on Sat 08 Mar 2025 03:35:29 PM GMT +0000. [03:12:30] RECOVERY - keitaiwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'keitaiwiki.com' will expire on Fri 14 Mar 2025 02:33:57 PM GMT +0000. [03:12:46] RECOVERY - terraforming.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'terraforming.wiki' will expire on Fri 21 Feb 2025 07:02:11 PM GMT +0000. [03:12:46] RECOVERY - farthestfrontier.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'farthestfrontier.wiki' will expire on Wed 12 Feb 2025 08:09:17 PM GMT +0000. [03:13:04] RECOVERY - thestarsareright.org - LetsEncrypt on sslhost is OK: OK - Certificate 'thestarsareright.org' will expire on Sat 08 Mar 2025 11:10:21 AM GMT +0000. [03:13:04] RECOVERY - aryavartpedia.online - LetsEncrypt on sslhost is OK: OK - Certificate 'aryavartpedia.online' will expire on Mon 20 Jan 2025 12:08:24 AM GMT +0000. [03:13:04] RECOVERY - thelonsdalebattalion.co.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'thelonsdalebattalion.co.uk' will expire on Thu 06 Mar 2025 09:28:33 PM GMT +0000. [03:13:16] RECOVERY - gielinorgames.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'gielinorgames.wiki' will expire on Sun 09 Mar 2025 10:35:49 PM GMT +0000. [03:13:29] RECOVERY - yokaiwatchwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'yokaiwatchwiki.com' will expire on Fri 14 Mar 2025 02:31:09 PM GMT +0000. [03:13:44] RECOVERY - journeytheword.wiki - Cloudflare on sslhost is OK: OK - Certificate 'CloudFlare Origin Certificate' will expire on Sat 27 Aug 2039 04:37:00 PM GMT +0000. [03:13:46] RECOVERY - electowiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'electowiki.org' will expire on Tue 11 Mar 2025 12:09:09 PM GMT +0000. [03:13:48] RECOVERY - christipedia.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'christipedia.nl' will expire on Sat 15 Mar 2025 02:14:33 PM GMT +0000. [03:14:03] RECOVERY - inourownwords.online - LetsEncrypt on sslhost is OK: OK - Certificate 'inourownwords.online' will expire on Sat 08 Mar 2025 09:05:57 AM GMT +0000. [03:14:06] RECOVERY - ourlifewiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'ourlifewiki.org' will expire on Sun 26 Jan 2025 06:18:01 PM GMT +0000. [03:14:16] RECOVERY - ncuwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'ncuwiki.com' will expire on Sun 09 Mar 2025 05:24:06 PM GMT +0000. [03:14:24] RECOVERY - crashspyrowiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'crashspyrowiki.com' will expire on Sun 09 Mar 2025 03:37:17 PM GMT +0000. [03:15:25] RECOVERY - kunwok.org - LetsEncrypt on sslhost is OK: OK - Certificate 'kunwok.org' will expire on Fri 07 Mar 2025 08:15:03 AM GMT +0000. [03:15:27] RECOVERY - androidwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'androidwiki.info' will expire on Mon 27 Jan 2025 03:47:10 PM GMT +0000. [03:15:56] RECOVERY - thegreatwar.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'thegreatwar.uk' will expire on Fri 14 Mar 2025 02:25:51 PM GMT +0000. [03:15:57] RECOVERY - ekumen.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ekumen.wiki' will expire on Thu 23 Jan 2025 10:33:12 PM GMT +0000. [03:16:28] RECOVERY - vylet.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'vylet.wiki' will expire on Thu 06 Feb 2025 02:42:38 AM GMT +0000. [03:16:31] RECOVERY - rodzinka.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'rodzinka.wiki' will expire on Wed 12 Feb 2025 08:04:19 PM GMT +0000. [03:16:35] RECOVERY - roll-of-arms.com - LetsEncrypt on sslhost is OK: OK - Certificate 'roll-of-arms.com' will expire on Sat 08 Mar 2025 11:49:41 PM GMT +0000. [03:16:41] RECOVERY - comprehensibleinputwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'comprehensibleinputwiki.org' will expire on Tue 11 Feb 2025 02:39:50 PM GMT +0000. [03:16:43] RECOVERY - blackmagic.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'blackmagic.wiki' will expire on Sun 23 Feb 2025 07:21:15 PM GMT +0000. [03:16:58] RECOVERY - ff8.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ff8.wiki' will expire on Sat 15 Feb 2025 01:05:30 AM GMT +0000. [03:17:00] RECOVERY - squareenixwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'squareenixwiki.com' will expire on Sun 09 Mar 2025 12:28:04 AM GMT +0000. [03:17:05] RECOVERY - landofliberos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'landofliberos.com' will expire on Sun 09 Mar 2025 08:42:53 AM GMT +0000. [03:17:27] RECOVERY - dccomicswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'dccomicswiki.com' will expire on Fri 28 Feb 2025 11:41:34 PM GMT +0000. [03:17:27] RECOVERY - fischipedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'fischipedia.org' will expire on Mon 10 Feb 2025 06:19:30 PM GMT +0000. [03:17:38] RECOVERY - jackbox.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'jackbox.wiki' will expire on Sat 08 Feb 2025 02:40:00 PM GMT +0000. [03:17:43] RECOVERY - stablestate.org - LetsEncrypt on sslhost is OK: OK - Certificate 'stablestate.org' will expire on Fri 07 Mar 2025 07:29:55 AM GMT +0000. [03:17:43] RECOVERY - hoseki.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hoseki.wiki' will expire on Sat 08 Mar 2025 01:09:01 PM GMT +0000. [03:17:47] RECOVERY - removededm.com - LetsEncrypt on sslhost is OK: OK - Certificate 'removededm.com' will expire on Sat 08 Feb 2025 12:51:28 PM GMT +0000. [03:18:02] RECOVERY - gimkit.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'gimkit.wiki' will expire on Wed 05 Mar 2025 02:35:49 PM GMT +0000. [03:18:09] RECOVERY - wikitide.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.net' will expire on Tue 28 Jan 2025 12:33:59 AM GMT +0000. [03:18:12] RECOVERY - acuralegendwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'acuralegendwiki.org' will expire on Sun 09 Mar 2025 02:34:56 AM GMT +0000. [03:18:54] RECOVERY - cp36 Nginx Backend for mw152 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8114 [03:18:58] RECOVERY - cp36 Nginx Backend for swiftproxy161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8206 [03:19:06] RECOVERY - cp36 Nginx Backend for mw184 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8128 [03:19:13] RECOVERY - cp36 Nginx Backend for mw171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8117 [03:19:14] RECOVERY - cp36 Nginx Backend for mw161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8115 [03:19:14] RECOVERY - cp36 Nginx Backend for mw153 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8121 [03:19:20] RECOVERY - cp36 Nginx Backend for phorge171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8202 [03:19:30] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [03:19:34] RECOVERY - cp36 Nginx Backend for test151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8181 [03:19:38] RECOVERY - cp36 HTTPS on cp36 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4140 bytes in 0.060 second response time [03:19:41] RECOVERY - cp36 Nginx Backend for mw173 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8125 [03:19:44] RECOVERY - cp36 Nginx Backend for puppet181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8204 [03:19:52] RECOVERY - cp36 Nginx Backend for matomo151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8203 [03:19:54] RECOVERY - cp36 Nginx Backend for mw164 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8124 [03:19:55] RECOVERY - cp36 Nginx Backend for mw154 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8122 [03:19:55] RECOVERY - cp36 Nginx Backend for mwtask171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8161 [03:19:59] RECOVERY - cp36 Nginx Backend for mw174 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8126 [03:20:05] RECOVERY - cp36 Nginx Backend for mw151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8113 [03:20:05] RECOVERY - cp36 Nginx Backend for mw182 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8120 [03:20:08] RECOVERY - cp36 Nginx Backend for swiftproxy171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8207 [03:20:19] RECOVERY - cp36 Nginx Backend for mw172 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8118 [03:20:20] RECOVERY - cp36 Nginx Backend for mwtask151 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8162 [03:20:23] RECOVERY - cp36 Nginx Backend for mwtask181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8160 [03:20:26] RECOVERY - cp36 Nginx Backend for mon181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8201 [03:20:26] RECOVERY - cp36 Nginx Backend for mw183 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8127 [03:20:30] RECOVERY - cp36 Nginx Backend for reports171 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8205 [03:20:34] RECOVERY - cp36 Nginx Backend for mw163 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8123 [03:20:37] RECOVERY - cp36 Nginx Backend for mw181 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8119 [03:20:37] RECOVERY - cp36 Nginx Backend for mw162 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8116 [03:20:38] RECOVERY - cp36 Nginx Backend for mwtask161 on cp36 is OK: TCP OK - 0.000 second response time on localhost port 8163 [03:20:45] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [03:20:55] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 4% [03:21:21] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 29 backends are healthy [03:22:34] RECOVERY - *.wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Mon 27 Jan 2025 07:57:39 PM GMT +0000. [03:22:48] RECOVERY - ssbuniverses.com - LetsEncrypt on sslhost is OK: OK - Certificate 'ssbuniverses.com' will expire on Mon 10 Mar 2025 06:31:05 PM GMT +0000. [03:23:02] RECOVERY - emmytherobot.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'emmytherobot.wiki' will expire on Tue 14 Jan 2025 01:22:21 PM GMT +0000. [03:23:02] RECOVERY - pokemonwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'pokemonwiki.info' will expire on Thu 02 Jan 2025 04:07:38 PM GMT +0000. [03:23:05] RECOVERY - geeu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'geeu.org' will expire on Tue 11 Feb 2025 02:36:30 PM GMT +0000. [03:23:24] RECOVERY - rippaversewiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'rippaversewiki.com' will expire on Tue 14 Jan 2025 01:24:58 PM GMT +0000. [03:23:37] RECOVERY - wikimas.kr - LetsEncrypt on sslhost is OK: OK - Certificate 'wikimas.kr' will expire on Sat 08 Mar 2025 09:56:22 AM GMT +0000. [03:24:06] RECOVERY - urbanshade.org - LetsEncrypt on sslhost is OK: OK - Certificate 'urbanshade.org' will expire on Wed 08 Jan 2025 03:18:01 PM GMT +0000. [03:24:22] RECOVERY - solaswiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'solaswiki.org' will expire on Sun 09 Mar 2025 11:41:38 AM GMT +0000. [03:24:24] RECOVERY - decimatedrive.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'decimatedrive.wiki' will expire on Sun 05 Jan 2025 10:46:22 PM GMT +0000. [03:24:35] RECOVERY - wikappedia.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'wikappedia.cc' will expire on Tue 07 Jan 2025 02:53:39 PM GMT +0000. [03:24:35] RECOVERY - freedomplanetwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'freedomplanetwiki.com' will expire on Sun 09 Mar 2025 05:01:39 PM GMT +0000. [03:24:35] RECOVERY - monarchists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'monarchists.wiki' will expire on Fri 07 Mar 2025 08:41:30 AM GMT +0000. [03:24:49] RECOVERY - issue-tracker.wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Mon 27 Jan 2025 07:57:39 PM GMT +0000. [03:24:50] RECOVERY - polcompball.net - LetsEncrypt on sslhost is OK: OK - Certificate 'polcompball.net' will expire on Wed 12 Feb 2025 01:32:10 AM GMT +0000. [03:24:50] RECOVERY - bonesword.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'bonesword.wiki' will expire on Sat 25 Jan 2025 06:14:29 AM GMT +0000. [03:24:58] RECOVERY - hastursnotebook.org - LetsEncrypt on sslhost is OK: OK - Certificate 'hastursnotebook.org' will expire on Mon 10 Mar 2025 07:54:11 AM GMT +0000. [03:25:04] RECOVERY - wikitide.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.org' will expire on Mon 27 Jan 2025 07:57:39 PM GMT +0000. [03:25:09] RECOVERY - speleo.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'speleo.wiki' will expire on Thu 06 Mar 2025 09:11:56 PM GMT +0000. [03:25:10] RECOVERY - icclopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'icclopedia.org' will expire on Sun 09 Mar 2025 02:26:09 AM GMT +0000. [03:25:10] RECOVERY - pso2ngs.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pso2ngs.wiki' will expire on Sat 18 Jan 2025 11:28:50 AM GMT +0000. [03:25:24] RECOVERY - rainverse.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'rainverse.wiki' will expire on Wed 05 Feb 2025 01:26:05 AM GMT +0000. [03:25:30] RECOVERY - pizzatower.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pizzatower.wiki' will expire on Sat 15 Mar 2025 06:20:47 PM GMT +0000. [03:26:01] RECOVERY - junxions.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'junxions.wiki' will expire on Wed 05 Mar 2025 02:30:23 PM GMT +0000. [03:26:05] RECOVERY - wikigenius.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikigenius.org' will expire on Wed 22 Jan 2025 11:09:01 AM GMT +0000. [03:26:19] RECOVERY - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediaofastrobiology.org' will expire on Fri 07 Mar 2025 08:41:41 AM GMT +0000. [03:26:25] RECOVERY - wikimoma.art - LetsEncrypt on sslhost is OK: OK - Certificate 'wikimoma.art' will expire on Fri 14 Mar 2025 02:42:04 PM GMT +0000. [03:26:46] RECOVERY - fashiondreamer.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'fashiondreamer.wiki' will expire on Fri 14 Mar 2025 01:50:18 PM GMT +0000. [03:27:05] RECOVERY - cpmusicwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'cpmusicwiki.com' will expire on Thu 30 Jan 2025 11:52:22 AM GMT +0000. [03:27:24] RECOVERY - civwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'civwiki.org' will expire on Fri 07 Mar 2025 11:39:55 PM GMT +0000. [03:27:31] RECOVERY - burnout.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'burnout.wiki' will expire on Thu 02 Jan 2025 04:09:56 PM GMT +0000. [03:27:31] RECOVERY - kingdomway.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'kingdomway.wiki' will expire on Wed 29 Jan 2025 06:48:21 AM GMT +0000. [03:27:44] RECOVERY - worldless.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'worldless.wiki' will expire on Thu 20 Feb 2025 07:02:16 AM GMT +0000. [03:27:45] RECOVERY - kingdomdeath.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'kingdomdeath.wiki' will expire on Fri 14 Mar 2025 08:40:52 AM GMT +0000. [03:28:19] RECOVERY - vesc.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'vesc.wiki' will expire on Tue 11 Mar 2025 01:54:22 PM GMT +0000. [03:28:31] RECOVERY - psycho.engineering - LetsEncrypt on sslhost is OK: OK - Certificate 'psycho.engineering' will expire on Tue 11 Mar 2025 12:21:00 PM GMT +0000. [03:28:35] RECOVERY - partopedia.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'partopedia.ru' will expire on Fri 21 Feb 2025 08:36:43 PM GMT +0000. [03:28:41] RECOVERY - warhammer40kturkiye.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'warhammer40kturkiye.wiki' will expire on Mon 13 Jan 2025 05:26:00 PM GMT +0000. [03:28:43] RECOVERY - iol.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'iol.wiki' will expire on Sun 09 Mar 2025 12:04:46 AM GMT +0000. [03:28:47] RECOVERY - encyclopediarobotica.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediarobotica.org' will expire on Sun 09 Feb 2025 02:29:45 PM GMT +0000. [03:29:52] RECOVERY - lostmediawiki.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'lostmediawiki.ru' will expire on Wed 22 Jan 2025 02:19:38 PM GMT +0000. [03:30:06] RECOVERY - opendatascot.org - LetsEncrypt on sslhost is OK: OK - Certificate 'opendatascot.org' will expire on Wed 12 Mar 2025 08:48:09 AM GMT +0000. [03:30:06] RECOVERY - worldtriggerwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'worldtriggerwiki.com' will expire on Sun 16 Mar 2025 12:03:12 AM GMT +0000. [03:30:09] RECOVERY - sytbay.site - LetsEncrypt on sslhost is OK: OK - Certificate 'sytbay.site' will expire on Mon 03 Feb 2025 10:49:41 PM GMT +0000. [03:30:56] RECOVERY - grayzonewarfare.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'grayzonewarfare.wiki' will expire on Wed 22 Jan 2025 08:13:00 PM GMT +0000. [03:30:56] PROBLEM - wiki.metaiitgn.org - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [03:31:10] RECOVERY - fanonpedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'fanonpedia.com' will expire on Wed 15 Jan 2025 01:02:33 PM GMT +0000. [03:52:44] PROBLEM - cp37 Current Load on cp37 is WARNING: LOAD WARNING - total load average: 7.75, 4.52, 2.05 [03:54:08] PROBLEM - cp36 Current Load on cp36 is WARNING: LOAD WARNING - total load average: 7.60, 5.76, 2.77 [03:54:44] PROBLEM - cp37 Current Load on cp37 is CRITICAL: LOAD CRITICAL - total load average: 20.44, 9.90, 4.30 [03:55:47] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 94% [03:56:08] PROBLEM - cp36 Current Load on cp36 is CRITICAL: LOAD CRITICAL - total load average: 12.54, 8.10, 3.98 [03:56:09] PROBLEM - cp36 HTTPS on cp36 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [03:56:11] PROBLEM - cp37 HTTPS on cp37 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [03:56:22] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 4 datacenters are down: 38.46.223.205/cpweb, 38.46.223.206/cpweb, 2602:294:0:b13::110/cpweb, 2602:294:0:b23::112/cpweb [03:56:32] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 68% [03:56:45] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 3 datacenters are down: 38.46.223.205/cpweb, 2602:294:0:b13::110/cpweb, 2602:294:0:b23::112/cpweb [03:57:38] RECOVERY - trollpasta.com - LetsEncrypt on sslhost is OK: OK - Certificate 'trollpasta.com' will expire on Thu 02 Jan 2025 04:17:18 PM GMT +0000. [04:00:32] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is WARNING: WARNING - NGINX Error Rate is 54% [04:00:42] PROBLEM - cp36 Varnish Backends on cp36 is WARNING: No backends detected. If this is an error, see readme.txt [04:02:15] RECOVERY - cp36 HTTPS on cp36 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4162 bytes in 1.182 second response time [04:02:38] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 29 backends are healthy [04:03:30] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 28% [04:04:32] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [04:05:07] PROBLEM - cp37 Varnish Backends on cp37 is CRITICAL: 17 backends are down. mw151 mw152 mw161 mw162 mw171 mw172 mw181 mw182 mw153 mw154 mw163 mw164 mw173 mw174 mw183 mw184 mediawiki [04:06:09] RECOVERY - cp37 HTTPS on cp37 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 4140 bytes in 0.055 second response time [04:06:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 0% [04:06:36] RECOVERY - cp37 Varnish Backends on cp37 is OK: All 29 backends are healthy [04:06:44] PROBLEM - cp37 Current Load on cp37 is WARNING: LOAD WARNING - total load average: 2.84, 7.15, 6.91 [04:06:45] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [04:07:19] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:08:49] PROBLEM - cp37 Current Load on cp37 is CRITICAL: LOAD CRITICAL - total load average: 15.10, 10.56, 8.21 [04:09:16] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 76% [04:10:27] PROBLEM - mw151 Puppet on mw151 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.42],Exec[git_pull_femiwiki-deploy-1.43] [04:10:36] PROBLEM - ping6 on ns2 is WARNING: PING WARNING - Packet loss = 0%, RTA = 338.84 ms [04:10:42] PROBLEM - mw164 Puppet on mw164 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.42] [04:10:51] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [04:11:21] PROBLEM - db182 NTP time on db182 is WARNING: NTP WARNING: Offset -0.1009038389 secs [04:12:15] PROBLEM - cloud18 NTP time on cloud18 is WARNING: NTP WARNING: Offset -0.116558671 secs [04:12:51] PROBLEM - ping6 on mattermost1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 155.36 ms [04:13:15] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset -0.1007243693 secs [04:13:25] RECOVERY - db182 NTP time on db182 is OK: NTP OK: Offset -0.09203931689 secs [04:13:40] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 97% [04:15:15] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.09629186988 secs [04:16:57] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 16% [04:17:14] PROBLEM - cp36 Puppet on cp36 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 28 minutes ago with 0 failures [04:17:27] PROBLEM - swiftobject171 NTP time on swiftobject171 is WARNING: NTP WARNING: Offset -0.10152179 secs [04:17:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 5% [04:17:47] PROBLEM - cp37 Puppet on cp37 is WARNING: WARNING: Puppet is currently disabled, message: Testing nginx rate limits --Void, last run 18 minutes ago with 0 failures [04:19:22] PROBLEM - mw152 NTP time on mw152 is WARNING: NTP WARNING: Offset -0.1005406976 secs [04:19:22] PROBLEM - mw153 NTP time on mw153 is WARNING: NTP WARNING: Offset -0.1026678085 secs [04:19:27] PROBLEM - mw151 NTP time on mw151 is WARNING: NTP WARNING: Offset -0.1019482017 secs [04:19:34] PROBLEM - swiftobject181 NTP time on swiftobject181 is WARNING: NTP WARNING: Offset -0.1002656817 secs [04:20:29] PROBLEM - mw183 NTP time on mw183 is WARNING: NTP WARNING: Offset -0.1168676019 secs [04:21:21] RECOVERY - mw152 NTP time on mw152 is OK: NTP OK: Offset -0.09381911159 secs [04:21:37] RECOVERY - swiftobject171 NTP time on swiftobject171 is OK: NTP OK: Offset -0.09791073203 secs [04:21:38] RECOVERY - swiftobject181 NTP time on swiftobject181 is OK: NTP OK: Offset -0.0977371335 secs [04:21:54] PROBLEM - mw171 NTP time on mw171 is WARNING: NTP WARNING: Offset -0.1019406915 secs [04:22:14] RECOVERY - cloud18 NTP time on cloud18 is OK: NTP OK: Offset -0.09797114134 secs [04:22:18] PROBLEM - graphite151 NTP time on graphite151 is WARNING: NTP WARNING: Offset -0.1017578542 secs [04:22:34] RECOVERY - mw183 NTP time on mw183 is OK: NTP OK: Offset -0.0995041132 secs [04:22:44] PROBLEM - cp37 Current Load on cp37 is WARNING: LOAD WARNING - total load average: 0.17, 4.45, 7.75 [04:23:21] PROBLEM - mw162 NTP time on mw162 is WARNING: NTP WARNING: Offset -0.1001785994 secs [04:23:21] RECOVERY - mw153 NTP time on mw153 is OK: NTP OK: Offset -0.09411662817 secs [04:23:30] PROBLEM - mwtask171 NTP time on mwtask171 is WARNING: NTP WARNING: Offset -0.102625221 secs [04:23:31] RECOVERY - mw151 NTP time on mw151 is OK: NTP OK: Offset -0.09669452906 secs [04:23:53] PROBLEM - test151 NTP time on test151 is WARNING: NTP WARNING: Offset -0.1017312407 secs [04:24:06] PROBLEM - mwtask181 NTP time on mwtask181 is WARNING: NTP WARNING: Offset -0.1020065248 secs [04:24:07] PROBLEM - cp36 Current Load on cp36 is WARNING: LOAD WARNING - total load average: 0.57, 2.93, 7.19 [04:24:21] RECOVERY - graphite151 NTP time on graphite151 is OK: NTP OK: Offset -0.09612777829 secs [04:24:54] PROBLEM - ping6 on ns2 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 345.28 ms [04:25:06] PROBLEM - db171 NTP time on db171 is WARNING: NTP WARNING: Offset -0.1010190547 secs [04:25:32] RECOVERY - mwtask171 NTP time on mwtask171 is OK: NTP OK: Offset -0.09729927778 secs [04:26:07] RECOVERY - cp36 Current Load on cp36 is OK: LOAD OK - total load average: 0.28, 2.02, 6.33 [04:26:08] PROBLEM - prometheus151 NTP time on prometheus151 is WARNING: NTP WARNING: Offset -0.1030931771 secs [04:26:38] PROBLEM - cloud16 NTP time on cloud16 is WARNING: NTP WARNING: Offset -0.1012549102 secs [04:26:44] RECOVERY - cp37 Current Load on cp37 is OK: LOAD OK - total load average: 0.22, 2.15, 6.05 [04:26:55] PROBLEM - bots171 NTP time on bots171 is WARNING: NTP WARNING: Offset -0.103985548 secs [04:26:56] PROBLEM - ping6 on ns2 is WARNING: PING WARNING - Packet loss = 0%, RTA = 348.49 ms [04:26:57] PROBLEM - ldap171 NTP time on ldap171 is WARNING: NTP WARNING: Offset -0.1002146304 secs [04:26:58] PROBLEM - db182 NTP time on db182 is WARNING: NTP WARNING: Offset -0.1024723053 secs [04:26:58] PROBLEM - reports171 NTP time on reports171 is WARNING: NTP WARNING: Offset -0.1000323892 secs [04:27:12] PROBLEM - rdb151 NTP time on rdb151 is WARNING: NTP WARNING: Offset -0.102594316 secs [04:27:14] PROBLEM - cp36 NTP time on cp36 is WARNING: NTP WARNING: Offset -0.1019372344 secs [04:27:25] PROBLEM - mon181 NTP time on mon181 is WARNING: NTP WARNING: Offset -0.1015426815 secs [04:27:26] PROBLEM - cloud17 NTP time on cloud17 is WARNING: NTP WARNING: Offset -0.1004639864 secs [04:27:32] PROBLEM - mw161 NTP time on mw161 is WARNING: NTP WARNING: Offset -0.1006633937 secs [04:27:33] RECOVERY - mw162 NTP time on mw162 is OK: NTP OK: Offset -0.09213247895 secs [04:27:35] PROBLEM - swiftac171 NTP time on swiftac171 is WARNING: NTP WARNING: Offset -0.1016778648 secs [04:27:37] PROBLEM - mw151 NTP time on mw151 is WARNING: NTP WARNING: Offset -0.1026777327 secs [04:28:00] PROBLEM - os161 NTP time on os161 is WARNING: NTP WARNING: Offset -0.102492094 secs [04:28:06] RECOVERY - mw171 NTP time on mw171 is OK: NTP OK: Offset -0.09997594357 secs [04:28:16] PROBLEM - mw163 NTP time on mw163 is WARNING: NTP WARNING: Offset -0.1019827425 secs [04:28:38] RECOVERY - cloud16 NTP time on cloud16 is OK: NTP OK: Offset -0.09858611226 secs [04:28:59] PROBLEM - swiftproxy171 NTP time on swiftproxy171 is WARNING: NTP WARNING: Offset -0.1017153263 secs [04:28:59] RECOVERY - bots171 NTP time on bots171 is OK: NTP OK: Offset -0.09879517555 secs [04:29:00] PROBLEM - ping6 on ns2 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 345.87 ms [04:29:02] RECOVERY - db182 NTP time on db182 is OK: NTP OK: Offset -0.08972686529 secs [04:29:03] RECOVERY - reports171 NTP time on reports171 is OK: NTP OK: Offset -0.084382236 secs [04:29:10] RECOVERY - db171 NTP time on db171 is OK: NTP OK: Offset -0.09991389513 secs [04:29:25] RECOVERY - mon181 NTP time on mon181 is OK: NTP OK: Offset -0.0885887742 secs [04:29:26] RECOVERY - cloud17 NTP time on cloud17 is OK: NTP OK: Offset -0.09458467364 secs [04:29:31] RECOVERY - mw161 NTP time on mw161 is OK: NTP OK: Offset -0.0952051878 secs [04:29:36] RECOVERY - swiftac171 NTP time on swiftac171 is OK: NTP OK: Offset -0.09752303362 secs [04:29:59] RECOVERY - os161 NTP time on os161 is OK: NTP OK: Offset -0.09162908792 secs [04:30:01] PROBLEM - swiftobject171 NTP time on swiftobject171 is WARNING: NTP WARNING: Offset -0.1018129587 secs [04:30:03] RECOVERY - test151 NTP time on test151 is OK: NTP OK: Offset -0.09157732129 secs [04:30:09] RECOVERY - prometheus151 NTP time on prometheus151 is OK: NTP OK: Offset -0.08236739039 secs [04:30:12] RECOVERY - mwtask181 NTP time on mwtask181 is OK: NTP OK: Offset -0.07145106792 secs [04:30:14] PROBLEM - cloud18 NTP time on cloud18 is WARNING: NTP WARNING: Offset -0.1036041379 secs [04:30:15] PROBLEM - mwtask151 NTP time on mwtask151 is WARNING: NTP WARNING: Offset -0.1039251089 secs [04:30:25] PROBLEM - mw153 NTP time on mw153 is WARNING: NTP WARNING: Offset -0.1069119871 secs [04:30:27] PROBLEM - puppet181 NTP time on puppet181 is WARNING: NTP WARNING: Offset -0.1073914766 secs [04:31:24] PROBLEM - mem161 NTP time on mem161 is WARNING: NTP WARNING: Offset -0.10564816 secs [04:31:42] PROBLEM - swiftproxy161 NTP time on swiftproxy161 is WARNING: NTP WARNING: Offset -0.1047641635 secs [04:31:46] PROBLEM - mwtask171 NTP time on mwtask171 is WARNING: NTP WARNING: Offset -0.1046037674 secs [04:31:48] PROBLEM - mw182 NTP time on mw182 is WARNING: NTP WARNING: Offset -0.1021487713 secs [04:31:55] PROBLEM - swiftobject161 NTP time on swiftobject161 is WARNING: NTP WARNING: Offset -0.1036309004 secs [04:32:05] RECOVERY - swiftobject171 NTP time on swiftobject171 is OK: NTP OK: Offset -0.09391605854 secs [04:32:16] RECOVERY - mwtask151 NTP time on mwtask151 is OK: NTP OK: Offset -0.09652394056 secs [04:32:17] PROBLEM - bast181 NTP time on bast181 is WARNING: NTP WARNING: Offset -0.1024311185 secs [04:32:19] PROBLEM - cp37 NTP time on cp37 is WARNING: NTP WARNING: Offset -0.1000409722 secs [04:32:20] RECOVERY - mw163 NTP time on mw163 is OK: NTP OK: Offset -0.09956118464 secs [04:32:30] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset -0.1058626771 secs [04:32:36] PROBLEM - graphite151 NTP time on graphite151 is WARNING: NTP WARNING: Offset -0.1049851775 secs [04:32:37] PROBLEM - mw174 NTP time on mw174 is WARNING: NTP WARNING: Offset -0.1055113971 secs [04:32:38] PROBLEM - cloud16 NTP time on cloud16 is WARNING: NTP WARNING: Offset -0.1042678356 secs [04:32:39] PROBLEM - mw181 NTP time on mw181 is WARNING: NTP WARNING: Offset -0.1042251885 secs [04:32:41] PROBLEM - os151 NTP time on os151 is WARNING: NTP WARNING: Offset -0.1039738655 secs [04:32:43] PROBLEM - changeprop151 NTP time on changeprop151 is WARNING: NTP WARNING: Offset -0.10423702 secs [04:32:57] RECOVERY - ldap171 NTP time on ldap171 is OK: NTP OK: Offset -0.09886777401 secs [04:32:58] PROBLEM - cloud15 NTP time on cloud15 is WARNING: NTP WARNING: Offset -0.1046605408 secs [04:33:10] PROBLEM - swiftobject151 NTP time on swiftobject151 is WARNING: NTP WARNING: Offset -0.1032109559 secs [04:33:15] PROBLEM - db182 NTP time on db182 is WARNING: NTP WARNING: Offset -0.1045092046 secs [04:33:18] PROBLEM - reports171 NTP time on reports171 is WARNING: NTP WARNING: Offset -0.1024841964 secs [04:33:35] PROBLEM - mw161 NTP time on mw161 is WARNING: NTP WARNING: Offset -0.1048460305 secs [04:33:35] PROBLEM - bast161 NTP time on bast161 is WARNING: NTP WARNING: Offset -0.1058609486 secs [04:33:42] PROBLEM - graylog161 NTP time on graylog161 is WARNING: NTP WARNING: Offset -0.1039575934 secs [04:33:43] PROBLEM - mwtask161 NTP time on mwtask161 is WARNING: NTP WARNING: Offset -0.1032416224 secs [04:33:44] PROBLEM - mem151 NTP time on mem151 is WARNING: NTP WARNING: Offset -0.1065305471 secs [04:33:46] PROBLEM - mw154 NTP time on mw154 is WARNING: NTP WARNING: Offset -0.1063068211 secs [04:33:49] PROBLEM - swiftac171 NTP time on swiftac171 is WARNING: NTP WARNING: Offset -0.1055791676 secs [04:33:51] PROBLEM - eventgate181 NTP time on eventgate181 is WARNING: NTP WARNING: Offset -0.1055881381 secs [04:33:51] PROBLEM - mw173 NTP time on mw173 is WARNING: NTP WARNING: Offset -0.1050077975 secs [04:33:51] PROBLEM - mw172 NTP time on mw172 is WARNING: NTP WARNING: Offset -0.1052033007 secs [04:33:53] PROBLEM - mw162 NTP time on mw162 is WARNING: NTP WARNING: Offset -0.1046696305 secs [04:33:55] PROBLEM - matomo151 NTP time on matomo151 is WARNING: NTP WARNING: Offset -0.1049990952 secs [04:33:58] PROBLEM - db161 NTP time on db161 is WARNING: NTP WARNING: Offset -0.1050010026 secs [04:33:59] PROBLEM - mw164 NTP time on mw164 is WARNING: NTP WARNING: Offset -0.1060404181 secs [04:34:05] PROBLEM - os161 NTP time on os161 is WARNING: NTP WARNING: Offset -0.1031613052 secs [04:34:07] PROBLEM - swiftobject181 NTP time on swiftobject181 is WARNING: NTP WARNING: Offset -0.1068076193 secs [04:34:09] PROBLEM - mw183 NTP time on mw183 is WARNING: NTP WARNING: Offset -0.1126515269 secs [04:34:26] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [04:34:45] RECOVERY - changeprop151 NTP time on changeprop151 is OK: NTP OK: Offset -0.08917722106 secs [04:34:55] RECOVERY - cloud15 NTP time on cloud15 is OK: NTP OK: Offset -0.08267071843 secs [04:35:11] RECOVERY - swiftproxy171 NTP time on swiftproxy171 is OK: NTP OK: Offset -0.07926928997 secs [04:35:23] RECOVERY - reports171 NTP time on reports171 is OK: NTP OK: Offset -0.09904575348 secs [04:35:24] RECOVERY - mem161 NTP time on mem161 is OK: NTP OK: Offset -0.09850943089 secs [04:36:10] PROBLEM - phorge171 NTP time on phorge171 is WARNING: NTP WARNING: Offset -0.1019639075 secs [04:36:11] PROBLEM - db172 NTP time on db172 is WARNING: NTP WARNING: Offset -0.113090694 secs [04:36:15] PROBLEM - test151 NTP time on test151 is WARNING: NTP WARNING: Offset -0.1220016778 secs [04:36:18] PROBLEM - swiftobject171 NTP time on swiftobject171 is WARNING: NTP WARNING: Offset -0.1220865846 secs [04:36:19] PROBLEM - db151 NTP time on db151 is WARNING: NTP WARNING: Offset -0.1199381053 secs [04:36:23] PROBLEM - jobchron171 NTP time on jobchron171 is WARNING: NTP WARNING: Offset -0.1203581691 secs [04:36:25] PROBLEM - mwtask151 NTP time on mwtask151 is WARNING: NTP WARNING: Offset -0.1218499839 secs [04:36:26] PROBLEM - mw171 NTP time on mw171 is WARNING: NTP WARNING: Offset -0.1219399273 secs [04:36:28] PROBLEM - mwtask181 NTP time on mwtask181 is WARNING: NTP WARNING: Offset -0.1218556166 secs [04:36:28] PROBLEM - mw152 NTP time on mw152 is WARNING: NTP WARNING: Offset -0.121707648 secs [04:36:28] PROBLEM - os162 NTP time on os162 is WARNING: NTP WARNING: Offset -0.1207306981 secs [04:36:30] PROBLEM - mw163 NTP time on mw163 is WARNING: NTP WARNING: Offset -0.1225915849 secs [04:36:31] PROBLEM - db171 NTP time on db171 is WARNING: NTP WARNING: Offset -0.1203027368 secs [04:36:35] PROBLEM - db181 NTP time on db181 is WARNING: NTP WARNING: Offset -0.1008006334 secs [04:36:38] PROBLEM - mon181 NTP time on mon181 is WARNING: NTP WARNING: Offset -0.1021150947 secs [04:36:42] RECOVERY - mw164 Puppet on mw164 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:36:42] PROBLEM - kafka181 NTP time on kafka181 is WARNING: NTP WARNING: Offset -0.1015709639 secs [04:37:11] PROBLEM - ping6 on ns2 is WARNING: PING WARNING - Packet loss = 0%, RTA = 347.95 ms [04:37:14] RECOVERY - swiftobject151 NTP time on swiftobject151 is OK: NTP OK: Offset -0.08891949058 secs [04:37:43] RECOVERY - graylog161 NTP time on graylog161 is OK: NTP OK: Offset -0.09745046496 secs [04:37:46] RECOVERY - mem151 NTP time on mem151 is OK: NTP OK: Offset -0.09666568041 secs [04:37:50] RECOVERY - mw154 NTP time on mw154 is OK: NTP OK: Offset -0.09586399794 secs [04:37:53] RECOVERY - swiftac171 NTP time on swiftac171 is OK: NTP OK: Offset -0.09789663553 secs [04:38:01] RECOVERY - mw164 NTP time on mw164 is OK: NTP OK: Offset -0.09818679094 secs [04:38:04] RECOVERY - mw182 NTP time on mw182 is OK: NTP OK: Offset -0.09573888779 secs [04:38:07] RECOVERY - os161 NTP time on os161 is OK: NTP OK: Offset -0.09524291754 secs [04:38:09] RECOVERY - phorge171 NTP time on phorge171 is OK: NTP OK: Offset -0.09503859282 secs [04:38:36] RECOVERY - db181 NTP time on db181 is OK: NTP OK: Offset -0.08482581377 secs [04:38:45] RECOVERY - graphite151 NTP time on graphite151 is OK: NTP OK: Offset -0.09368667006 secs [04:38:46] RECOVERY - kafka181 NTP time on kafka181 is OK: NTP OK: Offset -0.09503212571 secs [04:38:55] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 44%, RTA = 208.75 ms [04:39:14] RECOVERY - cp36 NTP time on cp36 is OK: NTP OK: Offset -0.08464309573 secs [04:39:28] PROBLEM - cloud17 NTP time on cloud17 is WARNING: NTP WARNING: Offset -0.1018687189 secs [04:39:37] RECOVERY - mw151 NTP time on mw151 is OK: NTP OK: Offset -0.09680983424 secs [04:39:43] RECOVERY - mwtask161 NTP time on mwtask161 is OK: NTP OK: Offset -0.09856256843 secs [04:39:54] RECOVERY - swiftproxy161 NTP time on swiftproxy161 is OK: NTP OK: Offset -0.09886959195 secs [04:39:57] RECOVERY - eventgate181 NTP time on eventgate181 is OK: NTP OK: Offset -0.09844833612 secs [04:40:04] RECOVERY - mw172 NTP time on mw172 is OK: NTP OK: Offset -0.099978894 secs [04:40:07] RECOVERY - matomo151 NTP time on matomo151 is OK: NTP OK: Offset -0.09812900424 secs [04:40:34] RECOVERY - mw171 NTP time on mw171 is OK: NTP OK: Offset -0.09527954459 secs [04:40:36] RECOVERY - db171 NTP time on db171 is OK: NTP OK: Offset -0.09949693084 secs [04:40:53] RECOVERY - os151 NTP time on os151 is OK: NTP OK: Offset -0.09669861197 secs [04:40:54] RECOVERY - puppet181 NTP time on puppet181 is OK: NTP OK: Offset -0.09930476546 secs [04:41:12] RECOVERY - rdb151 NTP time on rdb151 is OK: NTP OK: Offset -0.09275400639 secs [04:41:19] PROBLEM - prometheus151 NTP time on prometheus151 is WARNING: NTP WARNING: Offset -0.1034322381 secs [04:41:49] PROBLEM - graylog161 NTP time on graylog161 is WARNING: NTP WARNING: Offset -0.1023294032 secs [04:41:51] RECOVERY - mw173 NTP time on mw173 is OK: NTP OK: Offset -0.09732553363 secs [04:42:03] RECOVERY - db161 NTP time on db161 is OK: NTP OK: Offset -0.09963437915 secs [04:42:05] PROBLEM - changeprop151 NTP time on changeprop151 is WARNING: NTP WARNING: Offset -0.1027521193 secs [04:42:05] RECOVERY - swiftobject161 NTP time on swiftobject161 is OK: NTP OK: Offset -0.09636873007 secs [04:42:19] PROBLEM - mw182 NTP time on mw182 is WARNING: NTP WARNING: Offset -0.1147327125 secs [04:42:23] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.0001709163189 secs [04:42:25] RECOVERY - swiftobject181 NTP time on swiftobject181 is OK: NTP OK: Offset -4.696846008e-05 secs [04:42:25] RECOVERY - test151 NTP time on test151 is OK: NTP OK: Offset -0.09022775292 secs [04:42:27] RECOVERY - jobchron171 NTP time on jobchron171 is OK: NTP OK: Offset -0.0002468526363 secs [04:42:29] RECOVERY - mw183 NTP time on mw183 is OK: NTP OK: Offset -8.24034214e-05 secs [04:42:29] RECOVERY - mwtask151 NTP time on mwtask151 is OK: NTP OK: Offset -5.128979683e-05 secs [04:42:32] RECOVERY - swiftobject171 NTP time on swiftobject171 is OK: NTP OK: Offset -0.0001267790794 secs [04:42:34] RECOVERY - mwtask181 NTP time on mwtask181 is OK: NTP OK: Offset -0.000250428915 secs [04:42:36] RECOVERY - mw163 NTP time on mw163 is OK: NTP OK: Offset -0.0002895593643 secs [04:42:37] RECOVERY - cloud16 NTP time on cloud16 is OK: NTP OK: Offset -0.0003274083138 secs [04:42:39] RECOVERY - mon181 NTP time on mon181 is OK: NTP OK: Offset 8.246302605e-05 secs [04:42:40] RECOVERY - os162 NTP time on os162 is OK: NTP OK: Offset -0.0002695024014 secs [04:42:55] RECOVERY - mw181 NTP time on mw181 is OK: NTP OK: Offset -0.02416092157 secs [04:42:56] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 39.69 ms [04:43:05] RECOVERY - mw174 NTP time on mw174 is OK: NTP OK: Offset -0.0001436471939 secs [04:43:18] PROBLEM - ping6 on ns2 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 145.38 ms [04:43:19] RECOVERY - prometheus151 NTP time on prometheus151 is OK: NTP OK: Offset -0.0004417896271 secs [04:43:26] RECOVERY - cloud17 NTP time on cloud17 is OK: NTP OK: Offset -9.834766388e-07 secs [04:43:28] RECOVERY - bast161 NTP time on bast161 is OK: NTP OK: Offset -0.0002754926682 secs [04:43:30] RECOVERY - mw161 NTP time on mw161 is OK: NTP OK: Offset -0.0003724396229 secs [04:43:35] RECOVERY - db182 NTP time on db182 is OK: NTP OK: Offset -0.0001315176487 secs [04:43:49] RECOVERY - graylog161 NTP time on graylog161 is OK: NTP OK: Offset -0.0004214346409 secs [04:44:01] RECOVERY - mwtask171 NTP time on mwtask171 is OK: NTP OK: Offset -0.0001846551895 secs [04:44:07] RECOVERY - changeprop151 NTP time on changeprop151 is OK: NTP OK: Offset -0.0003571510315 secs [04:44:08] RECOVERY - bast181 NTP time on bast181 is OK: NTP OK: Offset 1.055002213e-05 secs [04:44:11] RECOVERY - db172 NTP time on db172 is OK: NTP OK: Offset -0.000144392252 secs [04:44:13] RECOVERY - cloud18 NTP time on cloud18 is OK: NTP OK: Offset -0.0001818537712 secs [04:44:16] RECOVERY - cp37 NTP time on cp37 is OK: NTP OK: Offset -0.0001330971718 secs [04:44:18] RECOVERY - mw162 NTP time on mw162 is OK: NTP OK: Offset -0.000272333622 secs [04:44:21] RECOVERY - mw153 NTP time on mw153 is OK: NTP OK: Offset -0.0002455413342 secs [04:44:23] RECOVERY - db151 NTP time on db151 is OK: NTP OK: Offset -0.0007340013981 secs [04:44:24] RECOVERY - mw152 NTP time on mw152 is OK: NTP OK: Offset -0.0001879930496 secs [04:44:24] RECOVERY - mw182 NTP time on mw182 is OK: NTP OK: Offset -0.0002871453762 secs [04:45:20] RECOVERY - ping6 on ns2 is OK: PING OK - Packet loss = 0%, RTA = 145.29 ms [04:48:52] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/rebuildall.php --wiki=deathbattlewiki (END - exit=0) [04:48:55] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/initSiteStats.php --wiki=deathbattlewiki --update (END - exit=0) [04:48:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:49:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:10:31] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 82% [05:10:55] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 83% [05:12:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 25% [05:16:31] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 94% [05:18:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 6% [05:18:55] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 4% [05:24:31] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 93% [05:24:56] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 85% [05:26:07] PROBLEM - cp36 Current Load on cp36 is CRITICAL: LOAD CRITICAL - total load average: 8.23, 7.06, 4.21 [05:26:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 28% [05:28:07] RECOVERY - cp36 Current Load on cp36 is OK: LOAD OK - total load average: 2.18, 5.23, 3.89 [05:30:31] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 88% [05:36:31] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 39% [05:36:55] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is WARNING: WARNING - NGINX Error Rate is 48% [05:37:02] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9208MiB (10% inode=98%); [05:38:55] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 3% [06:01:12] !log disabled Frankfrankultimate on Phorge for repeatedly disrespecting volunteers, violating the CoC and changing priorities [06:01:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:53:02] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/06eeb2aa3805bfeee9fef12fb4386a3820b97aca [14:53:02] 02puppet/03master 07paladox 0306eeb2a mattermost: upgrade to 10.3.1 [14:58:00] [02puppet] 07paladox pushed 1 new commit to 03master 13https://github.com/miraheze/puppet/commit/f443c071b377d748c25fa1edc814007b8acb8225 [14:58:00] 02puppet/03master 07paladox 03f443c07 mattermost: Set ScheduledPosts to true [16:22:13] PROBLEM - dragdown.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'dragdown.wiki' expires in 14 day(s) (Wed 01 Jan 2025 04:05:41 PM GMT +0000). [16:22:25] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/d8de5c56d46a32ddba3d3b314a5f4283d2e27d9e [16:22:25] 02ssl/03master 07WikiTideSSLBot 03d8de5c5 Bot: Update SSL cert for dragdown.wiki [16:34:25] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9723MiB (10% inode=98%); [17:04:09] [02dns] 07MacFan4000 pushed 1 new commit to 03master 13https://github.com/miraheze/dns/commit/609dc745724c5b2f8c19c285d4883379666f93ae [17:04:09] 02dns/03master 07MacFan4000 03609dc74 Create holostars.wiki [17:05:28] [02ssl] 07WikiTideSSLBot pushed 1 new commit to 03master 13https://github.com/miraheze/ssl/commit/208b6e1e9e3db57e4654ca1823a52a028a959b40 [17:05:28] 02ssl/03master 07WikiTideSSLBot 03208b6e1 Bot: Add SSL cert for holostars.wiki… [17:05:52] miraheze/dns - MacFan4000 the build passed. [17:24:54] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 81% [17:26:08] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 72% [17:26:53] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is WARNING: WARNING - NGINX Error Rate is 46% [17:27:52] PROBLEM - cp36 Current Load on cp36 is CRITICAL: LOAD CRITICAL - total load average: 11.07, 5.77, 2.52 [17:28:49] PROBLEM - cp37 Current Load on cp37 is CRITICAL: LOAD CRITICAL - total load average: 8.16, 5.71, 2.73 [17:29:49] RECOVERY - cp36 Current Load on cp36 is OK: LOAD OK - total load average: 3.57, 5.01, 2.65 [17:30:01] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 3% [17:30:48] RECOVERY - cp37 Current Load on cp37 is OK: LOAD OK - total load average: 1.62, 4.04, 2.48 [17:30:48] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 4% [17:40:17] PROBLEM - cp36 Disk Space on cp36 is CRITICAL: DISK CRITICAL - free space: / 4965MiB (5% inode=98%); [17:40:34] PROBLEM - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is CRITICAL: CRITICAL - NGINX Error Rate is 92% [17:41:40] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is CRITICAL: CRITICAL - NGINX Error Rate is 88% [17:49:21] PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 4702MiB (5% inode=98%); [17:50:23] RECOVERY - cp36 HTTP 4xx/5xx ERROR Rate on cp36 is OK: OK - NGINX Error Rate is 25% [17:51:25] RECOVERY - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is OK: OK - NGINX Error Rate is 4% [18:45:14] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importDump.php --wiki=fairutailwiki dump.xml --no-updates (START) [18:45:15] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importDump.php --wiki=fairutailwiki dump.xml --no-updates (END - exit=65280) [18:45:16] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/rebuildall.php --wiki=fairutailwiki (START) [18:45:17] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/rebuildall.php --wiki=fairutailwiki (END - exit=65280) [18:45:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:24] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importDump.php --wiki=fairytailwiki dump.xml --no-updates (START) [18:45:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:45:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:05:45] Aeotv [21:13:03] PROBLEM - fischipedia.org - LetsEncrypt on sslhost is CRITICAL: No address associated with hostnameHTTP CRITICAL - Unable to open TCP socket [22:43:37] PROBLEM - cloud17 APT on cloud17 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [22:45:38] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 11 packages available for upgrade (2 critical updates).