[01:46:46] PROBLEM - check_gcsip on payments2002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [01:50:15] PROBLEM - check_gcsip on payments1002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [01:50:15] PROBLEM - check_gcsip on payments2001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [01:50:15] RECOVERY - check_gcsip on payments1001 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 1.764 second response time [01:50:17] RECOVERY - check_minfraud_primary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.057 second response time [01:50:17] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:17] RECOVERY - check_minfraud_secondary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.036 second response time [01:50:37] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:53:16] PROBLEM - check_gcsip on payments2002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [01:53:20] RECOVERY - check_gcsip on payments2003 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 4.700 second response time [01:53:20] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:53:29] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 3.975 second response time [01:53:47] RECOVERY - check_gcsip on payments1004 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 4.442 second response time [01:53:47] RECOVERY - check_minfraud_primary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 2.015 second response time [01:53:49] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Cannot make SSL connection. [01:55:19] PROBLEM - check_gcsip on payments2001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [01:56:21] RECOVERY - check_gcsip on payments2002 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 9.524 second response time [01:56:31] RECOVERY - check_minfraud_primary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.607 second response time [01:58:21] RECOVERY - check_gcsip on payments2001 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 7.543 second response time [01:58:42] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.831 second response time [02:00:15] PROBLEM - check_gcsip on payments1002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:03:19] PROBLEM - check_gcsip on payments2002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:07:22] PROBLEM - check_gcsip on payments1003 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:07:22] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:10:17] PROBLEM - check_gcsip on payments2001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:10:17] PROBLEM - check_gcsip on payments1001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:10:17] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [02:10:18] PROBLEM - check_gcsip on payments1002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:10:18] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [02:10:37] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [02:12:27] RECOVERY - check_gcsip on payments1003 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 9.824 second response time [02:12:31] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:15:15] PROBLEM - check_gcsip on payments2001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:16:11] RECOVERY - check_gcsip on payments2002 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.160 second response time [02:16:17] RECOVERY - check_minfraud_primary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.029 second response time [02:16:17] PROBLEM - check_gcsip on payments1004 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:16:19] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:16:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:16:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:16:21] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.281 second response time [02:18:11] RECOVERY - check_gcsip on payments2001 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.168 second response time [02:19:07] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Cannot make SSL connection. [02:19:09] RECOVERY - check_minfraud_secondary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.061 second response time [02:20:15] PROBLEM - check_gcsip on payments1002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:20:17] PROBLEM - check_gcsip on payments1004 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:20:19] RECOVERY - check_minfraud_primary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.231 second response time [02:24:27] RECOVERY - check_gcsip on payments1004 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 8.111 second response time [02:25:17] PROBLEM - check_gcsip on payments2003 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:26:17] PROBLEM - check_gcsip on payments1001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:26:17] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:26:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [02:28:11] RECOVERY - check_gcsip on payments2003 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.159 second response time [02:28:11] PROBLEM - check_gcsip on payments2002 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:28:17] RECOVERY - check_gcsip on payments1002 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 1.596 second response time [02:28:17] RECOVERY - check_gcsip on payments1001 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 1.560 second response time [02:28:17] PROBLEM - check_gcsip on payments1003 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [02:28:18] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.044 second response time [02:28:18] RECOVERY - check_minfraud_secondary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.054 second response time [02:36:11] RECOVERY - check_gcsip on payments2002 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.162 second response time [02:36:17] RECOVERY - check_gcsip on payments1003 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 1.611 second response time [03:35:21] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:35:21] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:38:21] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [03:38:21] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [03:38:57] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [03:39:21] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:43:51] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.048 second response time [03:44:51] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:46:27] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:46:27] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:48:19] RECOVERY - check_minfraud_primary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.046 second response time [03:48:21] RECOVERY - check_minfraud_secondary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 3.061 second response time [03:49:17] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 1.552 second response time [03:49:17] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:49:17] PROBLEM - check_gcsip on payments1003 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [03:49:21] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:50:51] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:51:21] RECOVERY - check_gcsip on payments1003 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.951 second response time [03:51:22] RECOVERY - check_minfraud_primary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.039 second response time [03:54:25] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [03:54:49] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:58:21] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [03:58:21] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [04:00:47] PROBLEM - check_gcsip on payments1001 is CRITICAL: CRITICAL - Socket timeout after 121 seconds [04:00:57] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [04:01:21] RECOVERY - check_minfraud_primary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.048 second response time [04:01:21] RECOVERY - check_minfraud_secondary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 1.556 second response time [04:03:11] RECOVERY - check_gcsip on payments1001 is OK: HTTP OK: HTTP/1.1 200 OK - 343 bytes in 0.428 second response time [04:03:17] RECOVERY - check_minfraud_secondary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.038 second response time [04:03:17] RECOVERY - check_minfraud_primary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.034 second response time [04:03:17] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.051 second response time [18:20:50] (CR) Vedmaka Wakalaka: "> Patch Set 36:" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/517931 (https://phabricator.wikimedia.org/T124969) (owner: Vedmaka Wakalaka)