[01:34:13] cstone: hey.... yeah pretty sure I have [01:34:17] not sure how far it got tho [05:05:26] (CR) AndyRussG: [C: +2] "Woohooo!!!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) (owner: Eileen) [05:18:41] (Merged) jenkins-bot: Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) (owner: Eileen) [08:47:35] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 1 [=1] [08:57:31] RECOVERY - check_log_messages on frav1002 is OK: OK [09:41:13] (CR) Aklapper: "Palak: Which docs (link!) ask you to ever "git push"?" [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/677809 (https://phabricator.wikimedia.org/T279326) (owner: Palak199) [12:31:14] (PS1) Palak199: Replace link() function with linkRenderer() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/691166 (https://phabricator.wikimedia.org/T279326) [12:35:15] (CR) Palak199: "> Patch Set 5:" [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/677809 (https://phabricator.wikimedia.org/T279326) (owner: Palak199) [12:50:24] (Abandoned) Aklapper: Replace deprecated Linker::link with linkRenderer [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/677809 (https://phabricator.wikimedia.org/T279326) (owner: Palak199) [13:20:33] Fundraising-Backlog, donate.wikimedia.org, fr-donorservices: Lithuanian donation page shows old currency and not EUR - https://phabricator.wikimedia.org/T282771 (MBeat33) [13:24:49] (CR) Matěj Suchánek: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/691166 (https://phabricator.wikimedia.org/T279326) (owner: Palak199) [13:30:13] (CR) Matěj Suchánek: [C: +1] Replace link() function with linkRenderer() [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/691166 (https://phabricator.wikimedia.org/T279326) (owner: Palak199) [13:46:51] Fundraising-Backlog, donate.wikimedia.org, fr-donorservices: Lithuanian donation page shows old currency and not EUR - https://phabricator.wikimedia.org/T282771 (MBeat33) Thanks, @Pcoombe [14:22:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: minFraud_endpoint_critical 7 [=1] [14:27:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 20 [=5],minFraud_endpoint_critical 15 [=1] [14:32:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 15 [=5],minFraud_endpoint_critical 16 [=1] [14:37:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 25 [=5],minFraud_endpoint_critical 16 [=1] [14:42:35] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 17 [=5],minFraud_endpoint_critical 16 [=1] [14:47:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 23 [=5],minFraud_endpoint_critical 16 [=1] [14:52:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 23 [=5],minFraud_endpoint_critical 16 [=1] [14:57:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 1 [=1],cURL_error 31 [=5],minFraud_endpoint_critical 16 [=1] [15:02:35] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 21 [=5],minFraud_endpoint_critical 16 [=1] [15:07:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 19 [=5],minFraud_endpoint_critical 16 [=1] [15:12:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 14 [=5],minFraud_endpoint_critical 16 [=1] [15:17:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 19 [=5],minFraud_endpoint_critical 16 [=1] [15:22:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 15 [=5],minFraud_endpoint_critical 16 [=1] [15:27:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 23 [=5],minFraud_endpoint_critical 16 [=1] [15:32:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 18 [=5],minFraud_endpoint_critical 16 [=1] [15:37:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 18 [=5],minFraud_endpoint_critical 16 [=1] [15:42:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 18 [=5],minFraud_endpoint_critical 16 [=1] [15:47:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 19 [=5],minFraud_endpoint_critical 16 [=1] [15:52:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 17 [=5],minFraud_endpoint_critical 16 [=1] [15:57:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 11 [=5],minFraud_endpoint_critical 16 [=1] [16:02:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 18 [=5],minFraud_endpoint_critical 16 [=1] [16:07:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 15 [=5],minFraud_endpoint_critical 16 [=1] [16:12:20] fr-tech: it looks like these log messages errors are related to resolution for maxmind. it appears that maxmind is hosted on cloudflare which we as an org are seeing issues with right now. [16:12:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 15 [=5],minFraud_endpoint_critical 16 [=1] [16:12:55] dwisehaupt: ah cool thanks :) [16:17:31] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 13 [=5],minFraud_endpoint_critical 16 [=1] [16:17:53] i'm going to ack the alert for now while we find out if it's possible to flush caches to get a response. [16:18:55] ACKNOWLEDGEMENT - check_log_messages on frav1002 is CRITICAL: CRITICAL: cURL_error 13 [=5],minFraud_endpoint_critical 16 [=1] Dwisehaupt known cloudflare dns resolution issue being worked. [16:36:33] Fundraising-Backlog, Analytics, Analytics-EventLogging, Analytics-Kanban, and 4 others: Determine which remaining legacy EventLogging schemas need to be migrated or decommissioned - https://phabricator.wikimedia.org/T282131 (Ottomata) [18:22:31] RECOVERY - check_log_messages on frav1002 is OK: OK [18:42:13] fr-tech: the cloudflare issues that were messing with resolution of maxmind have cleared. we should be back to normal now. [18:42:26] nice dwisehaupt [18:43:12] yei :) [18:43:44] here's their incident report if you care to read about it (not that there is a lot in there): https://www.cloudflarestatus.com/incidents/g0m9sp8tsy5r [18:44:53] magic apparently [18:45:56] it's magic until it isn't. :) [18:53:33] wg 8 [20:17:53] dwisehaupt: thx! [20:42:58] (PS1) Cstone: Do Not Merge! Remove private config [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/691280