[00:02:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:03:19] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 3.475 second response time [00:03:53] RECOVERY - check_minfraud_primary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 1.539 second response time [00:05:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:07:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:08:21] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [00:10:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:10:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:11:53] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:14:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:14:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:14:19] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:14:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:14:20] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:14:20] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.056 second response time [00:14:40] Fundraising-Backlog, fundraising-tech-ops: Pats has gotten a new yubikey and needs the public key updated for access. - https://phabricator.wikimedia.org/T232640 (Dwisehaupt) Resolved→Open With @Ppena 's new machine came the need for a new ssh key. I walked her through the steps of creating a new... [00:15:30] Fundraising-Backlog, fundraising-tech-ops: Pats has gotten a new yubikey and machine and needs the public keys for ssh and the yubikey updated for access. - https://phabricator.wikimedia.org/T232640 (Dwisehaupt) [00:18:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:18:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:18:49] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:19:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:19:49] hello [00:20:12] dstrine: looks like 'the scrape' is working now [00:20:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:20:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:22:23] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:22:23] RECOVERY - check_minfraud_secondary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 3.037 second response time [00:23:53] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [00:24:51] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:25:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:25:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:25:28] (CR) Eileen: [C: +2] End of year summary: declare implicit properties [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538714 (https://phabricator.wikimedia.org/T195907) (owner: Ejegg) [00:27:23] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:30:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:32:23] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:32:23] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:34:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:34:21] RECOVERY - check_minfraud_secondary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 3.081 second response time [00:35:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:35:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:35:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:35:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:38:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:38:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:39:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:39:13] RECOVERY - check_minfraud_secondary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.066 second response time [00:39:19] RECOVERY - check_minfraud_primary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 3.104 second response time [00:39:53] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:43:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:43:21] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:44:21] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:45:19] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:45:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:45:21] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:46:31] (CR) Cstone: [C: +2] Adyen: add another ignored audit line type [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/538346 (https://phabricator.wikimedia.org/T233463) (owner: Ejegg) [00:46:59] RECOVERY - check_minfraud_secondary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 3.050 second response time [00:47:09] (Merged) jenkins-bot: Adyen: add another ignored audit line type [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/538346 (https://phabricator.wikimedia.org/T233463) (owner: Ejegg) [00:49:49] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:50:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:50:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:50:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:50:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:50:44] (CR) Krinkle: [C: +2] "Thanks!" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538676 (https://phabricator.wikimedia.org/T233538) (owner: TheSandDoctor) [00:52:23] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:54:19] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 3.092 second response time [00:54:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:54:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:54:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [00:55:13] PROBLEM - check_puppetrun on mintaka is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:56:23] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: connect to address minfraud-us-east.maxmind.com and port 443: Cannot assign requested address [00:56:23] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [00:59:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [01:00:13] PROBLEM - check_puppetrun on mintaka is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [01:00:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:00:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:00:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:00:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:03:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:03:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:04:23] RECOVERY - check_minfraud_primary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 3.042 second response time [01:04:23] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:05:13] PROBLEM - check_puppetrun on mintaka is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [01:05:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:05:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:05:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:05:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:06:43] (Merged) jenkins-bot: CentralNotice: Replace deprecated editToken with csrfToken [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538676 (https://phabricator.wikimedia.org/T233538) (owner: TheSandDoctor) [01:08:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:08:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:08:55] fundraising-tech-ops, Operations, netops: possible routing issue between eqiad and Maxmind network - https://phabricator.wikimedia.org/T233672 (Jgreen) [01:09:53] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:10:13] RECOVERY - check_puppetrun on mintaka is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [01:10:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:10:13] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:10:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:12:04] fundraising-tech-ops, Operations, netops: possible routing issue between eqiad and Maxmind network - https://phabricator.wikimedia.org/T233672 (Jgreen) p:Triage→Unbreak! Flipping this to "Unbreak Now!" since it's a timely issue, and service outage interfering with the donation pipeline. We do... [01:13:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:13:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:14:46] (CR) jenkins-bot: CentralNotice: Replace deprecated editToken with csrfToken [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538676 (https://phabricator.wikimedia.org/T233538) (owner: TheSandDoctor) [01:15:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:15:13] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:15:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:15:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:15:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:15:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:18:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:18:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:15] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:17] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:20:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:23:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:23:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:25:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:28:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:28:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:13] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:14] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:14] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:30:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:33:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:33:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:13] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:13] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:14] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:35:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:38:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:38:21] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:19] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:40:21] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:43:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:43:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:20] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:45:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:48:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:48:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:13] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:13] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:14] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:50:19] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:53:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:53:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:13] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:19] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:19] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:20] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:55:20] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:58:19] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [01:58:19] PROBLEM - check_minfraud_secondary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:15] PROBLEM - check_minfraud_primary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:17] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:17] PROBLEM - check_minfraud_primary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:19] PROBLEM - check_minfraud_secondary on payments1002 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:19] PROBLEM - check_minfraud_primary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:00:21] PROBLEM - check_minfraud_secondary on payments1004 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:02:39] PROBLEM - check_minfraud_primary on payments1003 is CRITICAL: CRITICAL - Socket timeout after 61 seconds [02:03:11] fundraising-tech-ops, Operations, netops: possible routing issue between eqiad and Maxmind network - https://phabricator.wikimedia.org/T233672 (ayounsi) [02:03:13] RECOVERY - check_minfraud_primary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.030 second response time [02:03:13] RECOVERY - check_minfraud_primary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.055 second response time [02:03:14] PROBLEM - check_minfraud_secondary on payments1001 is CRITICAL: connect to address minfraud-us-west.maxmind.com and port 443: Cannot assign requested address [02:03:15] RECOVERY - check_minfraud_secondary on payments1002 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.047 second response time [02:03:17] RECOVERY - check_minfraud_primary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.044 second response time [02:03:19] RECOVERY - check_minfraud_secondary on payments1004 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.038 second response time [02:03:46] fundraising-tech-ops, Operations, netops: possible routing issue between eqiad and Maxmind network - https://phabricator.wikimedia.org/T233672 (ayounsi) a:ayounsi All those IPs are behind Cloudflare. Opened a ticket with them. [02:06:19] RECOVERY - check_minfraud_primary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 787 bytes in 0.038 second response time [02:06:19] RECOVERY - check_minfraud_secondary on payments1003 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.092 second response time [02:08:13] RECOVERY - check_minfraud_secondary on payments1001 is OK: HTTP OK: Status line output matched 200,301,302 - 662 bytes in 0.044 second response time [02:47:57] fundraising-tech-ops, Operations, netops: possible routing issue between eqiad and Maxmind network - https://phabricator.wikimedia.org/T233672 (ayounsi) Open→Resolved Resolved by Cloudflare. [03:44:58] (PS1) Ejegg: EOY receipt: Quit filtering big query with function [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538727 (https://phabricator.wikimedia.org/T195907) [04:17:55] (PS1) Ejegg: WIP: Update SmashPig and related deps [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538729 (https://phabricator.wikimedia.org/T233463) [05:49:54] (CR) Eileen: "ejegg I've been looking at this but I can't see where totals is assigned." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538712 (https://phabricator.wikimedia.org/T195907) (owner: Ejegg) [08:03:07] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [08:11:58] (CR) Raimond Spekking: [C: +2] "false positive" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [08:13:10] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [08:22:24] (CR) jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538775 (owner: L10n-bot) [08:23:25] (CR) Raimond Spekking: [C: +2] "recheck" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [08:24:18] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [08:24:20] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [09:32:06] Fundraising-Backlog: French Thank You Email Typo - https://phabricator.wikimedia.org/T233620 (CDenes_WMF) oh neat! I thought we had a permissions blocker, but if that's not the case, I am happy to test out the new process ! [12:22:43] (CR) Raimond Spekking: [C: +2] "recheck" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [12:23:32] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [13:05:21] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [13:13:52] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, Wikimedia-Fundraising-Banners, FR-dlocal, Patch-For-Review: dLocal: on donation form, show Netbanking / UPI choice sooner in flow? - https://phabricator.wikimedia.org/T231452 (jgleeson) a:jgleeson [13:54:41] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, Wikimedia-Fundraising-Banners, FR-dlocal, Patch-For-Review: dLocal: on donation form, show Netbanking / UPI choice sooner in flow? - https://phabricator.wikimedia.org/T231452 (jgleeson) When the user arrives at the payments form, would we... [14:51:21] Fundraising-Backlog, fundraising-tech-ops: fundraising dev/database access request for Mariana Suijkerbuijk - https://phabricator.wikimedia.org/T232633 (MSuijkerbuijk_WMF) Hi Dallas, I've followed the steps now. Terminal says: Your identification has been saved in /Users/marianasuijkerbuijk/.ssh/id_r... [14:58:44] Fundraising-Backlog, Operations, SRE-Access-Requests: Banner History and page view data access for fundraising analysts - Jerrie and Erin - https://phabricator.wikimedia.org/T233636 (herron) p:Triage→Normal Hello! Once you have confirmed the details of the request, could you please create su... [15:06:23] (CR) Ejegg: "ah, it's not yet assigned! I supposed I should do that in this patch" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538712 (https://phabricator.wikimedia.org/T195907) (owner: Ejegg) [15:31:38] hey ejegg looking at doing this civi deploy there are a bunch of eileens patches in there, any reason they shouldn't be deployed? [15:32:00] hi cstone! [15:32:05] most of those are test fixes [15:32:31] so probably will all be deleted for the deployment branch [15:32:42] anyway, no need to fear. proceed! [15:33:01] hah okay [15:33:04] Then assuming all goes well i'm going to go ahead with a library update deploy [15:33:40] that one will pull in a year or so worth of changes to DonationInterface so I'll want to isolate it and watch it closely [15:34:06] specifically the old-globalcollect-recurring and the paypal orphan rectifier [15:45:22] ejegg question on the tests, a lot were deleted but it looks like there are 3 new ones, should I remove them? [15:45:42] cstone: yes please, just amend the merge [15:45:49] ok [15:49:48] (PS1) Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/538912 [15:54:00] Fundraising-Backlog: French Thank You Email Typo - https://phabricator.wikimedia.org/T233620 (Ejegg) So you can submit the fix to gerrit, it's just that for now someone in fr-tech will have to pull the fix on to staging to send a test email. [15:54:35] (CR) Cstone: [C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/538912 (owner: Cstone) [16:00:31] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, FR-Q2-FY2019-20-cleanup-list, Recurring-Donations, and 2 others: Annual totals receipt thank you email for recurring donors - https://phabricator.wikimedia.org/T195907 (CCogdill_WMF) Yes, omit endowment donations @Ejegg! [16:02:52] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, FR-Q2-FY2019-20-cleanup-list, Recurring-Donations, and 2 others: Annual totals receipt thank you email for recurring donors - https://phabricator.wikimedia.org/T195907 (CCogdill_WMF) And @Mbeat33 's suggestion above is fine for me, though I a... [16:08:23] hmm ejegg were these the tests that were broken for civi? [16:14:04] cstone they weren't quite cleaning up all the donations and contacts they created [16:14:16] which made subsequent test runs fail sometimes [16:14:20] ah nice [16:36:19] ejegg do we have to force the merge on these? was it an issue with the php version? [16:42:16] RECOVERY - check_swap on frav1002 is OK: SWAP OK - 97% free (7330 MB out of 7628 MB) [16:48:21] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, FR-Q2-FY2019-20-cleanup-list, Recurring-Donations, and 2 others: Annual totals receipt thank you email for recurring donors - https://phabricator.wikimedia.org/T195907 (CCogdill_WMF) Okay, from meeting with Elliott, it sounds like the multi-c... [16:50:11] cstone maybe still yeah we have to force merge [16:54:51] (CR) Cstone: [V: +2 C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/538912 (owner: Cstone) [17:08:43] hm does the submit button finally merge it then? [17:09:38] cstone yep, the submit button [17:13:50] !log civicrm revision changed from 5def62ab05 to feca96a2e3 [17:13:53] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:16:46] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, FR-Q2-FY2019-20-cleanup-list, Recurring-Donations, and 2 others: Annual totals receipt thank you email for recurring donors - https://phabricator.wikimedia.org/T195907 (Ejegg) Here are the language statistics for people with a recurring donat... [17:23:06] Fundraising-Backlog: Update IBM with TargetSmart Age data - https://phabricator.wikimedia.org/T233744 (KHaggard) [17:24:34] Fundraising-Backlog: Update IBM with TargetSmart Age data - https://phabricator.wikimedia.org/T233744 (KHaggard) p:Triage→High [17:27:11] Fundraising-Backlog: Update IBM with TargetSmart Age data - https://phabricator.wikimedia.org/T233744 (CCogdill_WMF) [17:37:36] (PS2) Ejegg: Update SmashPig and related deps [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538729 (https://phabricator.wikimedia.org/T233463) [17:38:00] well, recurring test looks good [17:38:18] I'mma go ahead and push those dependency updates to production Civi [17:38:55] (CR) Ejegg: [C: +2] Update SmashPig and related deps [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538729 (https://phabricator.wikimedia.org/T233463) (owner: Ejegg) [17:44:39] (PS1) Ejegg: Update SmashPig and related dependencies [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/538933 [17:45:01] (Merged) jenkins-bot: Update SmashPig and related deps [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538729 (https://phabricator.wikimedia.org/T233463) (owner: Ejegg) [17:46:59] (CR) Ejegg: [V: +2 C: +2] Update SmashPig and related dependencies [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/538933 (owner: Ejegg) [17:51:12] fr-tech a listening suggestion for today: the Sin Fronteras compliation by Apocalipsis records (All proceeds go to immigrant rights orgs): https://apocalipsistoday.bandcamp.com/album/sin-fronteras [17:52:27] Cool, I'll check it out. [17:53:18] (Merged) jenkins-bot: Update SmashPig and related dependencies [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/538933 (owner: Ejegg) [17:53:41] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/538935 [17:53:44] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/538935 (owner: Ejegg) [18:08:25] !log updated Fundraising CiviCRM feca96a2e3 to 52d2a24404 [18:08:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:19:10] (CR) Raimond Spekking: [C: +2] "@Ejegg: Could you override Jenkins pls?" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [18:22:32] ah, library out-of-dateness is screwing with DonationInterface tests [18:22:41] I'll update those then [18:24:50] good to do before the holidays [18:25:48] (PS2) XenoRyet: WIP No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 [18:26:34] (CR) jerkins-bot: [V: -1] WIP No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (owner: XenoRyet) [18:28:34] (PS1) Ejegg: Update libraries [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/538940 [18:28:49] (CR) Ejegg: [V: +2 C: +2] Update libraries [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/538940 (owner: Ejegg) [18:33:12] hey hey fr-tech :) [18:33:20] hi hi jgleeson [18:33:30] helllo jgleeson [18:33:35] Howdy [18:34:22] hi jgleeson [18:35:55] (Merged) jenkins-bot: Update libraries [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/538940 (owner: Ejegg) [18:38:12] (CR) jenkins-bot: Update libraries [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/538940 (owner: Ejegg) [18:41:13] fr-tech any objections to me deploying the most recent FRUEC? [18:41:20] None here [18:41:41] (PS3) XenoRyet: WIP No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 [18:42:04] XenoRyet: thanks! [18:43:29] (CR) jerkins-bot: [V: -1] WIP No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (owner: XenoRyet) [18:45:06] !log updated fruec from fb29cb74 to 97128874bf [18:45:08] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:49:08] AndyRussG: exiting stuff! FRUEC making it out to the wild :) [18:49:36] exciting* even [18:50:42] (PS4) XenoRyet: No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) [18:52:29] (CR) jerkins-bot: [V: -1] No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) (owner: XenoRyet) [18:57:54] (CR) jenkins-bot: Update libraries [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/538940 (owner: Ejegg) [19:07:22] jgleeson: yee release the floodgates [19:07:29] or should I say, the bug-gates? [19:08:06] jgleeson: fr-tech: for a first run of FRUEC I'll limit it to one day of logs. Ready for CR in localsettings is 9e5176bcce6ed59d [19:08:23] fr-tech .....and I have to miss standup :( e-mail forthcoming [19:12:23] >( [19:15:09] XenoRyet: what do you think of doing the variant with CSS instead of with JS? I'm not a big fan of the form changing on page ready [19:15:32] sorry to hear that AndyRussG [19:16:02] ejegg: oh it's ok... It's a gradual process of normalization! [19:16:05] Back in a bit! [19:17:46] ejegg: I'm not clear on how variants work with different CSS, how would I get it to do that? [19:18:18] I like the notion, I just didn't know it was a thing variants could do. [19:19:56] CSS is served up by resourceloader as part of modules, just like javascript [19:28:35] !log updated payments-wiki from 939b771800 to 5193dcdfa9 [19:28:37] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:02:27] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, FR-Adyen: Adyen audit parser fails on new line type - https://phabricator.wikimedia.org/T233463 (Ejegg) Open→Resolved [20:24:39] fr-tech or maybe the change mentioned in localsettings isn't subject to CR? Again, it's 9e5176bcce6ed59d, pushed to the repo on frpm1001 [20:24:51] It's just to make FRUEC only process one day of logs on the first go [20:25:09] (CR) jerkins-bot: [V: -1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [20:25:12] Let me take a look real fast. [20:25:44] AndyRussG i feel like I missed the beginning of that thought [20:26:44] mepps: just mentioned in my message earlier ^ a bit more than an hour ago [20:27:08] oh i see it now AndyRussG [20:27:12] It's just a temporary change to the fruec job to limit how long it'll run for on a first manual trial run [20:27:26] just in case something fails badly [20:28:25] XenoRyet: thanks much! [20:28:44] XenoRyet: you can run /srv/fruec consume -h for info on the added command line option [20:28:54] (on civi1001) [20:29:44] Oh oops actually that doesn't work [20:31:25] so is gerrit down for just me or everyone? [20:31:39] mepps: Yea, is for me. [20:31:44] down that is [20:31:55] thanks XenoRyet [20:33:04] hah came here to check the same thing [20:33:53] XenoRyet: try instead on civi1001: PYTHONPATH=/srv/fruec; python3 /srv/fruec/bin/fruec consume - [20:33:56] argh no [20:33:58] XenoRyet: try instead on civi1001: PYTHONPATH=/srv/fruec; python3 /srv/fruec/bin/fruec consume -h [20:34:01] ^ that [20:36:23] AndyRussG: That didn't seem to work either, but I think it's ok. That settings change looks fine to me. Syntax looks good, and I'm willing to believe you that the 'to' option is there and works. [20:36:24] (PS40) Mepps: Campaign fallback [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/517931 (https://phabricator.wikimedia.org/T124969) (owner: Vedmaka Wakalaka) [20:36:26] (PS1) Mepps: Change name, default, and add documentation for MaxCampaignFallback [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/538967 (https://phabricator.wikimedia.org/T124969) [20:36:31] I'd say you can go ahead and push the change. [20:37:13] (CR) Cstone: [C: +2] "Looks good, zipcode lines up nicely in Chrome and Firefox." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/535614 (owner: Ejegg) [20:37:13] Fundraising Sprint Sysadmin Kane, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Small fixes needed for Campaign fallback - https://phabricator.wikimedia.org/T232859 (mepps) I went with MaxCampaignFallback because I thought it had a better ring to it than using Iterations. I also noticed th... [20:37:18] Oh, fr-tech, I'm going to miss -talk today. I've got a conflicting appointment. [20:37:45] XenoRyet: thanks much!! [20:43:13] does anyone want to take point on reviewing the last bit of campaign fallback and +2ing fr-tech? AndyRussG i'm happy for it to be you, but didn't want to assume [20:43:44] mepps: I'm happy to do it or also happy for anyone else to [20:44:05] Someone could check to see if the last patch set conforms to what was requested [20:44:29] Though before someone +2's the whole thing, I would like to have one final once-over, either I could do that or someone else [20:44:48] I should be around again, possibly my kids' mom will take the kids for a few days now [20:45:13] i'd love review on my documentation--it was interesting to try to write it for this feature! [20:45:50] mepps: link? [20:46:10] it's further up in chat but here: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/517931 [20:46:24] wrong link [20:46:32] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/538967 [20:49:45] mepps: okok thnks! [20:50:14] (CR) Ejegg: [C: +2] "recheck" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [20:50:35] I guess we weren't expecting any further work from the original author on that patch... [20:51:13] (CR) Ejegg: [C: +2] "Raimond Spekking, I think I got the repos into a state where they shouldn't fail tests for the same reason again - that last failure looke" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [21:20:06] (Merged) jenkins-bot: Forms CSS: less scroll, fix field widths, minor cleanup [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/535614 (owner: Ejegg) [21:20:26] thanks for the CR cstone! [21:21:00] no problem! [21:30:05] (PS2) Ejegg: End of year summary: code formatting [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538713 (https://phabricator.wikimedia.org/T195907) [21:30:07] (PS2) Ejegg: End of year summary: declare implicit properties [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538714 (https://phabricator.wikimedia.org/T195907) [21:30:55] (PS2) Ejegg: EOY receipt: Quit filtering big query with function [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538727 (https://phabricator.wikimedia.org/T195907) [21:32:01] (PS1) Ejegg: End of year template fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538974 (https://phabricator.wikimedia.org/T195907) [21:33:04] (PS2) Ejegg: Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/538787 (owner: L10n-bot) [21:33:59] (PS5) XenoRyet: No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) [21:34:02] (PS2) Ejegg: End of year template fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538974 (https://phabricator.wikimedia.org/T195907) [21:37:59] (CR) jerkins-bot: [V: -1] No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) (owner: XenoRyet) [21:40:17] (PS6) XenoRyet: No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) [21:41:47] (CR) jerkins-bot: [V: -1] No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) (owner: XenoRyet) [21:46:42] (PS7) XenoRyet: No Card Selector Ingenico Variant [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/537775 (https://phabricator.wikimedia.org/T224929) [22:01:39] (PS3) Ejegg: End of year template fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538974 (https://phabricator.wikimedia.org/T195907) [22:08:56] (Abandoned) Ejegg: End of year template fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/538712 (https://phabricator.wikimedia.org/T195907) (owner: Ejegg) [22:39:48] fundraising-tech-ops, Operations, ops-codfw: rack/setup/install frqueue2001 - https://phabricator.wikimedia.org/T232630 (Dwisehaupt) After updating our stretch boot environment, I was able to boot and install the OS on this host. Additionally it has been tied into puppet and gotten its updates. Verif... [22:40:13] fundraising-tech-ops, Operations, ops-codfw: rack/setup/install frqueue2001 - https://phabricator.wikimedia.org/T232630 (Dwisehaupt) [22:41:24] dwisehaupt: Hi! We're just trying to run FRUEC for the first time [22:41:32] using a process-control job [22:41:40] and we got a permission denied error for /etc/fruec.yaml [22:42:21] Fundraising-Backlog, fundraising-tech-ops: fundraising dev/database access request for Mariana Suijkerbuijk - https://phabricator.wikimedia.org/T232633 (Dwisehaupt) Hello Mariana, Thanks for generating the keypair. What I need pasted into the ticket is the contents of the file: /Users/marianasuijkerbui... [22:49:35] Fundraising-Backlog, fundraising-tech-ops: Access and onboarding for new fundraising analyst (contractor) - Erin Yener - https://phabricator.wikimedia.org/T232982 (Dwisehaupt) Hi @EYener, here is some more info To add your key into the keychain/agent, you would run this command adjusting the key file if... [22:58:52] fundraising-tech-ops: EPIC: prepare for Debian Buster for fundraising systems - https://phabricator.wikimedia.org/T218258 (Dwisehaupt) I have pulled down the netboot kernel portions for buster. I will start testing them in my vm setup. [23:01:39] AndyRussG: hiya. let me take a look [23:02:55] hmmm... i see that it's trying to load the relative path of ./fruec.yaml [23:03:06] is that what you expect it to do? [23:07:31] i noticed --config is not specified in the command options. so what i think is happening is that it's checking for the config file in the local directory and getting the error. then it isn't falling through to the one in the /etc dir. [23:07:44] at least that's my first guess [23:56:27] Fundraising-Backlog, fundraising-tech-ops: Pats has gotten a new yubikey and machine and needs the public keys for ssh and the yubikey updated for access. - https://phabricator.wikimedia.org/T232640 (Ppena) Hi all. When I try to access entering ssh frdev1001 I get a "Connection closed by remote host". Th...