[02:24:19] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1142763 [02:24:35] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1142763 (owner: 10Eileen) [02:25:26] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1142763 (owner: 10Eileen) [05:48:11] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi recur fail message: confirm it checks for Gr4vy recurrings - https://phabricator.wikimedia.org/T393497#10798834 (10MSuijkerbuijk_WMF) Adding @Cstone - please let's prioritise this investigation for active recurring. [06:37:25] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_41) - 10https://gerrit.wikimedia.org/r/1142829 (owner: 10L10n-bot) [08:06:13] 06Fundraising-Backlog: India failed transactions - https://phabricator.wikimedia.org/T393553 (10RKumar_WMF) 03NEW [11:41:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: India failed transactions - https://phabricator.wikimedia.org/T393553#10799872 (10jgleeson) [11:44:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi recur fail message: confirm it checks for Gr4vy recurrings - https://phabricator.wikimedia.org/T393497#10799882 (10jgleeson) a:03jgleeson [11:47:45] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi recur fail message: confirm it checks for Gr4vy recurrings - https://phabricator.wikimedia.org/T393497#10799898 (10jgleeson) I chatted with @MBeat33 on [[ https://wikimedia.slack.com/archives/C045WH0QYS2/p1746542734786209 | Sla... [11:55:50] 03Fundraising Sprint Impossible shepherds pie, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2021: Create WMF standalone development build - https://phabricator.wikimedia.org/T372005#10799981 (10jgleeson) [11:58:22] 03Fundraising Sprint Impossible shepherds pie, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2021: Update development and CI enviroments for standalone - https://phabricator.wikimedia.org/T276272#10799982 (10jgleeson) [11:58:23] 03Fundraising Sprint Impossible shepherds pie, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2021: Create WMF standalone development build - https://phabricator.wikimedia.org/T372005#10799983 (10jgleeson) [12:54:32] 06Fundraising-Backlog, 06FR-donorrelations: Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594 (10MBeat33) 03NEW [12:55:42] 06Fundraising-Backlog, 06FR-donorrelations: Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594#10800202 (10MBeat33) A second donor has reported this as well, Zendesk 1667332, and provided screenshots in the ticket. [13:01:08] 06Fundraising-Backlog, 06FR-donorrelations: Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594#10800215 (10krobinson) Two donors from Spain have also reported this, so the issue seems widespread where there is SEPA. [13:01:54] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594#10800231 (10jgleeson) [14:10:30] 06Fundraising-Backlog, 06Web-Team, 05FY2024-25 WE3.2 Donation Entry Points, 07Regression: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601 (10Pcoombe) 03NEW [14:17:57] 06Fundraising-Backlog, 06Web-Team, 05FY2024-25 WE3.2 Donation Entry Points, 07I18n, 07Regression: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10800560 (10A_smart_kitten) [14:21:14] ^ that's a web team task, have pinged them about it [14:26:24] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi recur fail message: confirm it checks for Gr4vy recurrings - https://phabricator.wikimedia.org/T393497#10800615 (10jgleeson) I think I've tracked this down. We're sending the 'Recur failure email' in two places due to a recent... [14:30:53] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi recur fail message: confirm it checks for Gr4vy recurrings - https://phabricator.wikimedia.org/T393497#10800645 (10jgleeson) fr-tech notes: - recurring donation https://civicrm.wikimedia.org/civicrm/contact/view/contributionre... [14:36:33] 06Fundraising-Backlog, 06Web-Team, 05FY2024-25 WE3.2 Donation Entry Points, 07I18n, 07Regression: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10800674 (10ovasileva) p:05Triage→03High [15:02:58] 06Fundraising-Backlog, 06Web-Team, 10WikimediaMessages, 05FY2024-25 WE3.2 Donation Entry Points, and 2 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10800790 (10Jdlrobson-WMF) I can replicate this locally but haven't got to bottom of it. Right now I'm... [15:03:26] whoa, nagios bomb [15:03:39] zombie raids and time sinks [15:04:14] Jeff_Green: got some zombie defenses ready? [15:12:34] lol [15:21:36] !log civicrm upgraded from 6ffbde61 to dc096105 [15:21:37] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:36:07] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: India failed transactions - https://phabricator.wikimedia.org/T393553#10800925 (10Damilare) Hi @RKumar_WMF, for the transaction 8fd101c8-991a-44de-a71a-a86b6427f6ae, I think the 93 failure was from the issuer for the payment method. I noticed there was a... [16:26:35] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Missing Gravy - Dlocal Stitch transaction in CiviCRM - https://phabricator.wikimedia.org/T393461#10801282 (10Damilare) The fix for this has been deployed. Here's one of the [transactions in CiviCRM](https://civicrm.wikimedia.org/civicrm/contact/view/contr... [17:30:03] 06Fundraising-Backlog, 06Web-Team, 10WikimediaMessages, 05FY2024-25 WE3.2 Donation Entry Points, and 2 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10801616 (10Jdlrobson-WMF) 05Open→03In progress a:03Jdlrobson-WMF [17:34:02] ejegg: just gonna grab a coffee and the I'll join the standup link? that ok for you [17:38:45] back [17:39:19] jgleeson: I'm in another meeting [17:40:22] np ejegg we can catch up on tech-talk then. I can make a start on the review before to get warmed up [17:42:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594#10801687 (10Ejegg) Example link showing the issue: https://payments.wikimedia.org/index.php/Special:GatewayChooser?currency=EUR&country=ES&payment_method=rtbt&pay... [17:45:30] fr-tech recurring meeting folks are curious about this issue with the form chooser not finding a form for recurring sepadirectdebit ^^^ . I don't see an obvious issue - payment_methods.yaml has recurring: true for rtbt [17:45:43] will add another test case after i'm off the call [17:53:45] 06Fundraising-Backlog, 06Web-Team, 10WikimediaMessages, 05FY2024-25 WE3.2 Donation Entry Points, and 2 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10801719 (10Jdlrobson-WMF) It has been broken since at least 18th April: https://web.archive.org/web/2... [17:54:58] 06Fundraising-Backlog, 06Community-Tech, 06Web-Team, 10WikimediaMessages, and 4 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10801732 (10Jdlrobson-WMF) [17:56:05] 06Fundraising-Backlog, 06Community-Tech, 06Web-Team, 10WikimediaMessages, and 4 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10801748 (10Jdlrobson-WMF) @tstarling I assume reverting this is not the ideal solution since the patch is pretty large but c... [17:57:39] ccccccfhrkhflengckbcdjtbjjktrrbjgijhhkidllle [17:58:22] oops [17:58:53] ejegg: that sepa thing looks like it's falling over when it gets to this check in the gatewaychooser Iea1f2017e2093c28a6d297cdad61b92db33db042 [17:59:03] argh [17:59:04] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/1135131/1/special/GatewayChooser.php [17:59:28] it's looking for a key of 'recurring' on the submethod that isn't in the array [18:00:04] oh dang, empty is the wrong check there [18:00:07] thanks jgleeson [18:00:17] np [18:00:23] I'll make it so no submethod key == use method-level config [18:02:45] thanks for the speedy find! [18:05:18] (03PS1) 10Ejegg: Fix submethod-level recurring in GatewayChooser [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1143152 (https://phabricator.wikimedia.org/T393594) [18:06:39] 06Fundraising-Backlog: Inaccurate employer relationship on Civi records - https://phabricator.wikimedia.org/T393632 (10MDemosWMF) 03NEW [18:07:41] (03CR) 10CI reject: [V:04-1] Fix submethod-level recurring in GatewayChooser [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1143152 (https://phabricator.wikimedia.org/T393594) (owner: 10Ejegg) [18:09:57] 06Fundraising-Backlog, 10ContentTranslation, 10MediaWiki-extensions-CentralNotice, 10MW-1.44-notes (1.44.0-wmf.28; 2025-05-06), and 3 others: This page is using the deprecated ResourceLoader module "mediawiki.Uri" on page load - https://phabricator.wikimedia.org/T380079#10801818 (10Jdlrobson-WMF) 05Op... [18:10:52] what is Phan complaining about??? [18:21:44] (03PS2) 10Ejegg: Fix submethod-level recurring in GatewayChooser [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1143152 (https://phabricator.wikimedia.org/T393594) [18:26:51] (03CR) 10Jgleeson: [C:03+2] "This fixes the sepa form link for me!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1143152 (https://phabricator.wikimedia.org/T393594) (owner: 10Ejegg) [18:27:25] thanks jgleeson ! [18:28:11] np [18:53:45] (03Merged) 10jenkins-bot: Fix submethod-level recurring in GatewayChooser [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1143152 (https://phabricator.wikimedia.org/T393594) (owner: 10Ejegg) [19:00:29] (03PS1) 10Jgleeson: Add has-active-recurring check to Recurring Consumer Recur fail email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1143168 (https://phabricator.wikimedia.org/T393497) [19:01:52] fr-tech relocating downstairs. be a few mins late to tech-talk [19:08:38] no rush jgleeson it's just us two [19:22:48] (03PS1) 10Ejegg: Add Special:DonorPortal to allowlist [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1143170 [19:23:00] (03PS2) 10Ejegg: Add Special:DonorPortal to allowlist [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1143170 [19:23:16] (03CR) 10Ejegg: [V:03+2 C:03+2] Add Special:DonorPortal to allowlist [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1143170 (owner: 10Ejegg) [19:32:39] 06Fundraising-Backlog, 10FR-Donor-portal, 10FR-email-preferences: Gracefully handle errors fetching donor summary information from Civiproxy - https://phabricator.wikimedia.org/T393639 (10Ejegg) 03NEW [20:24:46] ok, i'm going to deploy that gateway chooser fix [20:25:32] 14Fundraising Sprint: didAnyoneTryThis(), 14Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog: Create new Benevity import - https://phabricator.wikimedia.org/T359219#10802339 (10MDemosWMF) @Eileenmcnaughton Did another test for the soft credit fix, but it looks like it might be off. Some examples: Co... [20:25:33] (03PS1) 10Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1143184 [20:31:12] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 17.66, 20.36, 13.54 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [20:32:12] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 8.54, 20.85, 15.14 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [20:37:12] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 39.77, 21.84, 16.12 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [20:41:14] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 35.02, 26.38, 17.73 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [20:42:08] PROBLEM - check_load on fransw2001 is CRITICAL: LOAD CRITICAL - total load average: 21.26, 24.34, 18.51 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_load [20:42:10] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 19.84, 23.81, 18.74 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [20:47:10] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 4.89, 12.00, 15.11 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [20:57:09] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1143184 (owner: 10Ejegg) [20:57:12] 06Fundraising-Backlog: Test and remove user information fields in Trustly ACH flow - https://phabricator.wikimedia.org/T393647 (10EMartin) 03NEW [21:06:14] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 32.11, 21.85, 16.16 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [21:07:08] PROBLEM - check_load on fransw2001 is CRITICAL: LOAD CRITICAL - total load average: 21.62, 20.02, 15.54 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_load [21:07:10] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 23.42, 22.50, 17.06 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [21:09:55] 06Fundraising-Backlog, 06Community-Tech, 06Web-Team, 10WikimediaMessages, and 4 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10802547 (10Tgr) `wikimedia-skin-minerva-donate-banner-url` is technically always in English (it has a `{{notranslate}}` tag)... [21:11:14] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 18.79, 20.53, 17.38 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [21:12:12] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 11.69, 18.10, 16.75 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [21:16:14] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 8.10, 13.76, 15.43 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [21:16:50] 06Fundraising-Backlog, 06Community-Tech, 06Web-Team, 10WikimediaMessages, and 4 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10802580 (10Jdlrobson-WMF) The issue also exists with `{{PAGELANGUAGE}}`but not `{{CONTENTLANGUAGE}}`. I think the latter wou... [21:19:00] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1143184 (owner: 10Ejegg) [21:28:26] 06Fundraising-Backlog, 06Community-Tech, 06Web-Team, 10WikimediaMessages, and 4 others: Sidebar donate link targets are always in English - https://phabricator.wikimedia.org/T393601#10802599 (10Tgr) >>! In T393601#10802580, @Jdlrobson-WMF wrote: > The issue also exists with `{{PAGELANGUAGE}}`but not `{{CON... [21:29:15] !log payments-wiki upgraded from 822bac34 to fac09775 [21:29:16] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:36:14] RECOVERY - check_load on fransw2003 is OK: LOAD OK - total load average: 0.51, 5.46, 8.87 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [21:37:10] RECOVERY - check_load on fransw2001 is OK: LOAD OK - total load average: 0.18, 4.20, 7.86 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_load [21:37:10] RECOVERY - check_load on fransw2002 is OK: LOAD OK - total load average: 0.10, 3.55, 7.62 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [21:39:45] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10MW-1.45-notes (1.45.0-wmf.1; 2025-05-13): Recurring SEPA form issue - https://phabricator.wikimedia.org/T393594#10802643 (10Ejegg) OK, we have a fix for this out in production [21:50:32] ejegg: https://lab.civicrm.org/dev/core/-/issues/5490 [21:51:55] oh nice, more love for the monolog extension! [21:52:15] ejegg: yeah & if bundeled with standalone that will be handy for us I think [21:52:23] yeah [22:05:12] well, logs look fine after that deploy [22:05:16] might step away for a while [22:18:16] (03PS2) 10Jgleeson: WIP: Add has-active-recurring check to Recurring Consumer Recur fail email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1143168 (https://phabricator.wikimedia.org/T393497) [22:39:58] (03CR) 10CI reject: [V:04-1] WIP: Add has-active-recurring check to Recurring Consumer Recur fail email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1143168 (https://phabricator.wikimedia.org/T393497) (owner: 10Jgleeson)