[00:18:09] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 21.97, 85.45, 49.78 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:23:09] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 20.54, 42.78, 40.70 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:28:09] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 4.53, 17.49, 30.19 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:33:09] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 4.26, 9.63, 23.24 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:38:09] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 1.03, 6.25, 18.28 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:53:09] RECOVERY - check_load on franio2001 is OK: LOAD OK - total load average: 0.12, 1.61, 8.28 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [00:55:07] PROBLEM - check_load on franio2003 is CRITICAL: LOAD CRITICAL - total load average: 19.75, 27.31, 17.18 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2003&service=check_load [01:10:05] RECOVERY - check_load on franio2003 is OK: LOAD OK - total load average: 1.02, 2.32, 7.79 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2003&service=check_load [01:28:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 17.14, 63.04, 40.13 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [01:33:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 24.23, 38.81, 35.59 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [01:38:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 0.44, 14.37, 25.82 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [01:43:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 6.58, 8.42, 19.94 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [01:48:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 7.11, 16.28 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [02:03:09] RECOVERY - check_load on franio2002 is OK: LOAD OK - total load average: 4.19, 4.03, 8.55 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [02:21:11] PROBLEM - check_nginx on payments1006 is CRITICAL: PROCS CRITICAL: 152 processes with command name nginx https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1006&service=check_nginx [02:26:11] RECOVERY - check_nginx on payments1006 is OK: PROCS OK: 66 processes with command name nginx https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1006&service=check_nginx [06:22:11] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 99% used. Largest process: trino-server-co (1277933) = 98.2% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:27:13] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 99% used. Largest process: trino-server-co (1277933) = 98.2% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:32:13] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 99% used. Largest process: trino-server-co (1277933) = 98.2% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:37:13] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 99% used. Largest process: trino-server-co (1277933) = 98.0% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:42:13] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 95% used. Largest process: trino-server-co (1277933) = 94.4% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:47:13] RECOVERY - check_memory on fransw2002 is OK: OK Memory 55% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [11:15:45] (03CR) 10Jgleeson: "Looks good. I had a question about the exception switch." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1270037 (owner: 10Damilare Adedoyin) [11:44:35] thanks for the CR jgleeson! I'd work on it. [12:10:20] np damilare [12:21:48] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270421 [12:22:06] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270421 (owner: 10Jgleeson) [12:22:50] fr-tech I'm gonna deploy donor portal to get the android fixes out that wfan merged (thanks wfan!) [12:23:20] damilare: looks like a few of yours are in that release too [12:23:45] okk checking [12:24:20] lgtm jgleeson [12:24:39] 👍 [12:29:35] jgleeson: so the reason I used the ApiUsageException is because of the ability to return the custom error code in the response and compatibility with the MW Api. The invalid exception only returns internal_api_error_InvalidArgumentException which doesn't give so much detail [12:33:57] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270421 (owner: 10Jgleeson) [12:43:57] I'll take another look damilare [12:44:59] !log donorwiki upgraded from 064a770e to 26f5451a [12:45:00] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [12:49:41] thanks! [12:50:44] damilare: I can see that message translation feature is handy [12:51:12] I guess I was wondering if we could preserve the preciseness of the exception itself [12:51:42] but if it's not convenient stick with your new one [12:52:19] are we not bubbling up "Bad parameter '$email' - should be an email address." [12:54:26] Not really, its only being concatenated into a string in the info property of the error object [12:54:37] on the frontend* [12:55:55] got it [12:57:02] like this error: {..., info: "[35875505398189043521c6ea] Exception caught: invalid_email"} [13:06:17] yeah that's no use [13:10:21] whew, noisy weekend [13:10:26] (failmail-wise) [13:10:35] thanks for looking into those already jgleeson [13:10:51] ejegg: I think we might need to delete and redownload the adyen report [13:11:10] ok, i'll do that deletion with a one-off job [13:12:58] right, it breaks off in the middle of a line [13:14:55] ya [13:15:40] ok, deleted [13:15:57] so we need to queue up the download job again, I guess? [13:16:41] let's see if that json blob is logged in the process-control output, or if we have to go to the listener logs [13:16:57] oh, or is it right in the email? [13:17:10] yep, nice, in the email [13:18:12] ok, that should re-run soon [13:19:33] jgleeson: if you have a chance, want to take another look at https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/1267873 ? I added some checks to make sure we're not setting the ID to null [13:20:29] will do. just testing damilare's email validation patch [13:22:45] oh dang, wait, was that a gravy d/l job failing too? [13:23:07] that first one was def adyen, right? I hope so, I pushed it to the adyen jobs queue [13:23:17] (03CR) 10Jgleeson: [C:03+2] "Works great for me locally! thanks" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1270037 (owner: 10Damilare Adedoyin) [13:23:38] Thanks jgleeson ! [13:24:07] yep ejegg it was adyen [13:24:20] I tried requeing the failed jobs in the damaged queue and two were gravy [13:24:26] they are unrelated [13:24:42] ah ok, but that gravy one probs still needs a CLI requeue then [13:24:59] oh, but gravy d/l jobs are not really retryable because they're always expired if we retry [13:25:17] (03Merged) 10jenkins-bot: Highlight invalid input errors on DonorPortal login page [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1270037 (owner: 10Damilare Adedoyin) [13:25:32] so.... i forget how we're supposed to get those. Maybe run something in the UI as the reporting user? [13:25:45] (03CR) 10Jgleeson: [C:03+2] "Thanks for the updates! LGTM." [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1267873 (https://phabricator.wikimedia.org/T421277) (owner: 10Ejegg) [13:25:52] ty jgleeson [13:26:19] (03Merged) 10jenkins-bot: Set gatewayRefundID on refund responses [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1267873 (https://phabricator.wikimedia.org/T421277) (owner: 10Ejegg) [13:26:43] ok, looks like that adyen report is downloaded again [13:27:02] and has a complete last line now [13:36:50] nice [13:38:55] oh it's not actually passing in the payment method on that line [13:38:56] $this->isEmailSourceTrusted($context['currentPrimary']['location_type_id:name'] [13:38:59] ) [13:39:14] let me add a guard around that [13:39:46] I wonder what the primary email type IS for that person [13:41:01] huh, it's showing ACH-bank in the edit dropdown but maybe that's because nothing is chosen in the db? [13:41:06] let's do a query [13:41:56] yah, location_type_id=0 on the primary [13:42:07] let's see how many have that [13:53:55] (03PS5) 10Aude: Check edit permission for insert close button and banner translation UI [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1268090 (https://phabricator.wikimedia.org/T420810) [13:54:23] huh, api4 Log::get is not the log i want [13:56:09] boo, and api3 logging get needs a log_conn_id [13:56:12] ok, to the db i go [13:57:42] ok, that location_type_id was set to 0 in 2022 [13:57:46] so not a new problem [13:59:16] hmm, latest set to 0 was updated on 3/10/26 [13:59:22] aha, and the contact_id changed too [13:59:28] so it's a merge issue [14:00:50] https://phabricator.wikimedia.org/T420492 ? [14:03:07] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Improve CiviCRM dedupe to better handle multiple emails of the same location type - https://phabricator.wikimedia.org/T420492#11814477 (10Ejegg) Looks like merge is also sometimes leaving the location_type=0 `lang=sql select log_action, contact_id, locati... [14:03:26] ok, so do we have a datafix class here? [14:04:03] ohhhh that last batch was anonymous anonymous [14:04:20] odd that thwy would end up with email addresses [14:05:12] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Improve CiviCRM dedupe to better handle multiple emails of the same location type - https://phabricator.wikimedia.org/T420492#11814493 (10Ejegg) Oh wait, that last example was merged into Anonymous Anonymous. Seeing a few more too. I wonder why, if they h... [14:08:00] ok, so the anon contact has just run out of email location types, which is why they get set to 0 [14:14:41] scanning for others being set to 0 this year, it looks like they mostly get restored (to e.g. type 6) on the same day as the merge [14:27:52] ohhh and that message was dropped because of a type error [14:28:11] I'll deploy the smashpig update to Civi so we start sending those to damaged [14:32:58] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 [14:33:00] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 (owner: 10Ejegg) [14:33:47] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 (owner: 10Ejegg) [14:34:17] (03PS1) 10Ejegg: Update SmashPig [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1270462 [14:34:19] (03CR) 10Ejegg: [C:03+2] Update SmashPig [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1270462 (owner: 10Ejegg) [14:34:31] (03PS2) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 [14:42:38] (03CR) 10Ejegg: [V:03+2 C:03+2] Update SmashPig [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1270462 (owner: 10Ejegg) [14:43:42] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Clean up emails with location_type_id=0 - https://phabricator.wikimedia.org/T423142 (10Ejegg) 03NEW [15:02:58] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Replace HEPdata employer matching data with something that updates a little more often - https://phabricator.wikimedia.org/T421296#11814922 (10Ejegg) Now might be a good time to look around - looks like our license has expired. "No valid license key found... [15:04:58] long UML diagrams don't look too good on mediwiki https://wikitech.wikimedia.org/wiki/Fundraising/Data_and_flow/PSP_integrations/Gravy#Gravy_Credit_Card_Activity_Diagram [15:05:27] I wonder if we could turn them into meraid.ai or plantuml links to open up the interactive diagram [15:19:09] did we turn the stripe connection back on? [15:19:42] 4 hrs ago there was another 'There is no active payment service for this transaction.' on a recurring charge [15:21:34] ah, paypal donor who donated on a US form, but has a CN address [15:24:23] I saw that one ejegg [15:27:56] think we could fix those with a small tweak [15:59:31] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations: Ensure country from form is saved to civicrm_contribution_recur_smashpig.original_country - https://phabricator.wikimedia.org/T423155 (10Ejegg) 03NEW [16:02:05] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Improve CiviCRM dedupe to better handle multiple emails of the same location type - https://phabricator.wikimedia.org/T420492#11815291 (10Lars) @Ejegg There is code on the merge hook that is supposed to prevent contacts ending up with more than one email... [16:03:45] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Improve CiviCRM dedupe to better handle multiple emails of the same location type - https://phabricator.wikimedia.org/T420492#11815297 (10Ejegg) Right, I was also wondering why we were merging contacts into the anonymous contact when we actually do have a... [16:04:58] larssandergreen: I was thinking of running the BlankLocation datafix script to deal with a chunk of those: https://phabricator.wikimedia.org/T423142 [16:05:39] does that seem like it's a good idea? Or likely to just choose confusing types? [16:05:42] https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/ext/org.wikimedia.datachecks/CRM/Datachecks/BlankLocation.php [16:06:03] ohhh that won't pick anything up anyway [16:06:08] it's only checking for NULL, not 0 [16:06:20] we have no nulls in the db [16:07:53] I'm not sure it really matters. It doesn't seem to harm anything to have 0s in there, they still show up in the UI as expected. [16:08:51] larssandergreen: it's breaking this bit of the new location type wrangling code [16:08:54] https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/ext/wmf-civicrm/Civi/Api4/Action/WMFContact/Save.php#L750 [16:09:00] $this->isEmailSourceTrusted($context['currentPrimary']['location_type_id:name']); [16:09:11] is sending NULL to isEmailSourceTrusted [16:09:28] we could fix the code to not crash [16:09:36] but it'd also be nice to get the data right, no? [16:13:13] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 (owner: 10Ejegg) [16:14:20] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270455 (owner: 10Ejegg) [16:14:45] (03PS1) 10Ejegg: BlankLocation datacheck: fix location_type_id=0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270489 (https://phabricator.wikimedia.org/T423142) [16:17:07] ejegg: fair. Are you thinking we recover the past location type from the logs or just set them all to 1 and call it done? I see 27k. [16:18:10] larssandergreen: That datacheck script should set them to the lowest number that isn't already in use for their contact [16:18:38] just needs this lil patch to make it consider id=0 not just null https://gerrit.wikimedia.org/r/1270489 [16:22:14] Sounds good [16:22:15] (03CR) 10Lars SG: [C:03+2] BlankLocation datacheck: fix location_type_id=0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270489 (https://phabricator.wikimedia.org/T423142) (owner: 10Ejegg) [16:23:15] thanks! [16:23:28] I'll just fold that into the current deploy [16:24:17] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270492 [16:24:20] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270492 (owner: 10Ejegg) [16:25:00] actually, I want to turn off the charge job to deploy the parent merge, so I'll just do that first [16:25:19] jobS plural that is! [16:26:58] !log disabled recurring donation charge jobs for code / settings update [16:27:12] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [16:28:18] ok, the jobs were already done for this cycle [16:29:24] !log fundraising civicrm upgraded from eb188fa2 to fa20eb0a [16:29:26] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [16:34:34] sorry if i'm causing failmail - just did a cv flush [16:34:48] maybe should have done something more targeted [16:34:55] just to get the managed and the menus [16:37:53] larssandergreen: any idea what could make a batch merge task in coworker fail with "DB Error: no database selected" ? [16:37:58] RPOW? [16:38:31] man, that flush command is dragging on [16:38:50] note to self: bad idea [16:44:39] ejegg: oh, CV flush is probably running into this: https://github.com/civicrm/civicrm-core/pull/35332 [16:44:42] just kill it [16:45:31] ok, will do [16:45:54] ah it just finished on its own finally [16:46:23] (03Merged) 10jenkins-bot: BlankLocation datacheck: fix location_type_id=0 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270489 (https://phabricator.wikimedia.org/T423142) (owner: 10Ejegg) [16:46:26] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1270492 (owner: 10Ejegg) [16:46:39] ohhh also larssandergreen when we start running refunds via the queue that'll probably hit permissions errors under coworker I think? [16:47:04] related to the batch version of that refund form: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1269005 [16:47:33] "no database selected" I don't know, would have to see a stack trace, I think as that's way down in the vendor code [16:47:34] I figured we would need to send >5 or so to the queue to avoid UI timeouts, so I added this API action: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1269005/5/ext/wmf-civicrm/Civi/Api4/Action/Contribution/RefundAndMarkIfFraud.php [16:48:00] but now I'm wondering what permissions I should put there [16:51:24] (03PS1) 10Ejegg: Allow env variable to change SmashPig stats names [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270495 [16:52:05] hmm, I need to remind myself where to look for better logs of coworker things [16:54:52] Jeff_Green: I've got a lil patch to maybe restore SmashPig stats, but I'm not sure if I just need to output to separate file names, or if we should output completely different stat names for the two jobs [16:56:31] ejegg: I think the refund permission makes sense there [16:56:45] larssandergreen: ok, and coworker won't barf on that? [16:57:03] or we can figure out how to give coworker that permission anyway? [16:57:45] ejegg: looking [16:58:08] Jeff_Green: the patch is just in review in gerrit, not a puppet thing yet [16:58:12] re. filename vs statname, I think if you have the same exact stat in two files, one is going to win [16:58:21] https://gerrit.wikimedia.org/r/1270495 [16:58:31] but all it takes to catch both is a different label [16:58:33] looking [16:58:48] ahhh right, a different label in the curly braces would be the way to go [16:58:50] all the stats across all the files get smashed together [16:58:54] yeah [16:59:00] so we can combine the error counts across both [16:59:05] ok, will redo that patch [16:59:42] (03CR) 10Ejegg: [C:04-1] "Will redo with the tag in the job-specific tag in curly braces, not changing the whole stat name" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270495 (owner: 10Ejegg) [16:59:55] ejegg: should be fine for coworker [17:00:02] thanks larssandergreen [17:01:34] oh cripes, cell-nucleus-level DRM: https://phys.org/news/2026-04-hackers-dna-encryption-cells.html [17:04:02] !log re-enabled recurring donation charge jobs [17:04:03] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [17:32:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Can't use Deduper with specific smart groups - https://phabricator.wikimedia.org/T423176 (10Lars) 03NEW [17:33:02] dwisehaupt: could you take a look at this one for a MariaDB setting change: ^^^ [17:39:11] 06Fundraising-Backlog: Update Fundraising Tech Hiring Task - https://phabricator.wikimedia.org/T422671#11816056 (10Damilare) [17:39:21] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog: Adyen Account Migration for WMF Staff - https://phabricator.wikimedia.org/T420198#11816059 (10Damilare) [17:39:22] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 05MW-1.46-notes (1.46.0-wmf.22; 2026-03-31): Upgrade FraudFilter field-reference comparisons to work with wildcards - https://phabricator.wikimedia.org/T421059#11816060 (10Damilare) [17:39:24] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-payments-ACH: Figure out what the ACH recurring failure flow looks like - https://phabricator.wikimedia.org/T420661#11816061 (10Damilare) [17:39:25] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07Epic, 07fr-combowiki, 07fr-current-sprint: Native Payments - https://phabricator.wikimedia.org/T415586#11816058 (10Damilare) [17:39:28] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Send double opt-in confirmation message whenever a contact's email is changed - https://phabricator.wikimedia.org/T407261#11816064 (10Damilare) [17:39:29] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-payments-ACH, 07payments-orchestration: Add support for a new parameter to show a Pending Thank You (TY) page - https://phabricator.wikimedia.org/T409500#11816063 (10Damilare) [17:39:31] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Epic, 07fr-current-sprint: Set double opt in activity to cancelled when double opted in contact opts out - https://phabricator.wikimedia.org/T417695#11816065 (10Damilare) [17:39:35] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, 13Patch-For-Review: Add token for CiviCRM emails to conditionally add Donor Portal link - https://phabricator.wikimedia.org/T419437#11816062 (10Damilare) [17:39:39] 03Fundraising Sprint G - 2026, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 4 others: Automate some Direct Mail white mail coding - https://phabricator.wikimedia.org/T386031#11816067 (10Damilare) [17:39:43] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, 07fr-current-sprint: Create Recurring Upgrade and Recurring Downgrade activities when completed by DR through CiviCRM UI - https://phabricator.wikimedia.org/T409097#11816068 (10Damilare) [17:39:47] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, and 2 others: Add token for recurring donation failure link - https://phabricator.wikimedia.org/T419046#11816066 (10Damilare) [17:39:52] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint, 13Patch-For-Review: Recurring failure email link improvements - https://phabricator.wikimedia.org/T415229#11816069 (10Damilare) [17:39:56] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 07fr-current-sprint: Improve automatic deduping - https://phabricator.wikimedia.org/T415806#11816070 (10Damilare) [17:40:00] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint, 05FY25-26 WE3.5 Donor Identification and recognition: CiviCRM is connected to MediaWiki - https://phabricator.wikimedia.org/T416950#11816071 (10Damilare) [17:40:04] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07payments-orchestration: Recurring Token Migration - Adyen - https://phabricator.wikimedia.org/T418619#11816072 (10Damilare) [17:40:08] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07payments-orchestration: Recurring Token Migration - Adyen Apple Pay and Google Pay - https://phabricator.wikimedia.org/T418760#11816074 (10Damilare) [17:40:12] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint, 07payments-orchestration: Recurring token migration - Paypal - https://phabricator.wikimedia.org/T405483#11816073 (10Damilare) [17:40:16] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Demote rather than overwrite non-Home email types when changing email in email preferences center - https://phabricator.wikimedia.org/T422051#11816075 (10Damilare) [17:40:20] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10FR-Donor-portal, 05MW-1.46-notes (1.46.0-wmf.22; 2026-03-31): Donor Portal - rename home link from "My donor portal" to "My donor account" - https://phabricator.wikimedia.org/T420610#11816076 (10Damilare) [17:40:24] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Adyen discrepancy - batch 1154 (caused by Gr4vy bugs) - https://phabricator.wikimedia.org/T420527#11816078 (10Damilare) [17:40:28] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Handle Braintree chargeback reversals - https://phabricator.wikimedia.org/T420556#11816080 (10Damilare) [17:40:34] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog: Fundraising Sprint F - 2026 - Priorities - https://phabricator.wikimedia.org/T420783#11816079 (10Damilare) [17:40:38] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 05MW-1.46-notes (1.46.0-wmf.23; 2026-04-07): Add appropriate consent/opt-in language to Payments form SMS capture - https://phabricator.wikimedia.org/T420271#11816081 (10Damilare) [17:40:42] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint, 05MW-1.46-notes (1.46.0-wmf.22; 2026-03-31), and 2 others: Integrate Pix Automatico to dlocal - https://phabricator.wikimedia.org/T419446#11816077 (10Damilare) [17:40:46] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` - https://phabricator.wikimedia.org/T420057#11816083 (10Damilare) [17:40:52] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07fr-current-sprint: CentralNotice - Scoping Immediate Issues - https://phabricator.wikimedia.org/T417064#11816082 (10Damilare) [17:40:56] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint, 05MW-1.46-notes (1.46.0-wmf.22; 2026-03-31), 07payments-orchestration: Add VIPPS in Norway - https://phabricator.wikimedia.org/T390870#11816084 (10Damilare) [17:41:00] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint: Spike: Fraud Tools - https://phabricator.wikimedia.org/T408750#11816086 (10Damilare) [17:41:04] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Chariot import - get api access and investigate pulling in DAF (in particular PayPal grants) directly - https://phabricator.wikimedia.org/T415825#11816087 (10Damilare) [17:41:10] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 05MW-1.46-notes (1.46.0-wmf.21; 2026-03-24): Filtering banners on campaign page resets "Limit traffic" checkbox and "Campaign type" - https://phabricator.wikimedia.org/T182343#11816085 (10Damilare) [17:41:14] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07fr-current-sprint: Channel sometimes empty for extra refunds / adjustments / chargebacks - https://phabricator.wikimedia.org/T414286#11816088 (10Damilare) [17:41:18] 03Fundraising Sprint G - 2026, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 2 others: Add Searchkit with daily email notifications / dashlet for donations missing required details - https://phabricator.wikimedia.org/T412463#11816090 (10Damilare) [17:41:22] 03Fundraising Sprint G - 2026, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10FR-WMF-Audit, and 2 others: Document Audit Settlement process - https://phabricator.wikimedia.org/T409995#11816089 (10Damilare) [17:41:28] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 14Fundraising-Backlog-Old, 10MediaWiki-extensions-DonationInterface, and 2 others: Some messages in the Donation extensions are outdated and should be removed - https://phabricator.wikimedia.org/T111677#11816092 (10Damilare) [17:41:32] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07fr-current-sprint: Handle Gravy SEPA payment_service_transaction_id change - https://phabricator.wikimedia.org/T416330#11816093 (10Damilare) [17:41:36] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: We retry failed recurring SEPA donations forever if the reason is retryable - https://phabricator.wikimedia.org/T420775#11816095 (10Damilare) [17:41:40] 03Fundraising Sprint G - 2026, 14Fundraising Sprint Michelin Star, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, and 2 others: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#11816091 (10Damilare) [17:41:44] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10FR-Smashpig: Create properties for auth and capture ID (and backend processor equivalents) on SmashPig responses - https://phabricator.wikimedia.org/T421380#11816096 (10Damilare) [17:41:52] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10donate.wikimedia.org, 07Accessibility, 07fr-current-sprint: Accessibility issues - Donate wiki - https://phabricator.wikimedia.org/T410586#11816094 (10Damilare) [17:41:56] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog: FR-Tech AI Safe Usage Guide - https://phabricator.wikimedia.org/T421487#11816097 (10Damilare) [17:42:00] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog: Actually populate auth id and capture id columns - https://phabricator.wikimedia.org/T420987#11816100 (10Damilare) [17:42:06] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Add an automatic default for $wgNoticeProjects in CentralNotice - https://phabricator.wikimedia.org/T419627#11816099 (10Damilare) [17:42:11] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add fields to Civi to better reflect reporting -GL Code - https://phabricator.wikimedia.org/T420156#11816101 (10Damilare) [17:42:17] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 05MW-1.46-notes (1.46.0-wmf.22; 2026-03-31): Remove $wgCentralHost in favor of $wgCentralSelectedBannerDispatcher - https://phabricator.wikimedia.org/T419633#11816098 (10Damilare) [17:42:21] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Clear Acoustic contact list before exporting Civi group to Acoustic - https://phabricator.wikimedia.org/T419471#11816102 (10Damilare) [17:42:25] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Audit file size - https://phabricator.wikimedia.org/T418967#11816103 (10Damilare) [17:42:29] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 07payments-orchestration: Recurring Token Migrate - Adyen CC - https://phabricator.wikimedia.org/T418759#11816104 (10Damilare) [17:42:37] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog: Integrate Chariot to Civi for automated payment info updates - https://phabricator.wikimedia.org/T419044#11816107 (10Damilare) [17:42:41] 03Fundraising Sprint G - 2026, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Engage <> Gr4vy for Direct Mail CC and ACH - https://phabricator.wikimedia.org/T417842#11816108 (10Damilare) [17:42:45] 03Fundraising Sprint G - 2026, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 3 others: Ensure new is_major_gift & gl code fields are fully and correctly populated - https://phabricator.wikimedia.org/T409994#11816106 (10Damilare) [17:45:48] 03Fundraising Sprint - Floor is Lava, 06Fundraising-Backlog: Fundraising Sprint F - 2026 - Priorities - https://phabricator.wikimedia.org/T420783#11816151 (10Damilare) 05Open→03Resolved [17:46:55] 06Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface: Refactor session use in DonationInterface - https://phabricator.wikimedia.org/T417831#11816159 (10Damilare) [17:47:50] sorry for the phab spam, just adding tags to the tickets in the sprints. [17:52:25] (03PS2) 10Ejegg: Allow env variable to change SmashPig stats output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270495 [17:52:31] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Clone button for CN campaigns - https://phabricator.wikimedia.org/T91078#11816185 (10Soylacarli) Hi! Is there any update on this? Or is there a way we can help move this feature forward? I haven't been a CN admin for long, but even in this short time... [17:56:41] damilare|brb: what are the log paths for donor portal? I wanted to add them to logdogs searchable paths to make it easy to track down any errors [18:08:18] (03PS3) 10Ejegg: Allow env variable to change SmashPig stats output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270495 [18:08:39] cstone: there wasn't a phab ticket to restore those nightly charge stats, was there? [18:08:50] not yet ejegg [18:09:20] ok . That patch should let us distinguish them with an environment variable anyway: https://gerrit.wikimedia.org/r/1270495 [18:12:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CiviCRM sets wrong email as display name for contact without a name - https://phabricator.wikimedia.org/T423181 (10Lars) 03NEW [18:14:02] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Can't preview monthly convert email in CiviCRM - https://phabricator.wikimedia.org/T423182 (10Lars) 03NEW [18:20:13] !log fundraising civicrm upgraded from fa20eb0a to fccf9b3a [18:20:14] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [18:21:06] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Can't use Deduper with specific smart groups - https://phabricator.wikimedia.org/T423176#11816352 (10Lars) a:03Lars [18:21:52] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Civi using incorrect donor name - https://phabricator.wikimedia.org/T422957#11816357 (10Lars) a:03AnnWF [18:22:22] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CiviCRM quicksearch: Don't return multiple rows per contact if the contact has the same email twice - https://phabricator.wikimedia.org/T422969#11816369 (10Lars) a:03Lars [18:24:43] ok, i'm going to run that blank location fix from the CLI [18:26:17] (03CR) 10Wfan: "good idea~" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1269791 (https://phabricator.wikimedia.org/T420992) (owner: 10Wfan) [18:29:00] oh hey, Datachecks github needs an update [18:32:49] submitted PR [18:35:47] ah dang, getting about a million Undefined array key "phone_type_id" errors [18:35:56] shoot, I meant to make it just look at email [18:36:25] but there's no way to do that [18:37:23] ok, it completed [18:37:29] let's see what that did to the 0 counts [18:38:49] oh wow, it's down to 0 [18:39:02] I guess it ended up assigning SOMETHING to the anon contact emails [18:39:48] (03PS2) 10Wfan: Backfill contact with special location type [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1269791 (https://phabricator.wikimedia.org/T420992) [18:41:50] huh, ach. I guess it got to the end of the loop for that contact [18:46:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: CRM_Core_Exception from RemoteUpdate.php - https://phabricator.wikimedia.org/T423189 (10AnnWF) 03NEW [18:50:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: TypeError when payload is not an array - https://phabricator.wikimedia.org/T423191 (10AnnWF) 03NEW [18:52:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Clean up emails with location_type_id=0 - https://phabricator.wikimedia.org/T423142#11816521 (10Ejegg) Ran the DataChecks fix and it seems to have cleaned them all up [18:55:01] jgleeson|mtg: its frdata, so archive is like /srv/archive/frlog1002/logs/frdata-20260411.gz [18:57:39] (03PS1) 10Wfan: job message payload type check [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270516 (https://phabricator.wikimedia.org/T423191) [19:03:23] (03CR) 10Ejegg: [C:03+2] job message payload type check [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270516 (https://phabricator.wikimedia.org/T423191) (owner: 10Wfan) [19:03:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: DownloadReportJob has empty payload, should be array - https://phabricator.wikimedia.org/T423191#11816554 (10AnnWF) [19:03:56] (03Merged) 10jenkins-bot: job message payload type check [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270516 (https://phabricator.wikimedia.org/T423191) (owner: 10Wfan) [19:06:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: DownloadReportJob has empty payload, should be array - https://phabricator.wikimedia.org/T423191#11816561 (10Ejegg) Just to note that the thing I was saying we should fix is the damaged message UI, so it can display and re-submit nes... [19:07:38] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: DownloadReportJob has empty payload, should be an array - https://phabricator.wikimedia.org/T423191#11816564 (10AnnWF) [19:08:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Damaged message UI requeues job messages with empty payload - https://phabricator.wikimedia.org/T423191#11816572 (10Ejegg) [19:11:01] oh man, dropped the mouse and now the scroll wheel is acting up. Boo [19:43:24] ugh mouse droppings [19:45:01] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07FR-Fraud, 03GNU England Shaker dresser, 13Patch-For-Review: Allow DR to refund transactions from Civi and flag as fraud - https://phabricator.wikimedia.org/T421277#11816645 (10Ejegg) a:03Ejegg [19:47:55] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 03GNU England Shaker dresser: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` and populate them - https://phabricator.wikimedia.org/T420057#11816660 (10Cstone) [19:48:23] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 03GNU England Shaker dresser: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` and populate them - https://phabricator.wikimedia.org/T420057#11816665 (10Cstone) [19:52:31] 06Fundraising-Backlog: Spike: Fraud Tools - https://phabricator.wikimedia.org/T408750#11816674 (10Damilare) [20:03:47] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Damaged message UI breaks nested arrays on re-queue - https://phabricator.wikimedia.org/T205316#11816726 (10Ejegg) [20:05:07] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Damaged message UI requeues job messages with empty payload - https://phabricator.wikimedia.org/T423191#11816729 (10Ejegg) →14Duplicate dup:03T205316 [20:05:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Damaged message UI breaks nested arrays on re-queue - https://phabricator.wikimedia.org/T205316#11816731 (10Ejegg) [20:07:48] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Damaged message UI breaks nested arrays on re-queue - https://phabricator.wikimedia.org/T205316#11816736 (10SBurnett-WMF) p:05Triage→03Medium [20:08:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: CRM_Core_Exception from RemoteUpdate.php - https://phabricator.wikimedia.org/T423189#11816740 (10SBurnett-WMF) p:05Triage→03Low [20:09:04] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations: Ensure country from form is saved to civicrm_contribution_recur_smashpig.original_country - https://phabricator.wikimedia.org/T423155#11816742 (10SBurnett-WMF) p:05Triage→03High [20:09:34] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Fill in missing CiviCRM core caching - https://phabricator.wikimedia.org/T422552#11816743 (10SBurnett-WMF) p:05Triage→03Medium [20:10:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Outreach to the donors from T420651 - https://phabricator.wikimedia.org/T421534#11816744 (10SBurnett-WMF) p:05Triage→03High [20:10:52] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Ensure recurring upgrade eligibility between Acoustic export and CiviCRM is consistent - https://phabricator.wikimedia.org/T421184#11816746 (10SBurnett-WMF) p:05Triage→03Medium [20:11:05] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Add a campaign to email activities for double opt in emails - https://phabricator.wikimedia.org/T422395#11816747 (10SBurnett-WMF) p:05Triage→03Low [20:12:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: MISSING_PREDECESSOR failmail on Recurring Sign-up messages - https://phabricator.wikimedia.org/T422839#11816750 (10Ejegg) These all went in without incident when I requeued them [20:14:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Recurring-Donations: Fix stats for smashpig recurring charge job - https://phabricator.wikimedia.org/T423196 (10Ejegg) 03NEW [20:14:58] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Recurring-Donations: Fix stats for smashpig recurring charge job - https://phabricator.wikimedia.org/T423196#11816767 (10Ejegg) https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1270495?usp=dashboard [20:15:07] (03CR) 10Lars SG: "Won't this get the same 1000 over and over (or at least any of them that haven't been changed in the last run)?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1269791 (https://phabricator.wikimedia.org/T420992) (owner: 10Wfan) [20:15:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Recurring-Donations: Fix stats for smashpig recurring charge job - https://phabricator.wikimedia.org/T423196#11816770 (10Ejegg) a:03Ejegg [20:16:53] (03PS1) 10Wfan: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270552 [20:17:38] (03PS4) 10Ejegg: Allow env variable to change SmashPig stats output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270495 (https://phabricator.wikimedia.org/T423196) [20:17:43] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: CiviCRM sets wrong email as display name for contact without a name - https://phabricator.wikimedia.org/T423181#11816775 (10Lars) Merged upstream, but this isn't super important for us, so we can just get it with our nex... [20:21:06] 10fundraising-tech-ops, 10Observability-Alerting, 13Patch-For-Review: Update firewall rules to allow frtech hosts to send alerts to production alertmanger - https://phabricator.wikimedia.org/T422888#11816791 (10SBurnett-WMF) [20:21:24] 06Fundraising-Backlog, 03GNU England Shaker dresser, 07payments-orchestration: Blik Payment Method- Poland - https://phabricator.wikimedia.org/T405606#11816795 (10Ejegg) a:03Ejegg [20:22:02] 06Fundraising-Backlog, 07fr-payments-ACH, 03GNU England Shaker dresser: Look into what async capture means for the smashpig recurring charge job - https://phabricator.wikimedia.org/T422863#11816799 (10Cstone) [20:22:07] 06Fundraising-Backlog, 07fr-payments-ACH, 03GNU England Shaker dresser: Look into what async capture means for the smashpig recurring charge job - https://phabricator.wikimedia.org/T422863#11816800 (10Cstone) a:03Cstone [20:23:31] 06Fundraising-Backlog, 06FR-donorrelations, 10Recurring-Donations, 07payments-orchestration: For gravy / dlocal, don't try to start recurring donations with debit cards - https://phabricator.wikimedia.org/T422566#11816805 (10SBurnett-WMF) [20:24:41] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: 2026-04-06 Gravy recurring CLP failing - https://phabricator.wikimedia.org/T422389#11816820 (10SBurnett-WMF) [20:27:05] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Epic, 13Patch-For-Review: Multi-arm bandit support for fundraising campaigns - https://phabricator.wikimedia.org/T422045#11816828 (10SBurnett-WMF) [20:29:28] (03PS1) 10Ejegg: Add Blik payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270554 (https://phabricator.wikimedia.org/T405606) [20:30:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 07fr-payments-ACH, 07payments-orchestration: Rescue ach recurring expired token issue to avoid charge back fee - https://phabricator.wikimedia.org/T421756#11816841 (10SBurnett-WMF) [20:32:23] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 03GNU England Shaker dresser: Exclude snoozed contacts from second recurring failure email send - https://phabricator.wikimedia.org/T422673#11816846 (10Lars) a:03Lars [20:42:43] 06Fundraising-Backlog, 10Recurring-Donations: Recurring: Optimize retry logic. - https://phabricator.wikimedia.org/T237655#11816886 (10Ejegg) →14Duplicate dup:03T413905 [20:42:45] 03Fundraising Sprint - Floor is Lava, 06Fundraising-Backlog: Update the recurring failure/cancellation logic - https://phabricator.wikimedia.org/T413905#11816888 (10Ejegg) [20:43:07] 06Fundraising-Backlog, 10Recurring-Donations: Recurring: Extend retry window. - https://phabricator.wikimedia.org/T237656#11816891 (10Ejegg) →14Duplicate dup:03T413905 [20:43:09] 03Fundraising Sprint - Floor is Lava, 06Fundraising-Backlog: Update the recurring failure/cancellation logic - https://phabricator.wikimedia.org/T413905#11816893 (10Ejegg) [20:44:09] 06Fundraising-Backlog, 10FR-Adyen: Adyen Capture timeout - don't send failmail if it's going to be retried - https://phabricator.wikimedia.org/T206864#11816895 (10Ejegg) 05Open→03Declined No longer using old WSDL code, haven't seen these errors for a long time [20:50:16] (03PS1) 10Lars SG: Exclude snoozed contacts from second recurring failure email send [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270560 (https://phabricator.wikimedia.org/T422673) [20:53:27] 03Fundraising Sprint - Floor is Lava, 06Fundraising-Backlog: Update the recurring failure/cancellation logic - https://phabricator.wikimedia.org/T413905#11816926 (10Ejegg) We have deployed all the code necessary to support changing our retry cadence, but the settings are all left at the current logic: 2 retrie... [21:02:28] (03CR) 10Eileen: [C:03+2] Add Blik payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270554 (https://phabricator.wikimedia.org/T405606) (owner: 10Ejegg) [21:02:34] (03PS2) 10Ejegg: Add Blik payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270554 (https://phabricator.wikimedia.org/T405606) [21:17:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Donor-portal: DonorPortal TY email for upgrade showing wrong frequency - https://phabricator.wikimedia.org/T421140#11817039 (10Lars) 05Open→03Resolved [21:17:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-CiviCRM-Searchkit: Change language in email preference center automated messaging - https://phabricator.wikimedia.org/T421150#11817040 (10Lars) 05Open→03Resolved [21:17:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 13Patch-For-Review: Update user job status when incomplete or complete with errors import has been completed via import API action - https://phabricator.wikimedia.org/T418416#11817041 (10Lars) 05Open→03Resolved [21:18:06] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 10FR-MGFE: Reduce queries being run on every CiviCRM load - https://phabricator.wikimedia.org/T421407#11817043 (10Lars) 05Open→03Resolved [21:18:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Contact hover pop-up - https://phabricator.wikimedia.org/T420741#11817045 (10Lars) 05Open→03Resolved [21:18:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Fix testRecurringQueueConsumeAutoRescueMessage when previous month lacks numeric day of current month - https://phabricator.wikimedia.org/T421773#11817046 (10Lars) 05Open→03Resolved [21:18:40] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: White Mail appeal code not importing - https://phabricator.wikimedia.org/T422065#11817047 (10Lars) 05Open→03Resolved [21:18:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: "mismatched contact types" when trying to import anonymous Fidelity donation - https://phabricator.wikimedia.org/T422213#11817048 (10Lars) 05Open→03Resolved [21:19:01] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 13Patch-For-Review: CiviCRM email location type is not set to default when adding new email on contact summary - https://phabricator.wikimedia.org/T421935#11817049 (10Lars) 05Open→03Resolved [21:41:12] (03PS1) 10Eileen: Adjust adyen batch date calculation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270573 [21:43:40] (03PS1) 10Ejegg: Add blik to Adyen-direct [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270575 (https://phabricator.wikimedia.org/T405606) [21:53:12] (03CR) 10Ejegg: [C:04-1] "This isn't quite it" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270575 (https://phabricator.wikimedia.org/T405606) (owner: 10Ejegg) [21:53:46] (03CR) 10Ejegg: [C:04-1] "Also, two extra files commit by accident" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270575 (https://phabricator.wikimedia.org/T405606) (owner: 10Ejegg) [21:54:25] (03PS3) 10Lars SG: Add is_eligible_for_donor_portal bool to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) [21:54:26] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, 03GNU England Shaker dresser, 13Patch-For-Review: Add token for CiviCRM emails to conditionally add Donor Portal link - https://phabricator.wikimedia.org/T419437#11817171 (10Lars) criteria more specifically is segment > 200, segme... [21:59:30] (03CR) 10Ejegg: "Might it be faster if you relied on contirbution_recur.payment_processor_id ?" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) (owner: 10Lars SG) [22:04:43] gotta run [22:11:08] (03PS1) 10Eileen: Rename variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270578 [22:11:08] (03PS1) 10Eileen: Gzip files when completed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270579 (https://phabricator.wikimedia.org/T418967) [22:30:37] (03PS1) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:31:07] (03CR) 10CI reject: [V:04-1] Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 (owner: 10Eileen) [22:31:51] (03PS2) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:32:00] minor patch for review ^^ [22:36:03] (03PS3) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:36:31] (03CR) 10CI reject: [V:04-1] Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 (owner: 10Eileen) [22:36:36] (03PS4) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:37:06] (03CR) 10CI reject: [V:04-1] Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 (owner: 10Eileen) [22:38:20] (03PS5) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:38:52] (03CR) 10CI reject: [V:04-1] Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 (owner: 10Eileen) [22:42:11] (03PS6) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:42:41] (03CR) 10CI reject: [V:04-1] Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 (owner: 10Eileen) [22:45:42] (03PS7) 10Eileen: Handle 'misccosts' from Adyen [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270581 [22:50:01] (03PS4) 10Lars SG: Add is_eligible_for_donor_portal bool to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) [22:51:28] (03PS5) 10Lars SG: Add is_eligible_for_donor_portal bool to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) [22:52:15] (03CR) 10Lars SG: "Thanks" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) (owner: 10Lars SG) [22:52:46] (03PS1) 10Eileen: Use separate contacts for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 [22:52:46] (03PS1) 10Eileen: Test for miscCosts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270588 [22:53:38] (03PS6) 10Lars SG: Add is_eligible_for_donor_portal bool to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1269066 (https://phabricator.wikimedia.org/T422670) [23:14:47] (03CR) 10CI reject: [V:04-1] Test for miscCosts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270588 (owner: 10Eileen) [23:17:59] (03CR) 10Lars SG: [C:03+2] Rename variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270578 (owner: 10Eileen) [23:25:58] (03PS2) 10Eileen: Rename variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270578 [23:26:15] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270578 (owner: 10Eileen) [23:26:22] (03PS2) 10Eileen: Test for miscCosts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270588 [23:29:34] (03PS2) 10Eileen: Gzip files when completed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270579 (https://phabricator.wikimedia.org/T418967) [23:29:34] (03PS2) 10Eileen: Use separate contact for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) [23:33:48] (03CR) 10Lars SG: "See comment" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) (owner: 10Eileen) [23:34:49] (03PS3) 10Eileen: Use separate contact for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) [23:35:11] (03CR) 10Eileen: Use separate contact for gateway fees (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) (owner: 10Eileen) [23:35:18] (03PS4) 10Eileen: Use separate contact for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) [23:36:11] (03CR) 10Lars SG: [C:03+2] Use separate contact for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) (owner: 10Eileen) [23:45:36] (03CR) 10Wfan: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270552 (owner: 10Wfan) [23:46:51] (03CR) 10CI reject: [V:04-1] Test for miscCosts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270588 (owner: 10Eileen) [23:47:00] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1270552 (owner: 10Wfan) [23:50:09] (03PS2) 10Lars SG: Add conditional donor portal link to tokens [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270041 (https://phabricator.wikimedia.org/T419437) [23:52:21] (03PS3) 10Lars SG: Add conditional donor portal link to tokens [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270041 (https://phabricator.wikimedia.org/T419437) [23:53:19] (03PS4) 10Eileen: Wire in Stripe Tests [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1266151 [23:53:48] (03CR) 10CI reject: [V:04-1] Wire in Stripe Tests [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1266151 (owner: 10Eileen) [23:57:08] (03PS1) 10Eileen: Update Braintree & Trustly to refer to backend_processor_reversal_id [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270599 [23:57:19] (03Merged) 10jenkins-bot: Use separate contact for gateway fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1270587 (https://phabricator.wikimedia.org/T415693) (owner: 10Eileen) [23:57:43] (03CR) 10CI reject: [V:04-1] Update Braintree & Trustly to refer to backend_processor_reversal_id [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1270599 (owner: 10Eileen)