[00:02:11] RECOVERY - check_mysql on frdata2002 is OK: Uptime: 259 Threads: 3 Questions: 179 Slow queries: 0 Opens: 130 Open tables: 124 Queries per second avg: 0.691 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:07:57] (03PS1) 10Eileen: Bump it into action again [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204705 (https://phabricator.wikimedia.org/T406193) [00:08:49] (03CR) 10Wfan: [C:03+2] Add trackingParams to packageFiles [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203588 (https://phabricator.wikimedia.org/T407644) (owner: 10Ejegg) [00:08:51] (03CR) 10Wfan: [C:03+2] Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) (owner: 10Ejegg) [00:11:14] (03Merged) 10jenkins-bot: Add trackingParams to packageFiles [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203588 (https://phabricator.wikimedia.org/T407644) (owner: 10Ejegg) [00:11:34] (03Merged) 10jenkins-bot: Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) (owner: 10Ejegg) [00:23:27] (03CR) 10CI reject: [V:04-1] Bump it into action again [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204705 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [00:30:12] (03PS1) 10Eileen: Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) [00:37:22] (03PS2) 10Eileen: Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) [00:41:04] (03PS3) 10Eileen: Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) [00:41:07] (03Abandoned) 10Eileen: Bump it into action again [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204705 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [00:52:48] (03PS4) 10Eileen: Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) [00:52:49] (03PS1) 10Eileen: Insert rows for channel = Email where mailing_identifier is not null [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204712 (https://phabricator.wikimedia.org/T406193) [01:12:21] (03PS10) 10Wfan: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [01:14:10] (03CR) 10CI reject: [V:04-1] Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) (owner: 10Wfan) [01:18:09] PROBLEM - check_ipsec on frban1002 is CRITICAL: Strongswan CRITICAL - ok: 1 not-conn: fran1002_v4 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1002&service=check_ipsec [01:23:09] RECOVERY - check_ipsec on frban1002 is OK: Strongswan OK - 2 ESP OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1002&service=check_ipsec [01:35:25] (03CR) 10Lars SG: [C:03+2] Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [01:52:14] (03Merged) 10jenkins-bot: Fix the last mis-labelled recurring gifts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204708 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [02:12:38] (03PS1) 10Wfan: move recurring modify to donor portal queue [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204719 [02:17:52] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204720 [02:18:06] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204720 (owner: 10Eileen) [02:19:29] !log civicrm upgraded from 90beebc7 to 654f12c2 [02:19:31] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:23:40] (03PS11) 10Wfan: Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [02:24:52] (03CR) 10Wfan: [V:03+2 C:03+2] move recurring modify to donor portal queue [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204719 (owner: 10Wfan) [02:25:48] (03CR) 10CI reject: [V:04-1] Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) (owner: 10Wfan) [03:22:51] (03CR) 10Lars SG: [C:03+2] Insert rows for channel = Email where mailing_identifier is not null [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204712 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [03:40:39] (03Merged) 10jenkins-bot: Insert rows for channel = Email where mailing_identifier is not null [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204712 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [03:42:27] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204723 [03:42:40] (03PS12) 10Wfan: Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [03:42:40] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204723 (owner: 10Eileen) [03:43:32] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204723 (owner: 10Eileen) [03:44:20] !log civicrm upgraded from 654f12c2 to a1faf78d [03:44:22] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:58:47] (03PS1) 10Eileen: Add a couple more utm_mediums [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204724 [03:58:47] (03PS1) 10Eileen: Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406139) [04:24:23] (03PS2) 10Eileen: Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406139) [05:24:37] (03PS1) 10Wfan: fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 [05:26:59] (03CR) 10CI reject: [V:04-1] fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 (owner: 10Wfan) [05:37:22] (03PS2) 10Wfan: fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 [05:39:58] (03CR) 10CI reject: [V:04-1] fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 (owner: 10Wfan) [06:35:00] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/1204457 (owner: 10L10n-bot) [06:35:41] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Get more clarity on Message / wmf_contribution_extra fields - https://phabricator.wikimedia.org/T400856#11369389 (10Eileenmcnaughton) a:03Eileenmcnaughton I did a bit of work on this & think it can be closed out - perhaps we can do mo... [06:37:13] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Map Settlement data to Civi field and table locations - https://phabricator.wikimedia.org/T397662#11369394 (10Eileenmcnaughton) a:03Eileenmcnaughton Current status is that we have most parts of this process in place and are refining /... [06:41:30] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07fr-current-sprint: Ensure new is_major_gift field is fully and correctly populated - https://phabricator.wikimedia.org/T409994 (10Eileenmcnaughton) 03NEW [06:42:33] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07fr-current-sprint: Rationalise our CiviCRM fields that map to reporting / finance reconcilliation - https://phabricator.wikimedia.org/T405986#11369416 (10Eileenmcnaughton) Moving this to done - I have s... [06:44:34] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Document Audit process - https://phabricator.wikimedia.org/T409995 (10Eileenmcnaughton) 03NEW [06:44:47] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Document Audit Settlement process - https://phabricator.wikimedia.org/T409995#11369430 (10Eileenmcnaughton) [06:45:34] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint, 13Patch-For-Review: Handle Settlement from Audit process - https://phabricator.wikimedia.org/T399782#11369432 (10Eileenmcnaughton) Moving this to done - focussing on specific mop up tasks now [06:47:26] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Handled double refund from Audit - https://phabricator.wikimedia.org/T409996 (10Eileenmcnaughton) 03NEW [06:50:11] (03PS5) 10Eileen: Update to cope with 2 files with the same batch in a run [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204097 (https://phabricator.wikimedia.org/T399782) [06:50:11] (03PS1) 10Eileen: Handle duplicate reversal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204745 (https://phabricator.wikimedia.org/T409996) [06:51:38] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Document Audit Settlement process - https://phabricator.wikimedia.org/T409995#11369451 (10Eileenmcnaughton) a:03Eileenmcnaughton [06:52:07] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Handled double refund from Audit - https://phabricator.wikimedia.org/T409996#11369454 (10Eileenmcnaughton) a:03Eileenmcnaughton [06:54:01] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, and 2 others: CiviCRM / Analytics data clean up - https://phabricator.wikimedia.org/T406193#11369458 (10Eileenmcnaughton) I've moved this to Done just because I'm swapping out the old 'do everything' phabs... [07:00:28] (03PS3) 10Eileen: Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406193) [07:05:47] 03Fundraising Sprint: Vixen, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 2 others: Channel QA - https://phabricator.wikimedia.org/T407179#11369473 (10Eileenmcnaughton) One the patch currently in review has gone out I will ask Joseph to review with regards to... [07:06:09] (03CR) 10CI reject: [V:04-1] Handle duplicate reversal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204745 (https://phabricator.wikimedia.org/T409996) (owner: 10Eileen) [07:09:03] (03PS2) 10Eileen: Handle duplicate reversal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204745 (https://phabricator.wikimedia.org/T409996) [07:21:33] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Handle invoicededuction in adyen audit file - https://phabricator.wikimedia.org/T409997 (10Eileenmcnaughton) 03NEW [07:23:09] (03PS1) 10Eileen: Add test cover for invoicededuction [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204748 (https://phabricator.wikimedia.org/T409997) [07:39:37] (03CR) 10CI reject: [V:04-1] Add test cover for invoicededuction [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204748 (https://phabricator.wikimedia.org/T409997) (owner: 10Eileen) [10:17:52] (03PS1) 10Wfan: Add flexibility for stack pending queue alter price up and down [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204811 (https://phabricator.wikimedia.org/T388755) [10:33:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi Unsubscribe Description Not Updating - https://phabricator.wikimedia.org/T409478#11370129 (10krobinson) Another one: https://wikimedia.zendesk.com/agent/tickets/1772581 / CID 19621344 - this d... [10:34:01] (03CR) 10CI reject: [V:04-1] Add flexibility for stack pending queue alter price up and down [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204811 (https://phabricator.wikimedia.org/T388755) (owner: 10Wfan) [10:41:54] (03PS13) 10Wfan: Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [10:43:02] (03PS3) 10Wfan: fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 [10:43:07] (03CR) 10CI reject: [V:04-1] Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) (owner: 10Wfan) [10:45:03] (03PS14) 10Wfan: Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [10:47:21] (03CR) 10CI reject: [V:04-1] Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) (owner: 10Wfan) [10:53:37] 10Fundraising analytics stack, 10fundraising-tech-ops, 06Privacy Engineering, 10Security Preview, 07SecTeam-Processed: [MinIO] Investigate packaging, install, security monitoring. - https://phabricator.wikimedia.org/T367123#11370175 (10jcrespo) Sorry for the late and random ping, but I highly recomme... [12:40:39] 10Fundraising analytics stack, 10fundraising-tech-ops, 06Privacy Engineering, 10Security Preview, 07SecTeam-Processed: [MinIO] Investigate packaging, install, security monitoring. - https://phabricator.wikimedia.org/T367123#11370503 (10IAckerman-WMF) Hi @jcrespo! Thanks. Would love to hear more about... [14:25:19] ugh, the exported matching gift policies don't seem to match what's in the DB.... how are they so out of date? [14:31:09] ????? [14:31:23] flushing caches [14:31:50] and still there! [14:32:14] regular API contact.get doesn't show the old data in the custom field [14:32:33] if I rewrite the export in API4??? [14:42:54] (03PS1) 10Ejegg: Move 'VerifyEmployerFile' to different entity [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 [14:54:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Employer field needs correction - https://phabricator.wikimedia.org/T404396#11370919 (10Ejegg) This is turning into one of /those/ tickets! Seems like the Philips 66 parent company hasn't had that typo-ed record in its subsidiaries... [15:00:01] (03CR) 10CI reject: [V:04-1] Move 'VerifyEmployerFile' to different entity [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 (owner: 10Ejegg) [15:06:08] (03PS1) 10Pcoombe: Tweak meta description to replace "sister projects" [extensions/FundraiserLandingPage] - 10https://gerrit.wikimedia.org/r/1204896 [15:06:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Employer field needs correction - https://phabricator.wikimedia.org/T404396#11370956 (10Ejegg) Ahhh, it's in a different custom field - "Name from matching gifts DB"! OK, fixing those too... [15:07:10] (03PS1) 10Ejegg: Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) [15:07:46] (03CR) 10Ejegg: [C:03+2] Tweak meta description to replace "sister projects" [extensions/FundraiserLandingPage] - 10https://gerrit.wikimedia.org/r/1204896 (owner: 10Pcoombe) [15:08:17] ah, worked on reload [15:08:58] ignore that ^^^, dredged up from typing history by pressing up and enter when I thought the keyboard focus was in a different window [15:09:14] So, unrelated test failures in CRM are always fun [15:09:27] SmashPigTest::testFailureEmailSentOnThirdFailedPayment and SmashPigTest::testCancelMessage are unhappy today [15:13:11] thanks for the super speedy +2 ejegg ! [15:14:08] 10Fundraising analytics stack: Ingest Zendesk data into MinIO - https://phabricator.wikimedia.org/T379121#11370981 (10AStein-WMF) Analytics folks are going to create a basic dashboard for kristine to review the data [15:14:59] 10Fundraising analytics stack, 10fundraising-tech-ops, 10FR-Tech-Analytics: Investigate Trino <> dbt TOO_MANY_REQUESTS_FAILED errors - https://phabricator.wikimedia.org/T395337#11370987 (10AStein-WMF) we havent seen this in a while so im going to close the ticket [15:15:08] 10Fundraising analytics stack, 10fundraising-tech-ops, 10FR-Tech-Analytics: Investigate Trino <> dbt TOO_MANY_REQUESTS_FAILED errors - https://phabricator.wikimedia.org/T395337#11370988 (10AStein-WMF) 05Open→03Resolved [15:16:30] 10Fundraising analytics stack, 10fundraising-tech-ops, 10FR-Tech-Analytics: Investigate Trino/ HMS <> dbt read timeout errors - https://phabricator.wikimedia.org/T395336#11370999 (10AStein-WMF) increasing the read timeout params for the iceberg and hive catalogs appear to have fixed this. closing [15:16:43] 10Fundraising analytics stack, 10fundraising-tech-ops, 10FR-Tech-Analytics: Investigate Trino/ HMS <> dbt read timeout errors - https://phabricator.wikimedia.org/T395336#11371000 (10AStein-WMF) 05Open→03Resolved [15:20:38] 10Fundraising analytics stack: Determine how to generate dbt docs - https://phabricator.wikimedia.org/T390202#11371018 (10AStein-WMF) i believe this is handled by our dbt-post-merge script which runs `dbt docs` and `dbt-metabase` to sync the definitions from dbt to metabase [15:20:55] 10Fundraising analytics stack: Determine how to generate dbt docs - https://phabricator.wikimedia.org/T390202#11371019 (10AStein-WMF) 05Open→03Resolved [15:23:16] 10Fundraising analytics stack: Ingest Zendesk data into MinIO - https://phabricator.wikimedia.org/T379121#11371035 (10AStein-WMF) analytics ticket: https://app.asana.com/1/3758245663860/project/1210424144163385/task/1211937797162579?focus=true [15:27:44] (03CR) 10CI reject: [V:04-1] Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) (owner: 10Ejegg) [15:28:14] ok, let's fix those other test failures [15:33:00] well, that's the immediate typos fixed and exported [15:33:07] still showing up in the autocomplete, probably cached [15:33:26] might as well finish the API4 switchover [15:34:32] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Port remaining Matching Gifts policy API actions to API4 - https://phabricator.wikimedia.org/T410052 (10Ejegg) 03NEW [15:35:55] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal: Change recurring donation frequency from monthly to yearly in the civi UI - https://phabricator.wikimedia.org/T396541#11371189 (10KHill-WMF) Present last monthly recurring amount (minimum gift of $5/year)... [15:37:21] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 13Patch-For-Review: Employer field needs correction - https://phabricator.wikimedia.org/T404396#11371204 (10Ejegg) OK, typos fixed and deployed. Should show up in the autocomplete as soon as the caches expire. [15:40:28] (03PS1) 10Ejegg: More updates for WP25 appeal [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204901 (https://phabricator.wikimedia.org/T407241) [15:40:50] (03CR) 10Ejegg: [V:03+2 C:03+2] More updates for WP25 appeal [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204901 (https://phabricator.wikimedia.org/T407241) (owner: 10Ejegg) [15:51:55] 03Fundraising Sprint: UTM_key lime pie, 06Fundraising-Backlog, 10FR-payments-general, 05MW-1.46-notes (1.46.0-wmf.2; 2025-11-12), 13Patch-For-Review: PaymentsWiki W25 design test - https://phabricator.wikimedia.org/T407241#11371270 (10Ejegg) OK, I've applied the new styles as well. And I think the q... [15:52:56] (03PS1) 10Ejegg: Another fix for WP25 appeal [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204902 [15:53:05] (03CR) 10Ejegg: [V:03+2 C:03+2] Another fix for WP25 appeal [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1204902 (owner: 10Ejegg) [15:56:13] (03PS1) 10Ejegg: Fix adding fail info to tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204904 [15:56:42] phew, plenty of paypal IPN listener failmail [16:13:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Wikimedia-production-error: ValueError: setcookie(): "expires" option cannot have a year greater than 9999 - https://phabricator.wikimedia.org/T410016#11371363 (10thcipriani) [16:14:52] (03CR) 10Lars SG: [C:03+2] Update to latest 6.8 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204672 (owner: 10Eileen) [16:18:25] (03CR) 10Ejegg: [V:03+2 C:03+2] Fix adding fail info to tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204904 (owner: 10Ejegg) [16:18:58] (03PS2) 10Ejegg: Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) [16:22:38] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Wikimedia-production-error: PHP Warning: Undefined array key "enabled" - https://phabricator.wikimedia.org/T409971#11371431 (10thcipriani) [16:25:26] (03PS2) 10Ejegg: Move 'VerifyEmployerFile' to different entity [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 (https://phabricator.wikimedia.org/T410052) [16:25:28] (03PS1) 10Ejegg: Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) [16:30:13] (03PS1) 10Ejegg: More type hints for matching gifts classes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204925 (https://phabricator.wikimedia.org/T410052) [16:31:08] (03CR) 10CI reject: [V:04-1] Update to latest 6.8 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204672 (owner: 10Eileen) [16:39:03] (03CR) 10CI reject: [V:04-1] Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) (owner: 10Ejegg) [16:43:10] (03CR) 10CI reject: [V:04-1] Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [16:47:03] (03PS3) 10Ejegg: Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) [16:50:07] (03CR) 10CI reject: [V:04-1] More type hints for matching gifts classes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204925 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [17:04:03] (03CR) 10CI reject: [V:04-1] Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) (owner: 10Ejegg) [17:06:28] (03PS15) 10Wfan: Donor Portal: Modify recurring amount [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1201197 (https://phabricator.wikimedia.org/T388755) [17:09:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: donor portal error when civiproxy down - https://phabricator.wikimedia.org/T410064 (10AnnWF) 03NEW [17:09:34] (03PS4) 10Wfan: fix response null issue when civiproxy down [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204731 (https://phabricator.wikimedia.org/T410064) [17:16:22] (03PS1) 10Wfan: fix typo [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204930 [17:16:44] (03PS2) 10Ejegg: Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) [17:16:44] (03PS2) 10Ejegg: More type hints for matching gifts classes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204925 (https://phabricator.wikimedia.org/T410052) [17:27:50] 03Fundraising Sprint: UTM_key lime pie, 06Fundraising-Backlog, 10FR-payments-general, 05MW-1.46-notes (1.46.0-wmf.2; 2025-11-12): PaymentsWiki W25 design test - https://phabricator.wikimedia.org/T407241#11371828 (10ppenloglou) Appreciate it @Ejegg ! Glad to hear the quirk is not from our side 👍 bummer... [17:36:00] (03CR) 10CI reject: [V:04-1] More type hints for matching gifts classes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204925 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [17:45:00] 06Fundraising-Backlog, 06FR-donorrelations: recent Gravy donations not in Civi - https://phabricator.wikimedia.org/T410067 (10MBeat33) 03NEW [17:45:26] 06Fundraising-Backlog, 06FR-donorrelations: recent Gravy donations not in Civi - https://phabricator.wikimedia.org/T410067#11371900 (10MBeat33) [17:45:55] 06Fundraising-Backlog, 06FR-donorrelations: recent Gravy donations not in Civi - https://phabricator.wikimedia.org/T410067#11371904 (10MBeat33) [17:52:55] (03CR) 10Lars SG: [C:03+2] Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [18:12:39] 06Fundraising-Backlog, 10fundraising-tech-ops, 07fr-acoustic: Report number of rows in Unsubscribe export file to prometheus - https://phabricator.wikimedia.org/T410071#11372041 (10Ejegg) [18:17:19] 06Fundraising-Backlog, 10fundraising-tech-ops, 07fr-acoustic: Report number of rows in Unsubscribe export file to prometheus - https://phabricator.wikimedia.org/T410071#11372071 (10Ejegg) [18:31:57] (03PS1) 10Ejegg: Fix warning in SmashPig tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204938 [18:41:07] (03PS1) 10Ejegg: Move isSiteBusy override to main phpunit bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204939 [18:41:33] fr-tech I think maybe ^^^ will get rid of the annoying CI-only test failures involving queues [18:42:27] gonna try adding back the queueing of recurring failure emails from the other place on top of that one ^^ [18:47:33] (03PS1) 10Ejegg: Send remaining recurring failure emails via queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204942 (https://phabricator.wikimedia.org/T356073) [18:47:34] (03PS1) 10Wfan: Add annual conversion activity type [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204943 (https://phabricator.wikimedia.org/T388753) [18:49:10] (03PS2) 10Wfan: Add flexibility for stack pending queue alter price up and down [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204811 (https://phabricator.wikimedia.org/T388755) [18:50:00] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: How to add addition Batch entry templates to top toolbar - https://phabricator.wikimedia.org/T410078 (10MDemosWMF) 03NEW [19:02:02] (03CR) 10Cstone: [C:03+2] Move isSiteBusy override to main phpunit bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204939 (owner: 10Ejegg) [19:17:35] (03Merged) 10jenkins-bot: Move isSiteBusy override to main phpunit bootstrap [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204939 (owner: 10Ejegg) [19:28:30] thanks cstone ! [19:28:52] 03Fundraising Sprint: Vixen: Implement recurring donation pay less for donor portal - https://phabricator.wikimedia.org/T410082 (10AnnWF) 03NEW [19:29:08] 03Fundraising Sprint: Vixen, 10FR-Donor-portal: Implement recurring donation pay less for donor portal - https://phabricator.wikimedia.org/T410082#11372306 (10AnnWF) [19:29:19] woohoo, and the other half of the queue-failure-emails patch gets a V+2 now too! https://gerrit.wikimedia.org/r/1204942 [19:55:37] (03PS4) 10Eileen: Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406193) [19:55:43] (03CR) 10Eileen: [V:03+2] Populate more missing channels [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204725 (https://phabricator.wikimedia.org/T406193) (owner: 10Eileen) [19:55:45] (03CR) 10Jforrester: [C:03+2] build: Update MediaWiki requirement to 1.46 [extensions/FundraisingTranslateWorkflow] - 10https://gerrit.wikimedia.org/r/1203941 (https://phabricator.wikimedia.org/T409239) (owner: 10Reedy) [19:56:53] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204949 [19:57:06] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204949 (owner: 10Eileen) [19:58:40] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204949 (owner: 10Eileen) [19:59:26] (03PS2) 10Eileen: Update to latest 6.8 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204672 [20:01:30] !log civicrm upgraded from a1faf78d to 59d75a7e [20:01:32] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:03:11] ejegg: was this one ok wrt the trxn_id - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1204745 - I couldn't see how else to do it... [20:09:48] (03CR) 10Cstone: [C:03+2] "wooooo!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204942 (https://phabricator.wikimedia.org/T356073) (owner: 10Ejegg) [20:18:44] (03Merged) 10jenkins-bot: build: Update MediaWiki requirement to 1.46 [extensions/FundraisingTranslateWorkflow] - 10https://gerrit.wikimedia.org/r/1203941 (https://phabricator.wikimedia.org/T409239) (owner: 10Reedy) [20:23:52] (03PS1) 10Ejegg: Update SmashPig [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204952 [20:25:03] (03CR) 10Ejegg: [C:03+2] Update SmashPig [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204952 (owner: 10Ejegg) [20:26:59] (03PS1) 10Ejegg: Update SmashPig and dependencies [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1204955 [20:27:03] (03Merged) 10jenkins-bot: Send remaining recurring failure emails via queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204942 (https://phabricator.wikimedia.org/T356073) (owner: 10Ejegg) [20:27:37] (03PS2) 10Ejegg: Update SmashPig [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204952 [20:27:39] (03CR) 10Ejegg: Update SmashPig [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204952 (owner: 10Ejegg) [20:28:32] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204956 [20:28:34] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204956 (owner: 10Ejegg) [20:28:56] (03CR) 10Ejegg: [V:03+2 C:03+2] Update SmashPig and dependencies [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1204955 (owner: 10Ejegg) [20:37:55] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Handle invoicededuction in adyen audit file - https://phabricator.wikimedia.org/T409997#11372535 (10Eileenmcnaughton) Seems the issue was I mis-tagged the release.... @ejegg is fixing [20:39:14] (03CR) 10Ejegg: [V:03+2 C:03+2] Update SmashPig [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204952 (owner: 10Ejegg) [20:39:22] (03CR) 10Ejegg: [V:03+2 C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1204956 (owner: 10Ejegg) [20:40:39] !log civicrm upgraded from 59d75a7e to 0675f10c [20:40:40] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:40:44] ok eileen that's out now ^^ [20:41:00] thanks ejegg [20:41:24] 06Fundraising-Backlog, 10fundraising-tech-ops, 07fr-acoustic: Report number of rows in Unsubscribe export file to prometheus - https://phabricator.wikimedia.org/T410071#11372549 (10Dwisehaupt) @ejegg You can write the output file into `/var/spool/prometheus/$filename.prom` and the node_exporter will pick it... [20:47:12] (03PS3) 10Eileen: Move 'VerifyEmployerFile' to different entity [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [20:47:23] (03CR) 10Eileen: [C:03+2] Move 'VerifyEmployerFile' to different entity (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [20:59:31] (03PS3) 10Ejegg: Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) [21:05:27] (03PS4) 10Ejegg: Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) [21:05:40] (03PS1) 10Ejegg: Fix undefined var in email template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204958 [21:06:06] (03PS1) 10Ejegg: Add donor langauge to prefs links [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204959 [21:08:57] (03Merged) 10jenkins-bot: Move 'VerifyEmployerFile' to different entity [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204892 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [21:12:46] (03PS4) 10Ejegg: Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) [21:14:19] (03CR) 10Jforrester: [C:03+2] build: Update MediaWiki requirement to 1.46 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1203966 (https://phabricator.wikimedia.org/T409239) (owner: 10Reedy) [21:20:42] (03PS1) 10Ejegg: Fix namespace case [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1204960 [21:21:35] (03CR) 10CI reject: [V:04-1] Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) (owner: 10Ejegg) [21:23:30] (03PS1) 10Ejegg: Handle convert declines in recurring queue consumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204961 (https://phabricator.wikimedia.org/T409966) [21:25:19] (03CR) 10Cstone: [C:03+2] Update to cope with 2 files with the same batch in a run [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204097 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [21:28:23] (03PS1) 10Ejegg: Send monthly_convert_decline messages to recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204962 (https://phabricator.wikimedia.org/T409966) [21:30:22] (03PS1) 10Ejegg: Remove monthly_convert_decline from donations queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204964 (https://phabricator.wikimedia.org/T409966) [21:30:38] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204672 (owner: 10Eileen) [21:30:53] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 13Patch-For-Review: Send 'decline-recurring' messages to a different queue from donations - https://phabricator.wikimedia.org/T409966#11372739 (10Ejegg) [21:37:32] (03Merged) 10jenkins-bot: build: Update MediaWiki requirement to 1.46 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1203966 (https://phabricator.wikimedia.org/T409239) (owner: 10Reedy) [21:39:56] cstone these patches to handle monthly convert declines in a different queue might speed donation processing a bit [21:40:11] We'd start with this one: https://gerrit.wikimedia.org/r/1204961 [21:40:23] that just adds the ability for the recurring qc to handle them [21:40:46] then we'd deploy this to the front end: https://gerrit.wikimedia.org/r/1204962 [21:40:52] (still needs local smoke testing) [21:40:55] brb [21:42:15] (03Merged) 10jenkins-bot: Update to cope with 2 files with the same batch in a run [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204097 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [21:47:22] yeah nice ejegg [21:55:12] (03CR) 10Ejegg: [C:03+2] fix typo [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204930 (owner: 10Wfan) [22:01:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Apple Pay donors seeing first & lastname error messages - https://phabricator.wikimedia.org/T407033#11372802 (10AMJohnson) Two more from today: - #1774204 / CID 67196845 "Get the following message when trying to use Apple Pay to... [22:01:40] hmm ejegg do you think sending them all to recurring will cause that queue to back up [22:05:11] let's see cstone, is it running out of time before it runs out of messages? [22:05:32] (03Merged) 10jenkins-bot: fix typo [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1204930 (owner: 10Wfan) [22:06:58] cstone: the most messages I've seen in process in a single run today is 211 [22:07:01] in recurring_queue_consume-20251113-103001.log [22:07:04] let's see how long that took [22:07:16] yeah imean with all the declines [22:07:24] hah like 10 sec [22:07:44] cstone: I don't think so - the recurring queue consumer still has lots of time [22:08:09] if it's run with a limit of 90 sec and maxing out at 9 sec runs [22:17:10] (03PS5) 10Ejegg: Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) [22:22:06] (03CR) 10Cstone: Handle duplicate reversal (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204745 (https://phabricator.wikimedia.org/T409996) (owner: 10Eileen) [22:22:25] 06Fundraising-Backlog, 06FR-donorrelations: recent Gravy-PayPal contribs without PII in Civi - https://phabricator.wikimedia.org/T410095 (10MBeat33) 03NEW [22:37:26] (03CR) 10CI reject: [V:04-1] Fix typos in another matching gift policy field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204897 (https://phabricator.wikimedia.org/T404396) (owner: 10Ejegg) [22:42:44] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: How to add addition Batch entry templates to top toolbar - https://phabricator.wikimedia.org/T410078#11372899 (10Lars) @MDemosWMF If you go to Administer > Customize Data & Screens > Navigation Menu, you can edit the menu there... [23:02:36] (03merge) 10dwisehaupt: Upgrade civi to 6.8.0 [repos/fundraising-tech/community-civicrm] - 10https://gitlab.wikimedia.org/repos/fundraising-tech/community-civicrm/-/merge_requests/26