[00:03:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 13Patch-For-Review: JO donation form: other amount issue - https://phabricator.wikimedia.org/T395093#10861748 (10AnnWF) a:03AnnWF [00:16:43] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10861798 (10Dwisehaupt) Most hosts updated to latest kernel. Awaiting the completion of a long running kafka cat job... [00:25:08] (03PS1) 10Wfan: JOD is three decimal currency [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151391 (https://phabricator.wikimedia.org/T395093) [04:30:51] (03PS1) 10Eileen: Add pre files - in particular force logging DSN [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151407 [04:39:35] (03CR) 10CI reject: [V:04-1] Add pre files - in particular force logging DSN [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151407 (owner: 10Eileen) [07:22:54] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 06FR-donorrelations: Gravy & the PayPal recurring cancellation process - https://phabricator.wikimedia.org/T393226#10862233 (10krobinson) Oooh interesting, thanks. I'm not sure we need it - the ZD confirmation is likely sufficient, no? If they are in... [13:37:41] (03PS3) 10Jgleeson: Add SEPA mandate notice to SEPA forms [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1149411 (https://phabricator.wikimedia.org/T394932) [13:37:46] (03PS2) 10Jgleeson: Mustache tweak: use new is_yearly flag [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1149697 (https://phabricator.wikimedia.org/T394932) [14:22:16] ok, finding a potential source of that stack-too-deep problem [14:22:31] inside SettingsBag.php line 124 [14:22:41] when it's unserializing stuff [14:22:47] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Omni activity load throwing exception when required file is missing - https://phabricator.wikimedia.org/T395457 (10Damilare) 03NEW [14:23:20] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Omni activity load throwing exception when required file is missing - https://phabricator.wikimedia.org/T395457#10863896 (10Damilare) [14:24:02] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: No support of required debit mandate for SEPA - https://phabricator.wikimedia.org/T394932#10863899 (10jgleeson) Hi @krobinson, @EMartin, I'm currently adding the SEPA mandate translations that @TomaszGorski provided. As I was review... [14:24:03] the mailing_backend is this strange nested thing: https://phabricator.wikimedia.org/P76622 [14:24:13] let's see if that crashes [14:24:30] ejegg: context please [14:24:33] :D [14:24:51] jgleeson: ah, when trying to run the unit tests under standalone it was crashing a ways through [14:25:00] then later any invocation of cv would crash [14:25:01] ah [14:25:34] with Xdebug has detected a possible infinite loop, and aborted your script with a stack depth of '256' frames [14:26:14] I had been testing a core PR relating to mail settings but cstone also got the same error [14:26:46] (also I was testing that core PR on dmaster not standalone) [14:26:57] Yeah mine was from just running the tests on wmf master [14:27:25] so lemme see if that's been stored in the DB or cache or what [14:27:41] oh that loadValues fn is db-specific [14:27:56] so I'll try to figure out where it's being mutated I guess [14:28:03] ejegg: I also had the infinite loop bug [14:28:07] I made a ticket [14:28:11] this was before standalone [14:28:20] ahh, ok, good data point [14:28:37] so it's most likely our own test harness code, I'd guess [14:28:57] https://phabricator.wikimedia.org/T394449 [14:29:18] oho, it's in that same Unserialize [14:29:30] and did you just have to wipe & reinstall to get it to work again? [14:29:45] I think so ejegg [14:29:56] interesting [14:30:25] you all still in the lighthouse place? [14:30:42] nope, I got back home last night [14:31:02] eileen might still be in transit [14:31:11] ah wb [14:31:17] thanks! [14:32:53] (03PS1) 10Jgleeson: Add French translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151710 (https://phabricator.wikimedia.org/T394932) [14:36:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Infinite loop error with local civicrm instance - https://phabricator.wikimedia.org/T394449#10863968 (10Ejegg) Seems to be a messed-up setting in the database. mailing_backend has become this Tetsuo-monster of serialization {P76622}. Clearing that setting... [14:46:41] (03PS1) 10Jgleeson: Add Dutch translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151715 (https://phabricator.wikimedia.org/T394932) [14:47:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Settled Adyen Donation - Settled Gravy Donation - Not Appearing in Civi - https://phabricator.wikimedia.org/T395340#10864042 (10SHust) [14:47:57] (03PS2) 10Jgleeson: Add French translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151710 (https://phabricator.wikimedia.org/T394932) [14:48:14] (03PS2) 10Jgleeson: Add Dutch translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151715 (https://phabricator.wikimedia.org/T394932) [14:53:54] (03PS1) 10Jgleeson: Add Spanish translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151718 (https://phabricator.wikimedia.org/T394932) [15:03:04] (03PS1) 10Ejegg: Fix mailing_backend mutation leading to crash [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151719 (https://phabricator.wikimedia.org/T394449) [15:03:25] yah it was in our test cleanup fr-tech ^^ [15:14:15] ooh, there's something that makes the tests get slower and slower [15:14:33] running the full suite, by the time it got to ImportTest each case was taking >30s [15:15:25] but if I filter to just ImportTest (even with xdebug.mode=profile) they mostly take 3-4 sec [15:17:01] oh i had the profile output_dir set to something that didn't exist [15:17:14] 06Fundraising-Backlog, 06FR-donorrelations: Civi recurring donations display question - https://phabricator.wikimedia.org/T393227#10864251 (10MBeat33) Thank you @Eileenmcnaughton and @Cstone this would be great to have in order to ease the DR team's use of Civi to help recurring donors [15:17:22] now that it's actually profiling they take 12-13 sec, but still less than during the full run [15:18:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Infinite loop error with local civicrm instance - https://phabricator.wikimedia.org/T394449#10864259 (10Ejegg) a:03Ejegg [15:19:16] sounds good [15:19:46] (03CR) 10Jgleeson: [C:03+2] "thanks for the fix!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151719 (https://phabricator.wikimedia.org/T394449) (owner: 10Ejegg) [15:20:10] wow, 10GB cachegrind file and it's only on 12/17 tests [15:20:19] I'm gonna have a break before the meeting vortex. back later [15:31:49] ok, afformscanner seems to be taking a lot of the time in those tests [15:31:59] but also the tearDown, deleting contacts [15:34:56] and it looks like a lot of that is the post-delete hook [15:37:07] ohh, this looks expensive: https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/core/ext/civiimport/Civi/Api4/Event/Subscriber/ImportSubscriber.php$140 [15:37:27] runs a CRM_Core_DAO_AllCoreTables::flush(); [15:38:30] (03Merged) 10jenkins-bot: Fix mailing_backend mutation leading to crash [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151719 (https://phabricator.wikimedia.org/T394449) (owner: 10Ejegg) [15:38:43] thanks jgleeson|away ! [15:56:48] ahhhh, afform time could be reduced greatly by caching allPaths [15:56:50] but.... [15:56:58] we'd have to turn off debug in core settings [15:57:34] https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/core/ext/afform/core/CRM/Afform/AfformScanner.php$105 [16:04:18] (03PS1) 10Damilare Adedoyin: Get payment method config and required fields from Gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1151739 [16:04:48] (03CR) 10CI reject: [V:04-1] Get payment method config and required fields from Gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1151739 (owner: 10Damilare Adedoyin) [16:11:32] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Verify which countries need a 'fiscal_number' to process a payment on Gravy - https://phabricator.wikimedia.org/T394051#10864758 (10AnnWF) >>! In T394051#10860809, @jgleeson wrote: > Cristiano from Gravy sent this list on Slack (contains duplicates): >... [16:12:10] (03PS2) 10Damilare Adedoyin: Get payment method config and required fields from Gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1151739 [16:12:31] (03PS4) 10Jgleeson: Add SEPA mandate notice to SEPA forms [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1149411 (https://phabricator.wikimedia.org/T394932) [16:13:34] (03PS3) 10Jgleeson: Mustache tweak: use new is_yearly flag [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1149697 (https://phabricator.wikimedia.org/T394932) [16:13:34] (03PS3) 10Jgleeson: Add French translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151710 (https://phabricator.wikimedia.org/T394932) [16:13:34] (03PS3) 10Jgleeson: Add Dutch translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151715 (https://phabricator.wikimedia.org/T394932) [16:13:34] (03PS2) 10Jgleeson: Add Spanish translation for SEPA mandate notice. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1151718 (https://phabricator.wikimedia.org/T394932) [16:29:22] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10864838 (10AStein-WMF) [16:30:21] (03CR) 10Jgleeson: [C:03+2] "lgtm!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1151388 (https://phabricator.wikimedia.org/T395093) (owner: 10Wfan) [16:36:34] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10864870 (10Jgreen) [17:15:03] 10fundraising-tech-ops, 06Data-Engineering, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Low volume in new webrequest feed - https://phabricator.wikimedia.org/T395089#10865052 (10Ahoelzl) [17:17:32] 10fundraising-tech-ops, 06Data-Engineering, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Low volume in new webrequest feed - https://phabricator.wikimedia.org/T395089#10865061 (10Ahoelzl) It looks like you are unblocked. Sorry about the non-communication about the Kafka partition changes that caused the... [17:21:12] PROBLEM - check_load on fransw2003 is CRITICAL: LOAD CRITICAL - total load average: 34.69, 22.66, 14.69 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [17:22:12] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 17.24, 20.86, 16.49 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [17:36:12] RECOVERY - check_load on fransw2003 is OK: LOAD OK - total load average: 0.58, 4.00, 8.63 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_load [17:37:12] RECOVERY - check_load on fransw2002 is OK: LOAD OK - total load average: 0.30, 2.93, 8.44 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [18:24:05] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Import templates missing / not saving new - https://phabricator.wikimedia.org/T394929#10865380 (10XenoRyet) [18:25:27] (03PS1) 10Ejegg: Update Standalonemigrate extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1151774 [18:52:31] weird, I just set up a monthly recurring via Gravy / PayPal and I don't see any saved payment at all on my PayPal page [18:54:30] it's got both ExpressCheckout - style monthly billing agreements and newer as-needed charges but nothing from wikimedia [18:55:04] also it still shows today's donation as 'Payment for items' [18:55:07] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 06FR-donorrelations: Gravy & the PayPal recurring cancellation process - https://phabricator.wikimedia.org/T393226#10865492 (10XenoRyet) Looking into this some more, it seems that the "balances due" messages in the paypal console are something that ar... [18:55:32] and doesn't mention anything about it being recurring. [18:55:50] I'm going to try to charge it again from the backend just to make sure we really do have a saved token [18:57:11] hmm, still not in civi either! [19:00:05] wat [19:00:21] and not on the gravy console [19:10:24] 10fundraising-tech-ops, 10BDC-Implementation, 06Data-Engineering-Radar, 07Epic: [MinIO] Investigate load-balancing approaches to eliminate SPOF - https://phabricator.wikimedia.org/T392249#10865544 (10Jgreen) [19:10:49] 10fundraising-tech-ops, 10BDC-Implementation, 06Data-Engineering-Radar, 07Epic: [MinIO] Investigate load-balancing approaches to eliminate SPOF - https://phabricator.wikimedia.org/T392249#10865546 (10Jgreen) [19:13:23] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10865551 (10Jgreen) [19:24:52] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10865577 (10Jgreen) >>! In T337585#10861798, @Dwisehaupt wrote: > Most hosts updated to latest kernel. Awaiting the c... [19:26:59] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10865595 (10Jgreen) [19:29:16] 06Fundraising-Backlog, 07payments-orchestration: Transaction ID listed on PayPal does not match processor_id in Gravy console - https://phabricator.wikimedia.org/T395512 (10Ejegg) 03NEW [19:31:24] 10fundraising-tech-ops, 06Data-Engineering, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Low volume in new webrequest feed - https://phabricator.wikimedia.org/T395089#10865625 (10Jgreen) 05Open→03Resolved >>! In T395089#10865060, @Ahoelzl wrote: > It looks like you are unblocked. > Sorry about t... [19:35:53] 10fundraising-tech-ops, 10BDC-Implementation, 07Epic: EPIC: Metabase Implementation - https://phabricator.wikimedia.org/T377363#10865649 (10Jgreen) a:05Jgreen→03None Un-owning because it's an epic task. [19:53:59] 06Fundraising-Backlog: Gravy dLocal placeholder fiscal numbers - https://phabricator.wikimedia.org/T395361#10865694 (10AnnWF) [20:29:24] 06Fundraising-Backlog, 07payments-orchestration: Transaction ID listed on PayPal does not match processor_id in Gravy console - https://phabricator.wikimedia.org/T395512#10865825 (10Ejegg) [20:32:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Infinite loop error with local civicrm instance - https://phabricator.wikimedia.org/T394449#10865836 (10Ejegg) 05Open→03Resolved [21:12:51] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Donors included wrongly in email send due to "country". - https://phabricator.wikimedia.org/T381437#10865937 (10Cstone) @ppenloglou @Pcoombe this issue is back again, any new ideas on how to stop t... [21:25:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Donors included wrongly in email send due to "country". - https://phabricator.wikimedia.org/T381437#10865958 (10Ejegg) There are so many ways a country can change in Civi, including a donor changin... [21:31:51] 03Fundraising Sprint: Jollof Rice, 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2425Q4 maintenance window - https://phabricator.wikimedia.org/T337585#10865968 (10Dwisehaupt) [23:43:50] (03PS2) 10Wfan: JOD is three decimal currency [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1151388 (https://phabricator.wikimedia.org/T395093)