[02:16:18] (PS3) Ejegg: Deal with gateway_txn_id in a standard way [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524535 [02:34:46] Fundraising Sprint Never Ending Query, Fundraising Sprint Office  , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (Eileenmcnaughton) @LeanneS sigh - the options for the values have changed again Currently for ne... [02:52:23] Fundraising-Backlog, FR-Ingenico: Follow up re: Performance of Ingenico MyCheckout hosted pages - https://phabricator.wikimedia.org/T228953 (Ejegg) [08:12:27] Fundraising-Backlog: 3DS2.0 - https://phabricator.wikimedia.org/T228945 (Peachey88) [08:12:51] Fundraising-Backlog: Apple Pay Spec for stand alone integration - https://phabricator.wikimedia.org/T228944 (Peachey88) [13:52:59] (PS1) Ejegg: India-specific PAN error messages [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/525545 (https://phabricator.wikimedia.org/T224506) [14:13:09] Fundraising Sprint Never Ending Query, Fundraising Sprint Office  , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (NNichols) @Eileenmcnaughton Yes please overwrite the existing net worth field. Some of the ones... [14:35:52] Fundraising Sprint Uptight Piano, Fundraising-Backlog: Paypal / Paypal EC issue in en6C? - https://phabricator.wikimedia.org/T178931 (Ejegg) [14:36:31] Fundraising Sprint Winter Wanderland, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Civimail bounce processing failing - https://phabricator.wikimedia.org/T181934 (Ejegg) p:Triage→Normal [14:37:03] Fundraising Sprint Asymmetrical Earth Theory, Fundraising-Backlog, FR-Ingenico: Ingenico audit wobble? 12/20 transactions not in Civi - https://phabricator.wikimedia.org/T183934 (Ejegg) p:Triage→High [14:38:34] Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising-Backlog, Wikimedia-Fundraising-CiviCR... - https://phabricator.wikimedia.org/T185041 [14:40:47] Fundraising Sprint Synchronized Screaming, Fundraising Sprint Uptight Piano, Fundraising Sprint turtles that are robotic that destroy the whole world with their foot, Fundraising-Backlog, Unplanned-Sprint-Work: donatewiki_counts and banner impressions m... - https://phabricator.wikimedia.org/T177331 [14:41:47] Fundraising-Backlog: Apple Pay Spec for stand alone integration - https://phabricator.wikimedia.org/T228944 (DStrine) @EMartin Thanks for this. We'll have to look at this a little later. Several years ago it looked like Adyen was going to be somewhat easier. Could you find any of their documentation and make... [14:44:16] Fundraising-Backlog: 3DS2.0 - https://phabricator.wikimedia.org/T228945 (DStrine) @EMartin @Ppena and @Ejegg This is good info. We just need to be sure the 1.0 is not applied 100% of the time for all countries. Eje [14:50:25] Fundraising Sprint Uptight Piano, Fundraising-Backlog: Paypal / Paypal EC issue in en6C? - https://phabricator.wikimedia.org/T178931 (CCogdill_WMF) Hmm... looks like I missed this two years ago :D FWIW, our normal threshold for bulk email is 1000 records. I supposed 91x12 = 1092 so we could extrapolate... [14:56:16] Fundraising-Backlog: rando spanish page in India flow - https://phabricator.wikimedia.org/T229006 (DStrine) [15:01:22] Fundraising Sprint USB stands for underhanded socket bureaucracy, Fundraising Sprint Vestigial tails shoot from the hip, Fundraising Sprint Window dressing is mostly olive oil, Fundraising-Backlog, Patch-For-Review: dash oauth stopped working - https://phabricator.wikimedia.org/T207209 (Ejegg) [15:06:39] Fundraising Sprint Synchronized Screaming, Fundraising Sprint Uptight Piano, Fundraising Sprint turtles that are robotic that destroy the whole world with their foot, Fundraising-Backlog, and 2 others: Back-fill missing pgehres data - https://phabricator.wikimedia.org/T178009 (Ejegg) [15:08:20] Fundraising Sprint Uptight Piano, Fundraising-Backlog: Paypal / Paypal EC issue in en6C? - https://phabricator.wikimedia.org/T178931 (Ejegg) Heh, yeah, I'm just going back and estimating things without story points. [15:20:12] cstone, ejegg I noticed that the processor_id on the contribution_recur record is now the original trxn_id, is that intentional? [15:20:35] it looks like it was a small int previously [15:21:04] (small int, in general and not the SQL field type) [15:21:23] jgleeson yeah, the new code is trying to be more in line with the Civi core usage [15:21:42] where processor_id refers to the recurring payment ID at the processor [15:22:26] don't we use that field somewhere else for mapping to our version of the gateway? [15:22:39] or maybe that's something else [15:22:42] there's also payment_processor_id [15:22:46] payment_processor_id? [15:22:47] ah yeah [15:22:47] very confusing [15:23:05] ok cool, thanks for explaining [15:23:11] sure thing! [15:23:36] OK, so I should pull down both your patch and cstone's patch, and try to get something working end to end [15:23:59] also figure out why that invoice ID format in your demo didn't have the . that I expected [15:27:02] (PS2) Ejegg: Separate do_not_solicit in merge code from otherwise calculated fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524393 (owner: Eileen) [15:27:56] (CR) Ejegg: [C: +2] "Looks good, and the calculated fields fn returns a more correct list now. Yay!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524393 (owner: Eileen) [15:28:45] Fundraising Sprint Men In Slack, Fundraising Sprint Never Ending Query, Fundraising Sprint Office  , Fundraising-Backlog: Replace MySQL-python with pymysql in fundraising python scripts - https://phabricator.wikimedia.org/T213757 (Ejegg) Verified that pymysql is already installed on the clus... [15:33:14] (Merged) jenkins-bot: Separate do_not_solicit in merge code from otherwise calculated fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524393 (owner: Eileen) [15:37:32] (PS2) Ejegg: Remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524391 (owner: Eileen) [15:37:47] (CR) Ejegg: [C: +2] Remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524391 (owner: Eileen) [15:43:18] (Merged) jenkins-bot: Remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524391 (owner: Eileen) [15:58:00] (CR) Ejegg: [C: +1] "This looks good, I just need to figure out where to set the environment in CiviCRM and in vagrant. We're also using the drupal module 'env" (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524960 (https://phabricator.wikimedia.org/T228713) (owner: Eileen) [15:59:16] (PS5) Ejegg: Add triggers_drop.sql [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524389 (owner: Eileen) [15:59:30] (CR) Ejegg: [C: +2] Add triggers_drop.sql [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524389 (owner: Eileen) [16:04:59] (Merged) jenkins-bot: Add triggers_drop.sql [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524389 (owner: Eileen) [16:06:45] Fundraising-Backlog: rando spanish page in India flow - https://phabricator.wikimedia.org/T229006 (MarcoAurelio) Confirm it is Spanish. Translation: please wait while you're being redirected. [16:10:13] (PS3) Ejegg: Updated triggers after extra fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524394 (owner: Eileen) [16:10:21] (CR) Ejegg: [C: +2] Updated triggers after extra fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524394 (owner: Eileen) [16:11:16] (CR) jerkins-bot: [V: -1] Updated triggers after extra fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524394 (owner: Eileen) [16:13:36] (CR) Ejegg: [C: +2] "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524394 (owner: Eileen) [16:19:19] (Merged) jenkins-bot: Updated triggers after extra fields [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524394 (owner: Eileen) [16:57:55] so the coworking space I'm at have an unlimited beer pump available on thursdays which explains why it's so busy today... [16:58:39] and I'm picking up roxy and the kids after this so I can't even sample it [16:59:12] maybe next time :) [17:00:13] whoa, that's a heck of a perk! [17:00:40] yeah and I only pay like £50 per month as I only use this space 1-2 times a week [17:01:05] I bet I could drink my membershipsworth if put to the test! [17:01:30] Especially if the beer is any good. [17:01:55] But that's still a great perk even with average beer. [17:05:32] Fundraising Sprint Never Ending Query, Fundraising Sprint Office  , Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (LeanneS) @Eileenmcnaughton Hm that's odd! Those values ($250,000 - $499,999, $750,000 - $999,999)... [18:07:53] Fundraising-Backlog: Apple Pay Specs - Adyen - https://phabricator.wikimedia.org/T229039 (Aklapper) Assuming this task is about #Fundraising-Backlog, hence adding project tag so others can find this task when searching for tasks under that code project. [18:09:59] Fundraising-Backlog: Apple Pay Specs - Adyen - https://phabricator.wikimedia.org/T229039 (EMartin) Thanks and sorry I'm not up on the project naemes as yet! I'll get better!! (I hope!) [18:48:18] (PS3) Cstone: Add recurring token support to creating a subscription. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524909 (https://phabricator.wikimedia.org/T227048) [18:48:38] cool cstone, will take a look! [19:42:35] (PS17) Vedmaka Wakalaka: Campaign fallback [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/517931 (https://phabricator.wikimedia.org/T124969) [19:45:50] (CR) jerkins-bot: [V: -1] Campaign fallback [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/517931 (https://phabricator.wikimedia.org/T124969) (owner: Vedmaka Wakalaka) [20:32:39] Fundraising-Backlog, FR-Ingenico, Recurring-Donations: Ingenico Connect: add new parameter to recurring transactions - https://phabricator.wikimedia.org/T229059 (Ejegg) [21:19:43] Fundraising Sprint A series of unfortunate event handlers, Fundraising Sprint Bert and Ernie's Excellent Adventure, Fundraising Sprint Casino Royale With Cheese, Fundraising Sprint Da Vinci Coder, and 15 others: Reduce recurring TY emails - https://phabricator.wikimedia.org/T213209 (Ejegg) @cston... [21:23:22] blarg, db is /still/ locking up [21:24:39] again, no long-lived query [21:25:07] just one with a group_contact_cache update, and another with an activity search [21:26:46] fr-tech does anyone know if the 'Update addressee for major gifts organization contacts' job would be updating the group_contact_cache? [21:27:08] it seems to be affected by the db hangs at the very least [21:28:08] cripes, and there's a totally useless civicrm_acl update statement in the mix [21:28:21] UPDATE IGNORE civicrm_acl SET entity_id = 3347208 WHERE entity_id = 34696866 AND entity_table = 'civicrm_contact' [21:28:37] oh wait, no, that's actually doing something [21:28:41] guessing merge-related [21:31:49] Fundraising-Backlog, FR-Adyen: Apple Pay Specs - Adyen - https://phabricator.wikimedia.org/T229039 (Ejegg) @EMartin just FYI, we also have tags for each payment processor integration: FR-Adyen, FR-Ingenico, FR-Amazon, FR-dlocal, FR-Paypal, and FR-Paypal-ExpressCheckout [21:33:28] ok, I'm going to try stopping the 'Update addressee' job and see if that helps [21:37:30] Fundraising-Backlog, FR-Adyen: Apple Pay Specs - Adyen - https://phabricator.wikimedia.org/T229039 (EMartin) Thank you. I will note this in the future. [21:40:04] !log turned off major_gifts_addresses fundraising job [21:40:10] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:53:49] fr-tech I'm having a hard time figuring out what's causing these locks. Anyone want to help investigate? [21:54:06] hey ejegg im back now [21:54:19] maybe the dedupe job colliding? [21:54:28] I'll try turning that off too [21:54:50] in the past it's always been obvious from a 'show full processlist' [22:01:11] so it seems to be failing on the contact insert, not the donation insert [22:01:30] which would mean it's probably not the fault of the new triggers [22:01:46] and to catch up here its failing cause someting else has locked tables involved? [22:02:00] !log turned off dedupe_civicrm_contacts fundraising job [22:02:12] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:02:49] cstone the failmail says 'database lock encountered' [22:03:16] which usually means there's a super-long-running query happening that we can find with 'show full processlist' [22:03:19] and kill [22:03:27] but this time I can't see anything long-running happening [22:03:38] ugh [22:09:13] aha, there's a big import happening [22:09:26] gonna shut down the donations queue consumer for now too [22:10:06] aha, but I'mma use the Civi UI switch for it [22:13:20] !turned off donations queue consumer using disable checkbox in civi UI [22:20:49] cstone heh, now I just see a couple of big queries under your username [22:21:07] doing some contribution searches? [22:21:25] haha not trying to do things [22:22:04] you can kill them if they are causing issues [22:22:10] weird, just loaded the front page then? [22:22:36] maybe they're totally normal queries that are just being impeded by something that's not showing up in the processlist? [22:23:01] i clicked on the contributions dash [22:23:30] is that normally a big one? [22:23:44] err, i dunno. I don't actually use it ever [22:26:10] crap, the org addresses update job is failing too [22:27:08] is that the big import that is happening? [22:27:30] cstone no, turns out there wasn't actually a big import happening [22:27:41] i misunderstood what nora said - she was about to start it [22:28:03] ah [22:28:05] the major gifts org addresses update is a normal recurring job [22:28:22] that just cleans up address records for organization contacts in major gifts groups [22:33:57] ok, looks really quiet in the db, I'mma try turning donations qc back on [22:34:18] !log re-enabled donations queue consumer [22:34:23] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:35:40] XenoRyet: I'mma go |food soon - can you help try to figure out what's up with these locks? [22:36:01] We might have to ask Jeff_Green for help if it's nothing we can find with civi-user-level db privs [22:36:07] Yea, I'll see if I can figure anything out. [22:36:59] thanks! [22:45:04] (PS1) AndyRussG: Banner history logger: remove loading of schema module [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525677 (https://phabricator.wikimedia.org/T227711) [22:47:52] (CR) jerkins-bot: [V: -1] Banner history logger: remove loading of schema module [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525677 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG) [22:50:55] (PS1) AndyRussG: Avoid usage of deprecated `ObjectCache::getMainWANInstance()` [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525680 [22:53:36] (Abandoned) AndyRussG: Banner history logger: remove loading of schema module [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525677 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG) [22:54:24] (CR) jerkins-bot: [V: -1] Avoid usage of deprecated `ObjectCache::getMainWANInstance()` [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525680 (owner: AndyRussG) [22:56:49] fr-tech hey, I'm getting a bunch of CI errors with the CN cherry-picks that I was gonna push out today [22:57:21] This one complained about a deprecated method and a phan check plugin version: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/525677 [22:58:15] When I tried to add the patch that fixes the deprecated method, I found got a zillion PHP unit test errors: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CentralNotice/+/525680 [22:58:37] Add that to the fact that there are QUnit tests that are not running on CI due to another bug [22:58:53] I'd say tips the balance against deploying on today's SWAT deploy [22:59:29] I can work on all this over the next few days, and hopefully we could do a SWAT of those two things on Monday. So, not deploying CN update [22:59:38] Hope that sounds right! [23:00:21] (Abandoned) AndyRussG: Avoid usage of deprecated `ObjectCache::getMainWANInstance()` [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/525680 (owner: AndyRussG) [23:40:40] fr-tech: I'm coming up dry on what's causing these locks, but I've gotta go pick up the kids. [23:42:22] ok, let's call Jeff_Green [23:55:49] hi Jeff!