[00:06:10] RECOVERY - check_mysql on frdb2002 is OK: Uptime: 898 Threads: 1 Questions: 7706286 Slow queries: 0 Opens: 1332 Flush tables: 1 Open tables: 723 Queries per second avg: 8581.610 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [00:06:52] (PS3) Eileen: Add new ckeditor5 extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/576443 (https://phabricator.wikimedia.org/T244323) [00:14:26] Fundraising-Backlog, fundraising-tech-ops: Automation / optimization of data cubes - https://phabricator.wikimedia.org/T238394 (EYener) Removed test cube from cron and set up a v1 production cube on a 10 minute schedule for inserts only. Next, I'll be working on updates on the whole cube, which would be... [00:17:10] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: All_endpoint 0 [=10] [00:18:10] Fundraising-Backlog, fundraising-tech-ops: Restore frdb2001 from backup - https://phabricator.wikimedia.org/T246045 (Dwisehaupt) Tested restores from multiple copies of the mariabackups of frdb1003. Each time the restore would fail on prepare with an error like: ` 2020-03-03 21:39:27 140076403128064 [ER... [00:18:33] Fundraising-Backlog, fundraising-tech-ops: Restore frdb2001 from backup - https://phabricator.wikimedia.org/T246045 (Dwisehaupt) p:Triage→Medium [00:18:55] Fundraising-Backlog, fundraising-tech-ops: Restore frdb2001 from backup - https://phabricator.wikimedia.org/T246045 (Dwisehaupt) Open→Resolved [00:18:57] fundraising-tech-ops, DC-Ops, Operations, ops-codfw: (Need by: ASAP) rack/setup/install frdb2001 - https://phabricator.wikimedia.org/T245566 (Dwisehaupt) [00:22:10] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: All_endpoint 0 [=10] [00:27:10] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: All_endpoint 0 [=10] [00:32:10] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: All_endpoint 0 [=10] [00:32:59] i'm checking into these messages. syslog is backed up on frav1002. [00:37:10] RECOVERY - check_log_messages on frav1002 is OK: OK [01:33:40] (PS1) Eileen: Override Setting api permissions so users without Administer CiviCRM can access. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577020 [01:44:45] (PS1) Cdenes: Edits to the Dutch TY email for the NLNL campaign coming up [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577024 [01:47:24] Fundraising-Backlog: NLNL TY Email - https://phabricator.wikimedia.org/T246962 (CDenes_WMF) [02:52:16] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, and 3 others: Cull low value data from activity table - https://phabricator.wikimedia.org/T245088 (Eileenmcnaughton) Digging into this I see our index sizes are | d... [03:07:59] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, and 3 others: Cull low value data from activity table - https://phabricator.wikimedia.org/T245088 (Eileenmcnaughton) I added this to ponder the issue https://lab.civ... [03:17:17] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, and 3 others: Cull low value data from activity table - https://phabricator.wikimedia.org/T245088 (Eileenmcnaughton) @MBeat33 @NNichols @LeanneS @rlewis the activit... [03:52:54] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Help Pats with her query - https://phabricator.wikimedia.org/T244194 (Eileenmcnaughton) Relevant queries tested on dev: **- Most eff... [04:28:33] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Help Pats with her query - https://phabricator.wikimedia.org/T244194 (Eileenmcnaughton) Let's see what fixing the casting does ` SE... [13:57:43] Fundraising Sprint Snoop (Dogg|Lion), Fundraising-Backlog, Fr-CentralNotice-Translation-Bugs, MediaWiki-extensions-CentralNotice, and 3 others: Central Notice message groups are slow to index - https://phabricator.wikimedia.org/T111189 (Pginer-WMF) Open→Resolved [14:20:40] (PS1) Jgleeson: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/577242 [14:30:53] (CR) Jgleeson: [C: +2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/577242 (owner: Jgleeson) [14:32:08] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/577242 (owner: Jgleeson) [14:42:40] (PS1) Jgleeson: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/577245 [14:47:05] hi fr-tech! [14:47:17] hey ejegg [14:47:43] (PS3) Ejegg: Adyen: add PaymentError for normal declines [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576877 (https://phabricator.wikimedia.org/T243340) [14:48:12] (just a rebase) [14:49:53] I've somehow broke my payments wiki since updating and recomposer installing grrr [14:50:10] now I'm getting ERR_SSL_PROTOCOL_ERROR errors which should be unrelated to code changes? [14:50:25] feel like my local https config is falling over for unknown reasons [14:50:30] good timing vagrant! [14:50:34] so jgleeson ohhhh [14:50:40] is it the skin redirect? [14:50:50] ?!?!?! [14:51:08] In the Ingenico test console, I changed the ejegg copy to redirect back to my mediawiki.d site [14:51:27] and made sure the frdev copy was pointing to the vagrant address [14:52:04] orr... are you getting ssl errors just trying to connect to the wiki in your browser? [14:52:48] yeah it's right at the start [14:53:11] just navigating to https://payments.wiki.local.wmftest.net:8080/w/index.php?title=Special:IngenicoGateway&appeal=JimmyQuote&ffname=cc-vmad&payment_method=cc&recurring=&uselang=en&language=en¤cy=USD&amount=35&country=US&first_name=Jimmy&last_name=Wales&street_address=1+Montgomery+Street&city=San+Francisco&state_province=CA&postal_code=94104&email=jwales%40example.com [14:53:22] gonna do some log diving see what's up [14:54:11] (CR) Jgleeson: [C: +2] Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/577245 (owner: Jgleeson) [14:54:15] actually before I do that I'll deploy that ^ [14:55:00] argh need to wait for zuul to do its bit [14:59:11] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/577245 (owner: Jgleeson) [15:20:57] fr-tech to cancel a recurring donation within civi do I set the Contribution Status to Cancelled ? [15:21:22] (was testing an adyen recurring donation after deploy) [16:04:59] jgleeson can you edit the recurring donation from the list on the contact page? [16:05:20] oh, yeah, that's the way to do it -- set it to Cancelled [16:05:31] sorry, didn't read your comment fully! [16:05:53] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [16:06:13] PROBLEM - check_disk on frpm1001 is CRITICAL: DISK CRITICAL - free space: /dev 7957 MB (100% inode=99%): /run 1441 MB (90% inode=99%): / 106 MB (0% inode=94%): /dev/shm 7974 MB (99% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 7974 MB (100% inode=99%): /boot 161 MB (64% inode=99%): /srv 704640 MB (84% inode=55%): /run/user/571 1594 MB (100% inode=99%) [16:11:13] PROBLEM - check_disk on frpm1001 is CRITICAL: DISK CRITICAL - free space: /dev 7957 MB (100% inode=99%): /run 1433 MB (89% inode=99%): / 5746 MB (10% inode=95%): /dev/shm 7974 MB (99% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 7974 MB (100% inode=99%): /boot 161 MB (64% inode=99%): /srv 686505 MB (82% inode=55%): /run/user/571 1594 MB (100% inode=99%) [16:16:11] RECOVERY - check_disk on frpm1001 is OK: DISK OK - free space: /dev 7957 MB (100% inode=99%): /run 1441 MB (90% inode=99%): / 24874 MB (46% inode=95%): /dev/shm 7974 MB (99% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 7974 MB (100% inode=99%): /boot 161 MB (64% inode=99%): /srv 664349 MB (80% inode=55%): /run/user/571 1594 MB (100% inode=99%) [16:27:09] (CR) Jgleeson: [C: +2] "This looks good but let's add a test for this scenario in another patch" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576877 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [16:27:28] thanks jgleeson [16:27:34] (Merged) jenkins-bot: Adyen: add PaymentError for normal declines [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576877 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [16:28:45] ejegg: I had a question about response codes [16:28:58] is there a place in the docs which shows which number relates to which response? [16:29:28] the example is "6" which sends back "Expired Card" [16:29:53] ahhhh found it [16:29:59] https://docs.adyen.com/development-resources/test-cards/result-code-testing/adyen-response-codes [16:30:14] oh wow [16:30:39] they're incrementing from 0 to 38 so we could write a test to run through them all [16:30:50] that's pretty convenient [16:31:43] although the 800 one seems to no longer exist which is kinda weird [16:35:10] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [16:36:06] jgleeson: yeah, it's apparently not exhaustive [16:36:23] oh [16:36:26] Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog: Fundraising tech engineers should have clear list of regular "chores" - https://phabricator.wikimedia.org/T246678 (mepps) Each shore should also be documented. [16:38:42] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [16:40:28] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Papaul) [16:40:30] fundraising-tech-ops, Operations, netops: DHCP routing issue with civi2001 - https://phabricator.wikimedia.org/T246812 (Papaul) Stalled→Resolved This is fixed. eth0 cable was connected to payments2002 eth1 [16:42:13] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Adyen_endpoint_critical 1 [=1],Amazon_endpoint_critical 1 [=1],Paypal_endpoint_critical 1 [=1] [16:43:26] Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Prepare CN deploy 2020-03 - https://phabricator.wikimedia.org/T246461 (mepps) These are the patches I see waiting for review (aside from Geotargeting and Banner templates). Do we have others @AndyRuss... [16:44:16] (CR) Mepps: "It looks like this patch is just waiting for the ids to be updated with mw-cn?" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/571767 (owner: Seddon) [16:47:13] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Adyen_endpoint_critical 1 [=1],Amazon_endpoint_critical 1 [=1],Paypal_endpoint_critical 1 [=1] [16:47:35] hmm fr-tech I'm trying to test out ejegg's response code patch https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/SmashPig/+/576891/ using the using the following test `php /vagrant/srv/SmashPig/PaymentProviders/Adyen/Maintenance/TestAdyenRecurring.php --token="1572252873.2" --currency="USD" --amount="10.00"` and I'm getting the following back from adyen's API 'internal 000 HTTP Status [16:47:37] Response - Internal server error' [16:47:41] 'internal 000 HTTP Status Response - Internal server error' [16:48:01] jgleeson: oh shoot, I'm getting the same errors [16:48:09] you had tried that earlier, right? [16:48:18] same sort of any2anyMap, etc [16:48:21] and same results? [16:48:26] yeah I think so [16:48:41] ok, I'll abandon that one [16:48:43] oh actually I dont think that stuff is being used [16:48:50] I don't include response-code [16:48:53] in the cli args [16:49:01] it's got a default value of 1 [16:49:05] for 'authorized' [16:49:28] (CR) Ejegg: [C: -1] "Doesn't seem to work. Maybe let's email Adyen and ask if it should work for SOAP?" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576891 (owner: Ejegg) [16:49:38] ah yeah [16:49:41] it works without that [16:52:13] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Adyen_endpoint_critical 1 [=1],Amazon_endpoint_critical 1 [=1],Paypal_endpoint_critical 1 [=1] [16:55:46] (CR) Jgleeson: [C: +2] Update spelling of gatewayTxnId [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576926 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [16:56:15] (Merged) jenkins-bot: Update spelling of gatewayTxnId [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/576926 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [16:57:13] RECOVERY - check_log_messages on frav1002 is OK: OK [17:01:32] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [17:12:13] PROBLEM - check_ipsec on frban2001 is CRITICAL: Strongswan CRITICAL - ok: 1 connecting: civi2001_v4 [17:16:52] right lets add that test [17:17:02] fr-tech anyone need any adyen review? [17:17:14] PROBLEM - check_ipsec on frban2001 is CRITICAL: Strongswan CRITICAL - ok: 1 connecting: civi2001_v4 [17:22:10] RECOVERY - check_ipsec on frban2001 is OK: Strongswan OK - 2 ESP OK [17:22:22] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [17:23:16] fundraising-tech-ops: rack/setup/install civi2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T242270 (Jgreen) [17:31:49] Fundraising-Backlog, fundraising-tech-ops: configure frdb1003 mariadb instance for FR Analytics use - https://phabricator.wikimedia.org/T247009 (Jgreen) [17:34:11] fundraising-tech-ops, Operations, ops-codfw: new fundraising Buster servers - bonded ethernet network error/warning - https://phabricator.wikimedia.org/T246492 (Jgreen) [18:06:28] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops, Fr-planning-ahead: Determine how much space is regained from contact deletion - https://phabricator.wikimedia.org/T246828 (Dwisehaupt) Testing on frdb2001 running the in place alter on the tables back to the same engine. On... [18:34:36] ejegg: I just noticed the branch name for this patch :) https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/SmashPig/+/576926/ [18:34:49] heh, yep [18:35:11] that r really should be there, but... [18:35:16] not for now [18:35:58] hah [18:36:02] ok, AndyRussG, I'm going to make a SmashPig patch to change those ErrorCode values to numbers [18:36:56] wondering if we should just start from 1, or if the fact that we're throwing something that gets sent into an API3 error code means we should start from something higher to hopefully not collide with other API3 error codes [18:37:32] shoot, something's come up and I need to head out for a bit [18:37:37] back soon, hopefully! [18:38:44] ejegg|afk: hmmm interesting... thx for doing that [18:39:01] I think maybe we want to avoid 0 because falsey [18:43:31] fr-tech yeah so like when did all of you want to try talking over stuff? [18:48:27] im free all day just will be relocating to a food place sometime later [18:49:17] cstone: ah okok... mmm I realized I have to go to a bank before it closes today, which is in 3 hrs [18:52:28] maybe I should try to do that sooner rather than later? [19:02:32] ok, I'll start from 1 million [19:02:34] :) [19:03:07] its better if it starts at 1 million! [19:03:26] my old company started order ids at 10000 to show "we had previous orders" [19:03:33] Wikimedia-Fundraising, Wikipedia-iOS-App-Backlog, iOS-app-feature-Feed: Fundraising banners still showing on iOS app? - https://phabricator.wikimedia.org/T242347 (JMinor) p:Medium→High One more thing that would be helpful for us: can we get the impression count for those banners during this s... [19:03:48] nobody wants to be customer #1 [19:04:36] (PS1) Jgleeson: Adyen decline handling unit tests [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577321 (https://phabricator.wikimedia.org/T243340) [19:05:01] (CR) jerkins-bot: [V: -1] Adyen decline handling unit tests [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577321 (https://phabricator.wikimedia.org/T243340) (owner: Jgleeson) [19:06:15] (PS1) Ejegg: Change values of error constants to numbers [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577322 (https://phabricator.wikimedia.org/T243340) [19:06:30] (PS2) Jgleeson: Adyen decline handling unit tests [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577321 (https://phabricator.wikimedia.org/T243340) [19:06:55] ooh, I like those tests! [19:08:06] ejegg: there's probably a bunch of refusal reasons that are in the "canRetry" method that might need double checking https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/SmashPig/+/577321/2/PaymentProviders/Adyen/Tests/phpunit/RecurringPaymentTest.php@140 [19:09:05] catch you all tomorrow o/ [19:26:34] AndyRussG: I should be free for another few hours [19:37:11] ejegg: ok gimme maybe 15 min here? :) [19:51:30] fr-tech oh yeah we have the analytics sync-up in a bit [20:18:46] Fundraising-Backlog: Invalid Pares Decline Error - https://phabricator.wikimedia.org/T247023 (Pcoombe) [20:26:06] ejegg: I got some interesting results in my testing the other day - the queries Pats are doing are faster if we DROP the contribution_status_id index - it's faster using no index because it doesn't cut down the field enough [20:37:43] eileen: i got numbers on the possible reclaimed space at: https://phabricator.wikimedia.org/T246828#5945663 [20:38:09] i could totally run that on the other tables but also looking at scripting it up to make it more sustainable. [20:39:06] dwisehaupt: yeah - I'm especially interested in the other tables as that could drive further decisions. I would love to track in grafana [20:39:25] let me see if it showed up in grafana in any useable form. [20:39:56] Also we do have some index alters we might do - they were not that slow on staging - I've been tinkering with indexes on dev_civicrm [20:43:06] yeah, we aren't capturing anything that would be of use for getting that fine grained. [20:43:25] we could write a collector to catch civi specific db stats that we'd like. [20:46:41] for reference, this alter took ~2mins and was done with the online DDL method that does it inplace and without needing locks. [20:50:37] nice - [20:56:06] Wikimedia-Fundraising-Banners: Desktop large opt-in is broken for RTL languages - https://phabricator.wikimedia.org/T246344 (Pcoombe) Thanks John. Just to close the loop: I've merged this into the bundle banner (https://he.wikipedia.org/?banner=B1920_0301_mlWW_dsk_p1_lg_cnt&country=IL) [21:16:32] fr-tech I guess I'm gonna run my bank errand, but still looking forward to some adyen talk today, this shouldn't take toooooo long!! [21:18:45] okok, I'll be around [21:21:32] I'll check that dedupe & maybe turn something off [21:22:52] hmm - it's some contact that's causing probs [21:23:37] ejegg: cool thx :) [21:23:44] also hi eileen :) [21:23:52] hi Andy :-) [21:29:44] ejegg: I have a small patch that you can maybe push through to solve MBeat 's dedupe permissions [21:30:00] ok eileen, I'll take a look [21:31:57] Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, and 3 others: Cull low value data from activity table - https://phabricator.wikimedia.org/T245088 (MBeat33) The Contributions tab is what DS uses to look up donation... [21:34:54] MBeat: I've been trying to narrow in on a dedupe problem but I found a process that works quite well in the UI - which you might like to try. [21:34:54] Basically if you allocated a range of say 5k contacts to an individual & they work through them then if you set the number of matches low (e.g 100) & then they just increment the contact id >= by 100 each hit then the the 'batch merge by contact' works really fast & you can hit that before you start deduping each 100 + with a small set everything is a bit snappier [21:34:54] (Also I have a patch for the permissions issue ) [21:36:27] thank you eileen ^^ I am working through that. where is the ‘batch merge by contact’ option? [21:36:36] in deduper? [21:36:40] yep [21:36:50] sorry it's just 'batch merge' [21:36:56] I can share screens if you want [21:38:33] would you have time for a short hangout to show Sandra and I, at some point? [21:39:50] yep sure [21:40:02] can do now - or just ping me next week [21:41:05] Sandra is free in 5 mins, would that work, eileen ? [21:41:11] sure! [21:41:14] cool [21:41:34] I will GChat invite [21:41:53] fr-tech i probably missed this from before but how does the smashpig recurring processor know to set up adyen credentials vs ingenico [21:42:25] cstone it uses the payment processor ID on the contribution recur table [21:42:31] lemme find that code [21:45:44] It's here cstone: https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/sites/default/civicrm/extensions/org.wikimedia.smashpig/CRM/Core/Payment/SmashPig.php$74 [21:45:44] ejegg: dedupe issue looks like someone has a last name of [21:45:45] [21:46:13] thanks ejegg [21:46:14] oh fun! [21:46:36] ah the pop up window family? [22:04:41] Fundraising Sprint Evil Twins For Everyone, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Investigate Automatic TY email improvements - https://phabricator.wikimedia.org/T246790 (DStrine) @Ejegg and @Eileenmcnaughton now that we have talked about this a bit more, I really just ne... [22:07:01] (CR) Ejegg: [C: +2] "This is a great test to have! It does point out some oddness in our can-retry / can't-retry logic, but we can revisit that." [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577321 (https://phabricator.wikimedia.org/T243340) (owner: Jgleeson) [22:07:49] (Merged) jenkins-bot: Adyen decline handling unit tests [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577321 (https://phabricator.wikimedia.org/T243340) (owner: Jgleeson) [22:10:59] (CR) Ejegg: [C: +2] "That's a heck of a conditional! Looks limited enough to be safe." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577020 (owner: Eileen) [22:11:47] cstone do these million-based error codes look good to you? https://gerrit.wikimedia.org/r/577322 [22:13:40] oh yeah, I guess maybe too many 0s? :P but maybe there are a million other types of errors [22:14:56] I just don't want it to collide with other CiviCRM error codes [22:15:06] yeah then perfect [22:15:31] (CR) Cstone: [C: +2] Change values of error constants to numbers [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577322 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [22:16:06] (PS2) Ejegg: Change values of error constants to numbers [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577322 (https://phabricator.wikimedia.org/T243340) [22:16:30] heh, that last-minute rebase will probably confuse zuul [22:17:34] (Merged) jenkins-bot: Override Setting api permissions so users without Administer CiviCRM can access. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577020 (owner: Eileen) [22:18:26] (PS1) Eileen: Code comments [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577360 [22:18:44] ohh - ejegg thanks - I just realised I had some uncommitted code comments - that last line above ^^ [22:20:27] (CR) Ejegg: [C: +2] "Thanks for the pointer, and I like that idea for access_arguments" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577360 (owner: Eileen) [22:25:30] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/577361 [22:25:43] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/577361 (owner: Eileen) [22:26:35] (Merged) jenkins-bot: Code comments [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/577360 (owner: Eileen) [22:33:06] fundraising-tech-ops: figure out Buster package for python3-mysql.connector for use with fruec - https://phabricator.wikimedia.org/T246823 (Dwisehaupt) We currently have the following mysql modules for python3 in the repos: ` python3_mysql_connector python3_mysqldb python3_pymysql python3_sqlalchemy ` Curre... [22:35:26] !log civicrm revision changed from 62e62e107c to 10506a9644, config revision is 734a7bfadd [22:35:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:35:45] MBeat: that should be the permissions fix deployed - can you get someone to test? [22:36:47] (PS3) Ejegg: WIP: Adapt SmashPig recurring processor for Adyen/Ingenico generic APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/576536 (https://phabricator.wikimedia.org/T242278) (owner: AndyRussG) [22:37:27] fr-tech I didn't see any other updates on that patch, so I went ahead and tried to map the errors to an appropriate PaymentProcessorException ^^^ [22:38:42] will check [22:42:42] (CR) jerkins-bot: [V: -1] WIP: Adapt SmashPig recurring processor for Adyen/Ingenico generic APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/576536 (https://phabricator.wikimedia.org/T242278) (owner: AndyRussG) [22:55:32] Fundraising Sprint Evil Twins For Everyone, Unplanned-Sprint-Work: Investigate / fix fatal errors on bad data - https://phabricator.wikimedia.org/T247039 (Eileenmcnaughton) [22:58:04] I fixed the data issue in our db causing the dedupe to fail but just checking why it did so [22:58:19] (PS1) Ejegg: Make create's isSuccessful return true on PENDING_POKE [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577372 (https://phabricator.wikimedia.org/T243340) [22:58:37] cstone: the SmashPig side needed one more tweak ^^^ [22:58:44] just smoke testing with Ingenico payments [22:59:38] ejegg for the errors would there ever be more than one coming back from the processor? [23:02:28] ok - looks like the issue is zealous escaping.... [23:04:05] cstone it's possible! [23:04:22] they do define the error fields as arrays [23:04:30] okay so that [0] part just grabs whatever one is on top to send up? [23:04:34] Fundraising Sprint Evil Twins For Everyone, Unplanned-Sprint-Work: Investigate / fix fatal errors on bad data - https://phabricator.wikimedia.org/T247039 (Eileenmcnaughton) OK - this is not a security issue but an oversecurity issue - the display name is escaped after being serialised - resulting in it n... [23:04:35] yeah [23:04:38] ok cool [23:04:55] my reasoning was that if any of the errors was a DO NOT RETRY that should take precedence [23:05:03] otherwise just grab the first [23:06:11] yeah makes sense [23:06:22] brb im gona relocate to a warmer location [23:12:08] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Adyen_endpoint_critical 3 [=1] [23:13:52] jeez, this throwing exceptions on failed auths is ugly [23:14:54] ugh, maybe we do need to fix that [23:17:09] RECOVERY - check_log_messages on frav1002 is OK: OK [23:24:00] well, I can get a successful ingenico payment charged and recorded fine [23:24:27] adyen is throwing a validation 130 Reference missing [23:24:49] hmph, what's that params['reference'] supposed to be? [23:25:00] let's use a standard parameter name for that [23:26:15] reference is our id for it [23:26:16] also back [23:26:24] it as in the payment [23:26:41] ahh, that should be called order_id then [23:26:45] ok, updating [23:26:51] thanks cstone ! [23:29:11] (PS1) Ejegg: Adyen: rename 'reference' parameter to 'order_id' [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577384 (https://phabricator.wikimedia.org/T243340) [23:38:46] oops, or should we be using invoice_id? [23:38:48] lemme see [23:39:40] Fundraising Sprint Evil Twins For Everyone, Unplanned-Sprint-Work: Investigate / fix fatal errors on bad data - https://phabricator.wikimedia.org/T247039 (Eileenmcnaughton) I'm kinda stuck on this - I documented where I got to https://github.com/civicrm/civicrm-core/pull/16694 & if no-one has any ideas... [23:39:56] no, it's order_id [23:44:47] does civi use order_id anywhere? [23:44:48] (PS4) Ejegg: WIP: Adapt SmashPig recurring processor for Adyen/Ingenico generic APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/576536 (https://phabricator.wikimedia.org/T242278) (owner: AndyRussG) [23:45:04] cstone nope, civi's term is invoice_id [23:45:15] and honestly i'd like to move us toward using that instead of order_id [23:45:31] hah okay cause order_id wasn't even on the list of ids hah [23:45:31] I think once we get rid of old globalcollect we actually can [23:45:36] ahh okay [23:46:10] So for all the other processors order_id = invoice_id = our-side-generated-thing [23:46:23] but for the old globalcollect order_id = gateway_txn_id [23:46:26] which I hate [23:46:35] and is one more motivation to get rid of that integration [23:46:51] anyway, maybe we can just start calling it invoice_id once that's gone [23:47:45] ah i see yeah [23:48:50] ok, PS4 of that got me through an Adyen payment. [23:48:59] lemme try again with Ingenico [23:49:51] PS4 still doesn't have special handling for the DO_NOT_RETRY [23:50:50] cstone also, I think I know why we never saw the 300260 errors up where we thought we might [23:51:22] like jgleeson pointed out, we're throwing an ApiException in SmashPig when any errors exist on the base 'errors' property of the ingenico response [23:51:46] (CR) jerkins-bot: [V: -1] WIP: Adapt SmashPig recurring processor for Adyen/Ingenico generic APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/576536 (https://phabricator.wikimedia.org/T242278) (owner: AndyRussG) [23:51:46] and only passing the numerical error back up if the error is on the paymentStatusResponse [23:52:18] so.... I guess we do need to fix that??? [23:52:42] so the 300620 was never getting up to where it was checked? [23:52:55] I think so? [23:53:11] lemme get some links, one sec [23:53:43] Fundraising Sprint Byzantine Empire Strikes Back, Fundraising Sprint CAPS LOCK CULTS, Fundraising Sprint Dampness, Fundraising-Backlog, and 2 others: Civi: build out TY email translations and Language Preference settings - https://phabricator.wikimedia.org/T227903 (MBeat33) Thanks, all. [23:55:41] cstone: ok, so here's where we throw an exception if the Ingenico API response has an 'errors' top-level property: https://phabricator.wikimedia.org/diffusion/WFSP/browse/master/PaymentProviders/Ingenico/Api.php$94 [23:55:51] (CR) Cstone: [C: +2] Make create's isSuccessful return true on PENDING_POKE [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577372 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [23:56:12] (Merged) jenkins-bot: Make create's isSuccessful return true on PENDING_POKE [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/577372 (https://phabricator.wikimedia.org/T243340) (owner: Ejegg) [23:57:01] and here's a standard 'not authorized' response: https://epayments-api.developer-ingenico.com/s2sapi/v1/en_US/java/payments/create.html?paymentPlatform=ALL#payments-create-response-402 [23:57:34] note that it has the 'errors' in the paymentResult/payment/statusOutput [23:57:44] but that it ALSO has the 'errors' up at the top level [23:58:22] so.... I guess the response for the 300260 ones was the same [23:58:30] and was also triggering an ApiException [23:58:44] I guess now is the time to fix it, huh? [23:59:10] I was wary a couple days ago when jgleeson suggested doing that [23:59:13] but he was totally right [23:59:36] current behavior is just broken and confusing to library users