[00:55:04] Fundraising Tech Backlog: Create Appeal option group and link to Direct Mail Appeal on dev - https://phabricator.wikimedia.org/T106105#1459269 (cwdent) NEW a:cwdent [02:14:19] (PS4) Ejegg: Remove unused logfile parsing code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/207742 (owner: Awight) [02:14:28] (CR) Ejegg: [C: 2] Remove unused logfile parsing code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/207742 (owner: Awight) [02:14:56] (Merged) jenkins-bot: Remove unused logfile parsing code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/207742 (owner: Awight) [12:55:30] Fundraising-Backlog, Astropay Integration: Update Brazil fiscal number to CPF - https://phabricator.wikimedia.org/T106137#1460158 (atgo) NEW [15:59:39] Hi awight ! I'm thinking of just rolling back the donation-interface dependency in CRM for now. Everybody's busy, and things like this worry me: https://gerrit.wikimedia.org/r/225265 [16:04:45] you mean rolling back to a previous version before the problem started? [16:04:51] yep [16:05:10] and rolling back yesterday's patch that accomodates the newness [16:05:44] yeah i dug for awhile but felt like there are about 2000 things i don't understand here [16:06:02] that failing test just creeps me out [16:06:10] i wonder if our time wouldn't be better spent porting globalcollect recurring to smash pig [16:06:52] how big of a job is that? weeks, months? [16:07:29] pretty big, unfortunately [16:07:57] all smashpig does now is parse listener messages and audit files [16:08:15] it has no idea how to initiate an api call on its own, or to interpret responses [16:09:17] would you say the code that does that stuff in DI is not portable? [16:09:22] or we could try merging the logging patches and run another batch of 1 [16:09:54] cwdent: not yet - it depends on a large number of gateway adapter calls [16:12:40] want to try merging logging first? [16:17:35] yeah some more info can't hurt [16:19:12] ejegg: can you link me that change? it's not in my list [16:19:26] oh sure, 1 sec [16:20:08] https://gerrit.wikimedia.org/r/225257 and https://gerrit.wikimedia.org/r/225263 [16:20:26] then I'll need to do another composer update patch [16:22:02] ejegg: Sorry, missed your ping. Yeah, rolling back is a great way to handle it for now. [16:22:20] http://wiki.civicrm.org/confluence/display/CRMDOC40/CiviCRM+Public+APIs [16:22:36] the examples of nested calls are syntax error [16:23:17] cwdent: hehe, not enuf punctuation [16:24:02] sun O))) [16:24:20] cwdent: here are the current docs http://wiki.civicrm.org/confluence/display/CRMDOC/Using+the+API [16:24:51] oh nice, dunno why i didn't turn that up [16:25:12] * ejegg looks up how to fake the composer.lock hash [16:25:12] Cos the search function is broken on that site :) [16:25:46] There's a banner warning about old docs, other than that it's easy to accidentally end up in the past [16:25:51] ejegg: gah [16:26:06] Maybe checkout -p the file? [16:26:09] ah, it's just used for out-of-date warnings [16:26:24] awight: sadly, i don't want to roll back all of vendor - we need the updated smash-pig [16:26:43] -p helps you revert just one chunk at a time... [16:27:45] oh, the d-i rollback is as easy as changing the one 'reference' line [16:27:48] Gotta switch rooms real quick... [16:27:49] yah [16:28:01] i just was afraid I had to re-hash the lock file to make it install [17:01:45] ejegg: how are you feeling about the expanded logging vs just rolling back for now? [17:03:29] cwdent: I'm still curious about logging [17:03:37] I'll prep a rollback patch too [17:04:00] cool, maybe try one more to get some clues and then roll DI back? [17:04:07] yeah [17:22:17] cwdent / XenoRyet I do need some code review on https://gerrit.wikimedia.org/r/225257 and https://gerrit.wikimedia.org/r/225263 to get that extra logging info [17:22:50] actually, if we just merge https://gerrit.wikimedia.org/r/#/c/225263/ I can run drush with -d and see it all [17:23:09] easier, too - no need to mess with composer / vendor submodule [17:27:22] cool yeah i've looked at the code and it seem fine although i don't know that much about these modules [17:27:44] i was going to check it out and run the tests but my working tree is dirty [17:28:01] In the DonationInterface.php one, it looks like you're just creating a new global and setting it true. Where does that value end up getting used? [17:28:29] XenoRyet: it's used in DonationInterface [17:29:56] I mean, DonationInterface the mediawiki extension [17:30:58] it's just another thing you need to spoof to get it to run? [17:31:11] under drupal [17:31:31] yeah, it's not needed to get it to run, but I want DI to send all of its logs to drupal [17:31:39] without that, it'll only send info and up [17:31:58] gotcha [17:35:41] Ok, seems cool [17:36:01] cwdent, you've had your nose in this more than me though. [17:36:17] hehe, maybe slightly [17:36:23] i'll merge them [17:36:31] ejegg: sound good? [17:36:59] cwdent: sure! I think I will just deploy the crm one for now, and try running drush with -d [17:37:08] but it would be nice to have the DI one for the future [17:37:16] cool [17:37:22] (CR) Cdentinger: [C: 2] Get all the log messages from DonationInterface [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225263 (owner: Ejegg) [17:37:24] (CR) Cdentinger: [C: 2] Quit demoting log messages to debug under drupal [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225257 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [17:37:38] thanks! [17:38:34] (Merged) jenkins-bot: Get all the log messages from DonationInterface [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225263 (owner: Ejegg) [17:38:36] (Merged) jenkins-bot: Quit demoting log messages to debug under drupal [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225257 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [17:39:56] (PS1) Ejegg: Roll back donation-interface lib, revert module updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225342 [17:39:58] (CR) jenkins-bot: [V: -1] Roll back donation-interface lib, revert module updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225342 (owner: Ejegg) [17:43:57] (PS2) Ejegg: Roll back donation-interface lib, revert module updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225342 [17:45:19] (CR) jenkins-bot: [V: -1] Roll back donation-interface lib, revert module updates [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225342 (owner: Ejegg) [17:46:16] oh fack [17:46:41] Worldpay case change means that rollback breaks worldpay audit tests [17:47:01] onward through the storm! [17:47:49] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225343 [17:47:50] ohshi [17:47:59] can we cherry pick that? [17:48:22] composer libs are kinda all or nothing [17:48:35] less we wanna re-fork DI [17:49:08] maybe just skip worldpay tests if we have to go this route [17:49:28] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225343 (owner: Ejegg) [17:49:30] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225343 (owner: Ejegg) [17:50:33] !log updated civicrm from fa724dd2e2e69545d81015c943cb7f52cf6de8e1 to 0acac037ce0c9a64e94a475463deb2d47e84193a [17:50:38] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [17:52:56] that build fail cause of the tests? [17:54:52] cwdent the smashpig failmail just now? [17:55:03] that was me re-running a batch of 1 [17:55:12] useful log data though! [17:55:26] Did you get your payments-jenkins account? [17:56:20] last thing DonationInterface logged was the response from DO_PAYMENT with NOK on expired card [17:56:41] huh! [17:56:46] just as had been supposed [17:56:56] dang thing strips the XML tags, or I'd be able to fix the worrying test in DonationInterface [18:25:11] Fundraising-Backlog, Astropay Integration: Validate CPF for AstroPay - https://phabricator.wikimedia.org/T106171#1460882 (Ejegg) NEW [18:34:06] Fundraising-Backlog, Astropay Integration: Validate CPF for AstroPay - https://phabricator.wikimedia.org/T106171#1460899 (atgo) p:Triage>Normal [18:36:41] (PS1) Ejegg: WIP Check CPF is numeric when exists, require for AstroPay [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225349 (https://phabricator.wikimedia.org/T106171) [18:37:44] Fundraising-Backlog, Astropay Integration, Patch-For-Review: Validate CPF for AstroPay - https://phabricator.wikimedia.org/T106171#1460921 (Ejegg) We'll need a new calculated validation function that takes country into account. This table may help: https://en.wikipedia.org/wiki/VAT_identification_num... [18:41:20] Fundraising Sprint ODB, Fundraising-Backlog, Astropay Integration, Unplanned-Sprint-Work, Patch-For-Review: Validate CPF for AstroPay - https://phabricator.wikimedia.org/T106171#1460924 (atgo) While you're in there... T106137? If not, we can hit it in the next sprint. [18:45:48] Fundraising-Backlog: Change error messages on new GC red text (was Change errors on GC forms from popup to red text) - https://phabricator.wikimedia.org/T106172#1460940 (Krenair) [18:49:57] Fundraising-Backlog: Change error messages on new GC red text (was Change errors on GC forms from popup to red text) - https://phabricator.wikimedia.org/T106172#1460949 (atgo) thanks @krenair :) [19:06:58] (PS1) XenoRyet: Rearrange order of CC logos on GC form for Japan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225359 [19:14:57] (PS2) XenoRyet: Rearrange order of CC logos on GC form for Japan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225359 (https://phabricator.wikimedia.org/T102496) [19:16:37] Is there a quick way to get gerrit to add everybody on our team as reviewers to a patch? [19:17:56] Nevermind, found it. [19:59:49] (PS1) Ejegg: Get around watchdog stripping tags [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225375 [20:00:17] XenoRyet: / cwdent one more tiny log patch? ^^^ [20:01:03] checkin... [20:01:35] crappy, but the essential thing i need to know in one of those logs is whether the tag name is STATUS or ROW [20:02:11] will anything besides globalcollect recurring hit this? [20:02:33] Only if we call DI from elsewhere in drupal [20:02:51] ...so no [20:03:01] cool [20:03:20] u guise still want to standup? [20:03:25] sure [20:03:32] Could. [20:03:39] Just in here is fine too if you guys want [20:03:47] ok. [20:03:51] I'm cool either way [20:04:02] i think i can get a real fix in for rgc soon [20:04:15] ejegg: i was hoping you could explain something to me quick [20:04:26] might be easier on the videophone [20:04:34] sure, ping me! [20:04:44] i'll just join standup [20:18:09] (CR) Cdentinger: [C: 2] Get around watchdog stripping tags [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225375 (owner: Ejegg) [20:18:31] (Merged) jenkins-bot: Get around watchdog stripping tags [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225375 (owner: Ejegg) [21:39:16] cwdent / XenoRyet : time to help merge https://gerrit.wikimedia.org/r/#/c/225491/ ? [21:41:33] sure just gonna push a patch up [21:42:02] got those option groups sorted? [21:42:25] heh sorta...it's fugly [21:42:46] if you don't mind just peeking at mine, it should only take 1 sec - just composer.lock [21:43:26] sorry to be frantic, just hoping to get the real fix up this evening [21:44:10] np! are the phpse parts intentional? [21:44:35] shouldn't hurt anyway [21:44:42] cool [21:45:03] (CR) Cdentinger: [C: 2] Update DI lib for logging fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225491 (owner: Ejegg) [21:45:22] thanks! [21:46:09] (Merged) jenkins-bot: Update DI lib for logging fixes [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225491 (owner: Ejegg) [21:46:41] (PS1) Cdentinger: amend update to fix Direct Mail Appeal custom field [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225492 [21:47:21] ^ pretty silly, but it's net negative code [21:47:29] nice! [21:47:37] (CR) jenkins-bot: [V: -1] amend update to fix Direct Mail Appeal custom field [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225492 (owner: Cdentinger) [21:47:44] sigh [21:48:33] (PS1) Ejegg: Update donation-interface dependency [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/225493 [21:48:57] (CR) Ejegg: [C: 2 V: 2] Update donation-interface dependency [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/225493 (owner: Ejegg) [21:49:45] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225496 [21:50:10] (PS2) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225496 [21:50:33] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/225496 (owner: Ejegg) [21:51:35] !log updated civicrm from 0acac037ce0c9a64e94a475463deb2d47e84193a to 78de1b9b74934984af3099afe9192fa53011bdaa [21:51:40] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:52:34] failmail in 5... [21:52:35] 4... [22:04:51] (PS2) Cdentinger: amend update to fix Direct Mail Appeal custom field [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/225492 [22:15:41] (PS2) Ejegg: Fix recurring GC false success report [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225265 (https://phabricator.wikimedia.org/T105848) [22:16:47] cwdent I think that should let us turn recurring back on ^^^ [22:17:24] cool, checkin it out [22:17:38] the test XML responses had been a bit out of touch with reality [22:18:25] these are what get picked up by the curl wrapper? [22:18:35] and I think the logic from march had two errors that cancelled each other out to get the right success status [22:18:39] cwdent: yep! [22:19:42] you set the code to xyz, and for transaction DO_PAYMENT in globalcollect it returns globalcollect/DO_PAYMENT_xyz.testresponse [22:19:52] so I added a new test for the NOK status [22:20:28] cool [22:23:33] gonna do my TestingGlobalCollect substitution in crm and make sure that patch works as advertised for both responses from inside drupal [22:24:09] sounds good! i'm getting my local stuff up to date [22:24:25] oops, needs rebase... [22:24:29] (PS3) Ejegg: Fix recurring GC false success report [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225265 (https://phabricator.wikimedia.org/T105848) [22:25:04] DI is a git repo inside a composer dir that is a submodule [22:25:08] heh [22:25:28] I've got an appointment I need to get to. I'll be back later. [22:25:37] see you then! [22:35:09] cwdent: OK, we still need to make that stub not throw an exception. Just going to return the message code right back for now [22:35:56] the getLanguage/formatMessage stubs? [22:36:33] at least the formatMessage one. Trying just that for now [22:37:50] cool, that [22:37:51] 's [22:37:53] all it needed [22:37:57] so DI must not have hit that code before, was that part of the refactor? [22:39:18] (PS1) Ejegg: Stop exceptioning in drupal formatMessage stub [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225508 (https://phabricator.wikimedia.org/T105848) [22:39:41] yeah, indirectly somehow! [22:40:03] I tried making the error message handling more consistent [22:40:16] and i think that pushed translation earlier [22:40:27] aah, ok [22:40:38] probably a bad design, since here we're not actually using the translation at all [22:40:39] but you discovered an actual failure that had nothing to do with that exception? [22:41:03] so, we were parsing the response to the first call completely wrong [22:41:11] in a way that would set the txn to failed [22:41:26] but then we weren't checking for failed status before doing the second transaction [22:41:51] does 2nd transaction mean retrying the failed one? [22:41:59] ah, no [22:42:04] for recurring, we make 3 api calls [22:42:25] and in donation interface, a single api call is called a 'transaction' in a lot of places [22:42:31] we really should rename that [22:42:49] oh ok, yeah confusing since it's mixed up with finances [22:43:04] so, a successful recurring payment is 3 api calls [22:43:19] a do_payment, a get_orderstatus, then a set_payment [22:43:36] but a failed recurring payment sends us NOK on that first api call [22:43:59] So, I'm not sure if the logic from march was always making 3 api calls, even when the first said NOK [22:44:11] since the DI internal calls were never part of the drush logs [22:45:00] huh, so was it only that code path that would somehow hit the translate exception? [22:45:11] anyway, a refactor I did after that split up the 'success' status into 'did the actual api call get a response' and 'did the payment work out' [22:45:24] oh, right, we set an error message on NOK payments [22:45:40] and my new logic was translating that earlier in the codepath [22:45:47] aha [22:46:19] so this patch fixes the original problem with not parsing api response # 1 correctly [22:46:39] then tests for payment success, not just api call return happened [22:46:48] before going on to calls 2 and 3 [22:47:17] ok cool, that makes sense, and all the tests pass. wanna go for it? [22:52:11] ejegg: am i right thinking the nature of the failures is that we were not properly checking the failed status of the previous row and trying to change in this case an expired card? [22:52:24] *charge [22:52:33] yeah, i think so [22:53:37] but the 2nd or 3rd api call was still failing in a way we were picking up as failed. So I think before my refactor, we were still getting the correct end status [22:53:48] just doing 1 or 2 unneccessary api calls [22:54:28] (which could theoretically get us fined by mastercard, for certain failure codes!) [22:55:07] yep, so it sounds like nothing was really going wrong we just need to not try to recharge txns that are known to fail [22:55:14] that's good [22:55:38] then i did my refactor and split up the success status, and the only thing stopping us marking failed txns as success was the exception in the method stub! [22:56:29] hehe, unexpected benefit [22:57:04] so would any failed transaction have thrown that exception? because it was trying to translate the error message? [23:01:35] Fundraising Tech Backlog: DonationInterface shouldn't say every single API call is a 'transaction' - https://phabricator.wikimedia.org/T106196#1461613 (Ejegg) NEW [23:02:23] cwdent: yep, all failed txns are currently ending in an exception [23:03:05] which may mean there are some rows in contribution_recur that are locked for processing but should be marked as failures [23:04:00] this should clean them up once they run? [23:04:40] no, i think i'll have to manually set them to failed [23:05:41] we're retrying all failures after a couple days, which is dumb. atgo has a task to perma-fail things like expired cards and send the donor an email [23:05:44] https://phabricator.wikimedia.org/T103449 [23:06:19] aha [23:07:44] gonna hop on the subway, but i'll keep irc on phone (subway tunnels have 4g now!) [23:07:54] still hoping to get this fixed tonight [23:21:33] (CR) Cdentinger: [C: 2] Fix recurring GC false success report [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225265 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [23:21:42] (CR) Cdentinger: [C: 2] Stop exceptioning in drupal formatMessage stub [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225508 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [23:21:55] (Merged) jenkins-bot: Fix recurring GC false success report [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225265 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [23:23:29] (PS2) Ejegg: Stop exceptioning in drupal formatMessage stub [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225508 (https://phabricator.wikimedia.org/T105848) [23:24:42] (CR) Ejegg: [C: 2] "Adding another +2 to get Jenkins to merge the rebase" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225508 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [23:25:05] (Merged) jenkins-bot: Stop exceptioning in drupal formatMessage stub [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/225508 (https://phabricator.wikimedia.org/T105848) (owner: Ejegg) [23:29:21] thanks! [23:30:16] np! are you going to deploy that from your phone? [23:31:06] i will do it if you want [23:31:20] heh, got my laptop here too [23:31:41] bus app says 10 minutes, i can at least get the composer package updated! [23:57:10] (PS1) Ejegg: Update donation-interface [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/225514