[11:12:12] Hi there. Just ran into [Wyo2bQpAICkAAJld-ssAAACE] 2018-06-20 11:11:41: Fatal exception of type "BannerExistenceException" [11:12:19] on CentralNoticeBanners [13:19:17] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Fatal exception of type "BannerExistenceException" - https://phabricator.wikimedia.org/T197793#4302659 (mepps) [13:28:05] fr-tech i made this based on Hauskatze's report ^^ [13:28:21] +1 [13:59:58] PSA: 18mo until python2 EOL [14:00:05] never too soon to start updating [14:02:30] https://docs.python.org/2/library/2to3.html [14:07:41] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Fatal exception of type "BannerExistenceException" - https://phabricator.wikimedia.org/T197793#4302790 (MarcoAurelio) I manually typed a banner name in the URL bar, it seems it didn't existed with that name and instead of returning a "this page doe... [14:09:22] mepps: thanks! fr-tech hi all [14:15:02] hey AndyRussG ; [14:15:04] ) [14:57:48] cwd ooh, good to know [14:58:19] are all of our tools scripts still running under python2 ? [14:58:49] That would be the audit downloaders, the silverpop export, and the paypal audit parser [14:59:19] unless they specify 3 [14:59:21] paypal audit parser should be rewritten in smashpig to match the rest [14:59:33] /usr/bin/python is a symlink to python2 [15:00:12] ok, I'll see what we've got for tickets [15:00:21] but 3 is available everywhere [15:08:20] Fundraising-Backlog: All fundraising python scripts should run on python 3 by 2020 - https://phabricator.wikimedia.org/T197805#4303079 (Ejegg) [15:26:28] ejegg, I finally managed to get a successful recurring transaction! [15:26:34] jgleeson: great! [15:26:45] Did you figure out what was the problem with the previous one? [15:27:15] And can we add some better error handling to make it easier to diagnose? [15:30:32] There was a mix of reasons, one of them was due to a shared token across different transactions. When testing, the token being generated for the test card details I was using (regardless of other input) was the same so during the donation saving process, it was linking any new recurring transactions to the earliest one to use that token which was confusing the hell out of me! I also think this was causing follow-on payments to [15:30:32] be taken in the wrong currency. I'm still a little unclear on if this is a problem that needs fixing or was some local env quirks [15:30:49] token returned from ingenico* [15:31:29] jgleeson: ok, got it [15:31:43] so the currency problem appeared to be that the first transaction was taken in EUR, but then recurring payments were attempted in dollars [15:31:50] so we can only use each token in a single currency? [15:31:51] and ingenico was rejecting this [15:32:00] I think so [15:32:26] I might amend my test payment cli script to allow a currency arg [15:32:29] hmm, their test cases are all have a single currency / amount for any given card number [15:32:31] then I can confirm that [15:33:02] i think they give us the same token for a given CC number each time [15:33:12] yep, that's true [15:33:26] btw, he is my quick access test card details I'm using https://docs.google.com/spreadsheets/d/1JOhxAlKk4BdR-Eo75fuSfG0bUOuEgAHr0LI-XMxVjkY/edit#gid=1759048558 [15:33:29] but maybe they'll give us a second one if we try to tokenize a different-currency initial payment [15:33:59] what's the error that's returned? [15:34:12] unauthorised [15:34:23] I'll see if I can recreate it [15:34:40] I think I emailed it over friday, the last failure repsonse from the API [15:34:52] sorry, monday [15:35:50] out of interest, if we take a recurring payment in EUR would follow on recurring payments be EUR or USD? [15:35:59] ejegg, ^ [15:37:03] jgleeson: always in the same amount and currency as the original donation [15:37:23] I need to add some documentation somewhere for testing payments with tokens, as I forgot how to use this tool until finding the PR https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/SmashPig/+/428945/ [15:37:31] jgleeson: oh hey, I should definitely add a non-USD test case [15:41:20] ejegg: for the FundraiserLandingPage schema, did we forget appeal-template? Or am I forgetting about some reason for not putting it? https://github.com/wikimedia/wikimedia-fundraising-tools-DjangoBannerStats/blob/master/fundraiser/analytics/management/commands/LoadLPImpressions.py#L341 [15:41:25] https://meta.wikimedia.org/wiki/Schema:LandingPageImpression [15:45:53] ok ejegg, I just test out two separate payments using the same token and if the value changes, regardless of currency, it fails [15:47:10] I was able to submit $35 and €35 but no other amount other than 35 [15:47:37] which effectively prevents cross currency transactions with a single token due to exchange rate [15:50:49] jgleeson: thanks for testing that out! [15:51:36] AndyRussG: good question! [15:52:04] AndyRussG: that sounds like a parameter we might not use much these days [15:53:38] hmmm... also https://www.mediawiki.org/wiki/User:AGreen_(WMF)/Draft:Mapping_of_EventLogging_properties_to_FundraisingImpressions_database [15:54:18] AndyRussG: ah, donatewiki_unique might actually be used [15:56:28] AndyRussG: yeah, that and donatewiki_counts are the self-hosted way to get email clickthroughs [15:56:51] ejegg: hmmm.... [15:56:59] since silverpop used to count those on a spammy-looking domain [15:57:54] they've started allowing custom domains, but ccogdill has been trying to get an https cert for a silverpop-hosted links.wikimedia.org for months now [15:58:36] got an update yesterday that they're still researching it... [15:58:42] if that's true :D [15:58:59] ccogdill: fingers crossed! [16:04:23] fr-tech meeting? [16:47:24] fr-tech anything for SoS? [16:47:35] Na, nothing here [17:15:10] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Matt Damon to head up Space Force, and 2 others: add Partner field to summary section ... - https://phabricator.wikimedia.org/T185048#4303536 [18:03:55] oops, i seem to have the power to create new gerrit branches but not delete them. [18:04:17] kinda jumped the gun on the fr/REL1_31 branch [18:11:15] (PS1) Mepps: WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) [18:11:37] (PS3) Ejegg: Delete unused extensions and skins [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/440415 [18:11:40] (PS3) Ejegg: Add WMF logos [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/440416 [18:11:43] (PS3) Ejegg: Add payments-wiki extensions [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/440417 [18:11:46] (PS3) Ejegg: Update vendor for merged fundraising deps [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/440418 [18:14:21] (CR) Ejegg: WIP: Return url without Title Class for Ingenico (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [18:15:52] (CR) Ejegg: "Patch looks good! Just hard to think of a use case for setting returnTitle rather than returnUrl in the options" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [18:33:33] XenoRyet: moar crash? [18:33:56] Na, just another intentional restart and forgot to bring IRC back up after [18:34:08] ahh [18:34:28] there might be info on the crash in /var/log/messages.1 or similar [18:35:05] It was my windows box that did it. Might be faulty PSU but worst case scenario I just do everything on the lappy today. [18:35:24] ah, in that case I dunno where the logs would go [18:36:23] It's in event viewer, but not much in there either, which is why I suspect PSU. Almost anything but a power interrupt would either blue screen or leave a log. [18:36:49] sounds like solid reasoning [18:38:20] XenoRyet: if you've got some free time, I'd be happy to discuss any doubts you have on the civi / recurring extension [18:38:56] hoping to deploy that today or tomorrow [18:39:00] Yea, let me get back to looking at that. I'll let you know if any questions or problems come up. [18:39:31] ok, feel free to suggest code re-organization or the like [18:40:07] Yea, will do. [18:52:25] fundraising-tech-ops: encrypt fundraising nfs traffic - https://phabricator.wikimedia.org/T170326#4303707 (Jgreen) Open>Resolved This is done--banner loggers now block incoming nfs connections unless they're ipsec, and block outbound non-TLS access to kafka-jumbo servers. [19:07:31] ejegg: okok so just to confirm updates for KafkaTree, we'll keep the donatewiki_unique table, and add the appeal-template property to the event (to be taken fro the URL to the landing page, just like the rest, no? [19:07:59] AndyRussG: yeah, that sounds good [19:08:08] ejegg: okok :) [19:08:38] I'm not sure how much value that appeal-template has (it's concatenated in to the landing page) so if that part is time consuming we can probably skip it [19:20:23] (PS1) Ejegg: JobQueueConsumer can hydrate non-JsonSerializableObjects [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441250 [19:25:03] (PS1) Ejegg: WIP job to add tokens to Ingenico payments [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441251 [19:25:27] (CR) jerkins-bot: [V: -1] WIP job to add tokens to Ingenico payments [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441251 (owner: Ejegg) [19:27:20] (PS2) Ejegg: WIP job to add tokens to Ingenico payments [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441251 [19:31:17] ejegg: it's actually just a second or three to update [19:32:11] for teck-tock, is it the hangout that's on the calendar, or the other one? [19:34:03] fr-tech -teck ^ ? [19:36:03] I guess either would work, so long as we all show up in the same one. [19:36:19] or we could all have both open at the same time [19:36:27] echo echo echo ... [19:36:33] heheh [19:36:44] eileen and I are in the one on the calendar [19:36:53] :-) [19:36:56] Me too now [19:37:12] Though I don't have much to talk about today [19:38:22] mepps: You still around for tech-talk? [19:41:21] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Matt Damon to head up Space Force, and 2 others: add Partner field to summary section ... - https://phabricator.wikimedia.org/T185048#4303805 [19:48:51] (CR) Mepps: "Ejegg that's taken from globalcollect--is it used there? i'll see if I can figure that out." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [20:05:08] thanks AndyRussG ! [20:07:48] (PS1) Ejegg: Simplify return stuffing [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441258 [20:09:29] (CR) Ejegg: "Oh weird, we WERE using it, but only in a spot where we ALSO set returnTo, so it never got called. If it had gotten called, it would have " [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [20:11:10] fr-tech anyone else had a chance to review the civi extension for smashpig recurring? [20:11:21] hoping to deploy that today or tomorrow [20:12:44] XenoRyet are you able to review the recurring extension with the new url? [20:14:00] Yea. Well, getting a step farther anyway. [20:14:48] I feel like the code looks good, I'd just like for one or more of us to be able to do a successful test. [20:15:42] jgleeson said that he finally got one through this morning [20:15:57] he had just been trying to use the same token with two different currencies [20:16:38] Cool, well I'm halfway into getting one through so let me bang on it a little more. [20:17:01] drush qc is complaining about something related to the geocoder though? [20:17:19] hmm, drush qc? [20:17:28] you have the geocoder extension installed? [20:17:51] Probably not unless vagrant style civi comes with it. [20:18:33] XenoRyet: it's checked in to the repo, and our wmf_civicrm install script should activate it on drush updb [20:19:10] hmm... let me go back and look at the output from updb [20:19:11] that's the extension that eileen created to share our postal-code-db-based geocoding with the world [20:20:16] (and so our geocoding was less based on custom hooks, and would show up in standard places) [20:27:57] a gust of wind just knocked over a glass on the courtyard tile and a few people rushed in to help clean it up [20:28:19] and simultaneously the network connection died for about 3 minutes [20:28:37] wifi can make you superstitious [20:28:39] spooky [20:35:43] XenoRyet: can you save a contact normally without any complaints about geocoder? [20:35:49] also, what's the exact error? [20:39:14] Oh, worked it out [20:39:45] Now stuff is getting consumed, but it's not showing up in the recurring table [20:40:40] XenoRyet: So the original donation didn't create a row in civicrm_contribution_recur? [20:40:54] right [20:41:19] XenoRyet: do you see payment processors defined at /civicrm/admin/paymentProcessor?reset=1 [20:41:59] also, anything in the table civicrm_payment_token? [20:42:53] yea, one row in payment_token [20:43:47] ok, weird, making that is the last step before creating the row in recurring [20:43:59] anything in your civi error logs? [20:44:00] and the token in there matches what was on the queue before [20:44:08] let me look [20:47:07] oh weird, the row is there now. [20:47:15] Does it take a minute for it to go in or something? [20:47:28] Anyway, proceeding [20:48:05] err, that would be weird [20:48:22] XenoRyet: just gotta update that next_sched_contribution_date [20:48:32] Yep, doing that now [21:01:12] Ok, yea. [21:01:19] Got stuff all the way through. Looking good [21:02:03] (CR) XenoRyet: [C: 2] "Code looks fine, and testing went well." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T188678) (owner: Ejegg) [21:03:12] ejegg: There you go. [21:03:21] Gonna go grab some food now. [21:08:19] (Merged) jenkins-bot: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T188678) (owner: Ejegg) [21:11:21] (CR) Ejegg: "That IS weird!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) (owner: Eileen) [21:13:32] (PS4) Ejegg: Clean up unused audit regex functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/436079 [21:13:56] (PS4) Ejegg: Reformat Translation.php with drupal style [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440023 [21:13:58] (PS4) Ejegg: Add custom ts() override fn to use MW messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440026 [21:14:13] those last two translation ones would be nice to have ^^^ [21:15:32] well, the translation override might actually be a horrible solution [21:15:32] but it'd be nice to have feedback on it [21:15:59] without them or some similar solution, we'll be using english on all the cc statements for the recurring charges [21:18:16] !log updated 'domain' settings in CiviCRM so name is now 'Wikimedia Foundation' and description is 'donate.wikimedia.org' [21:18:17] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:22:00] didn't /LOOK/ like that was going to break anything ^^^ [21:23:26] hmm, the 1 hour test did target enwiki, so maybe we're good to try the extension [21:32:13] ah dang, there's NOTHING in civicrm_payment_token ? [21:32:30] shoooooot [21:32:40] ok, gotta write a script to backfill that, I guess [22:03:36] ejegg: [22:03:51] I realise I never got back to giving you much feedback on your smashpig extension [22:07:18] (PS3) Eileen: Add location_id to civicrm_group_contact table. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) [22:08:52] (CR) jerkins-bot: [V: -1] Add location_id to civicrm_group_contact table. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) (owner: Eileen) [22:12:59] (PS4) Eileen: Add location_id to civicrm_group_contact table. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) [22:29:58] (CR) Ejegg: [C: 2] Add location_id to civicrm_group_contact table. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) (owner: Eileen) [22:35:06] (Merged) jenkins-bot: Add location_id to civicrm_group_contact table. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440981 (https://phabricator.wikimedia.org/T197045) (owner: Eileen) [22:49:07] (PS1) Ejegg: WIP tokenize recurring payments stuck in damaged queue [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) [22:49:31] XenoRyet: how does that last one look to you? [22:49:32] ^^^ [22:49:46] Somehow ALL of the recurring payments are missing tokens [22:50:16] which I guess makes some sense - we initially imported them before we created the paymentProcessor records [22:50:45] but if you look in the logs, you don't see a single result coming back with the tokens like we get in sandbox [22:50:59] so... idunno where they went [22:51:53] Let me take a look [22:53:04] it's as if every one of the recurring ones we attempted had 3dsecure activated [23:05:33] (PS2) Ejegg: WIP tokenize recurring payments stuck in damaged queue [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) [23:05:48] fixes a couple glaring errors ^^ [23:07:01] 10-4 [23:24:30] (PS19) Eileen: Add wmffraud into the showme/forgetme [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440253 [23:40:46] XenoRyet: any thoughts on that one?