[16:45:59] who's the best person to direct feedback on the new civi updates to? [16:55:45] (CR) Ejegg: [C: 2] (FR #1497) More import columns for communication prefs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/162187 (owner: Awight) [16:57:33] (CR) Ejegg: [C: 2] Import notes into Civi [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163082 (owner: Awight) [16:58:50] ejegg: thanks! [16:59:49] (PS1) Awight: Merge remote-tracking branch 'origin/master' into HEAD [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163602 [17:00:05] awight: is the standalone framework ready for review? [17:00:14] (CR) Awight: [C: 2] Merge remote-tracking branch 'origin/master' into HEAD [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163602 (owner: Awight) [17:00:32] ejegg: I think so! [17:00:37] good stuff [17:00:57] I'm writing a test for the recurring API, but that's split into its own commit... [17:01:22] ok, cool [17:12:16] (PS1) Awight: shuffling favicons [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163607 [17:12:31] (CR) Awight: [C: 2] "self-merging non-code" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163607 (owner: Awight) [17:13:08] (PS1) Awight: Merge remote-tracking branch 'origin/master' into HEAD [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163610 [17:13:17] (CR) Awight: [C: 2] Merge remote-tracking branch 'origin/master' into HEAD [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/163610 (owner: Awight) [17:28:29] Jeff_Green: didyu happen to see my question about mutating data during the prod->staging sync? [17:28:53] ...and, there was something we said we'd do this week... maybe finalize the /etc/ drupal settings thing? [17:31:41] awight mutating data: no, where was the comment? [17:31:45] (PS1) Awight: add PHPMailer [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/163615 [17:31:54] Jeff_Green: email from Saturday, no worries [17:32:26] odd. what was the Subject [17:33:32] Jeff_Green: I think I hid it from myself. oh, "srm'ing donor emails on staging" [17:33:43] Shall I resend? [17:34:06] looking [17:34:21] awight: i don't have mingle access [17:34:28] iirc last time we tried we were maxed out on accounts [17:39:49] Jeff_Green: I understand if you're allergic :p, but u could look in r/o mode for now? [17:39:53] ejegg: So... currency conversion. :) [17:40:23] awight: i can? looking... [17:40:39] ejegg: fyi I just deployed your drush command for exchange rates. [17:41:01] K4-713: Yeah, got a WIP on the backfill, just need to play with it locally a bit more [17:41:04] whoa, I can! who knew! [17:41:05] awight: thanks! [17:41:16] ejegg: Heh. Really? [17:41:41] Nice one. I just explained the situation upstairs, and they're interested in that getting done soon. [17:41:42] awight: actually, that's super-awesome since it should include the extra logging. [17:41:47] So... full speed ahead. :) [17:42:28] I was getting a 400 status doing the http request from drupal, but was able to curl it just fine from the command line [17:42:37] awight: i don' t fully understand. why transform all of the addresses in the db? [17:42:50] 400 makes sense for a bad api key, but drush get_variable showed it to match the one I used in curl. [17:42:51] Jeff_Green: I'm open to any other approach, but the goal is to never ever email real people [17:43:46] at the same time, we want to test email features, and have the letters go... to something like a shared account? [17:43:58] yep, that makes sense [17:44:16] awight: would be really cool if the mangled email addresses were still distinguishable a la verp [17:44:16] this mail would necessarily be leaving civi on lutetium? [17:45:29] ejegg: yeah! https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/1956 [17:46:17] awight two ideas: one would be to hack the mailer routine to recognize that it's in a dev environment, and if so adjust the envelope recipient [17:46:45] second would be to hijack *all* mail from lutetium that's originated by the web user, in the MTA [17:50:43] first idea is sketchy cos we use 2-4 different mailers [17:50:59] second sounds good, if you think we can make it 99.999% certain [17:52:34] ejegg: oh. I see there are definitely self-merges. [17:53:13] ejegg: at this end, we're discussing how this should be something MW-core devs deal with... [17:53:19] awight: it would be nice to filter out the submodule bumps with some gerrit-fu [17:53:25] yah [17:53:39] but gerrit query with a file regex tells me secondary index is not enabled [17:53:57] what is "secondary index"? [17:54:02] and I doubt they'll be enabling that during this sunset period [17:54:22] i'm not sure exactly, but it's the same error message via ssh command or web UI [17:54:40] when you try to filter on file:^extensions/.* [18:03:12] awight: ejegg K4-7131 there in one sec [18:12:35] ejegg: interesting, I hadn't considered going through the gerrit UI. I was imagining "git log" with a custom format, and lots of grep... [18:13:17] yeah, I just wanted to gerrit query since the list we have now is change numbers [18:14:18] you can query via script with an ssh command like this: [18:14:30] aha ok. And Change-Id is not as useful? [18:15:01] ssh -p 29418 ejegg@gerrit.wikimedia.org 'gerrit query --files --current-patch-set project:mediawiki/core status:merged limit:2' [18:15:12] that's awesome. [18:15:26] It's just not in the irc logs i was pulling from [18:16:22] there's probably a way to ask gerrit about self-merges directly, but I haven't figured that out yet [18:26:13] (PS1) Awight: Use Composer-install PHPMailer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163635 [18:26:30] (PS2) Awight: Use Composer-managed PHPMailer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163635 [18:26:46] (crap, it's the "no-shoe review" team") [18:31:37] (Abandoned) Awight: use stomp module from the future [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/115815 (owner: Awight) [18:37:01] (PS1) Awight: Use Composer-managed STOMP library [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163637 [18:42:28] ejegg: fyi, 1) wmf_communication\SilverpopImporterTest::testImport [18:42:31] Use of undefined constant NET_SSH2_DISCONNECT_BY_APPLICATION - assumed 'NET_SSH2_DISCONNECT_BY_APPLICATION' [18:42:47] (PS3) Awight: Use Composer-managed PHPMailer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163635 [18:42:49] (PS2) Awight: Use Composer-managed STOMP library [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163637 [18:45:51] awight: ooh, I'll see what's up with that [18:58:57] (PS2) Ejegg: Add exchange-rate-backfill drush command [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 [18:58:59] (PS1) Ejegg: Don't use http_build_query for OANDA retriever URL [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163642 [19:02:44] ejegg: what's the problem with http_build_query ? [19:03:46] it was escaping the ampersand between params on production, for some reason [19:04:19] that was the cause of my mystery 400 status [19:04:26] um [19:06:20] were you able to reproduce that on your local? [19:06:32] I just tested the function on the prod server and it's healthy [19:06:40] which makes me thing something else was doing the escaping... [19:06:42] no, locally it doesn't do it [19:06:49] very odd - string concatenation? [19:07:05] it's already a bit odd that we were using the function for two params, but not for "quote" [19:07:07] ok, let me try some stuff in interactive mode on prod [19:07:12] thx [19:08:11] yeah, there's no way to make it output multiple copies of the same param like that. feed it an array and it assumes you want indexes on each element [19:08:36] well, we could feed the imploded string into http_build_query? [19:08:55] but then it would escape all the ampersands [19:09:23] omg I see what you mean [19:09:39] Since we control all the inputs, i'm not too worried about using string concatenation [19:10:48] sure, I was only worried cos it seemed to be a magical no-op [19:11:04] the mystery escaping? [19:11:12] yeah [19:11:23] Also, is there a card for the backfilling? [19:11:25] i assumed it was some setting buried in an ini file [19:11:34] yeah, let me put that in the right column [19:11:43] and link the commit [19:11:44] I have simple questions I don't want to bother you with, like whether we're planning to reconvert contribution total_amounts [19:12:02] ahh, that will be another task, i guess [19:14:23] (CR) Awight: "Cool! I'm just blocking on the optional args..." (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [19:17:23] awight: if no start is given, it defaults to the last bank_update in the db. Guess I should make sure there actually is a bank_update [19:17:35] ejegg: oh! I missed that, sounds safe. [19:17:37] And maybe do that defaulting in the drush wrapper? [19:17:49] mmm good question. at least log about it? [19:17:56] ok [19:18:24] can't reproduce the extra escaping in interactive mode... [19:19:31] so very odd [19:21:03] But with the extra logging, we got definitive evidence of it happening. OANDA says 'unknown param (amp)' and the logged url has that first ampersand escaped [19:22:00] ejegg: wat. [19:22:17] so puzzling [19:27:41] I'm stumped, too. Fine with deploying the hack... [19:28:15] ejegg: are any of the other ampersands escaped? [19:30:22] nope! [19:30:52] gonna try a couple more things [19:31:21] I don't grep anything in /etc or crm/ setting the arg_separator.output character... [19:31:50] hmm [19:32:07] (which defaults to "&" of course) [19:32:23] drupal-specific? [19:33:01] ejegg: ooh, I lied and you might be right [19:33:22] I was looking in modules/, but check this out: grep -r arg_separator.output crm/civicrm [19:33:36] crm/civicrm/packages/HTTP/Request.php: ini_set('arg_separator.output', $separator); [19:33:50] ooh, ok... [19:34:16] * awight|lurch hops and swears like Q-bert [19:34:36] so,string concat it is! [19:37:50] awight I like the new civi logo :) [20:06:49] (CR) Ejegg: "replies inline" (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [20:11:30] (PS3) Ejegg: Add exchange-rate-backfill drush command [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 [20:12:37] ooo [20:15:43] SVentura: hehe, technically it's an old Civi logo, their new one doesn't look good as a favicon: http://civicrm.org [20:19:36] K4-713: Is it you that I come to screaming and crying about fundraising banner issues? [20:20:07] (CR) Awight: Add exchange-rate-backfill drush command (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [20:21:06] Deskana: Erm... sort of? What's up? [20:21:20] K4-713: Relatively minor things, but I thought I'd report them anyway. I'll come and chat. [20:21:35] Megan is over here, too. [20:21:38] (CR) Awight: [C: 2] "&Mysterious!;" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163642 (owner: Ejegg) [20:21:38] So... even better. :p [20:22:27] (PS4) Awight: Add exchange-rate-backfill drush command [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [20:22:32] (CR) Awight: [C: 2] Add exchange-rate-backfill drush command [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [20:22:40] (CR) Awight: [V: 2] Add exchange-rate-backfill drush command [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163311 (owner: Ejegg) [20:28:03] K4-713: we used to have workflow diagrams for GC... cannot find... [20:28:13] awight: We did? [20:28:22] yah! with status codes, in particular [20:28:28] Huh. [20:28:35] :'-/ [20:28:45] I was going to say: I don't remember doing that. [20:28:50] So, unless they came *from* gc... [20:28:55] What are you hoping to find? [20:29:03] I can no longer get into the Tech share... [20:29:07] really. [20:29:13] Symptoms? [20:29:24] The specific info I needed was the expected STATUSID from a recurring DO_PAYMENT [20:29:36] oho [20:29:39] it looks like it should be mapped as pending-poke, but the existing code checks for "pending" [20:29:52] hum. [20:29:53] I'm suddenly concerned we aren't hitting the SET_PAYMENT code path. [20:30:00] but also that I'm going crazy in multiple ways. [20:30:15] I get that too. [20:30:23] PPena: Do you remember seeing flow chart diagrams for GlobalCollect? [20:31:26] Alternatively, how do I get to their docs portal website? [20:31:36] awight: Thumb drive? I can get to the share just fine. [20:31:42] oh dear [20:31:54] I can get to Fundraising\Tech but not \Tech [20:32:09] That's... expected? [20:32:23] K4-713: Your browser question caught me off guard because I am lucky to not have to deal with browsers since I do apps. ;-) [20:34:14] Ha, no worries. [20:34:15] K4-713: yep totally in the PAyment Processing blackhole [20:36:51] PPena: ok, I need to look for docs on their site, can u help me with credentials? [20:57:22] ccogdill: when you have the time... Would you please look up a recurring GC charge (not an initial payment) and let me know what its status is? I'm scaring myself by thinking we've... never... been calling SET_PAYMENT to settle. [20:57:47] any recurring? [20:57:56] ccogdill: yah, want an ID? [20:58:01] sure, if you have one [20:58:05] np [20:59:14] ccogdill: orderid 9381205631 [20:59:39] thanks [20:59:58] no, TY! [21:00:19] so it started 12/29 and has recurred successfully every month since [21:00:28] it’s at status 800 right now but no reason to think it won’t settle [21:00:36] OH [21:00:42] what about the previous month? [21:00:58] yep, initiated 8/29 and settled 9/4 [21:00:59] Yeah I mean, it's "600" that I was worried about [21:01:04] ok thx [21:01:05] oohh no, it’s 800 [21:01:08] that’s expected [21:15:17] (PS7) Awight: new recurring charge API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/139476 [21:28:03] You know what really helps? When my laptop crashes all the time at strange intervals for no immediately obvious reason. [21:28:20] AWESOME. [21:28:21] I'm having that too. You update Bash yet? [21:28:33] Yes. [21:28:40] You? [21:28:54] need some more duct tape on my batteries :) [21:29:06] How about a penny? [21:29:22] Because, you know, things melting would be a serious improvement over crashing. [21:31:47] Gah. [21:31:50] What the heck was I doing. [21:32:16] (CR) Ejegg: "Looking great! Just a few questions inline." (4 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/139452 (owner: Awight) [21:34:26] msg nickserv [21:34:33] ...dur [21:50:30] (PS1) Ejegg: Send failmail on exchange rate update problems [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163742 [21:54:31] awight yes [21:54:35] awight still need them? [21:55:06] awight le tme ping you some links [21:55:28] PPena: awesome, TY [21:57:56] (CR) Awight: [C: 2] "Yesss..." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163742 (owner: Ejegg) [22:01:10] (CR) Ejegg: [C: 2] "I like it!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/163459 (owner: Awight) [22:01:56] (Merged) jenkins-bot: Discover test cases [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/163459 (owner: Awight) [22:03:57] So, should gross recalculation be a different drush job than backfilling the exchange rates db? [22:04:59] ejegg: Good question. Maybe it can be a single drush job, but the recalc will be a call into the wmf_civicrm module... [22:05:44] OK. I can see arguments either way. [22:06:04] i mean, they'll want to use the same start + end dates [22:06:32] recalc could be triggered by a boolean flag... [22:06:47] but if something goes wrong during recalculation, we don't want to redo all the rate requests [22:07:14] so maybe a separate drush job with an option on backfill to call the civicrm one on completions [22:07:26] That works fine by me! [22:07:29] cool [22:08:19] next trick is figuring out which ones to update [22:08:32] e.g. not cryptocoins [22:08:35] grargh! [22:08:45] anything else we get pre-converted? [22:08:48] Well, anything with original_currency != 'USD' [22:09:12] that *should* be populated for all contributions, cos it's done in a db trigger. [22:09:21] or hrm hook. [22:11:27] (CR) Awight: Merge standalone framework (4 comments) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/139452 (owner: Awight) [22:12:10] (PS22) Awight: Merge standalone framework [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/139452 [22:14:38] Jeff_Green: where did we leave the staging email question... [22:15:07] "upstairs" is waiting on this to try a new feature (spamming people from CiviEvent)... [22:20:16] K4-713: fyi, https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0AsYxO0Je1DGRdFNUZ1NyODVfNWZOV0JaSTBoTHhMbEE&usp=drive_web#gid=10 [22:20:39] hurm? [22:20:52] Woah. [22:20:53] seems to be where the GC payment flow charts live [22:20:55] Did you do that? [22:20:55] yeah [22:21:03] No, I think PPena did! [22:21:14] I have definitely never seen this. [22:21:18] Awesome. [22:21:29] hecklyeah [22:23:04] * ejegg requests permission to seee the doc [22:23:32] ejegg: done [22:23:35] ty! [22:26:12] awight: we'll have to follow up tomorrow, but my vote is for a code or config fix rather than munging the data in the database [22:26:39] imo it's safer if we can control all the mail flow [22:27:01] gotta run though, dinner time [22:32:37] Jeff_Green: yep, I agree in principle [22:32:43] k see ya! [22:34:47] ccogdill: is this a good time for me to send the tardy TYs in https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/1869 ? [22:35:02] as good as any [22:35:06] but can you give me like 5? [22:35:14] I want to add a tag in Zendesk for this [22:35:21] yeah it'll take me at least that long to set up, myself [22:35:24] perfect [22:35:27] thanks for the heads up! [22:37:35] ccogdill: oh dear, it's been too long. Do you remember if we decided to not bother with extra "we screwed up, sorry it's late" text? [22:38:06] ccogdill: ok nvm I think it's the "UnrecordedCharge" block... [22:38:54] umm [22:39:01] did you confirm? [22:39:12] I think it’s all this right? [22:39:14] We recently resolved a small technical issue which caused some monthly recurring donations to receive no tax receipt. This email is a confirmation of a donation made earlier in 2014, and is not indicative of a new donation. All thank you letters will send normally going forward. Thank you for your patience and your support, and please feel free to email donate@wikimedia.org if you have any questions. [22:39:27] I want to make sure I have the right text because that’s how Zendesk will know to tag it [22:39:28] awight [22:40:42] ccogdill: This is what I found: {% if "UnrecordedCharge" in contribution_tags %} We recently resolved a technical issue which caused a small number of donors to not receive a confirmation of their donation. Please accept this email as a thank you for your donation on [date]. We truly appreciate your patience and your support, and please feel free to email donate@wikimedia.org if you have any questions. {% endif %} [22:41:33] ahh that is right [22:41:38] yeesh, thanks [22:43:19] ccogdill: buckled? :) [22:43:26] locked and loaded! [22:43:31] okdoke [22:43:44] thanks again :) [22:45:17] ccogdill: 68 donations were affected. The letters should go out over the next few minutes. [22:45:29] okay great. 68 is no sweat [22:46:19] ccogdill: awesome, glad to have that off our plate. It'll be much easier next time, one can hope! [22:46:41] yeah for sure! all the back and forth was worth it now that we have something to fall back on later [22:48:01] Yeah, in theory all we have to do is tag a contribution with "UnrecordedCharge" and it will go out with the extra text... [22:48:18] thanks for ironing out the details! [22:49:51] nice, it’s cool to know it’s that simple [22:50:09] so if Rosie or I found a contrib with a no TY date later, should we tag it as such? [22:50:53] yep! There's still the grace period issue, we're planning to do something about that in the future. [22:54:14] right… [22:54:23] we’ll cross that bridge when we get to it, I suppose [22:54:32] which... means yeah it doesn't work like I said ;) [22:54:44] ok, confirmed that the letters went out. [23:06:48] wahoo! [23:21:01] (PS1) Awight: DO NOT DEPLOY - Use the merged DonationInterface library [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/163764