[02:45:58] (CR) Jforrester: [C: 1] "Good to go once the npm job is enabled." [extensions/ContributionTracking] - https://gerrit.wikimedia.org/r/205518 (owner: Legoktm) [10:35:16] Fundraising Sprint Indigo Girls, Fundraising Sprint James Brown, Fundraising Sprint Kraftwerk, Fundraising Tech Backlog, and 2 others: Write adapter controller logic to process front-end payments - https://phabricator.wikimedia.org/T90504#1333287 (atgo) @ejegg is this done? Can we move to done/close? [10:35:40] Fundraising Sprint House of Pain, Fundraising Sprint Indigo Girls, Fundraising Tech Backlog, MediaWiki-extensions-DonationInterface: Make the forms come from one template - https://phabricator.wikimedia.org/T86248#1333290 (atgo) [10:35:43] Fundraising Sprint Indigo Girls, Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, Fundraising Tech Backlog, and 2 others: Make form for Astropay Brazilian credit cards - https://phabricator.wikimedia.org/T97056#1333288 (atgo) Open>Resolved [10:36:02] Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, Fundraising Tech Backlog, Astropay Integration, Patch-For-Review: Add Astropay forms to DonationInterfaceFormSettings.php - https://phabricator.wikimedia.org/T90505#1333292 (atgo) Open>Resolved [10:36:05] Fundraising-Backlog, Wikimedia-Fundraising, Epic: Integrate processing in LATAM via Astropay - https://phabricator.wikimedia.org/T77907#1333293 (atgo) [10:36:24] Fundraising-Backlog, Wikimedia-Fundraising, Epic: Integrate processing in LATAM via Astropay - https://phabricator.wikimedia.org/T77907#831530 (atgo) [10:36:25] Fundraising Sprint Kraftwerk, Fundraising Tech Backlog, Wikimedia-Fundraising-CiviCRM, Astropay Integration, Patch-For-Review: Add gateway name to queue consumer switch, and custom reports - https://phabricator.wikimedia.org/T90508#1333294 (atgo) Open>Resolved [14:27:20] (PS2) Krinkle: resourceloader: Remove redundant getModifiedTime implementation [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/215382 (https://phabricator.wikimedia.org/T94074) [14:31:11] (PS3) Krinkle: Remove redundant ResourceLoaderModule::getModifiedTime implementation [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/215382 (https://phabricator.wikimedia.org/T94074) [15:00:02] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Few/no banners shown on 1 June? - https://phabricator.wikimedia.org/T101265#1333901 (Pcoombe) NEW [15:22:01] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Few/no banners shown on 1 June? - https://phabricator.wikimedia.org/T101265#1334033 (AndyRussG) Thanks! That's very strange! Just to confirm, we didn't do any CN deploys or config changes that would correspond to those variations. Maybe some proble... [15:22:55] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Few/no banners shown on 1 June? - https://phabricator.wikimedia.org/T101265#1334040 (AndyRussG) [16:38:37] Fundraising Sprint Indigo Girls, Fundraising Sprint James Brown, Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, and 3 others: Write adapter controller logic to process front-end payments - https://phabricator.wikimedia.org/T90504#1334292 (Ejegg) Open>Resolved [16:38:38] Fundraising-Backlog, Wikimedia-Fundraising, Epic: Integrate processing in LATAM via Astropay - https://phabricator.wikimedia.org/T77907#1334294 (Ejegg) [17:21:25] K4-713: what is civicrm_contribution.payment_instrument_id? doesn't appear to be a FK [17:21:46] oh, ha. [17:21:49] K4-713, AndyRussG, XenoRyet, cwdent: anything you want from other teams? [17:21:57] It is. But it's a PITA. [17:21:58] I'm good [17:22:12] don't think so ejegg [17:22:20] ejegg: not for now, thanks tho! :) [17:22:24] oh, is that one of those columns that FKs to civicrm_option_value? [17:22:54] K4-713: great comments there in the meating [17:23:08] Oh, thanks. I always wonder if I make sense at all. [17:23:19] Totally, many times over :) [17:23:21] i totally spaced that :( [17:23:25] how did it go? [17:23:32] Difficult to say. [17:23:36] teach me to filter group [17:23:39] I'll let you know if anything comes of that. [17:23:50] cwdent: it was iiiiiiiiinteresting [17:23:54] heh [17:24:11] Also, I couldn't see what anybody was saying on IRC. [17:25:47] cwdent: This should be somewhat enlightening: https://collab.wikimedia.org/wiki/Fundraising/Engineering/Fun_SQL_Queries#List_of_All_Payment_Instruments_By_ID [17:25:52] cwdent: I pm'd you the link to the youtube recording... P.S. don't get discouraged by stuff there, the truth is the WMF is a bubbling cauldon of potential for effecting positive change on so many levels! [17:26:43] There was a youtube? [17:26:59] I'm glad I don't pay attention to those details. [17:27:29] I say more when I don't think it's being recorded for posterity. [17:30:30] AndyRussG: oh yeah i am not discouraged, seems like normal organizational difficulties. i am still in the phase of "i can't believe people complain about working here" :) [17:30:51] K4-713: eeeverything is being recorded for posterity ;p [17:31:11] cwdent: The thing about here, is... [17:31:13] ...hmn. [17:31:16] How to explain. [17:31:43] Most other jobs, at some point, you can disconnect and just go "Well, it's a paycheck that I will think of as aggravation tax". [17:31:50] And nod and smile and not really give a damn. [17:31:56] Here... everybody gives a damn. [17:32:37] cwdent: heh :) well I think it's not complaining about working here, more like, as K4-713 said in the meating, that there seems to be a shift due to the newness and greeness of higher-ups, and are positivley wanting to help preserve what's special about the place [17:32:40] yep, it's a great thing, i'm just too green to know all the back story [17:32:45] Because, the second you decide you don't give a damn, there are like 1,000 places that pay better for less responsibilities that you don't give a damn about. [17:32:56] So, it's sort of self-selecting for people who take everything totally personally. [17:33:00] That's it. [17:33:43] yeah K4-713 that was a common sentiment at sparkfun too, and the minute i stopped giving a damn i left [17:33:59] Makes total sense. :) [17:34:02] K4-713: also self-selecting for people who don't want to become cold, dead, soulless shells barely reminiscent of what they might have been and done [17:34:10] Yes! [17:34:14] hehe, well said [17:34:29] The Hives have a song about that. [17:34:50] :P the caveat being of course, yes, there are other places that also self-select for that sometimes at least! [17:35:18] i think the "business model" of WMF (if you can call it that) makes for some really unique challenges [17:35:27] BAHAHA [17:35:32] * K4-713 coughs [17:37:01] Oh my. That was exciting. [17:37:12] awight: Want to look at that failmail avalanche? [17:37:42] oh good, civicrm_option_value is prepopulated so this shouldn't be so bad [17:38:03] Ongoing avalanche... [17:38:18] halp, i's blowing up [17:38:27] much explode [17:38:35] cwdent: yeah especially if you don't call it that :) [17:38:51] cwdent: my filters did not spambox that, for a change :) [17:39:21] yep made it to my inbox as well [17:39:43] civicrm_option_value not considered harmful? hardly [17:40:13] You guys *are* getting all that failmail, right? [17:40:24] am i reading this right as a yo dawg i hear you like datastores situation? [17:40:40] yep K4-713 i'm seeing them [17:40:41] K4-713: They started sending us files without the transaction ID, or something. Checking that now... [17:41:15] cwdent: congratulations, it's your custom schema... crammed all up into one table. [17:41:34] RDF graph nerds [17:41:47] btw, this glossy laptop screen is laughable [17:42:28] http://cdn.meme.am/instances/500x/58822751.jpg [17:42:42] new laptop awight ? [17:42:55] awight: Is it weird to you that the paypal audit job didn't decide to fail, there? [17:43:41] I mean, if you're going to explode over 100 times, it should probably fail the job. [17:43:49] ...seems to me. :p [17:44:08] Not that we have ever done anything like that before. I'm just saying it seems like a reasonable thing to do. [17:44:20] Probably percentage based, though. [17:44:36] K4-713: yeah that needs to happen soon. [17:44:37] I [17:44:39] Like, if you chug through 10 and 10 fail, give the hell up. [17:44:42] I'd say three is the magic number. [17:44:45] but yeah [17:45:03] Or, if you get to the end and nothing worked, mark as disaster. [17:45:04] I'd be surprised if there wasn't a task written already [17:45:20] Both of those things. [17:46:16] Not that I don't enjoy getting hundreds of emails in four minutes... hehe [17:46:27] This must be what popular people feel like. [17:52:19] Fundraising Tech Backlog, Fundraising-Backlog: Fundraising audit job should fail hard at some magic number of records - https://phabricator.wikimedia.org/T101287#1334618 (awight) NEW [18:01:31] ooooh _that_ fanmail... [18:01:43] Man the VE table editor rocks! [18:52:19] Hi friends! [18:53:21] hiya [18:54:37] Got irc on phone! [18:55:07] I just checked in with ejegg on gchat, but Let me know if there's anything I can do about this PayPal thing [19:18:10] Is it possible to... Turn off the fail mails that are still coming? [19:18:42] atgo: Sorry, I think you have to make a filter and leave it on for a few hours. [19:18:55] I'm going to write Kevin today, once I gather enough details about what happened. [19:19:15] One of my questions for him is, will this break again tomorrow or is it being resolved. [19:19:22] Cool. It's fine, I just need to keep alerts off on my phone because it killed my battery :P [19:19:26] omg [19:19:35] if you make a filter, it should not alert your phone. [19:19:50] that is hilarious though [19:20:06] Was just talking to a friend who said his company does logarithmic alerts. So it tells you about the first like 5, then the 10th, then the 100th, etc [19:20:07] plus, you're probably paying wanderfees [19:20:14] Haha yeah. I wonder if I can do that from mobile [19:20:15] yes [19:20:18] we want that [19:20:27] Free data and texting internationally! [19:20:35] The only thing awesome about T-Mobile! [19:20:53] Fundraising Tech Backlog, Fundraising-Backlog: Fundraising audit job should fail hard at some magic number of records - https://phabricator.wikimedia.org/T101287#1335133 (awight) 12:20 < atgo> Was just talking to a friend who said his company does logarithmic alerts. So it tells you about the first like... [19:21:26] Yay! I was going to add that once I got back to a computer :) [19:22:48] :p [19:35:32] awight: ejegg: cwdent: XenoRyet: https://www.mediawiki.org/wiki/Extension:CentralNotice/Campaign_and_banner_selection [19:37:58] looking good AndyRussG ! [19:40:16] ejegg: thx! Pls go ahead and fix any gaffes or goofs :) [19:40:38] Just was noticing that for other teams I've been having to explain this more than once, so I might as well put it up once and for good [19:59:01] (PS4) Ejegg: Finish removing hardcoded nobody@wikimedia.org [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215341 (https://phabricator.wikimedia.org/T101114) [19:59:03] (PS1) Ejegg: Stage email first, use staged for dependent values [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215710 (https://phabricator.wikimedia.org/T101114) [19:59:34] awight: I think I need https://gerrit.wikimedia.org/r/215710 to deploy the other default email thing you +2ed yesterday [19:59:59] ok, looking! [20:02:25] ejegg: Hey, is it a problem that one person can donor_id spoof another just by giving the same email? [20:03:00] maybe that should have a session salt in it? [20:05:12] hmm, yeah [20:07:06] Actually, not sure if that field matters at all - it's akin to the merchant reference 2 for worldpay [20:07:49] might be nice to find transactions by email address, even if you have to sha1 it first [20:07:57] (CR) Awight: Stage email first, use staged for dependent values (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215710 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [20:08:21] err that would be a sad way to search. AP doesn't support email search? [20:09:08] oh right, we do give that to 'em already [20:09:38] well I'm fine merging this, unless you want to tweak? [20:09:41] so yeah, no harm making that unique donor id unique per session [20:09:49] cool, I'll wait then [20:10:43] hmm, is DonationData's normalization applying other defaults? [20:10:58] it... should be. [20:11:08] I hate that we ever use "unstaged" data [20:11:50] IMO data should go, raw -> normalized -> staged for an API call, and response -> unstaged for return values [20:12:08] I think I fixed that in a few places... really confusing... [20:12:25] yeah [20:13:58] omfg, there will be 3k failmails by the time the mail queue empties [20:14:16] 3k bottles of failmail on the wall [20:14:21] hah, gmail is so confused. keep deleting those pp failmails and they keep popping back up in my inbox [20:14:39] (PS1) Cdentinger: make a/b widget return some fake data [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/215758 [20:15:09] awight: ejegg: atgo: cwdent: XenoRyet: Didja see this 'un? https://phabricator.wikimedia.org/T101265 I'm wondering if we should pop it into the sprint [20:15:22] atgo: ^ [20:15:26] ejegg: I think it's cos they're still coming in. Make a filter that catches the new ones... just remember to disable later today [20:16:15] Nah, it's the old ones just sporadically zombie-ing up. If I search bad_audit_line in:inbox, they don't show up in the results [20:16:20] AndyRussG: wheee, and this isn't kafkatee yet? [20:16:49] awight: nope. Actually might be lack of impressions, since donations went down too, no? [20:17:28] I was gonna ask ops if something burped down in the server dungeon [20:29:12] AndyRussG: I can't really tell if anything significant happened with the campaigns, it looks like the "enlvrosk" campaign continued pretty much as usual. [20:29:31] awight: hmm yeah I haven't even peeked yet 8p [20:30:01] I think that was the-wub's impression too, tho [20:33:09] (PS2) Ejegg: Stage email first, use staged for dependent values [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215710 (https://phabricator.wikimedia.org/T101114) [20:33:36] awight: using order ID for session uniqueness [20:34:07] ok, that is provided by AP eh? nice one. [20:35:15] nope, that's ours [20:35:31] ctid, I think? [20:36:55] k then [20:37:35] (CR) Awight: [C: 2] Stage email first, use staged for dependent values [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215710 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [20:38:00] thanks! [20:38:07] (Merged) jenkins-bot: Stage email first, use staged for dependent values [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215710 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [20:38:26] awight, did you see I make a couple comments over at github? [20:38:39] yes, thanks! [20:38:47] cool [20:50:42] ejegg: argh, I guess redis shards transparently in cluster mode, so transactions actually need to be on a single box or they would require two-stage commit or something... [20:52:15] oh, is that the issue? dang [20:54:31] Also, I don't think Predis even lets you take that chance. [20:55:12] I might be able to get rid of those transactions... [20:55:34] We already need to handle expired k/v entries that still have a zset entry. [20:58:09] awight: ah also I made a few shallowgithub comments [20:58:25] github made me make them so [20:58:42] Thanks for the help! [20:59:03] Heh likewise (many times over :) 9 [20:59:04] ) [20:59:07] I'll respond to those in a few hours... [20:59:11] awight: i figure if you hit an expired one when you pop, just keep popping till you get a good one [20:59:14] aww, my CR is pretty crummy [20:59:31] ejegg: yeah, that would be better than returning "null" [21:11:10] ejegg: I'm not so certain about the redis txn fail, now. Maybe it's not the sharding, but just a client thing--seems that we could make a single connection to a cluster through a load balancer, if sharding is not transparent. [21:11:22] erp English redundancy [21:12:23] yeah, i didn't see anything in the server docs that said txns weren't available in cluster [21:13:16] hrm. I think I should understand how we're clustering. [21:13:42] http://redis.io/topics/partitioning -> Redis Cluster says it does automatic sharding [21:14:19] awight: heyo. What happened with the paypal explode? [21:14:33] ejegg: I'll workaround for now by making a single connection... [21:15:06] K4-713: Paypal changed their audit file format for us. Looks like a mistake, I opened a ticket a minute ago [21:15:12] Oh, great. [21:15:20] I think I finally stopped getting emails over it. [21:15:26] subject = "Severe audit file problems" if you're reading from home [21:15:31] heh [21:15:36] 3k failmails, I think. [21:15:38] wheee! [21:15:47] plus... PII [21:16:19] In the old days, they would tie me down and give me chicken pox for something like this... [21:17:39] The only reason I haven't done that yet is that you're wfh. [21:17:48] :p not an accident [21:17:52] OIC [21:18:01] "work from in hiding" [21:18:04] * K4-713 rubs pox on your desk [21:18:31] But I'm manifesting the hex! [21:19:36] Jeff_Green: you might be interested in this discussion, ejegg and I were talking about paymentswiki Redis and whether it should be clustered. [21:20:12] I don't understand the alternatives we have available, but so far I'm not convinced it's critical enough to cluster. [21:20:38] On the other hand, potential Redis for contribution tracking would have to be not a SPOF to make it worth the trouble to migrate. [21:20:50] but that would be outside of frack. [21:24:45] awight: can you give me a bit of guidance on the listener queues? [21:25:11] anything i know is yours! [21:25:15] how can I help? [21:25:34] you probably want to include some fixture data so you can sort of TDD [21:25:55] If I just dump something into the 'verified' queue, that goes where? [21:26:18] that is the "donation" queue, sorry it has a different alias in every fucking subsystem [21:26:28] that's the main civi queue [21:26:32] ahh, ok. [21:27:21] AFAIR, "pending" if you encounter it is just a holding area while any asynchronous listener processing is happening. [21:27:36] Hmm, so we need to populate that message with all of the data. And if the processor's POST to the listener only has a few things, we need to grab the correlated info from limbo / pending [21:27:54] yes, from limbo [21:29:20] Hmm. [21:30:07] So, I'm currently sending the "completed" message at the result switcher if the signature validates and the result is 'paid' [21:30:31] I think I'm also deleting the limbo message when the come in to the result switcher, whatever the status. [21:30:59] For async donation methods, there will be a 'pending' queue message [21:31:18] that's probably not right, you should be sending the message to limbo at the end of the workflow, cos it isn't confirmed yet. [21:31:37] and astropay will post to the listener a while later, when it's either confirmed or expired [21:31:38] I think we have the same situation with both Paypal and Amazon, not sure if we handle correctly... [21:31:50] checking... [21:32:51] Remind me, isn't the final step in the AP workflow necessary to finalize the payment, or is that only for our own information about payment status? [21:33:15] just for our own info, I believe [21:33:42] ah then... it's right to send to the donations queue, sorry [21:34:27] Yeah... I'm not actually even making a second status call right now - just validating the signature on the status posted to the result switcher [21:35:57] The way our listeners currently work, we would do the verification calls, check the CRM database for the transaction, then add the message to the queue iff it does not exist (without extra limbo data). [21:36:21] That kind of sucks, in the future we should send the message no matter what, and the CRM will check a box somewhere that says "audited by listener" [21:36:29] same with the nightly reconciliation files. [21:36:41] then we can search for unverified messages in a new job... [21:47:30] sorry, not seeing where in smashpig we check CRM for the transaction. Is that just in the old listeners? [21:47:51] awight^^ [21:52:27] ejegg: oof, it's possible we're just dumping it all and relying on the consumer to discard duplicates. [21:53:21] That would make sense - pour stuff into the queue as fast as possible, then take as much time as you need to process on the other side [21:53:30] That would make sense cos we wanted to cut the wart off and isolate a bit from the CRM [21:53:41] that too [21:53:56] yeah, I prefer it that way cos it's a step towards treating the initial and listener event differently [21:54:51] OK, so basically all that the astropay listener needs is tests to get it towards initial deployment [22:01:07] hangouts plugin issues... [22:03:22] ejegg: the tests are for your own pleasure, really. No CI there. [22:05:11] atgo: Alive? [22:07:10] Oui! But I have a terrible headache and got sucked into something. Can you guys fend without me? Sorry :( [22:10:36] I'm back for reals next week. [22:13:28] K4-713: % [22:13:34] ^^ [22:13:46] kk [22:16:53] (CR) Awight: [C: 2] bannerController.lib: KV store with localStorage (1 comment) [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/210408 (https://phabricator.wikimedia.org/T94763) (owner: AndyRussG) [22:19:19] https://phabricator.wikimedia.org/T101265 [22:19:26] awight: thanks! [22:19:34] (PS5) Awight: Finish removing hardcoded nobody@wikimedia.org [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215341 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [22:23:41] (CR) Awight: [C: 2] Finish removing hardcoded nobody@wikimedia.org [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215341 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [22:25:09] (Merged) jenkins-bot: Finish removing hardcoded nobody@wikimedia.org [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215341 (https://phabricator.wikimedia.org/T101114) (owner: Ejegg) [22:27:15] AndyRussG: https://s-media-cache-ak0.pinimg.com/236x/f8/8b/9f/f88b9f3a9f1b3642346a2ae71e3bd2ac.jpg [22:27:29] Helping? [22:27:42] K4-713: wooo! [22:28:57] https://media3.giphy.com/media/ktW8FqKodoPUk/200_s.gif [22:29:32] http://i.imgur.com/xOV9mFb.jpg [22:35:44] yay, it's that hour! [22:49:42] well! [22:49:45] But it's not even Friday [22:51:08] Oh no, we're out of veggies! [23:17:47] hey team - sorry for being unavailable earlier. is there anything i can do to be helpful? [23:22:30] (PS4) Ejegg: Initial validation for mustache forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/214286 (https://phabricator.wikimedia.org/T100562) [23:22:32] (PS1) Ejegg: Make mustache form card selection act like GC [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/215813 [23:23:06] (CR) Ejegg: "PS4: rebase" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/214286 (https://phabricator.wikimedia.org/T100562) (owner: Ejegg) [23:26:43] (CR) Ejegg: "I like the way you break out of the loop! This code looks good, will merge after the phpqueue redis backend tweaks." (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/211062 (https://phabricator.wikimedia.org/T99017) (owner: Awight) [23:28:58] Hi atgo: Just a tidbit that you may want to be aware of, T101265 is heading to LouReedLandia... In 3... 2... [23:29:00] 1... [23:29:34] yeah that seems like a good idea :) [23:32:15] (CR) Ejegg: Orphan slayer reads from Memcache (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/211062 (https://phabricator.wikimedia.org/T99017) (owner: Awight)