[00:26:47] Fundraising Tech Backlog, Wikimedia-Fundraising-CiviCRM: Take timings to help with Civi optimization - https://phabricator.wikimedia.org/T109026#1538119 (awight) NEW [00:29:00] K4-713: ^ [00:29:09] apropos that other thing [00:30:39] heh [00:31:24] Fundraising Tech Backlog, Wikimedia-Fundraising-CiviCRM: Take timings to help with Civi optimization - https://phabricator.wikimedia.org/T109026#1538135 (awight) [01:03:31] Fundraising Sprint Queen, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work: Something broken in payments-antifraud and payments-initial message generation - https://phabricator.wikimedia.org/T109022#1538241 (awight) a:awight [01:04:10] (PS1) Awight: Restore 'freeform' flag [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/231474 (https://phabricator.wikimedia.org/T109022) [01:07:54] (PS2) Awight: Restore 'freeform' flag [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/231474 (https://phabricator.wikimedia.org/T109022) [01:11:27] Fundraising Sprint Queen, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work, Patch-For-Review: Something broken in payments-antifraud and payments-initial message generation - https://phabricator.wikimedia.org/T109022#1538248 (awight) Needs tests to prevent regression. [01:50:49] (PS7) AndyRussG: WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 [01:51:47] (CR) jenkins-bot: [V: -1] WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 (owner: AndyRussG) [01:52:46] (PS8) AndyRussG: WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 [03:30:21] (PS9) AndyRussG: WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 [10:25:39] (CR) Pcoombe: [C: 2] "Looks good. Thanks Adam!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/231432 (https://phabricator.wikimedia.org/T108703) (owner: Awight) [12:33:03] Wikimedia-Fundraising: Untranslated GlobalCollect iframes - https://phabricator.wikimedia.org/T109064#1539372 (Pcoombe) NEW [14:05:32] (PS10) AndyRussG: WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 [14:53:34] TCB-Team-Fundraising-Sprint-2015-08-12, TCB-Team: [WMDE-Fundraising] Create template switching mechanism for confirmation page - https://phabricator.wikimedia.org/T108812#1539711 (Tobi_WMDE_SW) a:WMDE-leszek [14:53:46] TCB-Team-Fundraising-Sprint-2015-08-12, TCB-Team: [WMDE-Fundraising] Move Piwik to its own VM - https://phabricator.wikimedia.org/T105289#1539712 (Tobi_WMDE_SW) a:WMDE-Fisch [16:31:28] Good morning. I'm using a relatively free morning to attempt more wiki edits. I'll be on email. [17:05:44] (PS1) Ejegg: Move Amazon Widget script and return URL into account [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/231585 (https://phabricator.wikimedia.org/T108112) [17:06:26] (CR) jenkins-bot: [V: -1] Move Amazon Widget script and return URL into account [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/231585 (https://phabricator.wikimedia.org/T108112) (owner: Ejegg) [17:06:35] (CR) Ejegg: Switch Amazon to Mustache form, add modules (1 comment) [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/230724 (https://phabricator.wikimedia.org/T108114) (owner: Ejegg) [17:09:13] (PS2) Ejegg: Move Amazon Widget script and return URL into account [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/231585 (https://phabricator.wikimedia.org/T108112) [17:35:24] Fundraising-Backlog, Astropay Integration: [epic] Processing via Astropay for Spanish-speaking LATAM countries - https://phabricator.wikimedia.org/T102143#1540389 (atgo) [17:52:37] (PS1) Ejegg: Clean up a bit of Amazon javascript [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/231595 (https://phabricator.wikimedia.org/T108112) [17:57:04] heading to boulder, back in a bit [17:57:41] (PS1) Ejegg: Use modern hook registration for Amazon [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/231596 (https://phabricator.wikimedia.org/T108112) [18:03:17] bad audit emails? [18:03:26] Should we be worried? [18:06:01] stupid paypal [18:06:06] dstrine: audit is not a deal-breaker [18:06:37] ok [18:06:39] we can always re-import the files [18:06:53] and it doesn't effect any donors upfront, just when they get their thank yous [18:07:58] Wikimedia-Fundraising: Untranslated GlobalCollect iframes - https://phabricator.wikimedia.org/T109064#1540567 (atgo) p:Triage>Low [18:08:33] Wikimedia-Fundraising: Untranslated GlobalCollect iframes - https://phabricator.wikimedia.org/T109064#1539372 (atgo) Triaging as low since we're a bit bound right now on what we can do. I think WorldPay is probably the right option, but we won't be able to get there in the near term. [18:08:52] fundraising-tech-ops, Traffic, operations, Patch-For-Review: Decide what to do with *.donate.wikimedia.org subdomain + TLS - https://phabricator.wikimedia.org/T102827#1540574 (CCogdill_WMF) After talking about this more with our email consultants, I don't think we have any other option but to chang... [18:10:28] ugh paypal. [18:23:11] (CR) Ejegg: Redirect to Amazon for login (4 comments) [extensions/DonationInterface] (payWithAmazon) - https://gerrit.wikimedia.org/r/230707 (https://phabricator.wikimedia.org/T108112) (owner: Ejegg) [18:33:28] dear Paypal: http://www.bumppy.com/wp-content/uploads/2015/02/AWAY-FROM-PARENTS-NAGGING.gif [18:33:51] P.S. http://bookriot.com/wp-content/uploads/2013/09/lalalalala.gif [19:21:02] K4-713: you went for food, yeah? [19:21:30] because i'm going to jamba [19:21:35] now-ish [19:21:40] I did. [19:21:42] ...:( [19:21:43] boo. [19:21:49] But! Exciting thing! [19:21:58] ? [19:22:03] Exciting thing that requires a printer! [19:22:07] ooh yay! [19:22:19] btw did someone turn off the paypal job? or should i expect continued spam all day? [19:29:13] atgo-eat: are you still getting paypal spam? mine seems to be relegated to 11:36 daily [19:29:30] * dstrine-lunch shrugs [19:29:43] summary of paypal this week: http://p.fod4.com/p/media/a4e1463ce9/YOYVX13IRp6Zhk4cTWOJ_Baseball%20Foul%20Hit.gif [19:30:05] nice [19:30:57] The'll get to the minors some day [19:34:19] bahaha... yes. [19:36:45] (PS6) AndyRussG: WIP Banner history logger campaign mixin [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/229560 (https://phabricator.wikimedia.org/T90918) [19:38:30] (PS7) AndyRussG: WIP Banner history logger campaign mixin [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/229560 (https://phabricator.wikimedia.org/T90918) [19:40:06] (CR) AndyRussG: "Small-ized EL call now part of this patch (squashed from Iceb916ee)." [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/229560 (https://phabricator.wikimedia.org/T90918) (owner: AndyRussG) [19:40:49] (Abandoned) AndyRussG: WIP Banner history logger: minify and cutoff for EL data [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/231060 (owner: AndyRussG) [19:45:20] ejegg: having mad autoloader problems with phpunit...crm tests complain about BaseChecksFileTest...DI tests complain about DonationInterfaceTestCase [19:45:29] i remember sorting this out before but can't remember wtf i did [19:45:37] does it ring any bells with you? [19:46:18] hmm, sounds like bootstrap file not loading [19:46:51] pointing to a suite.xml? and does that have a bootstrap php file listed? [19:47:42] hrm, just trying from the base DI dir, don't see a suite.xml anywhere [19:48:27] but just saying "phpunit tests/" does seem to find the tests [19:48:37] PHP Fatal error: Class 'DonationInterfaceTestCase' not found in /srv/DonationInterface/tests/Adapter/Adyen/AdyenTest.php on line 24 [19:48:56] cwdent ah, i always have to run from the mediawiki base tests/phpunit dir [19:49:05] and i use that dumb makefile [19:49:17] oh that's right, the makefile [19:49:33] i remember i had all the crm ones working [19:49:56] yeah, crm ought to just work [19:49:56] been trying to get DI working locally is how i got here [19:50:09] modulo some strict warnings [19:55:51] i really need to keep better notes [20:29:07] Fundraising Sprint ODB, Fundraising Tech Backlog, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work, and 2 others: DonationInterface spam in LocalisationUpdate logs - https://phabricator.wikimedia.org/T105850#1541126 (DStrine) [20:29:12] Fundraising Sprint ODB, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 3 others: GlobalCollect is not recurring: something wrong with "batch mode" in DonatioinInterface - https://phabricator.wikimedia.org/T105848#1541127 (DStrine) [20:30:58] Fundraising Sprint ODB, Fundraising Tech Backlog, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work, and 2 others: DonationInterface spam in LocalisationUpdate logs - https://phabricator.wikimedia.org/T105850#1541139 (Reedy) >>! In T105850#1465825, @Ejegg wrote: > Yep, the change is m... [21:13:07] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/231692 [21:13:24] just deploying that pt-br thank you letter [21:13:50] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/231692 (owner: Ejegg) [21:15:48] !log updated crm from 4f40ac6de0385982d8e672b1ed30ff1a2a2a2aa1 to fc0fcc8f5af262b56392d3f4f5998f8ea08c99a8 [21:15:52] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:19:51] cwdent: ejegg: LMK if u have a sec to help me check my thinking on something? Thx, sorry to interrupt if you're busy... [21:19:56] Also not urgent... [21:20:20] AndyRussG: sure, what's up? [21:21:07] cwdent: thx! So I'm thinking there's no way to even flag banner histories as belonging to donors or not without going through the same mechanism as we'd use to actually correlate 1:1 histories and donations [21:21:46] simply because the sample of banner history that includes everyone will happen shortly after page display [21:21:56] i was wondering about that too [21:22:08] and there's no way to know if the person will click on donate before leaving the page [21:22:09] AndyRussG: ooh [21:22:23] and the token has to follow them to payments wiki somehow [21:22:26] in order to correlate [21:22:28] Yeah [21:22:33] so, you send a random number with each history, click or no click [21:22:34] Even just to get a donate/didn't donate flag [21:22:41] ejegg: exactly [21:22:56] and that random number travels down the pipeline... [21:23:14] then if they /do/ click, you add that number to the payments link, right? [21:23:24] yea exactly [21:23:49] also I don't think "clicked on donate" is as useful a data point as "actually donated" [21:24:05] * K4-713 gets a funny look on face [21:24:06] and for the latter, the id has to travel all the way down, I think? [21:24:26] right, that's why the number has to follow them to the thank you page or thereabouts [21:24:44] This sounds remarkably like some things that the Contribution Tracking table was made to do. [21:24:53] that's actually something i wondered about -- do we have metrics on bail rate past donate click? [21:25:18] K4-713: which table? indoor or outdoor? [21:25:26] cwdent: yeah, i think the ct table has an entry for everyone that clicks [21:25:44] The whole point of the contribution tracking table, is to track people through the whole pipe. [21:25:44] then payments_initial means they at least submitted a form [21:25:58] I think it's currently broken to some degree on donate wiki. [21:26:01] But it used to start there. [21:26:08] and finally the contribution table in civicrm has the successful ones [21:26:14] K4-713: how does it track 'em? [21:26:15] Well... [21:26:26] those have a contact id right? [21:26:31] Everybody who doesn't have a contribution_tracking_id gets one. [21:26:42] As soon as they hit any of our systems, should be. [21:26:52] As they move through the pipe, the row gets more and more filled out. [21:27:15] The very last thing that happens, is the contribution_id in that table gets assigned the id for that donation in civicrm [21:27:24] It's... kind of the cornerstone of how we know anything. [21:28:04] Couldn't run A/B tests or know how effective anything was, without it. [21:28:06] K4-713: looks like we'll need a new column, banner history ID [21:28:13] Easy. [21:28:17] NExt problem. [21:28:19] :D [21:28:22] K4-713: but there's no way to get at the ct table on the same wiki that has access to the banner history [21:28:22] hrm, but won't that then be tied to an individual? [21:28:30] ejegg: And what wiki is that? [21:28:40] donate wiki can write to it. [21:28:45] enwiki [21:28:46] ...which is on the cluster. [21:28:48] or ptwiki [21:28:49] It's not generated from an existing table, just a random number, so designed to be unlikely to collide [21:29:02] I'd suggest, though, passing that id the same way we pass the utm data. [21:29:19] K4-713: yup! [21:29:29] mmm suggestions for a parameter name anyone? [21:29:36] utm_banner_history_log_id? [21:29:48] Eh... do we have to do the utm prefix? [21:30:00] doesn't it sound more finance-y? [21:30:00] urchin tracking monitor live on! [21:30:07] I think that's an industry standard name, for things we are not doing in standard ways anyway. [21:30:24] mmmmm K [21:30:45] i'm confused, won't this end up tying banner history to an individual? [21:30:57] ejegg: cwdent: so I'm correct in assuming we definitely need the non-persistent unique ID on all banner histories, no? [21:30:59] AndyRussG: you were hoping to not make that link, right? [21:31:03] Only after they click on donate and get to our systems. [21:31:08] cwdent: correct [21:31:28] Payments or donate wiki. [21:31:39] ejegg: not so much... only not tie it to users and readers, or make it something useful for broader tracking [21:32:12] So we don't put that banner history ID in the ct table or anywhere correlated to specific donations, just in the session [21:32:32] ejegg: uhh no I think we do [21:32:47] the banner history log fuzzes the timestamp +/- 20 seconds so it won't be possible to correlate with actual web logs or page revisions [21:32:56] then when we know they donated, mark that history as belonging to a donor in a whole different table [21:33:02] Just... don't write anything to the contribution tracking table until the user clicks on something to start making a donation. That is how we count landing page impressions. [21:33:42] it's only a log of banners you've seen or that were hidden from you, so I think so long as it's not tracking content production or reading and can never be tied to that, it's moderately kosher [21:34:10] ejegg: hmm that's a possibility [21:34:14] mmm, still gives a pretty good idea when you're browsing [21:34:15] Also, it's probably worth noting that the two wikis that write to contribution tracking are not under the regular privacy policy. [21:34:34] K4-713: yeah [21:34:36] ejegg: yeah [21:35:19] certainly gives you an overall view of a person's pattern of site usage. But only linked to any individual data after they click donate [21:35:35] hrmmmph. [21:35:49] Note also that banner histories only accrue when you're in a campaign that has the banner history feature enabled [21:35:53] not all the time [21:35:58] Sure, but if we /CAN/ get the info we need without tying contact info to any kind of site history, I'd say we should do that [21:36:12] ejegg: Agreed. [21:36:17] my only concern would be that the actual data collection is happening outside the relaxed privacy policy [21:36:31] It's occasionally (not often) *really great* to be able to say "we don't know that and can't find out." [21:36:36] Let's not know things when we can. [21:36:43] K4-713: +1 [21:36:47] K4-713: totally [21:37:34] i felt best about the initial idea, each pack of impressions is not really user data, just an example of a possible user experience [21:37:35] So I'm thinking another fredge table that's just a list of banner history ids that ended in a donation. nothing else, not even a timestamp [21:38:10] ejegg: yeah that sure sounds more privacy oriented [21:38:47] yeah i like the sound of that, and it's like one bit of additional data [21:39:04] The difference here with client-side stuff is that the user will not have the proof inside their own browser about what we know and what we don't [21:39:33] Well, I take that back, I guess they'll be able to see any code that sends the history id back in a separate logging call [21:39:52] ejegg: do we have EventLogging installed on donate and thank-you wikis? [21:40:08] AndyRussG: will it be sending information back before hitting the EL size limit in cases besides clicking donate? [21:40:21] AndyRussG: nope, not installed there [21:40:22] confusing sentence... [21:41:05] but like if we can store 10 impressions, will it ever phone home before collecting all 10, besides when donate gets clicked? [21:41:26] AndyRussG: but we can record the banner history ids someplace ellery's used to getting the rest of the data he crunches [21:42:03] cwdent: yeah, i was thinking about that too. no way of knowing whether that person's gonna donate on the next banner view [21:42:32] so i think we always want to send a sample back if they /do/ click, even if we sampled the same person before [21:42:55] yeah, depending how you aggregate i guess that stuff could even itself out [21:43:12] Do you guys have an outline of your current thoughts about the data set somewhere? Specifically, what you're phoning home about? [21:43:24] I have so very many old versions. [21:43:26] even if they donate on the 11th banner, that was still 10 banners in a row where someone didn't donate [21:43:42] good point [21:44:01] K4-713: i haven't been typing anything up anywhere [21:44:14] just blathering in irc [21:44:29] Okay. No rush, but I'd like to see where that is now. Maybe next week sometime. [21:44:38] AndyRussG: do you agree we want 100% sampling on click? [21:45:15] (sorry was afk just a sec, answering in order... ^ ) [21:45:19] oh right, we don't want to do double EL calls for clickers that got sampled previously... [21:46:14] ejegg: K no EventLogging on wikis in the donation pipeline, but I guess it'll be easy enough to have an endpoint where we can send just the banner history id and maybe their overall status in the pipeline, no? [21:46:42] cwdent: yes, the banner hsitory ids would go together with the whole bale of data going to EventLogging, every time [21:46:44] AndyRussG: yeah, we'd log it similarly to the utm data, just in a different table [21:46:51] ejegg: cool! [21:47:17] ejegg: sorry I meant ejegg where I said cwdent above [21:47:49] cwdent: sorry out of order! yes, it phones home randomly all the time. number of impressions in there doesn't make any difference [21:48:06] cwdent: so even occaisional users get included in the sample [21:49:08] K4|wandering: yeah here it is: https://www.mediawiki.org/wiki/Extension:CentralNotice/Notes/Campaign-associated_mixins_and_banner_history#Proposal_2 [21:49:09] Proposal 2 [21:50:37] K4|wandering: also there will be details in an upcoming e-mail to legal, that will cc to fr-tech [21:51:46] cwdent: yes, there will be many cases of histories being sent back before someone actually donated. Ellery will have to tease out the real info with statistical black magic [21:53:06] ejegg: yes we want 100% sampling on click to donate. We can just put up a flag when the history has already been sent back, and when donate is click'd only send back history if it hasn't been sent back already [21:53:21] yeah, that might make individual identification of donors even less important [21:53:53] if that information is just aggregated, which it seems like would be the only useful way to do it with those random sample sets [21:54:03] i also have no idea what i'm talking about [21:54:08] * cwdent no statistician [21:54:29] i was thinking that meant the random sample didnt need an id, just the click sample [21:55:51] ejegg: ah right but like u said ^ that could mean double EL calls... Though I guess not such a big deal, it'd only be for whatever percentage of donate-clickers that are included in the global sample, likely around 1%, I imagine [21:56:00] well ellery just responded, looks like it should be easy [21:56:01] amd if the random sample is 1% or less, losing them from the pool of ids leading to donations might not be too bad [21:56:12] just a "donated" flag in the click package [21:57:18] wohoo! OK [21:58:35] nice! [21:58:43] * cwdent goes back to donationinterface fiddling [21:59:39] ejegg: though, I dunno... would Ellery want to be able to catch the double-EL cases when a user sends back a history because selected for a sample, then sends the same history back again because donate-click? [22:00:14] AndyRussG: will it be the same history? doesn't it dump once sent? [22:01:08] cwdent: hahaha interesting point. No it doesn't. It just accumlates and old ones get purged, like a FIFO queue [22:01:11] oh, i was thinking of not double sending / not adding a history id to already - sampled clickers [22:01:27] ejegg: yeah that may be the right way to go to start [22:01:36] i gotta relocate, back in a few [22:02:38] cwdent|afk: in theory all of this has been vetted by Ellery since initial discussions about the feature, but we'll see how it goes when he actually starts numbercrunching [22:03:17] I think the fundamental idea is that his base unit will not be individuals, but page views [22:03:58] So you won't say, 3% of readers donated after 5 banner views [22:04:13] But rather, [22:04:38] 10% of page views during the campaign had 6 or more banner impressions [22:04:50] and then draw percentages on donors [22:05:19] 10% of donors during the campaign had seen a banner 10 times, and 60% of donors had seen a banner 1 tie [22:05:21] time [22:05:23] and such [22:06:22] So in that sense repeated inclusion of the same histories in the global non-click-triggered sample is OK... It's still a legitimate sample of banner histories for page views [22:06:27] I think [22:06:44] * AndyRussG checks multiple-universe convergence assumptions [22:07:24] * AndyRussG has no idea about this really 8p [22:14:52] ok, i'm heading out for the evening. have a great weekend, all! [22:16:18] AndyRussG: that makes sense [22:16:26] ejegg|away: cya! [22:16:29] and thanks! [22:17:46] cwdent: heh thanks, I do hope so :) I think there are a few similar cases of accrual of a data point that Analytics deals with [22:18:05] At least they'll know what to do, and whether we need to change stuff... [22:18:23] yep! i'm sure there is always some back and forth [22:18:36] cwdent: the main issue I guess is that you can only run code when the user goes to the site [22:19:03] hehe, unknown unknowns [22:19:04] So you can't say, timebox and send back all logs at a specific point in time... which might make more sense, if it were possible [22:19:26] If you tried that, I think you'd miss a lot of infrequent site users [22:38:02] Fundraising research, Research-and-Data, Research management: P/T fundraising data analyst JD - https://phabricator.wikimedia.org/T107822#1541574 (DarTar) JD circulated via personal network in France, please reach out if you know of potential candidates. [22:48:38] (PS8) AndyRussG: WIP Banner history logger campaign mixin [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/229560 (https://phabricator.wikimedia.org/T90918) [22:49:20] ^ adds not-persisted unique ID for banner history log when called manually (to be called when donate is clicked) [22:49:29] gotta dash! cya later!!! :) [22:50:56] * dstrine waves [22:50:56] (CR) jenkins-bot: [V: -1] WIP Banner history logger campaign mixin [extensions/CentralNotice] (campaign_mixins) - https://gerrit.wikimedia.org/r/229560 (https://phabricator.wikimedia.org/T90918) (owner: AndyRussG) [22:51:00] have a good weekend [23:01:34] Later friends!