[10:34:34] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, Unplanned-Sprint-Work, and 2 others: Deploy 2015 Lila thank you email + typo fix for br-pt - https://phabricator.wikimedia.org/T110232#1657648 (Pcoombe) Open>Resolved Thanks everyone! [10:57:36] Wikimedia-Fundraising: Fundraising banners on iPad are wrongly positioned - https://phabricator.wikimedia.org/T113220#1657708 (Pcoombe) NEW a:Pcoombe [11:46:39] Wikimedia-Fundraising: Fundraising banners on iPad are wrongly positioned - https://phabricator.wikimedia.org/T113220#1657807 (Pcoombe) Open>Resolved Turned out to be a really easy fix: add `left: 0;` to #frbanner. I've done this for the template banners, and all ones we've used recently, and re-enable... [13:38:30] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, Analytics-EventLogging, and 2 others: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657984 (Ottomata) Just deleted deployment-eventlogging02 [13:46:30] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, Analytics-EventLogging, and 2 others: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657993 (AndyRussG) Thanks so much! Closing... For MySQL querying of the banner... [13:46:40] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: [Mini epic] Activate Banner History! - https://phabricator.wikimedia.org/T112020#1657996 (AndyRussG) [13:46:43] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, Analytics-EventLogging, and 2 others: Beta Cluster EventLogging data is disappearing? - https://phabricator.wikimedia.org/T112926#1657995 (AndyRussG) Open>Resolved [15:52:08] * AndyRussG waves [15:53:32] ejegg: hi! how well would you say NetBeans does JS and PHP? I'm getting a bit annoyed w/ Eclipse and am thinking of giving NetBeans another go after 5 < n < 10 years... [15:54:21] Hi AndyRussG ! [15:54:30] Netbeans is pretty good [15:54:43] but there's an annoying bug under gnome shell 3 [15:54:51] Fundraising Sprint Tom Waits, Fundraising-Backlog, fundraising-tech-ops: Paypal IPN processing is broken - https://phabricator.wikimedia.org/T112919#1659187 (awight) [15:54:51] Hmmm wazzat? [15:55:04] where menus don't work right when it's maximized [15:55:11] is that the wm you use? [15:55:23] yepps [15:55:33] hmm that kinda sucks [15:55:36] yeah, I've been thinking I need to try something else [15:55:39] like phpstorm [15:55:57] Hmmm [15:56:22] I guess Eclipse is actually pretty good w/ PHP. I can do class hierarchies, autocomplete isn't bad [15:56:40] or just finally memorize all the keystrokes to use vim full time [15:56:53] yeah, the autocomplete is decent in netbeans [15:57:07] My main beef is w/ JS. I really wish it could detect the objects I'm declaring that're gonna be pubic, show a summary and let me mouse-click about [15:57:07] nice needle, haystack hints [15:57:33] I'm all party line not going the commerical IDE route I think [15:57:39] there's no js extension for eclipse? [15:58:04] Oh yeah there is, it just doesn't really cut it, as far as I'm imagining it what it should do [15:58:10] Not sure if others are that much better, in truh [15:58:13] truth [15:58:17] ah, bummer [15:58:24] Hmm yeah I think VIM would be a big investment, not sure if it's worth it either [15:58:36] I recommend vim, it's cut my productivity by 75%! [15:58:37] Maybe I'll try upgrading... But yeah last I checked the JS tools weren't even well maintained [15:58:50] awight: noooooooooooooo that's not enuf [16:00:17] vim can turn into a pretty complicated IDE with enough plugins [16:01:16] I have a hard time just getting vim to DTRT wrt tab stops [16:03:13] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/239863 [16:03:39] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/239863 (owner: Ejegg) [16:04:10] awight: http://editorconfig.org/ [16:04:59] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/239864 [16:05:11] Cool! Yeah I just have a bevy of :autocmd statements [16:05:21] (CR) Ejegg: [C: 2 V: 2] Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/239864 (owner: Ejegg) [16:05:38] awight: whatever doesn't kill me makes me stronger [16:06:26] suffering and smiling [16:07:34] awight: it is a pain though. i keep mine set at 4sp tabs that display as characters so it's easy to tell when i'm among spaces [16:07:40] then just do :set ep [16:07:51] covers most cases [16:08:15] until you get into non 4sp tab code [16:08:21] which always makes me sad [16:08:29] !log updated payments-wiki from 4d9d165c40070e036176dba8987243f6dbc7415e to 153418195a45cab820bc2aacf9a4f7dbc9dde768 [16:08:35] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [16:11:41] So vim better than vi? [16:11:48] I used to know emacs way back when [16:12:39] cwdent: I do something like autocmd BufNewFile,BufRead mediawiki-extensions/** set noet [16:13:17] AndyRussG: yeah, vi is the stripped down emergency editor these days, all the features have been added to vim [16:13:33] Ah k interesting [16:13:34] But don't worry, not much has changed in 30 years :p [16:13:40] * AndyRussG upgrades Eclipse [16:14:14] I sure hope not! With the pace of change today, I'm glad at least some things can be relied on to remain familiar [16:14:17] awight: that's a good idea [16:14:56] I'm not about to make "noet" my default ;) [16:18:03] cwdent: I'm just catching up on the WP France document. Did you determine whether we have access to the donor's name if we let WP collect it in the iframe? [16:18:39] On the other hand, if they hose the Unicode I don't know what to say. [16:22:29] awight: i left a comment on the doc, iframe post fails w/o name [16:22:57] cwdent: gotcha! but if we let WP collect the name, can we get that back in our system? [16:23:12] awight: i think so, lemme make sure [16:23:21] And, the unicode thing I'm referring to is just that WP stores intl characters as... nothing [16:23:43] So if we get mangled names back from them, this isn't such a productive line of action [16:23:57] latin-1 today, latin-1 tomorrow, latin-1 forever! [16:24:01] -wp [16:24:18] latin-wp, a subset which doesn't even do latin [16:24:51] They replace "q" with "^" and "v" with "&&", which then proceeds to break their database [16:24:54] ;) [16:25:02] hehe [16:25:20] awight: yeah i see the name in the response for the A request [16:25:25] ooh speaking of which, I fed a bunch of escape chars into a paypal notes field the other day... Gotta see what we caught in our db [16:26:11] cwdent: oh good. Mebbe try 吳稚暉 [16:26:38] Or Hervé if you wanna play nice [16:28:01] awight: actually looks ok in the response [16:28:13] but i see ?????? in the next request...losing it somewhere else [16:28:14] O_o [16:28:17] could be my setup [16:28:54] ejegg added a transliteration thing, which could be responsible. We might have to wire that up differently if we're relying on WP for the name. [16:29:30] hmm, yeah [16:30:00] also, that might be acting differently in 5.4+ dev setups than on prod's 5.3 [16:30:22] right, it switches libraries based on what's available [16:30:25] awight: i'm mostly worries about PPena's recent comments [16:30:31] cwdent: if you're trying to debug a prod thing, you may want to futz with EncodingMangler to force it to use iconv [16:30:43] Too bad we don't have multi-version CI [16:30:57] awight cwdent do my comments make sense? [16:30:58] if they can't send the esop request through the right backend processor that could torpedo the whole project [16:31:14] * awight head in hands [16:31:38] You'd think, as a reseller they... nvm [16:31:43] PPena: yeah i think so, i wasn't aware the different store IDs were...different processors [16:31:59] hope they can sort that out [16:32:13] PPena: yeah please lean on that if you get the chance. It's scary [16:33:39] cwdent yes. also, you mean merch ID. We have 3 (see WorldPay Master Processor doc- merch and store IDs sheet) [16:33:57] cwdent Store IDs are currencies- that we have a BUNCH :) [16:34:20] cwdent (store id versus merch ID (I knot, its confusing) [16:34:34] PPena: aah gotcha, yeah it's a big config section :) [16:35:12] Not trying to throw a wrench, but keep in the back of your mind this trivia: for one of their processors, the meaning of store ID and merchant ID are reversed. [16:36:05] sounds about right [16:39:19] cwdent: are the iframe languages for ESOP documented somewhere? [16:40:25] atgo: you mean language translations? not that i know of.. [16:40:30] ok, thanks. i'll ask [16:40:37] atgo cwdent WorldPay new integration options -screenshots has as much info as we know [16:40:45] english and french are live [16:40:52] PPena: i'm curious about hebrew [16:40:58] the remaining backlog is there, but not live yet [16:42:12] hey K4-713! you're around and working remote today, yes? [16:42:28] Yes. But I'll be on and off. [16:42:38] ...mostly getting on planes and stuff. [16:42:41] ahhh that [16:42:56] Also, today is my meetingwall checkin day. [16:43:03] yes, that's what i was asking about [16:43:08] then realized that we already rescheduled until tomorrow [16:45:20] AndyRussG: Did you have any new thoughts about the global vs per-campaign banner history? [16:54:37] awight: sorry I was just getting some wifi flakiness and then irc lient flakiness. Here are the last messages I sent, in case u didn't see them: [16:54:39] awight: I did think a bit about how it could be implemented globally. But I do have to confess that I feel now is not the time to be changing requirements that were discussed a while a go and that our users have already signed off on [16:54:41] awight: maybe I'm being a blind bureaucrat, I dunno... [16:54:43] but I think this was pretty clear to Ellery and co. and I'd really rather concentrate on getting it out the door and having it used as it is, first [16:54:50] awight: also I didn't see any replies you might have sent after that, if any [16:55:40] Pls LMK of course if u think I'm being completely wrong-headed [16:56:52] hey AndyRussG awight! [16:56:57] atgo: hi! [16:56:59] this feels like something i should be involved in the conversation about :) [16:57:09] atgo: yea! [16:57:47] awight just caught me up a bit, so i'm goign to take the question back to our users. i hear what both of you are saying [16:58:04] i think we may want to get to MVP being per campaign and then extend to global later [16:58:12] AndyRussG: My question though is whether it's actually the requirement... Totally agree that we should just get people using the MVP, but for example the increasingly complex TTL work doesn't need to happen if we decide to keep a global history. [16:58:29] remember that one of our use cases is for people who are getting low-level banners over time [16:58:29] atgo: to be very clear, the current implementation _is_ global and _can_ store banners for any number of campaigns. [16:58:51] awesome [16:58:59] That's a relief to me, too ;0 [16:59:02] yeah that's my one question - is there increased complexity for one of these paths? [16:59:10] awight: no, I disagree. We do need the global ttl because we want to move cookies to kvstoreage too [16:59:24] That's a different feature though [16:59:32] awight: but it would use the same kvstore infrastructure [16:59:56] so making bh always-on-global doesn't get us out of the ttl. Also, it's not that complicated. [17:00:26] atgo: the only way in which BH is not global is that it has to be manually enabled for any campaign to be collecting history into the global storage point [17:01:23] so, you can have 1, 2, 3 or 10 campaigns putting data in banner history. If the 10th campaign sends back the log, it can have data in it from any previous campaign that the user got that had bh enabled [17:01:47] I think it does get us out of the TTL work, cos we would be maintaining a single small circular buffer, and there would be no reason to expire items. [17:02:08] awight: no, because we'll be using it instead of cookies, too. [17:02:19] that's... a separate feature no? [17:02:19] Or would the cookie parts also have to take care of their own ttl? [17:02:46] awight: that's not a reason not to take it into account if it'll be using a shared piece of infrastructure [17:03:46] atgo: there are two catches to the current implementation, that awight is talking about. One is that you have to manually enable BH on all the campaigns that you want it to capture data for [17:06:23] The other one, which we are ^ apparently disagreeing about :) is whether it's more work and more complex to implement a global expiry system for the storage system that we have for BH, or whether it's just easier to enable it all the time everywhere and in theory not have to worry about the global expiry mecahanism. Also just to give you some more fun details, said global expiry mechanism is already implemented [17:06:23] and merged, but the performance team didn't like how we did it, and are suggesting a better way, which I think is not hard to do, and awight I think (I shouldn't speak for him, tho) disagrees ;p [17:08:36] atgo: the reason it has to do with other features down the road is that this global stoarge system, which currently is houses BH data and which the performance team wants us to patch up a bit more still w/ the improved expiry mechanism, is planned to also take care of things that are currently stored in cookies. I think if we eventually do that we'll just have to implement an (acceptable to performance team) [17:08:36] global expiry system anyway, so we might as well just do it now, and then not have to change the BH feature anymore [17:08:42] hehe. I'm just getting in your way here, sorry, the intention is at some level to save you the work, but it's not really playing out the way I'd hoped. [17:09:15] awight: finally checking on all those duplicate recurring things in civi, and we need to update a bunch of them [17:09:52] About the expiry though, there is a reason to not add it to BH if we're using a circular buffer--it increases the complexity and fragility of the system. [17:09:54] * AndyRussG tries not to recall his high school debating team days [17:10:07] I'm looking at recurring_import_subscr_cancel in recurring.module, but I'm not sure it's doing everything right [17:10:13] oh dear [17:10:24] shouldn't it set the contribution_status_id to Cancelled (=3) ? [17:10:28] I'm thrilled that you're looking at that before code freeze! [17:10:33] awight: not in the way at all! I think whenever there's a concern like this it's totally important to talk it through [17:10:35] That sounds fight [17:10:39] har [17:10:41] *right [17:10:59] awight: the circular buffer isn't an issue because all of BH is stored in a single KVStore item [17:11:25] I'm still not understdanding why we would expiry anything out of the BH log [17:11:25] so long as that item doesn't expire (and expiry time is refreshed on every access) it's all good :) [17:11:39] awight: well it can't be infinite [17:11:45] We can set high limits though [17:11:48] AndyRussG: so the only real change would be that we would be automatically sending vs. needing to opt each campaign in? [17:12:09] But if the log is full, we pop the last item when adding a new one [17:12:54] awight: we pop the oldest item out when adding a new one [17:13:18] and we can define "full" within whatever limits we want, it can be 100 or 1000 log entries if we think the clients can take it [17:13:29] So, what does expiry do? [17:14:31] The KV store will expire any KV store item, be it BH or anything else stored (say, stuff currently in cookies) a given amount of time after its last access. The default time in the first implementation is 1/2 a year, but it could also be made configurable quite easily [17:14:55] To make it, say, 1 year, as in the case of the donate cookies [17:15:08] a year after time of last update? [17:15:13] That would never happen for BH [17:15:18] awight: last time of access [17:15:38] for cookie-ish stuff, I guess we could make it not update on access [17:15:54] awight: currently it's 1/2 a year but it could be configured on a per-item basis [17:15:59] just like you can do now with cookies [17:17:03] AndyRussG awight i'm not feeling like this is clear enough to take to the stakeholders without knowing 1) additional effort involved and 2) end user changes [17:17:09] Stepping away from BH for a minute, it doesn't seem like a good idea to be writing a generalizable, awesome new storage method without some serious big-picture buy-in from other teams [17:17:35] effort can be reaaaally loose, but the actual difference is what i need to know about [17:17:36] I mean, you did it already which is really impressive, but now we're getting weird pushback stemming from how we're isolated. [17:18:12] What I'd like to see is that the Analytics team goes "hey, that's a brilliant new mechanism that solves our anonymity problem" and then takes up the banner [17:18:59] awight: hmm and thanks... I don't feel it's pushback, more like perfectionism, which also hints that they might like to iterate on it and generalize it one day [17:19:16] there's not a good track record of that happening at WMF :( [17:19:18] (tho the compliment is undeserved) [17:19:23] AndyRussG awight - i'm getting to where i'm going to talk to megan in 1 minute [17:19:33] so, what is the actual end user difference between these approachs? [17:19:52] I think, almost no end user difference [17:20:14] atgo: really just what you said before, for the users. In the current approach, to get BH from a campaign they have to manually enable it in that campaign [17:20:47] but the data will all be the same in the end? [17:20:49] atgo: a campaign doesn't add log entries unless BH is enabled for it. [17:21:05] atgo: yes, unless they forget to enable it for a campaign that they're interested in [17:21:18] ok, thanks [17:21:19] i'll check [17:21:43] AndyRussG: what about "pageview with no banner impression", I assume that would only be recorded in the "global" scenario [17:23:08] atgo: enabling it globally also has work-time consequences, so if they can live with it or at least try it out as is for the MVP, which is how we decided together a while ago, that'd be, IMHO the fastest and best route [17:24:07] awight: it depends on what the cause of the "pageview with no banner impression" is. If it's because a cookie or an in-banner script hid the banner, then it _is_ currently recorded in BH. [17:24:13] question AndyRussG: will they still be able to tie impressions to campaigns, etc. with global implementation? [17:24:47] (PS2) Awight: More inline help for mixins [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/239302 [17:25:04] atgo: the global implementation, if they were to wish for it, could work in any number of ways, which we haven't even talked through yet. But they could certainly get the same data [17:26:03] awight: currently the only scenario in which a no-banner-shown would not result in any log is if the user is not targeted by a campaign, because you have to be in a campaign w/ BH activated for it to run [17:27:17] That fits with my limited understanding. [17:27:32] * awight stealthily bumps https://gerrit.wikimedia.org/r/239302 [17:28:30] AndyRussG awight - the important thing is that we can get all the impressions across the different campaigns. if they have to be manually enabled, that's OK for a place to start for sure, as long as we can see a history with multiple campaigns [17:28:33] indeed we talked about this issue when discussing requirements. If it were always on and even recording pageviews when the user isn't in any centralnotice campaign, then it wouldn't be a BH feature, it'd be a global page view tracker [17:29:21] atgo: cool thanks!!! [17:29:36] ejegg: This is why the failure to set cancelled status hasn't been breaking things: AND ( civicrm_contribution_recur.end_date IS NULL ) in recurring.inc [17:30:30] * AndyRussG registers above described-and-thereby-enacted bump [17:30:32] AndyRussG: erm not quite a pageview tracker, we're not recording the page viewed [17:30:46] lol debate mode ;) [17:31:04] I did have a lot of fun the one time I went to a debate thing [17:31:18] awight: well it'd be recording time and wiki and logged-in status. [17:31:30] cwdent: Meeting? [17:31:38] omw sorry! [17:31:48] No worries. [17:31:52] This may be ridiculous. [17:31:54] AndyRussG: creepiness is definitely a good counterargument :) [17:32:43] awight: yeah, would still be nice to show the right status! [17:33:05] CRM_Contribute_BAO_ContributionRecur::cancelRecurContribution might be useful [17:33:08] ejegg: I guess it's possible to backfill from cancel_date is not null [17:33:12] oh! [17:33:12] yah [17:33:17] awight: yeah recalling how the discussion went, it was at least an implicit concern I think. From a more software-architecty POV, that'd also make it ouside the CN scope, really [17:33:40] AndyRussG: awight just to confirm 100% - we won't lose any data by doing per campaign, right/ [17:33:48] atgo: correct [17:33:54] cool, thanks [17:34:03] donno. I think just incrementing an internal counter "didn't see a banner" would be plenty [17:34:47] atgo: AndyRussG: The data we're losing is "reader saw 15 WLM banners, then didn't see a banner for 10 pageviews" [17:35:20] if BH is not enabled for the WLM campaigns, we'll have no record of people seeing those banners in-between our FR banners [17:35:49] awight: it'd have to be a "when" in the history. If you're storing 50 entries, 30 "didnt' se a banner" events followed by 20 banner events is different from 20 banner events with non-views interspersed throughout [17:35:59] atgo: ^ what awight sez is correct [17:36:29] atgo: except if the "didn't see a banner for 10 pageviews" is due to our own show/hide program [17:37:13] atgo: if the banners are hidden by our own code by design, as currently happens, those hides, as well as hides due to donate cookies or close-button-clicked cookies _are_ recorded in the current implementation [17:37:30] AndyRussG: Isn't there a thing to roll up multiple views of the same (non/)banner? [17:37:49] awight: huh? [17:38:00] awight: ah I see [17:38:10] No, there isn't. It's a straight out timeline [17:38:14] k, that's cool [17:39:14] (PS1) Awight: Revert "Shut it off." [wikimedia/fundraising/PaymentsListeners] - https://gerrit.wikimedia.org/r/239889 [17:39:26] atgo: ^ I don't think it should be hard to enable BH for intervening campaigns like WLM. We can also set the WLM campaign not to actually send back the history, just record it so it gets sent back for users when they get the FR banner, if that's desired [17:39:59] atgo: The kind of "didn't see a banner for 10 pageviews" that awight is talking about is when just users who weren't targeted by any campaign at all [17:40:47] K4-713: Want a fun one? https://gerrit.wikimedia.org/r/239889 [17:41:11] Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, Fundraising Tech Backlog, Fundraising-Backlog, and 3 others: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1660099 (Jgreen) > What is the highest sampling rate we can afford? Significantly... [17:43:34] Out for a couple of hours... [17:44:18] awight|away: Looks like you had a fun week. [17:45:40] (CR) Katie Horn: [C: 2] "Fun!" [wikimedia/fundraising/PaymentsListeners] - https://gerrit.wikimedia.org/r/239889 (owner: Awight) [17:46:44] Holy Business. This hotel is the loudest hotel I've ever hotel'd. [17:47:55] I sat down in a place that looked quiet earlier, and got Leonard Cohen *blaring* that Hallelujah song right... on my head. [17:48:01] Right there. [17:48:09] Wow [17:48:34] Not... exactly conducive to anything but a rising urge to kill, really. [17:48:55] I am now somewhere marginally better. [17:49:40] However, I think they don't bother to aim a speaker right at this chair because I'm close to the loud clanky kitchen. [17:50:37] ejegg, cwdent: Any tips for finding a quiet public place to take gchat calls? I am apparently terrible at this. [17:51:41] K4-713: rooftop? [17:51:47] Oooh. [17:51:55] I don't think I can get there from here, unfortunately. [17:52:08] I tried to take the stairs down one time and I think they sent security after me. [17:52:25] Why do people hate stairs now. [17:52:41] Also, why is paypal exploding. [17:52:54] grr, again? [17:52:59] Looks like. [17:53:07] BAD_AUDIT_LINE [17:53:20] oh, just got it [17:53:20] NO_BISCUIT [17:53:36] urlerror connection timed out? [17:53:55] guessing that's related to the new addresses we need whitelisted [17:54:02] * K4-713 sighs [17:54:07] yarp [17:54:13] See, this is the kind of thing that would already be handled if it wasn't a mutant. [17:54:31] got the whitelist blues with the amazon listener too... freaking aws ip ranges are gargantuan [17:54:47] The other audit scripts know the difference between a bad line of data, and a sad environment. [17:54:56] ah, true [17:55:15] so, that looks like a thing we should disable for now [17:55:41] Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, Fundraising Tech Backlog, Fundraising-Backlog, and 3 others: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1660173 (ellery) + 1 [17:55:42] Oh, probably. [17:55:44] K4-713: can you tether? [17:56:03] Can I... let's just say "Yes". [17:56:08] But it's not great. [17:56:12] it's convenient, but can get expensive [17:56:20] the LTE in longmont is faster than my home internet [17:56:24] which isn't saying much [17:56:26] #centurylink [17:56:27] I'm one of those people who got an actual mifi for emergencies. [17:56:40] nice [17:56:59] I'd hand it over to awight if I thought he'd fall for it. [17:57:05] !log disabled paypal audit parser [17:57:10] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:01:00] K4-713 - I noticed all the failmails and see you’re on it. Do you guys have a process for these bad_audit_lines? My recollection was this is a PP error and we always had to contact them to notify them things were exploding [18:01:15] so just adding context if that’s helpful… I can ask MBeat to follow up with them [18:13:20] ccogdill: Sorry, was on a call and then forgot to click back over here. [18:13:44] no worries, that was all I had to offer :p [18:13:55] Ah, honestly, I don't know anything about notifying paypal when the audit script explodes. If we do that, it's news to me. [18:14:33] IPN messages are a little different, because after a few they turn off the service until we tell them we're okay again. [18:14:44] Audit, though? Totally news. [18:14:48] well, it’s what we did during similar bad_audit_line failures in February and April… but I know there have been more since [18:15:02] It's probably worth asking them what the heck. [18:15:12] k [18:15:47] ccogdill: Also, I should put it on record that this is, you know... a Process at this point. [18:16:01] And I like that. [18:16:02] MBeat - are you aware of all these bad_audit_line failmails for paypal? [18:16:03] A lot. [18:16:13] I mean… do we have a process is place? [18:16:16] i haven’t seen then, ccogdill [18:16:19] Apparently we do. [18:16:20] :p [18:16:23] sorta :D [18:16:29] but will be happy to ask PayPal about them [18:16:36] Please edit the process stub... [18:16:39] MBeat this happened a few times last year. It seemed to me that Kevin fixed things [18:16:53] MBeat: Apparently it's not bad data. Looks more like we can't contact them. [18:17:06] Fundraising Sprint Kraftwerk, Fundraising Sprint Lou Reed, Fundraising Tech Backlog, Fundraising-Backlog, and 3 others: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1660255 (Jgreen) Ok, this is switched for all americium collectors. [18:17:10] cool, thx [18:17:21] but I don’t know what or how. You can find a few sample trxn ids in the failmails. I would usually send a few to Kevin and say this BAD_AUDIT_LINE thing was happening again [18:17:41] Which... I kind of wonder if this is related to the IP address whitelist changes I'm just now seeing go by... [18:17:54] hm… sounds promising [18:17:58] will do, thanks ccogdill & K4-713 [18:18:15] ejegg: Hey... you know anything about the paypal IP address whitelist changes going by? [18:18:22] Oh wait. Jeff_Green is here. [18:18:40] lalalala tralalala [18:18:48] thank you MBeat! [18:18:52] HI JEFF. [18:18:55] ccogdill, did you get the usual Failmails for these? [18:19:07] what do you mean "usual"? [18:19:09] K4-713: welcome back! apparently they changed their IP lineup, confirmed and updated on our end [18:19:28] Jeff_Green: So... funny thing. One of our paypal scripts suddenly can't phone home. [18:19:48] Wondering if some relevant range got nuked. [18:19:53] oh probably [18:20:00] like a “Failmail (wmf_common)” that mentions Bad Audit Line? [18:20:04] That would make it easy, then. [18:20:06] I certainly got them, MBeat. They’re in my normal FailMail folder :) There were over 100 on 9/17 [18:20:09] i did clean up some other IPs that were never documented [18:20:19] K4-713: do you know which hostname they're trying to reach? [18:20:20] Aaaaha. That might explain it. [18:20:22] it should be in the subject line… I will fwd [18:20:27] Jeff_Green: ...er. [18:20:37] paypal publishes a list of IPs for each hostname [18:20:45] hmm [18:20:46] Jeff_Green: No. But I should delegate this anyway. [18:20:58] ejegg: ...I think I'm tagging you in. [18:21:06] What do you know about the paypal audit scripts? [18:21:07] could you fwd one to me, ccogdill? I didn’t receive them [18:21:16] K4-713: where are you seeing the error? maybe I can figure it out from the appropriate log/spew [18:21:35] Oh... mostly a ton of BAD_AUDIT_LINE spam from an hour ago. [18:21:38] That's all I know, really. [18:21:59] oh hmm, lemme see if I got that mail [18:22:19] URLError: [18:22:49] done MBeat [18:23:03] thx [18:23:06] ccogdill, MBeat: This is likely something we did. [18:23:11] oh [18:23:18] ok [18:23:30] So, maybe hold off on contacting them until we know more. :) [18:23:36] fwiw, the 100 failmails from 9/17 do not present like the past bad_audit_line fails I’ve seen [18:23:55] they include no contact or transaction data whatsoever. Never seen that before [18:24:00] Note to All Selves: Better Fail Logging with Paypal Audit. [18:24:18] documentation blegh [18:24:47] ccogdill: I'm not especially surprised. It looks like we can't phone home to get the required info for this set. [18:24:57] Just... failing at a different point in the process. [18:25:08] well that’s good to know. [18:25:15] let us know if/how we can help from here on out [18:25:22] Definitely. :) [18:25:38] ccogdill: Thanks! [18:25:48] welcome back! :) [18:26:44] Thank you! [18:26:52] It was hard to leave the islands. [18:28:04] it *always* is. but at least you came home to nearly tropical weather [18:28:14] Well, I'm not there yet. [18:28:18] Still in Portland. [18:28:26] well then… if you come home this week, that should hold true [18:28:40] Oh yeah. I'm back in the office tomorrow. [18:29:00] The flight today was a *ton* cheaper than trying to leave yesterday... [18:29:12] yep. weekdays [18:29:17] Also, I think I'm getting the hang of cafe working. [18:29:23] It's kind of nice. [18:29:25] the energy can be fun [18:29:30] People are looking at me oddly. [18:29:40] K4-713: I think i may have figured it out [18:29:58] looks like it was probably api.paypal.com [18:30:03] Jeff_Green: oh? That's reassuring. :) [18:30:06] Also seems legit. [18:30:26] possibly r00tkit.payp4l.biz though [18:30:42] ! [18:30:43] :p [18:30:52] Fundraising Sprint Tom Waits, Fundraising-Backlog, Research-and-Data, Unplanned-Sprint-Work: Review banner history log data and confirm that it satisfies use cases - https://phabricator.wikimedia.org/T112986#1660327 (DStrine) [18:32:44] K4-713: one sec, having audio issues.... [18:32:52] No worries. [18:33:55] Fundraising-Backlog: Teach ccogdill to get failed payment data through grep - https://phabricator.wikimedia.org/T113268#1660337 (CCogdill_WMF) NEW [18:34:14] Sorry i missed your tag K4-713 - reading backscroll [18:35:03] No worries. [18:35:07] I think we got there. [18:35:17] K4-713: arg, K rebooting :( [18:35:36] ah, cool. found that awight email from a few hrs ago, 'New listener whitelist' ? [18:35:54] ejegg: Nutshell: When Jeff_Green gives the all-clear on the IP whitelist modifications, enable the audit job again. [18:36:02] And see if that was it. [18:36:05] will do! [18:36:05] oh sorry, that's done [18:36:09] Ha. [18:36:09] go for it [18:36:13] WOOO [18:36:17] hitting the switch... [18:36:40] there was another connection failure to another IP I haven't managed to track down yet (there's no reverse DNS) [18:36:54] !log re-enabled paypal audit parser [18:36:59] ok [18:37:00] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:37:15] Jeff_Green: It would be too easy otherwise, right. :) [18:40:37] (PS1) Ejegg: Use Civi method to cancel recurring via qc [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/239900 (https://phabricator.wikimedia.org/T110367) [18:42:19] audit's not exploding so far... that might have done it! [18:50:25] yay [18:51:48] Fundraising Sprint Snoop (Dogg|Lion), Fundraising-Backlog, Patch-For-Review: AstroPay audit parser failing - https://phabricator.wikimedia.org/T112779#1660417 (DStrine) [18:52:07] Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Tech Backlog, Fundraising-Backlog, Patch-For-Review: Settle unslain orphans - https://phabricator.wikimedia.org/T109668#1660421 (DStrine) [18:57:49] Fundraising Sprint Tom Waits, Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Nightly Amazon report download - https://phabricator.wikimedia.org/T109654#1660436 (XenoRyet) a:XenoRyet [19:03:28] Fundraising Tech Backlog: Missing Donation in Civi/Showing as Duplicate - https://phabricator.wikimedia.org/T113263#1660490 (Krenair) I'm going to guess tech? [19:09:04] Fundraising Tech Backlog, Wikimedia-Fundraising-CiviCRM: Missing Donation in Civi/Showing as Duplicate - https://phabricator.wikimedia.org/T113263#1660541 (LeanneS) [19:09:59] Fundraising Tech Backlog, Wikimedia-Fundraising-CiviCRM: Missing Donation in Civi/Showing as Duplicate - https://phabricator.wikimedia.org/T113263#1660195 (LeanneS) @aklapper sorry for missing that step! I have associated it with Wikimedia Fundraising - CiviCRM. [19:16:15] Wikimedia-Fundraising-CiviCRM: Missing Donation in Civi/Showing as Duplicate - https://phabricator.wikimedia.org/T113263#1660563 (Krenair) [19:19:11] Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, and 2 others: GC error causing donors to create multiple recurring donations - https://phabricator.wikimedia.org/T110367#1660577 (Ejegg) Update: 47 of the refunded donations s... [19:28:40] hey AndyRussG|school awight|away - for future reference, we can reach ellery probably by putting tickets into #research-and-data in phab [19:29:09] atgo: ah cool :) [19:36:58] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Missing Donation in Civi/Showing as Duplicate - https://phabricator.wikimedia.org/T113263#1660673 (atgo) [19:54:14] (PS1) Cdentinger: Record donor name after card authorize [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/239925 [19:56:13] Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, and 2 others: GC error causing donors to create multiple recurring donations - https://phabricator.wikimedia.org/T110367#1660701 (Ejegg) All subscriptions have been canceled,... [19:59:34] http://stackoverflow.com/questions/31612082/why-does-mt-rand1-php-int-max-always-return-an-odd-number/31614655#31614655 [20:00:52] wtf, really? [20:01:23] go php [20:03:28] bwahahaha that's amazing [20:04:38] XenoRyet and AndyRussG: standup time? [20:21:15] congrats cwdent!!! [20:21:39] :) [20:43:39] !log updated worldpay config on payments [20:43:43] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [20:53:26] lol... [20:54:08] * awight buys more Zend shares while it's cheap [20:54:12] You know how I keep getting emails meant for other Katie Horns? I just got the best one so far. [20:54:20] It's just a picture of a couch. No explanation. [20:54:24] good! [20:54:26] Clearly in somebody's living room. [20:54:34] You need something to start collecting again? [20:54:51] I want to write them an amazing reply. [20:54:54] My dad has an album just of mattresses on the sidewalk in Berkeley [20:55:06] ... but he won't accept submissions from third party photographers [20:55:11] "Your couch has inspired me to rethink the pillars of my life." [20:55:15] :D [20:55:40] "How did you know. How could you possibly have known that I needed to see that couch at that exact moment in time." [20:56:01] "Forever in your couchy debt," [20:56:35] ...but I can't just sign it the way I usually do. Needs disambiguation. [20:56:56] - CouchLuver1995 [20:57:04] bahaha [20:57:57] O_O [20:59:05] It's usually somebody else's bills. This is much better. [20:59:39] cwdent: Just confirmed that HHVM has the same bug in mt_rand [21:00:07] Giant, unrewriteable web stacks already depend on the zeroed bits [21:00:09] awight: that's sad [21:04:38] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661045 (LeanneS) Thanks, @XenoRyet! The file im... [21:06:10] ejegg: Can I undo the SmashPig Amazon spam suppression? [21:08:10] awight: yeah, I can turn off IPN at the merchant console for now [21:08:20] still need some whitelisting for it to really work [21:09:32] OIC. I'll just leave it jacked into your circuit, then [21:10:14] Did I swear my allegiance to Zanfel yet? [21:10:33] Planning to get tubes for everyone I know who goes outdoors [21:11:03] nice, it actually worked? [21:11:23] Mostly! Saving my life cos the one place where I didn't get the oil off in time is... decimated [21:11:33] whew [21:11:38] yikes [21:12:08] Half the fun is sanding off the boils with the grit they built into the cream [21:12:16] * awight wanders off, whistling [21:13:41] (CR) Awight: [V: 2] Revert "Shut it off." [wikimedia/fundraising/PaymentsListeners] - https://gerrit.wikimedia.org/r/239889 (owner: Awight) [21:14:09] Jeff_Green: Hey, didn't you set up Gerrit for the frack puppet repo? [21:14:25] no it's not in gerrit [21:15:01] I'm thinking of poking at a few of the config standardization issues as they come up, what's the best way to um coordinate or review or whatever? [21:15:35] (CR) XenoRyet: "Quick inline comment" (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/239925 (owner: Cdentinger) [21:15:43] it's just a shared repo on the build server, and as it stands only ops-hat people have write access [21:15:48] you can clone it though [21:15:56] !log legacy PayPal listener updated from 1c9ac2e66d11bbf768ea873d6e1a2522ca9841c1 to 55aeef63f6508381e3a8b7fcabddf9a3c3b73b8e [21:16:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [21:16:03] see motd on boron for the repo info [21:16:28] You mind if I take one home? [21:16:48] the main repo is fine to clone and have on your laptop yeah [21:16:51] Then I guess I'll just email you a format-patch or something? [21:16:54] sure [21:17:11] Are the missing private bits somehow documented, so I can make a working system/ [21:17:14] ? [21:17:40] there's a separate repo with private stuff, i don't think you'll have read access [21:18:14] What I'm wondering is how to locally fake out those missing variables, without knowing exactly what's missing? [21:18:15] I'm trying to move that repo to be very small--i.e. to populate puppet variables [21:18:16] XenoRyet: thanks for the comment! i'm guessing a bunch of terrible regex could approximate name split, if wrapped up and sequestered somewhere... [21:19:11] awight: related to megan's slides, the name we get back from the auth is of course one field [21:19:15] yah, that's a good question. what I do in virtualbox is create dummy data, but that's not easy without also being able to look at the real stuff [21:19:16] so fname/lname is tough [21:19:20] cwdent: Yea, it's one of those problems that there just isn't a super clean solution to, but it's fun to pick at. [21:19:53] cwdent: Megan's okay with asking for name twice? Seems like the sane choice in an insane world [21:19:54] awight I guess if you tell me in advance what you're working on I can create dummy versions of the stuff that's in the private repo [21:20:18] Jeff_Green: I’m going to get the CiviCRM tarball checked into a branch now & I’ll let you know when it’s there. I am not optimistic enough to expect it to upgrade smoothly first run… but would be good to see [21:20:49] Jeff_Green: oh--I can just search for "private" [21:20:50] eileen: ok great. I can get the test site ready and tied to that branch [21:20:55] awight: nice T2 reference. is she actually ok with that? [21:21:02] cwdent: :) [21:21:09] cwdent: That was my question, too [21:21:36] awight: yeah, that's a good starting point [21:21:53] Jeff_Green: jfyi, I'll start with the one I just noticed, where we need a way to shut off the legacy PayPal listener [21:22:21] Would it be easy to give fr-tech some kind of push and reload privs? [21:22:26] awight: you mean for planned maintenance? [21:22:44] The situation I just ran into was a 2AM emergency [21:22:59] I had to self-merge and deploy a code change that called exit() [21:23:13] ow [21:24:51] But yeah, from my uninformed perspective that's looking like the pain point if we try to move towards more puppet config--tech would have to call ops [21:25:25] right [21:25:29] thinking [21:26:19] I built knobs into puppet for this but there's no easy way to provide access to those knobs [21:26:34] hieradata? [21:27:10] I don't see how that would make a difference as opposed to class params [21:27:33] also puppet is a gnarly tool since it doesn't guarantee anything like timeliness [21:27:47] What problems are we solving? Avoiding privilege escalation of tech accounts? [21:28:32] essentally, yes. [21:28:39] gotcha [21:28:54] timeliness is not the most essential thing, I think we can live with +/- 5 minutes [21:29:11] probably more like +/- 10 min [21:29:27] That's pretty bad. No way to improve that? [21:29:41] It's just how long a puppet run might take? [21:30:09] puppet is supposed to run intermittently and not all at the same time [21:31:11] since it occasionally faceplants without explanation we have cron hacks to force it to restart periodically [21:31:32] looks like 2x per hour we force a run [21:32:07] We could do something really gross, and have the FR application- and service-level config in an isolated modules directory, which gets pulled in by the agent, but we also have an immediate puppet apply script which runs just on the knob modules [21:32:10] there's also an old tool, deprecated in newer versions (i.e. the one that's stock with trusty), to remotely force a run [21:32:27] hey cwdent - dstrine mentioned that there might be some churn on megan's requests for the WP form [21:32:32] anything i can help clarify? [21:32:33] I <3 puppet apply. [21:32:36] or prioritize? [21:33:21] puppet apply? [21:33:40] Guerilla way to run puppet w/o an agent, or behind the agent's back [21:35:04] atgo: did you get that google doc? [21:35:13] i did! [21:35:26] i made a couple of comments so far, but happy to talk through things [21:35:38] oh right on, lemme pull it back up [21:38:02] awight I don't see...yet...how puppet apply relates here [21:38:36] unless the idea is that we'd use it to override the puppet-propagated config per-node [21:39:20] atgo: regarding the name thing, https://gerrit.wikimedia.org/r/#/c/239925/ gets rid of the fields, but the name we get back from wp is one field, which makes splitting it into first/last/etc difficult [21:39:36] XenoRyet noted that the same problem exists for amazon: https://phabricator.wikimedia.org/T86720 [21:39:43] hmm [21:39:51] we have a way of parsing for Amazon [21:39:58] that works pretty OK [21:40:11] MBeat: do you have any thoughts about extending the parsing from amazon into worldpay? [21:40:13] Jeff_Green: It's a terrible idea, but yeah something like that. Saying it out loud, maybe what we actually need is to finish the /etc/fundraising config centralization. [21:40:25] it'll mean more inaccurate names... [21:40:31] also ccogdill might be intersted ^ [21:40:47] we also have an outstanding request to update the parsing that could help [21:40:56] if it’s possible sure [21:40:59] https://phabricator.wikimedia.org/T86720 [21:41:14] but if it will mess up TY salutations, we can use the wildcard to search Civi [21:41:39] it'd be the same as amazon, i assume. right cwdent XenoRyet? [21:41:58] MBeat it sounds like worldpay’s thing will send us the name all in one field? so that will kind of mess up salutations [21:42:27] The patch that's in review for WP right now does it slightly different than Amazon, but it is the same problem. [21:42:31] when we send an email, it’ll be Dear Michael Beattie - rather than Dear Michael. It’s a bit stiff :) [21:42:38] yup [21:43:00] hmm [21:43:05] well we've got a tradeoff [21:43:31] i'd be happy to use a generalized parser, maybe mw needs one [21:43:40] cwdent XenoRyet - let's leave this one out until after the 1 hour test and we can work through these decisions first? [21:43:44] seems like it's a bigger conversation [21:44:05] atgo: yeah it might have some scope creep [21:44:10] word [21:44:14] just waiting to hear back from them about the store ID problem [21:44:29] awight: imo rsync_blaster is probably a better way to push a fast change like that [21:45:02] Jeff_Green: Actually, the config file thing is trickier than I was imagining. Don't we have to generate at least some of the content with puppet templates cos it's machine-specific? [21:45:12] Fundraising-Backlog, fundraising-tech-ops: Finish config file centralization for Fundraising systems - https://phabricator.wikimedia.org/T113301#1661263 (awight) NEW [21:45:15] Some notes ^ [21:45:22] question up for grabs about the PayPal failmails: would the bad_audit_line fails look like a skipped payment with no error message in the PayPal console? [21:45:47] and cwdent there's no way to take the name info and feed it back into WP before we load the iframe? [21:46:05] awight moving forward yes, the generated file could have datacenter-specific settings [21:46:16] atgo: not that i've found, but i'll ask them [21:46:21] cool, thank you [21:47:07] Jeff_Green: https://phabricator.wikimedia.org/T113301 there's a bookmark for later, so I don't take up your whole evening with this [21:47:26] great [21:48:24] ccogdill MBeat am i understanding correctly that we don't parse the amazon names before the TY? [21:48:39] I don’t think that’s it atgo [21:48:41] Fundraising-Backlog: Should we remove name fields from WMF hosted part of WorldPay ESOP form? - https://phabricator.wikimedia.org/T113303#1661290 (atgo) NEW [21:49:06] Amazon just sends names to us in a weird manner, so if a donor has a middle name or middle initial, it ends up as a part of their last name [21:49:18] MBeat: Good question! We'll actually blow up whether or not the donation was in Civi already, so if it already existed there would be no effect on the Civi record. [21:49:21] right, but it's the same problem where it's just 1 field [21:49:26] the Amazon phab task is about trying to parse the M.I. as a part of the first name rather than last name [21:49:36] I think it’s two fields, but I could be wrong [21:49:44] it's not.. if you go into your amazon accrt [21:49:49] you can see it's just 1 [21:49:51] mine just says "anne" [21:49:54] MBeat: For recurring charges, they may have come in through the IPN listener already [21:50:14] But if not, yeah it will look like a missing payment in Civi, but the PayPal console should show the charge. [21:50:15] Yea, I was in there looking around. Amazon sends it as one feild, and so we have to make decisions about how to split it up. [21:50:28] ah, so I had the wrong impression [21:50:58] (PS1) Ejegg: Import refund data from CSV [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/239994 (https://phabricator.wikimedia.org/T110367) [21:51:03] Right now we split on the first space, which puts M.I. in the last name feild. We could do that differently, but it needs some figuring out and a conversation probably. [21:51:03] I’m pretty sure it’s parsed before the TY though [21:51:08] Fundraising-Backlog: [epic] worldpay 1 hour test in France - https://phabricator.wikimedia.org/T110113#1661301 (atgo) [21:51:09] Fundraising-Backlog: Should we remove name fields from WMF hosted part of WorldPay ESOP form? - https://phabricator.wikimedia.org/T113303#1661302 (atgo) [21:51:43] Jeff_Green: so, we can't actually avoid the long list of ip blocks, at least when it comes to the IPN listener :( [21:51:53] cwdent: I think you should apply the replicant test [21:52:01] ejegg no? [21:52:15] you see a turtle flipped on its back [21:52:20] yeah [21:52:24] my mother had a turtle [21:52:31] thanks awight - these are in Civi but have no error codes for the missing attempts at PayPal, so I’m going to chalk it up to bad_audit_line fails [21:52:42] listener needs to make requests to the notification servers in order to validate each notification [21:52:46] ccogdill MBeat cwdent i made a tracking task for the WP name discussin [21:52:55] thanks! [21:52:58] and the notification servers are on the amazonaws range [21:52:59] ooh sign me up atgo [21:53:00] please comment there so we have a better record (and our non-US friends can participate) [21:53:01] MBeat: I don't quite get it--what's in Civi? The subscription only, or all the charges too w/ none missing? [21:53:02] thanks atgo [21:53:10] MBeat: did :) [21:53:15] Is anything out of order? [21:53:27] the previous contrib in the subscription, awight, from August [21:53:39] Fundraising-Backlog: [epic] Accept credit cards in France via Enhanced Silent Order Post - https://phabricator.wikimedia.org/T110111#1661308 (atgo) [21:53:40] Fundraising-Backlog: [epic] Internal testing with French credit cards - https://phabricator.wikimedia.org/T110112#1661307 (atgo) Open>Resolved [21:53:40] ah okay so we're just missing the payment in Civi [21:53:56] ejegg: can you describe the request/response sequence a bit on the ticket? we'll whitelist what we have to, I just want everyone to understand why [21:54:25] cwdent: i'm also thinking we might be able to resolve some of these things after the 1 hour test - it'd be a bummer to have bad UI for the form, but the biggest issue is actually being able to run the campaign, so if we can prioritize the ciritcal issues (audit, etc) that woudl be good [21:54:29] MBeat: yep, that would be an audit failure. I think the outage window for that is Aug 25-27, and I should be able to get that missing data in shortly. [21:54:49] Jeff_Green: sure, will do [21:55:08] thanks. I'll look at it tomorrow, meanwhile I gotta run! [21:55:11] atgo: does audit mean civi import? [21:55:15] Fundraising Sprint Tom Waits, Fundraising-Backlog, Unplanned-Sprint-Work: Backfill from Aug 25-Sept 17 PayPal audit outage - https://phabricator.wikimedia.org/T112920#1661311 (awight) [21:55:16] Fundraising Sprint Tom Waits, Fundraising-Backlog, Unplanned-Sprint-Work: Parse PayPal audit data from Aug 25-Aug 27 + Sep 16 gaps - https://phabricator.wikimedia.org/T112989#1661309 (awight) Resolved>Open Sorry, having weird second thoughts that I didn't actually do this yet. [21:55:25] cwdent: yeah, more or less. i [21:55:29] uh [21:55:33] https://phabricator.wikimedia.org/T112807 [21:55:34] thanks awight. so these are not in Civi, but at PayPal there’s no attempt even - usu we see a ‘failed, will retry’ message [21:55:40] whoa! [21:55:52] I would have expected that PayPal would have the charge as usual. [21:56:06] This could be part of their automated system to not tell us about money we don't know we have [21:56:34] atgo: ok yeah, that stuff is working after all! ccogdill was able to find the payments. so as long as nothing changes with the merchant ID stuff [21:56:49] cwdent: we shold confirm that _all_ the ways that the data comes in work :) [21:57:44] atgo: definitely, i'm still a little foggy on some of it [21:57:52] fair enough [21:58:17] i know of 2 queue jobs that get scheduled at the end and i think they import the contrib into civi? [21:58:31] and afaik we don't need a listener for this one [21:59:09] what other things need to be checked? [21:59:22] cwdent: that's a great question for tech :) [21:59:49] so I don’t know if this is helpful, but I didn’t have to do any extra work to find your WP payment in civi, atgo. PPena may have made it difficult for tech to double check because she’s using a reference number that we don’t keep in Civi to talk about these payments [21:59:56] I wasn’t sure if that was causing confusion or not [22:00:10] awight: do you know of any other channels? [22:01:02] cwdent: Worldpay only gets donations through DonationInterface and the nightly audit job, no real-time component. [22:01:35] Fundraising-Backlog: Change "continue" button for credit card logos, suppress logos from within iframe - https://phabricator.wikimedia.org/T113304#1661321 (atgo) NEW [22:02:59] atgo: no task for the field alignment, unless there's one for the general mustache form... ejegg? [22:03:10] i'll make one if there isn't! [22:03:17] ejegg: you have 1 minute to respond :P [22:03:20] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661340 (XenoRyet) Great, sounds like everything... [22:03:24] cwdent: nope, I don't think there is [22:03:26] awight: nightly audit job... [22:03:28] atgo^^ [22:03:30] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661341 (XenoRyet) Open>Resolved [22:03:34] cool, on it [22:05:14] cwdent: See SmashPig/PaymentsProviders/WorldPay/Audit and crm/modules/wmf_audit/worldpay/ [22:05:37] ejegg cwdent is this the same as why it looks weird on the GC US form? [22:05:50] And just for explicitness's sake, the audit job is very different from Civi imports [22:06:58] Fundraising-Backlog: Style fields more cleanly for when donor enters form without currency or amount - https://phabricator.wikimedia.org/T113306#1661342 (atgo) NEW [22:07:22] ejegg cwdent - https://phabricator.wikimedia.org/T113306 if this is related to Mustache feel free to add it. it's a low priority since most donors won't see this unless they've done something weird [22:07:42] atgo: Wait, is this about the editable amount field? [22:08:20] oh yeah, that is a bit funny looking [22:08:27] awight: yeah, but just about the alignment [22:08:32] with the currency field [22:08:39] I though I just copied that from the GC form when I made the mustache template [22:08:47] *thought [22:09:26] but more broadly i think they'll come from the banner w/ an amount in the url so nobody will see it like atgo said [22:09:29] ejegg cwdent awight - i'm not really worried about it since donors, you know, won't ever really see it [22:09:41] just want to have a note that it should be fixed if we have a bunch of UI time someday [22:11:26] yah, I guess you do see it if you come in from 'ways to give' [22:14:40] awight / cwdent / XenoRyet : I made a thing to import refunds from CSV (https://gerrit.wikimedia.org/r/239994). Hoping to use it to resolve https://phabricator.wikimedia.org/T110367 [22:15:06] awesome! [22:15:34] atgo: Hey, does the creative team know about this editable amount? [22:15:54] I was under the impression that it was sort of skunkworks and we haven't had a requirements or design conversation yet. [22:16:13] cwdent: what part of this doc did you already address? https://docs.google.com/presentation/d/1Lbr_WmYxlyDDUfP-O0HRkxPF37AEB65S8zllIXtHjgE/edit#slide=id.p [22:16:17] I love the idea, but was thinking maybe we should hide it until it's a little more polished? [22:16:17] atgo: ^ [22:16:20] awight: why did we build it...? [22:16:34] atgo: Cos tech is incredibly annoyed at the way the amounts are hardcoded ;) [22:16:42] awight: it's the only way to enter an amount from 'ways to give' [22:16:47] aha [22:16:47] yeah, let's hide it until there's a user reason to have it [22:16:50] ohh [22:16:59] that makes sense [22:17:04] What do we do for other gateways from WTG? [22:17:15] i made a task, so we can look into it later. it's no big deal with so little traffic for now [22:17:18] I thought we would link to set amounts or something [22:17:19] they've got editable versions too [22:17:22] wat [22:17:27] Sorry, I've been under a rock [22:17:35] cwdent: was it the name fields? [22:17:50] unrelated: does anyone here know how to ask OTRS a question? as in how to ask if we've heard feedback about something? [22:17:53] also, if your amount is out of bounds, we show the editable version [22:18:02] dstrine: https://gerrit.wikimedia.org/r/#/c/239925/ [22:18:17] dstrine: except there's now the open question of how to parse the name field apart [22:18:30] so i don't think i can say it's been addressed yet [22:18:38] * awight mutters that we should have reusable code for that ^ [22:18:51] yeah, MW::parsename [22:19:10] atgo there’s a mailing list [22:19:14] cwdent: I like MBeat's idea in https://phabricator.wikimedia.org/T86720 [22:19:15] MBeat and I are on it [22:19:16] it's inevitably got to be a chain of regex [22:19:55] you can also ping Maggie Dennis atgo, she works with OTRS regularly and is always helpful - though maybe she has less time now that she’s Philippe [22:19:56] ah ha [22:20:11] ejegg: it just seems like there are going to be infinite regional variants [22:20:25] yah, i guess... [22:20:40] so something that takes care of the top 90% [22:21:38] i bet there are people typing their national titles and stuff like that [22:22:10] heh, this guy's trying to cover them all: https://github.com/joshfraser/PHP-Name-Parser [22:22:35] cwdent dstrine - there's a task for figuring out the name thing: https://phabricator.wikimedia.org/T113303 [22:22:51] ejegg: nice [22:22:52] My dream is that we move towards a single name field, too. What are the obstacles preventing that? [22:23:25] trying to be folksy with people in email greetings? [22:23:35] That can happen in the TY templating [22:23:48] we don’t only email from Civi though [22:23:51] :D [22:24:00] Though, I guess we do want to leave Donor Services free to edit people's preferred address [22:29:14] awight: cwdent: ... atgo: made a few tasks from megan's slide. https://phabricator.wikimedia.org/T113303 https://phabricator.wikimedia.org/T113303 https://phabricator.wikimedia.org/T113304 [22:29:22] non of them are in the current sprint [22:29:40] I don't think they need to interrupt anything else [22:29:58] atgo: would you say that the tasks in the current sprint are higher priority? [22:30:06] another lib attempting the same: https://github.com/jasonpriem/HumanNameParser.php [22:30:27] thanks dstrine [22:31:13] but I think the name field is worth discussing and probably needs to be in the next sprint [22:31:56] dstrine: i'm not sure what this one is: https://phabricator.wikimedia.org/T112946 [22:32:49] i'm not sure that it's important? but if it is to keep things working better for fr-online, sounds good [22:37:21] dstrine: we already have the name one in the current sprint for amazon [22:37:24] atgo: cwdent listed it as a requirement for the 1 hour test. See his email thread on 9/17 [22:37:50] yeah i'm just not sure where it came from dstrine - so if the-wub needs it to make banners better/easier, then let's do it before the 1-hour test, otherwise unclear? [22:39:04] atgo: cwdent: I can't make that call^ it sounded necessary [22:40:15] The "default to credit card" thing may be imaginary. I think the real bug is, we do bad things when a payment_method isn't specified in the URL. [22:40:32] It's gonna be really rare, but we should do something inoffensive and helpful when it does happen. [22:40:46] Currently, we dump people into a hole where they are probably unable to donate. [22:40:48] awight, atgo: wp asks me to remove the IsVerify field from the XML for the authorize attempt [22:41:25] and that's the same as default to cc? [22:41:45] seems a little kludgy...i had to comment it out of the structure and readd it if !ESOP [22:42:02] atgo: um, maybe not. Maybe we should redirect to Ways to Give. [22:42:19] It needs speccing. [22:42:31] i'm still not understanding the issue at hand [22:43:33] ohnoes! site's down! [22:43:54] oh hmm maybe not now [22:44:02] must be intermittent [22:44:13] atgo: If you send out a Special:Worldpay etc. URL but with no payment_method, bad things happen. [22:44:24] However, we would never do that in the real world. [22:44:40] AndyRussG: what is intermittent? [22:44:51] Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising-Backlog, Patch-For-Review: Implement Amazon payment notification listener - https://phabricator.wikimedia.org/T109649#1661450 (Ejegg) [22:44:58] dstrine: getting error pages on en.wikipedia.org [22:45:23] ok awight cwdent if there's a tech reason sounds good... just not sure about prioritizing because i don't understand how this happens and what the consequences are [22:45:57] atgo: sorry, my 2¢ is that we don't need to care at the moment [22:45:59] AndyRussG: emails probably related... [22:46:44] cwdent: which emails? [22:47:27] AndyRussG: just saw some to ops [22:48:17] cwdent: ah right! yeah I nearly forgot about that list ;p [22:50:00] Fundraising Sprint Tom Waits, Fundraising-Backlog: Parse Amazon last name data differently (or: review the way we receive Amazon name data) - https://phabricator.wikimedia.org/T86720#1661459 (Ejegg) Two PHP libs that attempt to solve this: https://github.com/joshfraser/PHP-Name-Parser https://github.com/j... [22:54:30] MBeat: thanks again for the status 600 report, it just led to uncovering a deeper issue! [22:54:40] cool [22:54:47] How long has the uptick been going on? [22:55:19] Fundraising-Backlog: Sprint U (9/30-10/14) goal: - https://phabricator.wikimedia.org/T109908#1661473 (DStrine) [22:55:40] Fundraising-Backlog: Sprint V (10/14 - 10/28) goal: - https://phabricator.wikimedia.org/T112180#1661474 (DStrine) [22:56:07] Fundraising-Backlog: Sprint W (10/28 - 11/11) goal: - https://phabricator.wikimedia.org/T112816#1661477 (DStrine) [23:03:17] atgo: private task, please... sigh [23:06:37] (PS1) Cdentinger: Only send IsVerified if !ESOP [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240007 [23:06:58] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661492 (LeanneS) @XenoRyet we just have one fol... [23:07:08] awight: if you have a chance to look, anything seem string to you? ^^ [23:07:48] That works! [23:07:49] their last email just confused me more [23:08:00] but we may as well try this [23:08:03] (CR) Awight: [C: 2] Only send IsVerified if !ESOP [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240007 (owner: Cdentinger) [23:08:18] sigh, s/string/strange/ [23:08:32] * cwdent all over th' place [23:08:34] (Merged) jenkins-bot: Only send IsVerified if !ESOP [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240007 (owner: Cdentinger) [23:11:48] Fundraising-Backlog, fundraising-tech-ops: paypal gateway log messages should be bucketed - https://phabricator.wikimedia.org/T113308#1661494 (awight) NEW [23:12:05] ejegg: how would you feel about a deploy? notice a few new things from you in there... [23:12:10] DI deploy that is [23:12:26] Fundraising-Backlog, fundraising-tech-ops: payments-payflow* log files can be deprecated - https://phabricator.wikimedia.org/T113309#1661501 (awight) NEW [23:12:45] cwdent: just checking, you have an hour to babysit? [23:12:50] cwdent: I deployed this morning - think that was all I had [23:14:36] awight: ugh probably not. my neighbor has been nagging me about a "programming problem" for days and won't back down. some windows thing i'm going to have NO idea about but waste an evening on [23:14:53] maybe hold off on the deploy, then? [23:15:30] yeah will do [23:15:42] That sounds hairy, I hope it's a fun chat at least [23:16:01] oh it won't be [23:16:02] I just did up a friend's Window$ laptop, pretty proud of my work. [23:16:07] KMSPico ftw [23:16:42] a $100 ssd bumped that VAIO up from doorstop status to sort of better than a macbook pro [23:16:51] he is a tech writer and IEEE member, he described a problem anyone with a rudimentary software problem should be able to solve [23:17:00] :p [23:17:08] which means it's effed up somehow and i'm going to suffer [23:17:08] he's a graffiti guru [23:18:08] I did manage to get most of his portfolio off the old hdd though at great human cost. About 1000 bad sectors, so I wrote this script to kill the rsync, add pasted filenames to the excludefile, and restart the job [23:19:53] gnarly [23:21:24] It's horrifying that this was faster than waiting for the disk to map its bad sector [23:28:18] Quick civi import question: If they don't want to send out automated email recipts, does setting no_thank_you or do_not_email true accomplish that? [23:29:38] set no_thank_you for that. The value can be anything non-null [23:29:49] Cool, thanks [23:29:54] At the moment, it's a dumping ground for free-form reasons [23:33:11] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661586 (XenoRyet) @LeanneS That's fairly straig... [23:36:55] Fundraising Sprint Tom Waits, Fundraising-Backlog, Research-and-Data, Unplanned-Sprint-Work: Review banner history log data and confirm that it satisfies use cases - https://phabricator.wikimedia.org/T112986#1661603 (ellery) @awight Having pageviews would make the data quite a bit more interesting.... [23:41:42] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661625 (atgo) Seems like a good @dstrine questi... [23:49:10] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661687 (XenoRyet) Good call on bringing in @DSt... [23:51:00] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661696 (DStrine) That change feels like it is p... [23:51:47] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661701 (XenoRyet) Cool, I'll handle it that way... [23:52:09] Fundraising Sprint Queen, Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint The Pogues, and 3 others: Set up import for Major Gifts events payment/invitation tool - https://phabricator.wikimedia.org/T101191#1661707 (XenoRyet) Resolved>Open