[00:00:00] Sorry lost my connection 4 a bit [00:00:19] ^ didn't see any messages over the last 15 min (if there were any) [00:01:01] :-) [00:30:21] ejegg|away: cwdent|afk: if you're back this evening and have a sec, couldja maybe give me a ping? thx! [00:33:53] (CR) AndyRussG: Migrate to a terser cookie name and format (2 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [00:52:30] (CR) AndyRussG: Migrate to a terser cookie name and format (4 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [01:42:13] (CR) AndyRussG: Migrate to a terser cookie name and format (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [01:51:43] (CR) Eileen: "I think it makes sense to merge these now - there should be no impact on working code & this is documentation of what I am doing - but my " (2 comments) [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/235359 (https://phabricator.wikimedia.org/T99836) (owner: Eileen) [01:59:13] (PS1) Eileen: CRM-17156 do not log user id in unit test context [wikimedia/fundraising/crm/civicrm] (4.6.9-deploy) - https://gerrit.wikimedia.org/r/240953 [02:00:06] (Abandoned) Eileen: CRM-17156 do not log user id in unit test context [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/240320 (owner: Eileen) [02:00:56] (Abandoned) Eileen: (wmf) CRM-17156 log user id against queries fix [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/240026 (owner: Eileen) [02:01:53] (CR) Eileen: "If this looks terribly familiar it's because I did it against the wrong branch before..." [wikimedia/fundraising/crm/civicrm] (4.6.9-deploy) - https://gerrit.wikimedia.org/r/240953 (owner: Eileen) [02:13:36] hi AndyRussG ! [02:13:56] ejegg: hi! [02:14:14] (ejegg: one sec) [02:24:49] ejegg: hi again! sorry for bothering you so late [02:25:07] AndyRussG: no problem! Just chatting with Krinkle over in -ops [02:25:41] ejegg: ah right! yeah I saw he had a question about DI [02:26:38] ejegg: if you still have a few drops of energy I was just going to ask your opinion about a request from ori to quickly merge and deploy a cookie-minifying patch... er, I think he was thinking of tonight [02:27:14] My first reaction is that it needs more testing on the beta cluster first, and that we'd also need to give a heads-up to FR-non-tech [02:28:19] Since he seemed kinda insistent, I said in any case that we should talk to others on fr-tech, that a flash CN deploy was not something that I was comfortable saying OK about without talking to the team [02:28:30] Did he say why it's so urgent? [02:28:39] ejegg: no [02:28:43] hmm [02:29:16] I suggested that we try to get it out at the same time we deploy other upcoming CN stuff, which hopefully would be Monday [02:29:40] However he said that he'd rather have the two deploys separate, I guess since if something goes wrong you can more easily know where that came from [02:30:18] However frankly it seems like a flash deploy of one ASAP is more risky [02:30:43] Yea really I wasn't sure what to say [02:31:27] In any case, I've made some minor, mostly-not-really-blocking comments on most recent version of the patch https://gerrit.wikimedia.org/r/#/c/235979/ [02:32:59] Hmm, maybe there's something to that separate deploys idea [02:33:13] less to test [02:33:56] but yeah, how long has the minified patch been on beta? [02:34:11] oh, it's not even merged to master yet? [02:34:37] Yeah... let's not deploy everywhere without at least some beta cluster testing [02:34:46] ejegg: nope not yet merged [02:35:56] ejegg: yeah OK I'm glad at least I'm not crazy 8p [02:36:35] ejegg: sorry I didn't quite know how to say "no" forecfully in that context... And really this is taking time from BH, which is more important also [02:38:10] Those comments look valid, but if you know exactly how you want to fix 'em we could definitely merge the existing one as-is and submit a fix patch [02:38:24] just to get the ball rolling on the beta cluster [02:39:08] Biggest issue seems like the delimiters being valid chars in campaign names [02:39:25] ejegg: hmmm maybe that's a the right approach [02:40:21] got a good scheme for campaign name escaping? [02:40:33] backslashes? [02:40:45] https://www.mediawiki.org/wiki/Manual:$wgLegalTitleChars [02:41:30] I didn't see any campaigns with those characters in the names. For me the biggest issue is the commit message, 'cause you can't change that in retrospect, AFIK, at least not with "normal" means [02:41:57] oh, what was the issue there? [02:43:38] Also commented in Gerrit. The commit message is just out of date WRT how the patch works now [02:45:54] Apparently it smoke tests but I haven't combed over every bit of functionality (for example, the new date format) [02:46:24] still worth testing on beta. [02:46:45] yeah fer sure :) [02:47:10] nice, even the numbers in js are minified: 14e8 [02:48:59] yeah! it's a neat idea [03:11:06] ejegg: on another note, Krinkle said he's OK w/ signing off on this patch after it's merged (he's traveling tomorrow) and awight already gave a +1 just to wait for approval from Krinkle, so if you feel like weighing in w/ your "poderosa" (Mexican slang for signature) https://gerrit.wikimedia.org/r/#/c/240170/ [03:12:03] looking [03:12:16] Many thanks! [03:23:59] hmm, that doIdleWork function could certainly get pushed up into some common toolkit [03:24:34] well, I guess this whole localstorage expiry thing would be nice to have all around [03:25:08] ejegg: yeah! The general version of doIdleWork is already being worked on. That's the phab task referenced in the code just above [03:25:20] ah, missed that [03:25:34] heh np it's not obvious [03:28:36] the code looks good at first glance, but I want to read more of the comments and test some locally before I weigh in with a +N [03:28:45] might have to do that in the morning [03:30:34] ejegg: k of course np! [03:30:53] on that note, have a good night AndyRussG ! [03:31:04] likewise ejegg thanks so much!!!!!!! :D [03:35:41] (CR) AndyRussG: "Great, thanks!! :) So, just a few comments still (see previous inlines), not all of which are blockers before merging. Seems to smoke test" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [04:05:23] (PS1) Eileen: T99836 remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240959 [04:07:58] (PS1) AndyRussG: BannerHistoryLogger: always send log ID, never send twice [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240960 (https://phabricator.wikimedia.org/T113421) [04:40:48] (Abandoned) Eileen: Anotate CiviCRM patches. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/235164 (owner: Eileen) [04:41:18] (Abandoned) Eileen: T99836 remove apiv2 Entity Tag call [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240925 (owner: Eileen) [04:42:29] (CR) Eileen: "Yep - makes sense" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/235371 (owner: Awight) [04:43:16] (CR) Eileen: [C: 2 V: 2] "tried again - I needed to tick those boxes didn't I" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/235371 (owner: Awight) [04:44:11] (Abandoned) Eileen: T99836 Civi46 update refund test for changed parameter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240927 (owner: Eileen) [04:44:34] (Abandoned) Eileen: T99836 Civi46: adapt to pre-hook standardisation [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240926 (owner: Eileen) [05:06:03] Fundraising Sprint Rowlf the Dog, Fundraising Sprint Snoop (Dogg|Lion), Fundraising Sprint Tom Waits, Fundraising Tech Backlog, and 4 others: Triage what to do with civi-core patches: extensions, upstream, or abandon - https://phabricator.wikimedia.org/T99836#1673576 (Eileenmcnaughton) Status upda... [07:47:01] TCB-Team-Fundraising-Sprint-2015-09-23, TCB-Team: [WMDE-Fundraising] Automate database updates for deployments - https://phabricator.wikimedia.org/T113465#1673669 (WMDE-leszek) a:WMDE-leszek [08:58:06] TCB-Team-Fundraising-Sprint-2015-09-23, TCB-Team: [WMDE-Fundraising] Automate database updates for deployments - https://phabricator.wikimedia.org/T113465#1673807 (WMDE-leszek) Pull request on github: https://github.com/wmde/fundraising-infrastructure/pull/51 [10:40:16] Fundraising Sprint Tom Waits, Fundraising-Backlog, Unplanned-Sprint-Work: Backfill missing associations between PayPal audited donations and contribution_tracking - https://phabricator.wikimedia.org/T113503#1674092 (jrobell) thank you @awight for resolving this, and thanks @pcoombe for backfilling the... [12:58:18] Fundraising-Backlog: [BUG] Donation form stuck with 0 as donation amount; donors can't modify it - https://phabricator.wikimedia.org/T113530#1674333 (MBeat33) The Belgian donor using Chrome (screenshot) also confirmed using a banner. ZD#157202 [13:18:50] Fundraising Sprint Enya, Fundraising Sprint Flaming Lips, Fundraising Sprint Grandmaster Flash, Fundraising Sprint House of Pain, and 8 others: CentralNotice bucket improvements, step 3 bis part 2: moar better cross-browser tests - https://phabricator.wikimedia.org/T86092#1674381 (zeljkofilipin) [14:02:43] (PS13) AndyRussG: KVStore: batched, more resilient implementation of TTL for items [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240170 (https://phabricator.wikimedia.org/T111385) [14:07:06] (PS2) AndyRussG: BannerHistoryLogger: always send log ID, never send twice [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240960 (https://phabricator.wikimedia.org/T113421) [14:07:25] (CR) AndyRussG: "Rebased" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240960 (https://phabricator.wikimedia.org/T113421) (owner: AndyRussG) [15:33:11] (PS2) Ejegg: T99836 remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240959 (owner: Eileen) [15:33:33] hey ejegg, would you be ok with a friday morning DI deploy? [15:33:37] there are just two commits [15:33:38] (CR) Ejegg: [C: 2] "Tidier! Thanks." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240959 (owner: Eileen) [15:33:56] cwdent: heh, trying to make a habit of it, are we? [15:34:05] :( [15:34:20] lemme see what we got [15:35:04] (Merged) jenkins-bot: T99836 remove trailing spaces [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/240959 (owner: Eileen) [15:35:27] in other news, that swat tool config change got merged, so we don't have to worry about ppl messing with our deployment branch any more [15:35:47] ah awesome [15:36:30] you're thinking just the 'reorder cards' and my 'leave overlay on redirect'? [15:36:50] yessir [15:37:08] pretty minor, but i think it's the last thing for the form to look good [15:37:23] well, seems trivial enough [15:37:38] sure, let's do it. +2 incoming [15:37:44] ok, ty [15:37:48] only one for today i promise [15:37:52] (CR) Ejegg: [C: 2] reorder cards and hide iframe card images [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240906 (owner: Cdentinger) [15:37:54] hehe [15:41:53] PPena: just gonna roll the card order change and i think we're pretty good [15:42:15] be back in a few [15:44:46] cwdent yesss... I want to write an interal email (that same thread) asking for the 'state of the project' [15:46:50] (Merged) jenkins-bot: reorder cards and hide iframe card images [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240906 (owner: Cdentinger) [15:49:52] PPena: sounds good! [15:52:22] (PS1) Cdentinger: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/241063 [15:56:51] (CR) Cdentinger: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/241063 (owner: Cdentinger) [15:57:08] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/241063 (owner: Cdentinger) [16:00:41] (PS1) Cdentinger: Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/241064 [16:01:18] Fundraising-Backlog: Hide name fields from WMF portion of form - https://phabricator.wikimedia.org/T113657#1674913 (Ppena) Waiting to hear from WP, but most probably if they have a fix (and are able to receive this info from us versus iframe), it may not be right away. [16:02:11] i'm just going to wait until ejegg|brb or awight is around to take a look at that before i deploy [16:02:19] watering the plants, brb [16:05:02] cwdent|brb: looks good to go! [16:05:13] Fundraising-Backlog, fundraising-tech-ops: Outdated cert on GC iframe? - https://phabricator.wikimedia.org/T112990#1674929 (CCogdill_WMF) GC sent an email today with a heads up that they will be updating their SSL cert (including upgrading to a SHA2 signature) so I don't think this is an issue. Should I... [16:05:19] Fundraising Sprint Tom Waits, Fundraising-Backlog: Style iframe contents more if necessary. (worldpay 1hour test) - https://phabricator.wikimedia.org/T112809#1674933 (Ppena) I think we could try to get the current CONTINUE button to say DONATE, just like we did with Amazon? [16:11:02] (CR) Ejegg: [C: 2 V: 2] Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/241064 (owner: Cdentinger) [16:15:10] thanks ejegg! [16:22:36] !log updated payments from dc78ff5157b59a8f475dc86194a1059c2d6b2fad to 3b0915a51a0fd567bdf22f3d4e17548a83e735d8 [16:22:41] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [16:23:51] ejegg: did you make a bike repair stand? [16:24:07] cwdent: most of one! I'm going back next week to finish it [16:24:34] sweet [16:24:36] I wanted it to be telescoping, so I needed a piece of stock that fit closely over the other part of the stem [16:24:51] had to add that to their weekly order [16:25:08] ah yeah, then do like a seatpost clamp type thing? [16:25:31] I was just thinking drill holes and put a pin through [16:25:46] or that [16:27:37] (CR) Ejegg: [C: 2] "This code looks great, and works in local testing! It would be great to see it promoted to core utils. It's a shame it only has one key " [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240170 (https://phabricator.wikimedia.org/T111385) (owner: AndyRussG) [16:27:52] i commuted by bike for like 10 years, used to build a lot of em too [16:28:13] nice. i have yet to build one [16:28:30] make... repair stand? awesome [16:28:32] i never actually built a frame though i have a friend who did several [16:28:50] have been collecting tools for a while, and can take all the bits off and get 'em back on [16:29:01] now that i never have to go anywhere i mostly just tool around on my '67 stingray: https://pbs.twimg.com/media/CBDmmgAUkAAA16V.jpg [16:29:02] silly thing I learned the hard way--yeah only clamp to the seat post and never the frame, cos denting [16:29:15] sick [16:29:44] nice banana seat [16:29:51] i rode fixed gears for a long time and then got into cyclocross [16:30:19] now my bikes gather dust :( [16:30:31] hopefully less dust once the shed is finished [16:30:38] at least less pollen [16:30:44] heh [16:33:30] ejegg: anyway i think the deploy looks good [16:34:40] great! [16:35:04] ejegg: thanks much!!! [16:35:18] rrrg gotta errand bassoon! [16:40:43] another Friday deployment? [16:40:51] * awight shakes a rain stick [16:41:30] awight: only one i promise [16:42:11] wassup? [16:42:28] Sorry I missed the action [16:43:57] awight: minor deck chair rearrangement [16:44:32] js tweak to leave the overlay up when about to redirect, WP css fix and card order swap [16:47:00] awight: yeah it was (hopefully) the last tweak to make WP testable [16:49:46] ookay, sorry to be a spoilsport [16:50:16] eh you're right [16:50:18] No icebergs in view [16:50:59] awight: i got the full scoop from tyler about what exactly it was about the deployment branch that goofed up swat [16:51:34] cool. Sorry I wasn't communicating that well [16:51:47] you probably are aware but i found it interesting, if they track a branch by name they don't cut the branch with the train deploy, they pull the latest state every time [16:51:54] He confirmed that we've been doing that to people for all of time, eh? [16:52:06] That's gotta be a bug [16:52:13] Don't make any sense [16:52:30] well i think it's so you can control the release cycle if you wanted it to be less frequent than weekly [16:52:49] Krinkle said yeah, it's been happening forever but he just bottled up his annoyance [16:53:30] yeah, tyler said errant commits show up pretty often in swat but not everyone will notice [16:53:49] also automatic submodule pointer bumps in core is relatively recent [16:54:35] Fundraising-Backlog: [epic] worldpay 1 hour test in France - https://phabricator.wikimedia.org/T110113#1675263 (cwdent) [16:54:36] Fundraising Sprint Tom Waits, Fundraising-Backlog, Unplanned-Sprint-Work, Patch-For-Review: Change "continue" button for credit card logos, suppress logos from within iframe - https://phabricator.wikimedia.org/T113304#1675260 (cwdent) Open>Resolved a:cwdent deployed [16:54:58] * Romaine waves to awight [16:55:02] anyway it was interesting to me to hear about that git workflow [16:56:46] cwdent: That makes a little bit of sense [16:57:09] but yeah, the solution is definitely to just put DI back on master for main cluster deployment [16:57:23] Romaine: hey! lmk if that snippet does the job [16:59:04] awight: yep, then it'll get branches cut and interim commits won't sneak into swat [16:59:12] (CR) Ejegg: "Looks functional, but I'm not sure if we need doIdleWork here." (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240960 (https://phabricator.wikimedia.org/T113421) (owner: AndyRussG) [17:00:00] So, my hosting provider vanished my mail node and web boxes this week. Like they never existed [17:00:21] not ludd.net? [17:01:33] luckily that one's chilling on another provider [17:01:38] ohgod [17:01:43] lovely [17:02:01] At least last time they were up front about SAN failure [17:02:06] The requested URL /~adamw was not found on this server. [17:02:11] hehe [17:02:15] yeah I got issues [17:04:39] pope gave a shout out to dorothy day and catholic worker yesterday [17:05:26] do you think getting boehner to resign was an amazing troll by the pope? [17:05:43] wait, what? [17:05:46] oh damn [17:05:50] he's a covert operative for the democrats [17:12:47] Anyone think it was funny when Boehner feigned to spearhead resistance to the imperial presidency? [17:13:05] awight: ccogdill said the data in civi looked good, are there other steps i should take to verify https://phabricator.wikimedia.org/T112807 ? [17:13:33] cwdent: You could check the source_type field in wmf_contribution_extra [17:13:46] That's how u know if a donation came in through the frontend, listener, or audit [17:15:28] that table joins on civicrm_contribution.id = wmf_contribution_extra.entity_id [17:15:50] cool, i'll take a look there [17:16:12] You could also peek at the audit job logs, and make sure you see some recent transaction IDs [17:29:48] Fundraising-Backlog: [epic] worldpay 1 hour test in France - https://phabricator.wikimedia.org/T110113#1675451 (jrobell) I have a few suggestions on how we can improve the French WP form. #1. Changing the second 'CONTINUER' button for 'CONFIRMER' , Continue gives you the impression that you will be able... [17:30:36] d'oh: wp audit parser build failed [17:30:48] oh wait, drush command missing? [17:31:03] dang, did I screw that up in my civi deploy yesterday? [17:31:05] looking [17:32:05] ejegg: I just noticed the job has been disabled for 4 mo [17:32:15] oh, huh [17:32:18] That's what happened when I kicked it [17:32:36] cwdent and I were chatting a little in the priv chan [17:38:09] so much fail [17:58:30] Fundraising Sprint Tom Waits, Fundraising-Backlog: Update WMF logo(s) in Amazon config - https://phabricator.wikimedia.org/T113563#1675593 (Ejegg) a:Ejegg [18:07:50] * AndyRussG waves [18:09:20] awight: Probably u missed the backscroll yesterday, Krinkle said he's happy to check out the TTL patch post-merge, just before we deploy. (He's traveling today.) [18:09:50] But we're not really deploying today, right? [18:09:54] friday and all... [18:11:10] I'm all for merging mostly stable chunks into master. [18:12:19] awight: ah no, the plan is Monday SWAT [18:12:31] cool! [18:12:45] awight: the TTL bit has merged :) [18:12:50] That's nice to have more hands doing the deploy bit [18:12:58] so we can focus on testing [18:13:02] yeah! [18:13:06] AndyRussG: oh, woot! [18:13:13] yippers! [18:14:09] Anything I can help with? [18:15:48] My kid is in a funny phase, I keep finding things lined up: 4 chairs in a square, 4 Chapulin Colorado dvds in a line... [18:16:17] She likes to make shrines of family photos, with a few in a line and then a border of blocks [18:16:35] I think adults would probably be just the same, without jobs to keep us busy [18:16:58] http://www.sciencemusings.com/blog/uploaded_images/bowerbird-756720.jpg [18:18:25] Fundraising Sprint Tom Waits, Fundraising-Backlog: Update WMF logo(s) in Amazon config - https://phabricator.wikimedia.org/T113563#1675653 (Ejegg) OK, got the little logo on the login page looking good. I didn't find any banner format logos, so I just moved the text out from under the circular bit to mak... [18:19:48] AndyRussG: yay TTL [18:20:21] dstrine: indeed! :) I'm gonna move the cards in a sec!! Just commenting on Gerrit on the other patch... [18:22:23] cool cool https://media.giphy.com/media/Ij87d6qbAhdJe/giphy.gif [18:24:12] Fundraising Sprint Tom Waits, Fundraising-Backlog: Update WMF logo(s) in Amazon config - https://phabricator.wikimedia.org/T113563#1675704 (Ejegg) @PPena, @MeganHernandez_WMF, please comment or close this task if the logos look good to you [18:26:00] Fundraising Sprint Tom Waits, Fundraising-Backlog: Fix Worldpay audit parsing - https://phabricator.wikimedia.org/T113782#1675724 (awight) NEW [18:28:02] (PS1) Awight: update SmashPig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241108 [18:28:04] (PS1) Awight: Don't assume configured directories end in a slash [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241109 [18:28:39] (CR) jenkins-bot: [V: -1] update SmashPig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241108 (owner: Awight) [18:28:57] (CR) jenkins-bot: [V: -1] Don't assume configured directories end in a slash [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241109 (owner: Awight) [18:29:17] awight: I thought audit parser stuff blocked "campaign ready" not 1 hour test? I saw you added this to the 1 hour test: https://phabricator.wikimedia.org/T113782 [18:29:56] (CR) AndyRussG: BannerHistoryLogger: always send log ID, never send twice (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/240960 (https://phabricator.wikimedia.org/T113421) (owner: AndyRussG) [18:31:03] there are going to be a bunch of bugs and changes that likely come out of the 1 hour test, I just want to be sure were prioritizing stuff right and not waiting too long to do the 1 hour test [18:31:06] dstrine: It blocks the parent task, but yeah we can do the 1-hour test w/o audit parsing [18:31:21] we should relink T112807 to block the campaign [18:32:34] awight: just updated smash-pig on packagist. dev-master is now at bf302444eae8 [18:32:39] so is it ok if I move both T113782 and T112807 to block campaign ready? [18:33:15] awight: ^ ejegg has already been looking at this, but if u wanna also poke about, I'm sure it's OK! Another fun thing to do would be to jiggle the now-merged TTL on the beta cluster, maybe throw a few different browsers at it that u have lying around. U can do use the bannerHistoryLogRate=1 URL param to force EL and tail -f on the eventlogging file on the beta cluster to check that the log gets through and looks [18:33:15] as it should :) [18:33:27] (...if u feel like doing any of that!) [18:33:58] ejegg: oh thx, I just composer.locked to a vendor tree-ish rather than the SP master [18:34:09] aha [18:34:32] (Abandoned) Awight: update SmashPig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241108 (owner: Awight) [18:34:40] awight: yeah I love that kid phase! Ceci did that a lot when she was small, too [18:35:43] aaahhh awight I got your last comment. I miss-read T112807 [18:35:57] I will move T112807 [18:36:13] dstrine: Good point though, I don't need to go too crazy fixing this audit thing [18:36:28] Fundraising-Backlog: [epic] Worldpay campaign ready in France - https://phabricator.wikimedia.org/T110114#1675768 (DStrine) [18:36:29] Fundraising Sprint Tom Waits, Fundraising-Backlog: Review audit parsing, review whether existing stuff is working. (Worldpay 1 hour test) - https://phabricator.wikimedia.org/T112807#1675769 (DStrine) [18:36:46] Fundraising-Backlog: [epic] worldpay 1 hour test in France - https://phabricator.wikimedia.org/T110113#1675770 (DStrine) [18:36:47] Fundraising Sprint Tom Waits, Fundraising-Backlog: Review audit parsing, review whether existing stuff is working. (Worldpay 1 hour test) - https://phabricator.wikimedia.org/T112807#1646663 (DStrine) [18:40:34] cwdent|afk: Things I'm slowly remembering. [18:41:15] There are two audit file streams coming from Worldpay, and a transaction comes through one or the other depending on bank backend used. [18:41:39] aaaaugh [18:41:41] smb://filesrv1/fundraising/Tech/Worldpay/EMIS is for the PISCES RECON garbage [18:41:59] It gets way better than that... [18:42:13] srsly, do they actually do any aggregation? [18:42:59] smb://filesrv1/fundraising/Tech/Worldpay/WP%20WPUS is for the TransDevVer2 files, which seem to have completely changed format though not version number since our code worked [18:43:11] ejegg: I think they take any chance they can get to eff up the data [18:44:45] I'm really hoping we learn from this 4th or so experience to amputate before putting this much work into the shipwreck's paint job [18:45:01] * awight mutters about bitter taste of mixed metaphors [18:45:22] dstrine: What's the new drop-dead date for WP? [18:46:24] I tried to bring up the topic but we didn't decide on one [18:47:18] we were waiting on a release yesterday and I guess more just got released this morning? [18:47:27] Fundraising Sprint Tom Waits, Fundraising-Backlog: Fix Worldpay audit parsing - https://phabricator.wikimedia.org/T113782#1675810 (awight) n.b. I'm going to gloss over the Transaction Detail Version 2 US Report failures for now, hopefully French campaign donations will be coming through the EMIS recon bac... [18:47:53] dstrine: I'm fine playing along until this fiasco is through, but we gotta draw some stark lessons [18:49:06] are there new developments this morning that are making things more complicated? [18:49:32] Pats gave them a deadline of EOD today to get their end straightened out [18:49:33] Just more attrition [18:50:12] We need to build the capability to say, screw this, find a new vendor, and reschedule the campaign. [18:51:34] On a more constructive note, we should decide on internal criteria for when to pull the panic level on the FR campaign [18:51:39] *lever [18:51:43] are there more example today or major issues [18:52:05] It's both. This is a historical snafu that we have to get into again [18:52:44] Long story short, there may be significant work to do to realign the audit parser with things that changed on WP's end, I can't say for certain yet. [18:53:24] But if we have to reopen that, it could be slow and involve more back 'n' forth. [18:53:59] Without speculating too much, I just wanted to drop the news + leave TODOs for our next steps [18:57:49] Fundraising Sprint Tom Waits, Fundraising-Backlog: Update WMF logo(s) in Amazon config - https://phabricator.wikimedia.org/T113563#1675834 (Ppena) Looks good! Closing the task :) [18:58:05] Fundraising-Backlog: [EPIC] Amazon embedded 1-hour test version - https://phabricator.wikimedia.org/T108106#1675837 (Ppena) [18:58:06] Fundraising Sprint Tom Waits, Fundraising-Backlog: Update WMF logo(s) in Amazon config - https://phabricator.wikimedia.org/T113563#1675836 (Ppena) Open>Resolved [19:07:11] awight: and I were just chatting on a hangout. The audit parsing problem is complicating WP and is more work. I'm going to email fr-tech so PPena has a heads up and an easier time reading the situation. this can be hard to follow in long IRC conversations [19:14:35] Fundraising-Backlog, Patch-For-Review: Unbreak WMF-hosted Worldpay workflow - https://phabricator.wikimedia.org/T112665#1675871 (Ppena) adding by Awhight request: Thinking about AMEX processing with merch ID 200639. Trying to think how to send the form country to WP, so they can differentiate processing f... [19:15:00] Fundraising-Backlog: TransDetVer2 audit parsing is broken - https://phabricator.wikimedia.org/T113787#1675872 (awight) NEW [19:17:26] Fundraising-Backlog: Only block French cards on France form. - https://phabricator.wikimedia.org/T113789#1675885 (awight) NEW [19:17:48] PPena: ^ a task in case we decide to go this way [19:20:15] awight: ejegg: cwdent|afk: XenoRyet: anyone got a sec throw about more ideas about missing bits of https://phabricator.wikimedia.org/T112022 ? If not now maybe a bit later today? [19:20:30] AndyRussG: sure! [19:20:36] awight: cool! [19:21:37] So thinking abot different scenarios... Last time we were talking about the case of users w/ out sendBeacon... Maybe taking up where we left off there would help me get an overview of how to work with the rest of users, too? [19:22:13] IIRC, we mentioned the idea of sending the BH log data as a post to the DI redirect-to-payment-processors page, for non-sendBeacon users.. [19:22:23] dstrine: cwdent|afk: I think the EMIS reconciliation actually works. [19:22:31] Will know for sure in a few more. [19:22:50] ^ :) [19:22:58] cool [19:23:50] AndyRussG: yeah it would have to be to donatewiki, which is currently not the redirect page, and doesn't have DonationInterface, only the ContributionTracking extension. So we'd have to add something in there. [19:24:28] ...yeah we could do the actual server-side EL log from donatewiki, which is the final destination, I think? In that case, how would the EL data make it through from the redirect to Paypal or whatever, and back? [19:25:09] There's a very stale feature in ContributionTracking to do almost exactly this, but * stale, * access to the contribution_tracking table from donatewiki seems like it's been broken for a long time anyway, * needs to parse the BH data [19:25:17] Fundraising Sprint Tom Waits, Fundraising-Backlog: Review audit parsing, review whether existing stuff is working. - https://phabricator.wikimedia.org/T112807#1675922 (DStrine) [19:25:33] donatewiki is rarely the final destination, we usually want to go banner -> paymentswiki [19:25:47] So this would be banner -> donatewiki invisible redirect -> paymentswiki [19:25:58] awight: ahhh K [19:26:07] There's another redirect at paymentswiki, unfortunately, but we don't don't need to consider it here [19:26:56] AndyRussG: c.f. ContributionTracking_body.php [19:27:27] ooh that's so bad. nvm, we should just delete that and start fresh. [19:27:30] awight: so for non-sendBeacon, as u said ^ banner -> donatewiki invisible redirect -> paymentswiki, and for sendBeacon-holders, just banner->paymentswiki, right? [19:27:45] yes, that would be ideal! [19:28:36] I think it's valid to finish the pipeline for just sendBeacon-having donors, and then add non-sendBeacon support in the next iteration. What's your preference? [19:29:05] awight: sure! sounds like that'd be the priority, definitely [19:29:05] My thinking is, Megan will know the data is low quality and biased, but at least she'll have something [19:29:07] (CR) Ori.livneh: Migrate to a terser cookie name and format (7 comments) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [19:29:09] k [19:29:35] (PS17) Ori.livneh: Migrate to a terser cookie name and format [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) [19:30:59] (CR) jenkins-bot: [V: -1] Migrate to a terser cookie name and format [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [19:31:47] cwdent|afk: one more setting for testing auditing: $conf['wmf_audit_log_archive_dir'] = [19:31:51] awight: yeah... Anyway, the silent redirect sounds to EL sounds like a great plan, I don't see why we wouldn't be able to pull that off :) [19:32:25] * AndyRussG represses more unfounded optimism [19:32:34] (PS18) Ori.livneh: Migrate to a terser cookie name and format [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) [19:32:47] That sounds a little fishy--just checking, we're not redirecting *to* EL, but making an EL server-side call from something on donatewiki? [19:33:09] (CR) Ori.livneh: Migrate to a terser cookie name and format (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/235979 (https://phabricator.wikimedia.org/T110353) (owner: Ori.livneh) [19:33:21] awight: yeah those are ghost fish... I meant, "in order to log via EL" [19:33:34] cwdent|afk: High five--EMIS reconciliation works, I see meganhernandez's 27819149.0 [19:34:03] ;) AndyRussG thx, I'm a little twitchy today [19:35:27] awight: heh me to! Maybe something in the coffee? [19:35:51] awight: just for my education...EMIS is part of audit parsing? [19:35:52] * AndyRussG checks for coffee-spiking leprechauns [19:35:53] I think it'll work great, too, and I'm sure even a few seconds of redirect churn is fine if someone is already determined to donate. [19:36:23] awight: K one sec lemme pul up our previous chat on the queue processing bit etc [19:36:32] dstrine: yeah, it seems to be the audit file format used for French donations [19:36:42] awight: that's the audit parser? [19:36:52] AndyRussG: it's fr-tech seasonal disaffection, arrived early [19:36:57] cwdent: yep [19:37:08] awesome! sorry i've been away -- what was it? [19:37:13] See that's why we should postpone the code freeze! [19:37:58] cwdent: I'm just gonna disable the borken US audit parser for the moment [19:38:33] AndyRussG: hehe I can't imagine you enjoyed last November's mad dash to deploy the CN changes [19:39:03] awight: dstrine: cwdent: XenoRyet: ejegg: I'm thinking of missing standup today, since I have to pick up my daughter from a birthday party (no school today, it's a "pedagogical day") exactly at 4 (1 PST). I could potentially sit in the car with her and do the call, maybe just a bit late, but I'm not sure it's worth it... [19:39:20] don't do it! [19:40:03] she'll know too much! [19:40:49] AndyRussG: wow I had to google that. Have fun! [19:41:41] (PS1) Awight: Disable Worldpay US auditing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/241125 (https://phabricator.wikimedia.org/T113782) [19:42:18] cwdent: So this should be all we need for now, ^ [19:42:31] plus a little deploy dance to get that out the other end [19:42:55] (CR) Cdentinger: [C: 2] Disable Worldpay US auditing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/241125 (https://phabricator.wikimedia.org/T113782) (owner: Awight) [19:43:08] word [19:43:10] cool, i've never deployed smashpig [19:43:16] and it's friday... [19:43:19] hah [19:43:21] AFTERNOON [19:44:15] awight: so to summarize, DI passes the BH id on to the queue consumer, which I may study and tweak in the wikimedia/fundrasing/crm, correct? [19:44:54] awight: so on that view, what are the two entry points I should be looking at, on the DI side and on the CRM queue side? [19:45:00] (PS1) Awight: update SmashPig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241128 [19:45:11] (CR) jenkins-bot: [V: -1] update SmashPig lib [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241128 (owner: Awight) [19:45:55] I'm off to chauffeur, I'll send my [stand(up]date) via e-mail [19:46:02] also will get any scrollback here [19:47:16] (Merged) jenkins-bot: Disable Worldpay US auditing [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/241125 (https://phabricator.wikimedia.org/T113782) (owner: Awight) [19:49:39] AndyRussG|a-whey: yes. wmf_civicrm_contribution_message_import is the important stuff called during queue consumption. [19:50:10] AndyRussG|a-whey: I would add something parallel to wmf_civicrm_insert_contribution_tracking, or even store the data as a new column in contribution_tracking, I forget whether we made a decision what to do wrt storage. [19:51:32] AndyRussG|a-whey: The DI side is funky, see DonationQueue::buildTransactionMessage and DonationData::populateData [19:56:39] (PS1) Awight: update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/241134 [19:57:13] (PS1) Awight: Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/241135 [19:57:15] (PS1) Awight: update SmashPig lib [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/241136 [19:57:17] (PS1) Awight: update vendor submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/241137 [19:57:35] cwdent: If you have any idea about how to improve the deployment fu, please jump in. I hate life every time [19:58:49] you do essentially the same thing for smashpig? merge and rsync? [19:58:53] (CR) Ejegg: [C: 2] update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/241134 (owner: Awight) [19:59:09] cwdent: the bad part is all this cruft ^^^ [19:59:19] submodule tango [19:59:30] yeah totally... that's the part that stresses me out [19:59:51] i mean...we could script that part [20:00:18] yeah I'd like a makefile that builds the release [20:00:22] i think the update and rsync work pretty well [20:00:25] (Merged) jenkins-bot: update SmashPig lib [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/241134 (owner: Awight) [20:00:31] awight: yeah i like that [20:00:33] me too! [20:00:53] a make file is a good idea [20:00:53] * awight hi-fives unsuspecting Jeff_Green [20:01:08] what did i break? [20:01:26] Jeff_Green: we were just saying that the update+rsync scripts are the only thing that works in our repos [20:01:33] ha [20:01:37] it's because it's perl [20:01:44] yep i really don't think there's a better way to deploy than rsync [20:01:55] Jeff_Green: actually, you might be interested in the borken part: we basically need to do a compilation step for the civicrm and some other repos. [20:01:56] :-) [20:01:58] people massively overcomplicate that shit a lot [20:02:05] (CR) Ejegg: [C: 2] update vendor submodule [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/241137 (owner: Awight) [20:02:42] Jeff_Green: it would be all local, we'd basically be building the release objects then keeping them checked into the deployment branch like we do now [20:02:53] It's just getting too complicated, with vendor/, submodules and shit [20:03:00] i see [20:03:06] so what would be in the repo? [20:03:16] standup is meow [20:03:17] awight: are we not checking composer.lock updates into master on crm? [20:03:19] master would be just code [20:03:23] Jeff_Green: sorry, gtg [20:03:26] ejegg: we are [20:03:45] ha ok, I will just pretend I never heard any of this, too much potential angst for a friday [20:15:29] the connection here is hosed [20:22:26] Jeff_Green: ;) [20:23:04] Fundraising Sprint Tom Waits, Fundraising-Backlog: Fix Worldpay audit parsing - https://phabricator.wikimedia.org/T113782#1676192 (awight) a:awight [20:23:52] (CR) Awight: "They say we can live with the extra garbage for now." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/238636 (https://phabricator.wikimedia.org/T105918) (owner: Awight) [20:35:29] dstrine: Amazon is not "in France", right? [20:35:57] asdasdfsadf [20:35:59] oops [20:36:01] correcting [20:36:49] http://mraw.org/blog/2011/05/24/Git_tip_of_the_day_insteadOf/ [20:37:52] awight: thanks [20:37:56] :) [20:38:02] * dstrine hangs head in shame... [20:43:20] Congress after papal address: http://www.myjewishlearning.com/wp-content/uploads/2015/04/pillar_of_salt.jpg [20:44:41] https://i.imgur.com/amQGBuN.gifv [20:45:54] That's crazy [20:53:46] That's so crazy the word crazy doesn't cut it [20:53:54] Why? [20:54:04] AndyRussG: ADHD [20:55:03] * AndyRussG tries to coax that acronym into something that means a person with a death wish [20:55:43] hehe yep the squirrel suit is definitely on the suicidal end of extreme sports [20:55:52] https://en.wikipedia.org/wiki/Dean_Potter [20:55:53] That can't have ended well [20:59:51] https://www.stilldavid.com/stuff/skitch/base_jumping_mortality_rate_-_Google_Search_2015-09-25_14-57-41.jpg [21:08:45] awight: thanks for the suggestions above, duly noted :) [21:26:53] http://idlewords.com/talks/what_happens_next_will_amaze_you.htm [21:28:31] super long but i love everything that guy writes [21:30:54] cwdent: interesting! [21:31:32] it's the guy who made pinboard [21:50:14] awight: ejegg: cwdent: XenoRyet: if you're not very busy, it'd be fun to get https://gerrit.wikimedia.org/r/#/c/240960/ into a deploy on Monday, if u think that makes sense :) apologies for the bother ;p [21:51:11] oh yeah, that all looked good except the thing I mentioned, and your reply to that made sense [21:53:17] hey all, I see an email with the subject "Request to whitelist a new bank" do we have to do this? [21:54:13] Fundraising-Backlog, Patch-For-Review: TransDetVer2 audit parsing is broken - https://phabricator.wikimedia.org/T113787#1676672 (awight) Here's the beginning of a file we were able to process, `TranDetVer2_246935_8-27-2014_8'28'02 AM.csv` ``` "1","2.0.0","1000709210","8/25/2014","8/25/2014","8/27/2014 8:2... [21:54:21] Fundraising Sprint Tom Waits, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work: Close Amazon order reference asynchronously - https://phabricator.wikimedia.org/T113809#1676674 (Ejegg) NEW a:Ejegg [21:54:31] Fundraising Sprint Tom Waits, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, Unplanned-Sprint-Work: Close Amazon order reference asynchronously - https://phabricator.wikimedia.org/T113809#1676682 (Ejegg) [21:56:30] (PS2) Ejegg: Don't close Amazon order reference in DI [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/240909 (https://phabricator.wikimedia.org/T113809) [21:58:14] Fundraising-Backlog, Easy: TransDetVer2 audit parsing is broken - https://phabricator.wikimedia.org/T113787#1676695 (awight) [21:58:53] (PS2) Awight: Don't assume configured directories end in a slash [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/241109 [22:00:51] I'm just goofing off cleaning up Gerrit, if anyone has more interesting things for me... [22:01:15] ejegg: cool thx! [22:01:18] dstrine-brb: No, I think that's a PPena deal. Something about calling banks to introduce them to other banks? [22:01:50] awight: u bet! wanna work on a hide-reason code shim? [22:02:28] awight: we need a global way to convert codeless reasons in existing cookies to codes in BH [22:02:59] dstrine-brb awight yes, Adam is right. Thats me and Finance. All good there- sorry for the spam [22:04:31] AndyRussG: Cool. We need to finalize the new reason code as part of that, too? [22:05:02] awight: yeah https://www.mediawiki.org/wiki/Extension:CentralNotice/Special:RecordImpression [22:05:12] All of these can be in existing cookies in the wild [22:05:52] awight: I recall you didn't like the solution, but I was thinking of a global reason -> reason-code conversion table inside ext.centralNotice.display.state.js [22:06:35] What should the deprecation period be? 6 mo? Maybe that's an ewulczyn question. [22:06:45] I know it seems completely unencapsulatey, but if you think of it as a i18n message table for computers, it's not so bad [22:06:56] awight: deprecation? for what? [22:07:00] AndyRussG: I think your solution sounds right on, I don't remember my objections at least. [22:07:37] I assume we want to deprecate the old hide cookies, in favor of localstorage and more code-y data type [22:07:56] awight: that it was grossly disrespecting encapsulation, though likely I didn't explain myself well before [22:08:00] hmm, no. localstorage, but with the text version of the key eh? [22:09:36] awight: eventually I'd lke to go to localstorage where available, but right now the issue is just to make all reasons like donate and close that are in cookies create the proper reason code in BH when the hide cookie triggers a banner cancellation [22:10:57] Okay, I'll put that into a task. [22:12:05] Remember, the BH log is minified, so instead of something like status: banner_canceled, reason: close, it should be "s":2.1 [22:13:02] awight: current code in the repository generates hide cookies that include a reason code, but there are scripts that don't. Adapting those scripts is a possible approach, but it won't do anything for existing hide cookies, which can stay around for a while (like donate ones) [22:13:36] Fundraising-Backlog, Easy: TransDetVer2 audit parsing is broken - https://phabricator.wikimedia.org/T113787#1676731 (awight) [22:16:22] AndyRussG: "global" reminds me--should we also do work to add pageview w/o impression counting? [22:17:10] I liked ewulczyn's last idea (at least I think it was him?), that we would keep that count, and attach the number of intervening non-impressions to each BH entry? [22:17:53] awight: I think so, yeah that makes sense [22:18:34] oh hey, is this still a thing? https://phabricator.wikimedia.org/T112947 [22:18:35] awight: I think it's lower priority than darning holes in the hide reasons [22:18:42] ok [22:18:56] ^ I meant re: out-of-campaign impression counting [22:19:12] Re: https://phabricator.wikimedia.org/T112947, I think so [22:19:21] (PS1) Ejegg: Close Amazon order reference when capture completes [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/241247 (https://phabricator.wikimedia.org/T113809) [22:19:35] Ouch--think it's High priority? [22:19:45] Or can ewulczyn get work done without this? [22:19:56] awight: my understanding is that he can, yep [22:20:57] eventually when he starts getting it into mysql I'm thinking maybe we should add some properties on the outer object that would help him query, such as campaign name, since the actual log contents will just go into a json blob in the MySQL [22:22:30] yes! always annoying [22:22:47] I see, so he's just manipulating json in hive for now. [22:22:54] could just dump it into mongo :) [22:23:02] or maybe he's already exporting ipynotebook [22:24:21] From ottomata's comment on the above task, "FYI, although not 100% announced yet, EventLogging data is going into HDFS now, so you should be able to query them using Hive there. " [22:24:23] Fundraising-Backlog: Banner history: encode all hide reasons - https://phabricator.wikimedia.org/T113813#1676789 (awight) NEW a:awight [22:25:51] AndyRussG: If you want to take a look ^ [22:26:14] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Banner history: encode all hide reasons - https://phabricator.wikimedia.org/T113813#1676789 (awight) [22:26:23] awight: +1! [22:27:06] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Banner history: Record count of pageviews where no banner was shown - https://phabricator.wikimedia.org/T113815#1676808 (awight) NEW [22:27:07] thanks! [22:27:36] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Banner history: encode all hide reasons - https://phabricator.wikimedia.org/T113813#1676814 (awight) [22:27:41] (CR) Ejegg: [C: 2] "Nice club, Ogg!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/238636 (https://phabricator.wikimedia.org/T105918) (owner: Awight) [22:27:49] (PS2) Ejegg: Include custom fields in default export [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/238636 (https://phabricator.wikimedia.org/T105918) (owner: Awight) [22:28:20] (PS3) Ejegg: Include custom fields in default export [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/238636 (https://phabricator.wikimedia.org/T105918) (owner: Awight) [22:29:16] hehe [22:29:21] (CR) Ejegg: "wtf? I seem to have rebased this onto itself" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/238636 (https://phabricator.wikimedia.org/T105918) (owner: Awight) [22:29:32] blecch [22:29:35] no idea [22:29:50] well, there's a submit button [22:30:38] voom! [22:31:03] Oh, seems to be merged after all [22:31:12] i just confused the heck out of gerrit [22:31:28] ejegg: har there's no patch left [22:31:48] yeah... [22:32:01] but it made it to civicrm master [22:32:19] i dunno what to do with the phantom gerrit bit [22:32:25] that's worth writing a bug about :( [22:32:27] ejegg: a gerrit-based perpetual motion machine? forever merging a patch and its ancestor (the same patch)? [22:32:44] I had something like that happen, random thing merged itself with no record in Gerrit [22:33:06] Do we need some divine intervention here? [22:33:18] I think that's what intervention looks like [22:33:57] A big, smoking hole, some mystery and an element of injustice [22:34:13] hah [22:34:57] ooh I don't like that the Gerrit author count has plateaued [22:35:12] http://korma.wmflabs.org/browser/ [22:35:44] slumped, more like [22:37:20] cool graphs [22:41:00] Huh I wonder why that is [22:41:42] IRC participation is also down [22:48:33] filed a gerrit bug: https://code.google.com/p/gerrit/issues/detail?id=3584& [22:51:23] awight: ejegg: cwdent: XenoRyet: where does a user return to after a successful donation? how does that work under the hood? [22:51:37] nice ejegg [22:51:52] AndyRussG: thanks page! [22:52:07] cwdent: that's on donatewiki? [22:52:07] AndyRussG: https://wikimediafoundation.org/wiki/Thank_You/en [22:52:41] But it's always payments-wiki that redireects them there [22:52:51] ejegg: ahh K that's the key [22:53:09] What is the payments-wiki page that redirects them? [22:53:10] so we'll know on payments whether to record the BHID in the 'donated' table [22:53:18] Yeppers [22:53:27] AndyRussG: Each of the payment processors has its own Special: page [22:53:37] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, I18n: Sort out PayPal language vs country - https://phabricator.wikimedia.org/T113819#1676860 (awight) NEW [22:53:51] (Abandoned) Awight: WIP: more paypal language fixes [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/63249 (owner: Awight) [22:53:53] Some of them have two, one for initial intake and one for processing after coming back from the payment processor's site [22:54:05] aaah right I remember seeing that in the DI documentation now [22:54:06] (Abandoned) Awight: WIP: More DI core changes to support DonationForm [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64440 (owner: Awight) [22:55:45] (PS16) Awight: WIP: DonationForm [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/61929 [22:55:50] (CR) jenkins-bot: [V: -1] WIP: DonationForm [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/61929 (owner: Awight) [22:56:06] the one I'm looking at is PayPal... It looks like it goes completely onto a PayPal-controlled page after the first redirect from the banner. How can we pass through params or data from the inbound Special page to the outbound one? [22:56:23] awight: .zargo? [22:56:33] (PS3) Awight: WIP: pull all the declarative stuff together [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/65003 [22:56:36] (CR) jenkins-bot: [V: -1] WIP: pull all the declarative stuff together [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/65003 (owner: Awight) [22:56:50] (PS4) Awight: WIP: DonationInterface refactor: pull all the declarative stuff together [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/65003 [22:56:52] AndyRussG: there's a quick bounce off the paymentswiki special page before paypal [22:56:54] (CR) jenkins-bot: [V: -1] WIP: DonationInterface refactor: pull all the declarative stuff together [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/65003 (owner: Awight) [22:56:59] awight: and PPena thanks for the note way above [22:57:00] we just do a server-side redirect [22:57:20] http://imgur.com/CqmFK0N [22:57:28] (PS2) Awight: WIP (FR #807) Recurring contributions keep the original contribution's gateway account name [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/96675 [22:57:30] (CR) jenkins-bot: [V: -1] WIP (FR #807) Recurring contributions keep the original contribution's gateway account name [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/96675 (owner: Awight) [22:57:40] happy little normal distribution [22:58:03] blisfully ignorant noramlcy [22:58:09] ejegg: ArgoUML, a silly corporate diagrams tool [22:58:14] ah, ok [22:58:40] ejegg: right, it goes banner -> DI redirects to PayPal -> successful donation (hopefully) -> DI redirects to TY page -> TY page... right? [22:58:54] But so how do we persist parameters from the first DI redirect to the second? [22:59:09] session [22:59:19] (PS2) Awight: WIP example worldpay audit conf [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/129212 [22:59:29] Or, what does the second visit to a redirecting DI page get from the payment processor that'll let us hook it up the first [22:59:31] Ah OK right [22:59:37] (CR) Awight: [C: -1] "Missing some new config keys." [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/129212 (owner: Awight) [23:00:28] ejegg: is it a Mediawiki session object? key? cookie? something weird, or norm-y? [23:01:02] just php session, no mw wrapper [23:01:09] AndyRussG: ejegg: we don't need to do any "is_donated" tracking on paymentswiki, and couldn't anyway... [23:01:20] That should happen when the message is consumed into Civi [23:02:01] ah ok, we'd just add the BHID to the queue message [23:03:12] AndyRussG: Do we eventually also want to track whether the reader started the donation workflow? [23:03:28] awight: I think so [23:03:53] ooh, are we waiting 'til a future iteration? [23:04:02] yeah we sure can [23:05:17] awight: ejegg: so basically the queue is passed down to civi only following a successful donation, and there is a DonationData or sumthn' object that I can put the BHID in before we go to the payment processor, and it'll still be aorund after the successful donation when it's time to talk to civi? [23:05:51] hey all sorry but I just got to the paypal email: "Major Impact to Batch Processing " is this anything to be worried about? Did it already get discussed? [23:06:06] sftp batch processing...i don't think so? [23:06:14] do we use that? [23:06:30] Fundraising-Backlog: Track whether the reader began a donation workflow, include in banner history log - https://phabricator.wikimedia.org/T113821#1676882 (awight) NEW [23:06:42] AndyRussG: yeah, pretty much [23:07:18] awight: any thoughts on the paypal email? ^ [23:07:44] ejegg: cool! [23:07:45] AndyRussG: I just realized there's another glitch with our ID passing scheme so far--PayPal only creates queue messages in the realtime listener, which wouldn't have access to paymentswiki session data. [23:08:31] awight: hmmm... what does that mean then? Is PP the only PP to do that? [23:09:56] AndyRussG: I think so. Other processors have more of a trickle of donations coming through the listener and auditor. I can pull the numbers easily, if we need to triage. [23:10:39] Fundraising Sprint Tom Waits, Fundraising-Backlog: Review audit parsing, review whether existing stuff is working. - https://phabricator.wikimedia.org/T112807#1676891 (cwdent) Patch that will roll on Monday fixes audit parsing for France. The US files still need some attention (T113787) but that shouldn't... [23:10:59] PayPal's especially creepy to think of a solution for, cos that listener is extreme legacy code that doesn't have access to the nice SmashPig libraries for secondary queues + stuff [23:11:47] i'm outta here for now, have a good weekend! [23:11:55] (PS2) Awight: DO NOT MERGE: (requies PHP 5.4.0) Use traits to consolidate TestingAdapters [extensions/DonationInterface] (php54_test_adapter_collapse) - https://gerrit.wikimedia.org/r/133508 [23:11:57] later cwdent [23:12:03] (CR) Awight: [C: -2] DO NOT MERGE: (requies PHP 5.4.0) Use traits to consolidate TestingAdapters [extensions/DonationInterface] (php54_test_adapter_collapse) - https://gerrit.wikimedia.org/r/133508 (owner: Awight) [23:12:14] (CR) jenkins-bot: [V: -1] DO NOT MERGE: (requies PHP 5.4.0) Use traits to consolidate TestingAdapters [extensions/DonationInterface] (php54_test_adapter_collapse) - https://gerrit.wikimedia.org/r/133508 (owner: Awight) [23:12:29] awight: could add a 'pending' queue message with BHID, then use the completion_message_id thing that AstroPay uses [23:12:38] (PS2) Awight: [WIP] Transparent workaround for access control [extensions/DonationInterface] (php54_test_adapter_collapse) - https://gerrit.wikimedia.org/r/133509 [23:12:47] (Abandoned) Awight: bump submodule to unpatched Civi 4.4 [wikimedia/fundraising/crm] (civi-4.4) - https://gerrit.wikimedia.org/r/133882 (owner: Awight) [23:12:47] cwdent|afk: cya! [23:12:51] (CR) jenkins-bot: [V: -1] [WIP] Transparent workaround for access control [extensions/DonationInterface] (php54_test_adapter_collapse) - https://gerrit.wikimedia.org/r/133509 (owner: Awight) [23:13:01] (Abandoned) Awight: Rewrite wmf_civicrm_get_civi_id to use new apis [wikimedia/fundraising/crm] (civi-4.4) - https://gerrit.wikimedia.org/r/133883 (owner: Awight) [23:13:43] awight: for PP, what info do we have on the first hit on DI that we can associate with the second hit on DI post-Paypal? [23:14:36] contribution tracking id, i think [23:14:52] (Abandoned) Awight: update bootstrap.inc to use new API [wikimedia/fundraising/crm] (civi-4.4) - https://gerrit.wikimedia.org/r/133884 (owner: Awight) [23:15:30] (PS5) Awight: WIP initial port of the dedupe review UI [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/133404 [23:15:36] (CR) Awight: [C: -1] WIP initial port of the dedupe review UI [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/133404 (owner: Awight) [23:15:46] ejegg: izzat it? So I guess we'll want to associate with that? [23:15:54] yeah [23:16:11] BTW, IIRC Ellery will also want one day to link histories w/ donation amounts and processors used [23:16:22] (Abandoned) Awight: WIP stupid things to make offline tests pass [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/147858 (owner: Awight) [23:16:26] (CR) jenkins-bot: [V: -1] WIP initial port of the dedupe review UI [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/133404 (owner: Awight) [23:16:33] (Abandoned) Awight: WIP don't need evil variable magic [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/147859 (owner: Awight) [23:17:31] (PS3) Awight: [WIP] Minor payment_submethod cleanup [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64236 [23:17:34] (CR) jenkins-bot: [V: -1] [WIP] Minor payment_submethod cleanup [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64236 (owner: Awight) [23:17:40] (PS5) Awight: GatewayAdapter::isSupported [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64872 [23:17:43] (CR) jenkins-bot: [V: -1] GatewayAdapter::isSupported [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64872 (owner: Awight) [23:17:51] (PS6) Awight: [WIP] GatewayAdapter::isSupported [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64872 [23:17:55] (CR) jenkins-bot: [V: -1] [WIP] GatewayAdapter::isSupported [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/64872 (owner: Awight) [23:18:05] (Abandoned) Awight: shared rapidhtml error handling in di.form.core [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/91302 (owner: Awight) [23:18:10] (PS3) Awight: fix comment [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/95739 [23:18:21] ejegg: do I need to know anything special then for the BHID->associate w/ donationID -> associate w/ successful PayPal donation path? [23:18:32] (PS3) Awight: [WIP] protect findAccount in case there is no -AccountInfo [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/95873 [23:18:35] (CR) jenkins-bot: [V: -1] [WIP] protect findAccount in case there is no -AccountInfo [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/95873 (owner: Awight) [23:19:18] AndyRussG: Not from the point of view of the banners [23:19:21] (PS4) Awight: bump PayPal "classic api" version [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/127672 [23:19:56] ejegg: but from the point of view of diving into the DI code and making a patch or three for this? [23:20:07] AndyRussG: hmm [23:20:32] you could look at how AstroPay works - it's similar in that we send them off to another site and they may not make it back [23:20:56] I'm going to head out for the day. Have a good weekend [23:20:56] We dump a message in the pending queue with all the info we have, identified by the contribution tracking ID [23:21:17] (PS5) Awight: bump PayPal "classic api" version [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/127672 [23:21:18] dstrine: likwise! cya! [23:21:37] then the IPN listener sends another message when it gets a successful contribution, but it adds a key called 'completion_message_id' [23:21:39] (PS2) Awight: [WIP] opt_out preferences apply to email addresses separately [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/133181 [23:21:41] (CR) jenkins-bot: [V: -1] [WIP] opt_out preferences apply to email addresses separately [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/133181 (owner: Awight) [23:22:04] ejegg: OK! that sounds copyableish [23:22:07] (Abandoned) Awight: possibly more efficient to use the integer index [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/133182 (owner: Awight) [23:22:22] when the queue consumer gets a sparse message with 'completion_message_id', it looks for the matching message in the 'pending' queue to get the rest of the information [23:22:31] (PS2) Awight: Kill "original_" [wikimedia/fundraising/PaymentsListeners] - https://gerrit.wikimedia.org/r/164390 [23:22:42] ejegg: that's SmashPig on the civi side, right? [23:22:42] later dstrine-away [23:23:08] AndyRussG: SmashPig does the listener stuff standalone [23:23:21] on the Civi side, it's our queue2civicrm module [23:24:17] ejegg: huh. So on the queue-consumption side, it's 2 parts? SmashPig consumes queue and passes it to civi, which stores, organizes and provides a UI? [23:24:55] awight: ...spring cleaning? [23:25:40] SmashPig gets an http post straight from the payment processor [23:25:45] and sends a message to civi [23:26:52] Sometimes, the payments processor doesn't reliably redirect donors back to our site after they successfully complete a donation [23:27:15] for those donors, we have to rely on the processor's out-of-band payment notification [23:27:36] which they post to our listener URL (running smashpig) [23:30:31] ejegg: interesting! Yeah I've sometimes had issues returning to a vendor site after paying w/ paypal infact [23:30:56] AstroPay is pretty bad that way too [23:31:35] I think I'm not gonna have a way to test this myself or anything... Hmmm so we have to make sure this is gonna work for all payment processors [23:31:51] Can I get the creds I'd need to smoke test or is that a long process? [23:32:09] ejegg: ^ [23:32:18] awight: ^ [23:34:57] K4-713: ^ [23:35:08] ? [23:35:09] AndyRussG: ejegg: sounds good so far. The issue however is that we'll have to rewrite the PayPal listener in SmashPig. Currently it's legacy, standalone PHP. This is on the long-term roadmap but not a thrilling code slush project... [23:35:45] awight: you can't just add a single key whose value is based on contribution tracking id? [23:37:26] For the PayPal thing, we're currently 95% coming through the listener, and there's no code in DI that I know of to pass data into a side channel. How would a key based on ct.id do anything? We do get ct.id in listener postback data, fwiw... [23:38:00] awight: we'd drop a message into 'pending' queue before redirect, as with astropay [23:38:09] that message's ID is based on ctid [23:38:22] so we can point to it from the smashpig message [23:38:34] ejegg: awight: maybe for the MVP we can just track whether people clicked on donate and made it to DI? [23:39:02] ejegg: ah oh, so we can just drop the BH ID into that pending channel then [23:39:07] yeah [23:39:33] the queue consumer already knows to check pending for extra info when it sees a 'completion_message_id' [23:39:35] AndyRussG: yeah mebbe we should have ewulcyzn prioritize which data he wants first [23:40:51] ejegg: good find! [23:41:03] That deserves some lines in our diagram of doom [23:41:10] @_@ [23:41:13] >_> [23:41:23] ooh, yeah [23:41:32] https://wikitech.wikimedia.org/wiki/File:DonationPipeline_201302.png [23:41:41] it started with AstroPay, but it's creeping into Amazon too [23:42:47] It's great. We should corral the organic growth some day, though--this would be improved by the same "refinement" feature we need for all the other data sources. [23:43:17] hmm [23:45:04] looks almost like that 'Need more data'? box up in the listener quadrant [23:45:39] e.g. audit files will have settlement information that we want to get into Civi even if the basic transaction is already recorded. And we should be comparing audit data with existing data, for accuracy. [23:46:04] yah, that too [23:46:09] like Amazon's fees [23:46:17] ok, I gotta head out [23:46:24] have a great weekend all! [23:47:18] see ya! [23:48:21] (PS2) Awight: Link to the regular wiki editor for banner content [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/181234 [23:48:42] ejegg|away: thanks likewise! cya! [23:48:47] (PS3) Awight: [WIP] Always log the message that would be sent to Civi [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/178981 [23:48:50] (CR) jenkins-bot: [V: -1] [WIP] Always log the message that would be sent to Civi [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/178981 (owner: Awight) [23:49:00] (Abandoned) Awight: Remove unused SystemStatus code [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/178889 (owner: Awight) [23:50:06] (PS2) Awight: [WIP] Fail the message if a currency or amount are given alone, rather than as a pair. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/164499 [23:50:32] (Abandoned) Awight: WIP for discussion [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/181237 (owner: Awight) [23:52:50] (Abandoned) Awight: WIP looking at single responsibility issues in RapidHtml [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/183172 (owner: Awight) [23:54:22] (Abandoned) Awight: updates [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/184704 (owner: Awight) [23:54:32] (Abandoned) Awight: Merge contrib-v4.4 into dev-v4.4 [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/184798 (owner: Awight) [23:54:48] (Abandoned) Awight: Merge contrib-v4.4 into dev-v4.4 [wikimedia/fundraising/crm/civicrm] (dev-v4.4) - https://gerrit.wikimedia.org/r/184800 (owner: Awight) [23:55:40] (CR) Awight: [C: -1] "Needs manual rebase." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/184847 (https://phabricator.wikimedia.org/T86945) (owner: Awight) [23:55:52] (PS2) Awight: [WIP] Automatic config port using convertExtensionToRegistration.php [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/186899 [23:56:04] (CR) jenkins-bot: [V: -1] [WIP] Automatic config port using convertExtensionToRegistration.php [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/186899 (owner: Awight) [23:56:16] (CR) Awight: [C: -1] "Needs manual rebase." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/187252 (owner: Awight) [23:56:55] (CR) Awight: [C: -1] "Needs manual rebase." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/190359 (https://phabricator.wikimedia.org/T91670) (owner: Awight) [23:58:13] (Abandoned) Awight: WIP convert wikimedia stuff from submodules to vanilla dirs [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/190398 (owner: Awight) [23:58:53] (CR) Awight: [C: -1] "Needs manual rebase" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/190484 (owner: Awight)