[00:01:37] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2559569 (Eileenmcnaughton) I found the code and the data is correct... [00:01:50] fr-tech: when we stopped logging referrer we started getting notices in the audit logs. Fix is here: https://gerrit.wikimedia.org/r/289887 [00:02:05] heading out for the night, see you all later! [00:02:12] see ya [00:03:33] cwd do you know who would know most about whether contribution_tracking table data should be kept over CiviCRM data? - I'm looking at https://phabricator.wikimedia.org/T96410#2559569 [00:04:31] (I tend to bug Adam a lot but I'm trying to diversify :-) [00:27:44] eileen: hmm, let me take a look [00:31:07] eileen: i don't have very much context but it definitely seems to me like we shouldn't be overwriting good data with bad data [00:31:52] cwd no that's cool - I will ask Elliot & Adam for more background - I just didn't want to assume it was them that knew what the background is [00:32:59] heh yeah sorry i don't have an answer, i am still pretty green when it comes to that stuff [00:35:05] :-) Do we have any examples of doing unit tests on python code? That silverpop export is python & there are tickets saying we SHOULD have tests but I don't know whay python testing precedent we have [00:36:13] i doubt we have any protocol if there aren't tests there, afaik that's the only python code we are responsible for [00:37:58] ok - & it's python rather than php for historical reasons I guess? [00:38:29] (there is not much code in it TBH ) [00:42:16] yeah, i think it was an attempt to get away from php [00:42:24] that went well :P [00:50:29] eileen: spoke too soon! just found some python in smashpig [00:50:44] wat python in SmashPig? [00:51:07] "inject.py" [00:51:10] I like it when we try to move away from something we don't like & just wind up spreading ourselves thinner.. [00:51:14] it's in the tests dir [00:51:29] Hmmm [00:51:30] eileen: pretty classic [00:51:44] n+1 layers of complexity [00:52:42] i strongly advocate for serious feasibility questioning of rewrites. a half completed rewrite is worse than no rewrite at all [00:54:38] cwd spot on - although mostly people don't plan to half-do things [00:55:13] but yeah we do need to keep checking if we really are going to have bandwidth for things [00:58:50] yeah, it's so easy to bite off too much [00:59:13] or not know where to stop [01:00:45] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2559660 (Eileenmcnaughton) @awight @Ejegg I think I need to pick you... [01:24:55] cwd: if you're around and not too sleepy still, would you have a sec to remind me of our DB backend layout? thx in advance [01:24:58] * AndyRussG looks for doc [01:25:35] There's a DI database, a Civi database, the new Pending database, and rogue tables in one of the first two, I think... [01:29:36] AndyRussG: sure [01:30:05] drupal.contribution_tracking is the first thing that gets touched [01:30:37] stores stuff when the donor first arrives and (i think) gets updated as they progress [01:31:07] civi runs on drupal and they have their own dbs [01:31:37] then fredge stores some meta stuff like payments-init and fraud scores [01:32:35] i don't think things are very logically segregated so you pretty much just have to learn where to look for stuff [01:34:32] cwd: yeah the lack of logic is why I think they need to be shown explicitly in a sequence diagram [01:35:03] heh that would be one complicated diagram [01:35:11] That's where it's headed yeah [01:35:17] Well, several [01:41:18] cwd: thx!! ah just found somthing: https://www.mediawiki.org/wiki/Fundraising_tech/Database_schema [01:41:32] Probably out of date, but still useful :) [01:43:30] Ah hmm recently moved out of collab [01:43:35] * AndyRussG checks collab too [01:43:40] ah nice [01:51:23] Hmm just this in collab, which I now recall having seen b'fore https://collab.wikimedia.org/wiki/Fundraising/Engineering/Fun_SQL_Queries [01:53:19] AndyRussG: this took me forever to find but will make you face palm: https://github.com/wikimedia/wikimedia-fundraising-crm/blob/master/sites/all/modules/wmf_common/WmfDatabase.php [01:53:31] speaks to the poor separation of concerns [01:54:55] Hummm [01:55:10] Yeah and duct tape from the past [01:57:01] And also a pretty eclectic bunch of systems [02:03:55] indeed [02:03:59] organic growth [02:04:07] Mmmm [02:04:10] Well put [02:06:09] What I'm not sure is what the long-term solution to "Multiple-database transaction" is [02:06:34] I guess figure out the true partitioning and non-partitioning needs and reorganize based on that [02:09:30] yeah i don't think there's an easy answer, just have to DBA your way out of it [02:09:55] multi db transaction seems like the pandora's box option [02:17:06] heh [02:19:24] Besides inconsistency and maintenance issues, also potential security and privacy problems due to not seeing easily where stuff is [02:20:27] yep, we've got debt problems [02:20:38] well, it makes life interesting! [02:20:59] there has been progress lately! baby steps [02:21:12] yeah actually strides :) [02:28:20] http://wikiverse.io/ [02:37:05] neat [02:40:03] Yeah! [02:40:10] Heh 'nuther visualization https://www.shipmap.org/ [02:40:21] cwd: see ya! [04:10:14] (PS1) Cdentinger: WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305172 [04:11:00] (CR) jenkins-bot: [V: -1] WIP smashpig paypal listener [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305172 (owner: Cdentinger) [05:06:12] (PS1) XenoRyet: Prevent multiple Ingenico iFrames [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) [05:08:09] (CR) jenkins-bot: [V: -1] Prevent multiple Ingenico iFrames [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [05:11:09] (PS2) XenoRyet: Prevent multiple Ingenico iFrames [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) [06:45:29] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review, Unplanned-Sprint-Work: Decimal dates are breaking our queue consumer - https://phabricator.wikimedia.org/T140667#2559862 (awight) [07:13:22] (PS1) Awight: [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) [07:15:04] (CR) jenkins-bot: [V: -1] [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) (owner: Awight) [07:15:23] (PS2) Awight: [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) [07:18:33] (CR) jenkins-bot: [V: -1] [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) (owner: Awight) [07:20:28] (PS1) Awight: Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 [07:20:35] (CR) jenkins-bot: [V: -1] Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 (owner: Awight) [07:27:14] (PS2) Awight: Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 [07:27:21] (CR) jenkins-bot: [V: -1] Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 (owner: Awight) [07:28:55] (PS3) Awight: Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 [07:29:07] (CR) jenkins-bot: [V: -1] Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 (owner: Awight) [07:42:33] (PS4) Awight: Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 [08:05:55] (PS3) Awight: [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) [08:06:41] (PS4) Awight: [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) [08:10:19] (CR) jenkins-bot: [V: -1] [WIP] Make another DateTime conversion more robust [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305186 (https://phabricator.wikimedia.org/T140667) (owner: Awight) [08:30:11] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, FR-Paypal, FR-WMF-Audit, Unplanned-Sprint-Work: PayPal audit parser sends date in the wrong format - https://phabricator.wikimedia.org/T142417#2560036 (awight) [08:30:49] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, FR-Paypal, FR-WMF-Audit, Unplanned-Sprint-Work: Queue consumer sometimes fails to parse string date - https://phabricator.wikimedia.org/T142417#2534257 (awight) [08:33:00] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, FR-Paypal, FR-WMF-Audit, Unplanned-Sprint-Work: Queue consumer sometimes fails to parse string date - https://phabricator.wikimedia.org/T142417#2560039 (awight) There was a code path which assumes the `date` field has been normalized... [08:34:59] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Monitor the number of contributions coming in without an association to contribution_tracking - https://phabricator.wikimedia.org/T143190#2560041 (awight) [08:35:48] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Monitor the number of contributions coming in without an association to contribution_tracking - https://phabricator.wikimedia.org/T143190#2560053 (awight) A line will be logged, beginning with "Newly inserted contribution tracking id". [08:37:32] (PS1) Awight: Normalize date earlier [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305198 (https://phabricator.wikimedia.org/T142417) [08:41:27] (CR) jenkins-bot: [V: -1] Normalize date earlier [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305198 (https://phabricator.wikimedia.org/T142417) (owner: Awight) [17:25:12] (PS1) Ejegg: Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) [17:25:35] fr-tech: any news for scrum of scrums? [17:25:51] None here. [17:25:57] nothing i can think of, thanks [17:28:28] (CR) jenkins-bot: [V: -1] Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) (owner: Ejegg) [17:37:08] fr-tech: chatting in the tech talk fyi [17:37:36] Be there in like 5 minutes. [17:58:20] Fundraising-Backlog, FR-ActiveMQ: Python fundraising-tools queue module should be abstracted and support Redis - https://phabricator.wikimedia.org/T130308#2561499 (XenoRyet) a:XenoRyet [18:00:16] Looks like I upgraded myself out of a job... [18:00:31] Can't use hangouts on my laptop and the tower melted into graphics card glitchmode [18:00:46] ohnoes! [18:00:51] aw jeez - broken in every browser? [18:01:05] Phone last option mebbe? [18:01:09] ah good point, maybe chrom* [18:01:21] Yea that one works w/out a plugin [18:01:37] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review, Unplanned-Sprint-Work: Fold 'donor comment' field into 'note' - https://phabricator.wikimedia.org/T142747#2561514 (Ejegg) Open>Resolved [18:01:49] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, FR-ActiveMQ, and 2 others: SmashPig tasks which read from pending queue should compare to pending db - https://phabricator.wikimedia.org/T140959#2561515 (Ejegg) Open>Resolved [18:02:14] hehe I joined just in time :p [18:02:30] cool, but yeah it sounded like all the things were ironed out [18:02:35] I'll CR XenoRyet [18:02:41] Cool, thanks [18:02:48] XenoRyet's iframe patch, then whatever ejegg has in store [18:03:08] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog: Major Donors on email suppression list who should be opted in - https://phabricator.wikimedia.org/T139102#2561519 (Ejegg) Open>Resolved [18:04:22] stuff for CR: paypal pending message patches ( https://gerrit.wikimedia.org/r/304930 , https://gerrit.wikimedia.org/r/304931 ) [18:05:16] cool [18:08:35] ejegg: this is the one that will read from redis and set donation queue messages? [18:08:46] (CR) Awight: [C: 2] Paypal: set order ID = ct_id, email is an optional key (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304930 (https://phabricator.wikimedia.org/T141654) (owner: Ejegg) [18:09:36] cwd that's just making the paypal gateway log a pending message from the front end [18:09:52] oh right sorry, pending queue [18:10:37] (Merged) jenkins-bot: Paypal: set order ID = ct_id, email is an optional key [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304930 (https://phabricator.wikimedia.org/T141654) (owner: Ejegg) [18:11:00] (PS2) Ejegg: Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) [18:12:07] (CR) Awight: [C: 2] "profitable!" (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304931 (https://phabricator.wikimedia.org/T141654) (owner: Ejegg) [18:14:22] (CR) jenkins-bot: [V: -1] Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) (owner: Ejegg) [18:14:40] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Added new Custom Field on Production - https://phabricator.wikimedia.org/T142624#2561586 (DStrine) a:RLewis>None [18:16:31] if these paypal IPNs come in as URL NVP, what format should the test data be in? php array? [18:17:04] I would go .json [18:17:06] Fundraising-Backlog: Civi email merge won't retain new email address - https://phabricator.wikimedia.org/T143236#2561596 (CCogdill_WMF) [18:18:01] (Merged) jenkins-bot: Log and send pending message for all redirects [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/304931 (https://phabricator.wikimedia.org/T141654) (owner: Ejegg) [18:18:02] awight: and have the test turn into an array? [18:18:43] Fundraising-Backlog: Civi email merge won't retain new email address - https://phabricator.wikimedia.org/T143236#2561615 (CCogdill_WMF) p:Triage>High [18:20:12] Fundraising-Backlog: Civi email merge won't retain new email address - https://phabricator.wikimedia.org/T143236#2561596 (CCogdill_WMF) [18:20:42] cwd: Hmm, yeah that sounds right [18:21:50] (CR) Awight: [C: 2] "Nice solution! Much more robust than our previous button disablery." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [18:22:36] The research showcase happening right about now looks really interesting [18:23:18] (Merged) jenkins-bot: Prevent multiple Ingenico iFrames [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [18:24:47] fr-tech: fyi ^ research showcase [18:25:05] Cool [18:25:07] oh cool [18:28:11] (PS3) Ejegg: Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) [18:35:27] neat! [18:41:19] !log disabled donations queue consumer, adyen job runner, pending queue consumer for db rename [18:41:24] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:52:10] !re-enabled jobs after db rename [18:53:47] Fundraising-Backlog, FR-Smashpig: Create 'smashpig' database to replace 'pending' - https://phabricator.wikimedia.org/T142515#2561736 (Ejegg) Open>Resolved [18:57:28] nice! [18:57:48] (CR) Awight: "Ah, also: it works :)" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [18:58:28] (CR) XenoRyet: ";)" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/305173 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [19:07:22] fundraising-tech-ops: install OS on fdb1001 - https://phabricator.wikimedia.org/T143245#2561807 (Jgreen) [19:09:06] fundraising-tech-ops: install OS on fdb1001 - https://phabricator.wikimedia.org/T143245#2561826 (Jgreen) [19:09:08] fundraising-tech-ops, Operations, ops-eqiad: Rack and setup Fundraising DB - https://phabricator.wikimedia.org/T136200#2561825 (Jgreen) [19:11:34] fundraising-tech-ops: install OS on frdb1001 - https://phabricator.wikimedia.org/T143245#2561830 (Jgreen) [19:16:47] Fundraising-Backlog: Leanne Schreibstein access to lutetium and MySQL - https://phabricator.wikimedia.org/T141945#2561870 (Jgreen) p:Unbreak!>Normal shell and mysql accounts configured, initial setup email sent, waiting for a reply [19:29:28] Fundraising-Backlog, Operations, Traffic, Patch-For-Review: Switch Varnish's GeoIP code to libmaxminddb/GeoIP2 - https://phabricator.wikimedia.org/T99226#2561904 (BBlack) [20:05:45] fr-tech - couple audit-related CRM things for review. [20:06:06] There's a notice in the logs every time we parse json cause there's no referrer in the logs any more [20:06:48] here's a fix for that: https://gerrit.wikimedia.org/r/#/c/289887/6 [20:07:07] Then we'll need this one for future queue stuff: https://gerrit.wikimedia.org/r/#/c/305305/3 [20:07:49] (CR) Ejegg: [C: 2] Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 (owner: Awight) [20:08:23] I feel so fancy--I have xwindows working on both computers again. [20:08:43] (Merged) jenkins-bot: Unpin from PHP 5, to allow development on PHP 7 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/305187 (owner: Awight) [20:11:01] network issues with the new os awight? [20:12:21] Every issue [20:12:29] It's looking good now, though [20:18:01] hmm, lots of amazon donation message imports finding the pending info in activemq but not the pending db. Records are there when I check, so I guess it's all timing [20:18:34] Could also be something wrong with the way we're checking from code [20:18:46] Although... timing wouldn't be surprising, due to the queue consumption step :( [20:19:20] yeah, i wonder if the pending queue consumer should always run before the donation queue consumer [20:19:48] dang, that's a difficult one [20:21:21] I'm not coming up with anything clever [20:23:01] Even running them like you said will still result in the last donation queue message lacking their precedessors [20:23:07] *messages [20:23:15] right [20:24:22] Unfortunately, we can't do something like stop the donation queue consumer once it hits the first unprecedented message [20:24:31] cos a single bad message would choke the consumer [20:25:02] * awight briefly imagines a donation queue consumer that dumps into a db rather than doing any processing [20:25:05] no, we're just going to bounce them to the damaged message store and requeue them in a half an hour or se [20:25:21] That's a really long delay for TY letter satisfaction [20:25:53] wow, we're being really redundant with the amazon messages... [20:26:15] I think they always hit the 'completed' queue at least twice [20:26:49] let me see if anything actually comes in with source listener [20:29:04] heading to the conf room... [21:04:29] google has chucked me off to re-do my account verification [21:05:42] waiting on code... [21:08:08] It always does that right in the middle of calls [21:08:18] I wonder how it knows... [21:11:34] :-) [21:28:05] I wonder is re the on-call the answer is just to stick with the status quo but to set a date to re-discuss & review? [21:30:05] dstrine: Am I missing something? You said 'see everyone in 5" - but the meeting is in 30 mins on my calendar [21:36:03] awight: one thing about last weekend that did make it a "something what needs fixing" type deal is it was sending massive failmails [21:36:56] I think they DID need to get us involved and actually you turning off that dedupe job turned out to be important [21:37:19] oh was there something actually broken there? [21:37:31] There was a problem with it (under some circumstances) and the lack of an immediate re-action to you turning it off was probably the query taking time to finish [21:37:50] The issue was that it was limiting the query based on an id range [21:38:07] but when no contacts were in the id range the limit was being dropped [21:38:12] resulting in a big query [21:38:23] hence it worked fine for 200,000 contacts [21:38:34] and then crashed on the next 200 where we had an id gap [21:39:05] ooh interesting. but turning off the dedupe job didn't seem to stop the mails [21:39:25] cwd I think the bad query took time to go away [21:39:34] hence the delayed response [21:40:42] fr-tech: Is 20 mins enough time to deploy DI, and does anyone want to jump on a chat and ELI5 how to do it? I'd like to get that iframe patch out before planning, and it's far past time I really learned how to deploy DI. [21:41:22] If after or tomorrow is better, that's fine too. [21:41:52] 20 min should be plenty [21:42:04] eileen: aaah i see [21:42:33] XenoRyet: first is just to merge to deployment and update the submodule in the fundraising/REL1_27 branch of core [21:43:56] you can self- +2 those [21:44:52] XenoRyet: side note--you can cherry-pick your patch from master to deployment, or "cherry -v" and merge everything that's waiting [21:45:01] we don't care about maintaining the order... [21:48:39] hey all I just discussed with Jeff planning a one-hour outage for one hour on 31 Aug 8pm EST [21:48:52] do any of you have any input before I email more widely [21:48:57] XenoRyet: you can also tease the steps out of here if you want: https://github.com/caseydentinger/frig/blob/master/prep.py [21:49:26] Oh, useful. [21:49:26] the outage is for deploying the CiviCRM dash (which is only a few minutes but needs Jeff) and Jeff has some other maintenance he will do at the same time [21:49:51] it probably still works but i sort of gave up [21:50:06] git is poorly scriptable above the "porcelain" layer [21:50:27] if anyone wants to tack on any mtce to the above I can suggest a longer outage [21:50:54] eileen: That sounds great [21:51:36] eileen: What kind of outage are you planning? Campaigns, Civi, everything? [21:51:53] well Jeff wants to include a kernel update & reboot [21:52:20] I guess campaigns can stay up but queues would have to come down [21:52:30] does that sound right? [21:53:17] (PS1) XenoRyet: Prevent multiple Ingenico iFrames [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/305404 (https://phabricator.wikimedia.org/T142059) [21:53:41] (CR) XenoRyet: [C: 2] Prevent multiple Ingenico iFrames [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/305404 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [21:54:26] (Merged) jenkins-bot: Prevent multiple Ingenico iFrames [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/305404 (https://phabricator.wikimedia.org/T142059) (owner: XenoRyet) [21:58:20] (PS1) XenoRyet: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/305406 [21:59:02] (CR) XenoRyet: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/305406 (owner: XenoRyet) [22:00:13] be right there [22:00:37] ack me2 [22:03:51] Fundraising-Backlog: Upgrade to drupal 7.5 - https://phabricator.wikimedia.org/T143268#2562488 (Eileenmcnaughton) [22:04:33] Fundraising-Backlog: Improve documentation for emergency & on-call procedures - https://phabricator.wikimedia.org/T143266#2562514 (Eileenmcnaughton) [22:06:42] Fundraising-Backlog, Operations, Traffic, Patch-For-Review: Switch Varnish's GeoIP code to libmaxminddb/GeoIP2 - https://phabricator.wikimedia.org/T99226#2562532 (BBlack) [22:07:00] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/305406 (owner: XenoRyet) [22:09:59] Fundraising-Backlog, Operations, Traffic, Patch-For-Review: Switch Varnish's GeoIP code to libmaxminddb/GeoIP2 - https://phabricator.wikimedia.org/T99226#2562548 (BBlack) [22:11:50] (PS1) Eileen: Further fix to criteria range on dedupe - exclude deleted [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305407 (https://phabricator.wikimedia.org/T142954) [22:12:43] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Fix GeoIP cookie synthesis or loading to prevent repeated calls to geoiplookup for IPv6 users who can't be geolocated by their address on that protocol - https://phabricator.wikimedia.org/T122092#1896447 (BBlack) Should should hopefu... [22:14:34] Fundraising-Backlog, Operations, Traffic, Patch-For-Review: Switch Varnish's GeoIP code to libmaxminddb/GeoIP2 - https://phabricator.wikimedia.org/T99226#2562582 (BBlack) I'm planning to merge up the new version of the Varnish GeoIP code from 371d7cc737d0 in the next couple of days. If anyone ha... [22:19:58] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Pipeline Report - https://phabricator.wikimedia.org/T128818#2562635 (RLewis) @Eileenmcnaughton just checking in on this, do all the changes we've recommended mak... [22:45:59] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 4 others: CRM reads 'completion messages' from pending DB and ActiveMQ, scream if no match - https://phabricator.wikimedia.org/T140484#2562750 (Ejegg) [22:57:47] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, FR-Smashpig, and 2 others: Migrate donations to new queue - https://phabricator.wikimedia.org/T131277#2161921 (Ejegg) The TransactionalWmfQueueConsumer should be a good class to use for this. You can take a look at https://gerrit.w... [22:58:29] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, WMF-deploy-2016-08-16_(1.28.0-wmf.15): Queue mirroring needs to copy source_* fields into the message body - https://phabricator.wikimedia.org/T141948#2562805 (cwd... [22:58:33] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 3 others: Donations message with empty body! - https://phabricator.wikimedia.org/T140515#2562806 (awight) Open>Resolved [22:58:34] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 2 others: Agree acceptance criteria & implement the address requirements - https://phabricator.wikimedia.org/T135062#2562807 (Eileenmcnaughton) Open... [22:58:36] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: [EPIC] dedupe exact matches - https://phabricator.wikimedia.org/T133807#2562808 (Eileenmcnaughton) [22:59:06] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising-Backlog, FR-Adyen, and 2 others: bug: Adyen JPY transactions have different amounts in CIVI vs Adyen - https://phabricator.wikimedia.org/T137607#2562810 (Eileenmcnaughton) [22:59:08] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 3 others: Handle refund mismatch more correctly - https://phabricator.wikimedia.org/T138542#2562809 (Eileenmcnaughton) Open>Resolved [22:59:13] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Unplanned-Sprint-Work: Is payments_initial missing Ingenico transactions? - https://phabricator.wikimedia.org/T142873#2562811 (Ejegg) Open>Resolved [22:59:21] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Epic: [EPIC] dedupe exact matches - https://phabricator.wikimedia.org/T133807#2244813 (Eileenmcnaughton) [22:59:23] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 3 others: CiviCRM dedupe causes runaway query - https://phabricator.wikimedia.org/T139964#2562813 (Eileenmcnaughton) Open>Resolved [22:59:37] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Civi reports: add filter for "is deceased" - https://phabricator.wikimedia.org/T140831#2562830 (Eileenmcnaughton) Open>Resolved [23:00:41] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 2 others: Disabled campaigns are causing messages to be rejected: '\'C15_Email1\' is not a valid option - https://phabricator.wikimedia.org/T139969#2562836 (Eileenmcnau... [23:00:45] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, FR-Smashpig, and 4 others: [Epic] Consolidate "pending" queue usages - https://phabricator.wikimedia.org/T130897#2562838 (Ejegg) [23:00:47] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 4 others: CRM reads 'completion messages' from pending DB and ActiveMQ, scream if no match - https://phabricator.wikimedia.org/T140484#2466525 (Ejegg) Open>Reso... [23:00:55] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising-Backlog, and 2 others: Disabled campaigns are causing messages to be rejected: '\'C15_Email1\' is not a valid option - https://phabricator.wikimedia.org/T139969#2562839 (awight) ... [23:01:21] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 2 others: Investigate Civi Load Time issue - https://phabricator.wikimedia.org/T138334#2562841 (Eileenmcnaughton) @CaitVirtue can we close this out? I t... [23:01:56] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Patch-For-Review, Spike: Spike: Civi Failmail Error: IMPORT_CONTACT Couldn't store email for the contact. - https://phabricator.wikimedia.org/T142954#2562845 (Eileenmcnaughton) Very small additional patch for review [23:03:23] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, FR-Smashpig, and 3 others: Audit processors should mirror outbound queue messages - https://phabricator.wikimedia.org/T141484#2562863 (Ejegg) [23:12:40] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Patch-For-Review, Spike: Spike: Civi Failmail Error: IMPORT_CONTACT Couldn't store email for the contact. - https://phabricator.wikimedia.org/T142954#2562891 (DStrine) [23:12:43] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Failmail code no longer recognizes comma-separated addresses - https://phabricator.wikimedia.org/T142420#2562893 (DStrine) [23:12:48] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, FR-Paypal, and 3 others: Queue consumer sometimes fails to parse string date - https://phabricator.wikimedia.org/T142417#2562894 (DStrine) [23:12:49] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, MediaWiki-extensions-DonationInterface, and 2 others: Clicking the credit card button multiple times loads duplicate iframes... - https://phabricator.wikimedia.org/T142059#2562895 (DStrine) [23:12:52] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Support CiviCRM dash changes - https://phabricator.wikimedia.org/T142051#2562896 (DStrine) [23:12:54] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Goal: SPRINT O (8-3 to 8/17) Goal: Queue work and bugs - https://phabricator.wikimedia.org/T142039#2562897 (DStrine) [23:12:55] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, FR-ActiveMQ, and 5 others: Move legacy PayPal listener to SmashPig - https://phabricator.wikimedia.org/T141654#2562898 (DStrine) [23:12:57] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, FR-ActiveMQ: UML activity and sequence diagrams for all processors - https://phabricator.wikimedia.org/T141617#2562899 (DStrine) [23:13:00] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, and 7 others: Run the orphan rectifier job from CRM Jenkins - https://phabricator.wikimedia.org/T141487#2562901 (DStrine) [23:13:04] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, and 6 others: Rewrite orphan rectifier to use the pending database and WmfFramework - https://phabricator.wikimedia.org/T141486#2562902 (DStrine) [23:13:08] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, and 4 others: Continuous integration: DonationInterface needs composer variant - https://phabricator.wikimedia.org/T141309#2562903 (DStrine) [23:13:09] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Decimal dates are breaking our queue consumer - https://phabricator.wikimedia.org/T140667#2562904 (DStrine) [23:13:12] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 4 others: Spike: Monitor deployment of: Suppress CentralNotice ResourceLoader modules on Special and act... - https://phabricator.wikimedia.org/T139439#2562906 [23:13:14] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 3 others: Investigate Civi Load Time issue - https://phabricator.wikimedia.org/T138334#2562907 (DStrine) [23:13:18] Fundraising Sprint Licking Cookies, Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, and 6 others: Spike: Monitor deployment rolling back our "googleoff" tag - https://phabricator.wikimedia.org/T137761#2562908 (DStrine) [23:13:20] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, and 2 others: Migrate fredge to new queue - https://phabricator.wikimedia.org/T131273#2562911 (DStrine) [23:13:21] Fundraising Sprint Jabberwock Slaying, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, and 3 others: Slow queries on staging related to groups - https://phabricator.wikimedia.org/T136158#2562909 (DStrine) [23:13:24] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising Sprint Jabberwock Slaying, Fundraising Sprint Killing Time, and 13 others: [Epic] All pending producers should write to a single pending queue - https://phabricator.wikimedia.org/T133195#2562910 (DStrine) [23:13:26] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, and 3 others: Pipeline Report - https://phabricator.wikimedia.org/T128818#2562912 (DStrine) [23:13:28] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, and 6 others: Quit looking in pending queue for completion message information - https://phabricator.wikimedia.org/T122641#2562913 (DStrine) [23:13:31] Fundraising Sprint Nitpicking, Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Unplanned-Sprint-Work: Help setting up a Top Prospects report in Civi - https://phabricator.wikimedia.org/T113904#2562914 (DStrine) [23:13:32] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2562915 (DStrine) [23:13:34] Fundraising Sprint P 2016, Fundraising-Backlog, FR-ActiveMQ: Python fundraising-tools queue module should be abstracted and support Redis - https://phabricator.wikimedia.org/T130308#2562916 (DStrine) [23:13:36] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, and 2 others: [Epic] Rewrite all queue clients to use a single shim library, improve library - https://phabricator.wikimedia.org/T133108#2562917 (DStrine) [23:13:39] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Failmail code no longer recognizes comma-separated addresses - https://phabricator.wikimedia.org/T142420#2562918 (Ejegg) Open>Resolved [23:13:41] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops, and 3 others: Migrate pending consumers to new queue and finish cleanup - https://phabricator.wikimedia.org/T131274#2562920 (DStrine) [23:14:13] Fundraising Sprint Octopus Untangling, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Regression, Unplanned-Sprint-Work: Failmail code no longer recognizes comma-separated addresses - https://phabricator.wikimedia.org/T142420#2534300 (Ejegg) [23:14:39] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, and 4 others: [Epic] Deprecate old pending queues - https://phabricator.wikimedia.org/T133433#2562922 (DStrine) [23:14:53] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, FR-Smashpig, and 3 others: [Epic] Deprecate old pending queues - https://phabricator.wikimedia.org/T133433#2231731 (DStrine) [23:15:11] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, and 4 others: Audit processors should mirror outbound queue messages - https://phabricator.wikimedia.org/T141484#2562924 (DStrine) [23:15:20] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, Spike: Spike: Deploy pilot new queue consumer - https://phabricator.wikimedia.org/T130306#2562925 (DStrine) [23:15:31] Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-ActiveMQ, and 3 others: Migrate donations to new queue - https://phabricator.wikimedia.org/T131277#2562926 (DStrine) [23:15:41] Fundraising Sprint P 2016, Fundraising-Backlog, FR-ActiveMQ, FR-Ingenico, and 2 others: Spike: investigate donations-gc-garbage - https://phabricator.wikimedia.org/T136022#2562928 (DStrine) [23:18:45] Fundraising Sprint Octopus Untangling, Fundraising Sprint P 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2562939 (awight) @Eileenmcnaughton U... [23:22:08] (PS1) BBlack: CN: use cookie exclusively for GeoIP data [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/305423 (https://phabricator.wikimedia.org/T143271) [23:27:08] (CR) jenkins-bot: [V: -1] CN: use cookie exclusively for GeoIP data [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/305423 (https://phabricator.wikimedia.org/T143271) (owner: BBlack) [23:31:16] Fundraising Sprint Octopus Untangling, Fundraising Sprint Pretending This Isn't Happening, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2562986 (Eile... [23:34:47] awight: can you look at this super small extension to the patch you previously reviewed https://gerrit.wikimedia.org/r/#/c/305407/ [23:40:04] eileen: sure! [23:40:32] (CR) Awight: [C: 2] "Oops! That makes sense." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305407 (https://phabricator.wikimedia.org/T142954) (owner: Eileen) [23:42:30] (CR) Awight: [C: 2] "Those look right" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/304289 (https://phabricator.wikimedia.org/T140959) (owner: Ejegg) [23:43:36] (Merged) jenkins-bot: Further fix to criteria range on dedupe - exclude deleted [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305407 (https://phabricator.wikimedia.org/T142954) (owner: Eileen) [23:43:38] (Merged) jenkins-bot: Remove some unused message fields [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/304289 (https://phabricator.wikimedia.org/T140959) (owner: Ejegg) [23:43:46] (CR) Awight: [C: 2] Not using referrer from messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289887 (https://phabricator.wikimedia.org/T110564) (owner: Ejegg) [23:45:29] (CR) Awight: [C: 2] Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) (owner: Ejegg) [23:45:54] (CR) Awight: "Ack, needs manual rebase!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/301630 (https://phabricator.wikimedia.org/T133433) (owner: Ejegg) [23:47:05] (Merged) jenkins-bot: Not using referrer from messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/289887 (https://phabricator.wikimedia.org/T110564) (owner: Ejegg) [23:48:34] (Merged) jenkins-bot: Add order_id and more to audit messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/305305 (https://phabricator.wikimedia.org/T141484) (owner: Ejegg) [23:48:42] (PS4) Awight: Quit deleting from pending queue, stop saying limbo [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/301630 (https://phabricator.wikimedia.org/T133433) (owner: Ejegg) [23:51:15] (CR) jenkins-bot: [V: -1] Quit deleting from pending queue, stop saying limbo [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/301630 (https://phabricator.wikimedia.org/T133433) (owner: Ejegg) [23:52:36] (PS5) Awight: [DO NOT MERGE] Quit deleting from pending queue, stop saying limbo [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/301630 (https://phabricator.wikimedia.org/T133433) (owner: Ejegg) [23:52:49] (PS3) Awight: Update SmashPig and PHP-Queue [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303701 (owner: Ejegg) [23:53:16] !log updated payments-wiki from b449f65dba5697905a87261592e934d7f4898a54 to 2b027e313ccecc2b93f214a94738b4f94899f347 [23:53:22] (CR) Awight: [C: 2] "Might be stale already, but it's a step forward." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303701 (owner: Ejegg) [23:53:23] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [23:53:33] (PS2) Awight: Use SmashPig config shortcuts, reset Context [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303717 (owner: Ejegg) [23:54:05] (CR) Awight: [C: 2] Use SmashPig config shortcuts, reset Context [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303717 (owner: Ejegg) [23:57:14] (Merged) jenkins-bot: Update SmashPig and PHP-Queue [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303701 (owner: Ejegg) [23:57:16] (Merged) jenkins-bot: Use SmashPig config shortcuts, reset Context [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/303717 (owner: Ejegg) [23:59:28] ejegg: Did I miss anything you need?