[00:13:29] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Deleted contacts: contributions tab has a count, but no contributions appearing - https://phabricator.wikimedia.org/T121717#2255557 (Eileenmcnaughton) I just clicked on the link & it's not the case now - I suspect something has fixed this in the meantime [00:14:40] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#2255558 (Eileenmcnaughton) Bringing this into sprint as it is to be resolved in this week's deploy [00:14:54] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#2255559 (Eileenmcnaughton) [00:23:28] Fundraising Sprint Internet Exploring, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Revisit triggers. We probably want them. - https://phabricator.wikimedia.org/T117055#2255562 (Eileenmcnaughton) [00:29:53] Fundraising Sprint Internet Exploring, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Revisit triggers. We probably want them. - https://phabricator.wikimedia.org/T117055#2255563 (Eileenmcnaughton) I pulled this in as we are deploying triggers this week & it duplic... [00:34:22] Fundraising Sprint Internet Exploring, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Revisit triggers. We probably want them. - https://phabricator.wikimedia.org/T117055#2255565 (Eileenmcnaughton) a:Eileenmcnaughton [00:34:30] Fundraising Sprint Internet Exploring, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Revisit triggers. We probably want them. - https://phabricator.wikimedia.org/T117055#1765636 (Eileenmcnaughton) [00:34:33] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#2255567 (Eileenmcnaughton) [00:34:40] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#1011901 (Eileenmcnaughton) a:Eileenmcnaughton [00:40:26] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Sync our buildkit with upstream - https://phabricator.wikimedia.org/T126266#2255572 (Eileenmcnaughton) related tickets https://phabricator.wikimedia.org/T120044 https://phabricator.wikimedia.org/T103180 https://phabricator.wikimedia.org/T115925 https://... [00:41:45] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Sync our buildkit with upstream - https://phabricator.wikimedia.org/T126266#2255574 (Eileenmcnaughton) Note this comment https://phabricator.wikimedia.org/T131232 In T131232#2163916, @Eileenmcnaughton wrote: Note that buildkit affects peoples installs... [04:59:59] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: setup civi 4.7 on staging - https://phabricator.wikimedia.org/T133806#2255700 (Eileenmcnaughton) I've been through the process of re-rolling our patches on the latest 4.6 & then on 4.7.7rc and have come up with... [06:13:27] (PS3) Eileen: CRM-18193 add default log_conn values for when interacting with mysql outside of CiviCRM. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/285263 [06:22:18] (PS1) Eileen: Updated MySql trigger code [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286390 (https://phabricator.wikimedia.org/T133745) [06:22:35] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Update trigger mysql in git to reflect latest change from upstream (connection_id) - https://phabricator.wikimedia.org/T133745#2255752 (Eileenm... [06:22:51] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Verify that the contact modified date trigger is installed - https://phabricator.wikimedia.org/T118346#2255753 (Eileenmcnaughton) [06:23:37] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#2255754 (Eileenmcnaughton) [06:23:40] Fundraising Sprint Internet Exploring, Fundraising Tech Backlog, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Revisit triggers. We probably want them. - https://phabricator.wikimedia.org/T117055#2255755 (Eileenmcnaughton) [06:54:40] (CR) Awight: [C: 2] "Yep! Empty string would have fooled isset." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286377 (https://phabricator.wikimedia.org/T133796) (owner: Eileen) [06:55:14] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Advanced search by country fails with database syntax error - https://phabricator.wikimedia.org/T133796#2255813 (awight) [06:56:34] (Merged) jenkins-bot: Fix for country search [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286377 (https://phabricator.wikimedia.org/T133796) (owner: Eileen) [12:23:38] Fundraising Tech Backlog, Fundraising-Backlog, fundraising-tech-ops: Adjust modsecurity rules to work for us - https://phabricator.wikimedia.org/T110227#2256373 (Jgreen) a:Jgreen [12:24:50] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Remove Engage Civi users and revoke SSL keys - https://phabricator.wikimedia.org/T114797#2256376 (Jgreen) a:Jgreen [12:26:41] Fundraising-Backlog, fundraising-tech-ops: Paymentswiki LocalSettings should not be writeable by fr-dev users - https://phabricator.wikimedia.org/T132392#2256379 (Jgreen) a:Jgreen [12:27:45] Fundraising-Backlog, fundraising-tech-ops: Paymentswiki LocalSettings should not be writeable by fr-dev users - https://phabricator.wikimedia.org/T132392#2196803 (Jgreen) Open>Resolved this is done [15:54:59] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Civi Question & Possible Task: Is there a more detailed change log than the one on the contact summary page? - https://phabricator.wikimedia.org/T88447#2257022 (DStrine) [16:15:36] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Verify that the contact modified date trigger is installed - https://phabricator.wikimedia.org/T118346#2257071 (DStrine) [16:15:41] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Verify that the contact modified date trigger is installed - https://phabricator.wikimedia.org/T118346#1797852 (DStrine) [16:36:15] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, and 4 others: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2257144 (cwdent) @MBeat33: the ones that are... [17:16:10] morning fellas [17:16:20] well...afternoon ejegg [17:16:32] heh, hi cwd [17:16:54] haloo! [17:31:13] hey Jeff_Green - there's a file in my home dir on barium, would you mind running it when you have a chance? just moves some audit files around [17:31:30] hey cwd, sure, looking [17:32:15] thanks! think i'm really close to having all these worldpay transactions accounted for [17:32:23] dine [17:32:26] done even [17:32:35] although dining is good too [17:32:49] worldpay done == yay [17:43:49] Fundraising Sprint Internet Exploring, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Patch-For-Review: CentralNotice: choiceData RL module hashes are flapping - https://phabricator.wikimedia.org/T133971#2257341 (AndyRussG) Open>Resolved [17:45:40] (PS2) Ejegg: Encapsulated amount validation for better messages [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286261 (https://phabricator.wikimedia.org/T105618) [17:47:21] Fundraising Sprint Internet Exploring, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Patch-For-Review: CentralNotice: choiceData RL module hashes are flapping - https://phabricator.wikimedia.org/T133971#2257347 (AndyRussG) @Catrope deployed the fix for this w... [17:47:23] (CR) jenkins-bot: [V: -1] Encapsulated amount validation for better messages [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286261 (https://phabricator.wikimedia.org/T105618) (owner: Ejegg) [17:47:54] Fundraising Sprint Internet Exploring, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work, Patch-For-Review: CentralNotice: choiceData RL module hashes are flapping - https://phabricator.wikimedia.org/T133971#2257348 (AndyRussG) [17:48:19] Fundraising Sprint Elevator Maintenance 2016, Fundraising Sprint Freshmaking, Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, and 4 others: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2257349 (MBeat33) Thanks, @cwdent, for stayin... [18:10:12] (PS3) Ejegg: Encapsulated amount validation for better messages [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286261 (https://phabricator.wikimedia.org/T105618) [18:14:29] (PS1) Ejegg: Don't register client-side var hook when failed [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286490 [18:16:47] (CR) Ejegg: "Oops, noticed another regression with this - we actually were using the check_not_empty param in GatewayAdapter::revalidate" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/281084 (owner: Awight) [18:17:07] That is quite the news on wikimedia-l... [18:17:17] oh? [18:17:48] From Denny. [18:18:12] oho, doc james details... [18:18:29] ejegg: Hey, you want me to make a task for the regression you just found? [18:19:05] awight: ah, I should do that, thanks for reminding me the proper procedure! [18:20:02] hehe just cos I will forget in ten seconds, is all [18:21:10] (PS1) Awight: Fix indentation [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/286491 [18:22:50] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Regression: no longer validating gateway-specific required fields server side - https://phabricator.wikimedia.org/T134189#2257398 (Ejegg) [18:23:13] Thank you! [18:23:17] should be a pretty straightforward fix though, just undoing the deletion [18:23:23] ah [18:23:48] That seems like a step backwards on the encapsulation [18:23:53] think I'll do that now to get DI into a less questionable state [18:24:04] Maybe this is a single responsibility fail? [18:24:14] ah, you think it should be handled differently? [18:25:00] If we're already doing common and adapter-level validations in dataobject and with transformers, seems like that's where the config/validations overrides should be moved [18:25:15] So each config lists the validators they need, and the validator checks both for parameter existing and for validity [18:25:24] mmm [18:26:02] We could keep it backward-compatible with config/payment_methods.validation [18:26:41] maybe a shared RequiredField validator for now... just to get it out of the adapter base class [18:26:51] and so that all validation code is called at one time [18:27:42] and the validator asks the gateway adapter which fields are required [18:27:51] ? [18:28:00] hmm. ok that seems like a flaw [18:28:28] Maybe the validator just calls adapter->getConfig [18:28:41] hrm gack. But it's a methodmeta thing [18:29:10] yah, and can depend on country too [18:29:13] how *does* methodmeta interact with validation... [18:29:23] getRequiredFields [18:29:31] called in revalidate [18:29:40] Ingenico has data_constraints, too [18:29:48] Those change depending on method and country [18:30:08] That seems like a good abstraction [18:30:18] hmm, ok [18:30:48] huh. Are those constraints on staged fields, though... [18:30:59] ooh, probably [18:31:11] We may be glossing over that distinction [18:32:22] It almost does make sense to push even required field validation into the transformer [18:32:38] We're already doing tests in there for data existence [18:33:14] right, but how about a StreetAddress validator [18:33:31] that does a luhn check and sets an error if there's a CC# there [18:33:35] The only down side is that feels like config-able boilerplate [18:33:43] does it have a subclass RequiredStreetAddress? [18:34:00] not quite following [18:34:14] So, sometimes street address is required [18:34:15] StreetAddress is the common class shared by all gateways? [18:34:33] I think, if a gateway has special rules then it does subclass StreetADdress [18:34:48] is 'address required' a special rule? [18:34:54] or 'address optional'? [18:34:58] And it's each enuf to have a switch($adapter->getPAymentMethod) in there [18:35:13] Good point, it would be hard to go the other way [18:35:14] where, in defineDataTransformers? [18:35:36] no, in SometimesRequiredStreetAddress::validate [18:35:47] s/each/easy [18:36:29] oh, so the SometimesRequiredStreetAddress knows which payment methods need address in which countries? [18:36:49] yes, it's where we encapsulate everything about a specific gateway x country x address fields [18:36:54] x method [18:37:00] Donno [18:37:11] It does solve the problem of, how to represent complex rules in a config file ;) [18:37:20] don't. [18:37:36] hmm, and how about the template renderer? Does it do the same logic to get the required fields? [18:38:26] gah [18:38:29] no [18:38:45] double gah [18:38:50] I feel like getRequiredFields might be a legit adapter responsibility [18:39:08] since there could be so many single-gateway methods [18:39:20] or gateway-specific field requirements [18:42:14] however, we're being a bit arbitrary about which validation rules are copied by the client [18:43:27] right, it would be nice to have a matching client-side module for each validator [18:43:55] hmm, that's probably the way to go for that client-side fiscal # thing [18:44:53] not trying to generalize rules, just writing validator functions and specifying which ones apply [18:46:00] found a cool van, and it's up in estes http://boulder.craigslist.org/cto/5565438851.html [18:46:06] where i am heading on weds for the reading offsite [18:46:15] data_constraints does seem like a nice thing, and there are very few edge cases handled in code currently. Anyway, I'm happy to revert the check_Empty_fields thing for now! [18:46:47] cwd: total dream van, unless you have to maintain it :p [18:47:01] cwd hardwood floors? [18:47:04] wat [18:47:25] haha, yeah [18:47:27] It's also the official official van, so if you don't trick it out with murals you can drive anywhere [18:47:40] totally [18:47:44] Flowers By Irene [18:48:08] i could probably replace the engine and trans and still be under $10k into it [18:48:12] and have a brand new van [18:48:16] Sweet. I did that with another friend, installed classy boardroom pergo in his van [18:48:56] i saw an econoline with 1M miles [18:49:05] whoa [18:49:26] couple of rebuilds but...there's not a lot to go wrong with them [18:49:26] immortoline [18:49:30] I'm so jelly of the roof rack [18:49:40] wow! okay I stand overcorrected [18:49:44] unless you make them into some crazy ass 4x4 thing [18:49:57] then there's a lot to go wrong [18:50:36] it's a V6 so i wonder if it's underpowered [18:51:15] Better range on a full tank, though... [18:51:47] yep [18:51:54] it made it up to estes so it can't be that bad :) [18:52:15] 4.6 is still a pretty giant engine [18:52:37] my dad's truck has a v8 that is 4.2 [18:53:42] related: https://en.wikipedia.org/wiki/We_Jam_Econo [18:53:57] one of my favorite music documentaries ever. the tag line is about their econoline. [18:54:11] but also a reference to communism i'm guessing [18:56:17] * awight sprays a pink hammer and sickle on the sliding door [18:57:40] looking at the queue task to create a pending schema and consumer [18:58:16] I see in the plan in mw.org stage 2 says maybe the pending store will be used for the limbo messages [18:59:00] but that seems like it'll be un-PCI [18:59:19] unless we move orphan slaying [18:59:31] ejegg: ah ha yes exactly [18:59:35] which actually seems like a good idea [18:59:54] so, orphan slaying as a smashpig job [19:00:08] everything reads from pending [19:00:12] https://phabricator.wikimedia.org/T131275 [19:00:17] yep [19:00:20] actually--no [19:00:42] awight, ejegg : i wanted to mention, that error when i was trying to show off frig was totally legit and if i would have read the error message it would have been obvious, it was because i didn't push the test change to remote so composer install was saying 'i can not find that sha' [19:00:44] only the pending consumer reads from pending, and it moves these into mysql [19:00:56] then everything reads pending messages from the db [19:01:02] ah right, I meant the db [19:01:27] cwd yah, that makes sense [19:01:39] cwd: ah sorry to get twitchy and ask you to disable the command! [19:01:57] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Write pending queue consumer and schema - https://phabricator.wikimedia.org/T133197#2257465 (Ejegg) a:Ejegg [19:02:32] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Write pending queue consumer and schema - https://phabricator.wikimedia.org/T133197#2224770 (Ejegg) [19:02:53] no worries! it's weird new territory so good to be ginger [19:03:14] any reason the pending queue consumer shouldn't be a smashpig job? [19:03:51] ejegg: caveat, you might consider doing this task at the same time: https://phabricator.wikimedia.org/T133965 [19:04:05] heh, I was just thinking that [19:04:16] ejegg: well, smashpig doesn't know about dbs yet [19:04:19] oh right, I'll want a dequeueLoop! [19:04:32] awight: ah, true [19:04:51] RelationDataStore [19:04:59] *Relational [19:05:02] I think I convinced myself that dequeueloop means we don't need to use popAtomic or pop outside of the loop implementation [19:05:09] grr i donno [19:05:15] It makes my hackles raise [19:05:41] Because all it really needs to be is a KeyValueStore? [19:05:48] but totally makes sense if we're doing the capture job from sP anyway [19:06:00] Hmm. I think it's more than K/V [19:06:10] we'll want many indexes [19:07:53] I really like keeping the pending data clear of the CRM, though [19:08:51] the code only ever needs to get/put by correlation ID [19:09:07] ah right, the cleanup job needs to get by age [19:09:18] and I think we need gateway + account? [19:09:44] correlation ID has gateway in it now [19:10:05] oh. tangentially, I just noticed that we're struggling to get contribution.trxn_id to play UUID, but contribution_tracking could be exactly that. [19:10:35] sure, that would be nice [19:10:36] it [19:10:45] or implode( ".", [ct_seq, api_seq]) sigh [19:10:53] nvm [19:11:03] s terrible that some gateways enforce a format on what the merchant reference can be [19:11:09] hehe [19:12:24] Is there any advantage in exposing pending db records in the CRM? [19:12:47] hmm, maybe for donor services [19:12:59] but I think you're right to want them in a whole nother db [19:13:04] Things will only hang out in there when we're waiting for e.g. a cash payment [19:13:27] to make sure they don't persist longer than we want em to [19:15:29] MBeat: What would you think about having a peephole into the pending messages, which we know about but have not been captured by the payment processor yet? [19:16:08] sounds good awight, any processor in particular? [19:16:53] MBeat: From all processors--we're trying to decide where to put the transient data about transactions that are waiting for extra steps [19:17:49] having it be viewable data would be cool, for sure [19:18:37] awight: ah, right, orphan processor will need to get by age + gateway [19:19:36] so, maybe not RelationalDataStore, but at least tabular [19:20:23] ehh, is there any compelling reason not to just use PDO for the consumed data? [19:20:54] MBeat: Thanks! If you get curious, feel free to subscribe to the epic about this... https://phabricator.wikimedia.org/T130897 [19:21:22] ejegg: PDO seems just right [19:21:33] word [19:21:39] Then we can use Oracle :p [19:21:44] or Azure! [19:21:45] no but seriously, yes. [19:21:50] hehe [19:21:57] omg I just got the "Assure" pun in that [19:22:06] wat [19:22:10] thanks! [19:22:46] oh man, as in we azure you your data is safe in our cloud [19:23:02] * ejegg <-slow on the uptake [19:23:16] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Write CiviCRM extension to view pending db records - https://phabricator.wikimedia.org/T134191#2257493 (awight) [19:23:25] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, Epic, and 3 others: [Epic] Clean up "pending" queue usages - https://phabricator.wikimedia.org/T130897#2257506 (awight) [19:23:34] (not putting that in the project) [19:23:52] eggsellent :p [19:24:06] cool, nice bonus feature but shouldn't hold up the core work [19:24:57] yah [20:07:46] (PS1) Cdentinger: wire 'cancel' in to another spot [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 [20:29:54] sorry guys - I could have sworn I checked last night & there wasn't a standup in my calendar [20:32:06] no worries! [20:32:32] eileen: it's all good. is everything ready for later today? You down to do this update? [20:37:56] could i pester anyone into taking a look at https://gerrit.wikimedia.org/r/#/c/286514/ ? will happily reciprocate CR, really wanna close this worldpay bug [20:47:45] (PS6) Cdentinger: Persist state/province selection, localize prompt [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/284623 (https://phabricator.wikimedia.org/T133253) (owner: Ejegg) [20:48:27] dstrine: yep! [20:48:37] (CR) Ejegg: [C: 2] New interface for atomic reads [wikimedia/fundraising/php-queue] - https://gerrit.wikimedia.org/r/284977 (https://phabricator.wikimedia.org/T131271) (owner: Awight) [20:48:57] sure thing cwd [20:50:06] thanks! [20:50:23] dstrine: i bet this would be relevant to burners' interests https://vimeo.com/126872053 [20:53:40] (CR) Ejegg: "I think you also want to add logic in the $total_missing['negative'] loop so that txns which were cancelled before they ever got to Civi d" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 (owner: Cdentinger) [20:54:54] (Merged) jenkins-bot: New interface for atomic reads [wikimedia/fundraising/php-queue] - https://gerrit.wikimedia.org/r/284977 (https://phabricator.wikimedia.org/T131271) (owner: Awight) [20:55:09] ejegg: thanks! there are a lot of checkpoints to find where the new status is supposed to live, and i don't have a good setup to test this locally [20:55:23] yep! [20:55:56] I remember speculating about that particular gotcha when I was first thinking about the cancel type [20:57:41] i would hope there's no case where we'd get the cancel line in the audit, then import the positive txn later [20:57:56] dunno, though [20:58:59] bah, we really need to keep track of gateway status + timestamp and make sure we're keeping the latest [21:00:03] there's a function that would do that (wmf_civicrm_set_gateway_status) but it's not being used and looks like it would breakthe php version we're using on prod [21:01:32] and... it hasn't been used since the code's been in git [21:01:43] heh [21:02:38] well, importing cancels only when the main exists and ignoring otherwise is still an improvement on existing behavior [21:02:55] i have run these files a bunch of times and haven't noticed any refunds or chargebacks getting blown away be rereading the initial stuff [21:03:00] so hopefully that won't be an issue [21:03:30] right, for those two types, we keep the file around if their parent txn is not yet imported [21:03:43] and try again tomorrow in the hopes that the main txn will be there [21:05:00] ah, so on some of these i see only an auth and a cancel [21:05:16] no capture [21:05:18] so if we merged that patch as is, cancels with no parent would make a lot of files stick around forever [21:05:24] ejegg: wanna see dedupe? [21:05:26] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising-Backlog, FR-Adyen: Adyen form for French (France), Japanese (Japan), Ukrainian (Ukraine), & Hebrew (Israel) - https://phabricator.wikimedia.org/T128812#2257769 (XenoRyet) @DStrine Here's that screenshot o... [21:05:30] but the auth is always before the cancel [21:05:37] eileen was demoing in hangouts... [21:05:56] oh yeah? I'll hop on and lurk [21:13:06] Fundraising Sprint Hermit Crab Husbandry, Fundraising Sprint Internet Exploring, Fundraising-Backlog, FR-Adyen: Adyen form for French (France), Japanese (Japan), Ukrainian (Ukraine), & Hebrew (Israel) - https://phabricator.wikimedia.org/T128812#2257792 (DStrine) Thanks @XenoRyet @Amire80 we w... [21:16:05] cwd: that's an interesting parade [21:17:46] eileen: and awight will you need to take campaigns down? [21:18:02] the secret is probably meth instead of acid [21:18:36] O_o [21:18:48] dstrine: yes.. I will send a reminder now [21:19:40] I have a hard stop at 5 today. awight are you able to manage campaigns? [21:19:55] dstrine: for sure, thanks for the reminder [21:20:34] awight: thanks! [21:22:51] ejegg: is what you mean that a cancel with no parent should not get added to $remaining['negative'], because that would cause it to remain in incoming? [21:26:16] yep, exactly! [21:26:25] cwd [21:26:55] ok thanks! i think i found the spot. so i should just make it do nothing instead? [21:27:18] I think that'd be fine for now [21:28:35] Not sure what the best other option is [21:29:34] Fundraising-Backlog, FR-PayPal-ExpressCheckout, FR-Paypal: Enable Express Checkout on production account - https://phabricator.wikimedia.org/T134201#2257814 (awight) [21:40:30] hi eileen, the outage plan looks great [21:41:27] Jeff_Green: cool - it's most what we put on the wiki & then a couple of extra steps [21:41:47] I forgot to mention crossing fingers [21:41:53] (PS2) Cdentinger: wire 'cancel' in to another spot [redux] [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 [21:42:41] how much time approx do you anticipate between 12-15? [21:43:32] well assuming the wait is only for the sql, not to finish the bottle - 2.5 hours [21:43:45] is around what it took on staging [21:44:02] ok cool [21:47:34] I'm going to head out for an hour or so, will be back in time for the thing [21:47:51] (CR) Ejegg: "Yep, that's the spot! Just missing a parameter for your new record_is_cancel call." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 (owner: Cdentinger) [21:50:27] (PS3) Cdentinger: wire 'cancel' in to another spot [redux] [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 [21:50:50] ejegg: derp...the joys of pushing code without running it [21:51:08] on that note, do you have a good setup for testing this? [21:51:17] it seems like a bear [21:51:35] It would almost be worth the vagrantization, if we weren't getting rid of log parsing complexity in the nonce [21:51:55] just the audit unit tests in crm [21:53:28] (CR) Ejegg: [C: 2] "That oughtta do it!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 (owner: Cdentinger) [21:53:54] thanks! [21:54:05] anytime [21:54:18] guys this is the trigger sql - I updated it yesterday & ran it locally & it worked apart from custom fields & a couple of fields that are on staging / live but probably should not be (civicrm_contribution.note) - I was about to ask Jeff to try running it on staging but I missed him. But, I guess he can trial run it while we are creating the log tables. Anyway - it might be better to get it merged first https://gerrit.wikimedia.org/r/ [21:57:48] gotta relocate, be back in a few! [21:58:50] (Merged) jenkins-bot: wire 'cancel' in to another spot [redux] [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286514 (owner: Cdentinger) [21:59:08] (PS1) Eileen: CiviCRM submodule update, country search fix [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286531 [21:59:12] hey Jeff_Green, would you mind running that same command from earlier again? [21:59:39] i'm wearing those files out [21:59:52] cwd: I think he may have run away for an hour. U could risk it from the one-off Jenkins job, I guess? [22:00:21] heheh, been waiting for a good excuse [22:00:42] He'll be back with vodka tho, so will be pliable if you still need something [22:00:44] oh yeah i see where he said that [22:01:00] sorry for the ping jeff [22:02:39] (CR) Eileen: [C: 2] "Submodule update, self-merging" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/286531 (owner: Eileen) [22:03:38] (PS1) Cdentinger: Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/286533 [22:03:57] eileen: are you about to deploy crm? [22:04:08] i am just merging a thing [22:06:00] cwd I was about to ask you about it [22:06:02] (CR) Cdentinger: [C: 2] Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/286533 (owner: Cdentinger) [22:06:16] I have a really minor civi patch I thought I'd deploy [22:06:24] yes this one is minor too [22:06:29] (Merged) jenkins-bot: Merge master into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/286533 (owner: Cdentinger) [22:06:32] wire 'cancel' in to another spot [redux] [22:07:06] cwd did that pick up my change too? maybe not? [22:07:38] doh sorry i should have waited [22:07:50] i can do a one command crm deploy now [22:07:58] :-) [22:08:09] if you haven't done it yet I'll merge the civi one in [22:08:33] i have not, go for it [22:09:44] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/286536 [22:10:47] (CR) Eileen: [C: 2] "Merge to deployment, self-approving" [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/286536 (owner: Eileen) [22:10:56] cwd it's merged now [22:11:28] eileen: i am fine to deploy whenever [22:11:31] want me to do it? [22:11:34] go for it! [22:11:50] dstrine: have you just associated our outage with Mission Impossible? [22:12:02] no airplane [22:12:08] that might be worse [22:12:11] was it a disaster movie? [22:12:47] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Advanced search by country fails with database syntax error - https://phabricator.wikimedia.org/T133796#2257983 (Eileenmcnaughton) [22:13:29] !log updated crm from b386a6821c71310950ccdcdcf2616add727e1af4 to f5e8f98d07a2280118b7153bc342bf52ee67edd5 [22:13:35] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:13:44] thanks cwd [22:13:51] np! [22:13:57] eileen: It's a super old movie: http://www.imdb.com/title/tt0080339/ [22:14:13] so i think i am going to hack the gibson with the jenkins website [22:14:26] i just typoed junkins and i like it [22:14:29] oh right - that's the one with the fish or chicken food choice... [22:14:33] :-) [22:15:34] eileen: exactly... the pilot with the drinking problem asks them stop pressuring him and that guy walk in and delivers those lines. [22:15:49] that movie has like 6 jokes a minute [22:15:53] Fundraising Sprint Internet Exploring, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Advanced search by country fails with database syntax error - https://phabricator.wikimedia.org/T133796#2257996 (Eileenmcnaughton) @CaitVirtue Do you to confirm this is all good now? [22:16:19] They don't make em like that anymore.... [22:19:03] awight: do you want me to keep the contents of the one off job somewhere? [22:19:11] so there's a record [22:19:41] cwd: I think there's something in the job config already, like separate shell tasks? [22:20:03] * awight cannot get foot from under Barbaloot to get usb key [22:22:30] (PS8) Awight: Initial reorganization of js for new forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/284508 (https://phabricator.wikimedia.org/T86248) (owner: Ejegg) [22:22:54] (CR) Awight: [C: 2] "Validation still works, nicely sutured!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/284508 (https://phabricator.wikimedia.org/T86248) (owner: Ejegg) [22:24:28] hmm, you can have separate steps but they'll all run [22:24:33] and i don't want to run the previous one [22:24:54] meh just comment the previous stuff, I guess? [22:25:06] that works [22:25:37] I thought there was something better, but it's not the biggest deal I suppose--the logs echo what commands were run anyway, one can hope [22:25:52] (Merged) jenkins-bot: Initial reorganization of js for new forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/284508 (https://phabricator.wikimedia.org/T86248) (owner: Ejegg) [22:28:37] hmm dang it sh: 0: Can't open /home/cwdent/wpfiles [22:28:47] does it not want to work outside some jail? [22:29:24] I'm pretty sure they're in nothing like a jail. This is the wild west [22:29:25] thanks awight ! [22:29:54] ejegg: Great to see the ship on stilts [22:30:29] heh [22:31:29] permission denied. does this thing su to my user to run commands? [22:31:44] it does say Started by user cwdent at the top [22:32:07] and yeah for some reason it wouldn't run something out of home but doesn't mind tmp [22:39:34] civi is going down in 20m right? [22:39:35] It runs as the service user. but can't access any home dirs for that reason [22:39:50] ah i thought it ran as jenkins [22:39:51] Puppet sets them to 0700 [22:39:56] it should [22:40:06] ~jenkins is the home dir you can't read? [22:40:14] err, that's not even a thing [22:40:27] you are reading from your own home dir? [22:40:45] well now i'm reading from tmp [22:40:57] but the files i'm trying to move are owned by jenkins [22:42:41] you can whoami from the script I suppose [22:47:57] oh well, who ever it is they ain't no good [22:48:31] maybe the parent directories have bad perms? [22:48:57] there's definitely some weird stuff going on with perms in there [22:49:14] ejegg owns a few files...do you know how that happened? [22:49:15] My motto is, umask 0002 [22:49:39] I don't know, probably running the downloader from CLI [22:50:22] I think that happened when I moved a couple files [22:50:32] that kept causing timeouts [22:50:35] how did you move them though? i can't touch anything in there [22:50:46] eileen: Think we should throw the checklist onto an etherpad? [22:50:52] From incoming I was just able to mv files [22:51:09] really? can you do that in the worldpay directory? [22:51:10] awight: good idea [22:51:13] eileen: Also--I'm going to take campaigns down now, if you agree [22:51:17] yep [22:51:21] k [22:52:18] * awight grimaces at wiki loves earth campaigns for each country [22:52:19] https://etherpad.wikimedia.org/p/outage [22:52:22] thanks! [22:54:39] well...oh well, i'm just gonna save this for tomorrow then [22:54:57] CAmpaigns are down [22:55:35] cwd lemme check [22:55:47] ejegg: sorry [22:55:51] I meant to say, [22:56:06] !log CentralNotice fundraising campaigns disabled for CiviCRM outage [22:56:13] ejegg: oh no rush, i think civi is about to go down anyway right? [22:56:14] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:56:21] cwd ah, true [23:00:00] (ejegg: ack, please ignore me!) [23:00:22] heh, no worries [23:04:25] awight: how come the extensions aren't submodules in vagrant/mediawiki-fr? [23:04:35] They... err [23:04:53] Interesting, that should be a conflict [23:05:17] They're done with the puppet mediawiki::extenion module, but that means the branch wasn't recursively cloned, eh? [23:05:18] wait...now they are [23:05:19] what did i do [23:05:27] maybe ran submodule update? [23:06:03] I wonder what we should do. [23:07:11] that was weird [23:07:14] i don't know what i did [23:07:17] but everything is fine now [23:07:26] cwd define, "fine" :) [23:08:26] eileen: Can you confirm that step 9 is to drop civicrm.log_civicrm? [23:08:29] Or is that staging? [23:08:42] yeah that's the extra db we created last week [23:08:50] ooh right [23:08:51] we put log_civicrm_setting table in it [23:09:03] but we have agreed to use the same DB in the end [23:09:10] k great [23:09:25] that needs to be done by Jeff I guess [23:09:26] so this is mostly cleanup. Seeing if I can get myself perms to do it... [23:09:45] Is the slow query killer off ? [23:10:24] ah. yeah we need to do that step in order. [23:10:28] i don't think we have a slow query killer any longer...but I will double check [23:10:30] It will also cause the site to fail [23:10:47] oh, I see you put into maintenance mode already [23:11:00] awight: partial, i didn't do the proxy flip part [23:11:03] (CR) Ejegg: "Sure does look better than all those silly array() s. Noticed some oddness in the cache key that's not the fault of this commit, and Maint" (5 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/285317 (https://phabricator.wikimedia.org/T133601) (owner: Awight) [23:11:04] is there anything that kills long php processes? [23:11:36] they kill themselves ;) [23:11:39] php timeout is the only thing I can think of that would [23:11:45] well there's that too :-) [23:11:57] OK - which doesn't affect cli so we should be ok [23:13:40] so, I think I can start & kick off the slow table create stuff… [23:13:52] Nice and smooth takeoff, so far [23:14:16] yeah, I don't think we have used any external query slayer in like 2 years [23:14:20] * awight resumes poking around in DonationInterface code [23:14:24] hehe good to know [23:14:30] OK - so I should pull the trigger on step 10? [23:14:34] the script is still in root's homedir on db1025, but it was manually invoked [23:15:01] eileen: I haven't done step1 yet [23:15:12] ah - yeah!! [23:15:16] lemme disable icinga alerts too [23:15:32] i appear to have striked them before actually doing them... [23:15:51] * awight ceases wondering about that :) [23:16:05] I can't be trusted [23:16:06] :-) [23:17:36] ok mysql monitoring-alerts are disabled [23:18:13] Actually need to think about the log_civicrm - if we drop that before dropping the 3 triggers then it will not be able to run the update [23:18:36] youch [23:18:49] I think it's ok to do 9 after 10 [23:18:57] don't we want to drop those triggers anyway? [23:19:01] yeah we do [23:19:10] & I was going to say the script would do it - but it won't [23:19:20] should we just do that now, and drop that db too? [23:19:25] ejegg: I see what you were saying--AstroPay is the right way to do nvp + redirect, gonna try to rewrite legacy paypal to be similar [23:19:40] cool! [23:20:01] Jeff_Green: yep - steps 9ish & 9 are on you... [23:20:02] regarding step 1 we have two dumps of civicrm from around 9:30 AM today [23:20:11] good enuf :D [23:20:22] from two different db machines, they're approx the same size but not exactly...as expected [23:20:42] appear to contain normal-lookin sql, so that's something [23:21:25] Always a good sign [23:22:17] also I stopped replication to fdb2001, we can leave that disabled until we see everything isn't broken and in flames [23:22:47] thinking thinking.... [23:24:40] do we need to put payments in maintenance mode, or is it ok to continue to have writes to the drupal db? [23:24:55] Let's do maintenance mode, thanks. [23:24:58] k [23:25:11] (CR) Ejegg: "Also, want to update composer.lock along with .json?" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/285317 (https://phabricator.wikimedia.org/T133601) (owner: Awight) [23:25:25] It's okay, but I agree with your instincts to avoid writing to other dbs whenever we're doing dramatic enough maintenance to stop replication [23:25:35] does any of the payments-listener stuff write to db? [23:25:46] No [23:25:52] k [23:26:00] Good question! [23:26:30] ejegg: ^ that's an interesting point, regarding our pending conversation from earlier. [23:26:47] Putting payments listeners into maintenance mode is something we really want to avoid. [23:27:08] awight: what's the downside? [23:27:35] Jeff_Green: It's one of our lines of defense around accurate data [23:27:36] I think all the processors will retry till they get a successful response [23:28:01] That might be true [23:28:10] though I guess it could be seen as rude [23:28:14] I'm pretty sure I remember seeing PayPal retry monthly, even [23:28:17] would there be significant transaction attempts if payments are disabled? [23:28:25] only monthly?! [23:28:35] ejegg: what if we have a very polite and apologetic error page :-) [23:28:54] Jeff_Green: some of the IPN messages are about things like boletos clearing [23:29:00] Jeff_Green: These can come from any time in the past, though [23:29:02] So they can come at any time [23:29:09] ok payments should go maintenance in 5 minutes max [23:29:40] We definitely disable the listeners for queue outages [23:29:42] It's not a show-stopper, it's just that without listeners we only have the audit processors, and that leaves lots of little gaps to lose transactions in [23:29:50] yep [23:29:59] I don't remember how to do that, fwiw... [23:30:08] ok, well we're fine with them up for this project anyway right? [23:30:13] yep! [23:30:45] Amazon IPNs retry 1x / hour for 14 days [23:31:07] that's quite polite! [23:31:38] Should be enough time to drain the lake [23:31:52] eileen: should I remove the triggers now? [23:32:00] Adyen does a back-off from 5min through 8 hrs, then again every 8 hrs for a week [23:32:07] Jeff_Green: yep [23:32:19] * ejegg should be writing this down [23:32:46] don't stop channeling! :p [23:34:07] ok triggers are removed [23:34:08] Huh, I honestly had no idea the IPN thing was giving such a fine SLA [23:34:28] eileen: ok to drop log_civicrm? [23:34:30] Cool Jeff_Green so if you can remove that extra logging db [23:34:32] yep [23:34:49] done [23:35:05] OK - all ready for me to attempt to kick off the slow thing [23:35:19] ok [23:35:40] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Find a way to backup triggers - https://phabricator.wikimedia.org/T134212#2258250 (awight) [23:36:28] show create table log_civicrm_activity; looks good [23:37:53] Jeff_Green: so now we have a long wait but I wanted to get you to re-run the trigger script on staging during this - step 12 [23:38:11] eileen: ok [23:38:13] & you might as well do that follow step too… [23:38:32] Fundraising-Backlog, FR-Astropay: AstroPay: internal exception caused by currency - https://phabricator.wikimedia.org/T134214#2258277 (awight) [23:38:44] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, FR-Astropay: AstroPay: internal exception caused by currency - https://phabricator.wikimedia.org/T134214#2258289 (awight) [23:38:46] I don't know if it's easy to grab that mysql from the gerrit link - might be easier if merged first & then pulled onto staging? [23:39:30] eileen: I usually deploy in review patches using the gerrit checkout commandline in the GUI [23:39:43] ah yeah - I'll put it on staging [23:39:43] I can probably figure it out either way, but I can't figure out which is preferable as a way of doing things [23:39:53] e.g. git fetch https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm/civicrm refs/changes/90/286390/1 && git checkout FETCH_HEAD [23:40:04] yeah I got what you meant [23:40:15] * awight grins sheepishly [23:40:38] it just sounded horrible so I wasn't sure if it was language [23:41:06] any sentence with gerrit sandwiched between two verbs bodes ill. [23:41:42] (PS2) Awight: Updated MySql trigger code [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286390 (https://phabricator.wikimedia.org/T133745) (owner: Eileen) [23:41:53] (CR) Awight: [C: 2] "Sorry for the delay!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286390 (https://phabricator.wikimedia.org/T133745) (owner: Eileen) [23:42:18] Jeff_Green: /srv/org.wikimedia.civicrm/civicrm/sql/trigger_create.mysql - on staging [23:42:26] eileen: ok [23:43:13] & then afterwards check the triggers look 'good' we are looking to see they are triggering to the same DB & they have that wierd COALLESCE construct around the @uniqueID [23:44:03] (Merged) jenkins-bot: Updated MySql trigger code [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/286390 (https://phabricator.wikimedia.org/T133745) (owner: Eileen) [23:46:18] eileen: just to make triple sure I'm sane, I'm applying that to dev_civicrm right? [23:46:24] yep [23:46:35] ok, done [23:46:46] that's a good sign - no errors... [23:47:06] there are now 435 triggers on that db [23:47:33] :-) [23:47:57] OK I just did an edit on staging & the change is in the change log [23:48:11] you should probably drop the extra staging db [23:48:30] * awight hops up and down with anticipation [23:48:58] you'll be pleases to know richard just brought me coffee - bang on in time for step 13 [23:49:20] !! [23:50:39] ohh we just get address table done - up to contact [23:51:35] awight: etherpad was a good idea [23:51:43] Always fun [23:52:31] * awight substitutes a beer for steps 13-15 [23:52:39] :-) [23:53:00] I can't progress past 13 until midday [23:53:08] 7 mins [23:53:17] that makes sense, time zone problems [23:54:41] eileen: should I drop dev_log_civicrm on lutetium? [23:54:54] Jeff_Green: yep & cleanup perms on it too [23:54:59] ok [23:55:47] eileen: Cool, I see you changed the DSN on staging as well [23:56:13] yep & with triggers re-created it's now working on staging unlike the … er demo [23:56:34] although staging is back on master branch so the merge screen is blocked again so you can't test... [23:57:12] Jeff_Green: eileen: rats, here's one for "do better next time". When we reenable jenkins, I think it's gonna be like after a blackout... All the jobs will be going at once. What we wanted was, to disable all jobs and then shutdown jenkins [23:57:33] awight: do you want to update this https://collab.wikimedia.org/wiki/Checklists_for_maintenance_%26_Recovery [23:57:52] also Jeff_Green I think you might want to add the ichinga things there? [23:57:54] yeah, thanks. I noted it in the etherpad for now. [23:57:56] awight: I think once upon a time we figured out how to hack the jobs while jenkins is down [23:58:06] oh dear, I remember something like that as well [23:58:14] the sound on that memory is all muffled though [23:58:24] think xml [23:58:33] I think we edited yeal .xml.tgz files or some such nonsense