[00:03:23] (PS7) Ejegg: Add risk score to Adyen pending queue message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/266295 (https://phabricator.wikimedia.org/T122244) [00:04:48] (PS1) Ejegg: Ensure Adyen result switcher logs correct OID [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/269339 [00:06:15] (PS4) Eileen: CRM-17951 ( merged into 4.7) add support for Chargeback contribution status and having an account relationship for refunds. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268459 (https://phabricator.wikimedia.org/T124979) [00:06:17] (PS3) Eileen: CRM-17951 form fixes for chargeback [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268618 (https://phabricator.wikimedia.org/T124979) [00:06:19] (PS4) Eileen: CRM-17951 form fixes to make chargeback more consistent with refund behaviour [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268617 [00:16:02] (CR) jenkins-bot: [V: -1] CRM-17951 ( merged into 4.7) add support for Chargeback contribution status and having an account relationship for refunds. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268459 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [00:42:34] (CR) Eileen: "recheck" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268459 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [00:57:19] (PS1) Eileen: (submitted 4.7) CRM-17999 add index to civicrm_contribution.source field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) [01:05:54] (CR) Eileen: "After running this onto staging and the other gerrit associated with this ticket I'm happy with the search perfomance (string = 'mxn%')." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) (owner: Eileen) [01:11:04] (CR) Eileen: "I'm assuming you have finished for the day now - but what I'd like to do is get the patches currently showing merged & then discuss whethe" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268459 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [01:13:42] Wikimedia-Fundraising-CiviCRM: Advise on reports - https://phabricator.wikimedia.org/T126297#2010454 (Eileenmcnaughton) NEW a:Eileenmcnaughton [01:38:37] (CR) Cdentinger: [C: 2] Ensure Adyen result switcher logs correct OID [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/269339 (owner: Ejegg) [01:39:59] (PS8) Cdentinger: Add risk score to Adyen pending queue message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/266295 (https://phabricator.wikimedia.org/T122244) (owner: Ejegg) [01:40:05] (CR) Cdentinger: [C: 2] Add risk score to Adyen pending queue message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/266295 (https://phabricator.wikimedia.org/T122244) (owner: Ejegg) [01:46:49] (Merged) jenkins-bot: Add risk score to Adyen pending queue message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/266295 (https://phabricator.wikimedia.org/T122244) (owner: Ejegg) [10:12:43] Fundraising-Backlog: No banner impression data since 4 Feb - https://phabricator.wikimedia.org/T126228#2010962 (Pcoombe) Open>Resolved a:Pcoombe It seems to have caught up now. Thanks again! [10:36:42] fundraising-tech-ops: db1008 (and fdb2001) periodically lags every day at the same time ~10:30UTC - https://phabricator.wikimedia.org/T126314#2011008 (jcrespo) NEW [18:50:47] (CR) Eileen: [C: 1] "Well I don't know this code well but the patch makes sense & does what the commit message says" (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/268025 (https://phabricator.wikimedia.org/T117816) (owner: Ejegg) [19:00:32] Fundraising-Backlog: Silverpop export not capturing language changes in Civi (not sure if this is language-only or not) - https://phabricator.wikimedia.org/T96410#2012149 (CCogdill_WMF) This problem is recurring. We don't seem to get a lot of donor complaints about it, but we'll never know for sure how many... [19:08:59] (CR) Ejegg: "Hi again," [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/263438 (owner: Mhutti1) [19:20:05] (CR) Nemo bis: "That's acceptable but not a real shortening: you just use the genitive as "pronominal". I think Siebrand didn't propose "your country's" a" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/263438 (owner: Mhutti1) [19:36:23] ejegg: should we talk refunds at some point - I think we are almost there ... [19:37:02] sure! Was distracted by other stuff, lemme just remind myself which one I was looking at next [19:37:46] :-) [19:38:08] ejegg: it's a bit of a marathon! [19:39:15] this contribution on staging civicrm/contact/view?reset=1&cid=12212649 has been updated using the csv refund script [19:39:29] cool, I'll take a look! [19:40:48] Oh, did you have some plans for handling the amount differently when there is no contra/chargeback account and we fall back to using the income account as to_financial_account on the refund trxn? [19:41:21] I guess that won't matter for us... [19:42:26] (CR) Ejegg: [C: 2] "Looks good!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268459 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [19:42:53] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Cat Herding, Fundraising Sprint William Shatner, and 5 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#2012317 (CCogdill_WMF) @Ejegg I'm not sure what I'm... [19:46:20] ejegg: I want to talk to you about that [19:46:36] if you look at civicrm/contact/view?reset=1&cid=12212649 [19:46:39] on staging [19:46:54] I have processed a refund for that person using the csv import process [19:47:05] so that uses mark_refund [19:48:07] Optionally we can assign an account - so I'll do that now in the UI for financial_type = cash [19:48:16] OK, I see the negative amount with the 'chargeback' status [19:48:33] that looks correct! [19:49:54] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Cat Herding, Fundraising Sprint William Shatner, and 5 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#2012339 (Ejegg) All the params on that email look g... [19:51:05] ejegg: so, by comparison - I just added the chargeback account relationship with 'CASH' - you can see that at civicrm/admin/financial/financialType/accounts?reset=1&action=browse&aid=9 [19:51:36] and I did another refund - this time through the UI but mostly the same deal - on civicrm/contact/view?reset=1&cid=14815253 [19:52:11] and you can see the 'Type' (Actually account as you astutely picked up in CR) shows Chargeback for the negative row [19:52:19] oh, that first was with no chargeback account? [19:52:34] looking at the other... [19:52:53] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Cat Herding, Fundraising Sprint William Shatner, and 5 others: Track email clickthroughs on donate wiki - https://phabricator.wikimedia.org/T114010#2012352 (CCogdill_WMF) Ah good thinking on the outa... [19:53:57] so the UI way has a different type displayed and the txn ID on both. I think both of them are pretty clear and accurate as far as we use the system! [19:54:24] I'm still pretty hazy on how actual accountants would use the 'type' [19:56:27] I don't see much visibility into accounts via the UI anyway [19:57:33] ejegg: no - it's not ui vs code - I made config change thru the ui [19:57:51] between the 2 [19:57:59] ah, right. [19:58:23] I haven't put that change into the upgrade script as I was on the fence about whether it added value [19:58:56] the trxn id is a field in the UI & I accepted the default (ie. same as the original) [19:58:59] I figured that would cause the 'type' display change, but thought maybe the UI form processing was responsible for copying the txn id over [19:59:15] heh, right, what you just said [19:59:20] :-) [20:01:39] I guess the advantage of doing the config is that for refunds (but not chargebacks) at the financial_item level it's a bit more consistent with historical ones - ie civicrm/contact/view?reset=1&cid=14815500 [20:02:19] yeah, that makes sense [20:02:28] k, gonna hop in this backlog meeting [20:11:27] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Advise on reports - https://phabricator.wikimedia.org/T126297#2012444 (DStrine) [20:11:56] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Sync our buildkit with upstream - https://phabricator.wikimedia.org/T126266#2012450 (Eileenmcnaughton) [20:16:34] Fundraising Sprint Cat Herding, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Advise on reports - https://phabricator.wikimedia.org/T126297#2010454 (DStrine) [20:19:40] Fundraising-Backlog, Wikimedia-Fundraising, MediaWiki-extensions-CentralNotice, Documentation: Banner and donatewiki style guide documentation needs updating - https://phabricator.wikimedia.org/T119821#1836550 (DStrine) @Pcoombe are you able to address this? [20:22:14] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Technical-Debt: Fundraising scripts should not use global scope - https://phabricator.wikimedia.org/T120084#2012522 (atgo) a:Pcoombe Hey @pcoombe - assigning to you per your last comment. Please unclaim if this is actually an fr-tech task. [20:25:35] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [Epic] Repair historical refunds to be consistent with new style - https://phabricator.wikimedia.org/T124983#2012546 (DStrine) Open>Invalid [20:25:37] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 3 others: [Epic] CiviCRM upgrade: Adapt refund processing & reporting to reflect changes since the upgrade. - https://phabricator.wikimedia.org/T116317#2012548 (DSt... [20:25:43] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [Epic] Rewrite automatic refund logic to perform a built-in Civi refund - https://phabricator.wikimedia.org/T124981#2012556 (DStrine) Open>Invalid [20:25:45] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint X-Ray Spex, Fundraising Sprint Yo La Tengo, Fundraising Sprint Zapp, and 3 others: [Epic] CiviCRM upgrade: Adapt refund processing & reporting to reflect changes since the upgrade. - https://phabricator.wikimedia.org/T116317#1775815 (DSt... [20:25:47] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Performance: Fundraising stylesheets and scripts should be minified - https://phabricator.wikimedia.org/T120083#2012560 (Ejegg) We're gradually adding functionality to Central Notice's campaign mixins to replace this sort of on-wiki javascript.... [20:32:07] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Create CiviCRM naming convention rule: In general, records should not begin with "The" - https://phabricator.wikimedia.org/T122592#2012598 (DStrine) [20:33:08] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, FR-Astropay: Prepare logos for AstroPay LATAM banks and cards - https://phabricator.wikimedia.org/T124320#2012605 (DStrine) [20:33:33] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, MediaWiki-extensions-DonationInterface, FR-Astropay, FR-Smashpig: Update payment methods and submethods for AstroPay LATAM rollout - https://phabricator.wikimedia.org/T124323#2012613 (DStrine) [20:33:43] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Create CiviCRM naming convention rule: In general, records should not begin with "The" - https://phabricator.wikimedia.org/T122592#2012614 (atgo) Hey @caitvirtue - we just talked through this one. What you're asking for is slightly complicated with the... [20:35:20] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, FR-Astropay: GatewayAdapter::getCurrencies should return currencies supported for the donor's country - https://phabricator.wikimedia.org/T124327#2012617 (DStrine) [20:57:42] ejegg: I'm leaning towards adding those account relationships (Chargeback/ Credit Contra) because I think it's probably more consistent with the way the earlier ones look [20:58:01] yeah, that sounds like the right way to go [20:58:28] ejegg: ok - I'll add another upgrade function for that [20:58:42] thanks! [20:59:29] NB re importing the existing ones - how does the process work in terms of michael provides a csv & we just log onto live, upload the csv & run the drush command ? [21:00:09] yeah, we haven't done it very often, but I think that's it [21:02:05] ok [21:04:01] composer vendor thing update from upstream https://github.com/composer/composer/issues/4717 [21:11:24] oh, that annoyance... [21:12:47] Reedy: thanks for keeping that in front of the composer folks! [21:12:58] heh [21:13:28] Their 'well you shouldn't be checking in the vendor directory' answer is such a cop-out [21:14:33] (PS1) Eileen: Add Financial Accounts to each Financial Type for Chargeback & Refund [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) [21:15:11] (CR) jenkins-bot: [V: -1] Add Financial Accounts to each Financial Type for Chargeback & Refund [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:16:05] (CR) Ejegg: [C: 2] CRM-17951 form fixes to make chargeback more consistent with refund behaviour [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268617 (owner: Eileen) [21:22:39] Jeff_Green: in the not too distant future I need to experiment on staging with turning logging on - which means adding a bunch of triggers [21:22:51] (CR) Ejegg: "Should this use cancelInfo_show_ids? Or does that relate to the patch for 17589?" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268618 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:23:11] (preferably I would try doing it through the UI) [21:23:57] I'm assuming that I'll need to arrange some permissions to do it... [21:24:19] to do that we would need to grant the civi user full access to the mysql instance [21:24:43] triggers cross databases, so it's not as though we can restrict it to just the dev_civi/dev_drupal instance [21:25:02] (PS2) Ejegg: (submitted 4.7) CRM-17999 add index to civicrm_contribution.source field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) (owner: Eileen) [21:25:10] ah r... [21:25:13] (CR) Ejegg: [C: 2] (submitted 4.7) CRM-17999 add index to civicrm_contribution.source field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) (owner: Eileen) [21:25:59] Jeff_Green: so really you want me to come up with the sql locally and for you to run it to create the triggers [21:26:12] I think that's preferable yeah [21:26:16] OK [21:26:29] also...I haven't come up with a sane way to dump the database and restore it with triggers [21:26:35] we're excluding them from dumps now [21:26:46] when we try to include them it breaks the entire reload [21:26:49] ie using —skip-triggers? [21:26:52] yup [21:27:15] (CR) jenkins-bot: [V: -1] (submitted 4.7) CRM-17999 add index to civicrm_contribution.source field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) (owner: Eileen) [21:27:27] yeah - I've always done that too - but we normally can then recreate easily through the UI as we don't normally have the limitations around replication [21:27:32] otherwise the restore fails on the triggers, it seemed to have something to do with the delimiter but I didn't manage to conclusively determine whether it's our syntax or a difference in mysql version [21:27:58] I haven't checked but presumably it's possibly to backup just triggers too [21:28:06] yeah it probably is [21:28:13] but to me triggers are code, not data [21:28:39] (Merged) jenkins-bot: (submitted 4.7) CRM-17999 add index to civicrm_contribution.source field. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269343 (https://phabricator.wikimedia.org/T123418) (owner: Eileen) [21:28:46] philisophically they should be treated as a code deployment [21:28:59] but that's just my opinion :-) [21:29:25] yeah - adding replication into the mix certainly adds weight to your argument [21:30:04] also when they get deployed, presumably we suddenly we start writing data differently, so... you'd want a full stop, adjust triggers, full start [21:30:28] When the logging gets turned on it adds triggers - but it also creates a shadow copy of most tables - which would take a long time! [21:30:52] and only testing on staging will tell us whether that approach will work for the WMF dataset [21:31:10] can you explain more re. shadow tables? [21:31:19] are they temporary? [21:31:25] no! [21:31:35] do they double the data footprint? [21:31:40] yes! [21:31:53] at least for the tables we track [21:32:00] which tables do we track? [21:32:07] and really if it's to be of any use that's all the big ones [21:32:18] uh [21:32:28] ie. the point is to be able see the history of things and potentially revert them [21:32:29] * Jeff_Green checks disk capacity on the master... [21:32:33] :-) [21:32:55] we are at 70% utilization on the master [21:33:34] staging is 74% - does that mean we can't even really test it under the current resource? [21:33:34] seems like that has bloated a lot even since I last checked [21:33:57] it likely has - the 4.6 DB structure does have some more tables. [21:33:59] not sure, i will investigate [21:34:32] the master has a ton of binlogs, seems like more than expected [21:35:07] ok.. [21:35:58] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:35:59] i guess we've been churning a lot, they're set to expire after 10 days but that's 196 binlog files [21:36:45] yeah that is quite a few…. [21:36:47] but it does look like it's at least doing what's expected... [21:36:54] that is a good thing :-) [21:37:38] 106GB of replication overhead [21:37:47] ouch! [21:38:19] plus there's a dump stored on that partition from just after we upgraded on 10/29 [21:38:40] is there any chance we would ever use that at this point? [21:39:05] I feel it can go - although we should probably ping awight before deleting [21:39:40] so we might be able to reduce our utilization a tiny bit but we're pretty close [21:40:05] (PS2) Ejegg: Add Financial Accounts to each Financial Type for Chargeback & Refund [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:40:46] (PS3) Eileen: Add Financial Accounts to each Financial Type for Chargeback & Refund [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) [21:41:37] are we talking about only the civicrm database? [21:41:55] Jeff_Green: hmm - so it's not terribly promising for testing something that uses a tonne more disk (obviously I only need to on staging at this stage) & yeah - just CiviCRM [21:43:11] the civi database is about 85GB [21:43:24] are the shadow tables indexed like the regular tables? [21:43:34] not normally [21:43:46] so that should reduce space quite a bit [21:43:48] but, my expectation is that we should look at indexing some of them [21:44:44] NB do you know how much space is going into indexes? There are some indexes that are much longer than they should be - but I haven't really looked into it & have no idea at the moment what performance hurtiness over-long indexes are causing [21:45:06] I should be able to figure it out but I'm not sure offhand [21:45:16] it was easier to tell in the myisam days :-) [21:45:28] (for example the pattern entity_id/entity_table gives us a long index on entity_table ) [21:45:47] blargh we have myisam tables still, gotta fix that [21:45:53] :-) [21:47:16] (CR) Ejegg: [C: 2] "Nice! The chargeback type's refund account and refund type's chargeback account probably wouldn't get much use, but there shouldn't be any" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269529 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:48:00] (CR) Eileen: "I think the cancelShowIDs is used in 4.7 but turned out actually to not apply to 4.6 & I had to do the tpl change here to adapt for 4.6 (I" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268618 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:49:07] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/261757 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [21:56:35] so very short answer: in terms of testing I don't see a problem, even if we were to double the civicrm database footprint we'd still have plenty of capacity [21:59:32] Jeff_Green: ok great - [22:00:16] I have a task to start getting on top of this that I'll be getting into shortly so you'll hear more on it - but I'll try to take it to Phab [22:00:27] ok [22:03:27] (CR) Ejegg: [C: 2] CRM-17951 form fixes for chargeback [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268618 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [22:08:26] ok i have more information regarding index space [22:11:56] Jeff_Green: yep? [22:12:18] trying to make it presentable [22:12:27] | TABLE_NAME | data | ix | total | [22:12:27] :-) [22:12:40] | civicrm_contact | 2675 | 5559 | 8234 | [22:12:40] | wmf_contribution_extra | 1809 | 6018 | 7827 | [22:12:40] | wmf_donor | 1359 | 5567 | 6926 | [22:12:45] | civicrm_entity_financial_trxn | 3192 | 3682 | 6874 | [22:12:46] | civicrm_contribution | 2380 | 4308 | 6688 | [22:12:46] | civicrm_financial_item | 2525 | 4089 | 6614 | [22:12:46] | civicrm_entity_tag | 2099 | 3130 | 5229 | [22:12:54] | civicrm_financial_trxn | 2628 | 2561 | 5189 | [22:12:54] | civicrm_address | 1147 | 3664 | 4811 | [22:12:55] | civicrm_line_item | 2033 | 2213 | 4246 | [22:12:59] in m/b [22:13:16] hmm interesting so index space > data [22:13:22] that's pretty common [22:13:23] fot most tables [22:13:26] yeah [22:13:34] these are the top ten by overall utilization [22:13:50] huh, are we tracking whether we have any unused indexes? [22:13:54] data is probably stored in the most space efficient way [22:13:57] I'm wondering that [22:14:04] I don't know how we would track that [22:14:09] we are actually adding some more indexes too [22:14:13] in the next deploy [22:14:39] we should probably fully review a tables indexes when we add a new one [22:15:11] looks like INFORMATION_SCHEMA.INDEX_STATISTICS is a thing we could refer to [22:15:23] I'm just looking at contribution table - but I don't think we can remove from there [22:16:04] ah, which requires set global userstat_running=1 [22:17:36] I can see one poorly thought out index on contact that could go [22:17:44] we're not collecting statistics at the moment [22:18:22] cool [22:19:06] I guess we should log a phab to review this [22:19:47] yeah [22:20:57] I gotta head out for the day, have a good day! [22:21:03] ok - see ya [22:23:31] (PS2) Ejegg: CRM-17951 (merged into 4.7) modify transaction view to show the financial account of the transaction [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268447 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [22:24:25] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Review CiviCRM indexes in order to reduce disk space & improve performance - https://phabricator.wikimedia.org/T126388#2013047 (Eileenmcnaughton) NEW [22:28:37] (CR) Ejegg: [C: 2] CRM-17951 (merged into 4.7) modify transaction view to show the financial account of the transaction [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/268447 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [22:30:54] ejegg: phew - only one left - although I'm working on one more data tidy-up one now... [22:31:30] nice! This last one is a bit larger, but it's looking good so far [22:33:06] ejegg: yeah the last one is really what all the others were leading up to! [23:15:21] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Cat Herding, Fundraising Sprint Zapp, and 3 others: Financial Transactions not visible for some transactions & possibly fee transactions are missing - https://phabricator.wikimedia.org/T122946#2013295 (... [23:24:32] (CR) Ejegg: [C: -1] "Cool! Just one necessary fix, a couple questions." (6 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/261757 (https://phabricator.wikimedia.org/T124979) (owner: Eileen) [23:24:42] (PS1) Eileen: Fix financial trxn records for upgraded contributions missing to_financial_account_id. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269556 (https://phabricator.wikimedia.org/T122946) [23:27:26] Fundraising Sprint Asbestos Removal 2016, Fundraising Sprint Bloodletting 2016, Fundraising Sprint Cat Herding, Fundraising Sprint Zapp, and 3 others: Financial Transactions not visible for some transactions & possibly fee transactions are missing - https://phabricator.wikimedia.org/T122946#2013326 (... [23:30:14] ejegg: do you understand this ? https://phabricator.wikimedia.org/T124842 [23:30:48] oh I know - in other modules [23:31:07] yah, make sure nothing is looking for old-style refunds [23:31:34] I can probably write at least a couple of those fixes [23:36:45] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Find and fix any references to parent_contribution_id or the "RFD" trxn_id prefix - https://phabricator.wikimedia.org/T124842#2013431 (Eileenmcnaughton) Functions that refer to parent_contribution_id globalcollect_audit_find_missing_gc_trxns wmf_civicr... [23:37:06] Fundraising Sprint Cat Herding, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Find and fix any references to parent_contribution_id or the "RFD" trxn_id prefix - https://phabricator.wikimedia.org/T124842#2013438 (Eileenmcnaughton) [23:38:35] ejegg: cool - that would be good - I thought we were so close but argh .. more [23:38:46] :$ hopefully not much [23:39:50] hmm, maybe we keep the 'RFD' convention for the secondary contributions used to balance mismatched refunds? [23:41:13] ejegg: ok - that probably makes sense [23:42:26] ejegg: is that just a case of going from 'contribution_source' => $refund_currency . " " . $amount_scammed, [23:42:34] to 'contribution_source' => 'RFD ' . $refund_currency . " " . $amount_scammed, [23:42:49] nah, the RFD goes in the gateway transaction ID [23:43:01] to make it distinct from the parent gateway txn id [23:43:25] ah ok - yeah I probably left that off because that table doesn't have a unique index on trxn_id [23:43:25] WmfTransaction->unique_id() handles that part [23:43:40] so you don't need to change anything in your current patch for RFD [23:43:49] unless you want to update that comment [23:44:05] I don't want to touch it or look at it ever again :-) [23:44:13] heh [23:44:55] huh, i thought sure we were enforcing unique (gateway, gateway_trxn_id) [23:46:10] oh weird, I do see something looking for RFD in contribution_source... That definitely needs to be updated [23:47:38] the civicrm_contribution table enforces unique trxn_id - but the civicrm_financial_trxn table doesn't [23:47:51] ah, i see [23:48:23] Actually there was no index on civicrm_financial_trxn.trxn_id - I added one & attempted to comment out the possibility of someone making it unique later [23:48:47] (but actually people very rarely make existing indexes unique anyway) [23:49:07] ok, so the balancing trxns still need a distinct trxn id though [23:50:03] at the civicrm-contribution level they will - unless we want to try to change that in core (no don't prolong the pain) [23:50:17] no, no core change! [23:50:41] we'll keep leaning on the wmfTransaction->unique_id for that [23:50:50] just use it a lot less often, only for mismatches [23:52:13] yeah - I'm having trouble facing https://phabricator.wikimedia.org/T124842 - I'm going to try https://phabricator.wikimedia.org/T124834 [23:52:46] cool! I've started on 842, I should put my name on it [23:53:30] ok! [23:53:44] (how could I say no :-) [23:54:56] Fundraising Sprint Cat Herding, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Find and fix any references to parent_contribution_id or the "RFD" trxn_id prefix - https://phabricator.wikimedia.org/T124842#2013561 (Ejegg) a:Eileenmcnaughton>Ejegg [23:56:38] (PS1) Ejegg: Don't require gateway_refund_id [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269567 (https://phabricator.wikimedia.org/T124842) [23:59:45] (PS1) Ejegg: Remove unnecessary keys from Globalcollect refunds [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/269569 (https://phabricator.wikimedia.org/T124842)