[00:00:01] will start at the start of the chain [00:19:09] whew, tricky [00:26:36] that one should be able to stand alone [00:27:11] ejegg: obviously I'd love them all merged but this one https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/608262 should be mergeable without others [00:27:29] ah, ok [00:27:39] was just reasoning through the dedupe refactor [00:27:43] (I'm finding the chains very confusing now...) [00:28:00] new interface... [00:28:24] yep yep, me too [00:28:36] oh i see, it just changes the column names [00:31:54] (PS4) Ejegg: Reorganize final export mapping fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608262 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [00:32:10] (CR) Ejegg: [C: +2] Reorganize final export mapping fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608262 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [00:32:23] cool [00:33:04] (Merged) jenkins-bot: Reorganize final export mapping fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608262 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [00:36:45] (PS1) Eileen: Prefix additional internal fields with foundation_ [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608493 (https://phabricator.wikimedia.org/T252245) [00:37:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1002:down [00:37:31] (CR) jerkins-bot: [V: -1] Prefix additional internal fields with foundation_ [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608493 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [00:38:42] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/608496 [00:38:58] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/608496 (owner: Eileen) [00:39:52] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/608496 (owner: Eileen) [00:42:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1002:down [00:47:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1005:down [00:52:12] RECOVERY - check_kafkatee on frban2001 is OK: OK: brokers:9 topics:2 [00:59:57] (PS6) Eileen: Silverpop export: refactor calculation of address and preferred_language [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607401 [00:59:59] (PS3) Eileen: Add some timings [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607624 [01:00:01] (PS5) Eileen: Silverpop refactor - reduce queries to build country & language [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607652 [01:00:03] (PS2) Eileen: Prefix all total colums in our staging tables with foundation_ [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608492 (https://phabricator.wikimedia.org/T252245) [01:00:05] (PS2) Eileen: Prefix additional internal fields with foundation_ [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608493 (https://phabricator.wikimedia.org/T252245) [01:00:07] (PS1) Eileen: Further field renaming to clarify foundation vs all funds [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608498 (https://phabricator.wikimedia.org/T252245) [01:01:21] (CR) jerkins-bot: [V: -1] Further field renaming to clarify foundation vs all funds [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608498 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [01:02:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1004:down [01:02:26] (PS2) Eileen: Further field renaming to clarify foundation vs all funds [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608498 (https://phabricator.wikimedia.org/T252245) [01:03:10] (CR) jerkins-bot: [V: -1] Further field renaming to clarify foundation vs all funds [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608498 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [01:04:18] (PS3) Eileen: Prefix additional internal fields with foundation_ [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608493 (https://phabricator.wikimedia.org/T252245) [01:06:58] (PS3) Eileen: Further field renaming to clarify foundation vs all funds [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608498 (https://phabricator.wikimedia.org/T252245) [01:07:12] RECOVERY - check_kafkatee on frban2001 is OK: OK: brokers:9 topics:2 [01:50:12] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1379 [01:55:12] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1679 [02:00:12] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 3059858 Threads: 2 Questions: 83360544 Slow queries: 178478 Opens: 1309574 Flush tables: 1 Open tables: 200 Queries per second avg: 27.243 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [02:16:30] (CR) Ejegg: [C: +1] "Cleverly done! Looks like it should have the same result as before, just more efficiently. Might have one more opportunity for cleanup." (3 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607401 (owner: Eileen) [02:18:26] (CR) Eileen: Silverpop export: refactor calculation of address and preferred_language (3 comments) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607401 (owner: Eileen) [02:20:35] ejegg: thanks for that - I just replied - basically it's not the end of the cleanup & I'd rather not change that commit as there are follow ons .... but incorporate later [02:21:03] ok, cool, I'll up that to a +2 [02:21:13] (CR) Ejegg: [C: +2] Add some timings [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607624 (owner: Eileen) [02:21:25] (CR) Ejegg: [C: +2] Silverpop export: refactor calculation of address and preferred_language [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607401 (owner: Eileen) [02:21:25] thanks [02:22:13] (Merged) jenkins-bot: Silverpop export: refactor calculation of address and preferred_language [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607401 (owner: Eileen) [02:22:28] (Merged) jenkins-bot: Add some timings [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/607624 (owner: Eileen) [02:23:00] ejegg: I think this is actually earlier in the chain? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/607652 finding it really tough with new gerrit at this stage [02:23:24] oh? [02:23:47] no, the others merged already [02:23:50] actually - those 2 merged so they must be first [02:23:52] snap [02:23:52] that's next up though [02:24:10] it's still stacking them with the earlier patches on bottom [02:27:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1002:down [02:28:37] heh, would be nice to have a case-insensitive diff [02:32:12] PROBLEM - check_kafkatee on frban2001 is CRITICAL: CRITICAL: kafka-jumbo1005:down, kafka-jumbo1007:down [02:32:17] oh I changed the cases huh? [02:33:35] I'm looking to change it so we use the most recent contribution email id not the highest - but I am not sure if that is before or after I add the extra fields - it doesn't matter if it's before or after until I switch to incremental.... [02:34:26] just in the build schema: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/607652/5/silverpop_export/rebuild_schema.sql [02:35:55] oh man, country and country_unicode columns... [02:37:12] RECOVERY - check_kafkatee on frban2001 is OK: OK: brokers:9 topics:2 [02:38:06] ejegg: yep for drupal join vs civi join [02:38:35] yeah, makes sense, just unfortunate that it's necessary! [02:38:39] :-) [02:38:53] well the only other thing would be to change contribution_tracking [02:38:56] but meh [03:00:50] that next one looks good! I'll smoke test it tomorrow - getting pretty late here. [03:00:53] see ya! [03:02:58] (PS1) Eileen: Add endowment_latest fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608501 [03:34:45] (PS2) Eileen: Add endowment_latest and endowment_highest fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608501 (https://phabricator.wikimedia.org/T252245) [03:34:47] (PS1) Eileen: Remove duplicate line [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608503 (https://phabricator.wikimedia.org/T252245) [04:18:28] (PS3) Eileen: Add endowment_latest and endowment_highest fields [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608501 (https://phabricator.wikimedia.org/T252245) [04:42:44] (PS1) Eileen: Refactor of staging queries - use stats table, don't update staging table. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608506 (https://phabricator.wikimedia.org/T253152) [04:42:46] (PS1) Eileen: Move autocommit back to the top [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608507 [05:30:03] (PS1) Eileen: Add endowment_highest_usd_amount field to final output [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608512 (https://phabricator.wikimedia.org/T252245) [05:30:05] (PS1) Eileen: Populate field for all_funds_highest_donation_date annd all_funds_highest_usd_amount and all_funds_latest_native_amount [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608513 (https://phabricator.wikimedia.org/T252245) [05:31:10] (CR) jerkins-bot: [V: -1] Populate field for all_funds_highest_donation_date annd all_funds_highest_usd_amount and all_funds_latest_native_amount [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608513 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [05:38:22] (PS2) Eileen: Populate field for all_funds_highest_donation_date annd all_funds_highest_usd_amount and all_funds_latest_native_amount [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608513 (https://phabricator.wikimedia.org/T252245) [05:39:25] (CR) jerkins-bot: [V: -1] Populate field for all_funds_highest_donation_date annd all_funds_highest_usd_amount and all_funds_latest_native_amount [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608513 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [05:42:33] (PS3) Eileen: Populate field for all_funds_highest_donation_date annd all_funds_highest_usd_amount and all_funds_latest_native_amount [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608513 (https://phabricator.wikimedia.org/T252245) [05:55:15] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1350 [05:57:59] ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1350 Dwisehaupt Silverpop import testing. [06:00:15] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 3074258 Threads: 4 Questions: 83530809 Slow queries: 179006 Opens: 1309648 Flush tables: 1 Open tables: 200 Queries per second avg: 27.171 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [06:08:24] (PS1) Eileen: Simplify handling of foundation_has_recurred_donation [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608514 [06:18:10] (PS1) Eileen: Silverpop refactor - further reduce updating of silverpop table [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608515 (https://phabricator.wikimedia.org/T253152) [06:52:08] (CR) Eileen: "recheck" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608501 (https://phabricator.wikimedia.org/T252245) (owner: Eileen) [07:53:46] (PS1) Eileen: Update timings [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/608549 [13:04:18] Wikimedia-Fundraising-Banners: [Enhancement] Auto-select email input field after user clicks RML link in Nag - https://phabricator.wikimedia.org/T255598 (Pcoombe) Open→Resolved Thanks @EWilfong_WMF, I've done this for the control as well: https://meta.wikimedia.org/w/index.php?title=MediaWiki:Central... [14:22:33] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Removing asterisks in payment page card number field - https://phabricator.wikimedia.org/T254032 (mepps) I'm currently getting an error trying to sign onto the production developer site :(. [14:42:52] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Japanese MC 2nd Upsell Email - https://phabricator.wikimedia.org/T255227 (Cstone) This has been deployed along with an option to test it with the `/admin/config/thank_you/test` page. In the Template drop down pick `recurring_notification... [14:44:50] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Japanese TY Email - https://phabricator.wikimedia.org/T254052 (Cstone) This has been deployed. [14:54:59] grr.... TypeError: Argument 1 passed to CRM_MatchingGifts_Synchronizer::__construct() must implement interface CRM_MatchingGifts_ProviderInterface, instance of Mock_ProviderInterface_20a0ed3d given [14:55:16] so Mocks don't implement the interface? [14:55:39] oh, maybe I should mock the concrete class [15:09:49] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Removing asterisks in payment page card number field - https://phabricator.wikimedia.org/T254032 (mepps) This change is on production. Leaving in review for others to confirm it works as expected. [15:10:23] that would make sense to me ejegg [16:25:29] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Japanese MC 2nd Upsell Email - https://phabricator.wikimedia.org/T255227 (CDenes_WMF) Thank you Christine! I've sent the email produced by testmeister to our Japanese translator for review. In the meantime, I noticed that there is no fo... [16:27:56] Fundraising-Backlog, MobileFrontend, WMF-Design, Design, and 3 others: Mobile web donate link - https://phabricator.wikimedia.org/T219793 (Edtadros) === Test Result - Beta **Status:** ✅ PASS **Environment:** beta **OS:** macOS Catalina **Browser:** Chrome **Device:** MBP **Emulated Device:** iPh... [16:28:50] Fundraising-Backlog, MobileFrontend, WMF-Design, Design, and 3 others: Mobile web donate link - https://phabricator.wikimedia.org/T219793 (Edtadros) [17:08:22] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Removing asterisks in payment page card number field - https://phabricator.wikimedia.org/T254032 (EMartin) Looks good on donate wiki! [image: image.png] {F31911939} [17:16:26] (PS18) Ejegg: New MatchingGiftPolicies api action 'sync' using job [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/605638 (https://phabricator.wikimedia.org/T249924) [17:16:28] (PS2) Ejegg: Test for matching gift policy synchronizer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608486 (https://phabricator.wikimedia.org/T249924) [17:23:14] (CR) jerkins-bot: [V: -1] Test for matching gift policy synchronizer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608486 (https://phabricator.wikimedia.org/T249924) (owner: Ejegg) [17:49:01] AndyRussG: I added more doc in the readme at least for PS18: https://gerrit.wikimedia.org/r/605638 [17:49:52] ejegg: cool! hey I was looking just now at that stuff, didn't see PS18 yet tho! [17:50:56] ejegg: looking at the matching gifts csv export, is it only supposed to export 25 at a time? [17:51:01] oh maybe PS17 had the Readme doc too [17:51:08] cstone oho! [17:51:22] cstone nope, it was supposed to get ALL of them [17:51:25] yeah I did see that readme stuff in ps17 [17:51:35] ejegg: I'm a bit torn about the lack of delete functionality, I'll confess, but I also don't want to suggest revisiting work already done or decisions already made [17:51:35] hah ok jut wanted to make sure there wasnt a limit I didnt know about [17:51:48] cstone I bet I'm hitting a limit on the internal api call i'm making there [17:52:08] yeah I was wondering if it was something like that I don't see 25 referenced anywhere [17:54:12] thanks for finding that before we deployed! I'll figure that out [18:01:43] (PS10) Ejegg: Add matching gift employer export [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) [18:01:54] cstone: ok, I think ^^^ should get > 25 [18:02:53] thanks ejegg one last thing I was looking at when its just one word it isn't exported with " but would 2918,Beck's cause issues? [18:07:12] ejegg: fr-tech: I have an idea for a follow-up change that would allow us to also include deleting a company if they suddenly don't appear in the data. For it to work, and transition smoothly, I don't think there are any changes needed in the current patch or data structure [18:07:39] oh? [18:07:39] I'm thinking I could maybe just add the task to the epic, and clearly mark it as "follow up" in the title? [18:08:04] Sure, I'd love to see how you think that could work [18:08:38] cstone I'm not sure about that - I guess we could try it in the DonationInterface front end that way [18:08:54] yeah its in the export now so ill try it [18:09:22] fundraising-tech-ops, Patch-For-Review: Determine reason for daily increasing proc count on fran1001 - https://phabricator.wikimedia.org/T256420 (Dwisehaupt) Having dug into this, the cause appears to be long running queries that then have smaller queries that get backed up behind them waiting on the tab... [18:09:27] also some of these subsidiaries are questionable to me haha [18:09:27] 2945,Cheese-It [18:09:27] 2945,Crispix [18:09:27] 2945,Eggos [18:09:51] ejegg: ok thanks! the idea would just be to download the entire search results without any last updated search parameters, and save it in a temp file or something like that. Then we could go through that file and update the entries that need updating (using the last updated field that the patch already stores) and removing the ones that have disappeared [18:10:25] So the current patch could go out as a MVP, since it already includes the last updated custom field [18:10:31] fundraising-tech-ops, Patch-For-Review: Determine reason for daily increasing proc count on fran1001 - https://phabricator.wikimedia.org/T256420 (Dwisehaupt) [18:10:46] AndyRussG: deleting a company from civi? or the export? [18:11:00] cstone: right mmmm yeah from the export, rather [18:11:17] so as a follow-on patch, for some later date [18:11:56] yeah we'd have to keep a record of the company, but we'd stop sending it to the UI if it no longer appears in the database of matching gifters [18:12:07] cstone ejegg does all that sound reasonable? [18:13:42] AndyRussG: like what this would do or something different? https://phabricator.wikimedia.org/T251201 [18:14:01] AndyRussG: you've got me thinking about it now and I'm wondering if we need to introduce a deleted/active type flag [18:14:24] ah okay I see [18:14:35] jgleeson: yeah I mean that could be added later [18:14:41] if Eggos goes out of business [18:14:42] AndyRussG: sure, I guess I'd probably run that job less frequently than the general one [18:14:51] not Eggos! [18:15:26] ejegg: I forgot to ask, did you figure out a way to pull back all companies ? [18:15:37] or are you looping through something [18:15:40] in the sync [18:15:51] cstone: ejegg: yeah the current patch to fetch the data won't remove any companies if they stop gifting [18:16:28] not just if Eggos goes out of business, also if they decide not to help us with their Eggo moneyz [18:16:40] there is a suppress_from_employer_field column it checks could we use that? [18:16:50] im ok with Eggos going but not cheese its! [18:17:36] cstone: that might be different because that'd be used for employers that do appear on the list based on our category filtering, but actually don't match with us [18:18:05] so we'd need a different field that would allow them to be potentially re-added if the start matching again [18:18:24] like an Eggo Phoenix, rising from the burnt crumbs at the bottom of the toaster [18:18:34] AndyRussG: where would be deleting things [18:18:41] in your proposed patch [18:19:14] jgleeson: I think not deleting, just marking somewhere as no longer matching due to not appearing in hep data [18:19:27] currently we're generating the export in the command that compares the latest with the last saved [18:20:01] jgleeson: yeah that wouldn't change, I think? [18:21:02] earlier on you mention "removing the ones that have disappeared" [18:21:21] ejegg: we could figure out job frequencies later yeah [18:21:51] so not deleting but marking records with a flag? [18:22:04] jgleeson I was thinking of civi, but yeah I was wrong, marking in civi with a flags the ones that no longer appear in hep data [18:22:15] not actually removing from civi [18:22:30] since it generates the csv from the contacts table using some column there then? although i dont see an outright choice [18:22:52] so we'd need to explode the subsidiaries also I guess [18:23:02] and if those no longer match, overwrite with the new? [18:23:10] unless that already happens in the general sync [18:23:49] jgleeson: I don't think so... I mean, for the export, we're already taking into account the suppress_from_employer_field, no? [18:23:51] isnt just the parent company stored in civi? [18:24:05] So it'd just be to take into account another similar field also [18:24:10] yay but the list of subsidiaries in in a json blob [18:24:13] on the row [18:24:28] jgleeson: I think we'd just keep updating that blob as a blob? [18:25:15] export is checking that suppress_from_employer_field =0 [18:25:46] ejegg: for the current sync api patch, what do you think about adding inline phpdoc on the methods that require specific keys in the parameters arrays passed, and documenting the structures of arrays returned, and noting what has to coordinate with what? [18:26:00] yeah I don't think this is related. I meant if the subsidiaries change. I guess if that happens the "blob" will also change [18:26:05] ejegg: and also consolidating the method for making the full settings string? [18:26:11] jgleeson: yeah [18:26:30] cstone: yeah it'd just be to check an additional field then I think [18:26:52] also apologies if I'm misunderstanding things and getting stuff all wrong, not unlikely [18:27:12] AndyRussG: I can definitely get that extra phpdoc in [18:27:23] ejegg: ok that'd be fantastic! [18:27:51] ejegg: the repeated settings string methods could also just be marked with a TODO indicating where the duplicate code is [18:27:54] the only issue with consolidating the full settings string fn is that with it on the sync object we don't have to pass it the provider name [18:28:17] so if I moved it to the factory e.g. [18:28:31] all of the calls to it would involve a bunch more boilerplate [18:29:10] ejegg: hmmm I haven't dug into the details, just following on your comment... I guess I was imagining a static method somewhere [18:34:23] ejegg: k just peeked at the code in ProviderFactory... couldn't we have in ProviderFactor a static method like this: fullSettingName( $setting, $providerName ) { ... } [18:35:31] Then in Synchronizer we could just say ProviderFacctory::fullSettingName( 'a_setting', $this->policyProvider->getName() ) [18:35:56] seems reasonably succinct? [18:37:18] oh, i'm going with the synchronizer still having its own protected instance version that calls the static providerfactory version [18:37:27] to not repeat $this->policyProvider->getName() a bunch of times [18:38:07] ejegg: ah yeah great idea! [18:47:52] (PS19) Ejegg: New MatchingGiftPolicies api action 'sync' using job [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/605638 (https://phabricator.wikimedia.org/T249924) [18:47:54] (PS3) Ejegg: Test for matching gift policy synchronizer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608486 (https://phabricator.wikimedia.org/T249924) [18:48:01] AndyRussG: ok, PS19 has more phpdoc [18:48:23] that test is still failing because of the progress table not installing :( [18:51:48] ejegg: okok thanks!!! and hmmm ;) [18:53:11] will mess around with the different install options in that followon patch [18:54:40] ejegg: thanks!!! eileen I think had some thoughts on that, I think she might be around for tech-talk? [18:54:55] (CR) jerkins-bot: [V: -1] Test for matching gift policy synchronizer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/608486 (https://phabricator.wikimedia.org/T249924) (owner: Ejegg) [18:59:26] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Japanese MC 2nd Upsell Email - https://phabricator.wikimedia.org/T255227 (DStrine) I just spoke with @TSkaff The footer text should exist somewhere. Please add it to the Japanese and english version. Just for reference the text appears i... [19:00:13] PROBLEM - check_disk on frdev1001 is CRITICAL: DISK CRITICAL - free space: /dev 257932 MB (100% inode=99%): /run 47483 MB (92% inode=99%): / 3599 MB (6% inode=96%): /dev/shm 257946 MB (100% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 257946 MB (100% inode=99%): /boot 180 MB (72% inode=99%): /srv 941450 MB (25% inode=99%): /run/user/573 51589 MB (100% inode=99%): /run/user/3609 51589 MB (100% inode=99%): /run/user [19:00:13] 00% inode=99%): /run/user/571 51589 MB (100% inode=99%) [19:05:13] RECOVERY - check_disk on frdev1001 is OK: DISK OK - free space: /dev 257932 MB (100% inode=99%): /run 47467 MB (92% inode=99%): / 20865 MB (39% inode=96%): /dev/shm 257946 MB (100% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 257946 MB (100% inode=99%): /boot 180 MB (72% inode=99%): /srv 920994 MB (24% inode=99%): /run/user/573 51589 MB (100% inode=99%): /run/user/3609 51589 MB (100% inode=99%): /run/user/16767 51 [19:05:13] =99%): /run/user/571 51589 MB (100% inode=99%) [19:23:35] dwisehaupt: Jeff_Green ^^ [19:23:53] ah I missed RECOVERY [19:23:58] jgleeson: yeah. already cleaned up. [19:24:06] thanks for keeping an eye out though. [19:25:00] Fundraising-Backlog: Follow-up: Flag matching gifters removed from provider data, don't show them in UI - https://phabricator.wikimedia.org/T256799 (AndyRussG) [19:25:13] Fundraising-Backlog: Follow-up: Flag matching gifters removed from provider data, don't show them in UI - https://phabricator.wikimedia.org/T256799 (AndyRussG) [19:25:48] fr-tech: i just wanted to verify the mw contribution tracking extension has been fully removed (T255216). if so, i'll move forward with T253653 to clean up the config [19:25:49] T255216: Archive the ContributionTracking extension - https://phabricator.wikimedia.org/T255216 [19:25:49] T253653: clean up payments-wiki configuration related to deprecated mysql contribution tracking connection - https://phabricator.wikimedia.org/T253653 [19:32:48] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Japanese MC 2nd Upsell Email - https://phabricator.wikimedia.org/T255227 (CDenes_WMF) Thanks David! Since the footer is the same as the footer in the 'standard' Ty email -- please find the Japanese translation of the footer within the Ja... [19:40:47] AndyRussG: no sorry not here for tt today [19:46:53] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Q4 19/20 investigate export and upload issues with the silverpop export - https://phabricator.wikimedia.org/T253152 (EYener) Hi @Eileenmcnaughton... [19:53:07] Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog: Removing asterisks in payment page card number field - https://phabricator.wikimedia.org/T254032 (XenoRyet) Looks good to me as well. I'm going to go ahead and move it over to the done column, but I'll leave it open a bit longer in case... [19:54:31] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Q4 19/20 investigate export and upload issues with the silverpop export - https://phabricator.wikimedia.org/T253152 (Eileenmcnaughton) @EYener the... [19:55:34] eyener: I'm shooting out in a few but I'm changing the underlying tables quite a bit - probably not the view so much [19:56:51] hi eileen! I imagine that there will be a change in the nomenclature for silverpop_export to reflect the new _all_funds fields as well as rename some fields to af_total_2019 and such? [19:57:31] yes - basically I'm trying to prepend foundation_ or endowment_ or all_funds_ to the fields [19:57:41] to reduce confusion [19:57:51] Overall I'm trying to get a clear snapshot in my head of what will look different between today and tomorrow in that db [19:58:05] so it's already updated [19:58:16] today and yesterday... [19:58:48] but it's not the last update to happen - I'm still working on changes - yesterday was the part that affected the final export mapping - ie the view [19:58:59] because that had to be co-ordinated with Katie [20:00:28] Okay I think I have it [20:01:11] the goal is a more incremental update - so it should be updated ideally more often with smaller changes [20:01:53] if you are wanting your lapsed, new, renew fields going in there I can look at that that [20:02:38] Nah, that's okay; currently silverpop is not directly accessible through the analytics replica server [20:03:03] (PS11) Ejegg: Add matching gift employer export [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) [20:05:02] (CR) jerkins-bot: [V: -1] Add matching gift employer export [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) (owner: Ejegg) [20:05:16] Thanks for the overview eileen - if you have a few minutes sometime this week to talk through it more, I'd be interested in a quick sync up to learn more about it! [20:12:15] eileen: ah right... just now ejegg was trying to figure out details of running the sql to create the table, I think for tests? If you have time, maybe peek the comments on line 13 here? (No worries if not also! :) ) https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/605638/16/sites/default/civicrm/extensions/matching-gifts/CRM/MatchingGifts/Upgrader.php#13 [20:13:08] Fundraising Sprint Lazy Loading Life, Fundraising Sprint MySQL is YourSQL and WeSQL, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Q4 FY2019/20 investigate export and upload issues with the silverpop export - https://phabricator.wikimedia.org/T253152 (Aklapper) [21:02:26] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Tony Le - https://phabricator.wikimedia.org/T256808 (DStrine) [21:02:42] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Julianne Joe - https://phabricator.wikimedia.org/T256809 (DStrine) [21:05:44] Fundraising-Backlog, FR-AutoTY-Email: Civi: possible email_greeting TY email feature request - https://phabricator.wikimedia.org/T253326 (DStrine) Has there been any update to this? [21:46:31] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Julianne Joe - https://phabricator.wikimedia.org/T256809 (Dwisehaupt) [21:46:48] dwisehaupt: is there a good spot on civi1001 to drop a file that can be accessed by all users on the machine? We have a process that is gonna create a csv file that someone else needs to be able to access. I tried touching in /var and /srv but perms are locked down. [21:46:50] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Julianne Joe - https://phabricator.wikimedia.org/T256809 (Dwisehaupt) a:Dwisehaupt [21:47:35] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Tony Le - https://phabricator.wikimedia.org/T256808 (Dwisehaupt) [21:49:01] well, /tmp is generally readable by everyone. keeping in mind that it lives on the / partition so we don't want to be dropping large files that could fill the device. also, /tmp gets cleared on reboot if that is an issue. [21:49:17] it's totally feasible for us to create a /srv/tmp if necessary [21:49:18] that might be an issue [21:49:31] (the reboot clearing) [21:49:49] Fundraising-Backlog, fundraising-tech-ops: Create civi cert for Tony Le - https://phabricator.wikimedia.org/T256808 (Dwisehaupt) a:Dwisehaupt [21:50:11] I guess all users can access /srv and /var right? so if the user running the process-control scheduled job can write to either of those it might be ok [21:51:08] what user runs the jobs here srv/process-control/civi1001 ? [21:51:19] is that puppet [21:51:39] looking. [21:51:44] the user puppet I mean [21:51:46] thanks! [21:52:24] fr-tech feel free to chime in if folks have done stuff like this before and know of a better approach [21:54:30] so, assuming you are asking which user runs process_control jobs as defined there, it's the jenkins user. [21:54:38] ah ok [21:54:48] does that user have write access to /var or /srv ? [21:56:26] although, in a process control definition, you can specify a user: value to have it run as a different user. [21:57:45] jenkins on its own doesn't have rights to write in /srv directly. but a directory could be created in there that could just hold files for this process [21:58:09] that's generally how i would do it anyway to keep /srv from getting cluttered with files anyway. [21:58:47] that's cool. /srv/tmp sounds ok to me? [21:58:56] if that's what you'd suggest [22:00:05] if they are just temporarily needed files (aka not forever or having special needs) that seems fine. unless it would benefit from having its own location. [22:01:08] that should be something like /srv/jenkins/tmp [22:01:28] the main downside i would say for /srv/tmp is that anything labeled /tmp stands out to be cleaned up first if there are space issues. [22:02:03] hmm [22:02:10] Platonides: yeah. i think i just need a little more info to give the best recommendation. [22:02:14] maybe /srv/matching-gifts [22:02:37] what is this file for? [22:02:48] the file is gonna contain a list of employers in csv format that will ultimately be scp'd to another box and then deployed [22:03:38] the manualness of this is due to compliance [22:04:00] compliance? [22:04:10] PCI compliance [22:04:30] in order not to break the isolation of the origin/target server? [22:05:15] in that case, i'd suggest /srv/jenkins/tmp or maybe something like /srv/$employer_list_process where $employer_list_process is whatever we want to call the process. [22:05:52] Platonides: yeah. to ensure files written to the target have the proper change tracking and audits in place. [22:06:11] why not make it go through git repository? [22:06:31] let jenkins send a changeset [22:06:43] git will provide the change tracking [22:06:50] and whoever reviews it, the audit [22:07:19] you may want a private repository, but it seems better than manual scps [22:08:44] i'm not sure enough of the process to collect the info to give any recommendation on that. [22:08:45] agreed. we've got a ticket to improve the transfer of the file between boxes so git might end up being the solution going forward for that [22:09:13] jgleeson: i have to head out to take care of the kids. i can catch up with you first thing tomorrow if that's ok. [22:09:23] sure np, thanks dwisehaupt [22:09:25] catch you tomorrow [22:09:43] jgleeson: AndyRussG and I are in techtalk if you want to join [22:10:05] sounds good. see you then. [22:10:43] thanks cstone but I'm heading off shortly [22:10:53] no worries [22:11:10] thanks for the idea Platonides [22:11:30] you're welcome, jgleeson [22:12:33] it may, or may not make sense in the end [22:36:23] (PS12) Ejegg: Add matching gift employer export [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) [22:56:00] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: add translations to generic email for failed recurring donations - https://phabricator.wikimedia.org/T255837 (CDenes_WMF) Hi all! The copy for 10 languages is ready. Please find the translations [[ https://docs.google.com/document/d/1rf63tFSXsxcALNsHZZ... [22:58:40] (PS1) Ejegg: Fix cursor for form fields [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/608742 [22:59:40] Fundraising-Backlog: Process to transfer of MG employers data file between civi1001 and frpm1001 - https://phabricator.wikimedia.org/T256825 (jgleeson) [23:02:05] Fundraising-Backlog: Process to transfer MG employers data file between civi1001 and frpm1001 - https://phabricator.wikimedia.org/T256825 (jgleeson) [23:17:24] (CR) Cstone: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) (owner: Ejegg) [23:17:51] new gerrit UI tricking me haha [23:19:14] (CR) Cstone: [C: +2] "Looks good, thanks for getting rid of the duplicate names." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/607633 (https://phabricator.wikimedia.org/T250328) (owner: Ejegg) [23:55:23] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: add translations to generic email for failed recurring donations - https://phabricator.wikimedia.org/T255837 (DStrine) cd