[01:02:15] RECOVERY - check_redis on frqueue1001 is OK: OK: REDIS 5.0.3 on 127.0.0.1:6379 has 1 databases (db0) with 5 keys, up 76 days 9 hours - memory use is 1.94M (peak 592.88M, 0.26% of max, fragmentation 11.24%), connected_slaves is 2, donations is 8, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 1, payments-init is 64, pending is 0, recurring is 0, refund is 0, unsubscribe is 0 [01:24:35] (PS1) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [01:25:12] AndyRussG: I've only tested that on paypal EC and Ingenico ^^^ [01:25:58] oh hmm, definitely needs an update for handling expired sessions - will mark it WIP for now [01:26:20] ejegg: woohooo K I'll start looking :) [01:27:07] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [01:28:39] AndyRussG: oh hmm, also more test setup needed [01:28:50] definitely not ready for prime time [01:29:08] but also not as complicated a fix as i thought we might need [01:38:47] oh hmm, def should return a better error when it fails [01:38:50] and not just {} [01:59:53] ejegg: besides fixing tests what else do you think it needs? I think for now the {} error is fine [02:05:51] (PS2) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [02:06:08] ^^^ should add an error message, but tests are still borken [02:07:41] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [04:45:15] RECOVERY - check_disk on civi1001 is OK: DISK OK - free space: /dev 7957 MB (100% inode=99%): /run 1437 MB (90% inode=99%): / 13976 MB (26% inode=93%): /dev/shm 7974 MB (100% inode=99%): /run/lock 5 MB (100% inode=99%): /sys/fs/cgroup 7974 MB (100% inode=99%): /boot 141 MB (56% inode=99%): /srv 578451 MB (69% inode=99%): /srv/archive/banner_logs 4132976 MB (54% inode=99%): /run/user/573 1594 MB (100% inode=99%): /run/user/16759 [04:45:15] e=99%) [07:32:10] (CR) AndyRussG: "Woohooo! So I guess we can assume that wmf_token is already always being sent with API calls for all processors, as in ext.donationInterfa" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [09:35:15] (CR) AndyRussG: "So, I'm trying to dig into how to get the tests to work... Seems not too easy!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [09:41:24] Fundraising-Backlog, FR-AutoTY-Email, I18n: Fundraising auto-thankyou email has incorrect localization for money sum - https://phabricator.wikimedia.org/T259947 (Pcoombe) [09:42:41] Fundraising-Backlog, MediaWiki-extensions-DonationInterface, I18n: Donation form should use the appropriate name order - https://phabricator.wikimedia.org/T259945 (Pcoombe) [11:23:25] Wikimedia-Fundraising-Banners: [Enhancement] India - Rearrange Visa payment logos to match other payment option button - https://phabricator.wikimedia.org/T259937 (Pcoombe) [11:27:40] Wikimedia-Fundraising-Banners: [Enhancement] India - Rearrange Visa payment logos to match other payment option button - https://phabricator.wikimedia.org/T259937 (Pcoombe) This is only a problem for India, and happened because of the last minute request to remove Amex. Other countries with 3 card logos e.g.... [13:16:24] Wikimedia-Fundraising-Banners, fr-donorservices: India banner: extra decimals on Other amount field gets PTF riled up - https://phabricator.wikimedia.org/T259762 (Pcoombe) >>! In T259762#6369797, @jbolorinos-ctr wrote: > I believe this is due to the fact that some places '.' or ',' is used for the thousa... [13:58:15] (PS3) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [14:00:27] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [14:18:38] (PS4) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [14:20:18] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [14:47:13] (PS5) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [14:49:15] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [14:57:45] (CR) Mepps: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/617562 (https://phabricator.wikimedia.org/T237350) (owner: Mepps) [15:26:14] (PS6) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [15:29:22] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [15:29:35] Fundraising Sprint P 2020, Fundraising-Backlog: Add two additional fields to silverpop export - https://phabricator.wikimedia.org/T257597 (mepps) a:mepps [15:31:26] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (Pcoombe) Huh, I actually don't know where that unsubscribe page is translated. @CCogdill_WMF ? [15:36:29] (PS7) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [15:39:41] (CR) jerkins-bot: [V: -1] Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [15:51:44] (PS8) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [15:59:39] woot, passing tests [15:59:54] ok, locally I have smoke tested ingenico and paypal ec [16:02:02] nice! [16:07:06] i have locally tested adyen and ingenico [16:08:09] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (CDenes_WMF) Quick note: I provide the translations in the MTT, I'm just not sure who does the coding! [16:10:22] ejegg: were you saying need to switch to the preprod ingenico creds for local? I realize now I got an error at the end [16:11:37] cstone ugh, yeah, looks like we need to use the pre-prod credentials and URL if we want to get to the point of testing the recurring convert [16:12:00] I think those should be current up on the fr puppetmaster machine [16:12:48] ok thanks [16:13:13] ejegg: I just tried a standard transaction and that failed also with ingenico [16:13:22] pre-prod is staging? [16:13:31] cstone: yep [16:13:34] I'll switch over to pre-prod if I'm not using it [16:13:40] lemme check [16:13:46] these are the SmashPig creds [16:14:05] https://eu.sandbox.api-ingenico.com [16:14:08] that's where I was hitting [16:14:22] yah, I think we need to switch to https://world.preprod.api-ingenico.com/ [16:15:20] trying with that one and matching creds now [16:16:35] ok i got a thanks page with the preprod but its in the iframe [16:16:44] but progress [16:17:00] me too cstone [16:17:05] cstone: oh dang, the TY page is in the iframe? [16:17:06] re:iframe [16:17:45] yeah ejegg [16:18:23] I also noticed: [16:18:25] A cookie associated with a cross-site resource at http://secured-by-ingenico.com/ was set without the `SameSite` attribute. It has been blocked, as Chrome now only delivers cookies with cross-site requests if they are set with `SameSite=None` and `Secure`. You can review cookies in developer tools under Application>Storage>Cookies and see more details at [16:18:27] https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032. [16:18:30] ok i tested again incognito and it behaved [16:18:34] but that might be nothing [16:18:38] thankyou page not in iframe [16:18:53] cstone hmm, so you're testing with the new patch? [16:18:53] cool [16:20:08] ejegg: with patch and I think it was some local chaos with the iframe issue its redirecting now [16:20:43] cstone: I've seen that before [16:21:12] I feel like the iframe liberator thingy might get tripped up when try again shortly after a successful donation [16:21:35] possible still in session as liberated [16:21:45] yeah jgleeson [16:22:53] ah, that makes sense [16:22:59] yeah it's playing nice for me also [16:23:17] so it's looking good for me with the preprod creds [16:23:33] I just wanna xdebug the actual changes on my side [16:23:34] and the recurring convert seemed to work too [16:23:39] just to see it getting through [16:23:45] cool jgleeson, sounds good [16:28:41] (CR) Mepps: [C: +2] "This looks good and was also tested upstream." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/618162 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [16:30:50] XenoRyet: be there in 1 minute [16:30:56] No worries [16:31:20] ejegg: it's working I just wanted to simulate sending in a bad token [16:31:28] what's the easiest way to do that [16:31:39] is it a hidden form field? [16:32:11] I'll try that after my 1:1 with XenoRyet in case it's still not merged [16:32:27] yep, it's a hidden form field jgleeson [16:32:29] wmf_token [16:32:57] right, I was thinking I should make the JS reload the form in case there's an expired token / session [16:37:45] (Merged) jenkins-bot: Fix obscure dedupe scenario where 'bad' location data can overwrite good data [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/618162 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [16:46:34] hrm, the 'badtoken' value i set with jquery seems to want to persist [16:46:47] across reloads [16:50:06] Fundraising Sprint Nyan cats for everyone, Fundraising-Backlog, fundraising-tech-ops: map out a civicrm.log_% table data expiration/delete strategy - https://phabricator.wikimedia.org/T257232 (Jgreen) >>! In T257232#6369920, @Eileenmcnaughton wrote: > @Jgreen that is about 200GB less than is was at t... [17:02:10] Wikimedia-Fundraising-Banners: [Enhancement] India - Rearrange Visa payment logos to match other payment option button - https://phabricator.wikimedia.org/T259937 (jbolorinos-ctr) Great! Thanks Peter for already taking care of that in other countries, I'll make a note to check this in future campaigns where... [17:05:00] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (CCogdill_WMF) @MNoorWMF can help or delegate this to trilogy. Thanks! [17:07:28] ok ejegg it fails as expected for me! nice work [17:07:35] (when injecting a bad token) [17:11:42] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (CCogdill_WMF) Also, re: question A -- this actually helps point out a funny inconsistency in our unsubscribe rules. Because Legal has tested our forms more than once, t... [17:24:05] cool! [17:25:54] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (MBeat33) > a bigtime edgecase and I'm not concerned about it happening to donors Thanks for confirming that, @CCogdill_WMF ! :) [17:25:57] Wikimedia-Fundraising-Banners: QA for en6C pre-test on August 12, 2020 - https://phabricator.wikimedia.org/T259918 (jbolorinos-ctr) Hey @spatton, given the task David mentioned T244420, will we be removing the Paypal option from these variants? [17:28:47] (PS9) Ejegg: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) [17:29:19] Fundraising-Backlog, fr-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935 (CDenes_WMF) I do not. We haven't run LATAM since I've been at the org. I took a look in the countries and translations google drive and found translations for LATAM for... [17:29:41] Wikimedia-Fundraising-Banners: QA for en6C pre-test on August 12, 2020 - https://phabricator.wikimedia.org/T259918 (spatton) Great question, @jbolorinos-ctr. As with other countries where we offer Paypal (notably the bundle countries, which Peter referenced today), Paypal comprises a considerable share of do... [17:39:01] (CR) Mepps: [C: +1] "Needs smoke testing but the code looks good. I had one silly question." (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [17:45:11] (CR) Cstone: [C: +1] "Smoke testing looks good on Adyen and Ingenico" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [17:46:45] (CR) Mepps: [C: +1] "Mostly looks good. Some tiny comments but I'm open to +2ing as is with discussion." (3 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619149 (owner: Eileen) [17:49:09] (CR) Mepps: [C: +2] Remove function moved to deduper [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618886 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [17:49:43] (CR) Mepps: [C: -1] "white space got worse" (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618922 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [17:54:03] (CR) Mepps: [C: +2] Remove blank address test [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619078 (owner: Eileen) [18:01:59] (CR) Jgleeson: [C: +2] "Code looks good. success and failure scenarios working as expecting for me and tests passing. Other folks in fr-tech have also tested. Nic" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [18:04:49] (Merged) jenkins-bot: Require and check tokens on donate API [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/619167 (https://phabricator.wikimedia.org/T244420) (owner: Ejegg) [18:16:38] fr-tech I'm just reading the latst Signpost special report https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2020-08-02/Special_report about declining active contributors and I feel like a less scary explanation might be that the number of articles across the wikis will be negatively correlated with the number of active editors so it might explain why there are less new editors as [18:16:40] there will be less low-hanging fruit in an article sense. Seddon I'd be interested to hear what you think [19:02:18] thanks for the CR jgleeson ! [19:02:29] And yeah, completeness seems like a probable factor [19:02:42] in decline [19:02:47] OK, so I'm going to deploy that patch [19:03:51] and then turn paypal back on [19:04:00] fr-tech today's album recommendation is the Donnie Darko movie soundtrack https://www.youtube.com/watch?v=LWz0JC7afNQ&list=PLEndOMDRcXvhZ3upjB2jYsfaEuEwWol5o [19:04:11] ok great ejegg [19:04:13] nice jgleeson! [19:05:25] nice. [19:06:24] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/619348 [19:06:40] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/619348 (owner: Ejegg) [19:07:43] nice jgleeson [19:07:55] I'm listening to Dreamland by Glass Animals no handy link [19:08:26] the album cover alone makes it list worthy! [19:10:10] checking it out cstone! [19:13:52] i was listening to this earlier. beware the cognitive dissonance of it sounding so right and wrong: https://www.youtube.com/watch?v=WtamPZzWmCg&list=PLoYpp-6MF1BRxVnVXl2aKxKQmee50ARUo&index=1 [19:14:15] Dreamland was quite dreamy indeed [19:14:29] :) [19:14:34] thanks! [19:15:05] I was trying to use Dreamland to stop Hamiltion from being a constant in my head but so far failed [19:15:57] dwisehaupt: 1:08 it goes crazy [19:16:13] turns into a jimmy page solo [19:16:17] :) [19:16:28] some of the game soundtracks out are amazing. [19:16:39] Fundraising-Backlog: Investigate moving to docker - https://phabricator.wikimedia.org/T260085 (DStrine) [19:17:22] cstone oh yeah, i had a hamilton song pop up earlier today [19:18:01] wow cstone the album title track is awesome [19:18:14] guns and ships is todays for me [19:18:24] feels like air safari on psychedelics [19:18:40] haha im a fan of stuff like that jgleeson reminds of some stuff animal collective has done and miike snow if you want more [19:18:50] moon safari* [19:19:36] cstone: you could watch the eurovision movie and get volcano man stuck in your head. [19:19:44] i need to do that [19:19:46] yeah send them over this one feels like it's a keeper! [19:20:06] I need to watch the eurovision movie and hamilton [19:20:41] but first we've got Braveheart to tick off the list for Roxy [19:20:52] FREEEEDOMMMMMMMMMMMMM [19:21:24] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/619348 (owner: Ejegg) [19:21:41] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/619353 [19:28:10] (CR) Ejegg: [C: +2] Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/619353 (owner: Ejegg) [19:28:35] derp, waiting there 7 minutes before I realized I still needed to +2 the mw branch patch [19:29:06] doh [19:32:00] fr-tech be there in 1 min! [19:32:32] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/619353 (owner: Ejegg) [19:45:06] Fundraising Sprint P 2020, Fundraising-Backlog: Create fundraising MediaWiki 1.35 payments deploy branch - https://phabricator.wikimedia.org/T257352 (DStrine) p:Medium→High [19:59:11] Fundraising-Backlog: prototype moving to docker - https://phabricator.wikimedia.org/T260085 (DStrine) [19:59:46] Fundraising-Backlog: prototype moving to docker - https://phabricator.wikimedia.org/T260085 (EBjune) [20:07:07] Fundraising-Backlog: Task creation link on #Fundraising-Backlog Phab project description does not work (requires unknown special rights) - https://phabricator.wikimedia.org/T259776 (Aklapper) Thanks for the edit! > seems to load correctly for me @Dstrine: That is likely, as you can probably access `T89899`... [20:10:52] !log updated payments-wiki from 932aacde54 to f199c071c3 [20:10:54] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:16:37] (CR) Eileen: Minor test cleanup (3 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619149 (owner: Eileen) [20:17:49] (Abandoned) Eileen: Not for merge - looking to see affected tests [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618865 (owner: Eileen) [20:18:08] (Abandoned) Eileen: Remove all remaining location handlinng from wmf_civicrm.module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618887 (owner: Eileen) [20:18:30] (Abandoned) Eileen: Remove handling of email conflicts from custom code, handling in deduper [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618168 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [20:18:47] (Abandoned) Eileen: Test to demonstrate bug with email handling [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618175 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [20:19:48] (Abandoned) Eileen: Add suppression list rebuild - not called - just for if needed to run the queries [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610400 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [20:20:01] (Abandoned) Eileen: Copy existing full build for export stats to rebuild file [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/612694 (owner: Eileen) [20:20:17] (Abandoned) Eileen: Extract running of queries to a function [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/610663 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [20:36:13] ok, patch seems to be OK up on payments-wiki [20:36:21] Fundraising-Backlog, Operations, User-Urbanecm, User-dancy, Wiki-Setup (Create): New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Pcoombe) >>! In T259002#6370883, @Urbanecm wrote: >>>! In T259002#6340904, @Pcoombe wrote: >... [20:36:37] pcoombe / fr-tech I'm all set to revert my donatewiki change and put paypal back on the forms there [20:42:02] Fundraising-Backlog, Operations, User-Urbanecm, User-dancy, Wiki-Setup (Create): New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Pcoombe) Oh, one more thing that I couldn't find in the settings: all users should have acce... [20:44:23] Fundraising-Backlog, Operations, User-Urbanecm, User-dancy, Wiki-Setup (Create): New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Ladsgroup) Okay then, I think we have everything we need. IMO, except the section (s5 or s3)... [21:29:21] Fundraising-Backlog, fundraising-tech-ops: Update fundraising_code_deploy payments config to deploy new employers.csv file - https://phabricator.wikimedia.org/T259662 (Dwisehaupt) a:Dwisehaupt Current plan is to get this file onto frpm1001 into the localsettings repo. Then we can distribute it by upd... [21:45:03] mepps: I commented on https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/619149 - the problem with chains of commits is it's a pain to add more to the early commits [21:47:33] (PS3) Eileen: Superficial cleaup [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618922 (https://phabricator.wikimedia.org/T256316) [21:48:10] (CR) Eileen: Superficial cleaup (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618922 (https://phabricator.wikimedia.org/T256316) (owner: Eileen) [21:48:26] Fundraising-Backlog, Operations, Patch-For-Review, User-Urbanecm, and 2 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Urbanecm) >>! In T259002#6374471, @Ladsgroup wrote: > Okay then, I think we have everything we ne... [21:50:54] Fundraising-Backlog, Operations, Patch-For-Review, User-Urbanecm, and 2 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Urbanecm) [21:54:30] cstone: I just spotted this is the last of those comment fixes from our tech talk- can you + 2 it ? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/618410 [21:57:20] Fundraising-Backlog, Operations, Patch-For-Review, User-Urbanecm, and 2 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Dzahn) @Pcoombe thankyou.wikipedia.org has been added to DNS Is an '.m.' mobile link needed/wa... [21:58:24] Fundraising-Backlog, fundraising-tech-ops: Update fundraising_code_deploy payments config to deploy new employers.csv file - https://phabricator.wikimedia.org/T259662 (Dwisehaupt) A stub employers.csv file has been put in place in the localsettings repo for both the payments-wiki and payments-wiki-stagin... [22:08:42] Fundraising-Backlog, fundraising-tech-ops: Update fundraising_code_deploy payments config to deploy new employers.csv file - https://phabricator.wikimedia.org/T259662 (Dwisehaupt) Open→Resolved Verified the deployments work correctly with payments-wiki and payments-wiki-staging projects. Should b... [22:11:41] fr-tech so it looks like the exchange rates update from friday at midnight did FAIL and the next on runs in about an hour or os. There isn't much in the process-control logs to say what broke but I can see in the db that the last update was thursday at mightnight and the exception code path the script fell into on Friday's attempt looks like we got a '0' http response code (timeout) [22:11:43] https://github.com/wikimedia/wikimedia-fundraising-crm/blob/685086bdec2a6a1c43b7e4ed02e982796395f911/sites/all/modules/exchange_rates/retrievers/OandaRetriever.php#L51 [22:12:27] thanks for looking into that jgleeson ! [22:12:50] Fundraising-Backlog, fundraising-tech-ops: Investigate increase syslog volume on civi1001 after buster upgrade - https://phabricator.wikimedia.org/T260000 (Dwisehaupt) Verified that the new debug like logs started ~Aug 7 05:27:41 for the drupal: entries and ~Aug 7 05:32:03 for the php: entries. This co... [22:13:10] It's possible oanda closed off our generic account when setting up a new account for Ppena [22:13:24] my guess is that account has a new API key we need to use [22:15:42] I'll make a ticket before I jump off ejegg and we can talk about it on midsprint check-in tomorrow [22:30:33] Fundraising-Backlog: Oanda exchange rates update job failing - https://phabricator.wikimedia.org/T260092 (jgleeson) [22:33:53] AndyRussG: ejegg I see you made sense of the silverpop up date one & I added your explanation into the commit - what is still needing to be done to get it merged? [22:34:03] this verbose logging issue may be better addressed at tomorrows tech talk where multiple people can see what i'm seeing. [22:34:49] hi eileen [22:34:52] it looks fine to me [22:35:01] i've definitely pinpointed that it wasn't present before the buster upgrade/reboot [22:35:02] AndyRussG is ooo today I think [22:35:09] Fundraising Sprint Pseudopretzels, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Q2-FY2020-21-cleanup-list: Remove foreign key in attempt to address Spate of database contention Deadlocks - https://phabricator.wikimedia.org/T258661 (Eileenmcnaughton) Open→Resolved This may have... [22:35:16] dwisehaupt: oh really, weird? [22:35:18] oh yeah of course he is [22:35:57] dwisehaupt: sorry about that truncate being slow last week - I missed that was there in amongst the drops [22:36:52] eileen: no problem. [22:37:30] ejegg: yeah. if you search either the fundraising-drupal or fundraising-misc logs for lines ending with a ( you'll see them [22:38:36] ending with an open-paren? [22:38:38] verified that they weren't in the log files before we reenabled services. [22:38:44] yeah [22:39:06] since it looks like the contents of an array being dumped to logs. [22:39:08] oh, the queue messages being dumped to log [22:39:30] hmm, I thought that was pretty standard [22:39:49] oh right, we dump them as a json blob [22:39:58] but here they're also being print_r'ed [22:40:24] so yeah, that's totally redundant [22:41:22] ejegg: does that mean you are OK to +2 https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/614853 [22:41:38] ccccccknfuvhfeuuudvhnkukenkithhfrgrlbbhurill [22:41:59] Fundraising-Backlog, Operations, Patch-For-Review, User-Urbanecm, and 2 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Pcoombe) >>! In T259002#6374625, @Dzahn wrote: > @Pcoombe > > thankyou.wikipedia.org has been a... [22:42:31] (CR) Cstone: [C: +2] Add code comments [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/618410 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [22:46:22] (CR) Ejegg: [C: +1] "Thanks for putting the explanation in the comment! Just looks like the first couple lines got duplicated." (1 comment) [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [22:47:43] (CR) Eileen: "OK - thanks - I just did an edit inn place on it" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [22:51:12] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [22:51:24] eileen: the edit seems not to have taken yet [22:51:29] maybe needs a publish? [22:51:51] mm I'll look - serves me right for using the Web interface [22:52:08] (PS6) Eileen: Add modified_date to silverpop_export and filter final view on it. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) [22:52:12] PROBLEM - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [22:52:14] yep that was it [22:52:18] PROBLEM - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [22:52:35] dwisehaupt: ok, looks like the code line to print_r the message has been there for a while - it's logging it at WATCHDOG_INFO level, which I would have assumed we've been logging at all along [22:52:45] it SHOULD go away [22:52:58] just not sure why it would have turned up now after not logging before [22:55:48] ejegg: cool. i'm not sure why either. [22:56:08] Fundraising-Backlog, Operations, Patch-For-Review, User-Urbanecm, and 2 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Urbanecm) @Ladsgroup Any idea how to get rid of mobile subdomain assumption? Is removing mobilefr... [22:56:12] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [22:56:26] unless there was a change in syslog or some other package between os versions that made it surface. [22:57:12] PROBLEM - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 10 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [22:57:18] PROBLEM - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 10 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [23:01:03] Fundraising-Backlog, Operations, MW-1.36-notes (1.36.0-wmf.4; 2020-08-11), Patch-For-Review, and 3 others: New wiki for fundraising Thank You pages with similar config as donatewiki - https://phabricator.wikimedia.org/T259002 (Reedy) >>! In T259002#6374756, @Urbanecm wrote: > @Ladsgroup Any idea... [23:01:12] RECOVERY - check_puppetrun on frdb2002 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [23:02:12] PROBLEM - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [23:02:18] PROBLEM - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [23:04:32] ACKNOWLEDGEMENT - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] Dwisehaupt mysql holds are causing complaints as the held version of the client is no longer in the repo [23:04:32] ACKNOWLEDGEMENT - check_puppetrun on frdb1003 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] Dwisehaupt mysql holds are causing complaints as the held version of the client is no longer in the repo [23:04:32] ACKNOWLEDGEMENT - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] Dwisehaupt mysql holds are causing complaints as the held version of the client is no longer in the repo [23:11:12] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 9 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[mariadb-client] [23:11:19] eileen: so the deleted contacts ARE an issue, just not introduced in this patch [23:11:56] if you look on prod, there are definitely deleted contacts in the silverpop export dating back to when the export was switched to be incremental [23:12:13] I guess I +2 this one and then we work out how to clear those out? [23:16:12] RECOVERY - check_puppetrun on frdb2002 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [23:17:12] RECOVERY - check_puppetrun on frdb2001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:18:59] fundraising-tech-ops: Look at mirroring the mariadb repos locally to cut down on puppet errors - https://phabricator.wikimedia.org/T260095 (Dwisehaupt) [23:22:18] RECOVERY - check_puppetrun on frav1002 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [23:25:01] ejegg: ah OK - yeah that suggests the delete isn't working right [23:25:27] ejegg: so in the export tables? [23:29:51] yep eileen, you join on the contact ID to the civicrm contact table [23:29:59] and see a bunch of civicrm contacts that are deleted [23:30:03] argh [23:30:25] i assume because we just look at things that are recently updated and NOT deleted [23:30:39] when we're building our silverpop tables [23:30:52] so we ignore things that are recently updated and deleted [23:31:11] So which table are you looking at? [23:31:31] the final view uses silverpop_export [23:32:03] which should be a small-ish subset FROM silverpop_update_world t [23:32:26] ohh, so the silverpop_expor table has the deleted contacts [23:32:38] but you're saying the final view shouldn [23:32:40] 't ? [23:32:52] because of the limit? [23:32:52] no silverpop_export shouldn't [23:33:34] maybe they are in silverpop_update_world ? [23:34:19] hmm, I haven't checked [23:34:50] though there are some in silverpop_export which are deleted in civicrm_contribution with a last updated date of e.g. 7/28 [23:35:05] so that's too far back to be in silverpop_update_world, right? [23:35:33] yeah [23:36:07] I wonder if the final filter shouldn't be in the view itself actually [23:36:24] (PS7) Ejegg: Add modified_date to silverpop_export and filter final view on it. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [23:36:33] to be honest there is an advantage of pushing the updates IF they were already in silverpop [23:36:52] (more visible than the suppression list) [23:37:38] (CR) Ejegg: [C: +2] "Looks good! The deleted contacts weren't introduced by this patch, and this patch will probably mean uploading fewer of them each night." [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [23:38:02] OK - so the contact isn't in export_world [23:38:14] hence it's not deleted when we do a delete [23:38:23] eileen, but we should make sure we have a non-deleted contact ID for each email that we send out [23:38:23] (Merged) jenkins-bot: Add modified_date to silverpop_export and filter final view on it. [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/614853 (https://phabricator.wikimedia.org/T253152) (owner: Eileen) [23:38:56] ok, so maybe we expand silverpop_export_world to include all the recently updated ones, deleted or not? [23:38:58] yeah - the contact I'm look ing at is 158056 [23:39:10] yeah maybe [23:40:29] hmm for that contact we really did delete his other email that might relate to the bug I am looking at on dedupe [23:40:31] so most of them will be merged into other contacts, and we just want a full recalc for htat email address [23:41:04] hmm, though I guess for merges there would always be another updated contact with the same email? [23:42:14] Yeah [23:42:36] the one I looked at the deleted contact's email is on the suppression list but we still pushed up the zero-data version [23:43:14] honestly I'd rather push up the empty contact with is_suppressed as a field because we had the issue where the merged contacts still showed ++ $ values [23:43:18] before this [23:43:38] because we had already pushed them up & when the got deleted we stopped updating them [23:44:30] however if they are not in update world their totals might not be updated either [23:47:10] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/619394 [23:48:27] ejegg - up to 3856 records affected (some will have been deleted since last recalc) [23:50:22] I guess we have a deleted contact where either [23:50:22] 1) their email still exists - they should be updated or [23:50:22] 2) their email no longer exists - in which case we have [23:50:22] a) they are not yet in silverpop or [23:50:22] b) they are already in silverpop [23:50:22] For 2a - I don't think they would be added [23:50:24] For 2b - I'm not sure zero-ing out their values is a bad thing. They also wind up on the suppression list [23:50:45] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/619394 (owner: Eileen) [23:51:18] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/tools into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/619394 (owner: Eileen) [23:52:12] (PS5) Eileen: Add code comments [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/618410 (https://phabricator.wikimedia.org/T253152) [23:52:36] (PS4) Eileen: Remove blank address test [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/619078 [23:52:51] (PS4) Eileen: Remove function moved to deduper [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/618886 (https://phabricator.wikimedia.org/T256316) [23:56:00] !log tools revision changed from 22550f38c5 to 9a89f45974 [23:56:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:57:20] I'm just trying running the export - dedupe job not currently running (schedule on - but I checked run-job -rl to make sure I could kick off) [23:57:37] checking to see it finishes