[00:03:02] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Ingenico, Recurring-Donations: Failed recurring donation attempts making it to status 600 in Ingenico - https://phabricator.wikimedia.org/T281091 (Cstone) Error message wise, starting at 2021-04-22 04:21:32 UTC we started... [00:06:23] eileen: this one https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm/+/681857 ? [00:28:09] eileen: oh hi glad to hear about the jury non-duty [00:28:20] eileen: dunno if you saw, I rebased the entire chain of patches [00:28:28] on top of the Civiproxy getpreferences api [00:28:39] since that got merged to master [00:30:22] cstone: I'm guessing it's something that contains this: https://github.com/civicrm/civicrm-core/pull/20146/commits [01:15:15] cstone: yep [01:15:35] AndyRussG: yeah - I'm relieved not to be picked [01:18:10] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682346 (https://phabricator.wikimedia.org/T279983) (owner: Eileen) [01:19:08] AndyRussG: I hit this test error locally on your preference stuff - Invalid data in e-mail preferences message. [01:19:26] it was the regex but I didn't know what the right answer wa [01:19:26] s [01:24:42] (CR) Cstone: [C: +2] "Looks good, thanks for the updates!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/677018 (https://phabricator.wikimedia.org/T268497) (owner: AndyRussG) [01:26:24] (CR) Eileen: Move logging to extension (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682346 (https://phabricator.wikimedia.org/T279983) (owner: Eileen) [01:27:01] eileen: ah ok hmmm I'll check a bit later! [01:27:03] thx! [01:28:20] (Merged) jenkins-bot: Email Preference Center: Guzzle connetor to Civiproxy [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/677018 (https://phabricator.wikimedia.org/T268497) (owner: AndyRussG) [01:29:13] (PS7) Eileen: Move logging to extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682346 (https://phabricator.wikimedia.org/T279983) [01:29:26] (CR) Eileen: "I've tweaked the comment" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682346 (https://phabricator.wikimedia.org/T279983) (owner: Eileen) [01:29:59] (PS7) Eileen: E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) [01:37:21] eileen: I broke something locally trying to test that security patch (I dont think the patch did it, something I did) and its getting late for me now, I can look at it again tomorrow [01:38:01] cstone: no worries - like I say - I don't think that security hole affects us so not urgent [01:45:24] (CR) jerkins-bot: [V: -1] E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [01:47:05] yay - logging on passing now [01:55:42] (PS4) Eileen: Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) [02:00:48] AndyRussG: ping me if you have time to talk about the tools one - I should get onto that I guess [02:10:49] (CR) jerkins-bot: [V: -1] Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) (owner: Eileen) [02:20:22] eileen: okok yeee it's next on my liste, I should be back working again in about 1 hr [02:29:14] cool [03:27:18] eileen hey looking at https://phabricator.wikimedia.org/T270731 now [03:27:40] thx AndyRussG [03:27:50] eileen :) [03:28:00] btw so you're able to run the export under Docker currently? [03:51:31] AndyRussG: there was a point when I think I could... [04:07:43] (PS1) Eileen: Keep imported org details on update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682793 (https://phabricator.wikimedia.org/T278892) [04:08:55] eileen: so probably we'd want to adapt the python code to read command line arguments using the argparse library [04:09:13] https://docs.python.org/3/library/argparse.html [04:09:29] AndyRussG: we need a whole new library? (python is a foreign country to me :-) [04:09:35] It might seem like a heavyweight solution, but it's quite straightforward to use [04:09:45] cool [04:09:50] and eventually I think we might want to accept other arguments [04:09:56] Yes - I agree [04:10:02] cool! [04:10:24] I think the python code suffers from generally not having had much love by anyone truly comfortable with python [04:10:26] then normally there should be a way to substitute python variables into the actual queries [04:10:41] all the 5-minute cleanups I'd do in php I can't do in python [04:11:00] hmmm right... I mean, it's more or less just a wrapper for sql [04:11:37] I'm having trouble finding doc about the way the python is talking to the db [04:12:14] but there must be some variable substitution way to pass data into the queries [04:13:36] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (Eileenmcnaughton) @RLewis @MDemosWMF I've put up a patch that is a... [04:14:32] ah found it [04:19:54] eileen: I guess all put notes there on the task then? [04:20:06] eileen: I guess all put notes there on the task then? [04:20:08] AndyRussG: yep [04:20:23] okok cool (sorry for duplicate message ^) [04:20:40] (I pressed up and then enter thinking I was in the terminal, not IRC) [04:20:45] lol [04:21:07] (CR) jerkins-bot: [V: -1] Keep imported org details on update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682793 (https://phabricator.wikimedia.org/T278892) (owner: Eileen) [04:22:44] eileen: on another topic, do you have easily on hand the command you used to run that queue consumer test that failed for you locally? [04:23:06] AndyRussG: I ran it through the unit test [04:23:26] https://docs.civicrm.org/dev/en/latest/testing/phpunit/ ? [04:24:04] https://www.irccloud.com/pastebin/r01cJSlt/ [04:24:35] but unit test I mean this one https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/682349/6/drupal/sites/default/civicrm/extensions/wmf-civicrm/tests/phpunit/api/v3/Preferences/CreateTest.php [04:26:31] (PS5) Eileen: Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) [04:26:45] (PS2) Eileen: Keep imported org details on update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682793 (https://phabricator.wikimedia.org/T278892) [04:29:53] eileen: docker@civicrm:/srv/civi-sites/wmff/drupal$ phpunit7 sites/default/civicrm/extensions/wmf-civicrm/tests/phpunit/api/v3/Preferences/CreateTest.php [04:29:55] PHP Fatal error: Trait 'Civi\Test\Api3TestTrait' not found in /srv/civi-sites/wmff/drupal/sites/default/civicrm/extensions/wmf-civicrm/tests/phpunit/api/v3/Preferences/CreateTest.php on line 17 [04:30:26] hmm are you picking up the bootrap.php? [04:30:40] no idea [04:31:03] but is that how you run the test? or ahhh I think you said you have your IDE run them? [04:31:20] yep [04:31:33] I right click & choose 'debug test' [04:31:46] hmmm interesting that it knows to do that inside the docker container [04:31:49] but you can run phpunit from the root dir of the wmff crm directory [04:31:50] or does it? [04:32:03] yep it does - it has a docker plugin [04:33:52] hmmmmm [04:34:12] getting the same error running inside the container from /srv/civi-sites/wmff/civicrm [04:34:23] move up one level [04:35:18] ahh ok that worked! [04:35:28] docker@civicrm:/srv/civi-sites/wmff$ phpunit6 drupal/sites/default/civicrm/extensions/wmf-civicrm/tests/phpunit/api/v3/Civiproxy/PreferencesTest.php [04:35:48] looks about right - although we normally use phpunit 7.5ish [04:35:56] (which is in in vendor) [04:36:10] but if it runs on phpunit 6 then that's fine [04:38:13] okok [04:38:17] yeah the test passes for me [04:38:32] but the command line invocation does fail with the "invalid data" thing [04:40:12] eileen: here's the problem: the hash is abx, but it should only have 0-9 and a-f in it [04:40:21] so in that sense the test is a failure at failing [04:40:25] it should fail but doesn't [04:40:30] AndyRussG: oh - of course! [04:40:45] but the preferred_language IS failing I think [04:41:50] (CR) jerkins-bot: [V: -1] Keep imported org details on update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682793 (https://phabricator.wikimedia.org/T278892) (owner: Eileen) [04:42:24] eileen: I don't see a failure from that? on the command line it should just be language, since that's the field name in the message [04:42:31] (CR) jerkins-bot: [V: -1] Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) (owner: Eileen) [04:45:34] weirdest thing also, when I turn on cli xdebug, phpunit doesn't start debugging, even though other php cli stuff does! [04:47:04] (PS3) Eileen: Keep imported org details on update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682793 (https://phabricator.wikimedia.org/T278892) [04:47:15] I think that command isn't running the test... no change if I change the expected language for example [04:48:58] as they say, never trust a test that you haven't seen fail.... [04:54:49] (PS6) Eileen: Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) [04:56:09] AndyRussG: you could just run phpunit.xml from the root dir with no filters? [04:56:25] (it would take longer but ...) [04:58:27] eileen: now I'm getting errors! after updating to your latest version of the patch [04:58:43] (haven't tried phpunit.xml yet) [04:58:48] the right errors - or the wrong errors [04:59:04] - did I repush a version locally over yours by accident? [05:00:17] yes, getting the right errors [05:01:17] eileen: and no, you didn't overwrite my changes with a previous version or anything... https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/682349/6..7 [05:01:25] phew [05:01:51] ;) [05:09:35] (PS1) Eileen: Remove more unused code [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682797 (https://phabricator.wikimedia.org/T270678) [05:15:22] (PS8) AndyRussG: E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [05:18:56] eileen: where does the Communication.opt_in field actually live in the db? [05:19:46] so it's a custom field - the civicrm_custom_group table contains info about all the custom field groups & will map the name of 'Communication' to a table name [05:20:15] civicrm_custom_field does the same for the field part (although I'm guessing the field name is opt_in) [05:22:04] eileen: ok cool... ahh I think there was an error in the Preferences.create api, which the test actually caught! one sec [05:22:29] - 'Communication:opt_in' => $params['send_email'], [05:22:31] + 'Communication.opt_in' => $params['send_email'], [05:25:06] also, oops, does CRM_Utils_Type::T_BOOLEAN equate truth with the param's presence and falsehood with its absence? [05:32:07] (CR) AndyRussG: [C: -1] "Hmmm it seems this doesn't set Communication.opt_in to false when send_email=false in the message..." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [05:36:19] AndyRussG: that sounds ug [05:39:49] (CR) jerkins-bot: [V: -1] E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [05:41:06] eileen: heheh fun fun... also note I did upload some minor fixes [05:41:15] but that bit is indeed broken [05:59:31] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (AndyRussG) >>! In T270731#7032420, @Eileenmcnaughton wrote: > Default offset - we ca... [06:05:46] I'm finishing up [06:10:55] eileen: yeah same, just about to head in :) cya! [12:31:36] PROBLEM - check_puppetrun on frmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 14 minutes ago with 3 failures. Failed resources (up to 3 shown): File[/etc/vim/vimrc.local],File[/etc/update-motd.d/99-footer],File[/etc/motd.tail] [12:36:36] RECOVERY - check_puppetrun on frmon1001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:15:14] Wikimedia-Fundraising-CiviCRM: Benefactor Page Listing Custom field - https://phabricator.wikimedia.org/T281268 (RLewis) [14:46:07] hi fr-tech! [14:46:26] hi AndyRussG - i see there was something more you wanted my input on for the QC patch? [14:46:32] sorry if that review was unclear [14:54:00] looking [14:55:36] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Benefactor Page Listing Custom field - https://phabricator.wikimedia.org/T281268 (DStrine) [14:55:52] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Benefactor Page Listing Custom field - https://phabricator.wikimedia.org/T281268 (DStrine) a:DStrine→None [14:59:16] oh, abandoned? [15:19:40] hey ejegg! yeah it's now in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/682349 [15:19:45] hi fr-tech :) [15:20:04] ejegg: the issue was where to put the queue consumer classes, but I think we have a place for now [15:20:20] there's an issue with the api-based patch ^ though, see the last coment [16:15:43] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (MDemosWMF) @Eileenmcnaughton thanks for explaining those scenarios... [16:15:43] oop, wikibugs is on break [16:15:44] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/682984 [16:15:51] updating our branch of core [16:16:05] with a few more things from upstream [16:16:09] (PS9) AndyRussG: E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [16:16:37] hmm, or maybe wikibugs just doesn't alert on our core branch here any more? [16:36:52] Hey fr-tech I'm trying to beat a headache this morning but hopefully I can win by sprint planning [16:37:30] cstone aww hope you feel better soon :) no worries of course [16:37:41] (wrt sprint planning) [16:38:25] I guess it's too late now to use "headache" as the first word in a sprint name [16:39:00] Fundraising-Backlog: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (krobinson) [16:40:04] hope you feel better soon cstone ! [16:55:23] weirdest thing, now on the queue consumer I'm getting an exception thrown when it tries to log: Civi::log('wmf')->info('Executing: Preferencesqueue.consume'); [16:55:36] WD php: Error: Class name must be a valid object or a string in Civi\Api4\Generic\DAOGetAction->_run() (line 91 of [error] [16:55:37] /srv/civi-sites/wmff/civicrm/Civi/Api4/Generic/DAOGetAction.php) [16:55:47] and that wasn't happening before, no idea what could be different now [17:05:03] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (TSkaff) [17:06:19] (PS8) AndyRussG: Move logging to extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682346 (https://phabricator.wikimedia.org/T279983) (owner: Eileen) [17:06:39] (PS7) AndyRussG: Save full name to addressee where supplied [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682356 (https://phabricator.wikimedia.org/T275445) (owner: Eileen) [17:07:27] (PS10) AndyRussG: E-mail pref ctr queue consumer using api [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [17:07:56] fr-tech for anyone working on civi stuff, I've rebased this whole chain: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/682349 [17:21:29] Fundraising-Backlog, FR-Japan: Add Japanese translations to Monthly Convert on payments wiki - https://phabricator.wikimedia.org/T281293 (TSkaff) [17:32:34] fr-tech I kept getting that DAO error, also when trying to rebuild stuff with civi buildkit [17:32:46] now resetting all the source code and persistent storage, to see if it works [17:33:09] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (Ejegg) Weird, it looks like they may have gotten a form using the special diagnostic language code 'qqx'. Here's how the main page of wikipedia looks with qqx: https://en.wikipedia.org/?u... [17:33:44] AndyRussG: I'm guessing ti has to do with the new logging channels [17:33:47] *to do [17:33:53] ejegg: indeed [17:34:22] I debugged into it, and it's trying to find a class called api\v4\Monolog or something [17:34:27] but it was working fine last night! [17:34:29] ahha [17:34:42] sounds like we may just be missing some namespacing? [17:34:44] lessee [17:34:49] something funny was happening when I was switching back and forth between civi branches [17:34:53] so the monolog extension got merged? [17:35:28] ejegg: no but I've rebased everything queue-consumer-email-preference-y on that change [17:35:37] ok, lemme get that payments-wiki patch up and then I'll pull down the new crm code [17:36:13] ejegg: cool thanks! I'll let you know if refreshing everything fixes it [17:36:28] might possibly need a composer update too? [17:36:35] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (krobinson) @Ejegg They are all replying to the Brazilian fundraising email send that went out today, not coming from the banners. Could one of the links in the email be off? [17:36:58] it's in DAOGetAction line 90, where it says $baoName = $this->getBaoName(); [17:37:03] and $baoName comes up null [17:37:11] and that dies 'cause it tries to call methods on that [17:37:16] thx!! [17:37:18] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (Ejegg) definitely worth asking what link they followed to get that form! [17:40:26] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (krobinson) ZD strips the responses of the links so we can't see them all, but these are the links from 943543 - does one of them look off? > ------ > [1] > https://donate.wikimedia.org/... [17:43:28] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (Ejegg) @krobinson nope, those all look fine. @Pcoombe there's no chance donatewiki could be swapping uselang=pt-br to qqx when passing donors on to payments, is there [17:46:51] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (krobinson) Adding the email team who might be able to share the original links. @nisrael @MNoorWMF @KHaggard any ideas? Can we see what email links were sent to CID 21078369 / CID 2243... [17:58:22] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (MNoorWMF) original links are as follows (separated it to easily see the parameters): https://donate.wikimedia.org/?utm_campaign=C2021_PwrEmail &utm_medium=email &appeal=Appeal-JimmyQuote... [18:02:05] !log update payments-wiki from 9a4eef1375 to 44570561f2 [18:02:12] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:14:25] there's a fair amount of noise in the payments error log from 1006 and 1007 [18:18:19] ejegg|afk: could be because what Jeff_Green mentioned? [18:18:35] ah no that's on frdev I think [18:18:45] I haven't started on frdev yet, just about to [18:18:58] ahh cool [18:20:52] ejegg fr-tech by removing and re-cloning the wmff source code, removing persistent storage, and re-running the civi buildkit steps via setup.sh, I got rid of the DAO error [18:21:01] and the latest patch set on the queue consumer does seem to work! [18:41:04] (PS12) Jgleeson: Add CiviProxy setup [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/673094 (https://phabricator.wikimedia.org/T268683) [18:41:08] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (nisrael) It looks like the %% are breaking the links and automatically sending donors to the English-language donatewiki page. @krobinson how many tickets with this type of feedback were... [18:42:16] (CR) AndyRussG: "> Patch Set 8: Code-Review-1" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/682349 (https://phabricator.wikimedia.org/T268511) (owner: Eileen) [18:44:48] (PS13) Jgleeson: Add CiviProxy setup [wikimedia/fundraising/dev] - https://gerrit.wikimedia.org/r/673094 (https://phabricator.wikimedia.org/T268683) [18:46:32] woohoo AndyRussG ! [18:53:09] ejegg: yesterday we chatted a little about deploying the email preferences stuff to live and realised we hadn't talked much about it before or discussed how we wanted to productionize the civiproxy stuff. The current thinking is we will add a new f_c_u / rsync_blaster job for civiproxy and add any prod env config to localsettings on frpm. Does that sound right to you? [18:53:37] sure thing jgleeson [18:53:54] ah yeah, the initial assumption was that it would all be managed with the CRM code I think [18:54:05] since the root of that repo is not the webroot [18:54:28] yup [18:56:21] I might hang around later than usual tonight to talk it through. Jeff_Green is up to speed also so just needs to know exactly what we want adding and where. [18:57:29] jgleeson: this reminds me--I took a quick look at the gerrit project for civiproxy, will there be a specific deploy branch? [18:57:43] dstrine: ideally wanted it all deployed somewhere ready to test end-to-end by the end of this sprint. the consensus seems to be it made sense to push it all to staging for now [18:57:58] ahh yes Jeff_Green I forgot all about that [18:58:04] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (Pcoombe) Confirmed that going to the malformed http://donate.wikimedia.org/?uselang=pt-br%%&country=BR shows English on donatewiki, but then gets changed to `uselang=qqx` on clicking thro... [18:58:11] jgleeson: cool [18:58:22] actually lemme check that [18:58:40] because this code base doesn't have non-prod stuff we don't want like unit tests and stuff [18:59:11] hmm it does have some unused bits we probably don't want in production though to reduce any potential attack surface [18:59:31] I guess we could remove all that in the main branch though and use that for deployment also [19:05:51] jgleeson: if it complicates things to remove stuff and there's specific stuff you want to restrict, there's also the option of making the deploy tools set file permissions so the webserver can't access them [19:07:55] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Docker, Patch-For-Review: fundraising-dev CiviCRM set up failures (with fixes and workarounds) - https://phabricator.wikimedia.org/T279669 (DStrine) Open→Resolved [19:08:32] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Slow query in acoustic export - https://phabricator.wikimedia.org/T280894 (DStrine) Open→Resolved [19:08:45] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM theme update - https://phabricator.wikimedia.org/T278888 (DStrine) Open→Resolved [19:09:08] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Move all remaining dedupe code out of wmf_civicrm to deduper extension - https://phabricator.wikimedia.org/T270679 (DStrine) Op... [19:09:10] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021, Patch-For-Review: Remove unused code from wmf_civicrm - https://phabricator.wikimedia.org/T270678 (DStrine) [19:10:46] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 3 others: Most recent email address becomes primary email address, former email address is moved to “Other” - https://phabricator.wikimedia.org/T276391 (DS... [19:11:04] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) Open→Resolved [19:11:25] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-AutoTY-Email: Production of New Annual Fund Thank You Email (due to ED transition) - https://phabricator.wikimedia.org/T278363 (DStrine) ... [19:11:32] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-AutoTY-Email: Edits to MC Upsell TY Emails in production [by ED transition] - https://phabricator.wikimedia.org/T278460 (DStrine) Open→Resolved [19:12:21] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (DStrine) Open→Resolved [19:12:56] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, MW-1.37-notes (1.37.0-wmf.3; 2021-04-27), Patch-For-Review: Non-english soft descriptor - https://phabricator.wikimedia.org/T277598 (DStrine) Open→Resolved [19:13:31] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice: Deploy updates, including translations of new messages in user preferences - https://phabricator.wikimedia.org/T278055 (DStrine) Open→Resolved [19:17:23] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Docker, Patch-For-Review: Docker dev setup: stand-alone smashpig - https://phabricator.wikimedia.org/T279872 (DStrine) [19:19:14] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Smashpig, fr-email-preference-center: Add new queue settings for email-preferences - https://phabricator.wikimedia.org/T268512 (Dwisehaupt) Added in the config for smashpig/main.yaml... [19:21:05] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, fundraising-tech-ops, and 2 others: Add new queue settings for email-preferences - https://phabricator.wikimedia.org/T268512 (Dwisehaupt) [19:23:19] Fundraising Sprint Princess Mongodb, Fundraising-Backlog, Fr-planning-ahead, Epic: Epic: Create Preference Center for donors to manage email subscription preferences - https://phabricator.wikimedia.org/T125272 (Dwisehaupt) [19:23:22] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, fundraising-tech-ops, and 2 others: Add new queue settings for email-preferences - https://phabricator.wikimedia.org/T268512 (Dwisehaupt) Open→Resolved a:Dwisehaupt [19:32:23] Fundraising-Backlog, FR-Japan: Add Japanese translations to Monthly Convert on payments wiki - https://phabricator.wikimedia.org/T281293 (DStrine) @TSkaff and @CDenes_WMF These can be submitted through translate wiki. no engineering needed. Have fun! [19:44:04] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-AutoTY-Email: Production of New Annual Fund Thank You Email (due to ED transition) - https://phabricator.wikimedia.org/T278363 (CDenes_WMF... [19:45:49] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, FR-Ingenico, Recurring-Donations: Failed recurring donation attempts making it to status 600 in Ingenico - https://phabricator.wikimedia.org/T281091 (DStrine) [19:45:51] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog: Acoustic export failing due to permissions issues - https://phabricator.wikimedia.org/T280354 (DStrine) [19:45:54] Fundraising Sprint I 2021, Fundraising-Backlog, FR-AutoTY-Email: New Endowment TY email - https://phabricator.wikimedia.org/T280574 (DStrine) [19:45:56] Fundraising Sprint I 2021, Fundraising-Backlog, fundraising-tech-ops: Enable SSL for CiviCRM DB connections - https://phabricator.wikimedia.org/T280080 (DStrine) [19:45:58] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, FR-Docker, Patch-For-Review: Docker dev setup: stand-alone smashpig - https://phabricator.wikimedia.org/T279872 (DStrine) [19:46:02] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, and 3 others: Figure out how to replace watchdog - https://phabricator.wikimedia.org/T279983 (DStrine) [19:46:04] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: develop more detail plan on queue refactor for drupal upgrade - https://phabricator.wikimedia.org/T279962 (DStrine) [19:46:06] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, FR-Adyen: Break down tasks for Adyen reintegration, drop in web - https://phabricator.wikimedia.org/T277121 (DStrine) [19:46:08] Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, and 4 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civi... - https://phabricator.wikimedia.org/T277500 [19:46:10] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (DStrine) [19:46:12] Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Error when searching Wikimedia Recipients SmartGroup in Civi - https://phabricator.wikimedia.org/T272415 (DStrine) [19:46:14] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (DStrine) [19:46:16] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, and 3 others: Creating New Fidelity File Civi Import - https://phabricator.wikimedia.org/T275445 (DStrine) [19:46:18] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Remove unused code from wmf_civicrm - https://phabricator.wikimedia.org/T270678 (DStrine) [19:46:20] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, and 2 others: Convert large donation module to an extension - https://phabricator.wikimedia.org/T270667 (DStrine) [19:46:22] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Convert wmf_campaigns to an extension - https://phabricator.wikimedia.org/T270676 (DStrine) [19:46:24] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (RLewis) @Eileenmcnaughton and @MDemo... [19:46:26] Fundraising Sprint Corrugated super slide, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 6 others: Create civiproxy on docker - https://phabricator.wikimedia.org/T268683 (DStrine) [19:46:28] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising Sprint Princess Mongodb, and 5 others: Consume new messages from Email Preferences form - https://phabricator.wikimedia.org/T268511 (DStrine) [19:46:30] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 8 others: Create new subpage for Special:EmailPreferences - https://phabricator.wikimedia.org/T268510 (DStrine) [19:46:33] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, and 2 others: Civi: enable agents to view and edit the dedupe 'nickname' rules list - https://phabricator.wikimedia.org/T244404 (DStrine) [19:46:35] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 8 others: Use Guzzle to make API request to CiviProxy to retrieve opt-in / opt-out fields. - https://phabricator.wikimedia.org/T268497 (DS... [19:46:37] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 9 others: Add CiviProxy to crm repo, write configuration or code for filtering API calls - https://phabricator.wikimedia.org/T268495 (DStr... [19:46:39] Fundraising Sprint Airline Passenger Experience, Fundraising Sprint Bee Wheel, Fundraising Sprint Corrugated super slide, Fundraising Sprint Downed power line jump rope, and 17 others: Fr-tech chores list - https://phabricator.wikimedia.org/T258527 (DStrine) [19:49:28] Fundraising Sprint I 2021, Fundraising-Backlog, fr-email-preference-center: Plan deploy for MVP for e-mail preference center - https://phabricator.wikimedia.org/T281302 (AndyRussG) [19:50:26] Fundraising Sprint I 2021, Fundraising-Backlog, fr-email-preference-center: Plan deploy for MVP for e-mail preference center - https://phabricator.wikimedia.org/T281302 (AndyRussG) [19:56:17] Fundraising-Backlog, MediaWiki-Vagrant, MediaWiki-extensions-DonationInterface: Payments vagrant role has duplicated settings, needs more documentation - https://phabricator.wikimedia.org/T135653 (XenoRyet) Open→Declined [19:56:19] Fundraising-Backlog, fundraising-tech-ops, Epic: epic: frtech onwiki documentation overhaul - https://phabricator.wikimedia.org/T209872 (XenoRyet) [20:02:25] Fundraising-Backlog: deprecate and or move wikitech documentation to Mediawiki - https://phabricator.wikimedia.org/T281304 (DStrine) [20:02:37] Fundraising Sprint I 2021, Fundraising-Backlog: deprecate and or move wikitech documentation to Mediawiki - https://phabricator.wikimedia.org/T281304 (DStrine) [20:04:04] Fundraising Sprint I 2021, Fundraising-Backlog: deprecate and or move wikitech documentation to Mediawiki - https://phabricator.wikimedia.org/T281304 (AndyRussG) [20:04:06] Fundraising-Backlog, fundraising-tech-ops, Epic: epic: frtech onwiki documentation overhaul - https://phabricator.wikimedia.org/T209872 (AndyRussG) [20:07:45] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint I 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (MDemosWMF) @RLewis thanks for that i... [20:20:02] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-AutoTY-Email: Production of New Annual Fund Thank You Email (due to ED transition) - https://phabricator.wikimedia.org/T278363 (DStrine) ... [20:20:17] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, and 2 others: Production of New Annual Fund Thank You Email... - https://phabricator.wikimedia.org/T278363 [20:21:48] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, FR-Docker, Patch-For-Review: Docker dev setup: stand-alone smashpig - https://phabricator.wikimedia.org/T279872 (AndyRussG) [20:39:23] fundraising-tech-ops: Create fundraising_code_update project for civiproxy - https://phabricator.wikimedia.org/T281311 (Ejegg) [20:55:49] Fundraising-Backlog: Civi stock TY email template missing gift date - https://phabricator.wikimedia.org/T281313 (MDemosWMF) [21:08:04] (PS1) Jgleeson: Remove unused CiviCRM Extension [wikimedia/fundraising/crm/civiproxy] - https://gerrit.wikimedia.org/r/683085 [21:09:52] ejegg: that patch removes the unused civi extension bundled in with the civiproxy project. once removed I'll cut off a new deployment branch from that [21:11:45] fundraising-tech-ops: Create fundraising_code_update project for civiproxy - https://phabricator.wikimedia.org/T281311 (jgleeson) [21:13:08] oops, I already created the branch in the gerrit UI at https://gerrit.wikimedia.org/r/admin/repos/wikimedia/fundraising/crm/civiproxy,branches [21:13:19] but it'll be simple to reset to the new pointer I think [21:13:20] also I just updated that ticket as we only need to drop in one file, config.php. turns out the other file we're dropping in in our docker setup is specific to our local env and how we get around our self-signed certs using civiproxy. [21:13:34] yah, easy to delet + recreate [21:13:41] cool [21:13:47] oh cool, just one file [21:14:21] (CR) Ejegg: [V: +2 C: +2] Remove unused CiviCRM Extension [wikimedia/fundraising/crm/civiproxy] - https://gerrit.wikimedia.org/r/683085 (owner: Jgleeson) [21:14:43] we're manually v+2-ing and submitting in that repo, right, jgleeson? [21:14:43] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (... [21:14:57] yeah there's no CI job [21:15:06] cool cool, shouldn't be needed [21:17:21] oh hey, i just noticed that the oreo coaster pic is by the WMF's own Kartik Mistry [21:18:21] I'll push up a draft civiproxy/config.php to local settings with the config I _think_ we'll need [21:19:18] jgleeson: ok, I just submitted that patch + updated the deployment pointer [21:29:29] Jeff_Green: dwisehaupt any ideas what the internal address of the civiproxy site will be? I need to provide the base url of the proxy service to the config [21:30:58] ah yeah, lemme file a task for network config [21:35:49] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (... [21:42:29] Papal payments :) [21:44:17] ok fr-tech I've pushed up the draft civiproxy config to localsettings [21:44:27] anything else we need? [21:47:15] rockin! [21:47:36] jgleeson: one more tech-ops task for the apache config maybe? [21:47:47] oh yeah good call [21:50:41] ejegg: I just added it I think above where you're typing [21:50:48] sorry nvm [21:50:50] :) [21:51:09] Fundraising-Backlog, FR-LATAM: Brazilian donors seeing odd donation form - https://phabricator.wikimedia.org/T281285 (Pcoombe) Open→Resolved a:Pcoombe So for some reason if the uselang parameter has a % sign in, then `wgUserLanguage` is set to qqx, and that's what donatewiki was passing to pa... [21:53:22] fundraising-tech-ops: Network configuration for Civiproxy - https://phabricator.wikimedia.org/T281320 (Ejegg) [21:54:04] whoa, weird, pcoombe. Nice sleuthing! [21:55:25] AndyRussG: feels like the dockerhub images thing might be a longer-term temporary arrangement.. maybe? should we create an fr-tech dockerhub account and use fr-tech@ as the account email? [21:55:54] Fundraising-Backlog, FR-EN6C, FR-UK, Thank-You-Page: Thankyou page link invalid for UK donors - https://phabricator.wikimedia.org/T276928 (Pcoombe) Open→Resolved a:Pcoombe I [added some simple javascript](https://thankyou.wikipedia.org/w/index.php?title=MediaWiki:Common.js&diff=prev&o... [21:56:18] I added one for civiproxy to my own account earlier today [22:01:45] enjoy tech-talk all. catch you tomorrow [22:03:37] fundraising-tech-ops: Add Apache config to serve Civiproxy - https://phabricator.wikimedia.org/T281321 (Ejegg) [22:04:16] fundraising-tech-ops, fr-email-preference-center: Add Apache config to serve Civiproxy - https://phabricator.wikimedia.org/T281321 (Ejegg) [22:04:31] fundraising-tech-ops, fr-email-preference-center: Network configuration for Civiproxy - https://phabricator.wikimedia.org/T281320 (Ejegg) [22:05:00] fundraising-tech-ops, fr-email-preference-center: Create fundraising_code_update project for civiproxy - https://phabricator.wikimedia.org/T281311 (Ejegg) [22:10:46] fundraising-tech-ops, fr-email-preference-center: Add Apache config to serve Civiproxy - https://phabricator.wikimedia.org/T281321 (Ejegg) [22:11:52] jgleeson|away: sorry, I was AFK for a bit there on the phone w/family [22:12:55] jgleeson|away: I suspect we're going to want to go with the fqdn of the host it resides on, which will make it easier to deal with https and certs [22:17:49] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (... [22:22:37] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (Eileenmcnaught... [23:05:55] Fundraising Sprint Humongous bacteria petting zoo, Fundraising Sprint Interstitial ads halfway down the coaster hill, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (Eileenmcnaught... [23:44:48] ejegg: silverpop_export now has 21802748 [23:45:23] SELECT TABLE_NAME,TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'silverpop'; [23:45:37] I think it might be OK to do a full export from the data there now [23:57:31] hmm, I got 24.5 million just doing the join from email_map to export_staging with the opted_out and opted_in conditions [23:57:50] so let's see, would the 3 million fewer be missing from email_stat? [23:58:12] or rather export_stat