[00:16:30] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Unexpected side-effect of CentralNotice wmf_deploy branch strategy - https://phabricator.wikimedia.org/T179536 (Krinkle) My suggestion would be to codify your desired permissions model onto the Git master branch for CN, instead using this additiona... [00:25:23] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Civi-Dedupe: Dedupe - document for DS the various bits of handling that are in the dedupe code - https://phabricator.wikimedia.org/T276392 (Eileenmcnaughton) OK - I'll write up what happens here & then cons... [00:27:11] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Developer Productivity: Unexpected side-effect of CentralNotice wmf_deploy branch strategy - https://phabricator.wikimedia.org/T179536 (Krinkle) [00:56:00] (PS1) AndyRussG: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/678667 [00:57:59] (CR) AndyRussG: [C: +2] Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/678667 (owner: AndyRussG) [01:03:18] (Merged) jenkins-bot: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/678667 (owner: AndyRussG) [01:20:58] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678668 [01:21:32] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678668 (owner: Ejegg) [01:35:35] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678668 (owner: Ejegg) [01:42:33] hmm, no wikibugs notification for our core branch? [01:48:44] hey ejegg [01:49:40] how do you feel about me taking advantage of the wmf day off to get out the civi point version [01:50:33] (CR) Ejegg: [C: -1] "Still having problems with the settings UI in PS8" (2 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676175 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [01:50:38] hi eileen [01:50:44] how goes [01:50:47] yeah, sounds good. let me try pulling it down [01:50:51] oh, it's going OK [01:51:02] those settings - they can take a bit of cache clearing [01:51:28] oh & I tested on the later civi version too [01:51:35] yep yep, I deleted the row from the db and flushed the system [01:51:59] hmm, but why not make it similar to the other multiselect settings on that same page? [01:52:46] if I change it to match those, I can persist stuff in the settings ui that then gets used just fine in the deduper [01:53:32] let's see, I'll pull the latest point release down too [01:54:53] yeah the serialize is supposed to be the 'right' way [01:55:15] (PS1) Eileen: Since merged change upstream [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/678673 [01:55:34] there is a very small change merged to 5.37 since I did that patch - I put it up as a follow on in case you have already pulled it [01:56:38] I should probably update contactlayouteditor & relationshipblock as the new functionality might need the later versions [02:03:11] (PS1) Eileen: Update contactlayouteditor & relationship block for 5.37 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678676 (https://phabricator.wikimedia.org/T278870) [02:04:45] dang, upgrade failed on a table alter adding an fk: FK_civicrm_group_saved_search_id` FOREIGN KEY (`saved_search_id [02:05:06] ug [02:05:15] ccccccknfuvhvclubnbjtbefgklgtcvbhlnjhnthvijv [02:05:18] Duplicate key on write or update")] [02:05:29] & this was a clean install? [02:05:39] hah, nope, my crufty one [02:05:50] sentimental reasons, I guess? [02:05:54] lol [02:06:03] yeah there have been changes around that [02:06:13] will do a clean install on the docker container and upgrade that, just need to plug in [02:06:22] looks like it already exists on live? [02:06:53] I'll squash that second commit in then since you obviously will take a bit to get the docker part sorted [02:07:28] (PS6) Eileen: 5.37 rc upgrade [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/675616 [02:07:42] (Abandoned) Eileen: Since merged change upstream [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/678673 (owner: Eileen) [02:23:57] huh, the sites/default dir was left -w so it wouldn't delete the old settings [02:24:50] setting up again [02:26:28] on docker? [02:26:43] yeah it can be a bit brutal - make sure you have the latest docker [02:29:50] as in latest fundraising-dev [03:02:13] back in a bit, baby needs lullabies [03:13:30] rockabye baby [03:34:13] hah, mostly showtunes and folk rock [03:34:34] man, she really didn't want to go back to sleep this time though [03:34:56] grr, why did it try to enable wmf_reports? [03:35:18] ok, restarting the install with a fresh clone of wmff [03:35:35] still don't see how it had that in the enabled_modules file [03:36:02] and already had the module deleted [03:45:45] yeah I thought that was gonner than a gone thing [03:45:58] don't worry she'll sleep well [03:46:02] once she's a teenager [03:46:04] .... [03:49:19] lol [03:49:49] hmm, even with that 'drush dis update' the module enablement is draggy as heck [03:50:07] there was another drupal setting to disable pings, right? [03:51:18] drupal_http_request_fails ? [03:51:29] yeah I don't know why it is so slow but perhaps there IS a pingback happening or trying to [03:55:09] oh hey, you can set an $override_function = variable_get('drupal_http_request_function', FALSE); [03:55:48] will try that next time I have to run setup [04:01:08] worth a shot! [04:05:24] well, it's a clean upgrade from a clean install [04:06:11] cool [04:09:18] (PS2) Eileen: Update contactlayouteditor & relationship block for 5.37 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678676 (https://phabricator.wikimedia.org/T278870) [04:09:20] (PS1) Eileen: Move rest of dedupe handling out of wmf_civicrm.module [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678707 (https://phabricator.wikimedia.org/T270679) [04:10:17] ok, i'mma pull that field skipping patch onto the docker site [04:12:34] k, got a shell on the civi container and am flushing system [04:13:39] eileen, even on a freshly upgraded 5.37 if I save two custom tables and return to that settings screen there's nothing shown as selected [04:13:44] cool - yeah civicrm_cache table can also hold stuff [04:13:59] hmm - [04:14:21] on your box does it repopulate with the multiple tables? [04:14:27] I mean - if something else works for you I'm happy to go with that! [04:14:49] I just don't understand how it should work as 'string' when the rest of the multiselects are 'array' [04:15:19] actually you are rigth - it only works with a single field [04:15:46] it should be 'an array of strings' & the serialize indicates array so combined with string that should mean an array of strings [04:16:00] but maybe just push up what works for you! [04:16:28] ok [04:17:39] (CR) Ejegg: [C: +2] "Works for our purposes! I'll push up a teeny change for the case of selecting multiple custom groups in the UI." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676175 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:20:19] thanks ejegg [04:21:28] (CR) Ejegg: [C: +2] "Looks good!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676180 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:22:22] (Abandoned) Eileen: WIP - see what fails [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/677062 (owner: Eileen) [04:30:21] yay - tests are passing post me ripping out the dedupe handling from wmf_civicrm [04:34:48] (Merged) jenkins-bot: Move skipping of wmf-donor fields to deduper [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676175 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:34:54] (Merged) jenkins-bot: Move handling for do_not_solicit field to extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676180 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:37:35] (CR) Ejegg: [C: +2] "Agreed with that filed bug - we should enter things more cleanly" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676181 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:40:27] (PS1) Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678710 [04:40:37] (CR) Eileen: [C: +2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678710 (owner: Eileen) [04:40:58] I'm gonna deploy those merged ones to make sure I get the settings - I might yet do another deploy shortly after [04:42:58] (CR) Ejegg: [C: +2] "Nice, the part where you find any potential instances of 0 primaries is especially clever" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/677061 (owner: Eileen) [04:44:13] ok [04:44:31] ejegg: should we push out 5.37? I think you have been testing on it? [04:44:46] eek, 677414 looks like it might be a bit complex for me to review before bed tonight [04:45:18] yah, I'll look more at the civicrm update for now and pause on that chain till tomorrow [04:45:28] yeah ejegg that one is hell [04:46:17] I thought I'd started to simplify the merge code but it's still nightmarish [04:47:03] however, the test cover is fairly good [04:51:04] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, FR-Civi-Dedupe, fr-donorservices: Civi: enable agents to view and edit the dedupe 'nickname' rules list - https://phabricator.wikimedia.org/T244404 (Eileenmcnaughton) I had a go at creating this locally with search-kit & it was pretty easy... [04:51:55] (Merged) jenkins-bot: Move preferred_language [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676181 (https://phabricator.wikimedia.org/T270679) (owner: Eileen) [04:57:45] hmm, repeatTransaction has an overhaul, not that we use it [05:01:08] yeah it is pretty well tested [05:03:37] well, the saved searches screen seems to work well [05:04:14] i guess testing with multiple users might find wonkiness but I'm happy to leave that for production :) [05:05:16] we shouldn't have any raw tokens in our templates that'll need updating, should we? [05:05:20] https://docs.civicrm.org/sysadmin/en/latest/upgrade/version-specific/#token-format [05:05:37] no - not at the moment [05:05:41] only place I can imagine would be scheduled reminders [05:05:42] ok, cool [05:05:52] yeah - it's specific to custom fields too [05:06:01] & it would only matter if we used them in an IF [05:06:07] (Merged) jenkins-bot: Add additional check on data before tearDown [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/677061 (owner: Eileen) [05:06:09] And I guess we can generate a signing key for future proofing [05:06:39] yeah I haven't dug into that much but it could be used in our preferences centre in future maybe [05:06:50] it's about secure remote forms [05:06:54] ok [05:07:30] ejegg eileen ahhh I need to add "Docker" to my ping words [05:07:35] lol [05:07:39] I'm about if I can be of assistance in anything [05:07:53] No - I think ejegg got up & running pretty quickly tbh [05:08:00] (CR) Ejegg: [C: +2] "Nice new features, no blockers found locally!" [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/675616 (owner: Eileen) [05:08:03] ah cool [05:08:19] heheh earlier I jumped into a conversation on another channel where CN was mentioned, and was greeted with "Hi CentralNotice" [05:08:21] hah AndyRussG, no, nothing specific to docker hanging us up here :) [05:08:26] hah [05:08:53] ah okok cool :) [05:09:07] AndyRussG: I suspect that sendmail thing made a difference [05:09:15] since ejegg didn't report any issues [05:09:30] just a brain-buster of a merge hook handler: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/677414/5/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/MergeHandler.php [05:09:58] ejegg: yeah & for background - I think I DID change approach somewhere along the way [05:10:07] ah yeah, on latest fr-dev + wiped wmff it installed with no issues [05:10:11] I started trying to mimic what the form would do much more [05:10:15] hmmmm [05:10:31] (just still slow) [05:11:00] I found another potential thing to make drupal not ping externally [05:11:06] will try it with the next build [05:11:41] ejegg: ah yeah was also going to say, the -w on the sites/default directory was something I also got, apparently something in the drupal/civi install left it that way when it errored out. But when the other thing was fixed, and it finished a clean install, the permisions on that dir were fine [05:12:03] are you sure it's an external ping? I thought it was just sending an e-mail to the local sysadmin [05:12:07] https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/drupal/includes/common.inc$800 [05:12:22] AndyRussG: I think maybe drupal pings home whenever a module is enabled [05:12:27] the email is separate [05:12:36] ohhh wow [05:12:42] that's horrific [05:12:48] but the module enablement is so slow I think it must be some kinda timeout [05:13:09] err, or maybe installed for the first time [05:13:25] is it the drupal modules or civicrm extensions (or both) [05:15:49] eileen the slowest part of setup is the drupal module enablement [05:16:52] ejegg: but doesn't it enable wmf_civicrm which enables the extensions? [05:17:16] oh hmm, that's a good point [05:17:32] lemme also echo those names out as they are enabled [05:17:43] good idea [05:22:08] (Merged) jenkins-bot: 5.37 rc upgrade [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/675616 (owner: Eileen) [05:23:14] (PS1) Eileen: Update CiviCRM to 5.37rc [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678711 [05:23:57] ejegg: it's hardly urgent but I think some of the 5.37 extra functionality relies on https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/678676 [05:24:16] (CR) Eileen: [C: +2] Update CiviCRM to 5.37rc [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678711 (owner: Eileen) [05:25:23] ah yeah, I'd be happy to +2 that, maybe we rebase it so it doesn't depend on the tricksy one? [05:25:36] oh sure I didn't realise it die [05:25:38] did [05:25:55] (PS3) Eileen: Update contactlayouteditor & relationship block for 5.37 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678676 (https://phabricator.wikimedia.org/T278870) [05:26:52] (CR) Ejegg: [C: +2] Update contactlayouteditor & relationship block for 5.37 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/678676 (https://phabricator.wikimedia.org/T278870) (owner: Eileen) [05:26:57] ejegg: actually https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/677414 was supposed to be before the tricky one - I thought they would need to be connected but they seem to be in different chains [05:27:09] actually - they are both tricky ones [05:27:38] oh yeah - that was the one you described as tricky [05:27:46] anyway - they both suck [05:28:19] and may conflict - but I'll deal with that when it happens [05:29:24] ok yep, it looks like it's the wmf_civicrm module enablement that takes the bulk of the time [05:29:39] ok - well that enables the extensions [05:29:54] and with the 'update' module disabled I'm not seeing drupal pings [05:30:13] ah good [05:30:19] (i set that drupal_http_request_function to something that just logs the url and returns a 404) [05:30:35] we disable that on live but locally it's actually useful as it notifies us about security updates [05:30:57] ah, ok [05:32:55] if it's slow maybe it can be off until the install is done? [05:33:22] well, doesn't actually look much faster [05:33:44] ah well I guess at least we know [05:33:47] that wmf_civicrm enablement is like 90%+ of the time [05:34:16] as in the module or the extension? [05:34:57] it's possible there is waaay to much cache clearing happening when it enables multiple modules [05:35:00] like 90% of the time to run the full civicrm buildkit is in that one enablement [05:35:14] then the rest of the modules come in pretty quick [05:35:28] oh, as in the drupal module [05:35:39] yeah - but if that module runs a long upgrade script AND installs a tonne of extensions it is also most of what is done [05:35:42] which enables the civi-specific extns [05:35:58] right right [05:41:21] (PS1) Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678712 [05:41:34] (CR) Eileen: [C: +2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678712 (owner: Eileen) [05:44:15] !log civicrm revision changed from ecc32d2a35 to 76bd8ff009, config revision is c5fc1b91e0 [05:44:21] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [05:51:04] g'night eileen + AndyRussG ! [05:51:16] ejegg: cya! [05:51:56] ya also gonna turn in indeed, cya eileen [05:52:08] night [06:41:41] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Figure out how to replace watchdog - https://phabricator.wikimedia.org/T279983 (Eileenmcnaughton) [08:01:15] (PS1) Raimond Spekking: Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 [08:06:15] (CR) jerkins-bot: [V: -1] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [08:13:13] (CR) Raimond Spekking: [C: +2] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [08:16:10] (CR) jerkins-bot: [V: -1] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [08:18:11] (CR) Raimond Spekking: [C: +2] "Could someone please merge this patch? I think the Jenkins error is not relevant to this revert but I do have the right to submit it. Than" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [08:23:35] (CR) jerkins-bot: [V: -1] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [13:21:38] PROBLEM - check_puppetrun on fran1001 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): File[/etc/vim/vimrc.local],File[/etc/update-motd.d/99-footer],File[/etc/motd.tail],File[/usr/local/bin/package_update_check] [13:21:38] PROBLEM - Host frdb1003 is DOWN: PING CRITICAL - Packet loss = 100% [13:26:38] RECOVERY - check_puppetrun on fran1001 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [13:26:54] RECOVERY - Host frdb1003 is UP: PING OK - Packet loss = 0%, RTA = 0.39 ms [13:59:31] Fundraising-Backlog, fundraising-tech-ops, FR-Tech-Analytics: Enable template processing - https://phabricator.wikimedia.org/T279292 (Jgreen) [14:51:27] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) [14:53:41] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) p:Triage→Medium [15:00:37] Fundraising-Backlog: Edits to MC Upsell TY Emails in production [by ED transition] - https://phabricator.wikimedia.org/T278460 (CDenes_WMF) Hi @DStrine @Cstone just a friendly reminder that the due date for this is April 15th due to the Katherine signatures. Thank you! [15:03:38] (CR) Jforrester: "Fundraising branches are extremely locked down; leaving to Elliott." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [15:06:36] ah drat, minfraud update is causing test fails [15:07:11] well, lemme roll that back to the previous version so we can just get 1.35.2 out [15:19:20] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, FR-AutoTY-Email: Coding HTML emails - https://phabricator.wikimedia.org/T277779 (CDenes_WMF) Hi @DStrine and @Cstone, I think we can mark this phab task as //done//; as the Chinese footer address... [15:20:49] (PS1) Ejegg: Roll back maxmind and respect for now [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678873 [15:21:10] (CR) Ejegg: [C: +2] Roll back maxmind and respect for now [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678873 (owner: Ejegg) [15:27:54] (CR) jerkins-bot: [V: -1] Roll back maxmind and respect for now [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678873 (owner: Ejegg) [15:35:26] (Abandoned) Ejegg: Roll back maxmind and respect for now [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678873 (owner: Ejegg) [15:38:21] cstone: some rabbit-related news in the NYT today: https://www.nytimes.com/2021/04/13/world/europe/darius-worlds-longest-rabbit-stolen.html [15:38:35] my friend just asked if I had stolen him [15:38:36] i have not [15:42:54] prime suspect ruled out... [15:51:33] owner is quite a character: https://en.wikipedia.org/wiki/Annette_Edwards [15:53:27] "reportedly at a cost of £5000 per year, covering 2000 carrots and 700 apples. " [16:02:41] that sounds similar to what my kids eat in a year. but maybe a little high on the apples. [16:13:23] Fundraising-Backlog: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) [16:23:20] ejegg: I read that the other day. Smells like an inside job! [16:23:34] howdy fr-teh [16:23:37] oops [16:23:39] fr-tech* [16:27:52] hi jgleeson fr-tech [16:28:19] jgleeson: are carrot really that expensive in the UK ? :) [16:28:28] *carrots [16:29:12] organic ones maybe... [16:30:22] it's possible Radagast needed a new rabbit to pull his sleigh [16:33:28] has anyone watched Seaspiracy on Netflix? [16:33:50] https://en.wikipedia.org/wiki/Seaspiracy [16:34:15] highly recommended. also plan not to eat fish again [16:47:17] jgleeson: ah yeah planning to watch that, though also afraid of that very consequence [16:48:00] alos hi fr-tech jgleeson ejegg cstone dwisehaupt :) [16:52:26] yeah AndyRussG it doesn't make easy watching if you like seafood. feels like it's a documentary that needed to be made though [16:55:35] jgleeson: yeah that's the sense I got from the trailer... for health reasons I try to serve fish or seafood to the kids at least once a week, though that might be wrong [16:56:21] I guess there might be some way to get local or local-ish seafood that's produced or harvested in a decent-ish way? [16:57:04] A couple years ago in Montreal I remember seeing people fishing in the St. Laurence river... which of course is where all the Great Lakes drain into... [16:57:58] I have eaten locally caught lake Ontario salmon but they do spend most of their lives not in lake Ontario :P [16:59:08] ahhh cstone that's a good point... yeah St. Laurence also has lots of affluents other than the Great Lakes [16:59:50] (most of which of course also have urban centres upstream... though smaller ones...) [17:00:28] the people who catch these salmon refuse to eat them (which is a whole different argument of why they are even doing it) [17:00:43] ya AndyRussG after reading up on some of the points made in the film I get the impression not all fishing is bad but industrialised fishing has reached a scale where is can't continue without destroying the ocean & us with it [17:00:57] were* [17:03:34] yeah that point sounds legit for sure [17:04:47] (CR) Ejegg: [C: +2] "Seems to have been caused by a recent library update. Reverted that for now, let's see if the tests pass." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [17:05:45] it dispels fairly widespread myths such as fish not feeling pain and the illusion that animals caught my mistake are release back into the water shortly... when the truth is most of them die the nets in the interim :( [17:05:57] by mistake* [17:12:38] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) [17:22:22] (PS1) Ejegg: Add return val to WmfFrameworkLogHandler [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678903 [17:22:33] (CR) Ejegg: [C: +2] Add return val to WmfFrameworkLogHandler [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678903 (owner: Ejegg) [17:24:22] ejegg: I did plan on checking out your log updates but got into a fight with syslog locally. hopfully get to them soon! [17:24:55] heh, i'm still fighting with it - hopefully ^^^ will make tests pass [17:26:28] (CR) jerkins-bot: [V: -1] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [17:37:29] (PS2) Ejegg: Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [17:42:40] (Merged) jenkins-bot: Add return val to WmfFrameworkLogHandler [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678903 (owner: Ejegg) [17:49:41] fr-tech for e-mail preferences, are we sure we want to use a queue to update donors' preferences, instead of just hitting the DB (via another CiviProxy call)? [17:50:08] decouple the world! [17:50:11] I feel like it's a kinda of situation where a user would expect the site to reflect their changed preferences right away [17:50:39] jgleeson welp I mean it's not a case where we can turn off the DB and have it still work, since we're going to the DB for the initial page load [17:50:41] i feel like the justification for that Andy would be to have clear lines between the "front-end" and the rest of the stack [17:51:02] i.e no direct link to the dbs from the frontend [17:51:15] jgleeson aren't those same lines just as blurred (or not?) with the call to get the initial pref values? [17:51:34] also, I don't think we'll every be expecting a super-high volume of preference changes? [17:52:15] yeah the api call in does muddy the waters a little [17:52:30] also seems like Civiproxy is a fine enough separator of front-end vs back-end [17:52:54] and it's like extra boilerplate in the form of another queue consumer job [17:53:19] I guess there is a different though as the initial call is read-only right? and the update is write [17:53:26] difference* [17:53:31] but I can see your point [17:54:09] in terms of code architecture I don't see much difference if it's read or write, civi api is still the back-end, Mediawiki extension is still the front-end, CiviProxy is still the go-between [17:54:33] also just trying to figure out how to wire up the queue consumer [17:54:49] I guess I'm thinking more in terms of having control over the writes on our side [17:55:00] from a security angle [17:55:02] hmmm [17:55:18] but it's a weakish stand [17:55:36] you mean, like being able to turn pause those writes temporarily to investigate an issue, without having to disable the user-facing thingy? [17:55:49] *like being able to pause [17:56:07] yeah I feel like being able to just shut off the consumers gives us a but more control of things writing to the db [17:56:20] but in that specific case we do lock things down tightly [17:56:26] the civiproxy calls, case [17:56:50] we might need some kind of maintenance mode for the e-mail pref center (or do we already get that for free with DI? I think maybe) [17:57:07] oh yeah we get that for free I think [17:57:19] I read that somewhere on our docs [18:00:34] jgleeson: if it were to indeed be a queue consumer, where would we get the name of the queue to consume, and where would we set it? I see some calls to $settings = Civi::settings() for example in ProcessSmashpigRecurring.php [18:00:43] but I'm not sure where those settings are [18:00:58] I know historically there has been a push to separate the frontend from the backend via queues which (I think) allows us to scale more easily and gives us some performance benefits. We decoupled contribution tracking last year. I guess if we can manage traffic via the civi-proxy layer that gives us more control than just a straight api call from mediawiki to civicrm. [18:01:31] hmm looks like that's in extensions/org.wikimedia.smashpig/settings/Smashpig.setting.php [18:01:54] ah AndyRussG that normally lives in smashpig config [18:02:13] are you using the smashpig queue library stuff? [18:02:28] jgleeson: ahh right and smashpig under civi will just still get its config from /etc [18:02:28] smashpig config, the yaml files [18:02:33] yeah [18:03:11] regarding using the sp queue library stuff, haven't gotten that far, but if u have code examples on hand that'd be fantastic! thx much btw :) [18:03:56] jgleeson: eileen made this example code for the set part of the preferences api: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/678464 [18:04:06] (I guess it should no longer be called getpreferences then) [18:04:56] I worked a bit on the contribution tracking queue consumer but that was in the old drupal/drush style. it probably has examples of how to pull in the smashpig stuff though so I'll dig it out [18:06:02] yeah good point on the name [18:06:47] (CR) Ejegg: [C: +2] Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [18:09:41] (Merged) jenkins-bot: Revert "Localisation updates from https://translatewiki.net." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/678689 (owner: Raimond Spekking) [18:12:33] grrr phpstorm isn't showing me the useful "show in github" link [18:12:39] it randomly loses that [18:14:52] oh it's my fault [18:15:14] since we moved the sites symlink it looks like I have to now add the remote for our drupal submodule for github also [18:15:30] sry AndyRussG... getting a link! [18:16:03] oh wait we don't have a drupal submodule :/ [18:16:27] wth can't it open the files in that directory then but it can the ones above in the project root [18:16:34] weeeeeird [18:17:08] https://github.com/wikimedia/wikimedia-fundraising-crm/blob/7b7315b3b8c80b0d92d059951026f2a72becd69a/drupal/sites/all/modules/queue2civicrm/contribution_tracking/ContributionTrackingQueueConsumer.php#L9 [18:17:20] Andy that's one of the consumers and it gets called here [18:17:45] https://github.com/wikimedia/wikimedia-fundraising-crm/blob/7b7315b3b8c80b0d92d059951026f2a72becd69a/drupal/sites/all/modules/queue2civicrm/contribution_tracking/wmf_ct_qc.module#L65 [18:18:43] all consumers extend from https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/c75040ecef74c6478bed6d2d6f6e7a6d5a500229/Core/QueueConsumers/BaseQueueConsumer.php#L21-L20 [18:18:55] (PS4) Cstone: Update 2021 thank you emails. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676681 [18:19:40] the constructor of that class, BaseQueueConsumer wires up the queue config here https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/c75040ecef74c6478bed6d2d6f6e7a6d5a500229/Core/QueueConsumers/BaseQueueConsumer.php#L77 [18:20:22] which in turn calls https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/c75040ecef74c6478bed6d2d6f6e7a6d5a500229/Core/DataStores/QueueWrapper.php#L31 [18:21:47] jgleeson: woohoo thanks so much, that's immensely helpful!! :) [18:23:51] fr-tech the updates to the ty emails are done if anyone wants to look real quick, just some wording changes https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/676681 [18:23:55] Fundraising-Backlog: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) [18:25:06] Fundraising-Backlog: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) p:Triage→High [18:25:07] The BaseQueueConsumer declares an abstract method 'processMessage()'. That method gets called on a per-message basis when $processed = QueueConsumer->dequeueMessages(); is called, which is usually the calling code for all the message consuming we do [18:25:27] jgleeson: hmmmm noice! [18:26:23] jgleeson: how do you think all this will be called on the command line from process control? for the ct queue consumer I see drush --user=1 -v -r /path/to/drupal ctqc [18:26:41] would we still have a similar entry point, or is that changing because moving away from drupal stuff? [18:28:32] i think the plan is still to use drush for process control calls and you can call civicrm api calls via the `drush civicrm-api` command [18:29:01] jgleeson: right that much I have [18:29:08] i think there's a few already in /srv/process-control on civi1001 [18:29:14] ah ok [18:29:27] jgleeson: yeah there are a bunch that call cvapi in process control, but I don't see any queue consumers per se [18:29:55] oh yeah that will be out first I think that lives exclusively within civicrm [18:30:12] ah okok [18:30:20] sorry if this isnt helpful but theres the optin queue consumer too that kinda would be similar? [18:30:23] jgleeson: but so if I understand correctly, it'd be a different civi api that would be called on the command line, and that would do the queue reading and such, and would in turn call the preferences api to set the new preferences for each queue message, is that correct? [18:30:38] cstone oh yeah that does sound helpful! [18:31:34] cstone so that one is just run via oqc command with drush [18:32:18] AndyRussG: possibly if we want to only speak to civicr via the https API. it might also be possible to implement the updates to the db directly from the consumer processing code. [18:32:19] cstone jgleeson ^ so in this case, it'll be similar, but the code will be a new civi api for consuming queue instead of whatever oqc drupal-ly thing is, right? [18:33:33] jgleeson: ahhh so in that case, we wouldn't have the setpreferences api per se as eileen suggested in her mock patch (https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/678464), no? [18:35:38] AndyRussG: you are talking about there ^ if you were gona undrupal it at the same time? [18:37:00] yeah i guess we could shortcut that if we wanted to. It might be nice to also do it via the API but it does add a little bit of an overhead (civi code calling out to its own web api adds a http call each time I think) [18:37:40] although if we do plan on sharing the extension the api route might be best [18:37:49] ejegg: ^^ [18:39:18] hmmmm [18:39:21] wait, why would an API call from other Civi code have to make http requests? [18:39:41] cstone yeah I thought a requirement was undrupal [18:40:09] ejegg: I'm sure we confirm this. when we call the cvapi drush command, it gets mapped to a http api request [18:40:23] somewhat sure! ha :) [18:40:53] ejegg right... though if it's a direct php call to do stuff in civi, is it worth the extra layer of an internal api? [18:41:23] I feel like with all the ORM stuff already in Civi, that might be over-engineering? [18:43:19] I'm just checking again to confirm it does make a http request [18:44:06] jgleeson: the API code can be called via http, via drush, or directly from other code [18:44:15] when you call it via drush, there is no http call made [18:45:00] hmm [18:45:10] i remember debugging this to confirm [18:45:11] AndyRussG: My initial thought was just to create a 'job' [18:45:30] I could very well be wrong [18:45:36] and that the job would be runnable via the cvapi drush command [18:45:57] and internally would make other API calls to existing Civi entities [18:46:12] well, at least one API call to update custom fields on the contact [18:46:28] so we wouldn't create any new API calls beyond that job [18:47:20] yah, so the job would look like the SmashPigProcessRecurring job [18:47:23] ejegg okok that sounds reasonable for sure [18:47:38] but its execute method would instatiate a new QueueConsumer subclass [18:47:56] ejegg: right [18:48:00] and that QueueConsumer subclass would make the contact update API call within its processMessage fn [18:48:20] K right [18:48:24] and then process-control can execute it from the command line using drush cvapi [18:48:38] under the job cvapi like job.process_smashpig_recurring, no? [18:48:43] bingo [18:48:50] okok cool beans :) [18:49:23] yah, i guess technically cvapi is a drush command that lets you make CiviCRM api calls via the drupal-shipped drush utility [18:49:36] and then each civicrm api call is structured as entity.action [18:49:45] e.g. contact.create [18:49:53] but with 'job' it's a bit more abstract [18:51:01] it's OK not to be doing the process_smashpig_recurring action on any particular instance of the 'job' entity though [18:51:25] what actually matters to Civi's API harness is that the function name is laid out as it expects [18:51:39] function civicrm_api3_job_process_smashpig_recurring($params) [18:51:45] well, that's for apiv3 anyway [18:52:12] i was assuming we'd do this one in apiv3 as well [18:52:14] hmmm I can't quite see [18:52:27] clicking through the code how it's resolving the call [18:52:39] lots of abstraction handling it in tidy fashion [18:52:44] phpstorm gets lost [18:52:53] I might need to run xdebug and call one to confirm [18:54:21] jgleeson: you could look at Civi/API/Kernel.php for starters [18:55:09] it creates a Civi\API\Request [18:55:16] which can be v3 or v4 [18:55:27] and that request has the entity + action + params [18:55:27] yeah I'm there [18:55:35] then it runs it [18:55:37] I can't get beyond runRequest [19:03:36] jgleeson: AndyRussG meeting? [19:03:50] coming! [19:03:55] same! [19:04:32] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, MW-1.36-notes (1.36.0-wmf.38; 2021-04-06): Remove Davidienda - https://phabricator.wikimedia.org/T277770 (DStrine) Open→Resolved [19:04:43] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog: es-419 updates to payments wiki smallprint/links - https://phabricator.wikimedia.org/T276367 (DStrine) Open→Resolved [19:04:45] Fundraising-Backlog, Wikimedia-Fundraising, FR-LATAM, MW-1.36-notes (1.36.0-wmf.33; 2021-03-02): Fundraising in Latin American Spanish - https://phabricator.wikimedia.org/T199680 (DStrine) [19:04:47] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM - update to 5.37 - https://phabricator.wikimedia.org/T278870 (DStrine) Open→Resolved [19:05:57] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Issue with New Field in Individual Engage Import File - https://phabricator.wikimedia.org/T275442 (DStrine) Open→Resolved [19:06:58] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Creating New Fidelity File Civi Import - https://phabricator.wikimedia.org/T275445 (DStrine) Open→Resolved [19:07:20] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, FR-AutoTY-Email: Coding HTML emails - https://phabricator.wikimedia.org/T277779 (DStrine) Open→Resolved [19:07:48] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: New weird dedupe error - https://phabricator.wikimedia.org/T277695 (DStrine) Open→Resolved [19:09:28] Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, FR-Docker, Patch-For-Review: fundraising-dev CiviCRM set up failures (with fixes and workarounds) - https://phabricator.wikimedia.org/T279669 (DStrine) [19:10:17] fundraising-tech-ops, Patch-For-Review: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (Dwisehaupt) Grant scripts updated to use 'create user or replace' (available since 10.1.3) which will allow us to run the scripts and just update the user account... [19:31:25] Fundraising-Backlog: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) @MDemosWMF is this where they are looking {F34379253} [19:32:05] Fundraising-Backlog: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) Hmm - you can select contact id ? {F34379268} [19:32:08] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (DStrine) [19:32:47] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (DStrine) p:High→Medium [19:32:54] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) a:Jgreen [19:39:04] Fundraising-Backlog: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) @MDemosWMF could you describe exactly how you are sending this letter? [19:41:28] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) @Eileenmcnaughton That's strange, for some reason I'm not seeing Contact ID in the dropdown. Am I missing it? {F34379365} [19:49:02] Fundraising-Backlog: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) @DStrine I tested it through my Civi profile CID 47095239 by creating a stock gift with stock value and stock description fields filled out as well as financial type and payme... [19:52:38] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) [19:54:20] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) @MDemosWMF that looks different in formatting too - what is the url you are seeing that at? (I went to https://civicrm.wiki... [19:55:36] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) the process you describe will send 2 emails. As far as we know, stock gifts are usually imported and the import will only send one TY email. A... [19:55:59] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) ie screenshot with a bit more context {F34379601} [19:56:43] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Figure out how to replace watchdog - https://phabricator.wikimedia.org/T279983 (DStrine) [19:56:45] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: develop more detail plan on queue refactor for drupal upgrade - https://phabricator.wikimedia.org/T279962 (DStrine) [19:56:48] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, FR-Docker, Patch-For-Review: fundraising-dev CiviCRM set up failures (with fixes and workarounds) - https://phabricator.wikimedia.org/T279669 (DStrine) [19:56:52] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: New Org Contact Fields in Engage Import not Importing to Record - https://phabricator.wikimedia.org/T278892 (DStrine) [19:56:54] Fundraising Sprint H 2021, Fundraising-Backlog: Edits to MC Upsell TY Emails in production [by ED transition] - https://phabricator.wikimedia.org/T278460 (DStrine) [19:56:59] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Remove unused code from wmf_civicrm - https://phabricator.wikimedia.org/T270678 (DStrine) [19:57:02] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Convert wmf_campaigns to an extension - https://phabricator.wikimedia.org/T270676 (DStrine) [19:57:04] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, and 2 others: Production of New Annual Fund Thank You Email (due to ED transition) - https://phabricator.wikimedia.org/T278363 (DStrine) [19:57:12] Fundraising Sprint Corrugated super slide, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 5 others: Create civiproxy on docker - https://phabricator.wikimedia.org/T268683 (DStrine) [19:57:15] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (DStrine) [19:57:18] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, FR-WMF-Audit: Paypal audit downloads failing - https://phabricator.wikimedia.org/T278878 (DStrine) [19:57:20] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, MW-1.37-notes (1.37.0-wmf.1; 2021-04-13): Non-english soft descriptor - https://phabricator.wikimedia.org/T277598 (DStrine) [19:57:22] Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, and 3 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civicrm - https://phabricator.wikimedia.org/T277500 (DStrine) [19:57:25] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, FR-Adyen: Break down tasks for Adyen reintegration, drop in web - https://phabricator.wikimedia.org/T277121 (DStrine) [19:57:27] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 3 others: Most recent email address becomes primary email address, former email address is moved to “Other” - https://phabricator.wikimedia.org/T276391 (DStrine) [19:57:29] Fundraising Sprint H 2021, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (DStrine) [19:57:33] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Move all remaining dedupe code out of wmf_civicrm to deduper extension - https://phabricator.wikimedia.org/T270679 (DStrine) [19:57:36] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Convert large donation module to an extension - https://phabricator.wikimedia.org/T270667 (DStrine) [19:57:41] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, FR-Smashpig, fr-email-preference-center: Add new queue settings for email-preferences - https://phabricator.wikimedia.org/T268512 (DStrine) [19:57:44] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising Sprint Princess Mongodb, Fundraising-Backlog, and 3 others: Consume new messages from Email Preferences form - https://phabricator.wikimedia.org/T268511 (DStrine) [19:57:49] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 6 others: Create new subpage for Special:EmailPreferences - https://phabricator.wikimedia.org/T268510 (DStrine) [19:57:59] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 6 others: Use Guzzle to make API request to CiviProxy to retrieve opt-in / opt-out fields. - https://phabricator.wikimedia.org/T268497 (DS... [19:58:07] Fundraising Sprint Bee Wheel, Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, and 8 others: Add CiviProxy to crm repo, write configuration or code for filtering API calls - https://phabricator.wikimedia.org/T268495 (DStr... [19:58:14] Fundraising Sprint Git Rebase Jump, Fundraising Sprint H 2021, Fundraising-Backlog, FR-Civi-Dedupe, fr-donorservices: Civi: enable agents to view and edit the dedupe 'nickname' rules list - https://phabricator.wikimedia.org/T244404 (DStrine) [19:58:16] Fundraising Sprint Airline Passenger Experience, Fundraising Sprint Bee Wheel, Fundraising Sprint Corrugated super slide, Fundraising Sprint Downed power line jump rope, and 16 others: Fr-tech chores list - https://phabricator.wikimedia.org/T258527 (DStrine) [19:59:25] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) @Eileenmcnaughton I did the same - went to: https://civicrm.wikimedia.org/civicrm/contact/view?rese... [20:00:41] Fundraising Sprint H 2021, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) Here's what my screen looks like: {F34379681} [20:05:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) Hmm, as far as I know there is no import for stock gifts. We get notifications from the bank through email for each individual gift and then... [20:08:01] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) Also if someone notifies us they did not receive their stock TY letter after the fact and needs a copy, I will go to the record to resend us... [20:14:06] AndyRussG: just read the reply at https://phabricator.wikimedia.org/T279669#6989928 [20:14:36] I'm guessing the local user means the code doesn't hit the '2021-04-09T03:41:14+00:00 c70a75671469 sendmail[1312]: 1393fEx0001312: SYSERR(UID1000): Who are you?' error line [20:15:56] jgleeson: could be, though further on in the log, it just says that's a warning [20:16:04] and continues to try to send the e-mail locally [20:16:09] and then gets connection refused [20:16:32] (that was before setting sendmail_path to /bin/true) [20:17:00] looking at other questions about sendmail on docker with similar connection refused issues also made me think that was it [20:17:07] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) a:Eileenmcnaughton [20:17:30] i.e., there's no daemon listening for connections, which is expected under Docker, since no init process that starts daeomons [20:17:52] I think init.d ships with debian [20:17:59] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog: Edits to MC Upsell TY Emails in production [by ED transition] - https://phabricator.wikimedia.org/T278460 (Cstone) a:Cstone [20:18:02] jgleeson: right but it doesn't run under Docker [20:18:12] although that's just sendmail being called directly right? [20:18:22] no deamon calls [20:18:29] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: develop more detail plan on queue refactor for drupal upgrade - https://phabricator.wikimedia.org/T279962 (Ejegg) The base consumer code is in Smashpig, so that shouldn't... [20:18:32] php is calling sendmail I think [20:20:24] jgleeson: hmm ok that certainly could be [20:20:42] I guess I'm not clear on how sending mail locally works [20:22:12] jgleeson: maybe it's a different service that should be running? so sendmail is for sending the e-mail, so that would initiate a connection to a smtp server on the destination host, but for that to work with a local send, there has to be something listening on that hose? [20:22:15] *host [20:25:58] so i just replicated the issue locally [20:26:14] with the new image and the new patch? [20:26:19] if I run `echo "Subject: docker user sendmail test" | sendmail -v docker@email.com` [20:26:36] on a standard debian docker image [20:26:39] jgleeson: on my host machine I have exim4 listening on port 25 [20:26:53] but then if I connect in to the same container as uid 9999 [20:27:03] and try the same thing I get [20:27:12] I have no name!@6b151f417d7b:/$ echo "Subject: 9999 sendmail test" | sendmail -v 9999@email.com [20:27:13] LOG: MAIN PANIC DIE [20:27:15] Failed to get user name for uid 9999 [20:27:17] I have no name!@6b151f417d7b:/$ [20:27:34] so it looks like sendmail can't work with anonymous users [20:29:49] k looking [20:32:59] I think it's just sendmail's internals trying to lookup the sender [20:35:15] jgleeson: what if you try adding: -f user@example.com [20:35:52] I have no name!@6b151f417d7b:/$ echo "Subject: 9999 sendmail test" | sendmail -f test@test.com -v 9999@email.com [20:35:53] that would specify a from address on the cli. may not make a difference. [20:35:54] LOG: MAIN PANIC DIE [20:35:56] Failed to get user name for uid 9999 [20:35:58] I have no name!@6b151f417d7b:/$ [20:36:02] I tried that but didn't work [20:36:09] bummer. [20:36:10] I was looking at ways round it [20:36:30] so dramatic with it's all caps panic die. [20:36:32] jgleeson dwisehaupt it works fine when I install and start exim4, which is what listens on that port.... one sec, sending a paste [20:36:47] I guess it's not a big issue for now but it explains why the local user fixed it AndyRussG [20:37:19] jgleeson dunno I'm getting quite a different result [20:37:23] one sec [20:37:33] jgleeson it's not urgent but we do eventually need to run sendmail [20:37:42] also thanks so much for digging in on this btw!!!! [20:40:58] exim4 is kinda cryptic [20:43:12] jgleeson: weird, I do get the LOG: MAIN PANIC DIE [20:43:14] Failed to get user name for uid 1000 [20:43:21] after I start exim4 inside the container [20:43:50] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) ok we'll look into this. [20:44:39] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (DStrine) [20:45:41] I didn't need to start exim4 [20:45:57] maybe it can be called directly and also launched in deamon mode ? [20:46:35] maybe [20:47:06] there are also other mta's that we could make docker use instead, which might not care about the uid not existing in /etc/passwd? [20:47:27] we do eventually need one I think [20:48:07] The -bd argument tells sendmail to run as a daemon [20:48:13] /usr/sbin/sendmail -bd -q10m [20:49:08] hmmm [20:49:39] I don't think it's much work to add a local user during the build steps to save having to try another MTA [20:50:00] jgleeson: it has disadvantages which is why I've kinda tried to avoid it a lot [20:50:06] it's doable though [20:50:19] we want the files created on the local filesystem to have the same uid as the host user [20:50:21] ah rly. what's the downsides [20:50:26] ^ [20:50:36] and the uid varies from one environment to another [20:50:41] but they would right? [20:50:49] because not everyone's local user is 1000 [20:50:51] we'd set the uid as the one to the local user [20:50:53] fwiw, we are using postfix as our default MTA [20:51:12] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) Thank you! [20:51:18] AndyRussG: you can easily set the uid to the correct uid [20:51:26] I wasn't suggesting we hard code it at 1000 [20:51:33] jgleeson right so you'd have to build the image differently for everyone. You can do that with arguments, but I'd rather we just be able to pull the image directly from the registry [20:51:38] it's an env var vy that point right? [20:51:40] by* [20:51:48] no no [20:51:52] not in the image [20:52:01] lemme check the post, maybe I wasn't clear [20:52:11] jgleeson you can't create a new uid without root [20:52:20] and we only have root when we're building the image [20:52:28] not when running locally inside docker-compose [20:52:28] Create a real local user for associated uid that docker exec uses to issue commands to the containers during build steps. We could add this during setup.sh and add a separate utility script for one-off uses in the event of containers being destroyed. [20:52:54] we have root in setup.sh right? [20:53:03] we can set any user at docker exec level [20:53:19] yeah we can run commands in the container as root, that's true [20:53:31] using -u 0 in docker-compose exec [20:53:32] that was under 'long-term fixes' on https://phabricator.wikimedia.org/T279669 [20:53:46] you can just do docker exec -u root 'container' 'cmd' [20:54:00] yeah or uid 0 [20:54:22] I get your point in the images, unless you build the images that's tricker [20:54:25] jgleeson: ah ok yeah I mistook that as something you were thinking should be done at the level of the image [20:54:35] but the other way should work with the setup.sh approach [20:54:51] K yeah I see, yeah hadn't thought of that [20:55:34] jgleeson: K sorry I now see you did even explain that in the task description [20:55:42] jgleeson: so apologies that I missed that, my bad [20:56:14] hmmm I guess also as you mention, it makes the containers not ephemeral [20:56:16] no worries at all. I know we've also talked about the other way too so easy to see how they were mixed [20:56:43] yeah that's the kicker, we lose it if they are destroyed [20:57:33] although i guess in the fundraising-dev readme, you encourage using stop/start over down/up which should mean folks won't be destroying them as much maybe? [20:58:04] hmmm I mean it's doable... I do tend to destroy them pretty often though [20:58:10] yeah me too [20:58:15] cleaner slate! [20:58:17] :) [20:58:22] yeah... I think I didn't find a way to completely reset the volumes without destroying the containers, too [20:59:28] I think you can get at the volumes with just `docker volume` [20:59:47] or did you mean something fancy to reset them all [20:59:48] hmmm [21:00:07] remove the data, but preserve the service [21:00:12] I meant* [21:00:39] no just docker-compose down -v (see line 148 of setup.sh) [21:00:50] I just tried using postfix instead of sendmail and still got the user id 1000 [21:01:01] sendmail: fatal: no login name found for user ID 1000 [21:01:10] (it's the same command name but a different program) [21:01:55] yeah I use that also to reinstall/reset. this is making me wonder if there is docker-compose volume cmd [21:02:02] specific to the stack [21:02:26] nope [21:02:34] right you want docker-compose to be the one that decides which volumes are associated with the docker-compose application [21:03:48] yeah right. I'm surprised we can't do that because if you run `docker volume ls` you can see the volumes are named in the docker-composey style [21:06:34] jgleeson: got it working with anonymous user [21:06:48] nice [21:07:09] echo "My message" | sendmail -f user@gmail.com foo@foo.com [21:07:12] -f is for from [21:07:18] and so that way it doesn't try to look up the username [21:07:28] gonna try it now with clean contianers [21:07:36] I swear we tried that earier? [21:07:38] (since I've been apt installing stuff in the containers) [21:07:40] lemme paste it [21:08:09] (that was with the postfix verison of sendmail) [21:08:30] ah ok [21:08:39] I just tried it with exim4 and it didn't like it [21:08:56] well, sendmail that by default is pointing to exim4 [21:09:52] I have no name!@6b151f417d7b:/$ ls -la /usr/sbin/sendmail [21:09:54] lrwxrwxrwx 1 root root 5 Mar 18 08:10 /usr/sbin/sendmail -> exim4 [21:14:04] Fundraising-Backlog, fundraising-tech-ops: Enable SSL for CiviCRM DB connections - https://phabricator.wikimedia.org/T280080 (Dwisehaupt) [21:14:47] jgleeson: ok here's how I have it working [21:15:06] 1) open a root shell in civicrm container [21:15:15] apt update [21:15:24] apt install postfix [21:15:40] at the configuration prompt, choose 5. Local only [21:15:48] set some whatever values [21:16:11] then when it's done: service postfix start [21:16:40] then without closing that shell, in another terminal open another shell on the civicrm container as the default (non-existent) user [21:16:57] then run: echo "My message" | sendmail -f user@gmail.com foo@foo.com [21:17:06] at least it seems to run fine and doesn't complain [21:18:38] jgleeson so I think this is how we can eventually get it working for testing mail sends, eventually: 1) install postfix in the image instead of sendmail, do something in the image so it's all configured [21:19:00] 2) set up a mailcatcher container that sendmail will connect to [21:19:30] 3) make php use -f option on sendmail, or something equivalent so it doesn't try to look up the user locally [21:20:48] Fundraising-Backlog, FR-Civi-Dedupe, fr-donorservices: Report request - https://phabricator.wikimedia.org/T277305 (SHust) [21:21:45] dwisehaupt also thanks for the note about prod! :) [21:22:21] yep that didn't throw an error [21:22:50] cool beans!!! :) :) [21:22:50] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (Eileenmcnaughton) I think this must be my browser caching the old version as I CAN see... [21:26:30] jgleeson: just for completeness, here is another option on how we could eventually add real users to the containers, while still making them ephemeral [21:26:51] we could add an additional build step specified in the docker-compose.yml file [21:27:05] so the base images, with almost everything set up, would still come from the registry [21:27:21] and so they'd still be identical across everyone's setup [21:28:24] and there would be also local Dockerfiles which would add the local user to the image as a follow-on build step, with arguments sent into the build to make the user have the same UID and GID as the user running stuff on the host machine [21:28:53] just a thought [21:33:21] sry was trying to debug civi [21:33:30] to work out this api confusion I had [21:33:58] I think I know what you mean [21:36:39] that make sense to me! might be a bit of work though! but makes sense [21:37:39] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678967 [21:37:41] (PS1) Eileen: Fix entity-ref search by id [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/678968 (https://phabricator.wikimedia.org/T280065) [21:37:49] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678967 (owner: Ejegg) [21:40:19] I was just checking the docker-compose reference to see if there was anything like that [21:40:33] something like an 'on-up' hook [21:40:54] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678971 [21:41:07] I'm fading now. good talking though AndyRussG. bye for now fr-tech [21:41:23] jgleeson|away: cya thanks!!! :D [21:44:40] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/678967 (owner: Ejegg) [21:45:12] I got a fix from Coleman for that bug that Melanie logged that we talked about in planning (678968 ) - pretty straight forward [21:45:34] (CR) Ejegg: [C: +2] Update DonationInterface submodule [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678971 (owner: Ejegg) [21:53:46] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_35) - https://gerrit.wikimedia.org/r/678971 (owner: Ejegg) [22:07:07] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) [22:07:59] !log updated payments-wiki from 70f5163816 to 9a4eef1375 [22:08:06] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:09:36] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) [22:17:18] (PS5) Ejegg: Update 2021 thank you emails. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676681 (owner: Cstone) [22:17:20] (CR) Ejegg: [C: +2] "Looks good, to the best of my language capabilities!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676681 (owner: Cstone) [22:34:21] (Merged) jenkins-bot: Update 2021 thank you emails. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/676681 (owner: Cstone) [22:36:36] (PS1) Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678980 [22:37:22] (CR) Cstone: [C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678980 (owner: Cstone) [22:38:18] (CR) Cstone: [V: +2 C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/678980 (owner: Cstone) [22:40:24] !log civicrm revision changed from 76bd8ff009 to a4c1a7b842 [22:40:31] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:43:41] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, and 2 others: Production of New Annual Fund Thank You Email (due to ED transition) - https://phabricator.wikimedia.org/T278363 (Cstone) @CDenes_W... [23:16:58] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Contact ID field for soft credit on contribution record is missing - https://phabricator.wikimedia.org/T280065 (MDemosWMF) Thanks @Eileenmcnaughton and Coleman! Let me know whe... [23:51:14] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM theme update - https://phabricator.wikimedia.org/T278888 (Eileenmcnaughton) [23:51:51] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: CiviCRM theme update - https://phabricator.wikimedia.org/T278888 (Eileenmcnaughton) @DStrine I pulled this in because I did this update to try to address a problem I was hitting (it didn't solve it... [23:52:21] (PS1) Eileen: Update shoreditch theme [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679019 (https://phabricator.wikimedia.org/T278888)