[01:11:29] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (Cstone) @MDemosWMF on entries already in the system if you use the Send Thank You option from cl... [02:58:53] eileen: hi! ahh qucik question, what extra thing do I need to tell Civi to tell it that my new api functions exist? [02:59:16] currently it's all, "API (Preferencesqueue, consume) does not exist (join the API team and implement it!)" [02:59:26] not sure what I'm missing [02:59:29] I did drush cc all [02:59:40] thx in advance! [02:59:49] (PS1) Ejegg: SFTP version of download_nightly_redux [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/680107 [03:00:33] (PS2) Ejegg: SFTP version of download_nightly_redux [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/680107 (https://phabricator.wikimedia.org/T278878) [03:02:19] AndyRussG: it might be your capitalization [03:02:35] it kinda expects preferences_queue or PreferencesQueue [03:05:01] also - if you are calling via drush you might need to pass version=4 [03:05:11] or otherwise make sure it knows you want v4 api [03:05:24] if it is v4 api [03:05:34] hmm I suspect it isn't so ignore [03:17:06] eileen: hmm k let me try changing capitalization [03:17:22] eileen: but I think it's v3 api? [03:17:32] yep I think it probably is [03:17:43] what is the function name? [03:19:48] function _civicrm_api3_preferencesqueue_consume(array $params): array { [03:19:50] return []; [03:19:52] } [03:20:02] and also function _civicrm_api3_preferencesqueue_consume_spec(&$spec) { [03:22:03] in wmf_civicrm/api/v3/Preferencesqueue/Consume.php [03:22:16] (trying to follow the pattern of Getpreferences.php) [03:22:19] eileen: ^ [03:23:22] btw same situation with the setpreferences mock that you send up (which I plan on following, thanks so much! just changing location and name and adding queue stuff...) [03:23:53] as in the Civiproxy.getpreferences works via drush command, but Civiproxy.setpreferences didn't, also got a message saying that API didn't exist [03:24:29] hmm - is your wip somewhere - I can pull it down & try it [03:25:15] ejegg: this will give us what we need re the ability to channelise https://github.com/civicrm/civicrm-core/pull/20079 [03:28:21] (PS3) Ejegg: Eliminate a couple redundant if conditions [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/676475 [03:28:27] eileen: not yet, one sec! it's completely just a shell of nothingness at this point [03:30:09] (PS1) AndyRussG: WIP E-mail preferences center queue consumer [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/680119 [03:30:33] eileen: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/680119 [03:32:51] cool - I'll make a cuppa & then look - also - did you know you should always try to drink water after a cuppa to prevent plaque (apparently stopping the staining reduces the plaque's ability to adhere) [03:35:43] eileen: hmmm I didn't know that, though maybe it only applies if you add sugar? [03:36:19] also, plaque might cause alzheimer's (really!) [03:36:31] thx! [03:41:23] AndyRussG: no - you know how the tanins stain your teeth [03:41:42] well that stain is easier to adhere to than unstained teeth it seems [03:42:23] AndyRussG: I think I see it [03:42:24] _civicrm_api3_preferencesqueue_consume [03:42:34] that function should have no preceding underscore [03:42:43] (the spec function should I think ) [03:43:00] yep the spec function does but not the api function [03:44:48] nice eileen, that looks good and simple. What's the recommended place to register a site-specific service? Right in civicrm.settings.php? [03:45:27] ejegg: no in a hook - like https://lab.civicrm.org/extensions/monolog/-/blob/master/monolog.php#L182 [03:45:53] eileen: yis! it works! thanks so much :) :) [03:45:55] ah, got it [03:45:57] we'll probably start with that extension & come up with some way to add enough configurability to meet our needs [03:46:03] cool cool [03:46:13] - currently is_syslog or is_normal_civi_whatever [03:46:23] is probably enough :-) [03:46:31] heh, that extension is still on monolog 1.x [03:46:49] ejegg: yeah - I proposed an update & made me a maintainer [03:46:55] nice [03:46:58] AndyRussG: yay - was that error in my sample code too huh? [03:47:15] k, signing off for the night. have a good one folks! [03:48:14] ejegg|away: cya! [03:49:58] eileen: no it wasn't, so there it must have been something else? (also could well have been something I was doing wrong on my end, of course) [04:29:45] (PS1) Eileen: community/feature-request#12 - Allow named logging channels [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/680137 (https://phabricator.wikimedia.org/T279983) [12:14:07] howdy fr-tech [12:14:26] Jeff_Green: did you catch ejegg|away's note on the silverpop failure [12:14:32] due to the same issue apparently [12:28:59] hey jgleeson, yep, looking at it now [12:29:07] it's kind of baffling [12:31:04] Jeff_Green: I'm just sending an email. I rerun the export in the off chance it would work and it did! [12:31:16] not sure if someone fixed it after ejegg|away's email early hours? [12:31:26] I just fixed it this instant! [12:31:44] I don't know why, but puppet didn't adjust the login group on frdev1001 [12:31:50] ah ok [12:45:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1201 [12:50:36] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1501 [12:51:36] Fundraising-Backlog: Acoustic export failing due to permissions issues - https://phabricator.wikimedia.org/T280354 (jgleeson) [12:52:26] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog: Acoustic export failing due to permissions issues - https://phabricator.wikimedia.org/T280354 (jgleeson) [12:55:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1801 [13:00:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2101 [13:05:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2401 [13:10:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2701 [13:15:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3001 [13:20:34] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3301 [13:22:52] fr-tech, heading out to pick up Oscar from school. the silverpop export job is still running (started 1h ago). I've created a ticket here to track the issue https://phabricator.wikimedia.org/T280354. If someone could let fr-online know when it completes that would be great! You can monitor the progress in the main log file for the job I triggered here: [13:22:54] civi1001:/var/log/process-control/silverpop_daily/silverpop_daily-20210416-122231.log [13:25:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3601 [13:30:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3901 [13:35:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3290 [13:40:38] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 1799047 Threads: 18 Questions: 43728931 Slow queries: 1474715 Opens: 312922817 Flush tables: 1 Open tables: 292 Queries per second avg: 24.306 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [14:16:58] Fundraising-Backlog: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (krobinson) [14:46:20] hey fr-tech :) [15:01:58] away [15:05:44] actually, g'day. just missed the / on the previous since apparently i'm still waking up. [15:06:05] howdy dwisehaupt AndyRussG [15:06:53] looks like the export has complete [15:07:22] Jeff_Green: does that issue need more work? [15:08:03] jgleeson: the issue of fixing the gid snafu? [15:09:53] the permissions error tripping the silverpop export [15:09:59] is that the same thing? [15:10:12] it is, it should be fixed now [15:10:47] I'm still cleaning up the underlying issue which was that the puppet useradd class had some logic issues that resulted in it not doing what we expected when we sert gid in various places [15:11:20] gotcha [15:11:24] so when that got fixed it exposed some logic problems in how we were assigning gid :-( [15:11:39] so I'm trying to clean up both once and for all [15:12:03] but this AM's problem was because puppet didn't change gid back to jenkins on frdev1001 because of the broken logic [15:12:39] ah ok cool [15:13:03] should I report out that the issue is being looked at still (to the email folks) or that it's a WIP [15:13:47] I think you can say it is fixed b/c we've fixed the two hosts where jenkins is used this way [15:14:34] I'd say we can think of the puppet tangle as under the hood stuff that is related but a separate problem [15:15:02] sorry that was meant to say either fixed or WIP [15:15:10] ok cool thanks [15:15:34] looks like there's no sends today or Monday so we're good on that front [15:15:43] cool [15:24:57] jgleeson dwisehaupt Jeff_Green :) [15:26:58] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog: Acoustic export failing due to permissions issues - https://phabricator.wikimedia.org/T280354 (jgleeson) Jeff has fixed this but let's close it after the next run completes as expected. [15:27:28] fr-tech as far as I understand, the new preferences queue consumer should be independent of all things drupal, no? so, no use of watchdog for logging, no? [15:27:39] I guess the thing to use now is Civi::log() ? https://civicrm.org/blog/totten/the-static-is-dead-long-live-the-static [15:40:12] sounds right AndyRussG [15:40:29] jgleeson: cool beans thx! ahh have we used that anywhere in our code yet, do you know? [15:40:33] hmmm I should grep [15:41:20] ah found an example in SmashPigRecurringProcessor.php [15:42:27] jgleeson: also do you know what the non-drupal equivalent of something like "$cycle_time = variable_get('ct_batch_time', 0);" (in wmf_ct_qc.module) would be? [15:43:21] also I don't see where such a thing is set... [15:44:15] you mean the drupal vars? [15:44:24] the civi equivalent [15:44:41] yeah [15:44:47] hmm I'm not sure tbh [15:44:49] also no idea what that's for [15:45:11] I think we timebox batch runs [15:45:16] hmmm [15:45:19] ok [15:45:26] and where are those set currently? [15:46:17] the variable_get fn is a built in drupal function that gets vars set in `drupal`.`variable` [15:46:42] it's commonly used when we want something configurable via the cms frontend [15:46:50] lemme show you an example [15:49:11] AndyRussG: if you open up https://civicrm.wikimedia.org/admin/config/queue2civicrm/contribution_tracking_qc you'll see the drupal module settings form for the contribution tracking queue consumer which includes that setting you pasted [15:49:34] the value in that field is retrieved in the variable_get() fn call [15:50:07] lemme show you the source to that form [15:51:45] ahhhh right I remember this now [15:52:02] all ui-y [15:52:39] do we ever configure that in like a config file, too? or it's set manually? and so maybe we don't even have a non-drupal-y equivalent? [15:53:20] yep! [15:53:31] so that form got generated here https://github.com/wikimedia/wikimedia-fundraising-crm/blob/8a783367ef21fa8f6b486e75e386bb3625955dd9/drupal/sites/all/modules/queue2civicrm/contribution_tracking/wmf_ct_qc.module#L29 [15:53:54] oh yeah, this is very drupal-specific [15:54:19] and I'm not sure if civi has it's own equivalent [15:54:41] that's a very cmsee behaviour [15:55:56] if I remember right AndyRussG, when you enable a drupal module, anything added to the $forms array here https://github.com/wikimedia/wikimedia-fundraising-crm/blob/8a783367ef21fa8f6b486e75e386bb3625955dd9/drupal/sites/all/modules/queue2civicrm/contribution_tracking/wmf_ct_qc.module#L29 is created in the `drupal`.`variable` so it can then be managed as a 'drupal variable' [15:56:29] although I appreciate none of this is useful in a non-drupal world [15:57:38] jgleeson: yeah I think for now I'll just add stubs/default values for this stuff until eileen is around to provide input, does that sound right? [15:57:49] maybe the expectation is that this bit will still be drupal-y [15:58:06] AndyRussG: I think civicrm might use it's 'settings' table in a similar fashion [15:58:37] jgleeson: ah ok I seem to recall seeing something like that somewhere, I can dig then :) [15:59:00] jgleeson: yeah looks right https://docs.civicrm.org/dev/en/latest/framework/setting/ [15:59:10] "If you're a Drupal developer, this system is analogous to the Drupal variables table and tools." [16:00:01] that's the ticket! [16:23:47] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (Cstone) a:Cstone [16:23:48] Fundraising-Backlog: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (Pcoombe) The only way I could reproduce this is by visiting an [India form](http://donate.wikimedia.org/?uselang=en&country=IN) and clicking through to payments first, then shortly after going to an... [16:30:32] Fundraising-Backlog, fr-donorservices: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (MBeat33) [17:23:19] fundraising-tech-ops, Privacy Engineering, SRE, Traffic, and 2 others: Visits to Wikimedia properties should not be used for Google ad targeting (FLoC) - https://phabricator.wikimedia.org/T279804 (Dwisehaupt) [17:25:33] Fundraising-Backlog, fr-donorservices: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (EMartin) As Kristie and Kristine are likely off line at the moment, I tested the links for the forthcoming campaigns and I do not see the PAN language. That's an interesting tw... [17:31:53] fundraising-tech-ops, Privacy Engineering, SRE, Traffic, and 2 others: Visits to Wikimedia properties should not be used for Google ad targeting (FLoC) - https://phabricator.wikimedia.org/T279804 (Dwisehaupt) Header added to fundraising nginx templates and deployed. ` [frack::puppet] 58ed92cf Add... [17:48:26] fundraising-tech-ops: investigate adding additional security headers to Fundraising civicrm, superset, etc. sites - https://phabricator.wikimedia.org/T280387 (Jgreen) [17:50:04] fr-tech if anyone can review the new shell script for the audit download I can deploy that + p-c settings today [17:50:34] just to not wait till tuesday to get that backlog of donations in [17:51:12] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/680107 [17:51:20] oh, it needs the 'sshpass' utility installed [17:51:28] and a file with the password in it [18:00:45] fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (Jgreen) [18:05:40] fundraising-tech-ops, SRE: (Need By: TBD) rack/setup/install payments100[5-8] - https://phabricator.wikimedia.org/T266481 (Jclark-ctr) [18:54:05] AndyRussG: here's an example of defining CiviCRM settings: https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/drupal/sites/default/civicrm/extensions/org.wikimedia.smashpig/settings/Smashpig.setting.php [18:55:08] And then using them as defaults for params not specified on the job command line: https://phabricator.wikimedia.org/diffusion/WFCG/browse/master/drupal/sites/default/civicrm/extensions/org.wikimedia.smashpig/api/v3/Job/ProcessSmashpigRecurring.php$27 [19:00:53] looking ejegg|away [19:07:40] hmm [19:07:48] I mean the code looks good [19:07:58] do you have a sample ftp you've tried it with? [19:09:54] lemme check localsettings to see if there's a sandbox ftp [19:12:30] oh ok I can see this is just a tweak to the existing process [19:12:50] so the bash is already tested to a degree [19:17:25] yeah. and sshpass is installed on the servers at this point so there is no holdup on our end. [19:19:02] I'm just spinning up an AWS sftp server using AWS Transfer Family [19:35:26] ok that turned out to be a nightmare to set up [19:36:11] I couldn't figure out how to point the AWS SFTP endpoint to my S3 bucket via the IAM roles.. [19:36:24] so I googled free sftp servers and found one that works [19:39:20] ejegg|away: ah cool thanks! [19:47:54] ok it appears to be working fo rme [19:47:57] for me* [19:52:09] +2'd ejegg|away [19:53:39] (CR) Jgleeson: [C: +2] "This is working for me. I tested it locally with a public sftp server (test.rebex.net demo/password) and it successfully downloaded the re" [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/680107 (https://phabricator.wikimedia.org/T278878) (owner: Ejegg) [19:54:37] (Merged) jenkins-bot: SFTP version of download_nightly_redux [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/680107 (https://phabricator.wikimedia.org/T278878) (owner: Ejegg) [20:58:06] Fundraising-Backlog, fr-donorservices: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (KHancock99) The PAN messaging is found //after clicking the// **Bank Transfer** button on each donation page. @Pcoombe @EMartin [20:58:52] Fundraising-Backlog: Adding PIX as a payment type for Brazil - https://phabricator.wikimedia.org/T280303 (Ejegg) @EMartin I forgot to tag you on the above comment - want to ask if Pix is available via Streamline and if the x_bank parameter PQ is the way to indicate it? [21:00:39] Fundraising-Backlog: Adding PIX as a payment type for Brazil - https://phabricator.wikimedia.org/T280303 (EMartin) Thanks Elliott, I saw it yesterday and Dlocal is looking into it. I'll update the task once I have the response. Thank you! [21:04:30] Fundraising-Backlog, FR-AutoTY-Email, fr-donorservices, Epic, FR-Email: Creating exceptions for when to send the recurring expiry/fail email - https://phabricator.wikimedia.org/T276959 (CDenes_WMF) Thank you Michael, we can get started on this. A few inputs please: - What languages do we ne... [21:04:43] (PS1) Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/680434 [21:05:06] (CR) Ejegg: [C: +2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/680434 (owner: Ejegg) [21:06:27] (Merged) jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - https://gerrit.wikimedia.org/r/680434 (owner: Ejegg) [21:08:40] !log updated fundraising python tools from ef54260b0d to 3d950fffbd [21:08:46] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:12:57] Fundraising-Backlog, fr-donorservices: Indian PAN messaging on Latam donation forms - https://phabricator.wikimedia.org/T280364 (EMartin) @Kristine Hancock @Peter Coombe Hmmm, I still don't see it: [image: image.png] [image: image.png] {F34401789}... [22:02:33] Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising-Backlog, fr-donorservices, MW-1.36-notes (1.36.0-wmf.34; 2021-03-09): Wikimedia's soft descriptor - https://phabricator.wikimedia.org/T272995 (EMartin) Hi - I see this descriptor coming through to th... [22:03:24] fundraising-tech-ops, Privacy Engineering, SRE, Traffic, and 2 others: Visits to Wikimedia properties should not be used for Google ad targeting (FLoC) - https://phabricator.wikimedia.org/T279804 (dpifke) Open→Resolved a:dpifke [22:16:45] (CR) Jforrester: [C: +2] build: Updating eslint-config-wikimedia to 0.20.0 [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/680318 (owner: Libraryupgrader)