[00:40:32] (PS1) Ejegg: Delete unused fn, log to see if we can delete another [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679531 [00:50:30] !log updated fundraising CiviCRM from c3342aa4ea to 35a8dd33ba [00:50:38] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:50:42] fr-tech ignore that one-off failmail plz [00:51:37] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679496 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [00:51:40] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679497 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [00:53:30] (CR) jerkins-bot: [V: -1] Delete unused fn, log to see if we can delete another [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679531 (owner: Ejegg) [01:00:39] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679498 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:00:54] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679499 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:07:33] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679498 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:07:35] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679499 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:07:42] (Merged) jenkins-bot: Remove tokenize-damaged [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679510 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:07:44] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679511 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:16:09] (PS2) Ejegg: Delete unused fn, log to see if we can delete another [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679531 [01:17:38] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679513 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:20:03] (CR) Eileen: [C: +2] Delete more wmf_civicrm update cruft [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679521 (owner: Ejegg) [01:37:30] (Merged) jenkins-bot: Remove more unused update functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679513 (https://phabricator.wikimedia.org/T270678) (owner: Eileen) [01:38:25] (Merged) jenkins-bot: Delete more wmf_civicrm update cruft [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679521 (owner: Ejegg) [01:47:34] (CR) Eileen: [C: +2] "this seems worth a go - & then when we get serious about looking at those triggers & moving them we will have more information" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679531 (owner: Ejegg) [01:52:31] (CR) Eileen: [C: +2] "Yep - agree this is cleaner/clearer - thanks ejegg" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679128 (owner: Ejegg) [02:06:03] (Merged) jenkins-bot: Deduper: consistently index ($location)Conflicts [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679128 (owner: Ejegg) [03:04:38] hmm, silverpop upload fail? [03:35:15] Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, and 4 others: Adyen: have audit file pick up manually settled donations - https://phabricator.wikimedia.org/T265545 (Ejegg) OK, looks l... [03:38:49] Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, and 4 others: Adyen: have audit file pick up manually settled donations - https://phabricator.wikimedia.org/T265545 (Ejegg) For the wor... [03:41:41] Fundraising Sprint Downed power line jump rope, Fundraising Sprint Esperantoland, Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, and 4 others: Adyen: have audit file pick up manually settled donations - https://phabricator.wikimedia.org/T265545 (Ejegg) for 8132890... [03:45:18] (PS2) Ejegg: Remove unused code to remove old cookies [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/677636 (owner: AndyRussG) [03:45:50] (CR) Ejegg: [C: +2] "Always good to clean up the clean-up code." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/677636 (owner: AndyRussG) [04:05:32] (Merged) jenkins-bot: Remove unused code to remove old cookies [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/677636 (owner: AndyRussG) [05:22:29] Fundraising Sprint Git Rebase Jump, Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-drupal-upgrade-2021: Figure out how to replace watchdog - https://phabricator.wikimedia.org/T279983 (Eileenmcnaughton) I've been digging into this a bi... [09:41:46] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (RLewis) @DStrine and @MDemosWMF just a heads up that I tested this out for Donor Advised Funds.... [13:40:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1201 [13:45:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1501 [13:51:04] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1801 [13:55:11] fr-tech I don't know when the export was kicked off again but Acoustic isn't finding the files on FTP. Since our sends are supposed to go out in less than 10 mins, I just unscheduled them. [13:55:40] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2101 [14:00:38] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2401 [14:00:48] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Patch-For-Review, SEO: CentralNotice code to fix the banner bump with “ESI and pageview+1 with possible exceptions for infrequent visitors or as needed” - https://phabricator.wikimedia.org/T279034 (awight) ESI would be a game-changer for Ce... [14:05:40] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2701 [14:10:40] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3001 [14:15:40] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3301 [14:20:45] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3601 [14:25:41] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3901 [14:30:39] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 1715647 Threads: 18 Questions: 42343476 Slow queries: 1406970 Opens: 301938103 Flush tables: 1 Open tables: 200 Queries per second avg: 24.680 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [14:33:39] hey katers [14:34:28] sorry just read your message [14:34:46] I was out picking my son up from school [14:34:50] lemme check the export progress [14:36:21] oh wow, it's still running! [14:37:05] thanks Jack! no worries. i appreciate you looking into it [14:42:59] np katers! I haven't run this process manually for a while but my memory is that it used to complete in around 30 minutes. I'm not sure why this one is running on longer. It's no errored and I can see it progressing but it does feel like a lot longer than usual. I guess I could check the recent logs to see how long other runs have been taking this week [14:44:40] Sounds good to me. And yeah I agree I remember it taking around 20-30 mins [14:45:46] Could there be other logs that are running that could slow it down? I think that was an issue once where there was a query running for days and it delayed the export process [14:46:05] katers: it looks like it's just finished [14:46:45] Oh, cool! I'll see if Acoustic can find it [14:46:54] sounds good [14:47:43] that run took around 1h30m. [14:53:38] Acoustic found it! yay [14:53:52] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Stock TY email in Civi - double send/different templates - https://phabricator.wikimedia.org/T280047 (MDemosWMF) Thanks @RLewis! Just tested that out - entering a new stock gift and not checking the... [14:54:17] I'll give it a quick lookover after it processes [14:55:08] awesome [14:57:07] All good now :) thank you so much Jack! [14:57:41] I'll let rest of email team know we can reschedule [14:58:18] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (DStrine) @KHaggard this is in the current sprint and we'll get you a full dump of th... [14:59:27] great to hear! thanks katers [15:00:05] Fundraising Sprint Humongous bacteria petting zoo, Fundraising-Backlog, FR-Email: Civi export data for 2 fields are preventing accurate data validation reports in Acoustic - https://phabricator.wikimedia.org/T270731 (KHaggard) Cool thanks, @DStrine - heads up that I'm floating those April WMF holida... [15:31:40] Fundraising Sprint File Systems Stage Show, Fundraising Sprint Git Rebase Jump, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Creating New Fidelity File Civi Import - https://phabricator.wikimedia.org/T275445 (MDemosWMF) Hi @Eileenmcnaughton just wanted to hop back in here and see if the im... [16:23:22] AndyRussG: ok. i found it. the one i was focusing on was https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/fundraising/tools/DjangoBannerStats/+/refs/heads/master/fundraiser/analytics/management/commands/LoadBannerImpressions.py [16:23:54] if you look at the top, you can see the sql statements. so it will collect the stats/impressions and load them into the db using django as the db connection provider. [16:24:33] i'm looking to add an options stanza on the django connection config and would just like to test it before doing it live. [16:25:06] dwisehaupt: ahh okok gotcha [16:25:12] so if there is a way to test that, it would be great. but i know there are a lot of moving parts in that setup so it may not be easy. [16:25:32] and this totally isn't something that needs to be done today. it's just part of a greater ssl-ification movement. [16:25:43] welp, I think you can download the source code locally and point it to a local mysql [16:26:03] and I guess you'd have to have your local mysql running with the ssl bit too [16:26:40] I guess you've already checked this, and I guess this'd be the same for all endpoints, but there are definitely cli command endpoints that are unused [16:27:30] IIRC it has moderately functional tracking of which files have been consumed and which haven't [16:27:57] cool. i'll try it on my vm setup to see if i can get something to test with. i'll just have to get some sanitized impression data to have it import which shouldn't be too difficult. [16:27:59] so if you try it live, and it breaks, it should still keep track of which data has been consumed [16:28:20] (though in that case I guess some backfill might be needed) [16:28:30] yeah. in this case, breaking would mean that the db connection will fail and data won't get populated into it. [16:28:36] yeah [16:28:58] yeah it's generally kinda awful, pls don't hesitate to lmk if there are details I could help with further along, btw! [16:29:22] ok. will do. i'll start by pulling it into the vm and seeing how far i can get. [19:12:21] Fundraising-Backlog, fundraising-tech-ops: timeboxed description of fr-tech vs WMF security settings and processes - https://phabricator.wikimedia.org/T280173 (Jgreen) Open→Resolved a:Jgreen [20:15:56] ejegg: I see you updated monolog in donation interface (yay) - but we need to tag a new donation_interface release now don't we? [20:16:29] durn, I guess we do [20:16:35] man, I want to sever that link [20:16:56] I think it's just the orphan rectifies that use donation interface under civi now [20:17:28] k, will put that tag up [20:20:30] 2.5.3 live on packagist, I guess we should update that in Civi [20:21:02] eileen: thinking about the watchdog logging replacement? [20:21:10] ejegg: yep [20:21:33] so i've been thinking we should replace the SmashPig weirdo logger with something PSR compliant [20:21:38] and that might be monolog [20:21:53] then we could just use those logging calls in Civi [20:21:59] Yep [20:22:07] but... I guess that would tie all of our extensions into SmashPig [20:22:10] so I was looking at https://lab.civicrm.org/extensions/monolog [20:22:16] oho [20:22:26] it's not a 'real' extension - it's a demo [20:22:34] but it is a good start [20:23:13] yep, just got a streamlogger for now [20:23:21] but would be easy enough to add a sysloghandler there [20:23:51] the thing is we don't want to log all civi errors to syslog - I was looking at using separate services & directing differently - along the lines of https://github.com/civicrm/civicrm-core/pull/20068 - but since we are mostly worried about our own logging we don't need that merged [20:24:57] https://www.irccloud.com/pastebin/FeJzRNT6/ [20:25:23] looks reasonable [20:25:50] and maps well to the watchdog calls that all had the category as the first arg [20:25:55] yeah - achieves the goal of writing code not tied to the CMS or the implementation [20:26:59] the in the extension we can create some config to allow us to specify what happens to various categories [20:27:16] hmm, it would also be good to be able to just 'get' a logger instance to e.g. supply to a future SmashPig that can log to any provided PSR-compliant logger [20:27:32] so that we don't have to look in different places for different logs [20:28:04] during the same process [20:28:22] e.g. the recurring charge jobs [20:28:38] right - within civi that ```Civi::log('our_name')``` should return that [20:28:54] would have a continuous-looking flow between the Civi logic and the SmashPig logic [20:28:57] yep yep [20:29:29] currently Civi::log() does supply a psr-compliant logger - it just doesn't log to where we want it to :-) [20:29:51] cool [20:31:28] hmm, even after composer cache-clear my local composer isn't updating donation-interface [20:31:37] with composer update monolog/monolog wikimedia/donation-interface coderkungfu/php-queue [20:31:56] yep - so this line is where the monolog extension is swapping it out [20:31:57] it just updates monolog from 1.25.1 to 1.26 and php-queue from 1.0.0 to 1.0.1 [20:31:58] https://lab.civicrm.org/extensions/monolog/-/blob/master/monolog.php#L182 [20:32:17] (but I don't think we want to swap out the main one - just our own ones) [20:32:26] right right [20:32:48] so that sounds like not too big a core change [20:33:03] I guess just deciding how to specify config for the alternate log streams [20:33:22] and what happens when you ask for a stream that's not configured [20:34:37] yeah - that 3rd q I want to ask Tim [20:34:57] we don't need a core change unless 3 requires it [20:35:31] & yeah on the config - we can offer up settings by 'channel' - which I think is the 'my_stream' in Civi::log('my_stream') [20:35:57] but not sure the config options other than 'syslog', 'file', 'civicrm_system_log' table [20:36:07] & normal ConfigAndLog [20:36:22] but for 'file' we need the extra param [20:36:42] (away for 10) [20:47:06] (PS1) Ejegg: Library update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679955 [21:23:50] ejegg: that looks good ^^ - do we think there is risk from upgrading google/apiclient? [21:24:03] nope [21:24:13] we can maybe even get rid of it [21:24:25] ok cool [21:24:35] I feel like any of the others would be hit by tests [21:24:36] it was what jgleeson used to push our exchange rates to a shared sheet [21:24:42] (CR) Eileen: [C: +2] Library update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679955 (owner: Ejegg) [21:36:31] eileen: for an example of maximal configurability you could look at the mediawiki monolog config [21:36:36] one sec, finding a link [21:39:10] https://phabricator.wikimedia.org/diffusion/WFDV/browse/master/config/email-pref-ctr/LocalSettings.php$204 [21:39:29] all the way to line 259 [21:39:39] ohhh [21:40:07] so those 'processors' mostly add context to each line [21:40:25] and then the 'handlers' decide how and where to write each line [21:41:34] then wgMWLoggerDefaultSpi is like the service wiring [21:42:04] (Merged) jenkins-bot: Library update [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/679955 (owner: Ejegg) [22:02:51] ejegg: am I right in understanding we attach syslog to all the default processors? [22:03:03] yep yep [22:45:36] Fundraising-Backlog, fundraising-tech-ops, fr-donorservices: Fundraising access request for Donor Services agent Lindsay - https://phabricator.wikimedia.org/T279287 (Dwisehaupt) Open→Resolved [23:03:25] Fundraising-Backlog: Adding PIX as a payment type for Brazil - https://phabricator.wikimedia.org/T280303 (EMartin) [23:21:47] Fundraising-Backlog: Adding PIX as a payment type for Brazil - https://phabricator.wikimedia.org/T280303 (EMartin) Developer spec: https://docs.dlocal.com/api-documentation/payins-api-reference/payment-methods/brazil#payment-methods-available [23:38:46] Fundraising-Backlog: Adding PIX as a payment type for Brazil - https://phabricator.wikimedia.org/T280303 (Ejegg) That developer spec is for a newer integration that we're not using yet. I don't see a two letter 'bank code' available for Pix in the older 'Streamline' integration docs: https://dlocal.com/devel...