[00:04:43] If it gets bounced to damaged messages where do I see them re-emerge? [00:29:32] Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for khancock - https://phabricator.wikimedia.org/T241154 (Dwisehaupt) Open→Resolved p:Triage→Normal Cert installed and running according to khancock. CRL update pushed. [frack::puppet::private] 475dd88 Reissuing... [00:31:54] Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for cstone - https://phabricator.wikimedia.org/T241153 (Dwisehaupt) p:Triage→Normal [00:31:55] Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for cstone - https://phabricator.wikimedia.org/T239767 (Dwisehaupt) p:Triage→Normal [00:31:59] Fundraising-Backlog, fundraising-tech-ops: Issue new SSL Client Certificate for bbarnes - https://phabricator.wikimedia.org/T236335 (Dwisehaupt) p:Triage→Normal [00:32:12] Fundraising-Backlog, fundraising-tech-ops: Access and onboarding for new fundraising analyst (contractor) - Erin Yener - https://phabricator.wikimedia.org/T232982 (Dwisehaupt) p:Triage→Normal [00:43:50] eileen: sorry, was afk a while without flag [00:44:22] there's a job that runs every 1/2 hr or so which picks things up from the damaged table if they have a retry date in the past [00:44:28] and drops them back in the original queue [00:44:44] so you would see them in a future log for the same queue consumer [00:45:02] ah ok - I think that a couple went back in [00:45:21] I put up a patch for one more place to catch for one that didn't (& requeud that one [00:45:29] still the redis one on the floow [00:45:34] floor [00:46:45] ok - yep I can see one of them requeued ok [00:47:32] hm but it was a duplicate by then & seems to be mentioned in other queues [01:09:31] ejegg: summary is promising - adding this will help https://gerrit.wikimedia.org/r/559627 - still digging around though [01:16:51] (CR) Ejegg: [C: +2] Another constraint / deadlock catch [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/559627 (https://phabricator.wikimedia.org/T182148) (owner: Eileen) [01:16:54] thanks! [01:23:22] (Merged) jenkins-bot: Another constraint / deadlock catch [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/559627 (https://phabricator.wikimedia.org/T182148) (owner: Eileen) [01:24:32] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/559640 [01:24:48] ejegg so how do we get back the record that fell off on the Redis delay? [01:25:20] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/559640 (owner: Eileen) [01:26:43] !log civicrm revision changed from 6ecdccd240 to f6f4aa1d86, config revision is 2e9bf6308b [01:26:48] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:47:37] eileen: we wait for the audit file :P [01:48:00] hmm, that's another argument for changing all the logged arrays over to json blobs [01:48:17] there's a command line script to take a json file and feed it into a queue [01:48:26] so that would make it easy to rescue a dropped thing [01:48:39] gotta sign off [04:08:18] (CR) Cstone: [C: +2] "There's some slight UI wiggling for me when enabling/disabling it but works nicely." [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/559610 (https://phabricator.wikimedia.org/T241000) (owner: Ejegg) [04:09:01] (Merged) jenkins-bot: Add toggle to include endowment in Big English charts [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/559610 (https://phabricator.wikimedia.org/T241000) (owner: Ejegg) [05:08:19] Fundraising Sprint YAMLton, the Musical, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review, Unplanned-Sprint-Work: Ensure we are coping with deadlock Exceptions properly & our q doesn't corrupt at high volume, reduce unecessary potent... - https://phabricator.wikimedia.org/T182148 [14:48:26] Fundraising Sprint A Wrinkle in Timezones, Fundraising Sprint Visual Basic Instinct, Fundraising Sprint X-rays, Fundraising Sprint YAMLton, the Musical, and 4 others: Rebase and finish Geotargeting (sub national targeting) - https://phabricator.wikimedia.org/T226438 (Ejegg) Add GB to region lists... [16:44:27] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/559907 [16:44:50] (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/559907 (owner: Ejegg) [16:45:27] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/dash] (deployment) - https://gerrit.wikimedia.org/r/559907 (owner: Ejegg) [17:48:56] !log updated fundraising internal dashboard from b75f9074de to 852f6871cf [17:49:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:15:49] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Queue consumers should log messages as JSON - https://phabricator.wikimedia.org/T241261 (Ejegg) [19:14:53] (CR) Jgleeson: [C: +2] "thanks! working great at this end." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/558794 (https://phabricator.wikimedia.org/T238691) (owner: Eileen) [19:16:49] (CR) Jgleeson: [C: +2] "I know we worked out that we don't need unique names anymore however I like the neater approach to building the tables and using the inbui" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/558802 (https://phabricator.wikimedia.org/T238691) (owner: Eileen) [19:17:14] (CR) Jgleeson: [C: +2] "Good idea!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/558813 (owner: Eileen) [19:32:13] hmmm ejegg looks like I jumped the gun. I can see the output on the cli of the new metrics for recurring jobs but I dont see anything added to the actual .prom output files. Digging in now to see what's happening [19:32:25] this is re https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/556264/ [19:32:52] oh hmm [19:33:02] actually, I need to smoke test that too :S [19:33:27] I guess I was really curious if you felt like we should call that hook after each donation [19:33:57] and send the whole context to the hook rather than just counts by status [19:35:08] how would our hook listener call the stats collector in the per-donation case so that it eventually wrote something [19:35:22] ? [19:35:46] would we have to add a finalize hook call afterward? [19:36:23] I feel like if we were gonna do more than just count the status messages the individual hook events might be better but then you're back in the realm of finding a singleton container to hold those during the process which would need some thought [19:36:53] the statscollector could do that yeah [19:37:11] I think the api is something like export() [19:37:15] at the end of the run [19:37:23] and it writes whatever is collected prior [19:38:19] I did add a to-do item to have listener behaviour for stats collection to achieve the hook-like behaviour https://github.com/jackgleeson/stats-collector#to-do [19:41:47] I feel like this patch is good enough for now if we just wanna count success/failure [19:42:20] if we wanted to break out by processor and other things, maybe we would then get more granular [19:49:57] interesting. it looks like the hook isn't being found [19:50:27] when civi goes through it's function name building steps [19:50:48] concatenating module names to hook functions [19:54:32] ahh dang [19:54:39] ok, I'll look into that [19:54:40] thanks! [19:54:44] gonna grab lunch now [20:08:10] Fundraising Sprint YAMLton, the Musical, Fundraising-Backlog: contact merge not working for MG - https://phabricator.wikimedia.org/T241276 (DStrine) p:Triage→Unbreak! [20:21:44] ejegg: it looks like hooks need to live in the .module file unless they are added in as a new hook group. This is according to the sparse drupal docs on the matter [20:22:11] although I just tried adding a 'hooks' group following this https://api.drupal.org/api/drupal/modules%21system%21system.api.php/function/hook_hook_info/7.x and that didn't work [20:37:17] jgleeson: hmmm, drupal hooks vs civi hooks [20:37:38] I'm trying to use the civi hook dispatcher here [20:37:59] so let me see where that one harvests its listeners from [20:50:16] Fundraising Sprint YAMLton, the Musical, Fundraising-Backlog: contact merge not working for MG - https://phabricator.wikimedia.org/T241276 (Ejegg) @DKaufman can you list the CIDs that you're trying to merge? [20:56:58] jgleeson looks like this patch of eileen's is a necessary precursor to the ones you've +2ed: https://gerrit.wikimedia.org/r/558793 [20:57:24] I guess some of what it does is changed in a later patch, but it looks OK in context [20:57:33] Any objection to my +2ing that one too? [21:45:35] (PS3) Ejegg: Report recurring stats to grafana via hook [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/556264 (https://phabricator.wikimedia.org/T199390) [21:51:51] (CR) jerkins-bot: [V: -1] Report recurring stats to grafana via hook [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/556264 (https://phabricator.wikimedia.org/T199390) (owner: Ejegg) [23:07:04] (PS4) Ejegg: Report recurring stats to grafana via hook [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/556264 (https://phabricator.wikimedia.org/T199390)