[14:57:33] morningsies! [15:23:57] hey AndyRussG [15:28:11] dstrine: :) [15:30:55] Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4204941 (MBeat33) [16:02:07] Fundraising-Backlog, FR-Ingenico: Ingenico audit wobble? iDEAL donations not in Civi May 1-3 - https://phabricator.wikimedia.org/T194296#4205043 (MBeat33) Netherlands fundraising email3 is scheduled for the 17th. Can we get this batch into Civi before then, to avoid bulk email complaints? [19:23:17] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4155376 (XenoRyet) a:XenoRyet [19:37:57] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4205576 (DStrine) [19:43:47] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, FR-Ingenico: Ingenico audit wobble? iDEAL donations not in Civi May 1-3 - https://phabricator.wikimedia.org/T194296#4205618 (DStrine) [20:06:59] cwd: Jeff_Green wanna chat? [20:07:29] ok be there in a couple mins [21:48:33] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4205981 (XenoRyet) The email lists and transaction lists for both GC and PayPal have been generated and are in the tmp folder... [21:48:48] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4205982 (XenoRyet) a:XenoRyet>None [21:52:52] Fundraising-Backlog, fundraising-tech-ops: Fundraising queues missing prometheus data since 4/26 - https://phabricator.wikimedia.org/T193456#4205999 (cwdent) Nothing obvious server side except that the numbers aren't there at civi1001:/var/spool/prometheus I believe they should be written by process-con... [22:16:00] I am turning off the process control job dedupe_civicrm_contacts_from_start.yaml [22:16:42] it has caught up to my main job & my current working theory is that we hit a data hiccup due to both jobs trying to process the same contact at the same time [22:24:50] !log update process-control to 97fd42b3ab disabling dedupe catch up [22:24:53] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:33:11] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4204941 (Eileenmcnaughton) What I can see is that there are 2 merge records at the same time for thi... [23:00:25] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4206089 (Eileenmcnaughton) For more on the scale of this issue - I find 2886 'same moment merges' fr... [23:16:19] am running a query on frdev on slave db - which is a bit slow. Am monitoring to see how slow but noting here in case it triggers alerts [23:20:16] Thanks :) [23:52:54] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4206149 (Eileenmcnaughton) @MBeat33 - I feel pretty confident I have stopped these from happening. H... [23:53:29] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Recover lost data from 'same-moment-merges' - https://phabricator.wikimedia.org/T194699#4206151 (Eileenmcnaughton) [23:55:17] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Restore live DB back to staging so I can test recovery of 'same moment merges' - https://phabricator.wikimedia.org/T194700#4206163 (Eileenmcnaughton) [23:55:29] Fundraising Sprint Junebugs prefer July, Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Civi dedupe script error? losing email addresses in recent merges - https://phabricator.wikimedia.org/T194662#4206173 (MBeat33) Great, thank you for the updates @Eileenmcnaughton Anything DS can do to help with... [23:56:20] Fundraising-Backlog, Fr-CiviCRM-dedupe-FY2017/18: Figure out a mechanism to 'reserve' contacts once they have been picked for dedupe - https://phabricator.wikimedia.org/T194701#4206174 (Eileenmcnaughton) [23:57:20] cwd - just logged a phab to get staging updated - https://phabricator.wikimedia.org/T194700 - just wanting to get a feel for when you might be able to do it?