[10:31:09] (PS1) AndyRussG: Initial general e-mail preferences prototype [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/672362 (https://phabricator.wikimedia.org/T263887) [10:32:30] (PS2) AndyRussG: Initial general e-mail preferences prototype [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/672362 (https://phabricator.wikimedia.org/T268510) [10:36:00] (CR) jerkins-bot: [V: -1] Initial general e-mail preferences prototype [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/672362 (https://phabricator.wikimedia.org/T268510) (owner: AndyRussG) [16:49:24] Fundraising-Backlog, fundraising-tech-ops: Fix duplicate entries in pgehres tables so we can convert them to utf8mb4 - https://phabricator.wikimedia.org/T277484 (Dwisehaupt) [17:34:44] (PS1) Cstone: WIP: Testing CI [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/672470 [17:38:05] (CR) jerkins-bot: [V: -1] WIP: Testing CI [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/672470 (owner: Cstone) [18:02:03] Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (jgleeson) Hi email folks. I can't see an obvious reason for this change. The data in the unsubscribe file we're looking at from today (15th) looks good. Co... [18:23:07] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Patch-Needs-Improvement: CentralNotice: JS code style, JSCS and JSDoc - https://phabricator.wikimedia.org/T111387 (Jdlrobson) [19:08:32] (PS1) Eileen: Move CiviCRM folder to the 'real' location [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/672513 (https://phabricator.wikimedia.org/T273822) [19:09:40] eileen: do the silverpop export unit tests work for you locally? [19:10:04] jgleeson: yes - I got them to work - running directly, not on the docker image I think [19:10:31] I had to hack the version it expected for python (do we have to hard code that?) [19:10:36] because I have 3.8 [19:11:08] (Does someone want to merge this now - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm-buildkit/+/669993 - we can run composer 2 - yay) [19:11:28] yeah I haven't touched silverpop on the docker side of things. I'm using mw-vagrant for that [19:12:00] I'm running python 3.5.3 on the local vm [19:12:58] eileen: did you run into the errors on CI on this patch? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/tools/+/671217 [19:13:12] I'm seeing them locally, complaining about utf8mb4_general [19:13:14] (CR) jerkins-bot: [V: -1] Move CiviCRM folder to the 'real' location [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/672513 (https://phabricator.wikimedia.org/T273822) (owner: Eileen) [19:13:28] although I've dropped and rebuilt the "test" database and set to the default collation [19:14:17] fundraising-tech-ops: improve civicrm dump/restore strategy - https://phabricator.wikimedia.org/T276100 (Jgreen) [19:14:37] I couldn't see in the CI config where we're adding that test database for the silverpop export tests [19:15:05] there is an example yml file that needs renaming [19:17:34] huh [19:17:38] in fundraising-tools? [19:19:12] Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (TSkaff) {F34162182} {F34162184} [19:23:24] Fundraising Sprint Esperantoland, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Composer: Update Fundraising tech C image to use upstream buildkit, no symlink for civicrm - https://phabricator.wikimedia.org/T277500 (Eileenmcnaughton) [19:24:04] Fundraising Sprint Esperantoland, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Composer: Update Fundraising tech CI image to use upstream buildkit, no symlink for civicrm - https://phabricator.wikimedia.org/T277500 (Eileenmcnaughton) [19:24:08] eileen: not sure if you've seen this but the silverpop export since the update to ut8mb4 is apparently sending less "duplicate" emails over. https://phabricator.wikimedia.org/T277366 [19:24:16] did we release a patch to that around the same time? [19:24:30] I don't think so [19:25:00] so we went from sending over 5.2mln duplicate records to 119 [19:25:31] it depends what "duplicate" means I guess [19:25:43] if it means, the same email in the single file repeated or... [19:25:56] an email address already on the acoustic unsubscribed list [19:26:09] in both cases, having less of them would be better I think [19:26:25] but still unexplained at this point [19:27:45] Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (nisrael) Hi Jack, here are the stats for today's import {F34162201} [19:27:51] Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (jgleeson) Thanks @TSkaff. [19:28:16] yeah agree - my ability to run the tests has gone backwards :-) [19:28:32] or rather :-( [19:28:57] hmm\ [19:29:13] so i thought it was due to my local "tests" db needing an update to the new collation but that didn't fix it [19:29:27] so for some reason it's treating something as utf8mb4_general [19:29:31] when comparing stuff [19:33:52] Hi all, at this point we're trying to assess whether to reschedule tomorrow's Sweden email send. Do you feel you have insight into whether or not there is risk? [19:38:07] tskaff: we're on a call now and it'll probably get discussed there so we should know more shortly [19:38:22] will come back once we conclude the call [19:38:24] Awesome thanks! Grabbing lunch [19:41:56] Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (DStrine) @bsisolak adding you here to see if you have feedback [19:47:47] Fundraising Sprint Esperantoland, Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (DStrine) p:Triage→High [20:43:08] Fundraising Sprint Esperantoland, Fundraising-Backlog, FR-Email: Civi to Acoustic export: Confirming value for Total Duplicate Addresses - https://phabricator.wikimedia.org/T277366 (Eileenmcnaughton) So if we upload the same 5 million addresses each day & each day we upload a handful of new ones then... [21:37:54] fundraising-tech-ops: FY2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Dwisehaupt) [21:42:00] Fundraising-Backlog, fundraising-tech-ops: Fix duplicate entries in pgehres tables so we can convert them to utf8mb4 - https://phabricator.wikimedia.org/T277484 (Dwisehaupt) [21:42:20] Fundraising-Backlog, fundraising-tech-ops: Fix our database to support emojis - convert tables to utf8mb4 - https://phabricator.wikimedia.org/T270218 (Dwisehaupt) Open→Resolved This work has been completed. Any further issues should be opened in new tasks, ie T277484. [21:42:22] fundraising-tech-ops: FY2020-21 Q3 maintenance window - https://phabricator.wikimedia.org/T268056 (Dwisehaupt) [21:44:06] fundraising-tech-ops: Access to Civi Terminal for Evelyn Martin for Reconciliation queries - https://phabricator.wikimedia.org/T257482 (Dwisehaupt) @EMartin Is this access still needed? [21:45:13] Fundraising-Backlog, fundraising-tech-ops: Setup shell access and civi access for civi staging for Coleman Watts - https://phabricator.wikimedia.org/T252693 (Dwisehaupt) Stalled→Resolved Closing out this task for now. Will reopen when then get around to needed the full access. [21:45:48] Fundraising-Backlog, fundraising-tech-ops: Setup shell access and civi access for civi staging for Tim Otten - https://phabricator.wikimedia.org/T252694 (Dwisehaupt) Stalled→Resolved Closing out this task for now. Will reopen when then get around to needed the full access. [21:51:47] Fundraising Sprint Esperantoland, Fundraising-Backlog, fundraising-tech-ops: Test silverpop export with utf8mb4 tables - https://phabricator.wikimedia.org/T276293 (Dwisehaupt) The work has been completed and silverpop runs are occurring. We are tracking an accuracy task in T277366 but there is curren... [21:54:26] Fundraising-Backlog, fundraising-tech-ops, fr-tech-ops-okr: OKR 2019-2020 Q3: Increase visibility and awareness of Fundraising system health and wellness - https://phabricator.wikimedia.org/T243110 (Dwisehaupt) Open→Resolved [22:28:24] fundraising-tech-ops: encrypt email traffic between fundraising servers - https://phabricator.wikimedia.org/T170324 (Dwisehaupt) Open→Resolved a:Dwisehaupt [23:12:16] fundraising-tech-ops: Access to Civi Terminal for Evelyn Martin for Reconciliation queries - https://phabricator.wikimedia.org/T257482 (EMartin) yes, @dwidehaupt, it is if possible