[00:04:26] (PS1) Eileen: Fix error on deleting activities with more than one link to the same contact. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469142 (https://phabricator.wikimedia.org/T204063) [00:11:12] (PS1) Eileen: Alter permission for 'forgetme' from administer CiviCRM to edit all contacts [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469144 (https://phabricator.wikimedia.org/T204063) [01:10:11] (CR) Eileen: "recheck" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 (owner: Eileen) [01:17:22] (CR) jerkins-bot: [V: -1] Drupal submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 (owner: Eileen) [01:24:11] (PS1) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [01:31:16] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (owner: Eileen) [01:53:12] (PS3) Eileen: Drupal submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 [01:58:50] (CR) jerkins-bot: [V: -1] Drupal submodule commit [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 (owner: Eileen) [02:03:21] (PS18) Eileen: Functional import using batch api. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/463898 (https://phabricator.wikimedia.org/T204008) [02:03:52] ejegg: I think I fixed the double up in output files - changed the fopen mode when instantiating [02:04:37] cool, i'll take another look [02:07:14] (PS4) Eileen: Drupal submodule commit (core security update) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 [02:07:19] eileen_: wait, wouldn't that just overwrite everything from the previous batch? [02:07:35] ejegg: no I think the files are only created once [02:08:02] & opened multiple times [02:08:05] hmm, lemme try with batch size 1 [02:08:25] ie. createOutputFile is only called from the start [02:09:53] eileen_: also, I think that drupal submodule update is already merged [02:10:04] (unless there's been another security update since last week) [02:10:06] ejegg: ah is that why it’s messing with me! [02:10:26] sorry, i should have mentioned I merged it [02:10:30] it's all deployed [02:10:58] ejegg: ok - but locally I still need that patch - maybe it’s something weird locally [02:11:29] hmm - do you have commit 25937313ab4b0475aa48aaa32a695a9709b10d85 ? [02:11:44] yeah I seem to [02:11:58] in master it's between 'Extract setMessages function' and 'Do our validation & header instantiation in construct, as long as we have the field uri' [02:13:14] hmm it doesn’t change the subcommit to the one that I have origin/master as being on the drupal side [02:14:09] I have 6a23ceb2639bb6cd75099f819a28 as a commit on top - which is our patches going back on [02:14:43] but it might be empty!!! [02:15:38] eileen_: OOOOHHHH, now I get it. There's no problem with the headers repeating, because the constructor doesn't get called on the batches after the first - the importer class is deserialized [02:15:43] sorry [02:15:44] perhaps that’s the issue - there is an empty commit in the tree - still I can approve that now & it will be resolved [02:16:14] ejegg: the headers weren’t repeating BUT multiple runs WERE appending to the same file [02:16:29] so I changed the a to w in createOutputFiles just before [02:17:09] Ahh, shoot, was it re-using the same filename before? [02:17:10] (CR) Eileen: [C: 2] "this wound up empty! still merging gets us in sync" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 (owner: Eileen) [02:17:14] I guess it was, huh [02:17:45] ejegg: yep - which I think is what you saw & identified as repeating headers [02:17:54] but they were from diff tests not diff batches [02:17:59] got it [02:18:57] so if someone uploads a file with the same name it'll overwrite old output files. Not the worst thing in the world, I guess [02:19:42] offtopic - seeing a lot of deprecated function warnings referring to code deep in Drupal [02:20:10] do you know how the work is going on Civi supporting Drupal 8? [02:21:54] so civi does work with drupal 8 but my understanding is the initial config is more painful than you would expect in a ‘finished’ product & some of the integratiosn (which we don’t use) are not done or nor mature [02:22:36] (Merged) jenkins-bot: Drupal submodule commit (core security update) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/468204 (owner: Eileen) [02:23:04] ok here is an embarrassing footnote - I think the reason I failed to demo imports correctly during civi fortnightly is just that I clicked on the wrong button [02:23:16] (although the meta reason may still be that I was pre-coffee) [02:33:18] (CR) Ejegg: [C: 2] "Looks good! I like the idea of the table for result CSVs." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/463898 (https://phabricator.wikimedia.org/T204008) (owner: Eileen) [02:33:50] ejegg: yay! [02:34:00] d'oh! demo-itis is definitely more likely in a caffeine-deprived state. [02:34:07] :-) [02:34:11] ok, gonna sign off for the night. have a good one! [02:40:32] (Merged) jenkins-bot: Functional import using batch api. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/463898 (https://phabricator.wikimedia.org/T204008) (owner: Eileen) [03:02:27] Fundraising Sprint They Live, Fundraising Sprint USB stands for underhanded socket bureaucracy, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Extend deletion to multiple silverpop databases - https://phabricator.wikimedia.org/T205332 (Eileenmcnaughton) a:Eileenmcnaughton @cc [03:05:51] (PS2) Eileen: Fix error on deleting activities with more than one link to the same contact. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469142 (https://phabricator.wikimedia.org/T204063) [03:05:53] (PS2) Eileen: Alter permission for 'forgetme' from administer CiviCRM to edit all contacts [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469144 (https://phabricator.wikimedia.org/T204063) [03:05:55] (PS2) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [03:05:57] (PS1) Eileen: Test data cleanup to better support multiple local runs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469157 [03:10:47] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (owner: Eileen) [03:14:50] (CR) jerkins-bot: [V: -1] Test data cleanup to better support multiple local runs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469157 (owner: Eileen) [04:23:10] (PS3) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [04:28:10] (PS4) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [04:28:27] (PS2) Eileen: Test data cleanup to better support multiple local runs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469157 [04:30:50] Fundraising Sprint USB stands for underhanded socket bureaucracy, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Scheduled contact reminder didn't work - https://phabricator.wikimedia.org/T206053 (Eileenmcnaughton) @LeanneS should we close this now? [04:34:01] (CR) jerkins-bot: [V: -1] Test data cleanup to better support multiple local runs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469157 (owner: Eileen) [04:35:11] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (owner: Eileen) [06:27:00] (CR) WMDE-Fisch: [C: 2] Update Gemfiles to use newest libraries [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/469060 (https://phabricator.wikimedia.org/T204119) (owner: WMDE-Fisch) [06:31:50] (Merged) jenkins-bot: Update Gemfiles to use newest libraries [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/469060 (https://phabricator.wikimedia.org/T204119) (owner: WMDE-Fisch) [06:40:55] (CR) jenkins-bot: Update Gemfiles to use newest libraries [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/469060 (https://phabricator.wikimedia.org/T204119) (owner: WMDE-Fisch) [13:11:17] (PS17) Jgleeson: WIP: Backend Stats Comparison Script [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/456016 (https://phabricator.wikimedia.org/T198752) [14:50:08] (PS18) Jgleeson: WIP: Backend Stats Comparison Script [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/456016 (https://phabricator.wikimedia.org/T198752) [16:49:19] (PS1) Ejegg: Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469223 [16:49:22] (CR) Ejegg: [C: 2] Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469223 (owner: Ejegg) [16:50:05] (Merged) jenkins-bot: Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469223 (owner: Ejegg) [16:55:12] (PS1) Ejegg: Update Amazon SDK fork [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/469226 [16:55:16] (CR) Ejegg: [C: 2] Update Amazon SDK fork [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/469226 (owner: Ejegg) [16:58:44] (CR) Ejegg: [V: 2 C: 2] Update Amazon SDK fork [wikimedia/fundraising/SmashPig/vendor] - https://gerrit.wikimedia.org/r/469226 (owner: Ejegg) [16:59:52] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469228 [17:00:03] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469228 (owner: Ejegg) [17:00:41] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469228 (owner: Ejegg) [17:03:29] !log updated standalone SmashPig deployment from 2292111bda to 18da9727d8 [17:03:31] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:06:08] !log rolled SmashPig back to 2292111bda [17:06:10] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:12:29] (PS1) Ejegg: Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469230 [17:12:41] (CR) Ejegg: [C: 2] Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469230 (owner: Ejegg) [17:13:07] (Merged) jenkins-bot: Update Amazon SDK fork [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/469230 (owner: Ejegg) [17:14:40] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469231 [17:14:48] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469231 (owner: Ejegg) [17:15:04] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - https://gerrit.wikimedia.org/r/469231 (owner: Ejegg) [17:18:50] !log updated standalone SmashPig deploy from 2292111bda to b638ca02bc [17:18:51] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:13:33] PROBLEM - check_ipsec on civi1001 is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: americium_v4 [19:13:43] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [19:18:33] PROBLEM - check_ipsec on civi1001 is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: americium_v4 [19:18:43] PROBLEM - check_ipsec on americium is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: civi1001_v4 [19:22:56] (PS2) AndyRussG: Remove testing field (for banner previews) from CNEvents [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/463110 (https://phabricator.wikimedia.org/T195594) [19:23:33] RECOVERY - check_ipsec on civi1001 is OK: Strongswan OK - 1 ESP OK [19:23:43] RECOVERY - check_ipsec on americium is OK: Strongswan OK - 1 ESP OK [19:24:26] (PS1) AndyRussG: Remove unused import in controller.py [wikimedia/fundraising/FRUEC] - https://gerrit.wikimedia.org/r/469251 [19:26:59] cwd: quick question, do we use MySQL or MariaDB? [19:27:32] mariadb [19:35:54] Fundraising-Backlog: Get Scrutinizer jobs working again for fundraising projects - https://phabricator.wikimedia.org/T207784 (Ejegg) [19:44:05] (CR) Eileen: [C: 2] Round things to the right number of places [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/463365 (owner: Ejegg) [19:49:39] (Merged) jenkins-bot: Round things to the right number of places [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/463365 (owner: Ejegg) [20:55:33] ooh, shoot, there IS something still creating null-opt-in recurring-paypals [20:55:44] so what the heck is different with the recurring? [21:10:12] looks like maybe the order_id doesn't come back with the .sequence ? [21:16:27] crap, speaking of recurring issues, what's up with that new failmail> [21:17:29] ? [21:28:29] is that the failmail from earlier today - of have I missed something more recently [21:31:11] eileen: there was a failmail half an hour ago with lots and lots of paypal subscription payments whose subscription IDs don't show up in our db [21:31:41] ejegg: hmm - I think I cleared them with my ‘overnight’ emails - assuming they had been dealt with [21:32:24] guessing it's related to their switch from S-XXX to I-XXX ids [21:33:04] jgleeson_: You had a patch that needed some CR? [21:33:15] oopps, flag still up again. [21:33:27] https://phabricator.wikimedia.org/T203126 [21:33:29] yup [21:33:58] Cool, I'll take a look [21:34:01] thanks! [21:34:57] argh, the more I look the more problems I find [21:35:18] are ALL the paypal EC orphan rectifier things failing? [21:36:47] ejegg, is the damage store a db tbl or queue? [21:36:50] ok, no [21:36:56] jgleeson_: it's a DB table [21:37:00] could count there to see the size of it [21:37:09] it used to be a queue, so you might see some references to that still [21:37:48] thanks [21:43:07] fr-tech so... all the paypal listener messages use the bare contribution_tracking_id as the order_id [21:43:40] while messages from payments-wiki, pending messages, antifraud messages, all use the ct_id.sequence [21:44:37] I'm pretty sure we need to change the listener messages [21:45:05] hmm, shoot, and the rp_invoice_id is being set from the PROFILEREFERENCE [21:45:13] not the INVNUM field [21:45:28] argh, this is not a thing I want to be messing with this late in the game [21:45:35] but it's WRONG... [21:45:48] fr-tech anyone want to sanity check that? [21:48:43] sure ejegg, where am I lookin [21:48:57] jgleeson_: ah, sorry, it's super late for you! [21:49:07] :) [21:49:07] I'll write this up in my end of day email [21:49:16] ok thanks [21:49:21] catch you all tomorrow ! [21:51:28] OK, so all the bad messages from the failmail an hour ago were audit-related [21:59:43] crap, and there's no new column for old subscriber ID [21:59:54] err, old subscription ID [22:00:35] guessing we have to make that subscr_modify stuff work [22:11:08] (PS5) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [22:12:47] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/469335 [22:14:43] (CR) Eileen: [C: 2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/469335 (owner: Eileen) [22:16:28] !log civicrm revision changed from bde28d4453 to 1c0a1b2406, config revision is c0a8be03a1 [22:16:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:16:37] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (owner: Eileen) [22:16:44] I just deployed the import fixes [22:17:46] Fundraising Sprint Sasquatches can't find us either, Fundraising Sprint They Live, Fundraising Sprint USB stands for underhanded socket bureaucracy, Fundraising-Backlog, and 2 others: Benevity import errors - https://phabricator.wikimedia.org/T204008 (Eileenmcnaughton) @LeanneS I just deployed th... [22:22:51] (PS3) Eileen: Test data cleanup to better support multiple local runs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469157 [22:29:32] (CR) XenoRyet: [C: 2] "Looks good to me, and I think it's fine to be reporting the invalid response here, even if it isn't strictly a curl error." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/458208 (https://phabricator.wikimedia.org/T203126) (owner: Ejegg) [22:37:11] (PS6) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 [22:42:56] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (owner: Eileen) [22:51:40] (PS7) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) [22:57:21] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) (owner: Eileen) [23:13:38] (PS8) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) [23:19:23] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) (owner: Eileen) [23:24:06] Fundraising Sprint Sasquatches can't find us either, Fundraising Sprint They Live, Fundraising Sprint USB stands for underhanded socket bureaucracy, Fundraising-Backlog, and 2 others: Benevity import errors - https://phabricator.wikimedia.org/T204008 (LeanneS) Will do! I should have a large file... [23:25:34] (Merged) jenkins-bot: Handle invalid GC SET_PAYMENT response and fix some xml formatting issues in stub responses. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/458208 (https://phabricator.wikimedia.org/T203126) (owner: Ejegg) [23:28:03] (PS9) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) [23:33:40] (CR) jerkins-bot: [V: -1] Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332) (owner: Eileen) [23:44:26] (PS10) Eileen: Handling for multiple erase databases [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/469155 (https://phabricator.wikimedia.org/T205332)