[00:16:23] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Improve robustness of forgetme with regards to silverpop - https://phabricator.wikimedia.org/T212382 (Eileenmcnaughton) [00:21:46] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: how did CID=6220811 end up with two primary emails? - https://phabricator.wikimedia.org/T212331 (Ejegg) Ooh, looks like CID 4724365 also did, according to https://phabricator.wikimedia.org/T199747#4833123 [00:24:03] (PS1) Eileen: Fix non-interaction with Silverpop when no mailing data. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/480886 (https://phabricator.wikimedia.org/T212382) [00:25:11] ejegg|afk: MBeat I found the forgetme was not sending an erase request to silverpop when there was no mailing data - fix here https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/480886/ - that would mostly affect when the first attempt failed but deleted some stuff & then it succeeded [00:26:27] eileen_: ooh, right, because it's part of the Omnirecipient extension [00:26:48] ejegg: yeah - the proposed fix is hacky but I’ve commented as such & added a test [00:26:51] and the pattern is to only forgetme if showme returns something [00:26:58] exactly :-( [00:27:15] well, let's see if it'll get us through the holidays :) [00:28:02] yeah - I mean I can start working on how to save & follow up delete requests but I don’t think I’ll get it done before I finish for the day/week/year [00:28:03] oh boy, that is a fun one :) [00:28:07] (CR) jerkins-bot: [V: -1] Fix non-interaction with Silverpop when no mailing data. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/480886 (https://phabricator.wikimedia.org/T212382) (owner: Eileen) [00:29:37] ug test challenges :-( [00:39:54] (PS2) Eileen: Fix non-interaction with Silverpop when no mailing data. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/480886 (https://phabricator.wikimedia.org/T212382) [00:40:12] ejegg: it just got worse :-( - this is how a patch written in the last half of the last day looks [00:41:25] haha, no_omnimail_hack [00:41:39] if not no hack, don't do the hack [00:42:19] :-) [00:42:45] wait, i mean DO the hack [00:43:11] the problem is figuring a sensible way to handle the http requests… in the main class [00:43:25] hence I amblatantly hacking [00:43:52] that can go in the tech debt column of our next sprint review [00:47:24] ejegg: looks like I have reached critical hack for passing tests :-) [00:51:14] hehe, nice [00:55:19] (CR) Ejegg: [C: +2] "Well, if it's hacky, at least it's blatantly hacky :)" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/480886 (https://phabricator.wikimedia.org/T212382) (owner: Eileen) [01:00:13] (Merged) jenkins-bot: Fix non-interaction with Silverpop when no mailing data. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/480886 (https://phabricator.wikimedia.org/T212382) (owner: Eileen) [01:04:27] (PS1) Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/480891 [01:06:59] (CR) Eileen: [C: +2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/480891 (owner: Eileen) [01:07:21] have a good night eileen_ [01:07:27] oh hey, and a merry xmas! [01:07:50] you too [01:07:58] (Merged) jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/480891 (owner: Eileen) [01:09:16] !log civicrm revision changed from 9d727e4708 to b33dcd3c94, config revision is 0f94a475b7 [01:09:17] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:30:26] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Improve robustness of forgetme with regards to silverpop - https://phabricator.wikimedia.org/T212382 (Eileenmcnaughton) Fix for 1 above deployed [08:30:05] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: how did CID=6220811 end up with two primary emails? - https://phabricator.wikimedia.org/T212331 (Eileenmcnaughton) This appears to be a merge issue - relating to manually merged contacts [13:01:28] AndyRussG: Hey Andy, you about? [13:07:32] Fundraising-Backlog, Analytics, Analytics-Kanban, Patch-For-Review, User-Elukey: Return to real time banner impressions in Druid - https://phabricator.wikimedia.org/T203669 (JAllemandou) Druid-kafka-supervisor task and how-to added to refinery in `oozie/banner_activity/druid` folder (https://... [15:03:11] Seddon: hewwo [15:03:14] I'm here now [15:06:35] AndyRussG: quick question. Are buckets randomised for each campaign or across the board? [15:06:42] Seddon: yes [15:07:01] They're completely separate for each campaign [15:07:17] Okay thanks :) [15:07:18] Newly chosen randomly the first time a user gets a campaign [15:07:50] randomly only A or B may be chose initially [15:08:04] then may be re-randomized between C or D if a campaign is doing the large banner-small banner thing [15:09:14] There may be issues with lack of complete randomness due to not using the most truly random numbers available (there's a task for that somewhere), but those should be really marginal cases, only noticeable, if at all, when looking at large amounts of data [15:09:17] Seddon: ^ [15:58:03] Seddon: hey btw any thoughts on the last comment on https://phabricator.wikimedia.org/T209873 ? [15:58:06] thx in advance [16:35:56] (PS4) Ejegg: Don't re-protect banner if it's already protected [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/478257 (https://phabricator.wikimedia.org/T210983) [16:44:09] (CR) Ejegg: "Now depending on a core change that lets us force loading restrictions from the master DB. Still a bit redundant - maybe better to have do" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/478257 (https://phabricator.wikimedia.org/T210983) (owner: Ejegg) [20:30:30] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: how did CID=6220811 end up with two primary emails? - https://phabricator.wikimedia.org/T212331 (Ejegg) [20:30:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Do some data integrity digging - https://phabricator.wikimedia.org/T212271 (Ejegg) [20:37:29] (PS1) Ejegg: Alphabetize hooks in extension.json [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481040 [20:38:07] oh nice cleanup ejegg [20:38:51] (PS2) Ejegg: Alphabetize hooks in extension.json [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481040 [20:41:44] (CR) jerkins-bot: [V: -1] Alphabetize hooks in extension.json [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481040 (owner: Ejegg) [20:42:18] AndyRussG: heh, almost! [20:42:54] oh, it's that flappy kvStore again [20:43:06] yeeeeeeeeee [20:43:09] flappy flappy [20:43:14] also, yeah, breakpoint! [20:43:21] woohoo! [20:43:42] so, the kvStore cleanup uses something that calls it after a random delay, right? [20:45:19] hmm, and the test is using an async assert [20:45:46] I don't remember at all [20:46:06] It's in the sprink though [20:46:11] oh i see, we're using requestIdleCallback to schedule it in the startup module [20:46:20] but in the test we're calling the maintenance function directly [20:48:16] weird, how does assert.async() work anyway? [20:55:36] ohhh, processKeys has ANOTHER requestIdleCallback in it [20:55:44] why would we need to nest that two deep? [20:56:29] ooh, each iteration gets another requestIdle, even [20:56:45] ah yeah I remember that stuff, fun heheh [20:56:55] if it's faster we can just disable the test temporarily btw [20:57:32] feel like I should spend another few minutes trying to understand it first [20:57:49] Aaaand now my IDE is crashing.................. [20:58:03] oh fun :P [20:58:32] rrrrghhh [21:00:34] AndyRussG: aha! It's the LEEWAY_FOR_REMOVAL [21:00:50] the 'Old' one to be deleted is only -1 day [21:01:01] and the LEEWAY_FOR_REMOVAL is 1 day in seconds [21:01:24] and the comparison for whether to delete is strictly < [21:01:44] so the test fails whenever the IdleCallback runs two iterations within the first second [21:01:58] oops no [21:02:10] the 'Old' on is -2 days [21:02:16] *one [21:02:23] hmmm [21:09:52] (PS1) Ejegg: DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 [21:11:54] (PS2) Ejegg: DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 [21:17:47] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (owner: Ejegg) [21:20:35] (CR) jerkins-bot: [V: -1] DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (owner: Ejegg) [21:20:47] awww [21:21:05] Wait, I kinda want to merge...... [21:21:18] Ahhh the +2 button.... [21:22:04] (PS3) Ejegg: DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 [21:28:36] (PS4) Ejegg: DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 [21:29:20] ok, my latest suspect is the requestIdleCallback inside getKeys [21:30:00] it DOESN'T iterate over multiple idle callbacks when the deadline times out - it just returns whatever keys it found in the first shot [21:31:29] (CR) jerkins-bot: [V: -1] DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (owner: Ejegg) [21:32:42] yep yep yep! [21:33:03] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (owner: Ejegg) [21:33:23] 'getKeys found 0 keys, index is 2, localStorage length is 3' [21:34:53] uff, that 'index-- > 0' confused me for a second too [21:35:26] but it's correct [21:36:23] so if we were to iterate that one too... [21:41:38] Fundraising Sprint Window dressing is mostly olive oil, Fundraising Sprint XML ate my homework, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: test ext.centralNotice.kvStore fails sometimes - https://phabricator.wikimedia.org/T208570 (Ejegg) p:Triage→Normal a:Ejegg [21:42:02] (PS5) Ejegg: DO NOT MERGE: kvstore test debugging [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) [21:47:52] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [21:52:09] ok, two possible solutions - shim requestIdleCallback in the test, or change getKeys to iterate over all of localStorage, even if it takes multiple idleCallback requests [21:55:52] (PS6) Ejegg: kvStoreMaintenance: getKeys tests all of localStorage [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) [21:56:29] ejegg: not sure if you saw https://phabricator.wikimedia.org/T209944 [22:01:12] (PS1) Ejegg: Shim requestIdleCallback in kvStore test [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) [22:04:31] (PS2) Ejegg: Shim requestIdleCallback in kvStore test [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) [22:04:57] Hauskatze: that's new to me, thanks! [22:05:24] Hauskatze: ah, that looks like another npm false alarm [22:05:52] we've got the package-lock file in there for a theme we're using, but since we're not developing on the theme none of us ever install the node modules [22:05:57] will write as much on the ticket [22:06:05] and maybe zap the package-lock [22:06:16] ejegg: just reporting the alerts I've got from GitHub so the people that deal with the project decides what to do :) [22:07:06] I'll check the upstream repo for that theme and see if they've still got the same versions listed [22:08:59] thanks :) [22:09:16] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:09:18] I think hashar deals with the github sec alerts these days though [22:09:22] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:10:22] Thanks Hauskatze, if we need to silence it administratively I'll go to him [22:10:47] I can do it but I prefer not to touch stuff I don't fully know :) [22:19:46] (PS1) Ejegg: Update Shoreditch to 0.1-alpha30 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/481094 [22:20:20] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:20:26] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:20:46] ^^^ just gonna spam those with 'recheck' a dozen times or so to see if they're solid fixes [22:25:25] (PS3) Ejegg: Shim requestIdleCallback in kvStore test [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) [22:25:32] (PS7) Ejegg: kvStoreMaintenance: getKeys tests all of localStorage [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) [22:26:04] just adding a note to the commit message pointing to the alternatives [22:30:50] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:31:00] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:32:06] AndyRussG: I'm feeling pretty good about those ^^^. Probably you'd prefer the test-shim-fix to the iterate-to-get-all-keys-fix, but both are ready for perusal [22:33:54] (CR) jerkins-bot: [V: -1] kvStoreMaintenance: getKeys tests all of localStorage [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:34:44] ejegg: ok thanks so much! [22:36:26] oooooh, there's a fail [22:37:38] well, I guess that's another reason to pick the test-shim-fix [22:39:26] yeah, I think I'll abandon the changes-key-functionality-fix [22:40:07] (Abandoned) Ejegg: kvStoreMaintenance: getKeys tests all of localStorage [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481047 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg) [22:40:54] (PS4) Ejegg: Shim requestIdleCallback in kvStore test [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) [22:48:32] (PS1) Ejegg: Add revision tag to CN admin edits [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481099 (https://phabricator.wikimedia.org/T209795) [22:50:10] ok fr-tech, have a great weekend and a happy xmas if you're into that sort of thing! [22:50:27] you too ejegg and have a great day tomorrow|! [22:50:39] Yea, congrats again. Enjoy it! [22:50:41] Merry Christmas to you too ejegg [22:50:47] Thanks! [22:50:50] :) [22:51:53] Fundraising Sprint XML ate my homework, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Patch-For-Review: Add 'CentralNotice' tag to all CentralNotice banner edits - https://phabricator.wikimedia.org/T209795 (Ejegg) p:Triage→Normal a:Ejegg [23:01:20] (CR) Ejegg: "recheck" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/481091 (https://phabricator.wikimedia.org/T208570) (owner: Ejegg)