[00:12:51] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4230439 (XenoRyet) Works for me. [06:47:00] Fundraising-Backlog, Google-Summer-of-Code (2018): [GSoC 2018] Machine Learning for Fraud Detection - https://phabricator.wikimedia.org/T190523#4230783 (saurabhbatra96) Meeting 1 discussion points - - Find out the original dataset for https://www.kaggle.com/mlg-ulb/creditcardfraud - Postpone featur... [10:20:46] fundraising-tech-ops, Operations, ops-eqiad: rack frpig1001 - https://phabricator.wikimedia.org/T187365#4231102 (ayounsi) [10:20:48] fundraising-tech-ops, Operations, ops-eqiad: rack frdata1001 - https://phabricator.wikimedia.org/T187364#4231103 (ayounsi) [14:00:23] (PS3) Mepps: Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) [14:02:53] (CR) jerkins-bot: [V: -1] Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [14:04:45] (PS4) Mepps: Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) [14:07:10] (CR) jerkins-bot: [V: -1] Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [14:12:28] good morning fr-tech! [14:12:38] good morning ejegg! [14:12:43] morningsies! [14:13:21] aaaaagh man what a rough morning I just had with the kids... Just as we were rushing to leave, they had a massive fight about a pencil sharpener and a pencil case [14:13:25] mepps good news: I found a better way to distinguish the connect API donations: https://gerrit.wikimedia.org/r/435066 [14:13:36] AndyRussG: oy [14:14:09] ejegg: yeah....... [14:14:09] what is the email type indicator ejegg? [14:14:29] mepps i dunno, but it on shows up in the Connect audit entries [14:14:34] and seems to be in all of them [14:14:48] and none of the old ones [14:14:49] I assume they'll get over it, but... man... [14:15:36] hmm, conflict resolution workshop time? [14:15:41] Maybeeee [14:16:22] Basically both their weaknesses came into play... Sofi's tendency to not take care of other people's stuff, and Ceci's tendency to sometimes get really, really angry [14:16:52] aaaarg they'll be ok [14:17:10] I'm just venting in a public work-related form... because [14:17:44] *forum [14:18:10] yeah that's a tough morning AndyRussG [14:18:49] mepps ejegg thx :) [14:20:35] I guess I need to find a way to spin it after school, to turn it into a positive learning experience for both [14:20:48] or maybe for all three :) [14:26:47] (PS5) Mepps: Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) [14:31:01] fr-tech hey... can someone remind me where I can see the job config for how DjangerBannoScripts are run? thx!!! [14:34:42] AndyRussG: sure! [14:34:46] it's up on frpm1001 [14:34:56] there's a repo for process-control jobs [14:35:12] oh wait, it's part of the general settings repo [14:35:39] in /var/lib/git/localsettings.git [14:35:46] so you clone that to your homedir [14:36:02] then inside your local localsettings/ [14:36:19] take a look in process-control/civi1001 [14:36:51] the regular job is defined in banner_impressions_loader.yaml [14:37:07] while there's a 'backfill' job only run manually in banner_impressions_backfill.yaml [14:38:05] Anyway, these are the two things that are run: manage.py LoadLPImpressions --verbose --recent and manage.py LoadBannerImpressions2Aggregate --verbose --top --recent [14:38:52] ejegg: fantasmic, thx! [14:42:08] mepps so next we need to make the log search look in both payments-globalcollect and payments-ingenico [14:42:32] I think I'm going to override the part where we unzip the files [14:42:47] and instead of just unzipping one file, unzip both and concatenate them [14:52:52] it's a little ugly, but it seems better than e.g. running the whole thing twice [14:53:54] and I think we're fine mashing them up, since order IDs are unique across both [15:08:20] Fundraising-Backlog: New scripts to ingress Kafkatee into MySQL - https://phabricator.wikimedia.org/T195594#4231719 (AndyRussG) [15:12:52] In dstrine's absence, I'm gonna bring T195594 into the sprink, since that's what I'm doing now [15:12:53] T195594: New scripts to ingress Kafkatee into MySQL - https://phabricator.wikimedia.org/T195594 [15:13:35] Fundraising Sprint Asymmetrical Earth Theory, Fundraising Sprint Bermuda Rhombus (where things disappear then reappear), Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising-Backlog, Epic: [Epic] Kafkatee changes - https://phabricator.wikimedia.org/T183978#4231755 (AndyRussG) [15:13:37] Fundraising-Backlog: New scripts to ingress Kafkatee into MySQL - https://phabricator.wikimedia.org/T195594#4231754 (AndyRussG) [15:14:08] AndyRussG: yeah sprinkle that on the Karma chameleons (also I'm not really here) [15:14:10] Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog: New scripts to ingress data from Kafkatee into MySQL - https://phabricator.wikimedia.org/T195594#4231719 (AndyRussG) [15:17:07] dstrine: cool thx! cya :) [15:19:00] Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog: New scripts to ingress data from Kafkatee into MySQL - https://phabricator.wikimedia.org/T195594#4231719 (AndyRussG) a:AndyRussG [15:28:27] whee, new Chromium is quite sped up [15:28:37] just installed from unstable [15:29:00] Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, FR-Ingenico, FR-Smashpig, and 2 others: Create Ingenico orphan rectifier maintenance script - https://phabricator.wikimedia.org/T163949#3215872 (Ejegg) a:mepps [15:29:02] modern browsers are quite amazing applications fer sure [16:16:57] no one on #operations has reacted to the term "swatty ploy" yet [16:17:13] I wonder what would happen if I just renamed them directly on wikitech? [16:19:11] haha [16:20:42] I could make Fab task requesting the name change [16:39:34] cwd: ejegg: Jeff_Green so LoadBannerImpressions.py doesn't run anywhere? Only LoadBannerImpressions2Aggregate.py and LoadLPImpressions? That's what I got from grehpping in /var/lib/git/localsettings.git/var/lib/git/localsettings.git [16:39:54] not sure off hand [16:40:11] it would be a process-control job right? [16:40:24] yep [16:40:24] ooh [16:40:47] /process-control/civi1001/banner_impressions_loader.yaml [16:40:58] and process-control/civi1001/banner_impressions_backfill.yaml [16:41:06] Those are the ones I see [16:41:16] (pointed out also by ejegg) [16:41:56] AndyRussG: yep i see what you see [16:42:59] i also see banner_history_queue_consume [16:43:27] but that looks entirely different [16:43:35] cwd: yeah unrelated [16:44:14] So these are in the civi1001 directory because that's where they run? and that's because of which DB they operate on, I guess? [16:47:58] AndyRussG: because of where they run [16:48:19] it's just how they get deployed [16:48:41] cwd: ah okok thx :) [17:06:37] mepps: You coming to standup or did you have some other meeting? [17:07:36] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232191 (XenoRyet) @MBeat33 @CCogdill_WMF Just... [17:09:19] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232196 (MBeat33) Good from DS-land, @XenoRyet... [17:11:07] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232198 (XenoRyet) @MBeat33 Looks like about 64... [17:14:26] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232202 (MBeat33) Great, that's not a crazy num... [17:31:20] (PS1) XenoRyet: Unintended Recur Part 3 [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/435195 [17:32:31] (CR) jerkins-bot: [V: -1] Unintended Recur Part 3 [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/435195 (owner: XenoRyet) [17:34:46] (PS2) XenoRyet: Unintended Recur Part 3 [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/435195 [17:36:26] (CR) XenoRyet: [C: 2] Unintended Recur Part 3 [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/435195 (owner: XenoRyet) [17:37:45] (Merged) jenkins-bot: Unintended Recur Part 3 [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/435195 (owner: XenoRyet) [17:46:50] !log updated civicrm from 4d797fc592 to 0b97f1f5b2 [17:46:54] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [18:47:11] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232404 (XenoRyet) Cancellations and refunds ha... [18:50:09] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations from Big EN - https://phabricator.wikimedia.org/T192958#4232423 (MBeat33) @XenoRyet awesome news, thank... [19:48:56] (CR) Mepps: [C: 2] Indicate correct gateway for Connect API donations [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/435066 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [19:49:43] (Merged) jenkins-bot: Indicate correct gateway for Connect API donations [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/435066 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [19:54:54] ejegg i think this is ready for review [19:57:42] mepps this? [19:57:53] ohh i copied the link but didn't paste [19:57:55] https://gerrit.wikimedia.org/r/#/c/435003/ [19:58:20] cool, taking a look [20:08:18] (PS1) Ejegg: WIP: Audit functions can deal with multiple logs for one date [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) [20:10:44] (CR) jerkins-bot: [V: -1] WIP: Audit functions can deal with multiple logs for one date [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [20:17:30] (PS6) Ejegg: Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [20:17:41] (CR) Ejegg: [C: 2] "Looks good!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [20:19:12] (Merged) jenkins-bot: Add cancelPayment for rectifyOrphan [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/435003 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [20:25:43] (PS2) Ejegg: Audit functions can deal with multiple logs for one date [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) [20:26:37] fr-tech: ok, hijacking the unzip step to concatenate two logs was just too hacky and error-prone [20:27:07] So I'm working to retrofit all the log search functions to handle multiple logs for the same date [20:27:18] but I'm splitting the work up into multiple commits [20:27:43] to hopefully avoid breakage [21:06:05] (PS3) Ejegg: Audit functions can deal with multiple logs for one date [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) [21:06:07] (PS1) Ejegg: Audit: allow log_filename functions to return arrays [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435286 (https://phabricator.wikimedia.org/T195337) [21:14:02] fr-tech does that look like a reasonable direction for the multi-log audit file thing ^^^? [21:14:44] it's not exactly applicable to the other use case (looking in multiple directories) [21:14:57] but it seemed like the least hacky [21:15:59] anyway, I'll put up the final patch in the chain soon [21:16:13] with a smashpig update and activating the multipatterns for ingenico [21:16:23] that one will be the real test [21:19:23] this approach is sub-optimal in that it searches across all files for a date to get the order info, no matter if it knows that the order is definitively one gateway code or the other [21:32:08] XenoRyet: I'm working on being able to find ingenico / globalcollect donations in either payments-ingenico or payments-globalcollect log files [21:32:22] I'm splitting the changes into 3 patches [21:32:38] the first two are ready, and shouldn't break existing behavior [21:32:51] want to take a look and let me know if you thing it's the right approach? [21:33:09] Ah, also, do they automatically show up for you in gerrit? [21:33:30] I saw most of the current team was missing from our projects in https://www.mediawiki.org/wiki/Git/Reviewers [21:33:45] so I signed folks up for the missing ones yesterday [21:34:15] (sorry I didn't ask for opt-outs) [21:36:34] Yea, I'll take a look. [21:36:55] And let me check if they automatically show up. I haven't really looked there before, I typically find 'em via email or here in the chat. [21:37:48] Yea, they seem to show up under My Changes in gerrit as incoming reviews. [21:47:07] great! [22:06:23] hmm, might have to update that smashpig audit bit again when we get an answer back about the id format [22:07:30] (Abandoned) Krinkle: Remove UnitTestsList hook subscriber [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/393697 (https://phabricator.wikimedia.org/T142121) (owner: Reedy) [22:37:08] (PS1) Ejegg: WIP Ingenico audit handles both APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435293 (https://phabricator.wikimedia.org/T195337) [22:52:16] (PS1) Ejegg: WIP: Fix IDs for Ingenico audit entries [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/435294 (https://phabricator.wikimedia.org/T195337) [23:11:54] have a great weekend everyone!