[01:11:15] <eileen>	 anyone looking at the failmail storm?
[01:11:41] <eileen>	 oh - looks like a hacking attempt
[01:11:43] <eileen>	 <SCRIPT>prompt(299792458);</SCRIPT>
[01:13:51] <eileen>	 mepps: not sure if you are about but is there something we can / should do if someone is trying to hack
[01:14:56] <eileen>	 ejegg|away: ^^ cstone dstrine anyone about?
[01:15:03] <mepps>	 Yikes Eileen! Contact security?
[01:15:16] <eileen>	 yeah I dunno - I mean I don't think they are getting in
[01:15:18] <mepps>	 Also Jeff_Green
[01:15:19] <eileen>	 but they are trying
[01:15:21] <mepps>	 Oh ok
[01:15:29] <eileen>	 If you look at all that failmail
[01:15:37] <eileen>	 AndyRussG:
[01:15:47] <eileen>	 they are entering js into the payment_method box
[01:16:01] <eileen>	 I assume it isn't doing anything - they just keep trying & we could block their IP?
[01:16:43] <mepps>	 I can’t see it on my phone
[01:17:12] <mepps>	 Yeah blocking their ip is a good idea
[01:17:23] <eileen>	 mepps: do you know how to?
[01:17:34] <mepps>	 I do not Eileen
[01:17:43] <mepps>	 It must be in local settings
[01:19:15] <eileen>	 ok - do you want to text Jeff? I can but it's international
[01:19:18] <mepps>	 There’s documentation on wiki of how to shut stuff down if we need to
[01:19:36] <mepps>	 Yeah I’m trying to remember where the contact sheet is
[01:19:43] <eileen>	 https://office.wikimedia.org/wiki/Contact_list
[01:21:20] <mepps>	 ThNks! It’s tough on my phone
[01:22:51] <eileen>	 they might have just stopped - I was . on the verge of turning off all the queues
[01:23:04] <eileen>	 maybe I did turn them off
[01:23:10] <eileen>	 will clear emails & turn on & see
[01:23:23] <mepps>	 I texted him
[01:24:01] <mepps>	 grateful for having you in NZ right now nonetheless
[01:25:14] <eileen>	 lol
[01:25:39] <eileen>	 well queues are off now - I'll see if he responds before trying to turn them back on
[01:25:47] <mepps>	 Cool thanks
[01:26:12] <mepps>	 dstrine are you around by any chance?
[01:26:34] <mepps>	 Or MBeat?
[01:26:44] <MBeat>	 hi mepps
[01:27:00] <MBeat>	 big fraud attack!
[01:27:14] <mepps>	 Yeah
[01:27:31] <mepps>	 Just wanted to make sure you saw queues are off
[01:27:32] <MBeat>	 https://collab.wikimedia.org/wiki/Fundraising/Engineering/Shutting_the_pipeline_down#Choosing_how_to_shut_down
[01:27:38] <MBeat>	 thank you
[01:28:03] <MBeat>	 I looked for all the activity on the IP in fredge, doesn’t look like anything got through as fraud
[01:28:04] <mepps>	 Hmm Eileen do we know what processor this was?
[01:28:10] <mepps>	 Oh good
[01:28:13] <eileen>	 seems to be ingenico
[01:28:41] <mepps>	 Okay if it doesn’t get through as fraud are we okay to leave ingenico up as far as you’re concerned MBeat?
[01:29:15] <MBeat>	 yes! we just sent out a batch of Japan emails, so I’m happy to monitor & not block the JP donors - thank you for asking
[01:29:27] <mepps>	 Eileen I might text dstrine too, seems like something he’d want to know
[01:29:49] <eileen>	 ok - I'm just creating a phab
[01:30:55] <eileen>	 not much detail yet but https://phabricator.wikimedia.org/T228822
[01:32:05] <mepps>	 Great, thanks for being on this
[01:32:33] <mepps>	 It’s hard from my phone! I can get my computer if needed though
[01:32:38] <eileen>	 I'm not too sure whether they stopped or whether my stopping the . queues happened first
[01:33:29] <eileen>	 SO I guess as long as they are all from one IP then we just have someone hammering away - or scripting away at our form
[01:33:40] <mepps>	 I haven’t heard from Jeff
[01:33:53] <mepps>	 Yes that makes sense
[01:34:27] <mepps>	 I think if we block that ip, we could turn the queues on and see what happens
[01:35:55] <eileen>	 yeah  -  I guess I'll go trawling for how to block an ip
[01:37:14] <mepps>	 I was just searching collab
[01:37:37] <eileen>	 hmm the amount of donations queued via Ingenico seems static at 20 https://frmon.frdev.wikimedia.org/d/Pq1YNMviz/fundraising-overview?refresh=1m&orgId=1&panelId=15&fullscreen
[01:37:44] <eileen>	 it might have stopped on it's own
[01:38:49] <mepps>	 Do you want to turn them on? I can join on watching the failmail
[01:40:10] <eileen>	 yeah I'll do it  now
[01:41:47] <eileen>	 ah ejegg|away was talking to us on the other channel & I missed it
[01:41:58] <eileen>	 anyway we are back on
[01:42:09] <icinga-wm>	 PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: payments-antifraud is 2152 2000 - REDIS 3.2.6 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 5 days 6 minutes - memory use is 3.40M (peak 3.63M, 0.08% of max, fragmentation 1.93%), connected_slaves is 2, donations is 224, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-init is 181, pending is 0, recurring is 2, refund is 0, unsubscribe is 2
[01:42:23] <MBeat>	 ty eileen
[01:42:58] <mepps>	 Oh ha I missed it too
[01:44:18] <mepps>	 My biggest learning from this is that the wiki office contact list is not super phone accessible and that we need better documentation of how to block ips
[01:47:14] <icinga-wm>	 RECOVERY - check_redis on frqueue1001 is OK: OK: REDIS 3.2.6 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 5 days 11 minutes - memory use is 1.90M (peak 3.63M, 0.06% of max, fragmentation 2.55%), connected_slaves is 2, donations is 12, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 73, payments-init is 12, pending is 0, recurring is 3, refund is 0, unsubscribe is 2
[01:50:48] <eileen>	 mepps: yeah - we should probably just all add each other's numbers to our phones
[01:51:11] <eileen>	 also I am ASSUMING that the hack attempts were futile & nothing useful could be achieved by it
[01:51:18] <mepps>	 Agreed. It turns out I had Jeff’s number but forgot I’d added it
[01:51:43] <eileen>	 I guess how it COULD have worked is IF the page we used to view them didn't escape them we could have run malicious script
[02:01:03] <eileen>	 !log payments-wiki revision changed from 224c6b2d7b to a28ad541ed, config revision is 8dcb77cf22
[02:01:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[08:37:58] <wikibugs>	 Fundraising Sprint Men In Slack, Fundraising Sprint Never Ending Query, Fundraising Sprint Office &nbsp;, Fundraising-Backlog, and 4 others: New total fields needed for MG prospecting - https://phabricator.wikimedia.org/T222958 (Eileenmcnaughton) Open→Resolved
[08:38:03] <wikibugs>	 Fundraising Sprint King Kong vs. Mozilla, Fundraising Sprint Land before Timeouts, Fundraising Sprint Men In Slack, Fundraising Sprint Never Ending Query, and 3 others: Add calendar year totals fields to CiviCRM & export ? some? of them to silverpop - https://phabricator.wikimedia.org/T170972 (Eil...
[09:23:43] <wikibugs>	 Fundraising-Backlog: Update email copy for the endowment Thank you email - https://phabricator.wikimedia.org/T228843 (jrobell)
[15:11:07] <ejegg>	 fr-tech any news for scrum of scrums?
[15:14:38] <jgleeson>	 non from me
[15:45:31] <wikibugs>	 (PS2) Ejegg: WIP deal with gateway_txn_id in a standard way [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524535
[15:47:27] <wikibugs>	 (CR) jerkins-bot: [V: -1] WIP deal with gateway_txn_id in a standard way [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524535 (owner: Ejegg)
[15:48:41] <ejegg>	 almost!
[16:04:06] <wikibugs>	 (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/525314
[16:04:43] <ejegg>	 cstone: did you see the failmail about the audit file failing? Might have something to do with the new India payment methods
[16:04:52] <ejegg>	 I haven't taken a look yet
[16:06:16] <wikibugs>	 (CR) Ejegg: [C: +2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/525314 (owner: Ejegg)
[16:07:32] <wikibugs>	 (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/525314 (owner: Ejegg)
[16:08:34] <wikibugs>	 (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/525315
[16:08:43] <wikibugs>	 (CR) Ejegg: [C: +2] Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/525315 (owner: Ejegg)
[16:10:36] <wikibugs>	 (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/525315 (owner: Ejegg)
[16:13:32] <wikibugs>	 Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), Performance-Team (Radar): Remove hacky EventLogging-duplicating code and use new lightweight EL facilities - https://phabricator.wikimedia.org/T214709 (Krinkle)
[16:13:42] <wikibugs>	 Fundraising-Backlog, MediaWiki-extensions-CentralNotice, MW-1.34-notes (1.34.0-wmf.7; 2019-05-28), Performance-Team (Radar): Remove hacky EventLogging-duplicating code and use new lightweight EL facilities - https://phabricator.wikimedia.org/T214709 (Krinkle) p:Triage→Normal
[16:17:30] <wikibugs>	 (CR) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/525315 (owner: Ejegg)
[16:21:11] <wikibugs>	 (PS3) Ejegg: Banner history logger: remove loading of schema module [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/523853 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG)
[16:21:20] <wikibugs>	 (CR) Ejegg: [C: +2] "Looks good!" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/523853 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG)
[16:25:25] <wikibugs>	 (Merged) jenkins-bot: Banner history logger: remove loading of schema module [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/523853 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG)
[16:31:20] <wikibugs>	 (CR) jenkins-bot: Banner history logger: remove loading of schema module [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/523853 (https://phabricator.wikimedia.org/T227711) (owner: AndyRussG)
[16:40:48] <wikibugs>	 (CR) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_31) - https://gerrit.wikimedia.org/r/525315 (owner: Ejegg)
[16:58:27] <wikibugs>	 (CR) Cstone: WIP: Add recurring token support to creating a subscription. (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524909 (https://phabricator.wikimedia.org/T227048) (owner: Cstone)
[17:05:13] <wikibugs>	 Fundraising-Backlog: Portuguese TY email: matching gift link error - https://phabricator.wikimedia.org/T209506 (TSkaff) I just sent myself test emails in pt and pt-br and the format looks ok and the link functions.  But I wonder whether:  [$matchingUrl programa de doações deste género]  should be  $matchingU...
[17:06:47] <wikibugs>	 (PS3) Ejegg: Add gateway_txn_id and subscr_id to subscr_start message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524538 (https://phabricator.wikimedia.org/T216560)
[17:06:58] <ejegg>	 cstone ^^^ now adds the subscr_id
[18:15:16] <ejegg>	 !log updated payments-wiki from a28ad541ed to 70b432d309
[18:15:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[18:22:17] <ejegg>	 ok, looks like new-API card donations still go through
[18:23:14] <ejegg>	 and old-API ones don't get a hosted payment page
[18:23:16] <ejegg>	 cool
[18:33:34] <cstone>	 ah ejegg one last thing for the installments, do you want to send any of that in the msg or should I just set it to 0 in the insert?
[18:46:02] <ejegg>	 cstone: you can set it to 0 in the insert
[18:46:10] <cstone>	 ok thanks!
[18:46:35] <ejegg>	 hmm, looks like XenoRyet's crm/vendor submodule update didn't actually include the SmashPig update
[18:46:49] <ejegg>	 that's why the last dlocal audit parse run failed
[18:46:58] <ejegg>	 lemme try to set that straight
[18:48:43] <wikibugs>	 Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: TY email not sent for recurring donation 7/22 - https://phabricator.wikimedia.org/T228922 (MBeat33)
[18:49:56] <ejegg>	 hmm, what's the matter here
[18:49:58] <ejegg>	 ?
[18:52:32] <wikibugs>	 Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: TY email not sent for recurring donation 7/22 - https://phabricator.wikimedia.org/T228922 (MBeat33) p:Triage→High Hoo boy, I see hundreds of recurring donations from the 22nd without TY email send time stamps.
[18:52:48] <ejegg>	 the packagist page shows sha 3dbd
[18:53:07] <ejegg>	 the repo shows tag v0.5.10 as commit 3dbd
[18:53:27] <ejegg>	 but composer install keeps cloning 5a056c7, which is v0.5.9
[18:53:32] <wikibugs>	 Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: TY emails not sent for recurring donation 7/22 - https://phabricator.wikimedia.org/T228922 (MBeat33)
[18:54:36] <ejegg>	 ahh, composer.lock issue?
[18:55:48] <ejegg>	 yep, somehow the sha change didn't get checked in
[18:57:43] <wikibugs>	 (PS1) Ejegg: Fix SmashPig version [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/525342
[19:04:04] <jgleeson>	 ejegg, were previous recurring donations always the same amount as the initial contribution?
[19:04:20] <ejegg>	 jgleeson: oh yeah, that too!
[19:04:40] <jgleeson>	 cool, I wanted to make sure before adding comments
[19:10:30] <wikibugs>	 Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: TY emails not sent for recurring donation 7/22 - https://phabricator.wikimedia.org/T228922 (MBeat33) Open→Resolved a:MBeat33 Cleared up after checking w/fr-tech, this is from the rollout of the new recurring TY process. Sorry for the a...
[19:16:34] <jgleeson>	 ejegg, are we using the effort_id as a counter for installments also?
[19:16:53] <jgleeson>	 in the queue context
[19:17:15] <ejegg>	 jgleeson: yeah, that's kind of a holdover from the old globalcollect
[19:17:23] <jgleeson>	 got it
[19:24:41] <wikibugs>	 (CR) Ejegg: [C: +2] Fix SmashPig version [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/525342 (owner: Ejegg)
[19:25:14] <wikibugs>	 (PS1) Ejegg: Fix SmashPig update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/525345
[19:25:27] <wikibugs>	 (PS4) Jgleeson: Update SmashPig recurring processor to handle making first payments. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/525098 (https://phabricator.wikimedia.org/T227051)
[19:25:37] <wikibugs>	 (CR) jerkins-bot: [V: -1] Fix SmashPig update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/525345 (owner: Ejegg)
[19:29:39] <wikibugs>	 (CR) Cstone: [C: +2] "Sending through the new parameters nicely!" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524538 (https://phabricator.wikimedia.org/T216560) (owner: Ejegg)
[19:29:52] <wikibugs>	 (Merged) jenkins-bot: Fix SmashPig version [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/525342 (owner: Ejegg)
[19:30:24] <wikibugs>	 (PS2) Cstone: WIP: Add recurring token support to creating a subscription. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/524909 (https://phabricator.wikimedia.org/T227048)
[19:30:42] <wikibugs>	 (Merged) jenkins-bot: Add gateway_txn_id and subscr_id to subscr_start message [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/524538 (https://phabricator.wikimedia.org/T216560) (owner: Ejegg)
[19:30:59] <wikibugs>	 (PS2) Ejegg: Fix SmashPig update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/525345
[19:34:38] <wikibugs>	 (CR) Ejegg: [C: +2] Fix SmashPig update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/525345 (owner: Ejegg)
[19:35:18] <wikibugs>	 (Merged) jenkins-bot: Fix SmashPig update [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/525345 (owner: Ejegg)
[19:37:13] <wikibugs>	 (PS1) Ejegg: Update sha for smash-pig in composer.lock [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/525350
[19:37:23] <wikibugs>	 (CR) Ejegg: [C: +2] Update sha for smash-pig in composer.lock [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/525350 (owner: Ejegg)
[19:42:29] <wikibugs>	 (Merged) jenkins-bot: Update sha for smash-pig in composer.lock [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/525350 (owner: Ejegg)
[19:43:08] <ejegg>	 !log updated fundraising CiviCRM from 875ab97742 to 121feb5d53
[19:43:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[19:45:55] <ejegg>	 OK, that should make the dlocal audit process correctly today
[19:48:07] <ejegg>	 fr-tech I've got to go catch a bus before they close the roads! back online in a few hours
[20:46:43] <wikibugs>	 (CR) jenkins-bot: Localisation updates from https://translatewiki.net. [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/525378 (owner: L10n-bot)
[23:31:28] <wikibugs>	 Fundraising Sprint Never Ending Query, Fundraising Sprint Office &nbsp;, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (LeanneS) This is great, thanks @Eileenmcnaughton! Yeah they do have some extra/unneeded columns i...
[23:32:55] <wikibugs>	 Fundraising Sprint Never Ending Query, Fundraising Sprint Office &nbsp;, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (Eileenmcnaughton) @LeanneS the 'full_address' is just street_address+supplemental_address+ supple...
[23:36:20] <wikibugs>	 Fundraising Sprint Never Ending Query, Fundraising Sprint Office &nbsp;, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Finalise & test field mapping - https://phabricator.wikimedia.org/T228712 (LeanneS) @Eileenmcnaughton No, for some reason, the street address is called that in their file....