[00:57:46] (CR) XenoRyet: [C: 2] Add form settings for new AstroPay methods [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/276168 (https://phabricator.wikimedia.org/T124323) (owner: Ejegg) [00:59:25] (Merged) jenkins-bot: Add form settings for new AstroPay methods [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/276168 (https://phabricator.wikimedia.org/T124323) (owner: Ejegg) [10:35:21] Fundraising-Backlog: Deploy Thank you letters to remove Lila's signature - https://phabricator.wikimedia.org/T129373#2106208 (jrobell) [10:36:23] Fundraising-Backlog: Deploy Thank you letters to remove Lila's signature - https://phabricator.wikimedia.org/T129373#2103736 (jrobell) The list has also been updated with Danish which had somehow been left out. [12:53:38] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: CentralNotice: DB timeouts when enabling more than one campaign at once from Special:CentraNotice - https://phabricator.wikimedia.org/T128869#2088740 (Pcoombe) I just ran i... [13:47:43] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: CentralNotice: DB timeouts when enabling more than one campaign at once from Special:CentraNotice - https://phabricator.wikimedia.org/T128869#2106711 (jcrespo) I think some... [15:58:33] (CR) Siebrand: [C: 1] "i18n/L10n reviewed." [extensions/FundraiserLandingPage] - https://gerrit.wikimedia.org/r/266032 (https://phabricator.wikimedia.org/T87917) (owner: MtDu) [16:01:50] (PS2) Paladox: Disable testIPTCParseForcedUTFButInvalid on PHP > 5.6.0 but < 5.6.10 [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/275035 (https://phabricator.wikimedia.org/T124574) [16:04:46] Fundraising Tech Backlog, Fundraising-Backlog, Epic: [epic] PCI gap analysis and Improvements - https://phabricator.wikimedia.org/T77912#2107286 (Jgreen) [16:06:51] fundraising-tech-ops: install redis on lutetium for prototyping an activemq replacement - https://phabricator.wikimedia.org/T129491#2107291 (Jgreen) [16:20:49] Fundraising-Backlog, fundraising-tech-ops: Provision Redis nodes on staging server - https://phabricator.wikimedia.org/T120465#2107407 (Jgreen) p:Triage>High a:Jgreen [16:21:39] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops, MediaWiki-extensions-DonationInterface: [Epic] SPOF: Replace ActiveMQ donation queues with Redis - https://phabricator.wikimedia.org/T108229#2107412 (Jgreen) [16:21:41] fundraising-tech-ops: install redis on lutetium for prototyping an activemq replacement - https://phabricator.wikimedia.org/T129491#2107410 (Jgreen) Open>Invalid duplicate ticket [16:22:24] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops, MediaWiki-extensions-ContributionTracking, MediaWiki-extensions-DonationInterface: Deploy Redis 3 to frack - https://phabricator.wikimedia.org/T120464#2107413 (Jgreen) a:Jgreen [16:39:17] fundraising-tech-ops: Please upgrade the version of R on lutetium - https://phabricator.wikimedia.org/T120878#1863856 (Jgreen) Lutetium is slated for replacement in FY2017, at which time we will upgrade from Ubuntu Precise to either Ubuntu Trusty or Debian Jessie. At that time a newer r-base release (trusty=... [16:41:21] Fundraising-Backlog, fundraising-tech-ops: Don't send us all the cron spam - https://phabricator.wikimedia.org/T119452#2107525 (Jgreen) a:awight bump. Adam does this sound like a sane fix to you? if so I will make it so and close the ticket. >>! In T119452#1847962, @Jgreen wrote: > For "cron only ma... [17:48:21] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: CentralNotice: DB timeouts when enabling more than one campaign at once from Special:CentraNotice - https://phabricator.wikimedia.org/T128869#2107931 (aaron) >>! In T128869... [17:53:31] Fundraising-Backlog, FR-Worldpay: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2108008 (cwdent) I have an email out to WP, they're looking into it. [17:53:49] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, FR-Worldpay: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2108023 (cwdent) a:cwdent [17:54:33] hi ejegg|away ppena , we still okay to start the adyen test in a few minutes? [17:54:42] I'll be keeping an eye on results [17:54:59] yes! let's do this [17:55:36] * cwd opens the logs [17:56:19] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: CentralNotice: DB timeouts when enabling more than one campaign at once from Special:CentraNotice - https://phabricator.wikimedia.org/T128869#2108032 (jcrespo) I think some... [17:56:22] hello! [17:57:50] hi ejegg! and thanks cwd for watching the logs [17:58:03] campaign is enabled, and will go up at 1800 UTC [17:58:18] the-wub, will that be in all countries? [17:58:35] MBeat: just the US [17:58:39] cool, ty [18:02:22] rolling in, no problems yet [18:02:26] okay cool, we're up. lots of clicks so far, had forgotten how fast the US traffic comes in! [18:06:34] uno failmail [18:07:08] ejegg: looks like maybe an IPN race condition? [18:07:11] hmm, we're logging declined cc as error too? [18:07:16] oh huh, will look [18:07:46] wohoo! Back to US testing! Exciting :) [18:09:58] cwd ejegg: I don't see any cc donations from the adyen banner in civi yet :( [18:10:08] results sheet here: https://docs.google.com/spreadsheets/d/17arGLTPupnrCmo7O0He38js-gmOoF3OY53sOwiAbAp0/edit#gid=0 [18:10:47] dang, i think that failmail has something to do with them not getting in [18:12:24] I'm not seeing that failmail - is it to fr-tech? think we should pull the test? [18:13:01] the-wub: if none of them are getting through, probably we should :( [18:13:35] damn [18:13:53] I saw the same condition in earlier tests and thought it would only happen in rare circumstances [18:14:03] but I guess I was wrong about what caused it [18:14:15] it seems like the capture on IPN thing would be a good candidate for a race condition [18:14:27] ejegg cwd ppena jessicarobell MBeat : okay, campaign disabled [18:14:38] cwd I think we're just leaving a queue connection open or something [18:14:58] there will probably still be people coming through for a few minutes [18:15:25] thx for quick response, all [18:15:27] thanks the-wub [18:16:11] sorry folks, I'll figure this out [18:16:12] thanks guys. no trnxs at Adyen's side... [18:16:29] wait, nothing on the adyen side either? [18:16:30] np. Let us know if its a quick fix and we can try again soon, or if we need to reschedule [18:16:58] so that must mean the capture job wasn't firing at all [18:16:59] definitely will need a code change [18:17:26] nope I dont see any news trnxs there [18:17:54] ejegg: nevermind [18:17:58] today IS the 10th [18:18:03] i do see a bunch [18:18:04] :D [18:19:16] ejegg: got 143 in [18:20:05] ok, cool! [18:20:18] I d see 4 sent for settle [18:20:29] Fyi [18:20:46] so not sure if the capture thing is THE thing? [18:20:46] Ah, so most didn't get captured.... darn [18:21:15] they are all authorized but not captured, but 4 were captured [18:21:27] I think we succeeded with the first few, then when the traffic got heavy, the mistake in how we talk to the queue jammed us up [18:21:40] maybe the last fix on the capture process? [18:21:46] i see [18:22:42] fundraising-tech-ops: Please upgrade the version of R on lutetium - https://phabricator.wikimedia.org/T120878#2108164 (mpopov) Open>Resolved a:mpopov Okie, dokie! [18:23:58] ejegg: are we going to have to make the listener poll the queue? [18:24:16] applying state to stateless protocols is hard [18:25:38] cwd maybe just close the connection after each job [18:25:52] on the plus side, we were up for only 13 minutes and got 839 donations via all methods. US must have been missing our banners ;) [18:26:02] wow, nice [18:26:03] heh, nice [19:14:00] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog: Update Square Cash import for changed column names and additional info - https://phabricator.wikimedia.org/T129174#2108505 (XenoRyet) a:XenoRyet [19:19:19] (PS1) Ejegg: Don't leave queue message dangling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/276538 [19:19:32] It's just possible that this is all we need ^^^ [19:25:37] ejegg: is there a task associated with that commit? [19:26:37] I'm just getting in and catching up. Do we need a task for an adyen bug? [19:27:06] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work: Update Square Cash import for changed column names and additional info - https://phabricator.wikimedia.org/T129174#2108548 (DStrine) [19:28:36] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Worldpay: Worldpay refunds not reaching Civi - https://phabricator.wikimedia.org/T129265#2108553 (DStrine) [19:30:19] nope dstrine,but i'll add one [19:30:34] thanks! [19:31:10] ejegg: and cwd|afk I see that slowly Adyen trnxs are getting captures... hopefully is you guys fxing it ;)? [19:33:49] ppena: so, the broken job can capture a few every five minutes [19:34:03] ohhh I see [19:34:11] it just gets clogged up quickly [19:34:29] so those will eventually get processed, even without a fix. Good to know [19:35:46] yeah, we'll just get another fail mail every 5 minutes when it clogs :( [19:46:27] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work: Update Square Cash import for changed column names and additional info - https://phabricator.wikimedia.org/T129174#2108727 (XenoRyet) Judging by the attached example report, looks like they changed things around... [19:53:27] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Adyen, FR-Smashpig: Adyen job runner failing at pending queue - https://phabricator.wikimedia.org/T129547#2108778 (Ejegg) [20:09:31] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Astropay: Remove fiscal number for Mexico form - https://phabricator.wikimedia.org/T129552#2108935 (atgo) [20:45:26] ejegg: i was talking to thcipriani a bit about the adyen thing [20:46:37] what if the listener queues another message instead of trying to capture [20:47:39] and then when either happens (IPN message or pending message) it can check if both are there and if so try to capture [20:54:43] cwd that's basically what happens [20:55:04] ipn message comes in -> listener queues a capture request job [20:56:14] job runner gets to capture request job, job looks at pending msg and avs / cvv results, either requests capture and leaves pending message in queue, or rejects and deletes pending message [20:56:34] then another ipn message comes in telling us we got a successful capture [20:56:44] and we queue a 'record capture job' [20:56:55] ...which may be overkill. [20:57:37] oh i must be confused about the actual nature of the failures then [20:57:54] i thought it was about the IPN message arriving before the pending job was there [20:57:54] anyway, the next job runner cycle picks up the record capture job, it finds the pending message again, and sends it to civi [20:58:30] cwd no, i think it's because we're using the same queue subscription for multiple attempts to read the pending queue [20:58:51] and when we leave something on the queue, I was just not acknowledging the message [20:59:10] if you disconnect after that, it's fine, it stays on the queue. [20:59:40] but if you stay connected, and try to read another message from the same queue, I think that's what gives the error message we were seeing [21:00:08] so that little patch calls a helper function that acknowledges the message, but immediately re-queues it. [21:00:43] anyway, we don't really want a queue at all for the pending stuff, we want a key-value store [21:00:57] yet another reason to get redis in there [21:01:35] yeah [21:01:38] ok that makes sense [21:01:40] thanks [21:02:25] every other place we use the php stomp lib, i think we disconnect and reconnect with every read attempt [21:02:56] so I had never really needed to worry about the actual message queue behaviors like acking and such [21:03:27] yeah...seems like unneccessary overhead [21:04:32] eileen1: standup? [21:08:16] XenoRyet: just this one left https://gerrit.wikimedia.org/r/276237 [21:09:01] 10-4 [21:17:07] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Astropay: Remove fiscal number for Mexico form - https://phabricator.wikimedia.org/T129552#2108935 (Ejegg) a:Ejegg [21:26:30] (PS1) Ejegg: Fix duplicate class name [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/276583 [21:32:21] (PS1) Ejegg: Don't ask for fiscal number in Mexico or Peru [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/276610 (https://phabricator.wikimedia.org/T129552) [21:33:02] AndyRussG or XenoRyet: ^^ should be all we need to get the tax number off the Mexico form [21:33:18] cool! [21:37:30] ejegg: it seems like the smashpig data store stuff is mean to be able to switch out the back end but...it also looks pretty specific to activemq. do you think we'll want to build something different for redis? [21:38:36] cwd yeah, i guess we want a data store with less queue-specific methods [21:39:24] probably don't need much besides the standard lib if i was to guess [21:44:42] (CR) XenoRyet: [C: 2] Add more credit cards (for Chile) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276237 (https://phabricator.wikimedia.org/T124323) (owner: Ejegg) [21:54:08] (CR) XenoRyet: [C: 2] Don't ask for fiscal number in Mexico or Peru [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/276610 (https://phabricator.wikimedia.org/T129552) (owner: Ejegg) [21:57:03] Thanks XenoRyet ! I'll deploy as soon as those merges finish [21:57:16] Good times [22:01:56] (Merged) jenkins-bot: Add more credit cards (for Chile) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276237 (https://phabricator.wikimedia.org/T124323) (owner: Ejegg) [22:01:58] (Merged) jenkins-bot: Don't ask for fiscal number in Mexico or Peru [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/276610 (https://phabricator.wikimedia.org/T129552) (owner: Ejegg) [22:09:46] (PS1) Ejegg: Merge master into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/276624 [22:10:20] (CR) Ejegg: [C: 2 V: 2] Merge master into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/276624 (owner: Ejegg) [22:11:18] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/276625 [22:11:37] (CR) Ejegg: [C: 2 V: 2] Update DonationInterface submodule [core] (fundraising/REL1_25) - https://gerrit.wikimedia.org/r/276625 (owner: Ejegg) [22:17:22] !log updated payments wiki from 07dcb0f3962143fd9497ccad19b7b682beb991fe to 7248c10613018c1a15a1754ab80242f79d04532f [22:17:25] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [22:19:41] OK, looks like that getBlahPage refactor didn't break stuff [22:23:31] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Astropay, Patch-For-Review: Remove fiscal number for Mexico form - https://phabricator.wikimedia.org/T129552#2108935 (Ejegg) Open>Resolved [22:32:16] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Unplanned-Sprint-Work: CentralNotice: DB timeouts when enabling more than one campaign at once from Special:CentraNotice - https://phabricator.wikimedia.org/T128869#2109760 (AndyRussG) Indeed 6 s... [22:33:28] AndyRussG: remember the other day when you were having that problem because vagrant fetches resources from the script patch at /var/www/w instead of the checkout where the actual files live? did you ever come up with a good solution? [22:33:49] cwd: only a solution, but not a good one [22:33:56] A solution for vagrant [22:34:47] Which is go to DonationInterface repo on under /vagrant/mediawiki (rather than /vagrant/mediawiki-fr) and put it on the same commit as the one you're actually working with under /vagrant/mediawiki-fr [22:35:11] I guess it doesn't even make the grade for "solution" [22:35:32] r u working under vagrant? [22:35:52] yep! trying to get into it [22:35:58] it's great for some stuff [22:36:13] cool! [22:36:17] i don't get why /var/www/w is a quasi-separate checkout [22:36:29] ah, because it's not a git repo [22:36:49] cwd: I guess the real solution would be to fix the web server routing in the vagrant role [22:37:06] So that static resources for payments wiki come from the right dir ;p [22:37:08] yeah...maybe i'll poke at that [22:37:14] coolio! [22:37:18] thanks! [22:37:25] np, likewise! [23:09:13] AndyRussG: the more i look, the less i know...where the heck is the vhost config for payments.local.wmftest.net? [23:10:11] i feel like i'm missing something basic here [23:11:52] (PS1) Siebrand: Call static Language::fetchLanguageName() statically [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/276641 [23:16:03] cwd: yeah I don't know either. Never messsed w/ web server config for vagrant [23:16:18] maybe grep about in puppet files for the fundraising role? [23:17:23] no mention of "mediawiki-fr" anywhere in /etc/apache2 [23:17:29] but it's clearly working [23:23:29] Hmmm [23:23:35] Yeahthere's definitely some magic [23:23:54] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/276651 [23:25:34] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/276651 (owner: Ejegg) [23:27:42] !log updated civicrm from cbcfafcb2e6d6e1dae12a2b2d554445871992aff to 7e21d5ad1f9ff404ac155a38f771b744bf238ccf [23:27:46] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [23:50:13] cwd I'm pretty sure this one won't hurt, in any case: https://gerrit.wikimedia.org/r/276538 [23:50:24] And I'm kind of curious to try it out [23:51:01] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [epic] Make deduping reversible - https://phabricator.wikimedia.org/T111704#2110183 (Eileenmcnaughton) [23:51:03] Fundraising Sprint Dirt Farming, Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog: Make dedupe reversible step 2 of 5: Identify and Address specific points of slowness i - https://phabricator.wikimedia.org/T127135#2110181 (Eileenmcnaughton) Open>Resolved I've converted a bunch o... [23:51:29] ah yeah i checked that out earlier but forgot to approve [23:51:42] shouldn't be necessary of course...more ammo against activemq [23:51:52] (CR) Cdentinger: [C: 2] Don't leave queue message dangling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/276538 (owner: Ejegg) [23:53:16] thanks!