[00:30:48] Fundraising-Backlog, Epic: [epic] Handle gateway (sub) accounts throughout pipeline - https://phabricator.wikimedia.org/T102701#2482637 (awight) A recent look at this problem. Code still creating contributions without a gateway account within the past month are: ``` select e.source_name, count(*) from... [00:31:59] Fundraising-Backlog, Epic: [epic] Handle gateway (sub) accounts throughout pipeline - https://phabricator.wikimedia.org/T102701#2482639 (awight) [00:32:01] Fundraising-Backlog, FR-PayPal-ExpressCheckout, MediaWiki-extensions-DonationInterface, Technical-Debt: account_config vs accountInfo vs account_name is a mess - https://phabricator.wikimedia.org/T130154#2482638 (awight) [00:58:12] (PS2) Awight: [WIP] Convert orphan rectifier to use the PendingDatabase [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300173 (https://phabricator.wikimedia.org/T131275) [00:59:11] spatton: The logs are looking very positive. I'm gonna take off in a minute, congratulations on the test! [01:00:24] (CR) jenkins-bot: [V: -1] [WIP] Convert orphan rectifier to use the PendingDatabase [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300173 (https://phabricator.wikimedia.org/T131275) (owner: Awight) [01:13:29] awight: Thanks much for monitoring with me! I appreciate it. [01:14:39] np! [01:16:42] (PS1) Awight: Conditionally include the home directory in config search [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/300208 [01:18:31] (PS3) Awight: [WIP] Convert orphan rectifier to use the PendingDatabase [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300173 (https://phabricator.wikimedia.org/T131275) [01:20:10] (CR) jenkins-bot: [V: -1] [WIP] Convert orphan rectifier to use the PendingDatabase [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300173 (https://phabricator.wikimedia.org/T131275) (owner: Awight) [01:31:11] For anyone watching at home, the Japan pre-test just ended. I'll email results after giving Adyen a few minutes to catch up. [03:29:38] fundraising-tech-ops, Operations, ops-eqiad: decommission aluminium, replace it with frqueue1002 - https://phabricator.wikimedia.org/T140676#2472440 (Dzahn) already removed from DNS in July 2015 and don't see anything in puppet either. --- commit 4c46ff39f1071816d8ed865d93d66daf3b3fc929 Author: jgr... [03:31:16] fundraising-tech-ops, Operations, ops-eqiad: decommission aluminium, replace it with frqueue1002 - https://phabricator.wikimedia.org/T140676#2482722 (Dzahn) only mgmt dns is left, since cables have been removed.. we can remove that too [03:32:47] fundraising-tech-ops, Operations, ops-eqiad: decommission aluminium, replace it with frqueue1002 - https://phabricator.wikimedia.org/T140676#2482723 (Dzahn) oh wait, you mean "aluminium.**frack.**eqiad.wmnet" (too) right [15:12:39] fundraising-tech-ops, Operations, netops: Cleanup layer2 firewall config from pfw-eqiad - https://phabricator.wikimedia.org/T111463#2483519 (Jgreen) [15:15:25] fundraising-tech-ops, Operations, netops: Cleanup layer2 firewall config from pfw-eqiad - https://phabricator.wikimedia.org/T111463#2483531 (Jgreen) p:Low>High bumping to high because this blocks adding pfw ports, which in turn blocks hardware refreshes [15:18:54] Fundraising Tech Backlog, Gerrit: Delete ContactPageFundraiser extension - https://phabricator.wikimedia.org/T123976#1942284 (demon) Marked as hidden in Gerrit, archived in Phab. If someone wants to clean up Github (usually deleting) feel free. [15:20:02] fundraising-tech-ops, Operations, netops, ops-eqiad: put pfw1- ge-2/0/11 in the 'fundraising' vlan for new host frqueue1001 - https://phabricator.wikimedia.org/T140991#2483556 (Jgreen) [15:22:18] fundraising-tech-ops: build two queue servers to replace silicon - https://phabricator.wikimedia.org/T137150#2483583 (Jgreen) [15:22:20] Fundraising Sprint Muggle Baiting, Fundraising Sprint Nitpicking, Fundraising-Backlog, fundraising-tech-ops, and 2 others: Provision Redis cluster for Fundraising - https://phabricator.wikimedia.org/T130283#2483584 (Jgreen) [15:22:22] fundraising-tech-ops, Operations, netops, ops-eqiad: put pfw1- ge-2/0/11 in the 'fundraising' vlan for new host frqueue1001 - https://phabricator.wikimedia.org/T140991#2483582 (Jgreen) [15:27:45] Fundraising Tech Backlog, Gerrit: Delete ContactPageFundraiser extension - https://phabricator.wikimedia.org/T123976#2483609 (demon) Open>Resolved a:demon Done in Github too. [15:56:16] Fundraising-Backlog, fundraising-tech-ops, Operations, Patch-For-Review: Allow Fundraising to A/B test wikipedia.org as send domain - https://phabricator.wikimedia.org/T135410#2483717 (CCogdill_WMF) After pushing IBM for a couple weeks, they finally sent us this response today: “After reviewing... [16:45:48] FRiends! I realized I won't be able to make it for the schema migration today... just thought I'd mention it. [16:46:36] over and out... [16:46:49] ;) [16:53:58] ooh i forgot about that [16:55:43] me too ;) [16:55:56] Luckily, getting the code out first makes this like zero risk. [16:58:04] yeah [17:02:55] have fun :D [17:02:57] gtg [17:15:53] Fundraising Sprint Muggle Baiting, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Patch-For-Review: Adding Gift Source option for Engage users - https://phabricator.wikimedia.org/T137569#2484240 (LeanneS) @Eileenmcnaughton they confirmed they are able to select Benefactor Gift as an option n... [17:37:15] dang, managed to get a null pointer dereference in php [17:45:57] heh [17:58:36] reloading vagrant fixed it [17:58:40] that was frustrating [18:00:38] (PS1) Ejegg: Merge master into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/300320 [18:07:45] (CR) Ejegg: [C: 2] Merge master into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/300320 (owner: Ejegg) [18:08:27] (Merged) jenkins-bot: Merge master into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/300320 (owner: Ejegg) [18:12:19] fr-tech: I'm going to deploy ^^ shortly [18:13:40] ejegg: is it risky? [18:13:47] also are we doing that schema migration today? [18:14:58] cwd it could potentially do nasty things if we've got false positives on fraud [18:15:45] yeah, looking up the window we asked for [18:19:59] ah yeah, 3-4 pm cwd [18:21:11] EST? [18:25:46] (PS1) Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/300324 [18:26:08] (CR) Ejegg: [C: 2] Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/300324 (owner: Ejegg) [18:28:10] (PS4) Ejegg: Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 [18:29:53] (Merged) jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/300324 (owner: Ejegg) [18:29:59] (CR) jenkins-bot: [V: -1] Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 (owner: Ejegg) [18:34:15] !log updated payments-wiki from f23f15656eb488f5008b45b940077abbaa779004 to 79d2b67067fd7e579372b63e0d619eccfa3b9143 [18:34:19] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [19:03:34] Jeff_Green, fr-tech: payments maintenance window is starting - shall I take down campaigns? [19:03:57] the-wub ^^ [19:03:59] sure [19:04:02] will do [19:04:28] oh yeah, go ahead ejegg. I had actually forgotten about that [19:07:25] !log disabled fundraising CentralNotice campaigns for paymentswiki schema update [19:07:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [19:08:20] ok, let's wait for traffic to subside [19:08:59] ejegg: while we're waiting we could push out the read/write config [19:09:35] are we still doing the payments1001-first plan? [19:10:06] won't the others replicate from there? [19:10:14] the mysql that is [19:10:23] yes, unless we stop replication [19:11:04] Jeff_Green: yep, I think that's the way to go [19:11:06] so don't we only have to do #1? [19:11:46] what's the upgrade notes link again? [19:12:16] i don't think i saw that [19:12:32] where is the #1 you're referring to? :-P [19:13:03] https://etherpad.wikimedia.org/p/Paymentswiki_upgrade [19:13:23] thx [19:14:04] hey ejegg, you accidentally disabled a couple of non-fundraising campaigns. I put them back up [19:14:17] the-wub: oops, thank you! [19:14:59] no problem. ours all start with C1516 or C1617 [19:22:27] (PS5) Ejegg: Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 [20:02:25] (PS31) AndyRussG: Migrate to extension registration [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/186899 (https://phabricator.wikimedia.org/T108721) (owner: Awight) [20:22:24] Fundraising-Backlog: Temporarily use Ingenico for Latin America credit card - https://phabricator.wikimedia.org/T141040#2485038 (Pcoombe) [20:28:05] !log re-enabled fundraising campaigns after schema update [20:28:10] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [20:30:50] fundraising-tech-ops: build two queue servers to replace silicon - https://phabricator.wikimedia.org/T137150#2485072 (Jgreen) don't forget that stock redis looks like it may need some tuning, notably around maxclients vs ulimits [20:39:20] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485123 (Ejegg) [20:43:44] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485156 (Ejegg) @Pcoombe, I think we need to change incoming links to ensure that payment_method is 'cc' and recurring is true, rather than havi... [20:50:04] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485190 (Ejegg) Definitely something off with that banner and recurring - I tried clicking recurring and $3, then credit card. Got one js error... [20:59:01] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485254 (Ejegg) OK, I have no idea how anyone saw that banner - it's not part of any active campaigns, and the utm_campaign value is C13_wpdr_en... [21:02:18] (PS1) Ejegg: Add order_id to sparse donation messages [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/300417 (https://phabricator.wikimedia.org/T140484) [21:05:12] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485267 (Pcoombe) Yeah, I was going to say: that banner is of 2013 vintage! We don't send rcc as a payment method anymore. [21:26:04] How was the migration? [21:26:28] Looks like it went well, from the server admin log... [21:28:08] afaik everything is fine [21:29:06] nice. Too bad we can't unhitch from php5.3 yet, though... [21:29:18] cos of Precise machines [21:35:26] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: GlobalCollect recurring failing for at least some donor - https://phabricator.wikimedia.org/T141043#2485432 (Ejegg) Open>Resolved a:Ejegg referrer is null, and the last of that campaign I see in bannerimpressions was 6/7/2015. Web... [21:36:01] awight: machines with DI? [21:36:14] awight: speaking of which, I added php53lint in to the change that removes REL1_25 tests [21:36:30] cwd crm machine [21:36:38] doing recurring payments [21:37:13] https://gerrit.wikimedia.org/r/299940 << CI change [21:38:47] aaah right [21:55:07] (PS1) Cdentinger: WIP: move PendingQueue into a library [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300429 [21:56:54] (CR) jenkins-bot: [V: -1] WIP: move PendingQueue into a library [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300429 (owner: Cdentinger) [21:58:48] cwd trying to write up all the steps we need for end-to-end testing here: https://etherpad.wikimedia.org/p/Fundraising_End_To_End [22:00:42] SICK [22:00:48] I'll be there in a minute [22:05:28] excellent [22:07:56] most of my day involved fscking with the composer autoloader but i think decoupling the pending queue from everything else will be pretty easy and clean, if we choose to go that route [22:18:12] Fundraising-Backlog: Strine's second spacer task ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - https://phabricator.wikimedia.org/T141059#2485605 (DStrine) [22:18:47] cwd: sounds amazing. Can I see? [22:19:43] awight: https://gerrit.wikimedia.org/r/300429 and https://github.com/caseydentinger/PendingQueue is all i have so far [22:19:49] pls eviscerate [22:21:05] mmm, entrails... [22:21:29] dang. how do I comment... [22:22:50] awight: oh right...how come we forked it? [22:23:14] Cos we added features faster than the maintainer could merge them ;) also some of the stuff I did is dubious. [22:23:23] There's are some tasks in the overhaul for unforking, [22:23:28] removing the keyvalue stuff especially. [22:23:31] and upstreaming. [22:23:55] needs README [22:25:19] yeah unforking would be the jam if we can do it [22:25:41] Needs schema, eh? [22:26:11] ah yeah, i only did the redis side today [22:26:17] I think we can. The author is easy to get along with :) [22:27:07] You depend on SmashPig classes but don't require it in composer.json [22:27:53] awight: sorry yeah, didn't even touch that file, probably shouldn't have copied it over [22:27:56] Are you planning to cut the classes out, or to make that dependency? Either is valid, o.c... [22:28:09] ok I'll stop CR'ing that, right [22:28:21] i think we can cut those classes out. custom date and exception, nbd [22:29:00] damn. how do I comment on code in GH without doing a blame and following to last patch, which is usually incorrect anyway [22:32:21] ugh, does it need to be a PR? [22:32:24] github is dumb [22:32:49] Gah. [22:33:08] How about I just send you a patch with comments? [22:33:41] This is awesome that you're making an attempt at severing from the other modules btw [22:35:29] needs a license ;) [22:36:27] cwd: So what's the plan with dependencies btw? [22:38:57] Sorry, I think I'm jumping on this a bit early. [22:39:49] I noticed a few other things about the old code, like we mention specific queue names, but I'm pretty sure the logic is actually agnostic of queues and we should leave that to calling code. [22:43:47] (CR) Awight: [C: -1] "I like the low-friction integration. Seems like it'll work, my only suggestion is that the library you split out has the scope "donation " (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/300429 (owner: Cdentinger) [22:45:08] ejegg: Do you agree we should include CentralNotice in this staging cluster? [22:45:43] awight: sure, eventually! I'm focussing on queue stuff now [22:45:45] I'm tempted to even throw in donatewiki, wondering if we can simply replicate the wikidb for that. [22:45:55] good point! [22:46:09] awight: which deps? [22:46:17] i feel like they should be pretty minimal [22:46:21] I guess we do want banner history log id in the test links [22:46:37] cwd: I can just wait and see, but I'm wondering what your thoughts are. [22:46:55] i think this module should only have deps on data stores [22:47:36] we might consider looking into the composer merge plugin at some point. nested composer projects has some sharp edges [22:47:38] ejegg: It might be a convenient way to debug--either that or a page of links like we have in vagrant [22:47:57] or we finally write that form generator--pretty easy and fun [22:49:03] hmm, if we want unique banner history log ids we'll have to do something dynamic [22:49:14] cwd: Yeah, the big downfall there seems to be the strict "vendor" dir structure. If it's an in-house module, it should be anywhere you want, anything *but* vendor/ [22:49:46] why not in vendor? [22:50:00] as long as composer is an inevitability [22:50:12] ejegg: The beta cluster might be good enough for testing banner history--does it harm anything to have bad BH ids in the queues or Civi? [22:50:46] not sure [22:50:47] cwd: maybe "lib" but "vendor" sounds like "contrib", implying that it's a third-party library, the directory can be rm'ed and regenerated, etc. [22:51:10] ejegg: I'm following you on just focusing on queue staging for now... [22:51:56] shouldn't our aim be to write things that can be used as 3rd party modules? [22:52:51] another way to say that is that vendor already provides all the equipment for includes and autoloads etc, why reinvent? [22:52:55] ejegg: A lot of what you're suggesting in the "end to end" doc sounds like a mock PSP. Maybe we can just extend the Testing*Adapters? [22:53:28] sorry but i gotta run! will read backscroll. see ya! [22:53:57] cwd: The thing that bothers me is that composer doesn't handle e.g. mediawiki/extensions/EXT, all I would want in that situation is that it reads composer.json from anywhere in a subdir, and merges those requirements into a superset in the top level. [22:54:30] I suspect that there are ways of making it do that now, though--I still haven't educated myself about the merge plugin, and there's also this installer thing that ejegg unearthed [22:54:32] awight: huh, so have a way to hit the Testing* adapters from the UI? [22:54:50] and then have them create the fake IPN messages? [22:55:40] ejegg: yeah, we dependency inject those in above the real adapters, then get our framework of canned responses--those could even be served by a slightly smarter class which has mini-workflows and stuff. [22:56:25] hmm, would sure be slick [22:56:28] right, there might be a config to either send a synthetic IPN message to self [22:56:56] K4-713 envisioned this a while ago, we should leave some whiskey out and see if she comes back to help with the project [22:57:49] I dunno - might be nicer to have the fake PSP be a standalone site [22:57:59] that we can just point our URLs at [22:58:23] that might even be useful for other people writing this kind of code [22:58:36] in fact... is there anything like that out there? [23:03:17] huh, not seeing it [23:04:02] The cool part about that is it would allow for real interaction and not just a test for a single code path [23:04:22] I was thinking of a SmashPig maintenance script that just creates IPN payloads matching pending db entries [23:04:34] However, it means we eternally chase a bunch of changing assumptions about how the remote end responds. Which I guess we're engaged in regardless. [23:05:20] ejegg: That sounds more reasonable ;) [23:06:55] Actually, templated sample data sounds pretty durn neat [23:13:47] I wonder, are we including an ActiveMQ in this staging cluster? It seems unsafe to assume there are no side-effects that touch the legacy stuff. [23:30:13] ejegg: holler if you have any thoughts about the server role specifications, too... [23:30:29] server role specifications? [23:32:18] lines 50+ in the etherpad... [23:32:35] oh, cool! [23:32:42] & i guess we need some tweaks to the localsettings repo [23:38:22] (PS6) Awight: Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 (owner: Ejegg) [23:39:12] (CR) Awight: [C: 2] Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 (owner: Ejegg) [23:40:44] (Merged) jenkins-bot: Add installer info to composer.json [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/298136 (owner: Ejegg) [23:44:14] does anyone have ssl working for vagrant? [23:44:29] I'm getting ;The connection to payments.wiki.local.wmftest.net:4430 was interrupted while the page was loading; [23:44:37] s/;/'/g [23:45:27] fr-tech: ^^ ? [23:51:05] ejegg: No I think I have http [23:51:29] phooey, amazon needs https js origins [23:51:52] * awight looks for new theatrical trailer [23:52:15] too new eh? [23:52:41] tryna be secure with payment info, i guess [23:52:42] IE 11+, is that the origin u talking about? [23:53:37] That's funny cos I seem to recall Simple Payments asking us to blindly trust a DNS lookup of Amazon servers [23:53:46] Mebbe they learned the lesson... [23:54:07] IE11? [23:54:22] No, i'm getting connection reset in Chromium and FF [23:55:09] ejegg: sorry, yeah I was just about to ask. So you are not talking about http://www.w3schools.com/jsref/prop_loc_origin.asp [23:55:09] that'd be silly [23:55:36] oh hey, i just needed to 'vagrant roles enable https' [23:56:07] awight: no, I mean the widget code needs to be requested from an https url to work [23:57:37] (CR) Awight: [C: -1] "Small fixups" (4 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/299816 (https://phabricator.wikimedia.org/T140484) (owner: Ejegg) [23:58:03] ejegg: rad, thx for passing that along! [23:58:34] not really loving the new gerrit interface... [23:58:48] The wrapping bug better fix itself [23:59:16] & the interface in general grabs too many of my keystrokes--I can't even -C text! [23:59:27] jeez [23:59:43] * awight quickly reviews own JS, realizing I've failed to test for modifier keys, everywhere.