[02:56:01] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice shifts down page content on load (causes mis-clicks) - https://phabricator.wikimedia.org/T52865#3253998 (Krinkle) >>! In T52865#2393116, @ori wrote: >>>! In T52865#554088, @Jalexander wrote: >> To be clear the biggest problem, by far,... [02:56:33] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice shifts down page content on load (causes mis-clicks) - https://phabricator.wikimedia.org/T52865#3254000 (Krinkle) [03:01:07] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Tracking: Reduce the "banner bump" (tracking) - https://phabricator.wikimedia.org/T109634#3254009 (Krinkle) [03:01:10] Wikimedia-Fundraising, MediaWiki-ResourceLoader, MediaWiki-extensions-CentralNotice, Operations, and 2 others: Provide location, logged-in status and device information in ResourceLoaderContext - https://phabricator.wikimedia.org/T103695#3254003 (Krinkle) Open>declined Declining as I I do... [03:20:40] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: CentralNotice shifts down page content on load (causes mis-clicks) - https://phabricator.wikimedia.org/T52865#3254010 (AndyRussG) >>! In T52865#3253998, @Krinkle wrote: > The real cause then becomes the fact that our page views are statically cache... [07:15:15] PROBLEM - check_procs on frdev1001 is CRITICAL: PROCS CRITICAL: 1105 processes [07:20:16] RECOVERY - check_procs on frdev1001 is OK: PROCS OK: 242 processes [10:37:10] fundraising-tech-ops, Operations, netops: BGP session between pfw clusters flapping - https://phabricator.wikimedia.org/T164777#3254591 (ayounsi) Nothing explicit in the logs. I've open case 2017-0511-0002 with JTAC [12:35:25] PROBLEM - check_mysql on payments2001 is CRITICAL: Slave IO: Preparing Slave SQL: Yes Seconds Behind Master: (null) [12:40:26] RECOVERY - check_mysql on payments2001 is OK: Uptime: 4127637 Threads: 3 Questions: 42998 Slow queries: 0 Opens: 29 Flush tables: 1 Open tables: 92 Queries per second avg: 0.010 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [12:45:25] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 5 minutes ago with 5 failures. Failed resources (up to 3 shown): Package[postfix-pcre],Package[mariadb-client],Package[postfix],Package[git-core] [12:50:25] RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [14:31:52] morning MBeat! the Hungary and Latvia test is up now [14:32:21] great, thanks for headsup pcoombe - will keep an eye on Zendesk & Ingenico [15:44:08] Hungary looking good so far. Latvia is v low traffic, but seems okay [15:46:07] MBeat dstrine : Paypal EC test for the US is all prepped, I can put it up whenever is convenient [15:46:30] how long will the EC test run, pcoombe ? [15:46:57] and is it US banners only? [15:47:31] MBeat: I think we did just 1 hour last time, that should be enough. yes US only, although I can add other countries if tech wants [15:47:58] thanks! any time is good [15:48:47] only 1 HU ticket, a donor who wasn’t sure their donation processed despite receiving the TY email: I asked if they saw and error of some kind [15:48:56] *an [16:02:12] pcoombe: thanks Dylan should be on any minute now. He can watch it on our side [16:02:20] ah lol [16:02:26] XenoRyet: just joined [16:02:41] oh :( [16:03:04] Hey, dstrine. What's up? [16:03:35] dstrine: no rush from my side, I only need to flick the switch [16:03:36] I don't know if you have backscroll but pcoombe was just saying he can do the EC test whenever. It's all prepped [16:03:56] Any time, including right now, is fine for me as well. [16:05:22] dstrine XenoRyet MBeat : cool, let's get it over with then :) [16:05:43] Let's do it [16:07:19] XenoRyet MBeat : US paypal test is up. setup is the same as last time: desktop, mobile and ipad [16:07:26] results sheet here https://docs.google.com/spreadsheets/d/1uETLfG5u1zRFyVwr5qJNZdF_EMaSXHLnjRFimSTeTic/edit#gid=173361939 [16:08:12] ty [16:17:06] looking good so far, just over 100 ec donations [16:20:18] yup, they’re in civi & ty emails sent quickly [16:20:52] Logs look nice so far. [16:21:39] One hit on the fraud filters, and one other error that I'll have to look into, but it's rare in any case. [16:40:36] MBeat: Hungary/Latvia test is done. I'll update the results in a bit and put them on the phab task [16:40:59] great, thanks pcoombe - I’ll generate elevate #s tomorrow [16:58:15] pcoombe: welcome to the beta cluster central notice administrators' exclusive club! [16:58:20] (should work now) [17:00:34] fr-tech: I have a dream. I have a dream that one day, on the red hills of Georgia, [17:00:34] the sons of former slaves and the sons of former slaveowners will be able to [17:00:34] sit down together at the table of brotherhood. [17:00:34] -- Martin Luther King, Jr. [17:00:35] -- discuss. [17:01:23] yeah! [17:02:00] Also elsewhere in the world [17:02:17] Too bad wage slavery is still a thing, tho [17:02:58] * AndyRussG glances sideways at a robot [17:05:14] thanks AndyRussG ! [17:07:17] XenoRyet MBeat : the EC test is over. results look pretty good on my end. I see some Civi fail mails though, not sure if that's related? [17:08:09] I saw one failmail that I wanted to look into, but I'm not sure the rest are related either. There was also one error that cropped up in the logs a few times that bears investigating, but over all things looked good here too. [17:08:29] great news [17:10:15] p-c is flapping...i am thinking the job frequency might be too quick for this volume [17:11:12] Fundraising Sprint Judgement Suspenders, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Large banner impression surge in NL and ES fr campaigns - https://phabricator.wikimedia.org/T164996#3255957 (AndyRussG) [17:11:30] pcoombe: np :) [17:33:24] pcoombe: dunno if I've already said this, just try to send test campaigns to beta cluster wikis other than enwiki, because that's where automatic browser tests run... thx!!! [17:33:54] ah okay, will do [17:34:37] Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising-Backlog, Patch-For-Review: Benevity import not updating emails for existing contacts - https://phabricator.wikimedia.org/T161666#3256074 (LeanneS) @Eilee... [17:34:39] K cool :) [17:37:20] Fundraising-Backlog: Improve process-control logging around locks - https://phabricator.wikimedia.org/T165072#3256082 (Ejegg) [18:10:08] PROBLEM - check_puppetrun on payments1003 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 5 minutes ago with 5 failures. Failed resources (up to 3 shown): Package[postfix-pcre],Package[mariadb-client],Package[postfix],Package[git-core] [18:15:08] RECOVERY - check_puppetrun on payments1003 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:16:41] XenoRyet: any ideas on that missing gateway_txn_id? [18:17:15] Hadn't looked just yet. Was trying to finish up the CR on that error encapsulation patch of yours before I dug into that. [18:17:32] It's looking good, I just want to try another thing or two before I give it the ol' +2 [18:18:08] oh wow, thanks! [18:18:17] Yea, no worries. [18:18:19] I'm available for a call if you want to talk out any bits of that [18:18:44] also, we should make sure we deploy the follow-on patch with the i18n updates along with the error encapsulation thing [18:18:53] Cool. It actually reads pretty easily though, and I see the intent. [18:19:05] Yea, I mean to CR that one along with. [18:19:12] thanks! [18:21:05] i am starting to wonder if the validation errors should even have message keys in them. Not sure how we'd indicate things like minimum and maximum donations otherwise [18:23:11] also, the duplication of the error state into both paymentTransactionResponse error and PaymentResult is odd. moreso paymentTransactionResponse, since that's a property of the adapter [18:23:32] but i think the patch is still a step in the right direction [18:23:58] Yea, it's not all the way to the promised land, but I think it's definitely a good step in the right direction. [18:24:25] Perfect being the enemy of good and all that. [18:25:44] Anyway, did anything jump out at you about that gateway_txn_id, or were you just getting started too? [18:26:20] Just getting started, but I notice there's a subscr_id and it's in the donations queue [18:26:47] The only other thing that showed up in the logs is some 'Request forbidden. No active donation in the session' errors, but I hadn't dug into those yet either. [18:27:05] huh, that's odd. [18:27:43] Anyway, I'll finish up this CR, then we'll figure it out. [18:27:50] k, thanks [18:30:40] (PS1) Ejegg: Use upstream PHP-Queue [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/353345 [18:31:13] fr-tech trivial CR there ^^^ [18:31:28] (CR) jerkins-bot: [V: -1] Use upstream PHP-Queue [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/353345 (owner: Ejegg) [18:31:28] last place we were pointing to our own fork of php-queue [18:31:31] d'oh [18:36:05] (PS2) Ejegg: Use upstream PHP-Queue [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/353345 [18:48:04] Wikimedia-Fundraising-Campaigns (FY Q4 Apr-Jun 2017), Wikimedia-CentralNotice-Administration (Q2-2017): WMF Fundraising - 2016-17 - Latvia/Hungary (Jun 6 - Jul 4) - https://phabricator.wikimedia.org/T164269#3256379 (Pcoombe) [18:48:06] Wikimedia-Fundraising: Fundraising banner pre-test in Latvia/Hungary - https://phabricator.wikimedia.org/T164405#3256377 (Pcoombe) Open>Resolved This ran today from 14:30 to 16:30 UTC. Traffic was quite low in Latvia (43 donations total), but there don't appear to be any changes in conversion compare... [18:52:15] (CR) Ejegg: [C: 1] "Thanks! I forgot to point this repo to the upstream php-queue lib, though - we'll need to merge Id3f11945265647f5 first to get past the co" [extensions/FundraisingEmailUnsubscribe] - https://gerrit.wikimedia.org/r/352780 (owner: Hashar) [18:57:10] Fundraising Sprint Deferential Equations, Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, and 9 others: Mediawiki namespace pages, including CentralNotice banners, are slow to sa... - https://phabricator.wikimedia.org/T158084#3256431 [18:58:24] Fundraising Sprint Homebrew Hadron Collider, Fundraising-Backlog, FR-PayPal-ExpressCheckout, FR-Paypal, Epic: Paypal Express checkout 1 hour test - https://phabricator.wikimedia.org/T131816#3256439 (Pcoombe) Just to note we ran a second test of this today, from 16:05 to 17:05 UTC in the US. R... [18:59:13] (CR) XenoRyet: [C: 2] "Looks good. Progress in the right direction." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/344042 (https://phabricator.wikimedia.org/T159897) (owner: Awight) [18:59:20] (CR) XenoRyet: [C: 2] Update translated error messages for new scheme [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/352995 (https://phabricator.wikimedia.org/T159897) (owner: Ejegg) [19:01:11] (Merged) jenkins-bot: Encapsulate errors [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/344042 (https://phabricator.wikimedia.org/T159897) (owner: Awight) [19:01:50] (Merged) jenkins-bot: Update translated error messages for new scheme [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/352995 (https://phabricator.wikimedia.org/T159897) (owner: Ejegg) [19:16:58] Fundraising Sprint Deferential Equations, Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, and 9 others: Mediawiki namespace pages, including CentralNotice banners, are slow to sa... - https://phabricator.wikimedia.org/T158084#3256464 [19:27:11] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Import email-only contacts from 'remind me later' links into CiviCRM - https://phabricator.wikimedia.org/T160949#3256471 (CCogdill_WMF) @Eileenmcnaughton you can use the ExportList API call to run a regular job importing a query from Silve... [19:31:18] Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, Fundraising-Backlog, FR-Smashpig, and 2 others: Handle iDEAL push notifications - https://phabricator.wikimedia.org/T161153#3256487 (Ejegg) [19:31:41] Fundraising Sprint Baudelaire Bowdlerizer, Fundraising Sprint Costlier Alternative, Fundraising Sprint Deferential Equations, Fundraising Sprint Dirt Farming, and 9 others: Store and update list of currently working iDEAL banks - https://phabricator.wikimedia.org/T128692#3256489 (Ejegg) [19:37:34] Fundraising Sprint Homebrew Hadron Collider, Fundraising-Backlog, FR-PayPal-ExpressCheckout, FR-Paypal, Epic: Paypal Express checkout 1 hour test - https://phabricator.wikimedia.org/T131816#3256539 (Ejegg) [19:37:36] Wikimedia-Fundraising-Campaigns: Fundraise in India - https://phabricator.wikimedia.org/T131118#3256540 (Ejegg) [19:37:38] Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, and 8 others: Assisted currency conversion for PayPal is broken again - https://phabricator.wikimedia.org/T98447#3256537 (Ejegg) O... [19:42:17] Fundraising-Backlog, FR-PayPal-ExpressCheckout, MediaWiki-extensions-DonationInterface: Recurring payment messages from frontend should go to recurring queue - https://phabricator.wikimedia.org/T165087#3256559 (Ejegg) [19:51:19] Fundraising-Backlog, FR-PayPal-ExpressCheckout, MediaWiki-extensions-DonationInterface: "Return to PayPal" error not handled right for recurring - https://phabricator.wikimedia.org/T165089#3256590 (Ejegg) [19:57:47] (PS2) Umherirrender: Typo fixes [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/351927 (owner: Ejegg) [19:57:53] (CR) Umherirrender: [C: 2] Typo fixes [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/351927 (owner: Ejegg) [20:00:08] (Merged) jenkins-bot: Typo fixes [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/351927 (owner: Ejegg) [20:12:48] (PS1) Ejegg: WIP deal with retry messages in pp recurring [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353382 (https://phabricator.wikimedia.org/T165089) [20:17:19] (PS1) Mepps: Update loadParamsAndArgs to use getOptionKit to fix errors causing T161260 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 [20:19:15] Fundraising-Backlog, Analytics, MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3256732 (AndyRussG) >>! In T115042#3252540, @mforns wrote: > 1) We spoke about the target segment of a campaign, and gave as an exam... [20:21:33] (PS1) Ejegg: Fix PP express for new encapsulated errors [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353386 [20:21:51] XenoRyet: ^^^ [20:22:03] Cool. Let me take a look. [20:22:13] I was doing that error encapsulation stuff in parallel with the return to paypal error [20:22:26] Ah [20:22:28] and ended up not modernizing that one part of the code [20:22:47] (PS1) Mepps: Smarty modifier converts all currencies available to civi T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 [20:23:12] ejegg just created two gerrit reviews [20:23:33] mepps: great! Looking... [20:23:47] about to fix the weird white space in smashpig [20:23:57] (CR) XenoRyet: [C: 2] Fix PP express for new encapsulated errors [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353386 (owner: Ejegg) [20:24:51] Alright, time for food [20:25:20] mepps: oh yeah, ws consistency is a problem for fr-tech [20:25:38] most of our repos use the mediwiki code standards [20:25:50] but our civi stuff is all over the place [20:25:56] (PS2) Mepps: Update loadParamsAndArgs to use getOptionKit to fix errors causing T161260 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 [20:26:51] i don't know why it's adding that space after inputLoaded = true... [20:26:51] (Merged) jenkins-bot: Fix PP express for new encapsulated errors [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353386 (owner: Ejegg) [20:31:42] (PS3) Mepps: Update loadParamsAndArgs to use getOptionKit to fix errors causing T161260 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 [20:34:23] (CR) Ejegg: "Unmangled! Might want to parameterize the query." (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (owner: Mepps) [20:35:39] (PS4) Mepps: Update loadParamsAndArgs to use getOptionKit to fix errors causing T161260 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 [20:46:03] Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, Fundraising Sprint Homebrew Hadron Collider, Fundraising-Backlog, Patch-For-Review: Benevity import not updating emails for existing contacts - https://phabricator.wikimedia.org/T161666#3256797 (Eileenmcnaughton... [20:47:07] (CR) Ejegg: [C: 1] "Nice! Can you just add a special case for the --help parameter making it boolean instead of adding it generically up in addDefaultParams ?" [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 (owner: Mepps) [20:49:32] (PS2) Mepps: Smarty modifier converts all currencies available to civi T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 [20:56:29] ejegg updated the civi extension pr [20:57:16] oh hmm [20:57:40] i'll need to update the template in the install script too [20:58:06] (CR) Ejegg: [C: 2] "Good stuff!" (1 comment) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (owner: Mepps) [20:58:25] Can be a followon [20:59:30] jfyi, the task id needs to be on its own line prefixed by 'Bug: ' to get the full phab linkage, with phab also linking back to gerrit [20:59:43] oh got it [21:00:45] (PS3) Mepps: Smarty modifier converts all currencies available to civi Bug: T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (https://phabricator.wikimedia.org/T151957) [21:01:15] (PS4) Ejegg: Smarty modifier converts all currencies available to civi Bug: T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (https://phabricator.wikimedia.org/T151957) (owner: Mepps) [21:01:40] (PS5) Mepps: Update loadParamsAndArgs to use getOptionKit to fix errors causing Bug: T161260 [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/353383 (https://phabricator.wikimedia.org/T161260) [21:11:31] (PS2) Ejegg: Deal with retry errors in pp recurring [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353382 (https://phabricator.wikimedia.org/T165089) [21:12:01] (CR) Ejegg: [C: 2] Smarty modifier converts all currencies available to civi Bug: T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (https://phabricator.wikimedia.org/T151957) (owner: Mepps) [21:16:31] (Merged) jenkins-bot: Smarty modifier converts all currencies available to civi Bug: T151957 [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/353387 (https://phabricator.wikimedia.org/T151957) (owner: Mepps) [21:22:16] mepps: I just looked at the extension you did for T151957 & I'm just trying to understand why it was an extension required, not a bug fix [21:22:16] T151957: Thank you letter from civi mangles non-$ currencies - https://phabricator.wikimedia.org/T151957 [21:23:00] hmm eileen1 you mean a civi bug fix? Not a bad point [21:26:28] that currency table should have columns for format and decimal places [21:26:56] fr-tech I told Cecilia the riddle from Standup, and she had a novel solution: send the dog and the grain over in the boat by themselves, and have the dog push the boat off the shore back to you [21:27:10] AndyRussG: nice! [21:27:43] Yeah, eh? Also highlights some hidden assumptions [21:37:53] mepps: yeah - that error was happening on the built in form when I saw it - but possibly not when sent from a different server - sorry I have to shoot out - after raising questions I'm running away [21:40:30] eileen1 that's okay but would love more context about the form [21:40:46] :-) [21:41:10] mepps: I'll catch up with you on it next week [22:07:32] XenoRyet: this should fix the thing that got us failmail this morning: https://gerrit.wikimedia.org/r/353382 [22:07:44] Cool, I'll take a look. [22:09:27] also, I'm not sure, but the error patch might have messed up showing error forms on return from cc processor [22:09:47] might just be local misconfiguration, testing again on the current deployment branch [22:23:54] Fundraising Sprint Deferential Equations, Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, Fundraising Sprint Gondwanaland Reunification Engine, and 9 others: Mediawiki namespace pages, including CentralNotice banners, are slow to sa... - https://phabricator.wikimedia.org/T158084#3257115 [22:37:47] (CR) XenoRyet: [C: 2] Deal with retry errors in pp recurring [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353382 (https://phabricator.wikimedia.org/T165089) (owner: Ejegg) [22:39:57] (Merged) jenkins-bot: Deal with retry errors in pp recurring [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353382 (https://phabricator.wikimedia.org/T165089) (owner: Ejegg) [22:51:37] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: New resultswitcher framebusting order messes up error display - https://phabricator.wikimedia.org/T165106#3257156 (Ejegg) [22:54:24] ugh, ^^^ is no fun [22:55:27] but... it's not any WORSE with the new error encapsulation code [22:58:48] heh that sounds like something that is my fault [23:01:06] ejegg: we never got any reports of this from donors? [23:01:29] I guess not... [23:02:04] the whole iframe thing [23:02:22] nothing would surprise me about a browser update ruining our whole scheme [23:03:32] we just didn't think it through, I guess [23:03:53] well i mean the breaking out of the iframe thing more generally [23:03:55] or none of us have the rapidFail thing enabled locally [23:05:16] i only vaguely remember but i'm sure this was solving a problem [23:05:31] the pre-breakout card charge [23:06:41] cwd I thought that we were losing donors between the popout and the processing [23:06:55] but the cure seems worse than the disease [23:07:29] does the failure happen in the wild? [23:07:51] yep, i just tried on production and got what looks like nothing happening [23:08:16] it doesn't even render the failform in the iframe, just sent response headers and zero bytes of content :( [23:09:49] maybe we broke mustacheErrorForm too? [23:10:05] i would just be surprised if this was happening for months and we never heard about it [23:11:58] blech, undefined index errors [23:13:56] (PS1) Ejegg: Don't crash error form on missing keys [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353482 [23:17:08] (CR) Cdentinger: [C: 2] Don't crash error form on missing keys [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353482 (owner: Ejegg) [23:17:23] ejegg: is that just to debug or was that the problem? [23:18:36] gotta roll! [23:18:58] (Merged) jenkins-bot: Don't crash error form on missing keys [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353482 (owner: Ejegg) [23:20:20] cwd just debug... [23:23:23] (PS1) Ejegg: Guard against missing params in PP-EC return URL [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/353483