[00:00:24] well, maybe tomorrow [01:13:47] (PS4) Eileen: Further upstream fixes in 5.3 release [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439788 [01:28:20] (PS1) Eileen: Add Logging.get api support for contact_id as a filter. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/441328 [05:09:14] (PS2) Eileen: Add Logging.get api support for contact_id as a filter. [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/441328 [05:56:24] (PS20) Eileen: Add wmffraud into the showme/forgetme [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440253 [06:11:38] (PS21) Eileen: Add wmffraud into the showme/forgetme [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440253 [06:53:08] (PS22) Eileen: Add wmffraud into the showme/forgetme [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440253 [06:53:10] (PS1) Eileen: WIP on showing logging [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441339 [06:58:20] (CR) jerkins-bot: [V: -1] WIP on showing logging [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441339 (owner: Eileen) [09:06:42] (PS2) Eileen: WIP on showing logging [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441339 [09:12:12] (CR) jerkins-bot: [V: -1] WIP on showing logging [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441339 (owner: Eileen) [12:41:23] PROBLEM - Host frbast1001 is DOWN: PING CRITICAL - Packet loss = 100% [12:41:52] D: [12:41:58] that's me [12:42:33] RECOVERY - Host frbast1001 is UP: PING OK - Packet loss = 0%, RTA = 0.37 ms [12:43:02] :D [14:43:41] hi mepps, jgleeson, and AndyRussG! [14:43:55] morning ejegg :) [14:44:06] mepps, AndyRussG [14:44:13] everybody psyched for peru v france? [14:44:41] ha [14:44:49] i keep seeing the games on the WMF calendar, cool that they're showing them in the office [14:44:52] hi all! [14:45:08] hi mepps! Working from someplace nice this morning? [14:45:17] I'm just loving the boston summer weather this week [14:45:29] hi ejegg, yup i'm at my comic book themed coffee shop :) [14:45:36] :) [14:46:27] ejegg, have you caught world cup fever too? I normally do but I've been slow to catch it this time around [14:46:52] jgleeson: just watching the peru and colombia games [14:47:24] since i was hanging out in peru when they qualified [14:47:38] it really is a spectacle all these countries coming together enjoying the game [14:47:51] and it's like their first world cup in 30+ years [14:48:09] nice! [14:49:03] both of 'em lost their first game, but here's hoping they turn it around! [14:50:16] so... ingenico stuff [14:50:23] I saw your patch [14:50:32] turns out ZERO of the ingenico recurring got in [14:50:40] oh really? [14:51:02] So, part of the problem is that we can't make token entries w/o payment processor entries [14:51:11] i really kinda dropped the ball on that [14:51:15] that was predictable [14:51:57] but what happened seems to have been that none of the tokens actually got assigned on the front end [14:52:28] so I'm not sure if maybe we had 3d secure on for everyone? [14:53:37] anyway, i'm hoping we can tokenize those stuck in the damaged table, 1 month after the fact [14:54:31] i need to just replicate the situation locally and try with sandbox stuff in damaged, then I'll take the patch out of WIP [14:54:43] PS1 had some glaring errors, but PS2 looks like it might work [14:56:12] concerning recurring, I've tested out the 3d-secure maestro payment without the isRecurring flag, and it's still not working for me so now just stepping through to see whats up with that [14:56:29] I'm starting to dislike Maestro [14:56:44] heh, i don't think we actually accept it anyplace [14:58:00] hmm, lemme move that fn out of .install, just in case we need it again [15:00:04] yeah ejegg, I'd really like to generate a 3d-secure experience using visa so I can walk through that flow. The fudging with Maestro feels dirty [15:00:20] but looks like it's the only way for now [15:00:25] :( [15:00:43] with the old integration there was a flag on the API call to turn it on [15:05:46] fr-tech hisies! [15:07:51] :) [15:14:07] hi AndyRussG ! [15:24:55] (PS2) Mepps: WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) [15:24:58] :) [15:32:00] (PS1) AndyRussG: Add appeal_template property to EventLogging [extensions/FundraiserLandingPage] - https://gerrit.wikimedia.org/r/441403 [15:32:19] (PS3) Ejegg: Tokenize recurring payments stuck in damaged table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) [15:32:53] jgleeson: ok, that version works against the sandbox payment I shunted into damaged ^^^ [15:34:13] will check it out [15:37:21] thanks! [15:45:11] ejegg fr-tech what about the log output of the DjangoBurnerStats stuff? Does anyone look at it? Is it saved? Should it continue, and does it need to maintain the same format? [15:45:27] (CR) Jgleeson: [C: 2] Tokenize recurring payments stuck in damaged table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) (owner: Ejegg) [15:45:36] AndyRussG: nobody cares about it! [15:45:39] (CR) Jgleeson: [C: 2] "Looks good!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) (owner: Ejegg) [15:46:04] AndyRussG: maybe just for debugging is all, but no other process reads it [15:46:34] thanks jgleeson! [15:48:10] ok, here goes [15:50:31] well, once that merges... [15:50:40] (Merged) jenkins-bot: Tokenize recurring payments stuck in damaged table [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441317 (https://phabricator.wikimedia.org/T195488) (owner: Ejegg) [15:58:19] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/441413 [16:00:00] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/441413 (owner: Ejegg) [16:01:15] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/441413 (owner: Ejegg) [16:20:28] Fundraising Sprint Lactose is unusually tolerant, Fundraising Sprint Matt Damon to head up Space Force, Fundraising-Backlog: Write a specification for mapping banner/landing page impression event properties -> database schema - https://phabricator.wikimedia.org/T196563#4305755 (AndyRussG) Here's the... [16:38:04] !log updated CiviCRM from 349d43eb8b to e2c8ddd70e [16:38:06] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [16:41:48] whaaaa... Ingenico error code - merchant not found [16:41:49] ???? [16:42:12] that's definitely the merchant ID we always use [16:43:30] :| [16:44:52] still working from prod... [16:44:58] lessee those logs [16:45:26] or rather, working from the payments box, just not from the civi box [16:54:01] weird weird weird [16:54:03] Fundraising-Backlog, fundraising-tech-ops: civi job conflicting with database dumps (?) - https://phabricator.wikimedia.org/T196394#4305817 (Jgreen) Open>declined This appears to have been an isolated event, and has stopped happening. [16:55:20] fundraising-tech-ops, Operations, ops-eqiad: rack frpig1001 - https://phabricator.wikimedia.org/T187365#4305821 (Jgreen) [16:57:07] fundraising-tech-ops, Operations, ops-eqiad: rack frpig1001 - https://phabricator.wikimedia.org/T187365#3973216 (Jgreen) OS install is done, puppet is enabled, and it has been added to the FR deploy tools. We need to test smashpig functionality on PHP7 and finally do the NAT switch and enable monitor... [16:59:04] Fundraising-Backlog: test smashpig on Debian Stretch / PHP 7 - https://phabricator.wikimedia.org/T197887#4305824 (Jgreen) [16:59:20] Fundraising-Backlog: test smashpig on Debian Stretch / PHP 7 - https://phabricator.wikimedia.org/T197887#4305834 (Jgreen) [16:59:22] fundraising-tech-ops, Operations, ops-eqiad: rack frpig1001 - https://phabricator.wikimedia.org/T187365#4305835 (Jgreen) [16:59:44] fundraising-tech-ops, Operations, ops-eqiad: rack frpig1001 - https://phabricator.wikimedia.org/T187365#3973216 (Jgreen) Open>Resolved a:Jgreen [17:00:02] Fundraising-Backlog: test and deploy smashpig on Debian Stretch / PHP 7 - https://phabricator.wikimedia.org/T197887#4305824 (Jgreen) [17:02:14] fundraising-tech-ops, Operations, ops-codfw, Patch-For-Review: rack/setup/install Prometeuse/Grafana host frmon2001 for fr-tech - https://phabricator.wikimedia.org/T196476#4305848 (Jgreen) a:Jgreen>None [17:03:47] hey codfw monitor host, nice [17:03:54] didn't know we even had one on order [17:05:51] Jeff_Green: i'll assign that to me, unless you are currently editing the task [17:14:27] ejegg|food, that reply looks promising [17:14:32] cwd cool [17:14:46] looks like we can disable 3d-secure on a per request basis [17:15:09] cwd dc-ops have been auto-assigning them to me as they rack them [17:15:32] I've exhausted all local 3d-secure Maestro test cards and none are working for me, gonna follow up on the original thread to Jerry [17:15:50] Fundraising-Backlog, fundraising-tech-ops, Epic: switch fundraising database replication from 'mixed' to 'row' - https://phabricator.wikimedia.org/T183140#4305906 (Jgreen) [17:17:18] fundraising-tech-ops, Operations, ops-codfw, Patch-For-Review: rack/setup/install Prometeuse/Grafana host frmon2001 for fr-tech - https://phabricator.wikimedia.org/T196476#4305907 (cwdent) a:cwdent [17:19:00] Jeff_Green: i was just talking to jaime about row replication like an hour ago [17:19:15] oh yeah? [17:19:21] and i agree we shoudl do it even though we aren't seeing that problem anymore [17:19:27] yeah totally [17:19:47] it seems like a probably big performance win for zero effort [17:19:53] or maybe 1 effort [17:19:58] I need to review what gotchas they hit in prod, I imagine our case is gonna be less complicated [17:20:22] also means we can exclude temp/cache tables [17:20:40] right [17:21:26] those are the only tables i could see causing a problem w/ row based [17:21:57] i.e. huge enough change sets to lag the server, though i'd be surprised if that was actually a problem [17:22:05] server/network [17:22:14] i wonder how those work in the replication model [17:22:31] another gotcha is if you have different schemas on the replicas which we don't [17:22:50] like if a single query induces 50K rows of change on the master, does the query have to complete on the master before it starts propagating the individual row changes? [17:23:38] hmm yeah [17:23:55] we could test by doing a long query and tailing the bin log [17:24:13] i'll see if I can find it in TFM [17:24:30] :) [17:24:56] we also need to double check that we've got a primary key for every table, there were a couple that didn't a while back but we may have added a viable index by now [17:25:09] ah yes [17:25:39] well i'm going to get dinner and pack, heading out early (7:30 here) [17:25:42] (pm) [17:25:47] cool [17:26:04] hoping the flights are better than the way over [17:26:21] yeah, that's a lot of flight time for just a few days :-( [17:26:57] my first one was 2 hours late [17:27:06] which meant that i missed all of them [17:27:13] yuck [17:27:26] last minute rescheduling [17:27:35] which means ~30 hours to get here instead of 15 [17:27:39] and all middle seats [17:27:48] that's insane [17:27:57] was it a lot of layover time, or crazy indirect flights? [17:28:08] both [17:28:21] santa fe -> denver -> newark -> frankfurt -> prague [17:28:26] plus some buses [17:28:29] holy mackerel [17:28:34] it was pretty brutal [17:29:01] maybe you should have flown west instead :-P [17:29:12] seriously [17:29:48] well have a good trip back, I hope it goes smoothly [17:29:54] thanks :) [17:29:57] ttyl! [17:37:30] calling it a day fr-tech [17:37:34] catch you all tomorrow [17:37:39] Later, have fun. [17:38:09] ejegg|food, fire over any thoughts you have in email or voice note on the backend job to process ingenico recurring and I'll pick up on it tomorrow [17:38:22] bye all [18:20:35] Fundraising-Backlog, fundraising-tech-ops: add primary keys or unique indexes to some tables in civicrm, drupal, and pgehres databases - https://phabricator.wikimedia.org/T176631#4306029 (Jgreen) [18:20:37] Fundraising-Backlog, fundraising-tech-ops, Epic: switch fundraising database replication from 'mixed' to 'row' - https://phabricator.wikimedia.org/T183140#4306028 (Jgreen) [18:20:52] Fundraising-Backlog, fundraising-tech-ops, Epic: switch fundraising database replication from 'mixed' to 'row' - https://phabricator.wikimedia.org/T183140#3844578 (Jgreen) [18:21:28] Fundraising-Backlog, fundraising-tech-ops: add primary keys or unique indexes to some tables in civicrm, drupal, and pgehres databases - https://phabricator.wikimedia.org/T176631#3631845 (Jgreen) [18:21:45] Fundraising-Backlog, fundraising-tech-ops, Epic: fundraising database improvements for 2018 - https://phabricator.wikimedia.org/T183138#4306034 (Jgreen) [18:21:48] Fundraising-Backlog, fundraising-tech-ops: add primary keys or unique indexes to some tables in civicrm, drupal, and pgehres databases - https://phabricator.wikimedia.org/T176631#3631845 (Jgreen) [18:23:32] fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321#4306038 (Jgreen) [18:24:09] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Ensure fundraising civicrm database triggers are always the same on staging as on production - https://phabricator.wikimedia.org/T159507#4306044 (Jgreen) [18:25:01] Fundraising-Backlog, fundraising-tech-ops: add primary keys or unique indexes to fundraising civicrm/drupal/pgehres database tables - https://phabricator.wikimedia.org/T176631#4306047 (Jgreen) [18:41:18] Fundraising-Backlog, fundraising-tech-ops, Performance: Paymentswiki load.php should be cached - https://phabricator.wikimedia.org/T110394#4306064 (Jgreen) Open>stalled The nginx reverse proxy can do this and by default will pay attention cache control headers from the origin server. Right no... [18:41:20] Fundraising-Backlog, fundraising-tech-ops, MediaWiki-extensions-DonationInterface, Epic, Performance: [Epic] Payments performance tuning - https://phabricator.wikimedia.org/T140311#4306066 (Jgreen) [18:42:43] (PS3) Mepps: WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) [18:43:55] Fundraising-Backlog, fundraising-tech-ops: fundraising monitoring fixes (EPIC) - https://phabricator.wikimedia.org/T197892#4306073 (Jgreen) [18:44:23] (CR) jerkins-bot: [V: -1] WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [18:44:29] (PS1) Mepps: Add returnUrl, remove returnTitle [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/441423 (https://phabricator.wikimedia.org/T163949) [18:45:05] fundraising-tech-ops: Create alerts for rsyslog rate limiting - https://phabricator.wikimedia.org/T176924#4306086 (Jgreen) [18:45:38] Fundraising-Backlog, fundraising-tech-ops: Fundraising queues missing prometheus data since 4/26 - https://phabricator.wikimedia.org/T193456#4306091 (Jgreen) [18:45:40] Fundraising-Backlog, fundraising-tech-ops: fundraising monitoring fixes (EPIC) - https://phabricator.wikimedia.org/T197892#4306073 (Jgreen) [18:47:27] (PS4) Mepps: WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) [18:49:03] Fundraising-Backlog, fundraising-tech-ops: fundraising monitoring fixes (EPIC) - https://phabricator.wikimedia.org/T197892#4306095 (Jgreen) [18:49:57] (CR) jerkins-bot: [V: -1] WIP: Return url without Title Class for Ingenico [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441243 (https://phabricator.wikimedia.org/T163949) (owner: Mepps) [18:50:32] Fundraising-Backlog, fundraising-tech-ops: fundraising monitoring fixes (EPIC) - https://phabricator.wikimedia.org/T197892#4306073 (Jgreen) Open>Invalid [18:51:49] Fundraising-Backlog, fundraising-tech-ops: Fundraising queues missing prometheus data since 4/26 - https://phabricator.wikimedia.org/T193456#4306104 (Jgreen) [18:51:51] fundraising-tech-ops, monitoring, Epic: overhaul fundraising cluster monitoring - https://phabricator.wikimedia.org/T91508#4306103 (Jgreen) [18:52:38] Fundraising-Backlog, fundraising-tech-ops: Prometheus 2 breaking change - https://phabricator.wikimedia.org/T185134#4306106 (Jgreen) [18:52:41] fundraising-tech-ops, monitoring, Epic: overhaul fundraising cluster monitoring - https://phabricator.wikimedia.org/T91508#1088471 (Jgreen) [18:53:35] fundraising-tech-ops, monitoring, Epic: [Epic] overhaul fundraising cluster monitoring - https://phabricator.wikimedia.org/T91508#1088471 (Jgreen) [18:55:33] fr-tech anybody want to help me debug this tokenizing call? [18:55:47] It sure looks like it's trying to make the right request [18:55:58] and it doesn't look like a firewall problem [18:56:17] but for some reason tokenize is getting a 404 [18:56:38] maybe it's the payment ID??? [19:00:12] hmm, cwd, does "HTTP error before end of send, stop sending" sound like a firewall issue to you? [19:00:35] Fundraising-Backlog, fundraising-tech-ops: add primary keys or unique indexes to fundraising civicrm/drupal/pgehres database tables - https://phabricator.wikimedia.org/T176631#4306110 (Jgreen) p:Low>High [19:00:52] Fundraising-Backlog, fundraising-tech-ops, Epic: fundraising database improvements for 2018 - https://phabricator.wikimedia.org/T183138#4306111 (Jgreen) p:Triage>High [19:00:54] hmm ejegg [19:00:57] if you grep for world.api-ingenico in syslog on the civi box you can see the requests [19:01:07] is this happening locally or on prod? [19:01:50] mepps on prod [19:02:00] does it happen locally? [19:02:23] locally I got a sandbox donation into the queue without its token, and the script tokenized it just fine [19:02:41] mepps you can even try it for yourself, just running drush updb [19:02:43] fundraising-tech-ops: rsyslog hung after firewall switch - https://phabricator.wikimedia.org/T176821#4306112 (Jgreen) Open>declined Closing this because we haven't seen the problem reoccur and there's not much to do unless it does. [19:03:00] fundraising-tech-ops: Investigate decommissioning two eqiad-frack vlans - https://phabricator.wikimedia.org/T174203#4306114 (Jgreen) Open>stalled [19:06:09] Fundraising-Backlog, fundraising-tech-ops: add "frtech_mail" syslog filter to thulium - https://phabricator.wikimedia.org/T186018#3931148 (Jgreen) I'm not sure what this task is about? Are you looking to make a prometheus collector (guessing from the old task name)? [19:06:53] where is the bug coming up ejegg? when you run the script on prod? [19:07:25] it doesn't look like i had any contributions to tokenize locally [19:07:35] fundraising-tech-ops: encrypt fundraising mariadb replication - https://phabricator.wikimedia.org/T170320#4306120 (Jgreen) a:Jgreen [19:07:42] mepps yeah, only on prod [19:08:04] i just meant is the script that's running into errors ejegg [19:08:11] and the weirdest thing is that it doesn't seem to be logging the error response on the curl output [19:08:31] that's frustrating [19:08:32] mepps it's the 7715 schema update for wmf_civicrm [19:08:38] gotcha ejegg [19:08:51] Fundraising-Backlog: test and deploy smashpig on Debian Stretch / PHP 7 - https://phabricator.wikimedia.org/T197887#4306124 (Jgreen) [19:08:53] fundraising-tech-ops: EPIC: migrate fundraising hosts to Debian Stretch - https://phabricator.wikimedia.org/T185013#4306123 (Jgreen) [19:09:02] so what shows up in the drush output is not as I'd expect either [19:09:38] it just spits out 'Ingenico error id XXXXXXX-XXXX-XXXX : Error code 700510: MERCHANT NOT FOUND YYYY' [19:10:01] when the error handling I tried to write should prefix that with the payment ID [19:10:13] ohhh, did I forget the var_dump 2nd arg? [19:10:42] ohh, i meant print_r, not var_dump [19:10:50] ok, that's one mystery solved [19:11:19] but in any case, that error code and id DON'T show up in the curl verbose logging that you see if you grep for the world.api-ingenico stuff [19:12:55] in those, you see the request headers and then Expect: 100-continue [19:13:01] then you see HTTP/1.1 100 [19:13:14] then HTTP/1.1 404, with some response headers [19:13:37] then after the response headers: "HTTP error before end of send, stop sending#012< #015#012* Closing connection 0#012" [19:13:56] (The #012 should be newline) [19:14:19] so.... no idea why it thinks the merchant is not found [19:14:57] could the settings be wrong somewhere ejegg? [19:15:40] wait i'm confused--doesn't ingenico use the eu api? [19:15:40] so, it's the same merchant ID in the error message as we're using on the payments cluster [19:15:48] oh wait that's the sandbox [19:16:08] and if I grep for world.api-ingenico in the payments cluster logs, I see them making some requests just fine [19:16:25] hmm [19:16:39] it's not the same API call [19:17:02] ? [19:17:09] and with a REST API, a 404 could mean that the payment I'm trying to tokenize just doesn't exist [19:17:20] hmm [19:17:33] mepps I mean we haven't tried making this particular API call from the payments servers [19:17:55] no? how do we tokenize payments now? [19:18:14] there's a flag we can set in the initial createHostedPayment call [19:18:33] ah okay [19:21:46] hmm, I could temporarily fudge that particular damaged message to see if a different one works [19:23:01] MBeat: I think my globalcollect console account may have expired again due to inactivity. if so, could you please flip that switch for me? [19:28:12] sure thing, ejegg [19:28:19] thank you! [19:29:39] mepps: ok, i just added something to the start of that particular one's error message so it will be skipped [19:29:42] gonna try another [19:33:33] ejegg where's the config for smashpig ingenico on civi1001 i didn't see it in etc [19:38:34] well, same error code on a different payment id [19:53:57] and the payments DO exist in the console [20:10:45] ejegg: doubt it re: firewall, that should just look like lost packets [20:13:06] cwd right, I've had a fair number of firewall problems and this sure looks different! [20:14:02] so, I feel like I've exhausted the obvious ways this could be a problem on our end [20:14:36] guess i'mma pester merchantservices a second time today [20:24:53] oh hey, and they already got back to us on the 3dsecure switch [20:24:59] and... we totally do control it [20:25:05] how did I miss that field? [20:38:49] well, I'll fix that first [20:48:20] oh jeez, we're not making many assertions about what our API calls look like [20:48:33] in the main ingenico class, anyway [20:51:47] (PS1) Ejegg: WIP skipAuthentication flag disables 3DSecure for Connect [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441483 [20:53:58] (CR) jerkins-bot: [V: -1] WIP skipAuthentication flag disables 3DSecure for Connect [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441483 (owner: Ejegg) [21:22:34] (PS2) Ejegg: skipAuthentication flag disables 3DSecure for Connect [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/441483 [21:35:06] (PS2) Ejegg: JobQueueConsumer can hydrate non-JsonSerializableObjects [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441250 [21:35:08] (PS3) Ejegg: WIP job to add tokens to Ingenico payments [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/441251