[00:02:10] (CR) jerkins-bot: [V: -1] Vendor update [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/364622 (owner: Eileen) [00:15:53] (PS3) Eileen: Do not add id if the field does not exist [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/364362 [00:19:45] (PS2) Eileen: Omnimail, enable view tab into mailing events. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364363 [00:33:28] (CR) jerkins-bot: [V: -1] Omnimail, enable view tab into mailing events. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364363 (owner: Eileen) [01:31:12] (PS4) Mepps: WIP: Implement HostedCheckout [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/363458 (https://phabricator.wikimedia.org/T163946) [01:32:47] (PS5) Mepps: WIP: Implement HostedCheckout [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/363458 (https://phabricator.wikimedia.org/T163946) [02:20:48] Fundraising-Backlog, Chinese-Sites, Technical-Debt: Clean up onwiki fundraising cruft - https://phabricator.wikimedia.org/T136134#3428835 (Krinkle) (I cleaned up the remaining `whichMessage` code from the Common.js of a handful of wikis that were still unconditionally loading it on all page views.) [11:36:36] Fundraising-Backlog, Chinese-Sites, Technical-Debt: Clean up onwiki fundraising cruft - https://phabricator.wikimedia.org/T136134#3430047 (Pcoombe) a:Pcoombe>None [12:18:42] Fundraising-Backlog: reevaluate legacy recurring paypal subscriptions tied to old fundraising domains - https://phabricator.wikimedia.org/T170414#3430170 (Jgreen) [12:26:58] Fundraising-Backlog: reevaluate legacy recurring paypal subscriptions tied to old fundraising domains - https://phabricator.wikimedia.org/T170414#3430187 (Jgreen) [12:29:51] Fundraising-Backlog, MediaWiki-extensions-DonationInterface: Inconsistent order of payment field errors - https://phabricator.wikimedia.org/T169797#3430191 (schoenbaechler) Hey @Ejegg, I would like to offer help here. Could you invite me the repo or send me the link to where I can find the files? Please... [12:58:18] mbeat / pcoombe : net ops got their thing done, so we should be good to include Amazon [12:58:30] great news, thanks ejegg|away [12:59:00] wonderful, thanks for the help ejegg|away ! [13:21:28] ejegg are you working this morning? [13:22:30] not fully activated just yet, but i wanted to check on the status of Amazon before the big english test [13:24:43] gotcha [14:42:05] ok mepps, all situated now [14:42:13] ejegg cool [14:42:30] so yesterday afternoon I was focussed on eileen's patches and didn't get back to the ingenico stuff [14:42:35] how's that going? [14:42:37] ejegg no problem [14:42:46] i got stuck on a weird issue that i resolved late last night [14:42:55] (didn't work continuously that whole time though!) [14:43:04] with types of line breaks [14:43:09] whew, I hope not! [14:43:18] whoa, newline issues? [14:43:48] yeah [14:43:54] luckily phpstorm helped me resolve it [14:44:16] basically the response parsing was looking for \r\n [14:44:25] but my editor by default was just putting in \n [14:44:36] phpstorm lets you designate what type of newline to put in though [14:45:03] (CR) Ejegg: [C: 2] "Cool, sounds good to at least make it possible to choose 'primary' before worrying about making it the default." [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/364311 (https://phabricator.wikimedia.org/T169025) (owner: Eileen) [14:45:05] this morning i've just been poking around the donation interface code, seeing where things are [14:45:35] mepps argh, that's right, I should put a warning somewhere [14:45:52] I guess \r\n made it into the http spec [14:46:01] ejegg my only concern was is there any chance that would be an issue with an actual response? [14:46:41] I think servers are pretty consistent with the spec on that note [14:47:07] okay cool [14:47:21] errr, no, you're right [14:47:33] 'be strict in what you emit, but tolerant in what you accept' [14:47:55] so yeah, if you want to fix it to recognize either, that would be an improvement [14:48:06] ha, hmm, when i tried to limit it to \n everything broke, let me look at it [14:48:28] still gotta chomp the \r s when they exist, of course [14:48:38] oh okay [14:49:08] oh hey, yeah, the spec says that servers MUST use \r\n, but that clients SHOULD be tolerant of bad implementations and accept \n [14:49:16] https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3 [14:51:53] (Merged) jenkins-bot: CRM-20759 enable Primary as an import location type. Upstream Prs https://github.com/civicrm/civicrm-core/pull/10594 https://github.com/civicrm/civicrm-core/pull/10565 [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/364311 (https://phabricator.wikimedia.org/T169025) (owner: Eileen) [14:53:42] ejegg MBeat: reminder that we're starting the Big English test in about 7 minutes. everything ready on your ends? [14:53:54] looks good, thanks pcoombe [14:54:02] we are using express checkout for paypal, and have amazon back in the US [14:54:04] pcoombe: yep, things look good. Just got a test Amazon donation through [14:54:24] and the IPN notifications and everything worked [14:54:45] thanks! I sent test donations through ec and amazon earlier, and they both looked good [14:55:21] fundraising-tech-ops, Operations, netops, ops-codfw: codfw: rack frack refresh equipment - https://phabricator.wikimedia.org/T169643#3430769 (ayounsi) Thank you! For the naming, please use: pfw3a-codfw and pfw3b-codfw for the firewalls and fasw-c8a-codfw and fasw-c8b-codfw for the switch members... [15:01:33] ejegg MBeat : banners are up! [15:01:40] cool! [15:02:33] great! [15:14:47] Fundraising Sprint Judgement Suspenders, Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog: Compile list of systems we depend on and should check all the time - https://phabricator.wikimedia.org/T163509#3430886 (Eje... [15:32:40] AndyRussG: and ejegg do you think you could call into an 8:30am PDT meeting either tomorrow or friday? [15:33:39] this would be for WMDE [15:39:46] dstrine: sure, either one would be good for me [15:40:00] thanks ejegg [15:45:09] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 10863 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 6 days 16 hours - memory use is 12.41M (peak 12.44M, 0.20% of max, fragmentation 1.32%), connected_slaves is 2, jobs is 0, jobs-adyen is 3, jobs-paypal is 497, payments-antifraud is 482, payments-init is 594, pending is 179, recurring is 18, refund is 0, unsubscribe is 1 [15:50:10] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 12090 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 6 days 16 hours - memory use is 12.79M (peak 13.12M, 0.21% of max, fragmentation 1.36%), connected_slaves is 2, jobs is 0, jobs-adyen is 5, jobs-paypal is 531, payments-antifraud is 198, payments-init is 259, pending is 71, recurring is 31, refund is 0, unsubscribe is 1 [15:50:32] (PS1) Ejegg: Fix thank you mailer watchdog message [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364765 [15:50:45] fr-tech tiny log cleanup ^^^ [15:51:22] ooh, looks like I better give the donations queue consumer more time [15:52:12] looks like we are running some campaigns? [15:53:06] * cwd reads email [15:54:04] ejegg should there be a variable in the array above in the patch above? otherwise good to +2 [15:55:13] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 12740 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 6 days 16 hours - memory use is 14.23M (peak 14.29M, 0.22% of max, fragmentation 1.28%), connected_slaves is 2, jobs is 0, jobs-adyen is 6, jobs-paypal is 511, payments-antifraud is 734, payments-init is 799, pending is 154, recurring is 53, refund is 0, unsubscribe is 1 [15:57:04] AndyRussG: what say you about a WMDE meeting? [15:58:04] mepps nope, the array's just a placeholder there [15:58:32] (CR) Mepps: [C: 2] Fix thank you mailer watchdog message [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364765 (owner: Ejegg) [15:59:31] man, TY mail sending is super lagged [15:59:52] turning off CiviMail records to see how much faster it goes [16:00:10] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 13483 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 6 days 16 hours - memory use is 14.43M (peak 14.43M, 0.23% of max, fragmentation 1.31%), connected_slaves is 2, jobs is 0, jobs-adyen is 1, jobs-paypal is 523, payments-antifraud is 454, payments-init is 546, pending is 360, recurring is 67, refund is 0, unsubscribe is 1 [16:00:10] ejegg this newline thing is turning out to be a bigger fix, only because we needed to use header_size and now i have to fix the tests that don't use setUpResponse [16:00:49] mepps: ah, if it's a distraction feel free to set it aside for now! [16:00:55] ejegg: MBeat : test is over and banners are down [16:00:58] ejegg i'm in at this point [16:01:01] thanks pcoombe! [16:01:06] thank you! [16:01:10] thank you :) [16:01:20] so, we definitely need to do something about the dead session errors [16:01:46] it's a fraction of a percent, but i'm guessing MBeat will see another dozen or so complaints from this hour [16:02:17] thank you for that ejegg - helpful to know the scope of a bug [16:02:29] I'd really like to understand why the sessions are dying, but I've got an idea for a workaround. [16:03:14] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, FR-PayPal-ExpressCheckout: PayPal EC dead session error - https://phabricator.wikimedia.org/T167923#3349710 (Ejegg) p:Triage>High a:Ejegg [16:05:08] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 14247 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 6 days 16 hours - memory use is 13.93M (peak 14.74M, 0.24% of max, fragmentation 1.38%), connected_slaves is 2, jobs is 0, jobs-adyen is 5, jobs-paypal is 404, payments-antifraud is 119, payments-init is 159, pending is 99, recurring is 11, refund is 0, unsubscribe is 0 [16:06:04] hmm, sending about 5 thank you emails per second, but still not keeping up with donation queue [16:06:29] Could definitely optimize the filesystem access [16:06:58] (Merged) jenkins-bot: Fix thank you mailer watchdog message [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364765 (owner: Ejegg) [16:08:19] ejegg: Is that just sending TY emails, or also getting things into civi? conversion looked a bit lower than last test but is picking up now, which makes me think there may be some delay [16:08:28] ejegg: what do you think is the slow part? [16:08:53] pcoombe the queue is a bit backed up too (see: icinga-wm alerts above) [16:09:22] oh okay, wasn't sure if they were related [16:10:09] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 12672 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 6 days 16 hours - memory use is 11.62M (peak 14.74M, 0.22% of max, fragmentation 1.53%), connected_slaves is 2, jobs is 0, jobs-adyen is 4, jobs-paypal is 56, payments-antifraud is 82, payments-init is 97, pending is 20, recurring is 17, refund is 0, unsubscribe is 0 [16:12:57] OK, queue counts are a bunch higher than actual donations, probably because we're queueing once on payments and a second time on the IPN listener. [16:14:58] ejegg: did it run faster under jenkins? [16:15:11] PROBLEM - check_redis on frqueue1001 is CRITICAL: CRITICAL: donations is 10830 10000 - REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 6 days 16 hours - memory use is 10.11M (peak 14.74M, 0.22% of max, fragmentation 1.76%), connected_slaves is 2, jobs is 0, jobs-adyen is 1, jobs-paypal is 28, payments-antifraud is 26, payments-init is 26, pending is 15, recurring is 20, refund is 0, unsubscribe is 0 [16:15:23] cwd hard to compare [16:15:39] I think the new db server is making some stuff faster [16:16:05] hmm, we definitely had some timing data someplace [16:16:07] lemme see [16:17:35] cwd https://phabricator.wikimedia.org/T146338 [16:18:13] so I guess the TY mail sending is about the same rate, but we're running it for shorter cycles now [16:18:43] ohh, but the initial record fetch is taking a good chunk of that time [16:18:50] so we should run it for longer [16:19:01] did we just not alert on queue bloat before? [16:19:14] maybe different thresholds? [16:19:33] yeah maybe [16:19:41] i guess we should just crank up the threshold [16:20:00] also, I think the old paypal method only sent queue messages from the IPN listener [16:20:08] and the new one sends both from IPN and from payments [16:20:15] so there are a lot of duplicates to discard [16:21:28] ah right on [16:21:30] that is paypal EC? [16:22:12] there's a listener patch to consult the c_t table and avoid queueing duplicate pending messages: https://gerrit.wikimedia.org/r/312563 [16:22:28] we could use the same check to avoid queueing duplicate donation messages [16:22:48] cwd yeah, today's test was with paypal all going to the EC gateway [16:22:57] nice [16:23:20] imo it's fine if it's slow to import [16:23:27] it seems like it's still on the order of a couple minutes [16:23:47] yah, but in a sustained campaign it would get further and further behind [16:23:56] i just want to adjust icinga accordingly so we don't get paged [16:24:01] this is true [16:24:40] i fear we are going to be falling on hard times with mysql [16:26:32] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email: Optimize filesystem access in thank_you email job - https://phabricator.wikimedia.org/T170435#3431198 (Ejegg) [16:30:45] dstrine: hey, sorry just saw your pings! Yes, 8:30 am PDT is totally fine [16:30:56] AndyRussG: ok thanks [16:30:57] either day [16:31:05] how are you doing? [16:34:14] fundraising-tech-ops: Unsubscribe AWight from icinga alerts - https://phabricator.wikimedia.org/T170437#3431249 (awight) [16:37:30] heh [16:40:11] RECOVERY - check_redis on frqueue1001 is OK: OK: REDIS 2.8.17 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 6 days 17 hours - memory use is 4.86M (peak 14.74M, 0.18% of max, fragmentation 3.09%), connected_slaves is 2, donations is 4009, jobs is 0, jobs-adyen is 5, jobs-paypal is 5, payments-antifraud is 13, payments-init is 11, pending is 4, recurring is 2, refund is 0, unsubscribe is 0 [16:50:29] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: "Thank-you Sent" field auto populating date - https://phabricator.wikimedia.org/T170440#3431347 (LeanneS) [16:56:41] (PS6) Mepps: WIP: Implement HostedCheckout [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/363458 (https://phabricator.wikimedia.org/T163946) [17:01:08] (PS1) Ejegg: Speed up thank you query [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364783 [17:01:52] fr-tech one tiny change to the thank you query cuts it down from 9 sec to less than one sec ^^^ [17:02:22] (CR) Mepps: [C: 2] Speed up thank you query [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364783 (owner: Ejegg) [17:02:25] I think because mysql is already using the receive_date in the where clause [17:02:27] thanks! [17:22:53] (PS1) Ejegg: Cache result of language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 [17:25:56] (PS2) Ejegg: Cache result of language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) [17:26:06] fr-tech any news for scrum of scrums? [17:29:05] Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Email, Patch-For-Review: Optimize filesystem access in thank_you email job - https://phabricator.wikimedia.org/T170435#3431617 (Ejegg) a:Ejegg [17:54:08] (PS1) Ejegg: WIP recover from dead sessions via EC token [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364794 (https://phabricator.wikimedia.org/T167923) [17:55:18] fr-tech / cwd ^^^ feels like a really kludgey workaround, but I'm having no luck figuring out why those sessions are dying [17:55:35] it's happening on all different devices and browsers [17:56:33] we're always calling session_addDonorData() when we do the inital call to paypal that sets up the payment [17:58:48] and it only happens with paypal? [17:58:57] cwd seems like it [17:59:04] only new paypal ec? [17:59:20] Well, the old paypal didn't require any processing on the return [17:59:39] My only guess is that it has to do with the immediate redirect [17:59:49] They're only making one request to paymentswiki [18:00:03] and that request comes back with a set-cookie header and a location header [18:00:25] browsers are all SUPPOSED to set that cookie before redirecting [18:00:55] but maybe sometimes they get impatient? [18:01:05] oh hey, travel briefing [18:01:56] i can't make it to wikimania :( [18:02:05] aw man, that sucks [18:02:17] this has been a travel heavy year already [18:03:32] phooey, we really want to restore that session before even creating the adapter [18:04:19] (Merged) jenkins-bot: Speed up thank you query [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364783 (owner: Ejegg) [18:04:31] ...and restore more than just the donor key if possible [18:05:16] wow, zuul is really backed up [18:05:31] oh right, we want to get rid of that extra node test if possible [18:14:00] (PS3) Ejegg: Resurrecting hashar's original composer json linter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/351351 [18:26:16] mepps / eileen: we're currently waiting a long time for zuul to do the tests in CRM [18:26:43] that patch combines two of the test tasks together [18:26:56] so we just run composer test, not both that and npm test [18:27:39] there's a companion patch in the CI repo: https://gerrit.wikimedia.org/r/364804 [18:45:31] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, and 2 others: PayPal EC dead session error - https://phabricator.wikimedia.org/T167923#3432007 (Ejegg) For paypal, the donor only hits our server once (as long as they... [19:00:47] (CR) Mepps: [C: 2] Resurrecting hashar's original composer json linter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/351351 (owner: Ejegg) [19:01:02] thanks mepps! [19:08:16] (CR) jerkins-bot: [V: -1] Resurrecting hashar's original composer json linter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/351351 (owner: Ejegg) [19:10:32] (PS1) Ejegg: PayPal EC: use ajax to get redirect URL [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364825 (https://phabricator.wikimedia.org/T167923) [19:11:04] fr-tech ^^^ turned out not to be too crazy. Think we should go for it? [19:11:35] The user experience is that they see the payments-wiki page in all cases, not just when there are errors [19:12:13] as soon as the page loads, the browser makes a call to the API and shows a loading overlay [19:12:46] then if that comes back fine the browser goes to paypal as before [19:13:14] ok, gonna snag some food. be back for standup! [19:21:33] Hey MBeat, question coming out of this FB ads test - do we accept money orders? It's not listed on Ways to Give [19:21:47] they are like checks, generally spatton_ [19:21:59] so yes [19:22:04] Cool, and same mailing address? [19:22:35] DC PO Box if US, SF office if international/other currencies [19:22:43] Thanks MBeat :) [19:22:47] np! [20:02:55] Fundraising Sprint Homebrew Hadron Collider, Fundraising Sprint Ivory Tower Defense Games, Fundraising Sprint Judgement Suspenders, Fundraising Sprint Kickstopper, and 5 others: CentralNotice: Admin UI for campaigns may submit input with errors - https://phabricator.wikimedia.org/T163057#3432483 (... [20:03:21] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: Epic: CentralNotice Banner squence - https://phabricator.wikimedia.org/T144615#3432486 (AndyRussG) [20:03:24] Fundraising Sprint Costlier Alternative, Fundraising Sprint Deferential Equations, Fundraising Sprint English Cuisine, Fundraising Sprint Far Beer, and 13 others: CentralNotice banner sequence: implement feature for MVP - https://phabricator.wikimedia.org/T144453#3432485 (AndyRussG) Open>R... [20:03:45] Fundraising-Backlog, MediaWiki-extensions-CentralNotice, Epic: Epic: CentralNotice Banner squence - https://phabricator.wikimedia.org/T144615#2605284 (AndyRussG) [20:03:48] Fundraising Sprint Judgement Suspenders, Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, and 3 others: CentralNotice banner sequence: unit tests - https://phabricator.wikimedia.org/T144456#3432488 (AndyRussG) Open>Resolved [20:04:01] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, and 4 others: Figure out a sane way to deploy SmashPig.yaml configuration - https://phabricator.wikimedia.org/T147503#3432491 (Ejegg) Open>Resolved [20:06:29] (CR) jerkins-bot: [V: -1] Cache result of language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) (owner: Ejegg) [20:08:15] Fundraising Sprint Kickstopper, Fundraising Sprint Loose Lego Carpeting, Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, and 3 others: [Tech Debt] Unit tests are sensitive to local SmashPig.yaml settings - https://phabricator.wikimedia.org/T160376#3432537 (Ejegg) Open>Resolved [20:12:01] (CR) jerkins-bot: [V: -1] WIP recover from dead sessions via EC token [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364794 (https://phabricator.wikimedia.org/T167923) (owner: Ejegg) [20:13:23] (CR) jerkins-bot: [V: -1] Resurrecting hashar's original composer json linter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/351351 (owner: Ejegg) [20:17:51] Fundraising-Backlog, FR-Ingenico: Determine if we need to change anything with Ingenico authorizations - https://phabricator.wikimedia.org/T170475#3432627 (Ejegg) [20:24:28] Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: "Thank-you Sent" field auto populating date - https://phabricator.wikimedia.org/T170440#3432661 (DStrine) [20:25:47] Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: "Thank-you Sent" field auto populating date - https://phabricator.wikimedia.org/T170440#3432664 (mepps) a:mepps [20:40:08] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1284 [20:40:09] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1284 [20:40:09] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1418 [20:45:18] RECOVERY - check_mysql on frdb1002 is OK: Uptime: 108929 Threads: 1 Questions: 4452806 Slow queries: 545 Opens: 1133 Flush tables: 1 Open tables: 587 Queries per second avg: 40.878 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [20:45:18] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 108390 Threads: 3 Questions: 4413873 Slow queries: 519 Opens: 1009 Flush tables: 1 Open tables: 449 Queries per second avg: 40.722 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [20:45:19] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1584 [20:50:18] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1884 [20:55:18] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2184 [20:59:12] Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, FR-PayPal-ExpressCheckout: Stop using kludgy workaround for PayPal EC recurring - https://phabricator.wikimedia.org/T170478#3432810 (Ejegg) [21:00:08] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 108470 Threads: 1 Questions: 4442011 Slow queries: 483 Opens: 1129 Flush tables: 1 Open tables: 572 Queries per second avg: 40.951 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [21:48:35] (PS1) Ejegg: PayPal EC recurring: initial charge is part of subscription [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364903 (https://phabricator.wikimedia.org/T170478) [21:49:18] Fundraising Sprint Murphy's Lawyer, Fundraising-Backlog, FR-PayPal-ExpressCheckout, Patch-For-Review: Stop using kludgy workaround for PayPal EC recurring - https://phabricator.wikimedia.org/T170478#3432810 (Ejegg) p:Triage>High [21:53:32] (CR) Ejegg: [C: -2] "Oops, need to cache by more than just language" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) (owner: Ejegg) [21:59:36] mepps: ejegg would this be able to go out the next time you are releasing something for civi? https://phabricator.wikimedia.org/T169025 [21:59:52] dstrine: for sure [22:01:35] thanks! MG is looking forward to it [22:04:36] (CR) Ejegg: "Thanks for doing the newline fix! Could you break that out into a parent commit?" (6 comments) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/363458 (https://phabricator.wikimedia.org/T163946) (owner: Mepps) [22:18:18] (CR) Ejegg: "Some potential next steps:" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364143 (owner: Ejegg) [22:18:33] (Abandoned) Ejegg: base ingenico test case [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364345 (owner: Ejegg) [22:19:15] (PS1) AndyRussG: Controls to purge banner content from front-end cache for a language [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/364910 (https://phabricator.wikimedia.org/T168673) [22:20:48] (PS2) AndyRussG: Controls to purge banner content from front-end cache for a language [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/364910 (https://phabricator.wikimedia.org/T168673) [22:21:33] (CR) Ejegg: WIP Ingenico gateway mimics GlobalCollect (1 comment) [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/364143 (owner: Ejegg) [22:24:06] (PS3) AndyRussG: Controls to purge banner content from front-end cache for a language [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/364910 (https://phabricator.wikimedia.org/T168673) [22:33:02] (PS3) Ejegg: Cache result of template loads, including language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) [22:34:33] (PS4) Ejegg: Cache result of template loads, including language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) [22:37:58] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/364913 [22:47:49] (CR) jerkins-bot: [V: -1] Cache result of template loads, including language fallback lookups [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/364790 (https://phabricator.wikimedia.org/T170435) (owner: Ejegg) [22:48:06] (CR) jerkins-bot: [V: -1] Controls to purge banner content from front-end cache for a language [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/364910 (https://phabricator.wikimedia.org/T168673) (owner: AndyRussG) [23:41:40] cwd do you know where the lint job is defined for our vendor dir? [23:41:43] https://gerrit.wikimedia.org/r/#/c/364622/ [23:42:16] eileen: not right away but i know someone who might...checking [23:42:42] cwd thanks - the failing file looks like a duplicate class for php 5.4 so it makes sense to ignore it [23:43:39] eileen: is this what you are referring to? PHP Fatal error: Cannot redeclare class CallbackFilterIterator in symfony/polyfill-php54/Resources/stubs/CallbackFilterIterator.php on line 28 [23:43:47] ah yeah 54 [23:44:43] yep [23:45:51] cwd we are shooting out (I'm not working today but wanted to see if I could figure out how to progress this) - so can you email me or open a phab if you do get an answer [23:46:08] eileen: no problem [23:46:39] thanks heaps!