[00:00:23] (CR) jerkins-bot: [V: -1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/439795 (owner: Ejegg) [00:00:24] (CR) jerkins-bot: [V: -1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/439795 (owner: Ejegg) [00:03:40] (PS1) Ejegg: Update libraries [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/439797 [00:03:49] (CR) Ejegg: [C: 2] Update libraries [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/439797 (owner: Ejegg) [00:05:05] (CR) Ejegg: [V: 2 C: 2] Update libraries [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/439797 (owner: Ejegg) [00:07:43] (PS2) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/439795 [00:08:32] (CR) Ejegg: [V: 2 C: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/439795 (owner: Ejegg) [00:10:04] (Merged) jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/439795 (owner: Ejegg) [00:16:36] !log updated CiviCRM from 69091d8b5f to f54f9810ab [00:16:40] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:02:33] (PS3) Ejegg: Clean up unused audit regex functions [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/436079 [01:02:57] calling it a night. So long folks! [01:21:47] (PS1) Ejegg: Fix NULLs in activity status filter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/439817 [01:31:41] Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, FR-PayPal-ExpressCheckout, Patch-For-Review: Paypal EC orphan rectifier fails with validation failure - https://phabricator.wikimedia.org/T191654#4274342 (Ejegg) [01:41:35] (CR) Eileen: [C: 2] "Ran fine locally + makes sense (in fact locally my filters were all bool already )" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/439817 (owner: Ejegg) [01:47:16] (Merged) jenkins-bot: Fix NULLs in activity status filter [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/439817 (owner: Ejegg) [02:57:46] (PS1) Eileen: Re-instate batch limit [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439833 [02:59:58] (PS2) Eileen: Re-instate batch limit [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439833 [03:00:00] (PS2) Eileen: Further upstream fixes in 5.3 release [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439788 [15:34:31] fr-tech, I'm trying to track down some log entries which correspond to the failmail generated on our Ingenico one hour test. So far, I've pulled down all logs from May 11th from frlog1001 /src/archive/logs, but I can't seem to find the errors I see in the failmail or any entries around the time of the failures... am I looking in the wrong place? [15:36:59] jgleeson_: let's see, it might be a case of postdated log files [15:37:19] I think the files in /srv/archive/logs often have a date one day later than you might expect [15:37:31] ah ok [15:37:33] I think they're stamped for the date on which they are rotated out, and in GMT [15:37:48] I'll pull down the day before [15:37:53] and give them a search [15:37:55] thanks! [15:37:56] cool [15:38:19] jgleeson_: also, I think we didn't have the log bucketing set up for ingenico till a couple days after that test [15:38:32] log bucketing? [15:38:34] so the ingenico logs probably ended up in payments-misc [15:38:39] ah, got you [15:38:40] rather than payments-ingenico [15:40:28] jgleeson_: OK, yeah, in payments-misc-20180512.gz, I see some lines starting with ingenico_gateway [15:40:53] the day after? [15:41:15] the lines are timestamped May 11 15:12:10 [15:41:36] but the file is dated for the day it was moved from the live file to the zipped archive [15:41:42] are you just zgreppin on the live box to see that? [15:41:46] yeah [15:42:11] cool, thanks [15:42:11] the box doesn't have a lot to do, no worries about zgrepping through lots of stuff [15:42:32] yeah, i wasn't sure how busy the box was [15:42:38] I guess I could of run top [15:42:40] :) [15:42:53] just want to remember to | less so you don't flood your terminal too bad! [15:43:03] worst case, you have to ^C [15:43:26] yeah, looks like puppet is the only thing taking any cpu [15:45:00] ejegg that's a little confusing about the log bucket dating [15:45:42] mepps yeah [15:47:04] looks like logs rotate around 06:30 UTC, so things from 6:30 AM UTC on Day N through 6:30 AM UTC on Day N+1 are in the file named for Day N+1 [15:50:39] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Patch-For-Review: Help switch over foundation pages to donate wiki - https://phabricator.wikimedia.org/T193663#4276004 (mepps) No problem @pcoombe! @Ejegg @DStrine [16:26:25] PROBLEM - Host payments2003 is DOWN: PING CRITICAL - Packet loss = 100% [16:27:25] RECOVERY - Host payments2003 is UP: PING OK - Packet loss = 0%, RTA = 36.26 ms [16:28:20] fr-tech: Looks like I forgot about an appointment so I'll have to miss tech-talk this morning. Someone let me know how it went when I get back. [16:31:13] fr-tech any requests for subjects for tech talk? [16:47:13] XenoRyet fr-tech i thought we'd talked about making this one later so eileen could join? [16:51:06] Mepps: I made the Thursday one in a hopefully Eileen friendly timeslot. I tried to be more Jack friendly with this instance. [16:51:22] XenoRyet i don't think eileen works on Thursdays [16:51:59] The calendar has her off fridays, but if she really is off thursdays too we can switch it. [16:53:12] Though that might be tricky too, since we have the later standup on Tuesdays. [16:53:14] hmm i might be wrong, ejegg do you know? [16:53:39] Anyway, I've gotta run to this appointment, we can figure it out when I get back. [16:54:29] sounds good XenoRyet? [16:54:48] oops i didn't mean the question mark [18:45:58] fr-tech let me know if there's anything more I can explain to help folks review the ingenico audit stuff! [18:46:50] https://gerrit.wikimedia.org/r/435220 https://gerrit.wikimedia.org/r/435286 https://gerrit.wikimedia.org/r/435293 [18:48:29] everything worked for me ejegg, although I was a little unsure of the overall functionality so +1'd to left someone else take a look too [18:48:52] let* [18:50:08] XenoRyet or mepps do you have time to double check the above commits? [18:50:11] thanks jgleeson_ [18:51:01] I'll probably have time to take a look after standup [18:51:06] (CR) Ejegg: [C: 2] Re-instate batch limit [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439833 (owner: Eileen) [18:51:49] unrelated one, fr-tech I stumbled across this today when looking for somewhere to save useful cli commands https://www.commandlinefu.com/commands/browse looks like a good find [18:52:14] ah yeah, that's a fun site! I'd forgotten about it - thanks! [18:57:14] (Merged) jenkins-bot: Re-instate batch limit [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/439833 (owner: Eileen) [19:27:01] oh hey, we're still not sure why those refunds are attached to the wrong installments... [19:30:36] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-Ingenico-integration_2017-18: Ingenico We don't have an order status after doing a GET_ORDERSTATUS. - https://phabricator.wikimedia.org/T194517#4200693 (mepps) Hmm just b... [19:32:51] oh good point ejegg--do we have an issue for it? [19:33:06] mepps ah we just left the refund issue open [19:33:49] hmm maybe it should be it's own so it doesn't get lost in the shuffle? we could also use one for better logging in the refund scripts (at least the paypal one) [19:34:43] (Abandoned) Ejegg: Insert records (mostly dummy) for processors [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/421663 (https://phabricator.wikimedia.org/T165695) (owner: Ejegg) [19:47:47] PROBLEM - Host bellatrix is DOWN: PING CRITICAL - Packet loss = 100% [19:48:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [19:48:32] uhm [19:48:48] Fundraising Sprint Junebugs prefer July, Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Patch-For-Review: Cancel & refund the remaining unintended recurring donations f... - https://phabricator.wikimedia.org/T192958#4276956 [19:50:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [19:53:07] RECOVERY - Host bellatrix is UP: PING OK - Packet loss = 0%, RTA = 36.41 ms [19:53:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [19:54:56] Fundraising Sprint Karma chameleons hide amongst us, Fundraising Sprint Lactose is unusually tolerant, Fundraising-Backlog, Fr-Ingenico-integration_2017-18: Ingenico We don't have an order status after doing a GET_ORDERSTATUS. - https://phabricator.wikimedia.org/T194517#4276979 (mepps) So far I c... [19:55:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [19:58:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [19:58:47] i am confused [20:00:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [20:03:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [20:05:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [20:08:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [20:10:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [20:13:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [20:13:58] well [20:14:24] ipsec tunnel between these boxes is missing [20:15:27] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [20:18:17] PROBLEM - check_ipsec on mintaka is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: alnitak_v4 [20:20:17] PROBLEM - check_ipsec on alnitak is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: mintaka_v4 [20:23:17] RECOVERY - check_ipsec on mintaka is OK: Strongswan OK - 1 ESP OK [20:25:27] RECOVERY - check_ipsec on alnitak is OK: Strongswan OK - 1 ESP OK [20:26:21] ejegg: did you see response on CiviCRM chat ? [20:35:22] thanks eileen, I see the pw reset email now [21:39:48] fr-tech I just realized it's been a month since the ingenico 1 hour test [21:40:23] Wondering if we got any recurring? [21:40:30] XenoRyet: yeah, exactly! [21:40:39] There were definitely some recurring [21:40:53] Yea, let's see if they recur properly [21:41:36] well, we'll need this: https://gerrit.wikimedia.org/r/426068 [21:42:13] I'd love to try that out on a real donation [21:42:32] seems to work for me in the sandbox! [21:43:43] jeez, needs another manual rebase [21:46:11] (PS46) Ejegg: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T188678) [21:49:00] hmm, it'll also need a translation override [21:52:05] (PS1) Ejegg: Reformate Translation.php with drupal style [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440023 [22:04:22] (CR) XenoRyet: [C: 2] "Yea, looks good to me too." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:09:14] (Merged) jenkins-bot: Audit functions can deal with multiple logs for one date [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435220 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:09:19] (CR) XenoRyet: [C: 2] Audit: allow log_filename functions to return arrays [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435286 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:10:16] Fundraising-Backlog: Cannot remove someone from group - https://phabricator.wikimedia.org/T197045#4277295 (NNichols) [22:11:15] (CR) XenoRyet: [C: 2] Ingenico audit handles both APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435293 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:15:55] (Merged) jenkins-bot: Audit: allow log_filename functions to return arrays [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435286 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:16:05] (Merged) jenkins-bot: Ingenico audit handles both APIs [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/435293 (https://phabricator.wikimedia.org/T195337) (owner: Ejegg) [22:42:41] (PS47) Ejegg: SmashPig payment processor extension [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/426068 (https://phabricator.wikimedia.org/T188678) [22:43:51] (PS2) Ejegg: Reformate Translation.php with drupal style [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440023 [22:43:53] (PS1) Ejegg: Add custom ts() override fn to use MW messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440026 [22:54:36] (PS3) Ejegg: Reformat Translation.php with drupal style [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440023 [22:54:38] (PS2) Ejegg: Add custom ts() override fn to use MW messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440026 [22:58:41] man, always forgetting to mark myself back when I come back. [22:59:05] (PS3) Ejegg: Add custom ts() override fn to use MW messages [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/440026 [22:59:45] gotta run an errand, but I can go over the recurring job when I'm back