[11:00:35] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Patch-For-Review: Deploy Thank you letters to remove Lila's signature - https://phabricator.wikimedia.org/T129373#2117539 (jrobell) Thank you @ejegg ! Indeed, it would be great if we could use this deployment to also update the Dutch... [14:12:09] (CR) Ejegg: [C: 2] "Self-merging dependency updates" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276915 (owner: Ejegg) [14:17:38] (PS1) Ejegg: Update SmashPig lib for audit fix [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/277240 [14:18:09] (CR) Ejegg: [C: 2 V: 2] "self-merging lib update" [wikimedia/fundraising/crm/vendor] - https://gerrit.wikimedia.org/r/277240 (owner: Ejegg) [14:20:48] (PS1) Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/277243 [14:23:32] (CR) Ejegg: [C: 2 V: 2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - https://gerrit.wikimedia.org/r/277243 (owner: Ejegg) [14:53:03] !log updated civicrm from 7e21d5ad1f9ff404ac155a38f771b744bf238ccf to 090d443c856574d45c80f89c4ae7ccb86c97448f [14:53:07] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [16:19:16] (PS6) Cdentinger: WIP: move IDEAL to mustache forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/274334 [16:19:34] (PS7) Cdentinger: WIP: move IDEAL to mustache forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/274334 [16:21:40] (CR) jenkins-bot: [V: -1] WIP: move IDEAL to mustache forms [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/274334 (owner: Cdentinger) [16:33:46] Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Allow fallback to alternative campaign if mixin would hide this one - https://phabricator.wikimedia.org/T124969#2118853 (Pcoombe) @AndyRussG Any idea how much work this would be for you? [16:56:11] Having a nicely rounded pi day? [16:56:42] happy pi day to you too awight! [16:57:01] how was the New York trip? [16:58:23] A lot of loud blustering, otherwise brief and silly. I do <3 NYC, though [16:59:12] Hehe, trains, planes and automobiles treat you OK? [17:01:32] Thanks, it was long but uneventful [17:01:48] nice [17:01:55] I recommend against taking a redeye on spring-forward eve, btw [17:02:22] oof, that one snuck up on me even without travel to exacerbate [17:02:39] so glad you could make it up my way [17:04:40] More of that when we relocate the WMF HQ to Slummerville ;) [17:06:18] (PS2) Ejegg: Call static Language::fetchLanguageName() statically [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/276641 (owner: Siebrand) [17:12:47] (CR) Ejegg: [C: 2] "Consistent with core usage in 1.25 and trunk" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/276641 (owner: Siebrand) [17:24:31] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: Civi: multiple errors when cancelling recurring Ingenico donations, and with refunds reaching Civi - https://phabricator.wikimedia.org/T128389#2119129 (MBeat33) p:Unbreak!>High I'm lowering the prio... [17:31:10] hello all! we okay to run an adyen test starting in about 10 minutes? [17:31:17] ejegg: ^ [17:31:19] works for me! [17:33:44] cool, thanks ejegg. I've set it to go up at 1740 UTC, and will be watching for results in civi [17:34:46] great! They should take about 10 minutes to get through the auth -> IPN -> capture -> civi pipeline [17:35:38] cool. let me know if you see any problems on your end [17:38:43] will do the-wub [17:40:52] ejegg jessicarobell ppena MBeat: Adyen banner is up! [17:41:01] wohoo [17:41:04] cool [17:41:06] thanks [17:41:08] thanks the-wub! :) [17:41:17] ty [17:41:49] I'll be putting results here: https://docs.google.com/spreadsheets/d/17arGLTPupnrCmo7O0He38js-gmOoF3OY53sOwiAbAp0/edit#gid=2003594159 [17:45:19] seeing the first nibbles [17:49:59] ejegg: I see 32 trnx and 22 still only authorized [17:50:02] is that normal? [17:50:13] ppena: yep, we run the capture job every 5 minutes [17:50:21] gotcha [17:51:15] on the 2's and 7's [17:51:51] we had a pretty slow start too (for all methods), think there must have been some banner caching delays [17:52:18] yeah, it was almost 4 min before I saw anything hit payments [17:52:39] i think the resourceloader caches for 5 min [17:56:20] ejegg: still no adyen donations in civi :( [17:56:46] no errors yet, they should come in soon [18:01:22] the-wub: hmm, the job runner looks like it's only trying to process 10 jobs each run - let me see where to turn that up [18:02:25] ah, I see it. I'll turn that up [18:04:26] first 5 in civi! [18:04:53] !log increased max jobs per run for Adyen job runner from 10 to 250, time limit from 60 to 120 [18:04:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:05:15] the-wub: those were some pretty low defaults! It should keep pace better now [18:08:25] (PS1) Ejegg: Don't log exception on refused CC authorization [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/277294 [18:08:50] cwd: ^^^ gets rid of some overzealous logging [18:09:26] dstrine: https://collab.wikimedia.org/wiki/Things_that_shouldn't_be_disclosed_publicly [18:10:15] ejegg: we don't want to bail on that response? [18:11:03] cwd failure is enough [18:11:12] ut-oh, job runner is doing something weird [18:11:20] going to set those limits back down [18:11:58] huh, TIL the gmail subject line character limit [18:12:13] (CR) Cdentinger: [C: 2] Don't log exception on refused CC authorization [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/277294 (owner: Ejegg) [18:12:51] ejegg: I'd recommend a 3 or 4 minute suicide alarm, like in the queue consumers [18:14:18] awight: Don't think it was a timeout thing, I think I just need to ack a message in another spot [18:14:30] ...and leave a logging context, or something [18:15:47] ah, cool. Still, we need to prevent overlapping job runners [18:18:12] (PS1) Ejegg: Ack and requeue when leaving authorization for review [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/277298 [18:19:10] awight: yeah, there's a --time-limit param for StompJobRunner. I've got it at 2 min, plenty short enough for the 5 minute repeat interval [18:19:32] cwd or awight, that patch should stop the exception^^^ [18:19:50] I'll have something for that atrocious log tag in a sec [18:21:48] (CR) Awight: [C: 2] Ack and requeue when leaving authorization for review (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/277298 (owner: Ejegg) [18:22:26] thanks awight [18:24:07] I think we're not in danger from the queue positions, since we're subscribing with a selector, right? Shouldn't that filter on the server side and give us the first matching message no matter how far back it is? [18:24:44] (Merged) jenkins-bot: Don't log exception on refused CC authorization [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/277294 (owner: Ejegg) [18:26:33] (Merged) jenkins-bot: Ack and requeue when leaving authorization for review [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/277298 (owner: Ejegg) [18:27:27] !log updated SmashPig from ad27a3a18cccd6bbd843e671218a1f5c190bdb8a to 9f08f6a1891b0a2bb70eacf460c2f9a8153c3b4e [18:27:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log, Master [18:30:34] (CR) Pcoombe: [C: -1] "There's a few paragraphs that have gone back to English for some reason, in Norwegian and Ukrainian. I re-published those translations on " (4 comments) [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276808 (https://phabricator.wikimedia.org/T129373) (owner: Ejegg) [18:42:31] jessicarobell ppena MBeat: Adyen test is done. ejegg is looking into some donations that were delayed [18:42:46] thank you the-wub [19:19:07] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Create drush job to requeue messages in bulk - https://phabricator.wikimedia.org/T129913#2119746 (Ejegg) [19:50:27] Fundraising Sprint Elevator Maintenance 2016, Unplanned-Sprint-Work, WMF FR: Ingenico ePayments Partners with Trustwave for PCI Compliance - https://phabricator.wikimedia.org/T129596#2119835 (atgo) @ppena please add all tasks to #fundraising-backlog in the future. thanks! [19:50:38] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, WMF FR: Ingenico ePayments Partners with Trustwave for PCI Compliance - https://phabricator.wikimedia.org/T129596#2119837 (atgo) [20:11:23] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, WMF FR: Ingenico ePayments Partners with Trustwave for PCI Compliance - https://phabricator.wikimedia.org/T129596#2119941 (awight) @Ppena is this the same as T129597? [20:14:17] Fundraising-Backlog, FR-Worldpay: Worldpay Audit Parser should catch more stuff - https://phabricator.wikimedia.org/T129917#2119949 (cwdent) [20:33:36] (PS3) Ejegg: Thank You letters for executive director transition [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276808 (https://phabricator.wikimedia.org/T129373) [20:34:46] Fundraising-Backlog, FR-Worldpay: Worldpay Audit Parser should catch more stuff - https://phabricator.wikimedia.org/T129917#2120036 (Ejegg) [20:34:48] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, FR-Smashpig, FR-Worldpay: Add 'Cancelled' type for refund messages - https://phabricator.wikimedia.org/T129714#2120037 (Ejegg) [20:41:13] ejegg: whoops, did i file a dupe? [20:41:43] they could definitely be separate tasks [20:42:26] if you think so...but i won't be offended if you merge em :) [20:53:28] awight: awesome, there are both 'EmptyQueueToDump' and 'PopulateQueueFromDump' scripts [20:54:17] RAD [20:54:26] * awight lifts a chalice to mwalker [20:54:59] gonna review a hardcopy for a few [20:55:09] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Unplanned-Sprint-Work: Create drush job to requeue messages in bulk - https://phabricator.wikimedia.org/T129913#2120085 (Ejegg) Open>declined @awight points out there already exist a pair of Sma... [21:05:44] gonna do that queue stuff as a jenkins 'one-off' [21:13:59] dang, "Decoded line does not have headers and body elements:"... [21:14:07] efff [21:15:59] ah, just need to dump as 'raw' [21:16:09] tricksy [21:16:33] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, WMF FR: Ingenico ePayments Partners with Trustwave for PCI Compliance - https://phabricator.wikimedia.org/T129596#2120194 (Ppena) @atgo Sorry I really thought I did... I followed some old examples on how to cr... [21:17:18] BTW. I stumbled across a, 1800s occurrence of "tricksy": https://books.google.com/books?id=f1c8dV_6Kc0C&pg=PA347&lpg=PA347&dq=tricksy+orb&source=bl&ots=KAoln2hZpG&sig=upx3P7cMUItQc7VCIBNQSUJmxUc&hl=en&sa=X&ved=0ahUKEwjDvO7ZjMHLAhUM62MKHQ2BBC8Q6AEIIzAB#v=onepage&q=ambrose%20bierce%20tricksy%20orb&f=false [21:17:27] I guess he used it in a few places [21:17:40] ambrose bierce, nice [21:17:47] oh I'm an idiot, nvm [21:17:48] https://books.google.com/ngrams/graph?year_start=1800&year_end=2008&corpus=15&smoothing=7&case_insensitive=on&content=tricksy&direct_url=t4%3B%2Ctricksy%3B%2Cc0%3B%2Cs0%3B%3Btricksy%3B%2Cc0%3B%3BTricksy%3B%2Cc0 [21:18:17] It's so hot right now :D [21:18:21] hehe [21:23:22] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [epic] Make deduping reversible - https://phabricator.wikimedia.org/T111704#2120246 (Eileenmcnaughton) @Jgreen how feasible is the sort of extra disk use mentioned above? https://phabricator.wikimedia.org/T111704#2110819 [21:31:23] (PS1) Eileen: CRM-18213 remove hard-coding of DAO names [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/277355 [21:31:25] (PS1) Eileen: CRM-18213 remove hard-coded DAO names [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/277356 [21:40:03] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [epic] Make deduping reversible - https://phabricator.wikimedia.org/T111704#2120273 (awight) > CiviCRM supports the log tables being in a different database. That seems like a great idea... Also, we could potentially make our culling strategy to simpl... [21:42:33] eileen1: Your audit logging investigation looks wildly succesful! [21:42:53] lmk if I can give any specific feedback beyond the comments I just left on the task [21:44:10] awight: just repling to those comments. Sounds like there using the logging is feasible [21:44:35] a major upside and downside is it records a lot more information than we need for this purpose [21:44:56] I've been itching for full audit logging since you mentioned it... two years ago, maybe? [21:45:05] That'll be a huge debugging aid, IMO [21:45:24] Users are becoming afraid of working in multiple tabs, which is really a shame [21:45:46] I'd like to track that down, and I'm sure audit logging would tell us exactly what that's about [21:47:35] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, WMF FR: Ingenico ePayments Partners with Trustwave for PCI Compliance - https://phabricator.wikimedia.org/T129596#2120296 (awight) Thank you for the note! smb://filesrv1/fundraising/Tech/PCI/SAQ_A_v3_2015.pdf [21:48:32] awight: yeah - it's great for forensics! [21:49:19] One think I LIKE about the archive table format is that it doesn't roll back - so you can catch a glimpse of anything that has rolled back [21:49:30] although I don't think it's a good enough reason to use it [21:49:51] Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM: [epic] Make deduping reversible - https://phabricator.wikimedia.org/T111704#2120303 (Eileenmcnaughton) 1) Culling - cold storage might make sense too. In general I feel we don't have to solve this yet 2) Yep - I'm still getting around 300 writes per sec... [21:50:55] eileen1: uh, oh, that brings up another issue [21:51:09] it's tricksy to roll back across multiple databases [21:51:22] I doubt it's already implemented correctly [21:51:38] so, this might be a problem if we wanted to store innodb audit logs in a separate db [21:52:31] hmm - interesting. Do you have any links on that - I haven't hit issues before but we have a lot of tables still in archive or in the same DB on Fuzion customers [21:53:01] and, i'm sure your mom is okay but it's *you* i'm worried about... https://en.wikipedia.org/wiki/By_His_Bootstraps [21:53:18] :-) [21:53:31] https://en.wikipedia.org/wiki/Distributed_transaction [21:53:37] The issue is during rollback [21:53:51] you could also be concerned that 7 clones of my mum exist in the DB [21:54:08] say you begin transactions on db A and B, then discover that you have to rollback. you rollback A, but something dies before you can rollback B [21:54:18] oh. commit is an issue as well [21:54:33] cos, you're simulating a distributed transaction by committing A and then B [21:54:42] it's very easy to commit in A, but not in B [21:55:20] I think the problem is impossible to solve perfectly without using server-side two-phase commit support [21:56:00] We're making the same error in crm/modules/wmf_common/Queue.php already... so we're in good company if we do decide to punt on this one [21:56:33] this is a better article, https://en.wikipedia.org/wiki/Two-phase_commit_protocol [21:57:04] MySQL does support it, but I hope we don't have to go there. http://dev.mysql.com/doc/refman/5.7/en/xa.html [21:58:29] Fundraising Sprint Elevator Maintenance 2016, Fundraising-Backlog, Unplanned-Sprint-Work, FR-Smashpig: Only capture a single payment against any order ID - https://phabricator.wikimedia.org/T129935#2120404 (Ejegg) [21:58:58] eileen2: 'scuse my ignorance, but does that @uniqueID persist across other queries from the same process? [21:59:11] hum. it must [21:59:14] cool! [22:02:00] awight: yes - the uniqueid works for the whole process - but I think we can add the option to revert to revert a limited part of the connection if we want [22:02:20] re the rollback [22:03:02] If the result is that only the main DB gets rolled back & not the logging I don't think that is a bad outcome [22:03:30] we are left with extra forensic information and the risk of that being misused seems very low to me [22:03:45] but if the risk is corruption of some kind then that is different [22:14:52] ejegg: it turns out rtbt has 5 submethods, so once you submit the form if there are errors or whatever it comes back to a form with all the buttons (this is the thing the show_submethod patch was about iirc) [22:15:32] i'm not sure what the best way around that is. i could hack the submethods array but it feels silly since this is a brand new feature [22:16:00] hmm, that's tricky [22:16:18] maybe we should be able to manually set show_submethods? [22:16:29] i could set it based on ffname [22:16:56] cwd maybe show_submethods could be a property of the method array? [22:17:45] that seems good, unless there might be an instance where the same method was used by multiple forms in different configurations [22:20:01] not sure about the other rtbt submethods [22:21:33] like, are they even a thing? [22:21:49] beats me! [22:21:58] yeah me too [22:22:02] could be cruft [22:24:27] (CR) Pcoombe: [C: 2] "Looks good, thanks!" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/276808 (https://phabricator.wikimedia.org/T129373) (owner: Ejegg) [22:46:29] (PS1) Ejegg: Cancel duplicate authorizations for a single order ID [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/277427 (https://phabricator.wikimedia.org/T129935) [22:50:43] (PS1) Ejegg: Add Adyen Cancellation IPN message class [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/277428 [22:52:40] hey all- my internet is dubious atm - I jut logged a fault - so if I drop in & out that's why :-( [22:53:11] sheep at the wires again... [22:53:56] :-) [22:53:56] maybe a possum… [22:54:33] it's always a bit dubious calling in faults because we always wonder if they are going to come out & say the trees on our property are causing interference [22:54:43] yikes! [22:54:52] but so far they have been external - other people's possums :-) [22:55:02] I am *not* down with OPP [22:56:03] just so you know possums in en_AU = "cute furry creatures" in en_NZ it means "noxious nasty pests" [22:59:03] is that because NZ is not filled with as many horrible beasts to compare them to? [22:59:16] :D [22:59:32] hey, the AU people are okay [23:00:05] just seems like a good place to get killed by the wildlife [23:00:41] if you make it out of the football stadium [23:01:12] https://www.youtube.com/watch?v=QTYID4lXhM0 [23:02:50] possums are native to AU but introduced to NZ & they do lots of damage here [23:03:24] Fundraising-Backlog, FR-Smashpig: Adyen job runner log prepending too many IDs - https://phabricator.wikimedia.org/T129946#2120639 (Ejegg) [23:16:14] Fundraising-Backlog: Scope remaining PCI work - https://phabricator.wikimedia.org/T129948#2120713 (atgo)