[01:53:02] (PS1) Eileen: FIx geocoder extension install to NOT use mgd methodology. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/420628 [01:54:25] Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, Fr-CiviCRM-dedupe-FY2017/18: Fix issues with Geocoder extension enabling disa... - https://phabricator.wikimedia.org/T190119#4063508 [01:54:57] (PS2) Eileen: FIx geocoder extension install to NOT use mgd methodology. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/420628 (https://phabricator.wikimedia.org/T190119) [02:17:44] Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, and 2 others: Step 1 , 2 and 3 of engage import - https://phabricator.wikimedia.org/T189617#4063569 (Eileenmcnaughton) I fixed up... [02:40:44] Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Missing sql fn gives a warning whenever we edit a comme... - https://phabricator.wikimedia.org/T188931#4063635 [02:41:07] Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Missing sql fn gives a warning whenever we edit a conta... - https://phabricator.wikimedia.org/T188931#4063636 [03:47:35] Fundraising-Backlog, fundraising-tech-ops: CiviCRM sql user has access to fredge on live but not staging - https://phabricator.wikimedia.org/T189760#4063718 (Eileenmcnaughton) @cwdent I'm a bit blocked on this - can you look tomorrow - forgot to ping you at a reasonable time today [03:49:30] eileen: hey, i am actually around doing a couple things [03:49:38] i can probably do that one [03:51:06] eileen: oh heh [03:51:11] # PRODUCTION SLAVE! DON'T GRANT WRITE PRIVS! [03:51:13] thanks jeff [03:56:41] cwd - thanks - I only need the one with dev_ in front which is not a slave [03:56:49] but reflects prod [03:57:33] oh right sorry [03:58:07] eileen: looks like civicrm user should already have all privs there [04:08:17] hmm [04:10:09] cwd - just did drush cvapi & then show databases & don't see it [04:10:14] on staging [04:10:20] (I do on prod) [04:12:05] | GRANT ALL PRIVILEGES ON `dev_fredge`.* TO 'civicrm'@'localhost' [04:12:18] eileen: ^ [04:12:21] that is in place [04:12:26] what are you seeing exactly? [04:17:42] cwd user is dev_civicrm [04:20:26] ah ha [04:20:47] eileen: ok looks like that user has: [04:20:51] select,insert,update,delete,create,drop,index,alter,create temporary tables,lock tables,trigger,alter routine,create routine [04:20:56] what is missing from there? [04:21:31] this is what I see [04:21:32] [nativecode=1142 ** SELECT command denied to user 'dev_civicrm'@'127.0.0.1' for table 'payments_fraud'] [04:21:53] & if I use drush to open mysql as that user I don't see dev_fredge as a db [04:22:16] (I see civi & drupal dev dbs though) [04:24:28] huh [04:32:20] eileen: ok i think i see now [04:32:33] should the civicrm user have access to dev_fredge? [04:32:41] or only dev_civicrm? [04:47:13] Fundraising-Backlog, fundraising-tech-ops: CiviCRM sql user has access to fredge on live but not staging - https://phabricator.wikimedia.org/T189760#4063733 (cwdent) Open>Resolved a:cwdent Turned out it was dev_civicrm user on dev_fredge db, should work now, re-open if not [11:36:26] Fundraising-Backlog, FR-Amazon: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4064304 (Pcoombe) [14:39:45] Fundraising-Backlog: Ingenico audit wobble? March 13 refunds not in Civi - https://phabricator.wikimedia.org/T190098#4064886 (MBeat33) Also GC 5771315413 refund from 3/9 is not in Civi [16:30:07] (PS1) Ejegg: WIP emit CSP headers on banner previews [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/420754 (https://phabricator.wikimedia.org/T190100) [16:30:25] oh huh, guess it's just the web UI that's down? [16:38:28] wow, there are some SNEAKY freaking ways to leak data to 3rd parties: https://github.com/cure53/HTTPLeaks [16:40:56] Fundraising-Backlog: Enable Content Security Policy on payments-wiki - https://phabricator.wikimedia.org/T190183#4065538 (Ejegg) [17:00:21] Fundraising-Backlog, Google-Summer-of-Code (2018): GSOC proposal - Machine learning fraud detection - https://phabricator.wikimedia.org/T190103#4065597 (DStrine) [17:04:15] ejegg|food: quick work there! nice! [17:04:32] I was thinking about how Varnish cashing might be impacted, but I assume not at all [17:04:39] Since logged-ins go straight to PHP [17:05:24] And I think nothing that's cache for a logged-in is sent to an anon [17:17:19] AndyRussG: not at all compatible with the core csp patch unfortunately [17:30:31] (CR) jerkins-bot: [V: -1] WIP emit CSP headers on banner previews [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/420754 (https://phabricator.wikimedia.org/T190100) (owner: Ejegg) [17:31:26] ejegg|food: hummm... link? [17:31:37] Man jenkins needs coffee today [17:54:09] Fundraising-Backlog, FR-Amazon: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4065925 (DStrine) p:Triage>High [17:54:49] fr-tech has anyone looked into this yet? https://phabricator.wikimedia.org/T190136 [17:55:16] i can look at that dstrine [17:55:57] Thanks mepps per our usual priority order this is not unbreak now but high priority [17:56:13] We're obligated to get this back up during regular business hours [17:56:39] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4065943 (DStrine) [17:58:36] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4064304 (DStrine) This is not Unbreak Now but it is high priority during regular working hours. I'm adding to the spring for t... [18:06:46] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", ... - https://phabricator.wikimedia.org/T185933#3929004 [18:07:03] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4066049 (Pcoombe) I've removed Amazon from donatewiki and Ways to Give for now. We were getting about 10-20 Amazon attempts pe... [18:11:01] AndyRussG: https://gerrit.wikimedia.org/r/253969 [18:11:37] ejegg: thx! [18:12:26] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", ... - https://phabricator.wikimedia.org/T185933#4066079 [18:39:53] (PS1) Ejegg: Update Amazon JS for new widget ready event [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/420808 (https://phabricator.wikimedia.org/T190136) [18:40:07] oops, just saw that mepps was already on the case.... [18:40:20] mepps that seems to fix it for me locally.... ^^^ [18:40:33] nice ejegg! [18:41:58] (CR) Mepps: [C: 2] Update Amazon JS for new widget ready event [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/420808 (https://phabricator.wikimedia.org/T190136) (owner: Ejegg) [18:44:40] (Merged) jenkins-bot: Update Amazon JS for new widget ready event [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/420808 (https://phabricator.wikimedia.org/T190136) (owner: Ejegg) [18:44:45] mepps do you have that working locally? [18:45:17] it's a bit of a pain.. need https, and also to register your personal URL as an authorized referrer in their sandbox console [18:45:19] i do now ejegg, but is there a test amazon account? [18:45:28] ahh gotcha [18:45:30] yeah, lemme see [18:45:40] i only have the redirect working [18:45:44] not https [18:45:54] oh right, you can create multiple test users in their console [18:46:17] i approved your change becuase of the high priority but was going to go ahead and test too [18:47:51] fr-tech anyone else want to mess with getting amazon working locally? [18:48:51] XenoRyet do you have a local install? [18:49:14] I was just checking on where my local DI is at right now. [18:49:33] I have the notion that I was midway through a vagrand rebuild [18:50:05] XenoRyet: Do you have https working with the vagrant / payments bit? [18:51:04] Not yet if it doesn't work out of the box with vagrant. [18:51:17] I think there's an https role [18:51:24] I'll try it and see what happens. [18:51:29] just not sure if it covers the payments one too [18:51:40] I think where I left off I still need to provision for fundraising role, so I'll do that first. [18:52:24] Oh wait, that's not right. [18:52:50] Let me do some reprovisioning and such [18:56:24] Fundraising-Backlog: Enable Content Security Policy on payments-wiki and donate-wiki - https://phabricator.wikimedia.org/T190183#4066275 (Ejegg) p:Triage>Normal [19:01:50] well, seeing as how it's already broken in prod, I'mma go ahead and deploy that fix without further messing about [19:04:04] Seems reasonable. [19:04:25] (PS1) Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/420817 [19:04:49] But is still a good reminder for the rest of us to check up on our local installs. [19:11:29] (CR) Ejegg: [C: 2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/420817 (owner: Ejegg) [19:11:41] (PS1) Ejegg: Update DonationInterface [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/420820 [19:11:53] (CR) Ejegg: [C: 2] Update DonationInterface [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/420820 (owner: Ejegg) [19:12:19] (Merged) jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/420817 (owner: Ejegg) [19:16:36] (Merged) jenkins-bot: Update DonationInterface [core] (fundraising/REL1_27) - https://gerrit.wikimedia.org/r/420820 (owner: Ejegg) [19:23:09] !log updated payments-wiki from 30f5f3edfb to 9e83e7f7a0 [19:23:13] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:27:18] ok, that looked good from the front-end [19:27:24] let's see if the IPN comes in... [19:32:35] `yes! [19:34:43] (CR) AndyRussG: WIP emit CSP headers on banner previews (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/420754 (https://phabricator.wikimedia.org/T190100) (owner: Ejegg) [19:35:15] AndyRussG: sorry, didn't mean for you to waste time on that one just yet... [19:36:12] ejegg: oh no worries, just getting my feet wet with this stuff [19:38:45] I'm gonna see how bawolff feels about splitting up the core patch [19:40:31] into one that makes the headers available [19:40:41] for enablement on selective pages [19:40:56] and another that turns them on sitewide [19:42:55] (PS8) Jgleeson: T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 [19:49:19] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Fr-Ingenico-integration_2017-18, Patch-For-Review: Ingenico adapter: parse response errors - https://phabricator.wikimedia.org/T176502#4066423 (jgleeson) Latest version here: https://gerrit.wikimedia.org/r/#/c/415907/ Th... [19:49:22] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon, MW-1.31-release-notes (WMF-deploy-2018-03-27 (1.31.0-wmf.27)), Patch-For-Review: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4064304 (Ejegg) @pcoombe thanks for no... [19:59:44] (PS3) Eileen: FIx geocoder extension install to NOT use mgd methodology. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/420628 (https://phabricator.wikimedia.org/T190119) [19:59:51] (CR) jerkins-bot: [V: -1] FIx geocoder extension install to NOT use mgd methodology. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/420628 (https://phabricator.wikimedia.org/T190119) (owner: Eileen) [20:01:10] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon, MW-1.31-release-notes (WMF-deploy-2018-03-27 (1.31.0-wmf.27)), Patch-For-Review: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4066446 (Pcoombe) Open>Resolved... [20:04:34] (PS9) Jgleeson: T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 [20:05:40] mepps, I just pushed an update to the previous commit with a link to some related notes on the *latest* attempt. Do you have a few minutes now to chat about the payment errors stuff you originally added coverage for? I just wanna confirm I need to port the new code over to that also before adding it [20:06:01] notes/comment is here: https://phabricator.wikimedia.org/T176502#4066423 [20:06:31] i'm confused jgleeson, i don't think i added any coverage... [20:06:48] coverage in the broadest sense [20:07:04] so basically I think you added a check for the key payment.statusOutput.errors [20:07:05] i'm not actually sure what' you're referring to [20:07:53] you were frustrated with the inconsistencies of the error locations, or the number of them [20:08:02] so you had to check for multiple keys [20:08:05] one of the was: payment.statusOutput.errors [20:08:14] I'll find the link to your patch [20:08:20] let me look at hwat you've done [20:08:21] hopefully make more sense [20:08:25] but your past patches had that covered [20:08:41] yeah... but the way that works has changed now so it's not there [20:09:02] and I can't find any existing code looking for that via test or process [20:09:14] that specific key "payment.statusOutput.errors" in isolation [20:09:33] so I wanted to double check if was still required before I port over the new stuff to IngenicoPaymentProvider [20:11:01] this is your original code mepps: https://gerrit.wikimedia.org/r/#/c/415085/8/PaymentProviders/Ingenico/Api.php@72 [20:11:13] oh so it's needed in the payment approval calls [20:11:36] which is called in ingenicopaymentprovider [20:11:37] Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, FR-Amazon, MW-1.31-release-notes (WMF-deploy-2018-03-27 (1.31.0-wmf.27)), Patch-For-Review: Amazon Pay is broken with javascript error - https://phabricator.wikimedia.org/T190136#4066498 (Ejegg) Postmortem: We need to... [20:12:01] which should probably jsut be called payment... [20:12:11] jgleeson ^^ [20:12:17] ok cool [20:12:23] there are no tests though [20:12:27] for the errors i mean [20:13:31] so the error case for that, is it intended to be detected in the response of getPaymentStatus or approvePayment [20:14:24] in the latest version, we are checking for it in getHostedPaymentStatus here https://gerrit.wikimedia.org/r/#/c/415907/9/PaymentProviders/Ingenico/HostedCheckoutProvider.php@51 [20:14:46] sorry, in the latest version of the hosted payments status errors checking*** [20:19:11] mepps ^ [20:19:49] jgleeson getPaymentStatus and approvePayment are different calls tahn getHostedPaymentStatus [20:20:02] yup [20:20:29] i think either call jgleeson--the idea here is to catch any errors that come back from an api call [20:20:37] so I'm trying to work out which one of them calls (getPaymentStatus or approvePayment) did you expect the error key in question to be populated [20:20:47] ah ok, so both? [20:21:32] (PS4) Eileen: FIx geocoder extension install to NOT use mgd methodology. [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/420628 (https://phabricator.wikimedia.org/T190119) [20:22:19] Fundraising Sprint Cottage Cheese isn't Made of Cottages, Fundraising Sprint Dinosaur Cookies co-existed with Gingerbread People, Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", an... - https://phabricator.wikimedia.org/T186883#4066532 [20:23:22] yeah jgleeson [20:23:57] I'm not sure if that's right [20:24:32] are you free for a few minutes for me to explain on hangouts? [20:25:16] nvm mepps seen you have to pop out, I'll catch you later/tomorrow and will explain more clearly what I'm trying to do [20:27:47] hey ejegg, would happen to be familiar with the area I'm asking mepps about? [20:28:32] * ejegg reads backscroll [20:30:22] jgleeson: so, the Api class is now just checking for the top-level errors, right? [20:30:22] I'm trying to confirm whether or not we need to check for payment status errors, on each API call. To me it looks like we may only need to do that on the actual status methods exposed by the ingenico API, and for the other methods, we get the more high level error response from ingenico when something goes wrong [20:31:01] yes ejegg, that is happening here https://gerrit.wikimedia.org/r/#/c/415907/9/PaymentProviders/Ingenico/Api.php@82 [20:31:04] jgleeson: yeah, totally, we should only check for those on the specific API calls where we expect them [20:31:11] and not in the API class [20:31:32] but in the IngenicoPaymentProvider class or subclass [20:31:40] that knows about that particular type of payment [20:33:06] jgleeson: yeah, like you're doing in the HostedCheckoutProvider class there [20:33:14] that's looking merge-able... [20:35:26] great. so my question above to mepps was around the specific errors relating to payment.statusOutput.errors. This is another key that mepps added coverage for in the original API class checkErrors() method but now that we've split it out that has gone. To me, that key 'payment.statusOutput.errors' looks like it related to the call getPaymentStatus() in IngenicoPaymentProvider and only that [20:36:43] and if that is the case it means I just need to port the new stuff in HostedCheckoutProvider over to IngenicoPaymentProvider for that one method [20:38:04] looking at the docs... [20:39:19] that was my next move also but I've not had great success with status codes and ingenico documentation so far so thought I'd ask here first :) [20:40:50] jgleeson ok, trying to figure out how much of the getHostedCheckoutStatus response is specific to hosted checkouts [20:41:48] the SDK object type 'Payment' seems to be something we'd want to understand in the IngenicoPaymentProvider class [20:42:35] though it would only be in the HostedPaymentProvider class that we would say to look for the payment object under createdPaymentOutput [20:43:25] looking at that here https://github.com/Ingenico-ePayments/connect-sdk-php/blob/master/src/Ingenico/Connect/Sdk/Domain/Payment/Definitions/Payment.php [20:44:51] mepps mentioned that it's needed/used/expected in the payment approvals [20:45:09] which does make sense as that API module also has a getPaymentStatus method [20:48:13] cwd - mysql perms working well now - thanks [20:48:31] eileen: great! yw [20:49:27] ejegg, the existing tests we have for the payments approval stuff does have a mock response which does line up with with the payment.statusOutput structure https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/master/PaymentProviders/Ingenico/Tests/Data/paymentApproved.response [20:50:48] so it's likely there is an 'error' block when we encounter an error scenario on the getPaymentStatus call [20:51:07] I guess I need to test it on the sandbox to confirm [20:52:58] cwd - re that sql function - does the function exist & civi user cannot see it - or does it not exist? [20:53:51] eileen: it exists, it's at the top of the trigger file [20:54:07] i think we just need to give the user permission [20:54:26] ok - yeah I think that makes sense manually [20:54:41] it should only need to be done once - I'm confused why it needs doing now [20:54:52] user runs [20:54:53] SHOW function status WHERE db = database() AND name = 'civicrm_strip_non_numeric' [20:54:56] jgleeson OK, does it still seem that any of the 'error' statuses inside that payment block are more 'errorResponse' than 'apiException' problems? [20:55:21] eileen where do we keep track of the db grant scripts? [20:56:36] ejegg: we track the trigger scripts but I'm not aware of DB grant scripts being tracked - unless ops do [20:57:15] yeah we keep track of grants in puppet [20:57:18] yeah, I couldn't find any [20:57:28] it's in the private repo [20:57:34] cwd ahhh, in that case we should add that fn to the grant scripts [20:57:50] instead of the trigger file? [20:58:01] do you mean the function definition or the grant for the civi user? [20:58:06] that's the thing, I can't really tell yet because I'm not sure if the 'errors' block will contain the same type of errors we seen in the corresponding hostedCheckouts block or not [20:58:19] cwd the grant for the civi user [20:58:23] reading here, seems to suggest they will be the same ejegg https://epayments-api.developer-ingenico.com/s2sapi/v1/en_US/java/statuses.html [20:58:32] yep sounds good [20:59:28] jgleeson OK, those definitely are not exception-worthy [20:59:42] rejected is a normal part of the flow [21:00:14] so, let's have a function in the paymentprovider class that understands that block and normalizes their statuses to ours [21:00:17] my thinking is that they warrant the same treatment as the HostedPaymentsStatus errors, which is just add them to the response [21:00:23] yeah, totally [21:00:31] just to* [21:03:44] ooh, dang, some of those DO sound Exception-worthy... [21:03:44] where are we using the calls in https://github.com/wikimedia/wikimedia-fundraising-SmashPig/blob/master/PaymentProviders/Ingenico/IngenicoPaymentProvider.php ejegg ? [21:03:55] IO_ERROR, hmm [21:04:25] jgleeson it's lame, we're putting those in the curl_transaction block inside the ingenico connect adapter [21:04:53] I'm confused [21:05:10] do you have a minute to jump on hangouts? [21:05:28] https://github.com/wikimedia/mediawiki-extensions-DonationInterface/blob/master/ingenico_gateway/ingenico.adapter.php#L154 [21:05:38] jgleeson: sure, be on in a sec [21:10:58] Fundraising-Backlog, Google-Summer-of-Code (2018): GSOC proposal - Machine learning fraud detection - https://phabricator.wikimedia.org/T190103#4066738 (srishakatux) @AKlapper Yes, this is not from the project featured in the list. My sense is that @Eileenmcnaughton and Saurabh have been jointly discussi... [21:16:28] Fundraising-Backlog, Google-Summer-of-Code (2018): GSOC proposal - Machine learning fraud detection - https://phabricator.wikimedia.org/T190103#4066772 (Eileenmcnaughton) I'm a bit on the fence about whether this should go to Civi or WMF as the GSOC org & was leaning to suggesting the student put it to b... [21:19:12] (CR) Ejegg: T176502 WIP: Updated Ingenico error handling (1 comment) [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 (owner: Jgleeson) [21:19:15] cwd I think staging is getting to the point of needing a sync from live [21:24:26] ah yeah been a while for sure [21:24:44] eileen: do you want me to do that grant? [21:24:54] Fundraising-Backlog, Google-Summer-of-Code (2018): GSOC proposal - Machine learning fraud detection - https://phabricator.wikimedia.org/T190103#4066801 (srishakatux) @Eileenmcnaughton I think you could make the best decision here in which organization this project would best fit in. Also, it is a good id... [21:25:31] cwd yes please [21:29:36] cwd - just realised the fredge DB is REALLY out of date [21:29:51] on dev? [21:29:55] more than the others? [21:32:51] AndyRussG: bd808 was already thinking about the possibilities for opting in gadget sources: https://phabricator.wikimedia.org/T130748#2320760 [21:33:22] cwd yep [21:33:31] ejegg: cool! [21:33:57] cwd should I create a phab to update staging DBs? [21:34:11] ejegg, AndyRussG: that is jsut about Toolforge though. not core [21:34:26] ah, got it [21:34:37] also... nothing on the prod wikis should *ever* need to talk to the outside world IMO [21:34:53] eileen: that would be great [21:35:09] i can kick it off soon just gotta finish some other stuff first [21:35:18] it takes like a lot of hours iirc [21:35:24] bd808: so, for users that want to use some crazy stuff in their own user js, we just tell them to copy it all locally? [21:36:04] ejegg: if it was up to me we would remove user js to solve that problem [21:36:20] bd808: you mean that we should serve 100% of the content on wp? [21:36:22] use greasemonkey instead if you'd like [21:36:23] cause yeah [21:36:59] is there a way to effectively comb for js that makes requests and say no? [21:37:06] is there a header for that? [21:37:13] bd808: hmm, yeah. I figured we could give people an interface to whitelist domains, then serve them custom CSP headers when logged in [21:37:16] yes, a strict Content-Security-Policy [21:37:49] custom per-user CSP is a lot of complexity for questionable gain [21:37:54] fwiw i've never seen privacy badger turn orange on WP before that banner [21:38:02] Fundraising-Backlog, fundraising-tech-ops: Update staging (all 3 DB) with latest prod DBs - https://phabricator.wikimedia.org/T190227#4066837 (Eileenmcnaughton) [21:38:05] cwd https://phabricator.wikimedia.org/T190227 [21:38:12] ty [21:38:47] nasty js stuff is not very common, but it has happened before. It is usually reverted quickly on a big wiki [21:39:07] smaller wikis have done things like adding Google Analyics tracking before [21:39:26] which can take a while for others to notice [21:39:33] bd808 I guess the 'gain' I'd be after here is getting the CSP on the site without user revolt [21:39:57] do a lot of banners request external scripts? [21:40:12] cwd no, banners just about never [21:40:27] I think the main blocker to CSP is a violation report collector that we think would survive the load [21:41:09] bd808: so the thing that's collecting reports from foundationwiki is not scalable? [21:41:41] foundationwiki has a CSP policy? [21:41:53] yep! I just learned about it friday from bawolff [21:42:49] huh. I don't know what's collecting reports for it, but yeah I would question if it can scale unless its varnish->kafaka->hadoop [21:43:12] yeah, I heard mention of a log file on mwlog1001, so I'm guessing no [21:43:39] anon traffic would generate CSP reports [21:43:40] I saw Sentry mentioned on that same ticket [21:43:48] so the volume would be huge potentially [21:44:20] I don't think we know how to scale Sentry to anon traffic either [21:44:54] In some small scale testing I've seen hundreds of false positive CSP reports from browser add-ons that end up blocked [21:45:13] right... does seem like these would have a lot of duplication, so something that could just tally counts of each one would be the way to go [21:45:20] freaking browser addons [21:45:35] My Firefox generates 2 per page [21:45:41] we're collecting client-side errors on paymentswiki, and get a lot of spam from addons [21:46:12] *nod* and de-dup is hard because of language and platform variation [21:46:45] at $DAYJOB-1 I wrote a lot of code to try and de-dup localized js error messages [21:46:58] * ejegg needs to read up on CSP violation report format [21:47:01] eventually I gave up [21:47:25] this feels like a thing we need to solve [21:47:26] its a json blob and there is a standard by only modern blink engine follows it closely [21:47:51] and now feels like the best momentum we're likely to get short of a really nasty incident [21:48:07] I actually started working last week on a "simple" system for collecting CSP reports on Toolforge [21:48:22] ooh! [21:48:34] i'd love to take a look [21:49:12] well, that standard looks reasonably parseable / de-dupeable [21:49:14] https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP#Sample_violation_report [21:49:19] ejegg: https://phabricator.wikimedia.org/source/tool-csp-report/ [21:49:26] thanks! [21:50:01] I'm only keeping a small slice of the data -- https://phabricator.wikimedia.org/source/tool-csp-report/browse/master/app.py;e806e9d3ad8d922821ecce058d00f4e0df40aed5$116-122 [21:50:26] its not live yet, but I'm hoping to turn it on Thursday and see what the data looks like [21:51:33] cool.... I'd love to see that [21:51:39] (the data_ [21:51:42] ) [21:51:44] (PS10) Jgleeson: T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 [21:51:56] thanks jgleeson! taking a look [21:51:57] (CR) jerkins-bot: [V: -1] T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 (owner: Jgleeson) [21:53:32] hmm ejegg, mepps asked me to drop the Ingenico from IngenicoPaymentsProvider and it seems to have caused a conflict [21:53:36] ejegg: me too :) I hope it mostly just cdnjs + google fonts [21:53:49] jgleeson: oops, my tokenize patch most likely [21:54:04] ahh ok then [21:54:06] I can rebase if you'd like! [21:54:14] it's so late there... [21:54:27] that would be great if you dont mind [21:54:33] heh, no problem [21:55:11] yeah, I'm gonna head off now. If mepps is happy on her review tomorrow, I'll add some tests for that latest code and then be happy to move it from WIP to ready to go [21:56:57] (PS4) Ejegg: WIP: initial test for Ingencio Hosted Payments Rejection Status [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/413786 (https://phabricator.wikimedia.org/T176502) (owner: Jgleeson) [21:56:59] (PS9) Ejegg: Tests passing, clean up [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415085 (https://phabricator.wikimedia.org/T176502) (owner: Mepps) [21:57:01] (PS11) Ejegg: T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 (owner: Jgleeson) [21:57:19] (CR) jerkins-bot: [V: -1] WIP: initial test for Ingencio Hosted Payments Rejection Status [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/413786 (https://phabricator.wikimedia.org/T176502) (owner: Jgleeson) [22:00:00] (PS5) Ejegg: WIP: initial test for Ingencio Hosted Payments Rejection Status [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/413786 (https://phabricator.wikimedia.org/T176502) (owner: Jgleeson) [22:01:13] ejegg, it looks like the very first commit made as part of mine and mepps joint effort is now complaining. I think we can safely rebase the commits on this into one as it was originally a step by step tdd approach between me and mepps [22:01:30] and we're close to the end [22:01:47] ah, you fixed it :) [22:02:23] heh, looks like that whole file gets deleted later tho... [22:02:27] what's up with that? [22:02:58] oh, it's for the now-unused way of parsing errors [22:04:06] ah, so I think mepps deleted one as we could put the tests in an already existing file, and I deleted the other as it was originally one mock error response but then discovered we needed multiple so added a new bunch with the error event code as a suffix [22:05:07] although I don't think that answers your question [22:06:27] ok I'm gonna jump off. I'll catch up with mepps first thing tomorrow and talk through the outstanding bits. Thanks for your help! g'night fr-tech :) [22:06:34] ok, have a good evening! [22:11:28] Fundraising Sprint Elevators were never intended to go down, Fundraising Sprint Fhabricator is spelled with an "F", Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM, fundraising-tech-ops: Missing sql fn gives a warning whenever we edit a conta... - https://phabricator.wikimedia.org/T188931#4066941 [22:49:05] hey ejegg|afk bd808 sorry I missed the ping earlier... Hmmm what about maybe only enabling csp on a small sample of pageviews, until most of the blips can be worked out? [22:51:42] sampling might work, but it might be more effective to sample the reports than the pages that are protected [22:52:58] (PS10) Ejegg: Tests passing, clean up [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415085 (https://phabricator.wikimedia.org/T176502) (owner: Mepps) [22:53:00] (PS12) Ejegg: T176502 WIP: Updated Ingenico error handling [wikimedia/fundraising/SmashPig] - https://gerrit.wikimedia.org/r/415907 (owner: Jgleeson) [22:53:02] we really should be able to build a robust reporting pipeline. It just will take some heavy lifting for the collection and de-dup to get to clean reports [22:53:23] I wish I had time/energy to work on it honestly [22:53:44] * bd808 is too much manager and too little hard problems hacker these days [23:12:45] (PS1) Eileen: CiviCRM 5.0 stock (7713e22ddbe8) [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/420930 [23:12:47] (PS1) Eileen: Reapply WMF patches [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/420931 [23:16:29] (CR) jerkins-bot: [V: -1] CiviCRM 5.0 stock (7713e22ddbe8) [wikimedia/fundraising/crm/civicrm] - https://gerrit.wikimedia.org/r/420930 (owner: Eileen) [23:23:52] bd808: mebbe we could find was to get it prioritized... Especially given it's a different internet these days (viz. some of today's news infact) [23:25:04] AndyRussG: its worth a shot. You could start by talking to John Bennett (new Director of Security) [23:26:47] bd808: K! [23:27:20] heheh gotta set up some privacy-by-design shields....