[00:00:00] ccogdill: Have you been hearing anything from donors that might mean that the credit card error page isn't working? I can't tell if what I'm seeing in the logs is just one person trying to pull a fast one, or if we possibly have a problem. [00:00:30] I mean, it seems like it's just one person. [00:01:23] nothing unusual at all K4-713 [00:01:57] ccogdill: Thanks for looking. Please let me know if you start seeing evidence to the contrary. :) [00:02:53] of course [00:02:57] K4-713 awight pizzzacat need any other quesitons answered from me? [00:04:40] atgo-home: : At 5pm, I certainly hope not. [00:05:56] awight: How's that database export thingy? [00:06:09] ...oh, ha. [00:06:30] * K4-713 turns brain tuner [00:31:52] awight: ...when I run this email thingy in the AM, this is possibly going to clog queue consumption for over an hour, isn't it? [01:19:50] (PS1) Awight: WIP (FR #1703) make banner hide cookie more forward-compatible [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/143232 [05:20:13] (PS1) Awight: Prepare to undo source revision fetching hardcode foo [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/143245 [07:05:25] (PS1) Awight: WIP Expose parsing without side-effects [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/143249 [07:30:07] (PS2) Awight: WIP Expose parsing without side-effects [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/143249 [07:34:25] (PS3) Awight: WIP Expose parsing without side-effects [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/143249 [08:00:40] (PS4) Awight: WIP Expose parsing without side-effects [wikimedia/fundraising/tools] - https://gerrit.wikimedia.org/r/143249 [11:17:39] (CR) Siebrand: [C: -1] "Doesn't merge. Needs review of the FR folks." [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/142616 (owner: Jforrester) [16:43:40] ccogdill: Good morning! [16:43:51] morning! [16:44:11] So, in reading those emails about the emails... are we not sending a round today? [16:45:12] Or, should I proceed with the 10am list cutting like we talked about last night? [16:45:25] yeah it sounds like we’re not sending today [16:45:32] we should still cut the list [16:45:39] because we want to be able to estimate list size [16:45:50] but I can do that manually if this is a time suck [16:46:00] Nah, it's no problem. I can do it now anyway. [16:46:09] okay awesome [16:46:10] thank you :) [16:46:12] It's honestly just one button I press. [16:46:27] haha okay whew [16:46:57] (the trick is, as usual, pushing the correct button. :p ) [16:48:16] ccogdill: Button pushed. Now we wait for, like, an hour or so. [16:48:35] yeeep [16:48:48] does it let you know when it’s done? [16:48:54] Not really. [16:49:04] I mean, you can watch the job... [16:49:11] ...but there's no *ding* if that's what you mean. [16:49:47] mm okay [16:49:56] it sends me an email when I do it manually so I wasn’t sure [16:50:00] ccogdill: Can we find out if it uploaded like it should be doing, when we tried this last night? [16:50:05] Or, did you already check? [16:50:24] I’m waiting to hear back from Peter to see if he can show me where he looks to confirm [16:50:27] because I’m not positive [17:46:10] ccogdill: The job finished when I wasn't looking directly at it. [17:46:49] So, if everything worked, you shouldn't have to do anything at this point... aside from find out if it really worked or not. [17:46:51] :) [17:52:30] thanks K4-713! I’ll email Peter/Eric again and see if one of them can help me confirm [18:14:05] ccogdill: Have you seen any complaints about rtbt recently? [18:14:17] rtbt? [18:14:21] is that bank transfer? [18:14:24] Realtime bank transfer. [18:14:28] yep [18:14:32] I’m actually sending an email now [18:14:35] ...orly. [18:14:39] Just the one? [18:14:39] well [18:14:55] I mean an email to fr-tech and creative [18:15:16] Ah. [18:15:21] it was hard to tell from zendesk tickets if there was an issue so I tested the workflow. the first time I went through it I couldn’t get past the address page [18:15:31] then I tried again and was sent to the correct page [18:16:03] looked in GC and we’ve had 18 other transactions fail in the same way my 1st attempt did today [18:16:11] (BT transactions, that is) [18:16:40] anyway I’ll email with some fun screenshots in a minute :) [18:16:43] Can you tell how long people have been reporting this? [18:17:08] there were a couple confused BT tickets yesterday morning [18:17:20] we just sent them our normal Japan BT info [18:18:18] Okay. In the future, I'd rather get a quick ping on IRC when you think you see something wrong, before a complete and thorough report goes out to the list. [18:18:29] Sometimes (like this) it's pretty clear on the backend what's going on. [18:18:44] there were only 1-2 tickets yesterday, and we get BT info requests all the time [18:18:50] that’s all they really seemed like yesterday :/ [18:19:04] but when I saw more today, it was testing it myself that let me know there was a problem [18:19:05] Turning on 100% campaigns does tend to make things shake out, yes. :) [18:19:09] haha yep [19:12:51] ejegg: You around for a relative sanity check? [19:21:53] K4: yep, sorry, was on the phone [19:23:24] pizzzacat1 - K4-713 and i are thinking about noms [19:23:31] ejegg you can come, too :P [19:23:39] aww [19:24:03] hehe [19:24:05] I think K4 wanted me to look at something - maybe the Japan BT fix [19:24:35] Yeah... I got stuck, though, because the thing I found & "fixed" seems to be all about iDeal, and... JP should have nothing to do with that. [19:24:42] dang [19:24:52] Oh well. I can do another one in the same style after lunch or something. [19:25:11] (PS1) Katie Horn: RTBT: Fixed some issues with the form template for iDeal. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/143380 [19:25:26] ejegg: There's the thing I did. It's got a test and everything. [19:25:36] rockin! I'll take a look [19:25:43] ...and TODOs for a real solution. :/ [19:27:52] woohoo, parsing HTML with regexes! (...as if I've never done that...) [19:57:12] (CR) Ejegg: [C: 2] "Looks good for immediate purposes. Also, nicely translated '123 Fake Street'" [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/143380 (owner: Katie Horn) [19:57:41] (Merged) jenkins-bot: RTBT: Fixed some issues with the form template for iDeal. [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/143380 (owner: Katie Horn) [20:43:55] K4-7131 what "tech area" would a jenkins job land in? [20:48:45] atgo: Depends on the job. [20:50:25] K4-713 non urgently, https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/1755 [20:50:42] i can also pitch this back to adam tomorrow [20:52:00] Huh. Yeah... [20:52:12] I'm eyeball-deep in mist-haves already. [20:52:15] Er. [20:52:17] Must-haves. [20:52:50] ccogdill: Do you know which specific version of rtbt Japan is using, off the top of your head? [20:52:55] Or, does this have nothing to do with that? [20:53:11] umm I don’t know [20:53:15] ...not to alter the results or anything, but the latter would make a ton more sense. [20:53:15] would PPena know? [20:53:37] ccogdill: Well, to start with: who was complaining about rtbt this morning? [20:54:00] that’s the thing… it’s hard to say if the JP tickets are really complaints or not [20:54:12] K4-713 what we are doing in Japan is offline BT and not RTBT (rtbt=real time bank tranfer) [20:54:13] a lot just say that forgot the reference number and ask us for it [20:54:14] But, there *is* rtbt in Japan, yes? [20:54:20] Oh. [20:54:31] K4-713 abou the 'version'… not sure they they have versions for BT [20:54:41] they just provide the bank account # as a hosted page [20:54:52] For rtbt, there's ideal... some other ones... [20:54:53] there is not a rtbt in japan [20:55:01] hum. [20:55:03] K4-713 yeah but not in japan [20:55:12] rtbt is only IDEAL and Sofort [20:55:16] curretly [20:55:39] I thought I had more enabled. [20:56:11] this BT is just like all those OFFLINE BTs we did in 2012 with GC [20:56:16] and we disabled them all later [20:56:18] all offline [20:57:37] PPena: I also have "enets" enabled, whatever that is. [20:57:46] SG? [20:58:08] yeah [20:58:50] yep ur right Enets is RTBT [21:04:23] K4: is there anything else you need right now? I'm about to head out, but will join the standup/hangout from my phone. [21:04:42] ejegg: I don't think I can get into too much more trouble today. [21:04:47] Thanks for checking. :) [21:05:00] good luck with the bank transfers! [21:07:22] K4-713 don't jinx yourself there :P [21:08:56] cc[tab]... d'oh. [21:09:13] atgo: I have something silly to tell you. [21:09:23] .... oh? [21:09:24] Well, a couple things, actually. [21:09:46] #1 - The problem I solved, appears to have nothing to do with the issues ccogdill was seeing this morning. [21:10:14] * atgo facepalms [21:10:16] #2 - The issues ccogdill saw this morning, may have nothing to do with anything: It looks like we timed out trying to talk to GC a couple times. [21:10:18] but good for fixing things [21:10:24] hm. that's weird. [21:10:29] is that on their side...? [21:10:44] That... is them not responding to our requests in a timely fashion. [21:11:09] Which would explain why some people said it just worked on a refresh. [21:11:14] hm. yeah that does make sense [21:11:24] it doesn't explain the weird behavior in the browser that cc was experiencing though [21:11:33] It doesn't? [21:11:35] What were they seeing? [21:11:36] well... maybe it does [21:11:46] Was it the total lack of iframe content? [21:11:48] here's what we saw on her machine [21:11:51] step 1 - banner [21:12:05] step 2 - LP1 with email + name only, hit continue [21:12:17] step 3 - LP with email, name, address [21:12:17] (That's LP2, but go on) [21:12:20] shit. [21:12:23] i need a glossary. [21:12:35] LP1 only happens on donate wiki or in banners. [21:12:40] LP2 = payments.wikimedia.org [21:12:41] step 4 - back to SP with email, name, address [21:12:44] oooh gotcha [21:12:55] SP? [21:13:05] so... anyway, then it was LP2 (small) > LP2 (large) > LP2 (large) forever [21:13:08] sorry... typo [21:13:13] really. really. bad typo [21:13:24] my hands got mixed up as to which was which or somethig :P [21:13:28] That's on the other side of the aeyboard. [21:13:39] yeah. umm.. [21:13:45] i have no explanation for that. [21:14:02] but [21:14:19] Large Forever, huh? [21:14:28] sigh [21:14:35] yes [21:14:37] that's what i'm getting at [21:14:49] so i don't know if that was some weird thing that we were doing on our side when we got the GC timeout? [21:14:55] nah [21:15:05] That happens totally independent of what we're doing. [21:15:25] Usually timeouts trigger some kind of hard stop. [21:15:44] Where is this CC issue happening? [21:16:26] it's not CC [21:16:31] it was through the BT flow [21:16:44] OH. [21:16:51] "cc" as in Caitlin. [21:16:59] ooooh haha yeah sorry... i hit tab but then.. she wasn't here [21:18:04] I *just* tested the BT workflow for JP and didn't get anything str... wait. I'm checking server logs for it being a different member of the cluster. [21:18:12] oooh good thinking [21:18:12] Though, we're all in the office, so... [21:18:18] Eh. [21:18:18] it did that for caitlin a few times and then worked later [21:18:23] wt... [21:18:29] yeah in a new tab [21:18:31] it was weird [21:18:36] She used her own email address, right? [21:18:47] i think so [21:18:57] * K4-713 disappears into server logs [21:19:01] she's not at her desk (but her laptop is) so i imagine she'll be back and i'll grab her [21:19:44] Don't worry about it. I got it. [21:22:40] atgo: Uhm. One of them is trying to do a US-based BT, and the other is JP. [21:23:55] Dunno why it would change the country from JP to US and back again. [21:24:06] I may have clues, though... [21:24:25] hmmm [21:26:13] i'm going to meet you guys in the collab space for standup in 7 min [21:26:15] wooo collab [21:26:34] Isn't it 2:30 now? [21:26:43] That's 4 minutes. [21:27:10] Oh, no... 2:33. My bad. :D [21:56:53] K4-713 420060 . 430900 . [21:57:17] atgo: Woot, thanks. [21:57:26] I'll see what I can... tease out. :/ [21:58:35] ...45. [21:58:39] 45 total. [21:59:00] Between the two of those. [21:59:52] atgo: And of those 45, most of them are people reloading 5+ times. [22:00:07] K4-713 it looks like they're all redundant.. as in they all have both codes [22:00:26] I'll know that in a minute... [22:00:50] well from GC that's how it looks [22:00:51] atgo: So, I'm thinking we leave it right where it is and try to repro... whatever the heck they were doing. [22:01:10] i mean even if it's only 45, it looks like there are, what, 450 total? [22:01:15] or am i way off... [22:01:25] hm. [22:01:41] (trying to figure out this console is hard) [22:01:45] 539 [22:02:40] atgo: But it's important to note that my 539 has no repeats on initial success. [22:03:00] 1% or something. [22:03:03] hm? [22:03:05] what does that mean? [22:03:15] I have 539 transaction endpoints. [22:03:21] ...for bank transfer, today. [22:03:55] 45 are failures, but the majority of those failures represent one individual trying over 5 times. [22:04:19] Successes only get one, and are not inflated. [22:04:26] So, this is a small population. [22:04:35] right [22:05:11] hm [22:05:48] i wish caitlin was around [22:06:09] Suspiciously, most of the error messages I'm seeing in the logs for the failures, are rendered in english... [22:06:18] ...which tells me it might have been mostly us. [22:06:27] hahahaha [22:06:44] looks like peter took BT down anyway [22:06:53] ...when? [22:06:57] don't know [22:07:01] oh.. ju8st kidding [22:07:06] i clicked a banner without it [22:07:54] Yeah, don't take it down. This may only have happened to a couple people. [22:08:05] ok so. i just went through it twice as me just fine [22:08:23] * K4-713 nods [22:08:25] 3x... it starts doing something weird [22:08:38] and i'm goign to stop so that i don't screw up the results too badly :P [22:08:40] * K4-713 coughs [22:08:50] still gets through [22:09:02] "something weird" [22:09:10] had to hit continue twice [22:09:25] atgo: Oh, you shouldn't be able to do that. [22:09:34] but by and large it's fine [22:09:52] yes i know.. but i can't reproduce [22:09:53] That also sounds like good old GC lag. [22:10:16] ok i found the problem. [22:10:22] * atgo wins [22:10:25] Because, we're waiting for them to get back to us. Why that button doesn't disable is... an infuriating question. [22:10:39] ...found the problem? [22:10:39] so it's when you get to short LP and then the country goes to US [22:10:40] reproduced [22:10:46] check the logs for me [22:10:47] ... [22:11:02] literal identical behavior to what i saw on caitlin's [22:11:05] What do you mean, the country goes to US? [22:11:12] so i'm in LP2 (short) [22:11:17] and then i change the dropdown from Japan to US [22:11:20] Wait 1. [22:11:25] then it sends me to the long one, and then i cycle there [22:11:30] Do you have a "ffname" in your URL? [22:11:41] ffname=bt [22:11:51] You... changed the dropdown, deliberately? [22:11:57] yeah to see if i could reproduce [22:12:07] so maybe caitlin did that [22:12:17] but why. [22:12:19] or could that be something that her browser did? [22:12:23] well.. [22:12:25] i don'tk now [22:12:41] are you crying or laughing? [22:13:04] I don't even know. [22:13:10] haha [22:13:34] however, it's still a not-great experience [22:13:56] woah [22:13:56] Good thing I hired a frontend engineer. [22:14:05] so now.. it looks like it sent me to a CC flow [22:14:13] if i look in the GC console [22:14:27] uh. [22:14:32] i have a couple of rejected BT and then a rejected MasterCard Online transaction [22:14:45] with this rejection code [22:14:46] 430415 . [22:16:08] for $3 USD [22:16:09] One of those forms it gave you was a cc form. [22:16:12] so something weird is happening there [22:16:20] but then it also gets me stuck [22:16:39] the error message is "SECURITY_VIOLATION" [22:16:52] so apparently we sent me somewhere sketchy :P [22:17:39] caitlin's attemps are the same in GC [22:17:53] ...the heck you say. [22:18:03] want to see GC's stuff? [22:18:29] so once we send people to the CC form, it's not in a way that makes sense [22:18:33] That OID you sent me isn't in the logs. [22:18:36] what? [22:19:03] that's...w hat it says [22:19:11] what's the last OID you have from me? [22:19:24] I don't know. Kind of a lot just happened. [22:19:34] mmk [22:21:28] hey meganhernandez ... how many BT attempts did you make today? [22:21:39] ish? [22:22:00] K4-713 don't rage too hard [22:22:13] every reject i'm looking at seems to be... us [22:22:14] atgo: Twelve. [22:22:16] ...for megan. [22:22:21] ... yeah... [22:22:28] these are all us [22:22:29] I'm telling you: Email is easy. [22:22:50] actually though. every one of these is ius [22:22:59] Okay. I'm going to request a moment of calm during which I look at the one that looks legit. [22:24:11] i bet he/she did the same thing [22:25:22] country code IN [22:25:54] K4-713 ^ [22:26:22] atgo: I have an idea. [22:26:52] How about we alter this form so it stops letting you try to perform nonsense? [22:27:11] yup [22:27:25] i mean, alternatively it's literally 1 person who's done this [22:27:31] I... honestly have no idea why there is a country dropdown on there that lets you pick countries THAT DO NOT SUPPORT THE CURRENCY YOU CAN'T CHANGE AT THIS STAGE. [22:27:38] i'm hesitant to lose more time on this. [22:27:42] yeah. that does make no sense [22:27:54] Sorry... I'm yelling at the past. [22:28:04] i get it [22:28:25] Also, we have dealt with this much more intelligently on other forms. [22:28:34] so... in the future, any time we have an issue, we should eliminate all of our team's testing transactions from the count [22:28:36] like... immediately [22:29:04] Like, there's a mechanism that removes inappropriate choices from dropdowns. [22:29:11] yeah that should definitely be happening [22:29:46] however, this is literally 1 person who made this mistake. i get that it's not an ideal experience, but that 1 person.... [22:29:55] There was a card in the backlog... don't know if it survived, that was saying we should reload the page when you pick something that changes everything in a dorpdown. [22:30:34] Last time we tried to get that work on the board, they just took out the dropdown that was bothering me. [22:31:02] ...heh. "Dorpdown" [22:31:10] More like derpdown. [22:31:13] hehehe [22:32:04] * K4-713 sighs [22:32:32] What if I spend 30 minutes looking for the thing that takes things out of other things, and see if I can use it here? [22:33:35] yeah alright sounds good [22:35:27] atgo: Please remind me to rant about testing at the next retro. [22:35:34] ... tomorrow? [22:35:38] Sure. [22:35:45] It'll be fresh. [22:35:48] heh [22:40:14] lmk when/if you find a fix (or abandon fixing) and i'll send out this recap email i've got drafted. [22:41:29] atgo: Yeah, I found the thing I was thinking of, and it's not an easy fix. [22:41:35] ok. let's not [22:41:48] I mean, we could always whack the dropdown entirely. [22:42:08] i mean, it's 1 person of like.. 500 [22:42:12] that actually tried to do this [22:42:19] If we don't, though, we have to maintain a list of all the country/currency combinations that GC has enabled for us, for BT. [22:42:22] yes, ideally we would do that, but will it paint us into some corner later? [22:42:42] Actually, I don't think that's ideal at all. [22:42:52] If they enable something for us, we should be smart enough to notice. [22:43:20] well.. i mean ideally we would have something smart on our forms that said "there's only one thign in this dropdown, don't display the dropdown" in general [22:43:24] because.. that's a silly usage of a dropdown [22:43:30] The longer fix is to have better error messaging when we get something back in that long series of errors. [22:43:46] Or, you know... a specific translated message on an error page. [22:43:56] is there anything we can do that won't kick them over into mastercard flow? [22:44:03] i mean, really any error [22:44:06] would be a good thing [22:44:12] compared to what we have :P [22:44:16] Actually, I have no idea why that went all credit card on you. That is a legit mystery. [22:44:21] Did you see a cc form? [22:44:31] it gave me the long LP2 [22:44:59] You never saw a cc form, but they say you went into a cc workflow at some point? [22:45:08] yeah in the GC console it has a rejection at mastercard [22:45:09] hang on [22:45:15] They could *also* have a bug, y'know. [22:45:21] what?! madness. [22:45:35] Particularly when you do deliberately strange things. [22:45:56] haha yeah [22:45:59] it didn't do that to the other person [22:46:02] so.. whatevs. [22:46:14] but it did to caitlin [22:46:17] it's just the US people [22:47:17] whatever. this is not worth more time [22:47:30] if it surfaces again, absolutely [22:47:36] but as is i think we should move on with our lives :) [22:58:41] aaand then i locked myself out of the floor. [23:05:57] oops. [23:06:19] atgo: ...I think I need to WFH some day this week so I can get back to SEPA. [23:06:34] Otherwise somebody will chase me up a tree and set fire to it. [23:07:22] whatever you need to do [23:07:40] tomorrow woudl be the only day... because thursday is metrics and friday is fireworks [23:07:58] Shooooooooooo. [23:08:10] Well, never mind. [23:33:32] (PS1) Katie Horn: Submit form when return is pressed [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143512 [23:33:34] (PS1) Katie Horn: Fix expiration date message translation [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143513 [23:33:36] (PS1) Katie Horn: RTBT: Fixed some issues with the form template for iDeal. [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143514 [23:34:31] (CR) Katie Horn: [C: 2] "+2 for Deploy" [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143512 (owner: Katie Horn) [23:34:45] (CR) Katie Horn: [C: 2] "+2 for Deploy" [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143513 (owner: Katie Horn) [23:34:56] (CR) Katie Horn: [C: 2] "+2 for Deploy" [extensions/DonationInterface] (deploy-payments_1.22) - https://gerrit.wikimedia.org/r/143514 (owner: Katie Horn) [23:49:11] !log updated payments cluster to c5689f385b2f0a7 [23:49:15] Logged the message, Master