[00:00:11] pizzzacat: you were making the new branch the same as master, for now. if you set the initial SHA-1 to the commit *before* your first feature patch, that is more correct [00:00:28] huh: hi [00:00:45] oh awight the last part of that makes it make sense [00:01:18] awight: I know the "X" button exists, BTW [00:01:51] huh: I think I swore at the fullscreen banner myself, when I noticed that I couldn't dismiss it with the usual or clicking outside the dialog. [00:02:07] but it won't come back, now that you've closed it :) [00:02:09] huh: but is it annoying /enough/ ? :P [00:02:18] ;) [00:02:29] moar pink! Call in Jimmy photoze! [00:02:52] There are reasons we don't get to design banners. [00:04:23] (CR) Ejegg: [C: 2] typo-O [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/175591 (owner: Awight) [00:05:57] No colors for grrrit? [00:06:18] (Merged) jenkins-bot: typo-O [extensions/DonationInterface] - https://gerrit.wikimedia.org/r/175591 (owner: Awight) [00:07:27] huh: Just two. :) [00:11:23] awight: Do you need anything else to go out in the payments deploy I'm about to do? [00:11:33] ejegg and pizzzacat too. [00:11:40] not I [00:11:47] K4-713: no [00:12:03] Good. This should be simple. [00:12:05] ... [00:12:11] I have no idea why I typed that just now. [00:12:15] :) [00:12:21] * K4-713 would like more trouble [00:15:01] I'm good [00:15:28] ejegg: what about yr WP styles? [00:15:54] awight: Oh, we could do the button stuff [00:16:02] GC though, I think [00:16:17] The RTL bit will require talking to GC [00:16:28] ejegg: oh! [00:16:28] (PS1) Katie Horn: Merge branch 'master' into deploy [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/175612 [00:16:33] hehe too late [00:16:46] you guys are rad [00:16:57] i'm heading out - see you mañana! [00:17:06] later skater [00:17:06] atgo: Later! [00:17:13] Yeah, for RTL we need to tell them to source a different stylesheet for specific langs [00:17:14] ...you win this time, awight. [00:17:16] so long atgo [00:17:19] K4-713: that was creepy [00:17:28] * awight dons tin foil [00:17:31] what K4-713, you're not going to jinx him? [00:17:34] btw, i owe you a coke. [00:17:35] (CR) Katie Horn: [C: 2] Merge branch 'master' into deploy [extensions/DonationInterface] (deployment) - https://gerrit.wikimedia.org/r/175612 (owner: Katie Horn) [00:18:13] If I keep this up, everybody's going to notice I can read minds. [00:18:17] hahaha [00:18:37] Just please don't tell me what happens in tomorrow's episode. [00:18:55] back in a bit, gonna relocate [00:19:21] ewulczyn: just confirming that you saw this handoff: https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/2175 [00:19:48] ewulczyn: There was a 100% test under the new client-side banner code, today, 2300-2400 UTC. [00:20:04] If there's a Faraday cage around my desk when I get to work tomorrow, I'll understand. [00:20:47] ewulczyn: We wanna know that the impressions are at the same proportion of total pageviews as in the last few days... [00:23:14] fyi the 100% US test came down about 20 mins ago PPena AndyRussG awight K4-713 [00:23:50] Cool. :) [00:24:05] meganhernandez: Ah OK... Uh that was due to...? (sorry if I didn't see relevant backscroll...) [00:24:31] AndyRussG: good to test 100% US with buckets A, B, C & D tomorrow morning? I was aiming for around 6 o 6:30am pacific [00:24:50] AndyRussG: it was the full hour [00:25:00] awight: Ah OK thanks [00:25:03] meganhernandez: sure! I'll be here [00:25:07] time flies when you're having fun [00:25:31] A fix for the previewing banners on wiki issue is almost done, should be deployed this eve [00:25:43] meganhernandez: ^^ [00:26:07] oh right, yeah we’d need that before we can test those new banners [00:26:53] meganhernandez ok thanks [00:27:06] (PS2) AndyRussG: Fix previewing banner on-wiki [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175606 [00:27:09] Woah. New phabricator spam. [00:27:18] hah. I tricked you. [00:27:36] CURSES! [00:27:40] Foiled again. [00:27:42] (CR) AndyRussG: "Tests now pass..." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175606 (owner: AndyRussG) [00:28:02] Okay, deploying payments things. [00:28:21] (CR) Awight: [C: 2] Fix previewing banner on-wiki [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175606 (owner: AndyRussG) [00:28:34] :) ^ [00:28:41] AndyRussG: what do you think about https://gerrit.wikimedia.org/r/#/c/175597/ ? [00:28:50] awight: just about 2 rebase... [00:28:54] k [00:30:28] awight: that patch won't bork produciton configs, I guess? It's actually not set to false anywhere? [00:30:40] (PS2) AndyRussG: default to client banner choice [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175597 (owner: Awight) [00:31:11] AndyRussG: it's set to true, unconditionally, now [00:31:18] (Merged) jenkins-bot: Fix previewing banner on-wiki [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175606 (owner: AndyRussG) [00:31:29] ooo hooo so now it'll be set to true twice [00:31:38] :) [00:31:39] Double positive makes a negative? [00:31:55] Tests pass... https://gerrit.wikimedia.org/r/#/c/175597/ [00:31:55] !log updated payments to 3e3cda8f07af9f7f7 [00:31:57] double positive causes suspicion [00:31:59] Logged the message, Master [00:32:05] K I'll merge [00:32:24] nice to see that the qunit actually would have caught the bug :) [00:32:31] If I only ran it correctly ugggh [00:32:44] so close! [00:33:52] K4-713: is the WR1 thing going to be admitted to the operating room?... [00:34:08] I just need to poke it with sticks for a while. [00:34:17] ...until I get time to fix it for real. [00:34:27] ok [00:34:29] And by "fix it" I mean use the WX files. [00:34:41] awight: funny I don't see the bannerController.lib tests running on Jenkins: https://integration.wikimedia.org/ci/job/mwext-CentralNotice-qunit/248/consoleFull [00:34:49] ...and then gently nudge the old code out to sea. [00:35:08] Still let's merge 4 now [00:35:11] AndyRussG: ouch, 00:30:48 Warning: No route to fetch banner choice data configured. [Called from CNBannerChoiceDataResourceLoaderModule::getChoices in /srv/ssd/jenkins-slave/workspace/mwext-CentralNotice-qunit/src/extensions/CentralNotice/includes/CNBannerChoiceDataResourceLoaderModule.php at line 60] in /srv/ssd/jenkins-slave/workspace/mwext-CentralNotice-qunit/src/includes/debug/MWDebug.php on line 300 [00:35:32] awight: yeah saw that. That's also due to default config [00:35:34] AndyRussG: but I do see the tests running: 00:30:53 ext.centralNotice.bannerController.lib - allocations test cases...OK [00:35:57] Unfortunately, we don't get a line per scenario... but they're there. [00:36:25] ccogdill: obt payment methods are back up. :) [00:36:32] That's... ideal, bpay... [00:36:40] woo! [00:36:41] awight: ah yes ur right... (derp again) [00:36:48] I wasn’t expecting it so soon, thanks k4-713 [00:37:09] We should have independent confirmation, and then have the-wub re-enable the buttons on donate. [00:37:34] (CR) AndyRussG: [C: 2] "\o/" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175597 (owner: Awight) [00:38:05] (Merged) jenkins-bot: default to client banner choice [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175597 (owner: Awight) [00:38:11] ccogdill: Wait, actually, that's obt and rtbt. [00:38:45] ccogdill: bpay, sofort, ideal... enets? [00:39:13] yandex? webmoney? [00:39:38] Probably! [00:39:40] awight: see any issues ahead of deploy? [00:39:44] woop! [00:39:54] ccogdill: It was a silly thing. [00:40:01] But it was in there pretty good. [00:40:07] glad to hear it, silly is easier [00:40:14] it’s not on the form yet - any idea on an ETA [00:40:31] AndyRussG: looks good, but I'm not looking :) [00:40:31] I mean, it's fixed in production now. [00:40:37] ccogdill: ^^ [00:40:53] ohh that’s a the_wub thing huh k4-713 [00:40:57] Yep! [00:41:14] got it, thanks! [00:41:16] I'll send out an email about it in a minute. [00:41:23] Just thought you'd like to know early. [00:41:55] oh excellent, saves me from sending one :) [00:42:11] awight: O_O [00:42:18] * AndyRussG looks and looks [00:49:08] K4-713: cute photo, whenever you need one... [00:49:17] * K4-713 swivels [00:53:04] meganhernandez are those the final numbers on the A/b spreadsheet? [00:55:30] (PS1) Awight: Merge master to wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175621 [00:55:42] (CR) Awight: [C: 2] Merge master to wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175621 (owner: Awight) [00:56:15] (Merged) jenkins-bot: Merge master to wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175621 (owner: Awight) [01:00:26] waiting for Jenkins merge... [01:01:35] K4-713: oooof, I broke all that? [01:01:42] Who knew "return" could be so important [01:01:48] Looks like some of it was broken a bit before. [01:01:54] :p [01:03:12] K4-713: oh, that Complete Discography was of Minor Threat. Don't know how that affects your decision to cp. [01:05:03] awight: yeah Jenkins has been a bit pokey tonite [01:07:14] awight: thanks for preparing the patches, also smoke tested locally [01:07:14] AndyRussG: I noticed it runs both test and gate+submit jobs, even after I've CR+2'd. [01:07:26] It should cancel the "test" jobs IMO [01:07:26] Hmmm [01:07:37] otherwise, it's wasting 50% of the cpu! [01:07:38] Maybe it tests the merged version? [01:08:16] ...what the heck, fredge. [01:11:38] K4-713: ? [01:11:50] Are you not getting the failmail? [01:12:01] Not yet... [01:12:03] Oh, for... [01:12:05] What fanmail? [01:12:12] "Data too long for column" [01:12:19] Fail mail. [01:12:20] :p [01:12:33] Well, that's dumb. [01:12:36] i'm not a big fan of fail [01:12:47] Apparently, I love it. [01:13:21] This is what I get for testing weird payment methods with... outrageously long names. [01:13:28] Hmmm [01:13:35] awight: anything I can do BTW? [01:13:36] who'da thunk it would be auf deutsch? [01:13:41] Right? [01:15:01] The weirder thing is that some weirdo put it in our system in long form. [01:15:34] !log disabling fredge consumer [01:15:39] Logged the message, Master [01:18:21] I think I'm just going to... increase these field sizes from varchar 16 to 32. [01:18:43] go for it! [01:22:59] awight: is deploy? [01:23:55] Oh I see ops [01:30:10] AndyRussG: ejegg: yep, I stepped on a mine. Trying to fix my shtitstorm at the moment [01:30:54] awight: erp it happens, don't worry, it'll be fine [01:32:08] lemme know if I can do anything [01:33:08] AGH. [01:33:17] awight: there are no deploys scheduled right after this one, so provided the sites are open and u have time, there's no rush [01:33:38] GC apparently sent us an order ID of "GCTEST 141121003". [01:33:49] ...you may notice that most of that is not numeric. [01:34:17] I think it's a new format for expressing irrational numbers [01:35:13] Now I have to deploy things to get a clean run. [01:36:06] AndyRussG: ok thanks for checking [01:36:42] base 36/ [01:36:44] ? [01:39:23] * K4-713 facedesks [01:39:36] We were doing so well. [01:39:41] Kind of. [01:40:39] K4-713: it'll be fine... [01:41:54] I should clarify. *You* guys have been doing awesomely. [01:43:02] K4-713: you're doing great! don't facedesk, that must hurt on a stand-up desk... also pls lemme know if I can help with anything... [01:43:27] * K4-713 blinks slowly at an uncommitted hairball on local [01:43:36] AndyRussG: deployed the CentralNotice patches... [01:43:44] This complicates things. [01:43:51] awight: thanks much!! [01:43:52] * awight borrows a desk for face-ing [01:44:13] Yeah I can test banners [01:45:13] http://en.wikipedia.org/?banner=B14_1030_enUS_dsk_gwy_gc&force=1&debug=true [01:45:39] Still gotta leave the debug=true in there until the cache oodles out [01:47:48] Cache fully oodled now! [01:48:36] I'm gonna reply to James A. on centralnotice-admin list, fiddle with Mingle... [01:49:10] awight: thanks again for so much deploying effort! [01:51:31] (PS1) Katie Horn: Downgrading the severity of something that GC just did to us [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175634 [01:52:14] omg [01:52:45] Who wants to play "find the actual two lines of real change in the whitespace mess?" [01:53:02] BLARGH [01:53:15] why... [01:53:35] well anyway my Gerrit diff view defaults to ignore whitespace :) [01:53:48] Oh. Well, never mind, then. [01:54:02] nice try ;) [01:54:19] Actually... you know what? [01:54:24] (CR) Awight: [C: 2] "Designed to be tampered." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175634 (owner: Katie Horn) [01:54:26] (Merged) jenkins-bot: Downgrading the severity of something that GC just did to us [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175634 (owner: Katie Horn) [01:54:30] uh oh [01:54:39] I just decided that I did that for a really good reason. [01:54:46] Just kidding. [01:54:49] Ehm. [01:54:50] :p [01:54:59] I'm through with this day... [01:55:05] U need more supports? [01:55:19] And: I can't believe you're using a standing desk. Masochist. [01:55:24] Please stop that. [01:55:26] (PS1) Katie Horn: Revert "Downgrading the severity of something that GC just did to us" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175637 [01:55:34] wat [01:55:59] (CR) Awight: [C: 2] "/me keeps eye on the shells..." [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175637 (owner: Katie Horn) [01:56:01] (Merged) jenkins-bot: Revert "Downgrading the severity of something that GC just did to us" [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175637 (owner: Katie Horn) [01:56:12] awight: That number gets used in an exec statement. [01:56:22] oh my god [01:57:23] awight: I need to think about this for a second. [01:57:33] I should be able to bail out on that whole transaction at that point. [01:57:43] K4-713: can u update the regex to match the stupid new nonnumeric thing? [01:57:43] But it's bigger than that. [01:57:59] awight: I just want to skip everything that is unusual at this point. [01:58:20] Also: Strictly numeric is safe. [01:58:25] fair enough. What are the circumstances in which GC tries to kill us? [01:58:41] Unknown. [01:58:48] This is the first time they have done this. [01:59:08] we should send those transactions back, along with a bill for our time. [01:59:12] ...so, if I had to guess, it's got 100% of everything to do with holding the door open for our wr1 implementation. [01:59:28] I am getting really tired of being right about this kind of thing. [02:00:21] I'm pretty sure they said that they're only generating these for us, and a very small number of others. [02:02:04] why add a digit... when you can totally change the format and start using random punctuation to get more bits! [02:02:17] Also, spaces where there are usually none. [02:02:23] no. [02:02:29] Oh yes. [02:03:42] * awight checks pizzzacat's drawer. [02:03:44] Only one left [02:03:48] * awight bares teeth [02:03:56] what are you looking for awight? [02:04:04] beerz? [02:04:11] awight: If you beat me unconscious with it first, that's okay. :) [02:04:14] pizzzacat: yes hidden celebratory supplies [02:04:23] K4-713: it's on. [02:04:29] whatever happened to K4's tequila stash??? [02:04:36] *alleged [02:04:40] I had to give it back. [02:04:44] * awight checks fridge before getting all Lord of the Flies on it [02:04:45] * K4-713 looks sad [02:04:51] awwwww [02:07:11] awight: Confirmed, btw. These test transactions do not appear in the WX files. [02:07:19] I had to look. [02:14:28] The really unacceptable thing about this iTunes monopoly is that it can't decode FLACs. [02:22:27] meganhernandez: fyi, AndyRussG fixed the banner= issue, as of a few minutes ago. [02:23:10] meganhernandez: and awight courageously deployed :) [02:25:12] huh? what about the aweseome possum? [02:26:19] AndyRussG: He quit. [02:26:56] that’s fantastic, i see banners! [02:27:01] ok will prep banners for the morning test [02:27:21] AndyRussG: re that banners to logged-in people, do you think it might have been cached RL data? Or is anonymous filtering also performed in the client code? [02:27:51] meganhernandez: cool! Pls give a shout if there are any issues... [02:28:02] awight: huh? [02:28:23] did something bad happen to logged-in people? [02:29:34] AndyRussG: oh sorry, I CC'd you to a Phabricator thing, still learning how to use it. https://phabricator.wikimedia.org/T75812 [02:30:43] awight: ohh, I'm still learning I should check my phabricator spambox [02:30:50] whoooo [02:34:06] awight: I can confirm that's happenning, booo :( [02:34:20] AndyRussG: really? I couldn't reproduce it locally. What did you do? [02:34:24] Log in? [02:34:33] hmm [02:34:41] https://en.wikipedia.org/w/index.php?title=Main_Page&country=FR&reset=1 [02:35:18] AndyRussG: I don't get one [02:35:31] awight: and you're logged in? cleared cookies? [02:35:36] ... which slightly increases my belief in the theory I gave above [02:35:40] clearing now... [02:36:07] awight: check your browser's call to Special:BannerLoader to be sure [02:36:14] AndyRussG: nope I can't reproduce [02:36:28] Are you sure you aren't clearing your logged-in cookie? :) [02:36:32] I'm getting it if I log in with an incognito / private browsing session [02:36:44] ejegg: weird [02:36:45] seriously. how are you logged in w/o a cookie [02:37:03] awight: clear cookies, then log in? [02:37:22] Yeah, cookies persist within the window in the private browsing session in FF [02:39:33] weirdly, I don't get the banner. But I do see a bunch of junk in the RL response. [02:40:02] I think the RL data response is cached, and does not fragment across logged-in [02:40:29] well that would really suck [02:40:34] pardon my language [02:41:46] awight: are you getting the banners if you log out? Does it vary for you on logged in/logged out? [02:42:05] RL must vary on user because it sends user data in to the client [02:42:47] AndyRussG: yeah I see banners if logged out [02:43:04] And then you log in and they go away like they're supposed? [02:43:08] AndyRussG: so you think the RL data module class might be getting the wrong logged-in-ness? [02:43:18] AndyRussG: yeah. however, I see them in the RL data response [02:43:38] awight: that's reall weird. [02:43:50] No, we're not filtering for logged-in-ness on the client since that's supposedly done on the server [02:43:51] What do you get for https://bits.wikimedia.org/en.wikipedia.org/load.php?debug=false&lang=en&modules=ext.centralNotice.bannerChoiceData%2CbannerController%7Cext.centralNotice.bannerController.lib%7Cext.centralauth.centralautologin.clearcookie%7Cext.eventLogging%7Cext.uls.init%2Cinterface%2Cpreferences%2Cwebfonts%7Cext.visualEditor.track%2Cve%7Cext.visualEditor.viewPageTarget.init%7Cjquery.accessKeyLabel%2CbyteLength%2Cclient%2Ccookie%2CmwExten [02:44:13] So something more wierd must be going on for you not to see them, can u debug? might be another issue to track down... [02:44:26] I was pretty sure I checked that we were getting the user correctly [02:44:31] How did we expect that to work... [02:45:11] If my theory is correct, the only fix is to send both anon and logged-in banners in the RL data response, and the columns showAnon and showUser... [02:45:15] then filter in the client [02:45:49] Yea that would also suck [02:45:59] though not sooo terribly I suppose [02:46:04] grrr [02:47:16] AndyRussG: just realized there is a smaller but terrifying chance that the RL response will get cached in response to a logged-in user [02:47:24] in which case, many many people will not get banners [02:47:26] urp [02:47:33] I'm having a problem pushing my day's patch to gerrit [02:47:43] :( [02:47:43] AndyRussG: what are your work hours tonight :) ? [02:47:56] Let's summon Krinkle............ [02:48:29] AndyRussG: we're talking here about turning the client-side switch off again [02:48:30] Krinkle: boo! Have we done something terrible that doesn't properly distinguish logged-ins w/ anons? [02:49:06] awight: gotta do watcha gotta do [02:50:02] awight: also pinging Roan on -dev [02:50:25] AndyRussG: I'm happy to wait until we get some RL heads to think about this [02:50:30] anyone know what this means? ! [remote rejected] HEAD -> refs/publish/master/BigEnglishModules (change 173632 closed) [02:50:39] but I don't see how the RL request could possibly distinguish between logged-in and anon [02:50:56] or, working the other way (pushing without using gerrit) ! [remote rejected] HEAD -> refs/for/master (change 173632 closed) [02:51:04] pizzzacat: that it's been merged or abandoned already. [02:51:20] the patch I'm trying to push? [02:51:21] pizzzacat: in this case, merged :) [02:51:29] yeah [02:51:38] how is that possible, since I haven't pushed it yet? *scratches head* [02:51:51] pizzzacat: https://gerrit.wikimedia.org/r/#/c/173632 ? [02:52:04] That's the one it's complaining about. is that the head? [02:52:08] it's after this [02:52:26] ok. then you have an amended or rebased version of ^^ preceding the patch you're pushing. [02:52:32] rebase your new work [02:52:35] so I sometimes get this: [02:52:51] well I won't paste [02:52:54] AndyRussG: If I'm right about the non-fragmentation thing, we definitely have to turn the client choice off until fixed [02:52:56] basically a few commits [02:53:03] and it asks if I want to push them [02:53:14] awight: yeah, roan is looking now [02:53:18] AndyRussG: ok great [02:53:25] and I told this to someone and they said I could just push them and it ignores the ones I've already done [02:53:27] meganhernandez is here too, we're trying to triage [02:54:59] what am I supposed to be rebasing awight?] [02:55:40] this is what I was trying to avoid earlier. :-/ [02:56:07] seems like you guys are busy so I can just deal with it in the morning but it sucks having spent all day and not pushing it. :( [03:06:35] AndyRussG: fyi I've opened a deployment window, I'm turning off client-side choice until we can fix. It's better this way... [03:06:47] Yep. [03:06:56] * K4-713 looks stern [03:07:00] :) [03:07:42] awight: Yeah right way 2 go [03:08:02] I'll put in a comment on the phabricator report [03:09:15] We can differentiate anon/logged-in on the client, and maybe if it's decided that RL could be patched to distinguish, then we could switch back to filtering on the server [03:09:16] Bah [03:09:39] eh I think it's a minor change--there isn't much additional payload, if that's what you're thinking about [03:09:44] awight: as before, lemme know if I can do anything [03:09:58] Yeah, it's not terrible. But also too much to do in 5 minutes or less [03:10:15] I'll work on the patches for fixing and we can see if we turn it on again tomorrrow? [03:10:20] definitely! [03:10:31] K :) thanks BTW [03:12:06] AndyRussG: ok the switch is off... [03:12:17] I'll wait around for a few, then going home. [03:12:37] awight: K thank U... I'll check and ping when it's all cached out [03:16:33] awight: before u go---any news on the reason why you weren't getting the banners? (if not we can investigate tomorrow) [03:17:43] awight: K enwiki is back to Special:BannerRandom, thanks once again [03:18:57] meganhernandez: as per above, we ran into a problem distinguishing between logged-in and anonymous users with the new system, and logged-in users were suddenly getting fundraising banners. We've turned the new system off and will see if we can fix it tomorrow. [03:19:34] AndyRussG: Hi [03:19:38] ok thank you for the update AndyRussG [03:19:46] awight: [03:19:49] What's up? [03:20:05] i’ll still run a test in the morning, but i don’t think you need to monitor since the new central notice changes are not out [03:20:18] meganhernandez: OK sounds good... we'll keep everyone updated... [03:20:29] thank you! [03:20:46] np! likewise, especially thanks 4 being patient... :) [03:20:51] Hi Krinkle, uhh, we realized that we were mistaken in thinking the RL module could distinguish logged-in vs anon status [03:20:52] AndyRussG: fwiw I won't be around tomorrow, but ejegg|away can do the config deploy to turn it back on. [03:20:59] modules are shared by wiki(dbname), skin and user language (e.g. french user with monobook on enwiki) [03:21:10] awight: yeah! ejegg|away will do great! [03:21:18] modules are otherwise stateless. [03:21:28] that confirms our fear. [03:21:33] ok thx, gotta run! [03:21:39] awight: bye, take care! :) [03:22:21] AndyRussG: How big is the banner load if you were to do that on the client side as well? Like all the other factors you're doing client side now as well (cause you're sending them all to the client right and selecting there instead of server-side) [03:22:23] Krinkle: yeah we just learned about the magic by which a user ID for logged-ins is sent with the page HTML [03:22:57] Krinkle: yeah we can do it, it's just a bit of refactoring, here there and everywhere [03:23:35] plenty of scripts are logged-in-users-only (e.g. the load() call only occurrs for logged in users, this can be done server side from an OutputPage-accessible hook using User::isAnon() and calling addModules conditionally) [03:23:38] So we turned off the feature, which otherwise was performing quite nicely, and will see if we can get it up and running again tomorrow... [03:23:52] but the source code of a module itself, once loaded, can't vary on user. [03:24:23] AndyRussG: OK. [03:24:43] Krinkle: hmmm, wait a minute... [03:24:52] AndyRussG: If the size of the banner pool is significantly larger when selecting both anons and lius, let me know. There may be a few tricks we can try. [03:25:07] But.. tomorrow. [03:25:08] Uhh yeah no it's not really that big [03:25:44] Krinkle: based on what u said, it occurs to me then that we could add _different_ modules for logged-in vs anon users [03:25:49] https://github.com/wikimedia/mediawiki-extensions-CentralNotice/blob/master/includes/CNBannerChoiceDataResourceLoaderModule.php#L37 [03:25:54] I should've seen that as a red flag [03:26:11] Eh np :) yeah that's where we borked [03:26:19] it using wgUser is strange though [03:26:29] as if that's somehow different from context->getUser; it isn't [03:26:48] Well.. I didn't see user params coming in on the RL URLs [03:27:19] Indeed. That's done for legacy user scripts modules. [03:27:36] Hmmm [03:27:40] But you don't wanna do that, since you'd throw away your client-cache almost completly, and explode server-side cache. [03:27:59] right [03:27:59] as it uses username [03:28:04] yeah [03:28:32] and also bypassses the resourceloader delivery system (e.g. not compatible with mw.loader.load and addModules) [03:28:37] Yeah, so I saw the URL param was checked ( $this->user = $request->getVal( 'user' ); in RLContext line 62) [03:29:20] and also saw that $wgUser was checked in a few code paths there.. [03:29:48] However I should have asked explicitly and also (ahem) just smoke tested that the system correctly distinguished logged ins vs. anons [03:30:04] It's easy to miss. [03:30:48] Hmmm... and also easier to ask when you're not sure, and remember that there was something there to double-check, as I should have [03:31:40] AndyRussG: How was the result in prod earlier today? [03:32:06] Anyway, if the options are sending a bit more data to the client vs. causing more cache explosion or doing strange things w/ RL modules... I guess the first option takes it [03:32:18] Was it double checking the conditions client-side (e.g. less banners were shown than we should've) or were we potentially showng anon-only banners to logged-in users or visa versa [03:32:20] Krinkle: the result was not bad, save this and one other issue [03:33:02] AndyRussG: We might be able to craft a solution that would involve fragmenting it by anon/not-annon (as opposed to by anon/{username} which is the legacy solution you refer to) [03:33:37] Krinkle: no client-side double checking, and we _did_ show logged-ins fundraising banners, and someone complained, thankfully right away [03:33:58] Yeah that wouldn't be too much fragmenting [03:34:24] The main backend part where it fragments is Varnish, which is a non-issue for this purpose. [03:34:33] Hmm OK [03:34:36] The main front-end concern is the size of the startup module [03:34:49] that would contain two modules for all users (anon and non-anon) instead of one. [03:34:55] From a practical perspective: there is a year-end fundraiser looming, and we have only a few days to either get this working and test it sufficiently, or leave it off definitively until January [03:35:17] and only end up loading one of them. The other being kind of registered akwardly. And if someone where to load it somehow, who knows what'd happen. [03:35:51] Yeah sounds complicated [03:36:03] I'm not sure how bad the current performance is, but if that's feasible overal, and this being an improvement, then I'd go with the safe route and stick to one module and just end anon/non-anon to the client and decide there. [03:36:06] I think whatever solution is taking for now (i.e. tomorrow) it'll have to rely on something that's tried and true [03:36:17] send* [03:36:37] Though I'm curious exactly how they compare. I'm worried it's biased towards backend performance. [03:36:56] Krinkle: yeah I think that's it for tomorrow, it's doable and at least we have some unit tests ticking away [03:36:59] I can't imagine the current non-RL solution being that bad for the front-end. Considering it must be much smaller in terms of bandwidth, right? [03:37:38] Krinkle: it's not really bad for the frontend but it's reached limits that Fundraising needed to go beyond [03:37:39] but it does mean 1 non-RL http request (which we'd not have when using RL) [03:37:45] and presumably the current request is non-cachable. [03:38:18] Well the current (hopefull soon to be previous) system makes a request to a funky page called Special:BannerRandom [03:38:26] That is actually cached and is not really random [03:38:39] To make that work, there's a thing called slots [03:39:02] So the client randomly chooses from among 32 possible slots and puts that into a param on the URL [03:39:31] That means that banner allocations are quantized, you can't have a campaign running at exactly 1% for example [03:39:43] Though that's an annoyance, not the real problem [03:39:51] The real issue is that fragmentation was kinda at the limit [03:40:12] And we needed to add more parameters to the banner selection process, that is, with the current system we would have had more fragmentation [03:40:43] So that's what the change to client-side banner selection aims to solve: being able to use more criteria for choosing a banner [03:40:44] AndyRussG: Right, so performance wise the old system did quite well. Both front- and back end. [03:40:52] But at the cost of being limited in abilities and parameters. [03:40:52] Yeah [03:40:58] Alrighty. [03:40:58] Yeah [03:41:07] The new system also has a few perfomance advantages [03:41:31] 1 less http request. [03:41:35] In addition to a less fragmented cache, it makes one less background request for users that we know definitely would not get a banner [03:41:48] Right [03:42:10] yea, in the previous system everyone makes the request even if they're not in a project or language that's targeted [03:42:28] Though the cache fragmentation of that part only affects squid/varnish special-page urls. I assume in the backend cache could and remains non-fragmentedly cacheable already. [03:43:17] Hmmm, in which cache? [03:46:26] I don't know of any explicit caching in the backend code... [16:46:17] (PS1) AndyRussG: Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 [16:48:11] (CR) jenkins-bot: [V: -1] Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [17:03:35] Morning ejegg! [17:06:55] Hi AndyRussG ! Looking over CN fix. Seems good so far [17:07:16] ejegg: Fantastic, thanks! I'm just fixing tests and tweaking a bit [17:07:27] AndyRussG: That was fast. [17:07:36] * AndyRussG waves at K4-713 [17:07:42] Helloooo [17:08:26] K4-713: Thanks... fortunately it's fresh enough that I have most of the code layout still in short/medium-term memory [17:08:34] Looks like it's a fun morning, so far. [17:08:47] It's a beautiful day in the neighbourhood! [17:09:11] awight: ... you're not here, are you? [17:09:48] nope [17:10:20] maybe awight plans to take double time off on Thursday ;) [17:10:45] :) [17:11:01] meh, I'm just making a bookshelf today, and the studio isn't open 'til 10. [17:11:30] awight: Neat. So, what's up with recurring? :) [17:11:31] Plus, this helps steel my resolve to kill a veggie turkey :p [17:11:36] (PS3) Ssmith: Add board stats piece [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/175161 [17:11:38] (PS4) Ssmith: Big English header piece [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/175055 [17:11:40] (PS1) Ssmith: Add totals earned chart widget [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/175742 [17:11:47] K4-713: Something in the standalone DI is dying, and slowly. [17:11:56] I'm looking in syslog... [17:11:58] Seriously. [17:12:08] * pizzzacat1 thinks she figured out the issue from last night when she couldn't push to gerrit [17:12:41] pizzzacat: Woot! [17:12:49] You're not in town today, are you? [17:13:14] I am in town, just wfh to try to pound this out by the end of the day... [17:13:26] Ah, makes sense. [17:13:33] I leave at the crack of dawn [17:13:35] :) [17:13:48] where 2 now? [17:14:01] ohh. right, the looney bin [17:14:03] Seattle for Thanksgiving with friends [17:14:06] :) [17:14:12] hehe [17:14:24] Don't make fun of my looney bin! [17:15:12] Jeff_Green: I have a strangeish request. [17:15:22] You're not going to like it. :) [17:15:47] K4-713: cURL transaction to Global Collect failed: (7) couldn't connect to host [17:15:50] poodle. [17:15:56] awight: That happens sometimes. [17:16:32] K4-713: every time... [17:16:40] awight: Huh. [17:16:57] I think I'm the one who said "we have absolutely nothing to fear from the poodle fix other than fear itself" [17:17:10] Fear, spiders, public speaking... [17:17:34] GC isn't all the way down right now, is it? [17:20:08] Checked. No, it's fine. [17:20:10] Okay. [17:21:02] Well, at least that makes sense? [17:21:18] yep. I shall disappear, then [17:21:54] Someone else can reenable the job and set it to reprocess... everything since Nov 19 ? [17:22:55] Yow. [17:23:00] Okay. [17:23:31] ccogdill... might want to know, too. [17:23:40] yep. Always. [17:23:43] 6 days isn't too bad though [17:24:35] For us? [17:24:36] :p [17:24:42] HEHE [17:25:24] kbye [17:25:24] (PS2) AndyRussG: Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 [17:25:35] awight: bye! [17:25:57] (CR) jenkins-bot: [V: -1] Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [17:38:42] (PS3) AndyRussG: Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 [17:39:12] (CR) jenkins-bot: [V: -1] Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [17:43:26] blarg jslint [17:44:10] Jeff_Green: So, that weird thing I was mentioning earlier. [17:44:30] I need to modify one of the GC nightly files. [17:44:52] They added garbage that is ruining everything, and coding around it creates a security hole. [17:44:59] ejegg: the patch I think is ready now, just a silly jslint error boo [17:48:17] (CR) Awight: [C: 1] "Looks great!" (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [17:48:22] K4-713: sounds positively swell [17:48:40] It is one of my more favorite breaking things in the last week. [17:48:47] Which at last count was, in fact, everything. [17:49:02] part of chaos week, which I forget the name of already [17:49:08] Freak Week! [17:49:11] that one! [17:49:32] Freak Week + buried in cruft = ...? [17:49:46] fabtabulous [17:51:25] Jeff_Green: I'm looking for the exact thing I need whacked now, but it might be weird, because of course it's a directory that somehow moves things into archive. [17:51:53] AndyRussG: OK, looking it over again [17:52:14] ejegg: Thx! I think awight acught the silly jslint thing... [17:52:37] (PS4) AndyRussG: Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 [17:52:58] K4-713: ok [17:53:14] oh, the comma. i see [17:53:30] yeah! hrmf [17:53:48] * AndyRussG retracts all the complaints he's ever made about jslint [17:56:05] (CR) AndyRussG: Filter logged-in status on client (1 comment) [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [18:00:30] You guys see about the banner impresisons? [18:00:43] ejegg: looking at today's deploy schedule.. There's a 1.25wmf9 MW train at 11 PST. If we merge this into that branch, do we get to ride the train, even though that version is already deployed elsewhere? If we could take the train, we'd get a free deploy to non-wikipedia sites, which we could test for a bit, and maybe really deploy later? [18:00:57] K4-713: I haven't... good, bad or bizarre? [18:02:15] K4-713: Since we had the issue of sending FR banners to logged-ins, and then turned off the feature, I hadn't thought ot check them... [18:02:58] AndyRussG: So, let's do... you know, one thing at a time. [18:03:09] K4-713: Yurp [18:03:33] I think we're going to get some qa assistance from ellery once we get the CN stuff patched and re-enabled. [18:03:36] K4-713: Mmm so your question was whether we'd checked the impressions, or... is there some news about them? [18:03:48] Ah, no. There's a definite problem. [18:03:53] It's been described in email. [18:04:00] AndyRussG: isn't on those emails [18:04:04] But it's been a problem for a while. [18:04:07] ...oh, flip. [18:04:19] Why did I think I actually did that in a timely manner? [18:04:28] * K4-713 makes a ticket [18:05:29] AndyRussG: OK, think that the 11 train is a good plan [18:05:42] with this change impressions should be essentially the same, with just a few different params on empties and non-empties [18:06:22] ejegg: cool! So we _can_ ride on it even though wmf9 was already deployed elsewhere? [18:07:16] AndyRussG: mmm, i'm pretty sure we can [18:07:37] ejegg: cool! ...umm do you want me to ask, or are U sure enuf? [18:07:56] can't hurt to ask [18:08:23] atgo: thanks for forwarding the e-mail :) got it, but it's missing the attached file Ellery mentions... [18:08:30] oh 1 sec [18:08:37] ejegg: K [18:09:00] AndyRussG: forwarded that one, too [18:09:07] atgo: thanks! :) [18:11:35] ejegg: K just asked in operations [18:14:13] (PS1) Reedy: Filter logged-in status on client [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175768 [18:14:14] ejegg: Reedy says it wouldn't normally get deployed as part of the train, but that he can deploy it when he drives the train in a bit (if we ask nicely) [18:15:28] Ah, glad you asked! [18:15:30] ejegg: whoa that was fast, sorry I didn't mean to have it supserscede your CR, apologies [18:16:58] Reedy: mind if I get that merged into master first? [18:17:00] heh [18:17:02] Sure [18:17:10] I don't mind too much how you do it [18:17:24] just making one more pass over it, should be 5 min at most! [18:17:32] Reedy: ejegg: thanks! [18:17:35] Window isn't for 43 minutes anyway [18:18:37] ah, cool [18:21:25] Okay, so this is strange. I'm running the recurring job, and yet according to icinga the number of recurring_gc_failures_missed is going up. [18:21:42] Jeff_Green: Can you tell me where icinga is getting that number? [18:29:28] (CR) Ejegg: [C: 2] Filter logged-in status on client [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [18:30:14] woohoo! [18:31:55] looking good [18:33:12] (CR) Ejegg: [C: 2] Filter logged-in status on client [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175768 (owner: Reedy) [18:33:27] ejegg: Reedy also asked if he should add it to wmf8, too, i.e. all wikipedias [18:33:44] AndyRussG: How do you feel about that? [18:33:51] Well the configuration is currently turned off everywhere. [18:33:53] Switch is still off [18:33:55] Yeah [18:34:00] Let's go for it! [18:34:02] Yeah, even on non-wp sites [18:34:05] OK! :) [18:34:18] fyi team - just added ew's banner impression issue to the sprint: https://wikimedia.mingle.thoughtworks.com/projects/online_fundraiser/cards/2196 [18:34:21] When that's merged, we just need to do the bumps in the core wmf branches [18:34:24] I'll thanks atgo [18:34:35] K4-713: it's polling the database [18:34:41] ejegg: huh? [18:34:42] Reedy: OK, I'll submit submodule bumps in wmf8 and wmf9 [18:34:54] that'd be great if you could :) [18:34:56] atgo: thanks for adding the card for that [18:35:03] Jeff_Green: Where can I find the query it's using? [18:35:10] ah no problem! i guess the "i'll" was confusing :) [18:35:12] sorry, running my sentences together [18:35:26] K4-713: sec [18:35:33] Reedy: great... yeah it looks like we will go for deploying everywhere including wmf8 (that's what you were asking/suggesting?) the config switch is still off, and we can still turn it on bit by bit [18:35:41] Maybe this thing is just... surprisingly slow. [18:35:48] cool [18:36:33] K4-713: boron:/etc/puppet/modules/nagios/files/plugins/check_fundraising_jobs [18:36:41] thank [18:36:43] you [18:36:45] ...ugh [18:36:57] AndyRussG: Though.. Depending on when you were considering turning it on [18:37:04] WMF8 goes away tomorrow anyway [18:38:50] Reedy: yeah we may turn it on today... if the issue we're addressing goes away and everything seems to be in order, it'd be nice to have it active a good chunk of tomorrow to monitor results while people are still working... :) [18:42:41] wmf9: https://gerrit.wikimedia.org/r/175775 [18:42:56] wmf8: https://gerrit.wikimedia.org/r/175776 [18:45:18] hey K4-713 did you see the paypal email that pats forwarded? [18:45:44] ejegg: thanks! Reedy ^ [18:46:22] K4-713: it's the same day that the recurring GC issues started... any chance they're related? [18:46:39] i would think not, but big coincidence [18:46:45] Other than it's the week that everything went mad? I don't think so. Maybe. [18:46:59] AndyRussG: I +2ed 'em since they're not actually code [18:47:06] just waiting on zuul [18:47:15] yeah! looks great :) [18:48:10] strange [18:48:35] atgo: ? [18:48:39] atgo: Well, this is cool. We fixed some communication things, and I re-ran the recurring job, and the servers are still screaming. [18:48:46] AndyRussG: not you :P [18:48:54] ... yay. [18:49:13] I'm going to have to keep looking at this until I know what the what. [18:49:48] ejegg: wanna think about a schedule for turning the feature back on gradually while we test [18:49:48] ? [18:51:02] shall I grab a slot for 13:00 PST? [18:51:57] Sure, 13 PST sounds good [18:55:29] ejegg: OK will do! [19:02:48] ejegg: all set: https://wikitech.wikimedia.org/wiki/Deployments#Tuesday.2C.C2.A0November.C2.A025 [19:09:45] AndyRussG: Hey, how's it going? [19:10:01] Just looking through https://gerrit.wikimedia.org/r/#/c/175732/ at the moment. Looks good :) [19:10:46] meganhernandez: Just wanted to let you know, that you should be good to put bpay back up. [19:11:08] We have an extremely ugly manual solution. [19:11:29] And, it seems to have done the job for the moment. [19:15:09] (CR) Krinkle: "In viewing this I noticed a few oddities:" [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175732 (owner: AndyRussG) [19:15:39] AndyRussG: Not code review but just curiosities as I'm interested in learning how CN works. Feel feel to answer later via IRC or elsewhere. [19:16:11] Krinkle: thanks! yeah actualy there's a significant wooops you noticed! [19:16:54] AndyRussG: true vs 1 ? [19:17:05] ejegg: yeah oooooops!!! [19:17:12] aw shoot [19:17:12] lemme fix that quick [19:17:55] (CR) Krinkle: "toggleNotice or toggleBanner? toggleNotice looks still used on metawiki." [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/173661 (owner: Awight) [19:19:26] (PS1) AndyRussG: Fix check of banner logged-in status properties [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175786 [19:19:40] ejegg: Reedy: ^ [19:20:24] AndyRussG: want to get that consistent in the test fixture too? [19:20:27] AndyRussG: lol, is it not quite right? :P [19:20:49] Reedy: exactly, it's lol not quite right, blrgg [19:21:09] ejegg: I have the feeling the test fixture is OK [19:21:47] ejegg: ah wait I see what u mean [19:21:49] I think [19:21:57] yeah, just true vs 1 again [19:22:44] ejegg: in the test fixture all I see is true, which is what's desired [19:22:52] oh? Let me check again [19:23:12] The 1 was this BS from the database that I fixed, but forgot to fix one part in bannerController.lib [19:23:45] ohhh, guess i remember that from looking at the old thing [19:24:33] grrr sorry about that [19:24:42] AllocationsFixtures.json line 20? [19:25:20] what the... [19:25:36] that shouldn't even be there... [19:26:33] (PS2) AndyRussG: Fix check of banner logged-in status properties [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175786 [19:27:43] ^ sorry when I said test fixture, I was looking/thinking of the actual tests... [19:28:32] Hmm. Seems good to me [19:29:15] famous last words! [19:35:01] This recurring job is surprisingly slow. [19:36:51] Two hours to do 1000? [19:37:00] That's just... terrible. [19:41:28] ejegg: smoke testing above the anon/logged-in filter fix works good [19:41:53] AndyRussG: If they're always boolean, then this seach might help https://github.com/wikimedia/mediawiki-extensions-CentralNotice/search?utf8=%E2%9C%93&q=display_anon [19:42:00] The phpunit one says 1 as well it seems [19:42:32] but will work fine as long as actual logic does boolean [19:42:37] it's one way compatible :) [19:48:42] Krinkle: ejegg: the provider gives a true boolean value, true or false. ( false != 1 ) is true, is ( true != 1) false? I guess it could be if the 1 is truthily cast to a boolean true [19:49:59] Krinkle: it's confusing because in the DB the values are indeed 0 and 1. But I wanted to keep that mess in the DB and work with actual booleans, since that's the intention behind these vars [19:50:30] AndyRussG: !0 = true, !1 is false. That's what I meant with casting. direct comparison is not compatible javascript in either direction [19:50:42] a few exceptions there but don't count on it [19:51:06] so if (!display_anon) is "safe" [19:51:40] it's more important for the logic to switch to boolean than the data to be boolean or integers. Ideally it's bools everywhere of course. [19:52:13] btw, if this uses the standard api value result interface, boolean properties will be empty strings instead of boolean or integer. [19:52:24] due to xml [19:53:11] When the Web API is used it comes in as Json and I'm pretty sure I checked the types are preserved. In this case we're not using the Web API code path [19:53:41] hey meganhernandez - was there more you wanted to talk over today? i hvae our checkin as a placeholder in 5 [19:53:41] Just straignt PHP data struct encoded up as Json like you see in the RL module [19:53:54] Jeff_Green: I have more for you. The iginca alert? It looks like two of them are looking at the same named parameter. [19:54:19] Those numbers should actually be totally different. [19:56:26] hi atgo [19:56:32] i can call in 4. [19:56:43] i think we can be quick [19:56:53] Krinkle: nice github codegrep there BTW! it looks like everything's cool there, a lot of stuff that doesn't have to do with this change [19:56:55] how about i just call your phone and avoid any possible hangout drama? [19:57:46] haha sure [20:00:00] AndyRussG: It goes wikimedia-wide as well. https://github.com/search?q=display_anon+%40wikimedia&type=Code&utf8=%E2%9C%93 [20:00:21] though nothing outside CentralNotice for that search, naturally. [20:00:48] cool! [20:00:52] (PS1) Ejegg: WIP: add is_YYYY_donor and latest don data to wmf_donor [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175794 [20:00:58] (CR) jenkins-bot: [V: -1] WIP: add is_YYYY_donor and latest don data to wmf_donor [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175794 (owner: Ejegg) [20:11:25] Krinkle: ejegg: strange tho it may seem, I think the displayAnon vs display_anon in the test fixture is correct, at least for now. When it's dislayAnon it's in reference to the parameter $displayAnon in the php code Banner::addTemplate(), where as display_anon is the expected result from the BannerChoiceDataProvider, which mirrors the DB column cn_templates.tmp_display_anon [20:11:35] K4|lunch: which ones? [20:12:43] AndyRussG: yeah, that makes sense. I was only concerned about just the stray display_anon: 1 that you just zapped [20:13:30] (CR) Ejegg: [C: 2] Fix check of banner logged-in status properties [extensions/CentralNotice] - https://gerrit.wikimedia.org/r/175786 (owner: AndyRussG) [20:16:18] ejegg: yeah I thought I'd zapped that stray display_anon earlier... must've accidentally gotten thru on a commit --amend [20:16:52] ejegg: whereto no? [20:16:58] now? [20:17:00] I meant [20:17:13] deploy window in 45 min, right? [20:17:22] * AndyRussG goes for moar coffee, needed to revive self after near death from embarassement [20:17:26] yep [20:18:01] Let's start with code to wmf9 & re-enable on test + aabooks + mw.org [20:18:08] Reedy: ejegg: sorry to have wasted people's time due to the above-corrected oversight [20:18:40] ejegg: K, sounds good, thanks [20:21:10] yurp, indeed on my JS console: [20:21:10] > true != 1 [20:21:10] < false [20:25:02] (PS1) Ssmith: Add distance-to-goal chart views to BE board [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/175801 [20:44:07] (PS1) Ejegg: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175804 [20:44:30] AndyRussG: want to CR that merge (over a cherrypick) [20:44:33] ? [20:44:51] ejegg: OK [20:47:11] (CR) AndyRussG: [C: 2] Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175804 (owner: Ejegg) [20:52:53] (PS2) Ejegg: WIP: add is_YYYY_donor and latest don data to wmf_donor [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175794 [20:54:43] ejegg: config change here: https://gerrit.wikimedia.org/r/175849 [20:55:10] thanks! [20:55:28] np likewise... I've never done one of those before, so forgive me if it's backwards.. [20:57:15] Might want another set of parens around the wikibooks + aa [20:58:44] ejegg: done! [21:01:06] ejegg: I would make the wmf8 and wmf9 changes, but since we're down to the wire, I'd kinda rather defer to your experience, is that OK= [21:01:07] ? [21:01:55] Yeah, unfortunately still waiting for zuul to merge CN to wmf_deploy [21:02:36] Hmm! [21:03:02] (Merged) jenkins-bot: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - https://gerrit.wikimedia.org/r/175804 (owner: Ejegg) [21:03:08] there it is! [21:03:18] yalp! [21:03:26] zuul is slo today [21:03:38] jeez, 15 min or so [21:04:16] yurf [21:05:16] ejegg: pls lemme know what I can or should do... [21:05:35] nuffin yet [21:06:07] just getting the wmf9 bump merged [21:10:55] ok, merged. [21:11:01] heading up to deploy [21:11:05] Jeff_Green: the icinga check_recurring_gc_jobs_required check, is looking at recurring_gc_failures_missed. It also seems there's a recurring_gc_failures_missed check, which is looking at recurring_gc_failures_missed. [21:11:18] ejegg: Cool! Maybe given the slowness we should merge wmf8 and the partially-enabling config change right away, too? [21:12:37] hurf. Looks like you guys are having fun. [21:12:42] let's do this one step at a time [21:12:59] ejegg: OK [21:13:17] K4-713: looking forward to more fun checking out impressions in a sec... :) [21:13:30] aye [21:14:50] aagh sorry i missed standup! [21:15:02] my computer isn't yelling at me about meetings like it normally does [21:15:20] In one sense I don't think any of us _missed_ the standup. [21:15:23] But it went by. [21:15:23] hahahaha [21:15:28] well, indeed. [21:15:29] AndyRussG: OK, 1.25wmf9 deployed [21:15:34] K4-713: not fully understang the schema, I have no idea, I just work here :-) [21:15:37] Wee!!! [21:16:02] again testing it's not busting stuff when turned off [21:16:13] Jeff_Green: Ha, no problem. There's a number to check, that seems to match the name of the icinga check that's looking at the other one's number. [21:16:23] That... may not have made any sense at all. [21:16:34] Wait 1. [21:16:34] it might make sense [21:16:47] let me look at how the test is actually run, maybe I typod somewhere [21:17:31] AndyRussG: cursory check shows OK with the switch off. Let's do your config change [21:17:35] K4-713: indeed that's it [21:17:38] look: [21:18:00] er, actually. hrm [21:18:03] ejegg: OK, yeah things look OK from here too! [21:18:23] at least that merged quick [21:21:00] ejegg: seeing the new code on mediawiki.org [21:22:03] K4-713: should be better now [21:22:17] Jeff_Green: Thanks! [21:22:18] OK, about to throw the switch [21:22:52] OK, synced the config change [21:23:57] ejegg: OK, I'm getting banner choice via the new system on mediawiki.org [21:24:41] K lemme test that I can get a fundraiing banner somewhere, then that I don't get it logged in... [21:25:19] rockin [21:29:01] ejegg: I don't see what "project" mediawiki falls under.. [21:29:16] wikimedia, confusingly enough [21:29:26] heheh OK [21:32:02] I'm going to deploy the code to wmf8 since it seems not to break things when off [21:32:43] ejegg: OK sounds great! I'm going to see how to test on production, I don't see any FR campaigns except on wikipedias... [21:33:07] reactivate aabooks campaign or esperanto mediawiki? [21:33:26] I think it takes too long. Rather I think I'll just go into the debugger and change settings [21:33:40] OK [21:33:51] uurg no that won't work, since that's data we get from the server... [21:34:01] Well I could copy it into choiceData [21:34:03] atgo: I think I finally got out of the woods. [21:34:05] * K4-713 blinks [21:34:16] woo!! welcome to the light K4-713 [21:34:31] recurring alive? [21:35:04] I mean, don't get me wrong. I love it when servers starts squaking at the exact moment I board the train in to work. [21:35:12] Those are THE BEST DAYS. :| [21:35:27] maybe... is it better than when you're boarding the train home? [21:36:01] I suddenly want to graph all the alerts I've ever gotten. [21:36:45] speaking of alerts, actually, megan and i were wondering if we should be getting any of the alerts? [21:36:46] If you rule out the ones an active fr-tech person caused directly (me!), I feel like they disproportionately cluster between 1am and 5am. [21:37:04] ha [21:37:06] atgo: That is an interesting question. [21:37:10] #1 - I think you do. [21:37:16] Most of them go to fr-tech. [21:37:31] are those the grack ones? [21:37:37] Er. [21:37:39] Frack? [21:37:42] #2 - None of them are in any way pretty, or descriptive of what the problem is. [21:37:43] ahhhaaa some of them skip my inbox [21:38:18] So, I guess if you want to make extra people panic, but have the extra added bonus that they can't do anything about whatever it is that's trying to tell you something has gone wrong... [21:38:55] Keep in mind, also, that a large number of our alerts do not indicate that users are experiencing problems. [21:38:56] yeah the only thing would be for example the GC thing last week - go and hop on the form and see if it's something there, too [21:39:02] yeah [21:39:10] AndyRussG: OK, code is out on wmf8 [21:39:11] I wouldn't want to take down banners for 75% of the mess that goes on here. [21:39:33] hmm ok [21:39:34] ejegg: fantastic! almost got this tested, silly browser tools issuz [21:39:38] that's sort of what i figured [21:39:48] ejegg: http://makeameme.org/media/created/Brace-yourself-Winter.jpg [21:39:51] are they all the ones from "root"? [21:40:00] They're from all over the place. [21:40:21] I have at least four distinct styles of explosion-related email. [21:40:23] i think the only ones i have that i skip inbox on are the ones from root@ [21:40:34] that sentence was poorly structured [21:40:38] I get a lot of things from root. [21:40:45] Not all of them are catastrophes. :) [21:41:06] Reedy: hope this campaign doesn't last as long as a Westeros winter! [21:41:50] hmmm... ok. maybe i'll try to restructure my filters a bit [21:41:50] ejegg: Can we get fundraising team shirts that say "Winter is coming"? [21:42:05] yes plz [21:42:50] I'm going to. Even if I have to do it myself, with puffy paint. [21:43:39] atgo: So, this is weird. [21:43:54] K4-713: especially with puffy paint. [21:44:03] There's nothing in the backlog or dev-ready, that isn't CN something. [21:44:08] AndyRussG: reactivated esperanto/mediawiki.org test campaign [21:44:17] ejegg: OK makes sense [21:44:34] hm.. interesting K4-713 [21:44:34] 50%throttle, anons only [21:44:40] welp [21:44:42] atgo: I suppose I should record what the heck I was doing this morning, huh? [21:44:49] AndyRussG: also, country=US [21:45:00] That was worth at least one emergency point. [21:45:05] ejegg: finally successfully tested anon/logged filtering [21:45:23] Aw yeah [21:45:31] Had to jump thru a few hoops, but it seems to work, tweaked data on browser console [21:45:48] that would be good to do [21:45:59] Anonymous, banner, log in, clear cookies... d'oh. :p [21:46:52] chromium doesn't seem to be respecting my breakpoints and FF doesn't let me run stuff on the console when paused [21:48:26] atgo: I like our fat sprints. [21:48:37] fat sprints? [21:48:43] They remind me of ice cream bars. [21:49:04] bahaha, the overview graph. [21:49:08] AndyRussG: my tests with the esperanto thing and private browsing seem to be consistent [21:49:13] hahaha i know, right K4-713? [21:49:27] AndyRussG: so, turn it back on everywhere? [21:49:47] ejegg: yep!! same result here :) [21:50:39] K4-713: i know ejegg is in the weeds on the CN stuff, maybe could use a hand with 1944 or 2078? [21:51:08] I've got to say, I'm just about the last person you want in CN in a hurry. [21:51:21] ha yeah but there are non-CN things [21:51:21] K4-713: definitely could use some help with the lybunt stuff. [21:51:42] Heh. Don't know that I'll do much better there, but I'll give it a shot. [21:51:44] ejegg: https://gerrit.wikimedia.org/r/175863 [21:52:07] adding a ton of rollup columns to the wmf_donor table so that data doesn't depend on email [21:52:22] atgo: Is there really nothing on fire on the payments cluster right now? [21:52:23] but only have the beginnings of that up for review yet [21:52:44] so what's in the sprint is what's critical for Monday [21:52:54] Huh. [21:52:59] certainly, there are other things in the world that would be useful :) [21:53:15] This WR1 hangover, for example. [21:53:22] yeah that, and PCI [21:53:34] You mean the consultant review? [21:54:48] ejegg: everything continues to check out on mediawiki.org [21:55:19] OK, turned it back on everywhere [21:55:32] ejegg: Yea! just got it on enwiki [21:56:14] K4-713: yeah [21:57:07] AndyRussG: looking good. country=FR gets me fundraising banners logged out and WLA banner logged in [21:57:47] ejegg: checking... I just did get a FR banner logged but it could be some caching issue (new config avail but not the new code?) looks good now [21:59:04] AndyRussG: under what circumstances did you just get the FR banner logged in? [21:59:48] Just got another one, https://en.wikipedia.org/?country=FR&debug=true, it's all the way from the choicesData, wanna check the campaign WLA 2014 FR? [22:00:11] Banner is WLAfricaFR [22:00:19] WLA isn't fundraising, and is supposed to be served to logged-in users [22:00:25] so we're good [22:00:37] ah right [22:01:05] ejegg: not sure... [22:02:47] about what? [22:03:28] Well on enwiki logged in I got again a FR banner and possibleBanners also let an anon-only banner through [22:03:41] Just refreshed browser cache and it looks OK tho [22:03:46] ah dang [22:04:04] let's keep testing a bit? [22:04:08] on enwiki, were you doing country=FR? [22:04:17] ejegg: yep [22:07:49] ejegg: OK, now it seems to be OK on enwiki [22:08:05] Must have been some caching weirdness somewhere [22:09:08] now only non-fundraising banners, ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R ctrl-R [22:09:15] when logged in [22:09:27] veddy good [22:09:35] ...and yes fundraising when logging out [22:10:06] Lemme try w/ the german banner [22:11:14] https://en.wikipedia.org/w/index.php?title=Main_Page&country=DE&uselang=de [22:12:38] ejegg: OK, everything seems to be sehr güt [22:12:50] danke! [22:12:56] bitte! [22:13:03] * K4-713 blinks [22:13:24] gnau mensch [22:14:09] k, i'mma turn my attention back to dash & LYBUNT fun [22:14:20] AndyRussG: let me know if anyone starts hollering [22:14:27] mm that should be genau [22:14:49] OK I'm gonna poke 'em a bit to speed things up [22:14:52] thanks a ton!! [22:15:21] thank you for all the coding! [22:15:30] likewise... :) [22:16:14] ejegg: K4-713: is it OK if I go onto a community IRC channel (like wikipedia-fr) to see if anyone wants to test that they don't get fundraising banners when logged in? [22:16:44] I don't see why not. :) [22:16:56] K thanks [22:17:04] actually looks like that's not the right channel, hmm [22:18:20] ah no I just spelled it wrong, blrg [22:31:47] (PS3) Ejegg: WIP: add is_YYYY_donor and latest don data to wmf_donor [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175794 [22:31:52] (CR) jenkins-bot: [V: -1] WIP: add is_YYYY_donor and latest don data to wmf_donor [wikimedia/fundraising/crm] - https://gerrit.wikimedia.org/r/175794 (owner: Ejegg) [22:32:13] ejegg: K4|wandering: one user wikipedia-fr confirms that we're OK [22:32:43] That's all I need! [22:33:14] :) [22:33:27] I'm gonna leave a note on the phab task just to let people there know it's re-enabled [22:33:47] and also send a note w/ the whole story to centralnotice-admin, if that's OK [22:38:35] AndyRussG: are we all good? that's awesome! [22:41:46] atgo: Looks like we're good! [22:41:58] yay!! [22:42:31] atgo: yay indeed! [22:48:56] K4|wandering: is it OK if I take a break (2 hrs or so) before getting going on the impressions issue? My brain needs a rest, and domestic tasks beckon [22:51:36] meganhernandez: happy to announce that the new banner choosing code is back on. We're monitoring for new issues... [22:52:08] fantastic [22:52:11] thank you AndyRussG [22:52:25] we can set up another test with all 4 buckets for the US [22:52:40] if everything looks good from your side [22:52:44] np, huge thanks to ejegg, awight and others for tons work they've done on this... [22:53:06] meganhernandez: I think so... at least, none of the previous issues seem to be croping up :) [22:53:17] and no new ones found yet, either... [22:56:56] AndyRussG: You should totally do some wandering. [22:57:18] If anything wild happens, we can always killswitch. [22:57:45] K4-713: Ok thanks! Yeah gonna deal with the "I forgot to #{something}"s... Also I won't be far from the computer for very long at any time [22:57:56] :) [23:15:09] (CR) Ejegg: [C: -1] "Comments inline" (4 comments) [wikimedia/fundraising/dash] - https://gerrit.wikimedia.org/r/175055 (owner: Ssmith) [23:38:09] you guys rock! hasta manana [23:39:25] Everything got really, really quiet. [23:39:41] * K4-713 waits for ripples in the water glass [23:42:15] ejegg: How are you? [23:42:51] doing pretty well! [23:43:05] just trying to get the lybunt data into wmf_donor [23:43:19] and keep it up to date with civicrm_post hooks [23:46:58] ejegg: Sounds good. Need anything from me? I went from "OMG ALL THE THINGS" to crickets. [23:47:51] Wait, I take it back. GC is doing something strange again. [23:48:12] phew, can't let K4 get bored! [23:50:16] No rest for the ridiculous. [23:53:58] Ha. I think I just found one of *their* deploy windows. [23:54:09] I should email them just in case they think that was sneaky.