[00:11:51] bmansurov: hmm, maybe i just messed up the board u guys keep, but we have so many tags and projects these days... [00:14:11] i think i meant mhurd there [00:14:14] gah, too many nicks as well. whatever, someone will find htem i hope :) [01:19:17] jhobs, around? [06:14:26] bd808, jdlrobson: the getting beaten with a rubber hose thing is why i didn't deploy the change earlier [06:14:41] there's still a merge and sync step that needs to be done even with a bc-only change [09:04:08] hey hey hey [09:08:31] morning! [11:14:00] amazon are doing same-day prime delivery to my house now [11:16:12] phuedx: hello there [11:26:20] hey d3r1ck [11:39:07] phuedx: whats up men? [11:39:10] its been a while [11:40:51] yeah, seems like a long time [11:41:21] yeah. So can we PM, lets not spam others in #wikimedia-mobile :) [12:26:02] i'm out to walk the dog and lunch [15:07:09] so hopefully mobile gurus are around :D [15:07:28] the beta cluster has a bunch of notices related undefined php variables from operations/mediawiki-config [15:07:39] Notice: Undefined variable: wgMFPageActions in /srv/mediawiki/wmf-config/mobile.php on line 95 [15:07:50] Warning: array_diff() expects parameter 1 to be an array or collection in /srv/mediawiki/wmf-config/mobile.php on line 95 [15:07:50] filling that [15:08:14] hashar: there is a patch for that [15:08:19] in swat today i think [15:09:20] oh [15:10:05] bmansurov_away: great, thank you ! [15:10:11] np [15:43:39] hashar: what's the status of the swat window? (i understand that you're running the train) [15:43:44] y'know what [15:43:51] i'll go over to -operations! [15:43:54] ^ that guy [16:10:16] hullo [16:21:00] this font is gorgeous https://github.com/nathco/Office-Code-Pro cc/ phuedx [16:21:07] * phuedx looks [16:23:09] it's like source code pro, but modified, and I'm not sure why it looks so much better [16:26:57] joakino: i'll try it out for the next couple of days [16:36:33] bearND: mdholloway: puppetswat is in 30 mins, could you guys prepare the deploy repo so that we can proceed smoothly when the time comes? [16:37:33] mobrovac: sure, i actually had the deploy commit built and ready to push to gerrit yesterday, i could just push that one or rebuild today, either way [16:37:58] mdholloway: sure, push away [16:38:58] mobrovac: bearND: pushed! [16:39:35] hey phuedx — can Wikimedia tweet a reference to your simulator? [16:39:43] woah [16:39:46] mdholloway: ty [16:39:46] erm [16:39:48] sure! [16:39:52] you’re famous dude [16:40:08] Developing for low bandwidth applications? Wikimedia engineer has built low-cost simulator. https://github.com/phuedx/micro-device-lab [16:40:28] the idea is brilliant btw — facebook has a fancy lab for the same purpose [16:40:56] what did i tell you about the british and crippling anxiety around being seen or heard? [16:41:15] lean in my friend — it’s all about your personal brand [16:42:01] in seriousness, I know, but it actually might be useful for other people! [16:53:58] https://twitter.com/Wikimedia/status/694924430837854208 [16:58:56] bearND: mdholloway: ready? [16:59:23] mobrovac: yep! [16:59:50] mdholloway: ok, i've just merged the patch, please go to deployment.eqiad.wmnet and do the deploy [16:59:59] mdholloway: but no service restarts please, just the deploy [17:00:09] out for a while [17:00:13] (dinner and bath) [17:00:15] later [17:00:19] mobrovac: ok [17:01:55] mdholloway: let me know when you're done [17:02:01] mobrovac: just finished [17:02:10] kk thnx [17:02:17] np! [17:02:18] mdholloway: ty [17:02:47] mdholloway: you should !log the deployment in #ops [17:03:54] mobrovac: done [17:03:58] thnx [17:15:20] bearND: mdholloway: all good, the change went through fine [17:15:32] excellent, thanks mobrovac! [17:15:37] mobrovac: great! :) [17:16:08] thank you guys :) [17:36:12] jdlrobson, phuedx: my attempt to slip https://gerrit.wikimedia.org/r/#/c/267812/ into SWAT failed :( [17:36:39] there might be a window after puppetSWAT and before the train that we can slide into [17:42:57] How can I get to the mobile view of https://wikitech.wikimedia.org from my desktop? To I need to fake my UA or something? [17:44:01] bd808: https://wikitech.wikimedia.org/w/index.php?title=Main_Page&mobileaction=toggle_view_mobile [17:44:10] hey bd808 [17:44:15] awesome thanks dbrant [17:44:19] https://wikitech.wikimedia.org/?useformat=mobile [17:44:22] is my preferred method [17:44:26] ^better [17:44:27] using mobileaction has side effects [17:44:38] (sets cookies/can make it stick) [17:44:45] but if you just want to check out something do that [17:44:45] didn't know about that one! [17:44:58] bd808: thanks for attempting a SWAT [17:45:06] does it need to go in the normal swat window though [17:45:35] ah. I want to look at how the css is loaded on mobile there. I'm seeing a FOUC effect with Mobile.css and assume that is because it is bottom loaded [17:53:59] jdlrobson: no but today is a pretty busy deploy day [17:54:42] jdlrobson: I found more breakage from switching to extension.json too -- https://gerrit.wikimedia.org/r/#/c/268147/1 [17:54:54] aude is working on a better fix I think [17:58:13] bd808: it might take me some time to test my fix [17:58:20] (let me put it on gerrit) [17:58:35] or it might not work [18:15:31] dbrant: bearND kaity standup [18:24:22] aude: I think the config patch I proposed will at least fix the problem in the short term by moving the use of the MFE config variable to after the point it has been declared by the loader [18:24:23] I think [18:27:14] bd808: link? [18:27:39] aude: https://gerrit.wikimedia.org/r/#/c/268147/ [18:27:42] ok, found it and thought of that [18:28:09] oh, wgMFSearchAPIParams should go in there too right? [18:28:21] https://gerrit.wikimedia.org/r/#/c/268182/ is what i propose but either way is ok if it works for now [18:28:44] probably both should go there [18:33:04] aude: yours is probably a better fix isn't it? Mine is pretty much just a hack [18:34:52] bd808: it just has to be maintained if the default in MF changes [18:35:05] and i should probably add a comment saying why we are doing it this way [18:35:13] joakino: hey, can you re-assign the lazy loading patch to yourself? I'm getting unnecessary notifications :( [18:38:22] bmansurov: how do i do that [18:38:46] change the Change-Id of the patch [18:38:56] and abandon my patch [18:39:11] jdlrobson: keep your eye on job #4113 at https://integration.wikimedia.org/ci/view/Beta/job/beta-mediawiki-config-update-eqiad/ [18:39:26] when it is done your config change should be on beta cluster [18:39:33] joakino: there? [18:39:36] yep [18:39:40] thinking about what bmansurov said [18:39:43] about to leave though [18:39:44] joakino: not sure i follow your question about Gather creating a skin [18:40:20] bmansurov: just surpress gerrit notifications via google filters - most of them are useless :) [18:40:22] jdlrobson: when I load an article locally, the Skin code path is run twice, one from mobile.startup/Skin and another one from the gather code [18:40:34] jdlrobson: i'm getting them in irc [18:40:47] joakino: oh i sees [18:40:54] mm [18:41:13] not sure why though, I thought that codepath would only run on Vector [18:41:18] let me take a look i ran into similar issues yesterday [18:41:20] jdlrobson: does it happen to you too? [18:41:47] i'm going to reassign the patch to take all the credit lol [18:41:52] joakino: it's to do with hijacking watchstar i suspect [18:42:36] joakino: errg it's to emit changes yuck [18:43:05] joakino: i can attempt to untangle that today [18:43:51] will create a blocking task in current sprint [18:45:24] jdlrobson: moved the patch to https://gerrit.wikimedia.org/r/#/c/268185/ bc bman [18:45:28] follow up there, sorry [18:46:43] jdlrobson: i've sent the gather-exporter to reading-wmf, if you can try it and see if it works for you it'd be great [18:46:47] joakino: with respect to the parser hook stuff - i commented on the patch with a suggested hook [18:46:53] joakino: will do [18:47:19] i saw it, i'll have a stab at it tomorrow if you don't [18:47:21] going for a smoke and i'll be back for a few minutes [18:47:36] joakino: thanks [18:49:37] Has anyone seen Nirzar today? [19:00:37] YO [19:06:30] phuedx: i have no idea what's going on with those hamburger clicks [19:06:38] they all seem to be going down despite moving to top [19:06:41] yeah… [19:06:43] craziness [19:06:50] except that when you look at historical data [19:07:02] you see HUGE SPIKE, 60%, 30%, HUGE SPIKE [19:07:12] issues with unreliable EventLogging? [19:07:17] bots? [19:07:21] but, presumably, at a higher sampling rate [19:07:30] beacon changes? [19:07:31] because the numbers are 3x [19:08:06] really not sure -- it's definitely worthy of me expanding on my comment though [19:08:18] i'd like to wait a bit and see if this downturn is periodic [19:08:46] The sampling rate did change at Mon Dec 28 17:35:08 2015 from 50% to 10% [19:08:56] for clicks to the hamburger [19:09:01] well that's useful [19:09:57] phuedx: https://phabricator.wikimedia.org/T108723 [19:10:01] just trying to work out when that would have gone out [19:10:26] jdlrobson: honestly, let's turn it off [19:10:33] how often do we look at it? [19:10:42] why aren't we instrumenting /only what we need/ [19:12:26] It would be useful if were were using it for detecting anomalies e.g. clicks go down to a certain element. We've manually identified negative impacts to user experience via graphs e.g. clicks to save mobile options went considerably down when a button moved below the fold AND when we moved the hamburger code to bottom clicks went down there too quite [19:12:26] considerably. I think all our schemas need revisiting - why not start an email thread as really this has to be decided by Adam [19:12:36] We should either use them better or get rid of them [19:43:15] phuedx: joakino bd808 first experiment seems to have gone unexpectedly well.. [19:45:11] how's so [19:52:37] 1s joakino ill post [19:52:55] so we only have 1 result so far [19:53:03] so not gonna jump to conclusions just yet as it could be a blip [19:53:14] HTML Bytes jumped from 187.7k to 152.5k (19% decrease) [19:53:23] https://phabricator.wikimedia.org/F3307936 [19:53:32] https://phab.wmfusercontent.org/file/data/7kwo3w255abthd3k7xlm/PHID-FILE-eentcj2dixyqhoooyvzi/Screen_Shot_2016-02-03_at_11.33.24_AM.png [19:53:48] Before: https://phabricator.wikimedia.org/F3307947 [19:53:58] after: https://phabricator.wikimedia.org/F3307949 [19:54:05] Render time on the beta site saw a 68% improvement (18.3 - 5.9 / 18.3) [19:54:09] Fully loaded improved by 33% (36s - 24s / 36 ) [19:54:13] TTFB unexpectedly saw a 73% improvement (14.7s - 3.9s / 14.7) [19:54:35] but one data point so don't read too much into the data [19:55:18] aha [19:55:29] jdlrobson: i can't see the files, it says only you have permission [19:55:38] waaa [19:56:05] okay will the data on the graph -> https://grafana-admin.wikimedia.org/dashboard/db/mobile-2g?panelId=46&fullscreen [19:56:50] given downwards spike on 29th though it could be a blip [19:56:52] this one better https://grafana.wikimedia.org/dashboard/db/mobile-2g?panelId=46&fullscreen [19:56:57] i'm not an admin ;) [19:57:32] but pretty clear in https://grafana.wikimedia.org/dashboard/db/mobile-2g?panelId=69&fullscreen [19:57:33] interesting [19:57:36] when our change went live [19:57:39] doesn't make much sense though [19:57:43] joakino: use your wikitech user/pass to gain access to admin [19:57:44] joakino: it needs your wikitech/gerrit username and password (e.g. BryanDavis for me) [19:58:03] if you are not in the wmf ldap group we can get that fixed [19:58:31] next result due around 22:00 UTC [19:58:46] (so 2hrs time) [19:58:55] bd808: i think i am, I just can't remember the password, lastpass wont autocomplete there, and i'm too lazy to go fetch it [19:59:02] heh [19:59:03] jdlrobson: looks promising! [19:59:04] once i've got that, we should enable on stable on beta cluster and see if the results are consistent [19:59:16] you and phuedx may need to take over the analysis from there [19:59:48] jdlrobson: send an email end of day with info! [20:01:14] Will do [20:01:25] cool.io [20:01:41] i'm out! 💃 [20:01:51] 🍺 time [20:01:53] i'd like to see a higher number of measurements before we move it to stable [20:01:57] later joakino \o/ [20:02:01] WHISKY TIME [20:02:13] YOU WIN [20:02:18] seriously [20:02:22] just fly over to my house [20:02:53] not a competition joakino [20:02:56] go enjoy your beer [20:03:10] phuedx: my wife and I were talking about the feasibility (or lack of same) of flying to Londaon for dinner last night :) [20:03:16] http://media1.giphy.com/media/Fng6twXJMmCcg/giphy.gif [20:03:17] *London [20:03:28] to be fair i'm already having some rum on the rocks lol [20:04:20] bd808: if you need a place to crash for a night, then you're welcome to stay at ours [20:04:40] just be warned that my children and i wake up at 6 am [20:04:45] ;D [20:43:13] niedzielski: so guess what -- each of those two crashes are from a single user! (i.e. an endless crash loop on startup) [20:43:26] dbrant: yeesh [21:03:34] niedzielski: just sent 'adb shell am broadcast -a android.intent.action.BOOT_COMPLETED' to my Nexus 6P and it was received [21:04:02] mdholloway: alright, must be something funky with my device. thanks for checking. would you mind closing the ticket? [21:04:18] no prob, i'll close it [22:41:51] sigh... [23:05:07] thedj: what's the sigh for?