[14:03:33] hey ottomata qchris [14:03:43] yo, in standup :) [14:03:48] Standup time! [14:03:52] Right :-) [14:03:59] omg -- I'm running away [14:04:01] ;) [14:04:05] I'll ping you in a bit [14:04:37] qchris_meeting: ottomata mind having me lurk in out of curiosity ? :D [14:04:42] I feel lonely! [14:05:10] baw please [14:05:12] comeo on in [14:05:18] https://plus.google.com/hangouts/_/calendar/d2lraW1lZGlhLm9yZ19jYjM3bXU0OGNuaHRkN2hybmE4czI3b25hb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29t.c6j7qidqs491nhi7ovk9pi4h14?authuser=1 [14:20:55] you guys out of standup? [14:21:36] people talkin [14:21:39] whatttsssup? [14:24:23] wanted to get some info on the mobile page views -- were you able to agree on a name? [14:24:53] ja, it will be hosted under the name pagecounts-all-sites [14:25:01] donde? [14:25:09] and, i have a patch to get it out [14:25:10] it will be at [14:25:24] dumps.wikimedia.org/other/pagecounts-all-sites/ [14:26:57] thanks I feel replenished [14:27:11] anytime hashar [14:27:17] we're here all week [14:27:20] tnegrin: Documentation is up at https://wikitech.wikimedia.org/wiki/Analytics/Pagecounts-all-sites [14:27:35] nice qchris [14:27:42] the site 404s currently [14:27:57] (the path ottomata sent) [14:28:03] Yup. [14:28:14] That's expected. [14:28:24] https://gerrit.wikimedia.org/r/#/c/164124/ [14:28:28] needs to get merged. [14:28:50] This sets up an rsync that fills the webserver with the needed files. [14:29:09] got it -- I'll communicate thusly [14:29:16] ottomata said to work on that today. [14:29:50] Meanwhile, the data is available on stat1002 ... in case that helps. [14:30:05] /mnt/hdfs/wmf/data/archive/webstats/ [14:30:12] is the path to the data on stat1002. [14:31:25] here's what I'm planning on saying at the metrics meeting [14:31:27] We’re releasing new files containing desktop and mobile page views imminently [14:31:28] Working with stats.grok.se and other community-run services to surface [14:31:29] Uses new Hadoop infrastructure (yay!) [14:31:30] Collaboration between Analytics dev, Tech-ops, LCA [14:31:31] We’ll let the analytics/wikitech lists know when data is available [14:31:56] Sounds good to me. [14:31:58] I am very happy to make this announcement -- thanks for the hard work [14:32:12] you good with this ottomata [14:32:43] yay all the way :) [14:33:02] :) [14:34:36] tnegrin: yup good w me [14:34:42] thanks all [14:35:18] yuri's going to talk about the graph extension too! [14:35:27] it's like data day [14:35:42] now if only we could make the PV data easily queryable... [14:39:24] get on it man [14:44:09] YuviPanda: query it in hdfs? :) [14:44:12] in hive? [14:44:16] it is in a table there [14:44:17] ottomata: publicly, I mean :) [14:44:21] aye :) [14:45:05] ok, moving locations back in just a bit [16:03:00] (CR) Gilles: [C: 2] Update schema revision number for NavigationTiming [analytics/multimedia] - https://gerrit.wikimedia.org/r/164084 (owner: Gergő Tisza) [16:03:08] (Merged) jenkins-bot: Update schema revision number for NavigationTiming [analytics/multimedia] - https://gerrit.wikimedia.org/r/164084 (owner: Gergő Tisza) [16:27:32] Say nuria_, can you move the EL delays meeting per our conversation yesterday? [17:19:43] milimetric: did you happen to be at scrum of scrums yesterday? [17:20:17] awjr: yes! and I actually wanted to pick your brain [17:20:23] :) [17:20:29] what do I have to do after I facilitate [17:20:30] milimetric: how did it go? who wound up facilitating? [17:20:37] ah, you, i presume ;) [17:20:42] right, yes :) [17:20:51] milimetric: https://www.mediawiki.org/wiki/Scrum_of_scrums#Facilitator_responsibilities [17:20:59] oooh! great, reading [17:21:34] awjr: so that doesn't say where to post the notes [17:21:43] you send them to developers-l, but do you bcc anyone? [17:21:48] how you choose to interpret that is up to you; i tried to always post the notes on mw.o (linked to that page), distill some of the more urgent things that came up into a few bullet points, and mail out the notes to the engineering list [17:21:49] no [17:21:52] i mean - no to bcc [17:22:15] wait, which list exactly do you send them to? [17:22:16] i would usually just mail out a link to the notes posted on mw.o, with the distilled list of urgent bullet points copied into the email [17:22:26] sorry, I filter all lists to archive now because email is evil [17:22:29] engineering@lists.wikimedia.org [17:22:30] lol [17:22:31] k [17:23:03] feel free to update the documentation for more clarity [17:23:13] cool awjr, will do [17:23:18] milimetric: thanks! [17:23:21] I'm eating lunch but I'll get to it during the metrics meeting [17:23:23] milimetric: how did it go yesterday? [17:23:25] right on [17:23:29] oh went well [17:23:44] good :) [17:23:45] i think people were right on a couple weeks back when you asked [17:23:55] in what sense? [17:24:02] when they said it's usually generally good and some people tune out [17:24:08] but when it's useful it's incredibly useful [17:24:31] and yesterday it sounded like a few people had ops dependencies and I made sure to follow up on those in person [17:25:21] ah yes - terrific [17:26:14] milimetric: since you filter the email lists to archive, you might look for the thread on the engineering@ list subject line 'scrum of scrum, ocrober 1 2014' [17:26:31] milimetric: otto and matt flaschen posted some stuff on that thread since they were absent [17:26:47] oh cool, thanks for the heads up [19:30:18] Analytics / Visualization: Metric definition links are invisible - https://bugzilla.wikimedia.org/71568 (Ryan Kaldari) NEW p:Unprio s:normal a:None At https://metrics.wmflabs.org/static/public/dash/, if you load a metrics view, the title is conveniently a link to the definition of that metr... [19:30:45] Analytics / Visualization: Wikimetrics: Metric definition links are invisible - https://bugzilla.wikimedia.org/71568 (Ryan Kaldari) [19:39:49] qchris: where is teh end point for the mobile pageview files? [19:40:36] nuria_: http://dumps.wikimedia.org/other/pagecounts-all-sites/ [19:59:15] Analytics / Visualization: Wikimetrics: Metric definition links are invisible - https://bugzilla.wikimedia.org/71568#c1 (nuria) Thanks for reporting, design of the tool belongs to design team so we will defer on design decisions to them. Seems like having titles been underlined on, at least, hoover wil... [20:03:49] guys which elliot is the fundraising elliot? [20:04:47] Ironholds: do you know which elliot is the fundraising elliot? [20:05:55] milimetric, er. What's he done? [20:06:17] ah, I think I found him: http://comments.gmane.org/gmane.science.linguistics.wikipedia.technical/77938 [20:06:20] and, I don't, but if you can find awight or pizzacat, they'll know [20:06:21] aha [20:06:36] yeah, he's apparently awight's friend from childhood! :) [20:06:52] he's done nothing wrong, but I've wronged him in forgetting to ping Christian about a Gerrit issue [20:06:58] he mentioned it yesterday in SoS [20:10:33] Analytics / Visualization: Wikimetrics: Metric definition links are invisible - https://bugzilla.wikimedia.org/71568#c2 (Dan Andreescu) Ryan - thanks. This is what I get for rushing a feature right before the demo :) We'll circle with design and they'll set us straight, no worries. [20:11:27] milimetric: do you have a link to the gerrit issue? [20:11:47] qchris: damn, I tried to keep you out of it by using your code name "Christian" [20:11:53] i just sent an email [20:11:57] k [20:12:12] but basically Elliot needed to search all gerrit changes that were self-merged [20:12:34] or, I guess not reviewed by someone (for PCI compliance) [20:12:45] I guess he'll not like the answer :-/ [20:12:51] But I'll chime in on the email. [20:18:32] qchris: since this is potentially a lot of work for them, and error-prone, would it help if I tried to query the gerrit db directly? [20:19:08] You have access to it? [20:19:10] :-D [20:19:38] It depends a bit what they see as "self-merge" and what not. [20:19:50] E.g.: If you upload a change [20:20:00] I modify it, but keep you as author, [20:20:12] and them merge it ... is it a "self-merge"? [20:20:44] Depending on the answer of those questions ... the query might be either easier in answered using gerrit's db or git directly. [20:20:44] heh, yeah, didn't even think of that [20:20:58] But I responded to the email. [20:21:02] thanks very much [20:21:15] ok, feel free to loop me in if the answer is: go query the db directly [20:21:18] thank you for forwarding the request. [20:21:20] i'm happy to write sql, as always :) [20:21:28] Ok. Thanks. [20:24:10] super good job kevinator with presentation [20:24:34] qchris, i forget, do oozie properties get passed to sub-workflows by default? [20:24:46] or do I have to specify them in again? [20:24:51] Thanks nuria_ ! [20:24:55] from your stuff, it looks like they are passed [20:25:06] I got it done in 6 minutes :-) [20:25:39] ottomata: At some level, there is a switch for it. Let me double check whether or not it is the sub-workflow level. [20:26:19] ottomata: Yup. It is the sub-workflow level. You want: [20:27:00] https://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a3.2.6_Sub-workflow_Action [20:27:09] "The propagate-configuration flag, if present, indicates that the workflow job configuration should be propagated to the child workflow." [20:27:25] perfect [20:27:39] thank you [20:30:12] * milimetric has this very bad feeling that he's trading one scrum mastering role for another :) [20:30:47] milimetric: which scrum mastering role? [20:31:06] heh, i'm half-kidding but I just sent out the scrum of scrums notes to engineering [20:31:22] and I see Andrew and I doing that kind of often on account of nobody else volunteering [20:31:32] *Andrew and me. damnit! :) [20:33:20] woa - this is really cool: [20:33:21] {{Special:PrefixIndex|prefix=Scrum of scrums/|hideredirects=1|stripprefix=1}} [20:44:51] milimetric: i think others will volunteer [20:44:53] matt maybe? [21:44:26] milimetric: did you deploy Wikimetrics today with the updated edits and pages created? [21:53:17] (CR) Milimetric: "I like the other changes, just one comment on the state manager." (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/160685 (https://bugzilla.wikimedia.org/70887) (owner: Nuria) [21:54:23] kevinator: no i did not, oops [21:54:24] doing so now [22:01:12] kevinator: ok, all good [22:01:46] kevinator: i ran some anecdotal cohorts I have for testing deployments and the numbers with the new *all namespaces* option are *much* bigger [22:01:51] than with just namespace 0 I mean [22:02:03] the default for that field is still 0 [22:02:32] but i'm just pointing that out - in case anyone's freaking out about HUGE numbers, that'd be the first place to look :) [22:03:06] also - our recurrent edits and pages created metrics are going to start being Crazy pants after today [22:03:26] we have to re-schedule them and manually change the parameters to be what we want, then delete all the old results and backfill [22:03:41] kevinator: I can try to take care of that tomorrow [22:07:31] milimetric: thanks! [22:07:48] I’ll let Jaime know in the mean time that the deploy happened and the numbers will be bigger than normal [22:25:01] Say nuria_, can you move the EL delays meeting per our conversation yesterday? [22:27:32] (CR) QChris: "> EhhHHHhhh, I'd rather this not be a hard and fast rule, but" [analytics/refinery] - https://gerrit.wikimedia.org/r/164102 (owner: QChris) [22:28:10] (Abandoned) QChris: Document agreement on HiveQL filenames starting in a verb [analytics/refinery] - https://gerrit.wikimedia.org/r/164102 (owner: QChris) [22:42:39] (CR) QChris: "> > The name" (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/162589 (owner: QChris) [22:42:53] halfak, yes, sorry i though you were going to do it [22:43:02] Can't. Tried. [22:43:17] :) Stupid google. [22:43:35] It's weird to work at the wiki place and have so many things that require permission and are anti-*bold* [22:52:43] leila: what should we do about staff meetings when everyone is in town? [22:54:25] like 1:1s? [22:54:58] I've asked everyone to cancel as much as possible, but if you'd like to keep your 1:1s, that's cool, given that face-to-face 1:1s are much better. [22:55:14] tnegrin, ^ [22:55:40] I'm less sure about standups, sprints, etc. [22:55:41] 1:1s are cancelled [22:55:52] as are the scrum things [22:56:04] so, it's just staff meeting? [22:56:16] both of them -- I'm inclined to combine them [22:56:56] sure. if you like to have one at all. Alternatively, you can grab coffee with us when you feel like it. [22:57:25] kk [23:32:58] (CR) Nuria: Bootstrapping from url. Keeping state. (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/160685 (https://bugzilla.wikimedia.org/70887) (owner: Nuria) [23:43:35] (CR) Milimetric: Bootstrapping from url. Keeping state. (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/160685 (https://bugzilla.wikimedia.org/70887) (owner: Nuria)