s [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa) [15:09:18] <grrrit-wm> (Merged) jenkins-bot: Fix <title>s [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa) [16:39:21] <wikibugs> Analytics-Cluster, Analytics-Kanban: {wren} - https://phabricator.wikimedia.org/T100027#1303807 (kevinator) NEW [17:37:10] <mforns> kevinator, I just found out that the travel cancellation was my only fault [17:37:25] <kevinator> how? [17:37:55] <mforns> kevinator, in the first email that karen sent me with the schedules, she asked me for a confirmation, which I didn't notice... [17:38:03] <mforns> kevinator, so my bad [17:39:24] <kevinator> ah, ok. so they need to search for a flight again and they you confirm it? [17:39:42] <kevinator> er "and THEN you confirm it" [17:44:23] <mforns> kevinator, yes, will write to her [18:18:16] <milimetric> mforns: checking in [18:18:22] <mforns> milimetric, hey [18:18:27] <milimetric> so, I got TimeseriesData working and merging the way I want, I think [18:18:38] <mforns> cool! [18:18:42] <milimetric> I'm now halfway into converting the wikimetrics-timeseries data converter to use it [18:19:02] <milimetric> once that's done, I have to change the separated-values converter (used for the pageviews) [18:19:17] <milimetric> and once that's done, I'll have to change the vega timeseries graph [18:19:20] <mforns> I have had one on one with Kevin and other stuff he asked me to do in the meantime, so still a long way to go for me [18:19:30] <mforns> milimetric, aha [18:20:09] <milimetric> ok, cool. Want me to commit just the TimeseriesData part? [18:20:35] <mforns> milimetric, this would be good! [18:20:58] <milimetric> you could base your patch on that and output a TimeseriesData set from your converter [18:21:06] <mforns> aha [18:21:10] <milimetric> using the pivot thing that we talked about [18:21:14] <mforns> yes [18:21:18] <milimetric> ok, i'll push just that for review, one sec [18:21:22] <mforns> cool [18:23:23] <grrrit-wm> (PS1) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212799 [18:23:42] <milimetric> mforns: oops, I screwed up :( [18:23:56] <milimetric> I added everything instead of just what I wanted [18:24:29] <mforns> milimetric, maybe I can go with that [18:25:31] <mforns> milimetric, no problem, I can work on that and make it a dependency [18:25:53] <grrrit-wm> (PS1) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212800 [18:26:12] <milimetric> mforns: nono, i'm fixing [18:26:14] <milimetric> hang on :) [18:26:18] <mforns> milimetric, ok, that works too :] [18:26:27] <mforns> milimetric, thanks! [18:26:31] <grrrit-wm> (Abandoned) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212799 (owner: Milimetric) [18:26:43] <milimetric> mforns: ok, this one's ok: https://gerrit.wikimedia.org/r/#/c/212800/ [18:26:58] <milimetric> I'm trying to keep everything more atomic in this chain :) [18:26:59] <mforns> milimetric, ok [18:28:00] <mforns> milimetric, hehe, it's ok, it was a lot of code, no way that you could have split it more... [18:28:54] <milimetric> I definitely should have, and maybe it wouldn't have been as much of a mess. It's ok, you live you get better :) [18:59:40] <milimetric> I'm gonna step out for a bit, brb [19:39:37] <milimetric> (I'm back btw) [21:48:31] <grrrit-wm> (PS1) Milimetric: Refactor Wikimetrics layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/212821 [22:29:08] <grrrit-wm> (PS2) Milimetric: [WIP] Refactor Wikimetrics layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/212821 [22:29:21] <milimetric> woo! almost done :) [22:29:31] <milimetric> TimeseriesData working nicely [22:29:45] <milimetric> have a nice weekend everyone [22:30:10] <milimetric> Marcel, if you're around, you can feel free to jump on top of either my latest patch: https://gerrit.wikimedia.org/r/212821 or the earlier one we talked about [22:30:34] <mforns> milimetric, ok, have a nice weekend! [22:30:36] <milimetric> both are ok points, but there was a slight change in TimeseriesData since that earlier patch [22:30:43] <milimetric> thx :) [22:30:44] <milimetric> you too [22:30:52] <mforns> ok! [23:40:37] <grrrit-wm> (CR) Ricordisamoa: "When will the live site be up to date with the code?" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa)

[02:03:20] Analytics, WMF-Product-Strategy: Provide metrics for WMF quarterly report on January-March 2015 - https://phabricator.wikimedia.org/T97344#1303206 (Tbayer) Obtained all numbers in time and published them on May 15 as part of the report ([[https://commons.wikimedia.org/w/index.php?title=File:Wikimedia_Fou... [02:03:30] Analytics, WMF-Product-Strategy: Provide metrics for WMF quarterly report on January-March 2015 - https://phabricator.wikimedia.org/T97344#1303208 (Tbayer) Open>Resolved a:Tbayer [04:32:14] Analytics-Tech-community-metrics, Engineering-Community, Wikimedia-Hackathon-2015: A new events/meet-ups extension - https://phabricator.wikimedia.org/T99809#1303297 (Qgil) I think this proposal should be advertised among organizers of such events. My first impression is that it is a too ambitious p... [06:09:47] (PS1) KartikMistry: Update Language dashboard with latest deployments [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/212755 [06:11:48] Analytics-Engineering, Analytics-Kanban: Normalize the domain names while querying for uniques based on last-access cookie - https://phabricator.wikimedia.org/T98257#1303334 (kevinator) [06:45:21] (CR) KartikMistry: [C: 2] Update Language dashboard with latest deployments [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/212755 (owner: KartikMistry) [06:45:26] (Merged) jenkins-bot: Update Language dashboard with latest deployments [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/212755 (owner: KartikMistry) [08:12:42] Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table. - https://phabricator.wikimedia.org/T96044#1303426 (JAllemandou) We have a UDF that extracts the project part of the host (en.m.wikipedia.org --> en.wikipedia for instance). A "project" filed will soon b... [08:29:32] Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table. - https://phabricator.wikimedia.org/T96044#1303430 (Yurik) @jallemandou, not fully - ideally I would like any "non-normalized" hosts such as `wikpedia.bg` to be shown as '-', and only our proper normaliz... [09:34:43] Hey halfak ! [09:34:50] ALL HAIL HALFAK [09:34:50] You work french hours ? ;) [09:35:06] :P I'm in Italy ATM. [09:35:12] Nice :) [09:35:19] yuvipanda, I accept this new responsibility and I require grapes. [09:35:34] I have been on a steady diet of 2-3 oranges a day [09:35:36] the last few weeks [09:35:47] That sounds like a scary celebrity diet. [09:35:54] I'm also eating food as well [09:36:02] but *also* oranges [09:36:19] Oh. :) [09:36:31] Just protecting against scurvy? [09:36:31] I haven't gone crazy yet! [09:36:49] no, I discovered that oranges in the US taste different from 'Mosambi' in India (which is also nice) [09:40:03] halfak: still in conf mode or have a minute to chat ? [09:40:44] Conf mode. :\ Lots of irrelevant talks, but not this one. I'll ping when I hit a lame one. [09:40:58] joal, ^ [09:41:17] ok np :) [10:18:44] Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table. - https://phabricator.wikimedia.org/T96044#1303486 (JAllemandou) Filtering for correct hosts is done on our common use cases using the is_pageview flag. I am not sure we want to add another domain format... [10:24:09] joal, boring talk. What's up? [10:24:26] Wanted to loop with you again on scheduling [13:33:03] joal: let's do page table and get title from page_id that is the real solution! :) :) [13:33:19] ottomata: :) [13:33:39] We have pageId for every page_view ? [13:36:15] I haven't double checked so :) [13:36:20] ottomata: --^ [13:38:01] Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table. - https://phabricator.wikimedia.org/T96044#1303621 (Ottomata) is_pageview will help, but this could be useful for non pageviews too, so it might be nice to have. Yurik, do you really need bad host/proje... [13:38:06] joal: ALMOST [13:38:11] except mobile apps/ api [13:38:14] everything else yes [13:38:21] hm [13:38:22] i will check on mobileapps /api [13:38:28] ok [13:38:28] and poke it [13:38:37] https://phabricator.wikimedia.org/T92875 [13:38:46] While you are checking, maybe you double check on standard pageviews as well ? [13:38:49] ;) [13:39:09] Arf ;) [13:39:21] you have enough with the hivecontext and impala stuff :) [13:39:30] I'll double check this weekend :) [13:39:34] Have a good one ! [13:39:47] Analytics-Engineering, Analytics-Kanban: Normalize the domain names while querying for uniques based on last-access cookie - https://phabricator.wikimedia.org/T98257#1303624 (Ottomata) [13:39:48] Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table. - https://phabricator.wikimedia.org/T96044#1303625 (Ottomata) [13:39:59] Analytics-Engineering, Analytics-Kanban: Normalize the domain names while querying for uniques based on last-access cookie - https://phabricator.wikimedia.org/T98257#1263141 (Ottomata) Thanks Madhu, I merged this with Yurik's ticket. [13:40:55] oh you are out! :) ok byyyeyeyey [13:41:07] joal|weeken: i haven't verified page_id at all, but at the very least it is being set by mediawiki [13:41:19] ok :) [13:41:46] thx for the reply on host splitting stuff [13:42:01] Let's agree on something next tuesday [13:42:04] Ciao ! [13:45:10] Analytics-Engineering, MediaWiki-API, Wikipedia-Android-App, Wikipedia-iOS-App: Add page_id and namespace to X-Analytics header in App / api requests - https://phabricator.wikimedia.org/T92875#1303634 (Ottomata) Ok, thanks Matt. So, does that mean that the API code needs to figure out which title... [13:48:58] joal|weeken: just in case you are still lurking, there is also this: [13:48:59] http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_rn_hive_ki.html [13:49:05] Important: Hive on Spark is included in CDH 5.4 but is not currently supported nor recommended for production use. If you are interested in this feature, try it out in a test environment until we address the issues and limitations needed for production-readiness. [13:49:10] also, madhuvishy^ [13:49:24] oh, maybe that is for use within hive [13:49:27] engine spark or whatever [14:04:20] Analytics-Wikimetrics, MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631#1303649 (Milimetric) @Memeht, honestly I feel your pain. Every person I know who's tried to work with mediawiki-vagrant runs into some problems like this. I don't see why apt-get wou... [15:09:12] (CR) Yuvipanda: [C: 2] Fix s [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa) [15:09:18] <grrrit-wm> (Merged) jenkins-bot: Fix <title>s [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa) [16:39:21] <wikibugs> Analytics-Cluster, Analytics-Kanban: {wren} - https://phabricator.wikimedia.org/T100027#1303807 (kevinator) NEW [17:37:10] <mforns> kevinator, I just found out that the travel cancellation was my only fault [17:37:25] <kevinator> how? [17:37:55] <mforns> kevinator, in the first email that karen sent me with the schedules, she asked me for a confirmation, which I didn't notice... [17:38:03] <mforns> kevinator, so my bad [17:39:24] <kevinator> ah, ok. so they need to search for a flight again and they you confirm it? [17:39:42] <kevinator> er "and THEN you confirm it" [17:44:23] <mforns> kevinator, yes, will write to her [18:18:16] <milimetric> mforns: checking in [18:18:22] <mforns> milimetric, hey [18:18:27] <milimetric> so, I got TimeseriesData working and merging the way I want, I think [18:18:38] <mforns> cool! [18:18:42] <milimetric> I'm now halfway into converting the wikimetrics-timeseries data converter to use it [18:19:02] <milimetric> once that's done, I have to change the separated-values converter (used for the pageviews) [18:19:17] <milimetric> and once that's done, I'll have to change the vega timeseries graph [18:19:20] <mforns> I have had one on one with Kevin and other stuff he asked me to do in the meantime, so still a long way to go for me [18:19:30] <mforns> milimetric, aha [18:20:09] <milimetric> ok, cool. Want me to commit just the TimeseriesData part? [18:20:35] <mforns> milimetric, this would be good! [18:20:58] <milimetric> you could base your patch on that and output a TimeseriesData set from your converter [18:21:06] <mforns> aha [18:21:10] <milimetric> using the pivot thing that we talked about [18:21:14] <mforns> yes [18:21:18] <milimetric> ok, i'll push just that for review, one sec [18:21:22] <mforns> cool [18:23:23] <grrrit-wm> (PS1) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212799 [18:23:42] <milimetric> mforns: oops, I screwed up :( [18:23:56] <milimetric> I added everything instead of just what I wanted [18:24:29] <mforns> milimetric, maybe I can go with that [18:25:31] <mforns> milimetric, no problem, I can work on that and make it a dependency [18:25:53] <grrrit-wm> (PS1) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212800 [18:26:12] <milimetric> mforns: nono, i'm fixing [18:26:14] <milimetric> hang on :) [18:26:18] <mforns> milimetric, ok, that works too :] [18:26:27] <mforns> milimetric, thanks! [18:26:31] <grrrit-wm> (Abandoned) Milimetric: Add generic timeseries data representation [analytics/dashiki] - https://gerrit.wikimedia.org/r/212799 (owner: Milimetric) [18:26:43] <milimetric> mforns: ok, this one's ok: https://gerrit.wikimedia.org/r/#/c/212800/ [18:26:58] <milimetric> I'm trying to keep everything more atomic in this chain :) [18:26:59] <mforns> milimetric, ok [18:28:00] <mforns> milimetric, hehe, it's ok, it was a lot of code, no way that you could have split it more... [18:28:54] <milimetric> I definitely should have, and maybe it wouldn't have been as much of a mess. It's ok, you live you get better :) [18:59:40] <milimetric> I'm gonna step out for a bit, brb [19:39:37] <milimetric> (I'm back btw) [21:48:31] <grrrit-wm> (PS1) Milimetric: Refactor Wikimetrics layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/212821 [22:29:08] <grrrit-wm> (PS2) Milimetric: [WIP] Refactor Wikimetrics layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/212821 [22:29:21] <milimetric> woo! almost done :) [22:29:31] <milimetric> TimeseriesData working nicely [22:29:45] <milimetric> have a nice weekend everyone [22:30:10] <milimetric> Marcel, if you're around, you can feel free to jump on top of either my latest patch: https://gerrit.wikimedia.org/r/212821 or the earlier one we talked about [22:30:34] <mforns> milimetric, ok, have a nice weekend! [22:30:36] <milimetric> both are ok points, but there was a slight change in TimeseriesData since that earlier patch [22:30:43] <milimetric> thx :) [22:30:44] <milimetric> you too [22:30:52] <mforns> ok! [23:40:37] <grrrit-wm> (CR) Ricordisamoa: "When will the live site be up to date with the code?" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/211294 (owner: Ricordisamoa)