[09:04:12] <grrrit-wm>	 (CR) Joal: [C: 1] "Looks perfect to me :)" [analytics/refinery] - https://gerrit.wikimedia.org/r/273557 (https://phabricator.wikimedia.org/T108618) (owner: BryanDavis)
[10:04:36] <wikibugs>	 Analytics, Pageviews-API, Wikimedia-General-or-Unknown, I18n: PLURAL needed on $1 - https://phabricator.wikimedia.org/T129633#2111302 (Purodha)
[10:40:06] <wikibugs>	 Analytics, Pageviews-API, Wikimedia-General-or-Unknown, I18n: PLURAL needed on $1 - https://phabricator.wikimedia.org/T129633#2111367 (Liuxinyu970226)
[12:01:52] * elukey watch oozie doing its job
[12:35:24] * elukey goes afk for lunch!
[15:10:15] <grrrit-wm>	 (CR) Joal: [C: 2 V: 2] "Self-merging." [analytics/refinery] - https://gerrit.wikimedia.org/r/276534 (owner: Joal)
[15:22:35] <ottomata>	 +1 joal! :)
[15:26:44] <joal>	 :)
[15:26:48] <joal>	 Will deploy on monday
[15:33:58] <mforns>	 ottomata, hi! quick question: to use the hdfs user in stat1002's reportupdater, is it enough to pass 'hdfs' in puppet in the reportupdater class statement?
[15:34:10] <moritzm>	 ottomata, elukey: this morning I enabled some iptables debugging rules on carbon and noticed that kafka1002 (and only that host) is trying to contact carbon.wikimedia.org every 30 mins on UDP/9649. any idea what that may be, is that legtimate traffic we should allow? if you don't know it offhand, I'll run a tcpdump for a closer look
[15:34:42] <ottomata>	 mforns:  i did that yesterday, ja?
[15:35:06] <ottomata>	 moritzm:  hmm, offhand i don't know.   only that host?  weird
[15:35:57] <moritzm>	 yeah, only 1002. I'll make a dump and have a closer look
[15:35:58] <ottomata>	 moritzm: it looks like jmx metrics...to ganglia?
[15:36:39] <ottomata>	 yeah
[15:36:48] <ottomata>	 jmxtrans is configured to send to ganglia on carbon
[15:36:50] <elukey>	 going to carbon?
[15:37:05] <ottomata>	 its also confoigured to do so from kafka1001
[15:37:09] <moritzm>	 but why only 1002?
[15:37:24] <ottomata>	 and also kafka1012 (in analytics kafka cluster)
[15:37:30] <ottomata>	 looks like all kafka brokers are configured to do that
[15:37:42] <ottomata>	 elukey:  probably carbon is the default ganglia aggregator?
[15:37:47] <ottomata>	 in puppet?
[15:38:05] <ottomata>	 OH
[15:38:10] <ottomata>	 or, actually
[15:43:18] <ottomata>	 i betcha these are leftover
[15:43:29] <ottomata>	 not puppetized anymore, but previously were, and not cleaned up
[15:43:34] <ottomata>	 cause we don't really use these stats in ganglia
[15:43:38] <mforns>	 ottomata, oh! cool. Sorry, I saw you decided to go that way, but didn't see the code change
[15:43:38] <ottomata>	 checking, if so, i'll manually remove them
[15:43:51] <ottomata>	 sorry mforns i shoulda added you, i think i pinged you here but you were offline
[15:44:09] <ottomata>	 mforns:  i didn't move the previously created data from the stats owned output dir
[15:44:11] <mforns>	 np ottomata! so now I should be able to just run hive, no?
[15:44:17] <ottomata>	 i think cron is running it
[15:44:18] <ottomata>	 no?
[15:44:29] <ottomata>	 the old output is in /a/reportupdater.stats
[15:44:35] <mforns>	 yes, yes, I mean from the scripts
[15:44:37] <ottomata>	 and the new (hdfs owned) is in /a/reportupdater
[15:44:39] <ottomata>	 um
[15:44:43] <ottomata>	 can you run manually?
[15:44:45] <ottomata>	 uhhhh
[15:44:47] <ottomata>	 can you sudo to hdfs?
[15:44:52] <mforns>	 hehe
[15:44:55] <mforns>	 ottomata, wait
[15:45:00] <ottomata>	 ya you can
[15:45:02] <ottomata>	 you are in analytics-admins
[15:45:03] <ottomata>	 so ja
[15:45:35] <mforns>	 I mean that now, the query scripts that were running beeline with hardcoded user and host, now can be rewritten to use just hive no?
[15:45:44] <ottomata>	 moritzm: elukey, nope, it is puppetized to do that
[15:45:48] <ottomata>	 to send to carbon
[15:45:53] <ottomata>	 dunno why it would only be kafka1002
[15:45:56] <ottomata>	 they are all configured to do that though
[15:46:07] <ottomata>	 mforns: ?
[15:46:14] <ottomata>	 hardcoded user and host?
[15:46:18] <ottomata>	 oh, without the username, ja
[15:46:21] <ottomata>	 because the hdfs user will run them
[15:46:26] <mforns>	 ottomata, ok
[15:46:31] <ottomata>	 so it should use hdfs as default hive user
[15:46:32] <mforns>	 thx!
[15:46:57] <moritzm>	 but why the port? ganglia isn't even listening on 9649?
[15:47:13] <ottomata>	 moritzm: https://github.com/wikimedia/operations-puppet/blob/production/modules/role/manifests/kafka/main/broker.pp#L81
[15:47:45] <ottomata>	 moritzm: https://github.com/wikimedia/operations-puppet/blob/production/hieradata/eqiad.yaml#L27
[15:48:41] <ottomata>	 https://github.com/wikimedia/operations-puppet/commit/b4846929a3d71db7f1534fbe4d203269a2e9435e
[15:49:12] <moritzm>	 yeah, but Giuseppe told me that the value in Hiera is only the starting port to calculate the actual, nothing should communicate directly to 9649
[15:49:23] <moritzm>	 (actual port)
[15:49:33] <ottomata>	 ?
[15:49:53] <ottomata>	 what is 9649 then?
[15:50:09] <moritzm>	 gmond only listens on 9651 onwards
[15:50:16] <ottomata>	 why 9649 at all?
[15:50:22] <ottomata>	 welll, either way, i think we can just remove the  ganglia jmx stuff, we don't really use it
[15:50:55] <moritzm>	 sorry, I have no idea about how ganglia allocates and uses ports
[15:51:08] <ottomata>	 heheh
[15:51:35] <moritzm>	 but if it's unused, let's just remove it to avoid logspam
[15:51:43] <ottomata>	 yeah
[15:51:45] <ottomata>	 removing
[15:57:19] <wikibugs>	 Analytics-Kanban: Build a Dashiki dashboard that shows basic OS and browser request breakdowns - https://phabricator.wikimedia.org/T129102#2111935 (mforns)
[15:58:03] <nuria>	 https://www.irccloud.com/pastebin/NOChDfWg/
[15:58:06] <nuria>	 cc madhuvishy
[15:58:50] <nuria>	 madhuvishy: this is our package
[15:58:51] <nuria>	 https://github.com/wikimedia/analytics-ua-parser
[16:03:16] <grrrit-wm>	 (PS1) Mforns: Add the query folder as the last parameter of scripts [analytics/reportupdater] - https://gerrit.wikimedia.org/r/276758 (https://phabricator.wikimedia.org/T127326)
[16:03:41] <grrrit-wm>	 (CR) Mforns: [C: -1] "Still testing." [analytics/reportupdater] - https://gerrit.wikimedia.org/r/276758 (https://phabricator.wikimedia.org/T127326) (owner: Mforns)
[16:16:41] <milimetric>	 I'm around, btw, but dealing with internet issues at the new apartment
[16:20:13] <grrrit-wm>	 (PS1) Mforns: Improve the browser queries [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/276763 (https://phabricator.wikimedia.org/T127326)
[16:20:47] <grrrit-wm>	 (CR) Mforns: [C: -1] "Still testing." [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/276763 (https://phabricator.wikimedia.org/T127326) (owner: Mforns)
[16:26:40] <nuria>	 ottomata: I do not have permits to ssh into stat1001 but *I think* i have used that machine before
[16:27:43] <ottomata>	 stat1001.eqiad.wmnet
[16:27:43] <ottomata>	 ?
[16:28:02] <ottomata>	 hm!  you are right
[16:28:02] <ottomata>	 you don't
[16:28:03] <ottomata>	 oh
[16:28:05] <ottomata>	 most people don't nuria
[16:28:11] <ottomata>	 do you need it?
[16:28:56] <nuria>	 ottomata: if possible yes cause i would like to look at the directories that serve our datasets
[16:29:04] <nuria>	 ottomata: if it is hard, no big deal
[16:29:12] <ottomata>	 just gotta do that access request thing
[16:29:13] <ottomata>	 uhhh
[16:29:39] <ottomata>	 statistics-web-users
[16:29:41] <ottomata>	 is what you want
[16:30:16] <mforns>	 a-team, having problems with hangouts
[16:30:50] <madhuvishy>	 a-team, getting to office fot the breakfast meeting
[16:30:52] <nuria>	 a-team: standduppp
[16:32:34] <madhuvishy>	 Will join for the end of standup if I make it. I worked on the server side UDP cleanup changes yesterday, and have them deployed on beta. It looks good so far.
[16:43:13] <grrrit-wm>	 (CR) Nuria: [WIP] Add a legend to each graph in the tabs layout (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/276649 (https://phabricator.wikimedia.org/T129497) (owner: Milimetric)
[17:08:21] <grrrit-wm>	 (PS2) Mforns: Improve the browser queries [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/276763 (https://phabricator.wikimedia.org/T127326)
[17:10:49] <grrrit-wm>	 (CR) Mforns: [C: -1] "Still testing." [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/276763 (https://phabricator.wikimedia.org/T127326) (owner: Mforns)
[18:02:47] <elukey>	 a-team: going offline! byyyeeee
[18:02:54] <elukey>	 talk with you on monday
[18:03:01] <mforns>	 bye elukey !
[18:04:05] <ottomata>	 byyyye
[18:06:46] * milimetric goin to lunch - bbl
[18:08:04] <wikibugs>	 Analytics, Analytics-Cluster, Deployment-Systems, scap, Scap3 (Scap3-Adoption-Phase1): Deploy analytics-refinery with scap3 - https://phabricator.wikimedia.org/T129151#2112246 (dduvall)
[18:11:55] <mforns>	 bye team! have a nice weekend!
[18:12:58] <nuria>	 HaeB: hola
[18:13:26] <HaeB>	 hi
[18:14:57] <nuria>	 HaeB: regarding the blog...
[18:15:10] <nuria>	 HaeB: how can I deploy the piwik snippet to automatic
[18:15:33] <HaeB>	 i'll reply to your email soon. i think there may have been some misunderstandings
[18:18:39] <nuria>	 HaeB: i know you are not in charge of the blog anymore. just wanted to know if you know whether it is documented anywhere how to update the code there
[18:18:48] <nuria>	 HaeB: none seems to know...
[18:19:26] <madhuvishy>	 milimetric: you've moved to NYC! Congrats :D
[18:21:33] <HaeB>	 i can deploy code, but that's not the issue. again, will reply soon to hopefully resolve those misunderstandings
[18:35:43] <wikibugs>	 Analytics-Kanban: Update UA-Parser with latest definitions - https://phabricator.wikimedia.org/T129672#2112410 (madhuvishy)
[18:40:08] <grrrit-wm>	 (PS1) Madhuvishy: Update uap-core with latest code from upstream [analytics/ua-parser] - https://gerrit.wikimedia.org/r/276787 (https://phabricator.wikimedia.org/T129672)
[19:00:48] <milimetric>	 madhuvishy: not quite yet :)  I'll be in Philly until April 1st
[19:01:01] <milimetric>	 But the new place is almost done, I'm just here to finish up with the contractors
[19:35:21] <wikibugs>	 Analytics-Kanban: dashiki should report metrics to piwiki in production - https://phabricator.wikimedia.org/T128866#2088649 (Nuria) a:Nuria
[19:37:08] <wikibugs>	 Analytics-Kanban: dashiki should report metrics to piwiki in production - https://phabricator.wikimedia.org/T128866#2088649 (Nuria) Made site on piwik, this is the correct tracking code:  <!-- Piwik --> <script type="text/javascript">   var _paq = _paq || [];   _paq.push(['trackPageView']);   _paq.push(['enab...
[19:42:34] <grrrit-wm>	 (PS1) Nuria: VitalSigns dashboard reports metrics to production piwik [analytics/dashiki] - https://gerrit.wikimedia.org/r/276798 (https://phabricator.wikimedia.org/T128866)
[19:43:33] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] VitalSigns dashboard reports metrics to production piwik [analytics/dashiki] - https://gerrit.wikimedia.org/r/276798 (https://phabricator.wikimedia.org/T128866) (owner: Nuria)
[19:43:50] <nuria>	 fastest CR evah
[19:49:07] <madhuvishy>	  :D
[19:49:41] <madhuvishy>	 ottomata: I pulled the latest code for a submodule from a remote repo - it has some 30 new commits - what is the right way to push it to gerrit?
[19:50:02] <ottomata>	 madhuvishy: git add path/to/submoule
[19:50:06] <ottomata>	 git commit
[19:50:08] <ottomata>	 git review
[19:50:10] <ottomata>	 blabla
[19:50:26] <madhuvishy>	 ottomata: i already did that - no need to git review from inside the submodule right?
[19:50:46] <madhuvishy>	 ottomata: https://gerrit.wikimedia.org/r/#/c/276787/
[19:52:58] <ottomata>	 madhuvishy:  that looks correct
[19:53:01] <ottomata>	 no
[19:53:16] <ottomata>	 you are updating the parent repo's submodule pointer to the sha of the submodule repo listed
[19:53:17] <ottomata>	 in this case
[19:53:19] <ottomata>	 095b648d9350854e05d442137326d73c3a401882
[19:53:23] <ottomata>	 so when someone pulls
[19:53:29] <ottomata>	 after this is merged
[19:53:37] <ottomata>	 and then runs git submodule update
[19:53:41] <ottomata>	 it will chekc out the submodule at that commit
[19:54:27] <madhuvishy>	 oh cool
[19:54:43] <madhuvishy>	 okay then
[20:10:24] <harej>	 I can't keep track of all the statistics pages. Where can I find the page view data for Wikimedia projects, including the comScore information?
[20:12:25] <grrrit-wm>	 (PS12) Ottomata: Add initial oozie job for ApiAction [analytics/refinery] - https://gerrit.wikimedia.org/r/273557 (https://phabricator.wikimedia.org/T108618) (owner: BryanDavis)
[20:14:51] <Ironholds>	 harej, we're not using comscore any more afaik?
[20:15:18] <harej>	 So there's no more information about what percent of a continent's population is reading Wikipedia?
[20:15:32] <Ironholds>	 No, because that information was totally inaccurate
[20:17:00] <harej>	 Alright, thank you
[20:17:47] <grrrit-wm>	 (CR) Ottomata: [C: 1] "Ok, I just tested this, and ran into a few unexpected errors. The latest patchset fixes these." [analytics/refinery] - https://gerrit.wikimedia.org/r/273557 (https://phabricator.wikimedia.org/T108618) (owner: BryanDavis)
[20:31:13] <nuria>	 harej: we are working on data of our user base and we hope to release that pretty soon, as in the next month
[20:32:36] <nuria>	 harej: but comscore numbers were not accurate at all
[21:19:45] <grrrit-wm>	 (CR) Yuvipanda: "I'm going to merge this soon." [analytics/quarry/web] - https://gerrit.wikimedia.org/r/266925 (https://phabricator.wikimedia.org/T76466) (owner: Alex Monk)