[01:01:57] <grrrit-wm>	 (CR) Awight: [C: 1] Improve server-side cohort upload form validation [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/138151 (owner: AndyRussG)
[01:29:52] <grrrit-wm>	 (CR) Awight: [C: 1] "Looks good! I'm only worried about the /w/api.php assumption, see for example https://bugzilla.wikimedia.org/show_bug.cgi?id=46992" (5 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/126927 (owner: Awight)
[08:29:22] <grrrit-wm>	 (CR) Nuria: "Besides the comment on line #41 the only other thing I would mention is that" (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946) (owner: Milimetric)
[09:09:00] <grrrit-wm>	 (CR) Nuria: "I think the use case we are trying to satisfy is:" (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/142045 (owner: Terrrydactyl)
[09:14:46] <grrrit-wm>	 (CR) Terrrydactyl: "One of the problems with having just a search function with wiki users is that Jaime wants to be able to see what wiki users are in the co" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/142045 (owner: Terrrydactyl)
[09:26:34] <grrrit-wm>	 (CR) Nuria: ">Since we're doing away with loading the wiki users on the main cohorts >page, the next logical place is to have it on its own separate pa" [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/142045 (owner: Terrrydactyl)
[12:05:59] <grrrit-wm>	 (CR) Milimetric: Add script to bulk insert cohorts or reports (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946) (owner: Milimetric)
[12:06:44] <grrrit-wm>	 (CR) Milimetric: "The cohort screen validation problem needs to be fixed in my other patch, doing that now." [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946) (owner: Milimetric)
[12:36:44] <grrrit-wm>	 (PS1) Milimetric: Fix wiki cohort display [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/142514
[14:16:59] <ottomata>	 milimetric: I merged christian's stuff, you can update labs instances whenever
[14:17:12] <milimetric>	 cool
[14:17:27] <milimetric>	 so the pointers from ops/puppet to the wikimetrics module are updated too?
[14:18:10] <ottomata>	 yup
[14:57:27] <grrrit-wm>	 (CR) MarkTraceur: [C: 2] Separate the opt-in/opt-out actions into their own tab/graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140888 (owner: Gilles)
[14:57:36] <grrrit-wm>	 (CR) MarkTraceur: [C: 2] Add opt-out and opt-in totals to graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140890 (owner: Gilles)
[15:29:45] <grrrit-wm>	 (CR) MarkTraceur: [V: 2] "Right, no Jenkins" [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140890 (owner: Gilles)
[16:15:10] <grrrit-wm>	 (PS3) MarkTraceur: Add opt-out and opt-in totals to graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140890 (owner: Gilles)
[16:15:19] <grrrit-wm>	 (CR) MarkTraceur: [V: 2] Add opt-out and opt-in totals to graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140890 (owner: Gilles)
[16:16:48] <grrrit-wm>	 (CR) MarkTraceur: [V: 2] Separate the opt-in/opt-out actions into their own tab/graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140888 (owner: Gilles)
[16:17:36] <grrrit-wm>	 (PS4) MarkTraceur: Display opt-out totals for each wiki on a global graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140892 (owner: Gilles)
[16:17:56] <grrrit-wm>	 (CR) MarkTraceur: [C: 2 V: 2] Display opt-out totals for each wiki on a global graph [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/140892 (owner: Gilles)
[16:24:59] <Ironholds>	 hey, analysts!
[17:10:16] <grrrit-wm>	 (PS3) Terrrydactyl: Added projects to csv output [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/138474
[17:45:58] <Ironholds>	 hey guys, I actually got an error out of hadoop
[18:00:21] <grrrit-wm>	 (CR) MarkTraceur: [C: 2] Don't query tables that have only data older than 30 days [analytics/multimedia] - https://gerrit.wikimedia.org/r/139780 (owner: Gilles)
[18:00:28] <grrrit-wm>	 (Merged) jenkins-bot: Don't query tables that have only data older than 30 days [analytics/multimedia] - https://gerrit.wikimedia.org/r/139780 (owner: Gilles)
[18:00:33] <ottomata>	 what'dya get Ironholds?
[18:00:43] <Ironholds>	 ottomata, attached to the bug :)
[18:00:47] <Ironholds>	 maximum number of something something
[18:00:55] <Ironholds>	 buried, as usual, in 30 other messages
[18:38:44] <marktraceur>	 milimetric: tgr would like access to limn1 plox
[18:39:02] <milimetric>	 plox?
[18:39:03] <milimetric>	 box?
[18:39:08] <marktraceur>	 Please :)
[18:39:14] <milimetric>	 adding
[18:39:23] <tgr>	 thx
[18:39:44] <milimetric>	 tgr: is your name on wikitech tgr too?
[18:39:57] <tgr>	 the shell name, yes
[18:40:12] <tgr>	 the wikitech username is "Gergő Tisza" or something like that
[18:40:27] <tgr>	 labs had that weird policy for a while
[18:40:55] <milimetric>	 tgr - added
[18:41:11] <tgr>	 \o/
[18:41:53] <marktraceur>	 "please" -> "plz" -> "plx" -> "plox"
[18:42:58] <tgr>	 milimetric: one more question, where can I request access to NavigationTiming data? I have EventLogging access, but the NT tables seem to have separate permissions
[18:43:21] <milimetric>	 they do?
[18:43:24] <milimetric>	 I didn't know that
[18:43:34] <milimetric>	 uh, nuria would know, and ori, but I'm not familiar enough yet
[18:43:46] <milimetric>	 so do you have analytics-store access? and you still can't query them?
[18:44:27] <tgr>	 I have access to stat1003, and the research MySQL password
[18:44:47] <tgr>	 and I get a permission error for "use NavigationTiming_XXX"
[18:45:00] <tgr>	 um, select from, I mean
[18:45:29] <tgr>	 other than that, I only tried MediaViewer-related log tables, those worked fine
[18:46:22] <milimetric>	 hm, that's what I have too, lemme stupidly do the same thing and expect different results :)
[18:47:41] <milimetric>	 tgr, try using the same research user / password, but connect to this box instead: host=analytics-store.eqiad.wmnet
[18:49:51] <milimetric>	 and tgr, use the "log" database there too
[18:50:02] <milimetric>	 I just tried it and it let me select
[18:50:08] <tgr>	 that works, thanks!
[18:50:36] <tgr>	 actually, it seems to work on the slaves now, too
[18:50:49] <tgr>	 maybe I made some stupid mistake earlier
[18:51:14] <tgr>	 generally, is it better to use the slaves?
[18:51:23] <tgr>	 or makes no difference?
[18:56:35] <tgr>	 milimetric: one last question, is there a way to access the squid page view data?
[18:56:50] <tgr>	 I saw it mentioned somewhere that it is stored in hadoop
[18:57:13] <milimetric>	 "squid" no longer exists, it's just named that way to not freak people out
[18:57:17] <milimetric>	 everything comes from varnish
[18:57:19] <tgr>	 ok, ok, varnish
[18:57:22] <milimetric>	 do you mean sampled?
[18:57:33] <milimetric>	 or unsampled raw?
[18:58:11] <tgr>	 well, I want to check file page view count changes after deploying MediaViewer, if it's sampled that's no problem as long as it is unbiased
[18:58:15] <milimetric>	 unsampled raw is the big beast we've been trying to tame.  It's not ready for public consumption and we're just doing mainly ad-hoc work on it internally
[18:58:47] <milimetric>	 sampled, you can see that on dumps: http://dumps.wikimedia.org/other/pagecounts-raw/
[18:59:25] <tgr>	 I was kind of hoping to avoid setting up a database for it myself :)
[18:59:56] <milimetric>	 right, http://stats.grok.se/ allows you to query it, but we have no database or anything fancier
[19:00:04] <milimetric>	 you can access raw sampled logs as files on the server
[19:00:13] <milimetric>	 but we have nothing better than shell access to them either
[19:00:40] <milimetric>	 brb
[19:01:10] <tgr>	 I was hoping for something which lets me sum page views over huge groups of pages
[19:01:34] <tgr>	 like all description pages for images which are used in articles
[19:02:40] <tgr>	 worst case I could use the NavigationTiming data for that, but it only measures certain browsers so it might be biased
[19:09:17] <milimetric>	 k back
[19:09:51] <milimetric>	 tgr: what you'd like is definitely something we're working towards, but there's no great way to accomplish that yet
[19:10:34] <tgr>	 ack
[19:10:52] <tgr>	 thanks for the help
[20:24:27] <ottomata>	 milimetric or nuria, either of you around to try somethign for me
[20:24:28] <ottomata>	 in a few?
[20:24:34] <ottomata>	 (cdh5 in vagrant)
[20:24:36] <milimetric>	 k
[20:24:45] <ottomata>	 i'm double checking my run now, going to commit soon...
[20:29:43] <ottomata>	 ok, milimetric
[20:29:44] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/142698/
[20:29:58] <ottomata>	 pull that change down in mediawiki/vagrant (probably one you don't use)
[20:30:06] <ottomata>	 and then run vagrant destroy && vagrant up
[20:30:11] <ottomata>	 you can just vagrant up if you like...
[20:30:12] <ottomata>	 but, not sure
[20:30:23] <ottomata>	 oo, also
[20:30:28] <ottomata>	 follow the instructions at
[20:30:28] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/142698/1/puppet/manifests/roles/analytics.pp
[20:32:17] <milimetric>	 k, back
[20:34:03] <ottomata>	 k milimetric, ja you can test? :D
[20:34:14] <ottomata>	 vagrant works ok for you now, right?
[20:34:33] <milimetric>	 i just destroyed it
[20:34:36] <milimetric>	 checked out the change
[20:34:39] <milimetric>	 and ran submodule update
[20:34:41] <milimetric>	 now I'm reading
[20:37:22] <ottomata>	 k and vagrant provision...?
[20:37:23] <ottomata>	 running?
[20:37:25] <milimetric>	 and now I did vagrant up
[20:37:54] <milimetric>	 now it's running puppet
[20:39:00] <ottomata>	 cool
[20:39:01] <ottomata>	 right
[20:39:01] <ottomata>	 right
[20:39:08] <ottomata>	 yeah fingers crossed it all jsut works
[20:41:07] <milimetric>	 :)
[20:41:13] <milimetric>	 no errors yet that I could see
[20:43:16] <ottomata>	 :)
[20:44:09] <milimetric>	 k, all ran, no errors
[20:44:18] <milimetric>	 should I do anything else to test?
[20:45:39] <ottomata>	 !
[20:45:40] <ottomata>	 amazing
[20:45:40] <ottomata>	 um
[20:45:43] <ottomata>	 maybe just hive
[20:45:49] <ottomata>	 does hive start up all fine and dandy?
[20:45:51] <ottomata>	 do an
[20:45:54] <ottomata>	 hdfs dfs -ls /
[20:45:57] <ottomata>	 while you are at it
[20:45:57] <milimetric>	 no command found hive
[20:46:19] <milimetric>	 did i not enable some role?
[20:46:25] <milimetric>	 oh of course
[20:46:37] <ottomata>	 oh!
[20:46:37] <ottomata>	 ha
[20:46:44] <ottomata>	 yeah sorry
[20:46:50] <ottomata>	 vagrant enable-role analytics
[20:46:51] <ottomata>	 i think
[20:47:24] <milimetric>	 Could not find class role::oozie for mediawiki-vagrant.home at /tmp/vagrant-puppet-1/manifests/roles/analytics.pp:34 on node mediawiki-vagrant.home
[20:49:24] <ottomata>	 wha
[20:49:24] <ottomata>	 hm
[20:49:25] <ottomata>	 oh!
[20:49:27] <ottomata>	 needs added!
[20:49:28] <ottomata>	 oops
[20:50:43] <ottomata>	 ok, milimetric
[20:50:48] <ottomata>	 uploaded another patchset
[20:50:51] <milimetric>	 checking
[20:53:44] <milimetric>	 it's taking longer now
[20:55:06] <ottomata>	 lots to do!
[20:56:31] <milimetric>	 it's been on this:
[20:56:32] <milimetric>	 notice: /Stage[main]/Apt/Exec[apt-get update]/returns: executed successfully
[20:56:33] <milimetric>	 for a while
[20:56:50] <dr0ptp4kt>	 Ironholds: you there?
[20:57:17] <ottomata>	 hm k its probably fine
[21:04:02] <milimetric>	 k, it's still going, puttering along
[21:11:56] <ottomata>	 milimetric:  how goed it?
[21:16:06] <ottomata>	 hm ok!
[21:23:09] <milimetric>	 it's on this:
[21:23:09] <milimetric>	 notice: /Stage[main]/Cdh::Hadoop::Namenode/File[/var/lib/hadoop/name]/ensure: created
[21:23:12] <milimetric>	 ottomata: ^
[21:23:49] <milimetric>	 sorry, was cooking some naeng myun
[21:25:36] <nuria>	 tgr, the navtiming measures all browsers
[21:26:51] <nuria>	 tgr, it's just that for browsers taht support the api there are more measures
[21:27:02] <ottomata1>	 milimetric: stll going?
[21:27:10] <nuria>	 but at least there is 1 measure per browser for all browserrs
[21:27:10] <milimetric>	 yep :)
[21:27:13] <ottomata1>	 hmmm
[21:27:20] <milimetric>	 notice: /Stage[main]/Cdh::Hadoop::Namenode/File[/var/lib/hadoop/name]/ensure: created
[21:27:20] <ottomata1>	 das weird, shouldn't take that long
[21:27:33] <ottomata1>	 ok...
[21:27:34] <ottomata1>	 hm
[21:27:35] <milimetric>	 no errors, just chugging
[21:32:04] <nuria>	 hola milimetric
[21:32:28] <milimetric>	 hi nuria
[21:32:34] <nuria>	 i thought we said that there was a centralauth role on vagrant , am I missrememebering that ?
[21:33:00] <milimetric>	 i don't know, i think someone said it might have been added at the hackathon
[21:34:15] <yuvipanda>	 nuria: milimetric there is a centralauth role
[21:34:22] <yuvipanda>	 it was added during the hackathon
[21:34:36] <nuria>	 ah, that is what i thought ...
[21:35:35] <terrrydactyl>	 yuvipanda: is it listed when you run vagrant list-role?
[21:35:39] <nuria>	 I do have it, i was just wondering whether it was just a placeholder
[21:35:43] <yuvipanda>	 terrrydactyl: should be
[21:35:46] <nuria>	 it is
[21:35:49] <nuria>	 centralauth
[21:36:00] <terrrydactyl>	 it’s not in my vagrant..
[21:36:06] <yuvipanda>	 terrrydactyl: pull to master?
[21:36:10] <nuria>	 you probably need to pull
[21:36:12] <nuria>	 right
[21:36:16] <terrrydactyl>	 ahhh
[21:37:39] <milimetric>	 ottomata1: progress!
[21:37:39] <milimetric>	 notice: /Stage[main]/Cdh::Sqoop/Package[sqoop]/ensure: ensure changed 'purged' to 'present'
[21:37:52] <milimetric>	 but yeah, crazy slow
[21:39:41] <nuria>	 milimetric , did you bump up the memory in vagrant?
[21:39:45] <milimetric>	 no
[21:39:54] <milimetric>	 he didn't say to :(
[21:42:28] <nuria>	 i bumped it up to 3G in vagrantfile, lemme look...
[21:42:39] <milimetric>	 i sadly only have 4G :)
[21:42:47] <milimetric>	 i mean, total
[21:43:03] <hashar_>	 I have 4G but my LTE mobile is not compatible with the french network baah
[21:43:04] <nuria>	 'vagrant_ram' => 3024,
[21:43:12] <nuria>	 on Vagrantfile
[21:43:18] <milimetric>	 :)
[21:43:39] <hashar_>	 have a good week-end folks!
[21:43:39] <milimetric>	 boy it's Friday night
[21:43:42] <milimetric>	 :)
[21:43:46] <milimetric>	 I was just gonna say hashar_
[21:43:49] <milimetric>	 have a good one
[21:43:55] <milimetric>	 good luck in the roudn of 16!
[21:44:16] <hashar_>	 had to look it up in google hehe
[21:44:27] <hashar_>	 I don't follow soccer. I am too busy with Jenkins/Python!
[21:44:35] <milimetric>	 gasp
[21:44:50] <milimetric>	 then you missed zidane?  Wizards are fascinating regardless if you like the sport they play
[21:45:21] <hashar_>	 in 1998 I was on IRC and got frightened when the whole Parisian neighborhood started to scream.
[21:45:39] <hashar_>	 figured out we were in world cup final and landed our first goal.  I went back to irc
[21:45:41] <hashar_>	 good time
[21:45:51] <hashar_>	 anyway didn't want to distract from your vagrant box
[21:45:54] <milimetric>	 lol
[21:46:14] <milimetric>	 have a good one
[21:46:36] <nuria>	 ciao harshar_
[21:46:46] <nuria>	 hashar
[21:48:05] <grrrit-wm>	 (PS2) Terrrydactyl: [WIP] Add ability to global query a user's wikis [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/129858
[21:48:26] <terrrydactyl>	 ignore this code change, my vagrant is being buggy and i wanted to push my code in in case something goes wrong
[21:53:18] <milimetric>	 ottomata1: it errored out, but I think because of memory issues
[21:54:10] <ottomata1>	 did you up the ram?
[21:54:13] <ottomata1>	 milimetric: ?
[21:54:33] <milimetric>	 It was at 2048
[21:54:37] <milimetric>	 in setttings.yml
[21:54:55] <ottomata1>	 and you did that before you created the vm?
[21:55:01] <ottomata1>	 or you did vagrant reload in between?
[21:55:43] <milimetric>	 i did that before creating the vm, yea
[21:55:44] <milimetric>	 before vagrant up rather
[21:55:49] <milimetric>	 after vagrant destroy
[21:55:59] <milimetric>	 but I upped it to 3072 and I'll try again
[21:56:57] <milimetric>	 just to make sure, this is what my settings look like:
[21:56:58] <milimetric>	 ---
[21:56:58] <milimetric>	 GIT_USER: 'milimetric'
[21:56:58] <milimetric>	 vagrant_ram: 3072
[21:56:58] <milimetric>	 forward_ports:
[21:56:59] <milimetric>	     5000: 5000
[21:57:01] <milimetric>	     8888:  8888,    # Hue
[21:57:03] <milimetric>	     8088:  8088,    # Hadoop Job GUI
[21:57:05] <milimetric>	     50070: 50070,   # Hadoop NameNode GUI
[21:57:08] <milimetric>	     11000: 11000    # Oozie
[21:59:41] <grrrit-wm>	 (CR) Nuria: Add script to bulk insert cohorts or reports (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946) (owner: Milimetric)
[22:01:15] <milimetric>	 (i'll fix that in a minute then nuria, just gotta deploy the new backup system)
[22:02:11] <nuria>	 i can deploy if you want (unless you are in teh middle)
[22:27:50] <milimetric>	 ottomata1: working!
[22:46:55] <grrrit-wm>	 (PS6) Milimetric: Add script to bulk insert cohorts or reports [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946)
[22:47:16] <grrrit-wm>	 (CR) Milimetric: Add script to bulk insert cohorts or reports (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/141077 (https://bugzilla.wikimedia.org/65946) (owner: Milimetric)