[08:59:08] <grrrit-wm>	 (CR) QChris: [C: 2] Tox configuration to run flake8 (python linter) [analytics/geowiki] - https://gerrit.wikimedia.org/r/134334 (owner: Hashar)
[08:59:18] <hashar>	 \O/
[08:59:37] <hashar>	 qchris: there is a few more patches for geowiki and wp-zero repo. I haven't actually tested the code though
[08:59:47] <qchris>	 Yes. Finally getting to them.
[08:59:53] <qchris>	 Sorry for taking me that long :-(
[09:01:46] <hashar>	 well those changes are not going to change the face of the world and I don't qualify them as being urgent :D
[09:03:26] <grrrit-wm>	 (CR) QChris: [C: 2] Lint setup.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/134339 (owner: Hashar)
[09:05:12] <grrrit-wm>	 (CR) QChris: [C: 2] Replace git URL to git.wikimedia.org [analytics/geowiki] - https://gerrit.wikimedia.org/r/134340 (owner: Hashar)
[09:09:51] <grrrit-wm>	 (CR) QChris: [C: 2] Lint geowiki/process_data.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/134347 (owner: Hashar)
[09:12:31] <grrrit-wm>	 (CR) QChris: [C: 2] Lint geowiki/mysql_config.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/134342 (owner: Hashar)
[09:13:56] <grrrit-wm>	 (CR) QChris: [C: 2] Lint geowiki/geo_coding.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/134345 (owner: Hashar)
[09:17:16] <hashar>	 bah they don't merge
[09:17:18] <hashar>	 Zuul is locked
[09:17:19] <qchris>	 hashar: Zuul page looks unusually busy ... https://integration.wikimedia.org/zuul/
[09:17:21] <hashar>	 fixing it
[09:17:30] <qchris>	 Oh same thought :-)
[09:17:31] <qchris>	 Thanks
[09:17:39] <hashar>	 which gives me an opportunity to debug it
[09:17:51] <hashar>	 hmm no
[09:22:30] <grrrit-wm>	 (CR) QChris: [C: 2] Tox configuration to run flake8 (python linter) [analytics/wp-zero] - https://gerrit.wikimedia.org/r/134314 (owner: Hashar)
[09:22:54] <grrrit-wm>	 (CR) QChris: [C: 2] Lint: allow commenting of code [analytics/wp-zero] - https://gerrit.wikimedia.org/r/134320 (owner: Hashar)
[09:42:29] <hashar>	 bah
[09:42:37] <hashar>	 jenkins-bot is not allowed to submit patches :-D
[09:43:08] <qchris>	 That's something I can change :-)
[09:43:33] <hashar>	 yeah it is not listed at https://gerrit.wikimedia.org/r/#/admin/projects/analytics,access
[09:43:41] <hashar>	 can't remember the exact group name though
[09:44:03] <hashar>	 qchris: JenkinsBot is the Gerrit group
[09:44:14] <hashar>	 basically me and Zuul  ( https://gerrit.wikimedia.org/r/#/admin/groups/14,members )
[09:44:55] <grrrit-wm>	 (CR) QChris: "recheck" [analytics/wp-zero] - https://gerrit.wikimedia.org/r/134314 (owner: Hashar)
[09:45:05] <hashar>	 qchris: that will just rerun the jobs
[09:45:08] <hashar>	 not actually merge them
[09:45:20] <qchris>	 So I Submit them by hand
[09:45:31] <hashar>	 I could potentially make it so recheck submit the jobs if they have been +2ed but not sure how to handle the Zuul conf to do that
[09:45:32] <qchris>	 And see if it works for the new changes?
[09:45:36] <hashar>	 so yeah, manual submit would do
[09:45:41] <qchris>	 Ok.
[09:46:39] <hashar>	 all geowiki ones are in :)
[09:46:53] <qchris>	 Thanks!
[09:47:08] <qchris>	 The two that are remaining from the wp-zero ones will need more time.
[09:47:14] <grrrit-wm>	 (PS1) Hashar: Jenkins job validation (DO NOT SUBMIT) [analytics/geowiki] - https://gerrit.wikimedia.org/r/136293
[09:47:41] <qchris>	 (The code we use in production there is ahead of master, and I'll have to merge them in)
[09:48:24] <grrrit-wm>	 (Abandoned) Hashar: Jenkins job validation (DO NOT SUBMIT) [analytics/geowiki] - https://gerrit.wikimedia.org/r/136293 (owner: Hashar)
[09:48:52] <hashar>	 yeah that is fine
[09:48:58] <qchris>	 I'll clean up the repos and make flake8 pass.
[09:49:00] <hashar>	 I will be happy to rebase the wp-zero on top of the prod one
[09:49:16] <hashar>	 you can ignore some flake8 changes by editing setup.cfg
[09:49:23] <hashar>	 ex:
[09:49:23] <hashar>	 ; E265 block comment should start with '# '
[09:49:23] <hashar>	 ; E501 line too long (X > 79 characters)
[09:49:23] <hashar>	 ignore = E265,E501
[09:49:28] <hashar>	 some are rather annoying
[09:49:32] <qchris>	 Yup. I saw that one.
[09:49:33] <qchris>	 Thanks.
[09:49:49] <hashar>	 the reason I push tox / flake8 is to get people used to tox and running the command
[09:50:08] <qchris>	 :-) It certainly helped for me.
[09:50:15] <hashar>	 hoping that it will create some good habit and eventually have devs to run/write tests
[09:50:39] <qchris>	 That would be great.
[09:51:19] <hashar>	 one of you repo (I think that is wikimetrics) depends on having a redis server though :-(
[09:51:30] <qchris>	 Right.
[09:52:19] <qchris>	 What do you suggest to work around that?
[09:52:28] <qchris>	 (Tests also need databases)
[09:54:15] <hashar>	 I have no clue :-]
[09:54:27] <hashar>	 maybe have the test suite boot a redis server in  def setUp()
[09:54:33] <hashar>	 or mock the responses
[09:54:44] <qchris>	 That's music to my ears :-)
[09:55:02] <hashar>	 yeah you taught me about mocking a while back
[09:55:17] <qchris>	 Hah :-D
[09:55:20] <hashar>	 python has a rather nice mocking library
[09:55:46] <qchris>	 Only ... we hardly use it (at least in analytics)
[09:55:54] <qchris>	 Having mostly legacy systems around ...
[09:56:02] <qchris>	 those repos are a bit short on testing.
[09:59:03] <hashar>	 legacy legacy :-(
[10:49:01] <grrrit-wm>	 (PS1) QChris: Lint scripts/restore_from_files.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136297
[10:49:03] <grrrit-wm>	 (PS1) QChris: Lint scripts/make_limn_files.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136298
[10:49:05] <grrrit-wm>	 (PS1) QChris: Lint geowiki/wikipedia_projects.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136299
[10:49:07] <grrrit-wm>	 (PS1) QChris: Lint geowiki/mysql_config.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136300
[10:49:09] <grrrit-wm>	 (PS1) QChris: Lint geowiki/geo_coding.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136301
[10:49:12] <grrrit-wm>	 (PS1) QChris: Remove unused geowiki/format_output.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136302
[12:07:25] <grrrit-wm>	 (CR) Hashar: [C: 1] "Straightforward :)" [analytics/geowiki] - https://gerrit.wikimedia.org/r/136297 (owner: QChris)
[12:08:52] <grrrit-wm>	 (CR) Hashar: [C: 1] Lint scripts/make_limn_files.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136298 (owner: QChris)
[12:09:40] <grrrit-wm>	 (CR) Hashar: [C: 1] Lint geowiki/wikipedia_projects.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136299 (owner: QChris)
[12:09:59] <grrrit-wm>	 (CR) Hashar: [C: 1] Lint geowiki/mysql_config.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136300 (owner: QChris)
[12:10:36] <grrrit-wm>	 (CR) Hashar: [C: 1] "Bye bye unused imports and variables." [analytics/geowiki] - https://gerrit.wikimedia.org/r/136301 (owner: QChris)
[12:17:32] <grrrit-wm>	 (CR) Hashar: [C: 1] Remove unused geowiki/format_output.py [analytics/geowiki] - https://gerrit.wikimedia.org/r/136302 (owner: QChris)
[14:01:31] <qchris>	 ottomata: Standup :-)
[14:01:33] <ottomata>	 on it
[14:01:40] <ottomata>	 i'm clicking buttons ALREADY
[14:07:10] * hashar listens to button clicks
[14:09:09] <YuviPanda>	 you can't listen to them anymore. they are flat
[15:22:09] <grrrit-wm>	 (PS1) Milimetric: Update column types for logging table [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136328 (https://bugzilla.wikimedia.org/65944)
[15:26:47] <grrrit-wm>	 (PS2) Milimetric: Update column types for logging table [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136328 (https://bugzilla.wikimedia.org/65944)
[15:35:04] <ottomata>	 so qchris, milimetric, python question
[15:35:13] <milimetric>	 ya
[15:35:24] <ottomata>	 subprocess.Popen
[15:35:31] <ottomata>	 DOES behave differently with lists vs strings
[15:35:33] <ottomata>	 right?
[15:35:38] <ottomata>	 when you use pipes in your command
[15:36:11] <milimetric>	 no idea, testing
[15:36:20] <qchris>	 Pipes in subprocess are a problem to me, as you cannot detect when one of the commands in the pipeline failed.
[15:36:59] <qchris>	 But about what kind of different behaviour are you talking about?
[15:37:19] <ottomata>	 https://gist.github.com/ottomata/c11b4aea1bcb933f381e
[15:37:55] <milimetric>	 btw ottomata, if you want to see details on something in ipython, add ??
[15:37:56] <milimetric>	 like:
[15:37:56] <milimetric>	 subprocess.Popen.__init__??
[15:38:13] <ottomata>	 ha, cool
[15:39:28] <qchris>	 double quoting around sed's expression?
[15:39:38] <qchris>	 (line 2)
[15:45:00] <qchris>	 ottomata: You're using shell=True
[15:45:01] <qchris>	 then:
[15:45:05] <qchris>	 "If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself."
[15:45:11] <ottomata>	 naw i have to quote it, its a string
[15:45:21] <ottomata>	 hmmmm
[15:45:23] <ottomata>	 lemme try
[15:45:26] <qchris>	 That's why the "echo" is threated as the command to run.
[15:45:34] <qchris>	 And "hi" is parameter to shell.
[15:45:55] <ottomata>	 naw, same result
[15:46:21] <qchris>	 Not for me :-)
[15:46:29] <ottomata>	 In [36]: p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
[15:46:29] <ottomata>	 In [37]: stdout, stderr = p.communicate()
[15:46:29] <ottomata>	 In [38]: stdout
[15:46:30] <ottomata>	 Out[38]: '\n'
[15:46:33] <ottomata>	 oops
[15:46:38] <ottomata>	 and
[15:46:38] <ottomata>	 command = ['/bin/echo', 'hi', '|', '/usr/bin/env', 'sed', '-e', 's@hi@wee@']
[15:46:40] <ottomata>	 right?
[15:46:47] <qchris>	 Ok.
[15:46:56] <qchris>	 That should give an empty line.
[15:47:06] <qchris>	 (When using "shell = True")
[15:47:21] <ottomata>	 In [39]: p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
[15:47:21] <ottomata>	 In [40]: stdout, stderr = p.communicate()
[15:47:21] <ottomata>	 In [41]: stdout
[15:47:21] <qchris>	 And it gives the empty line for me.
[15:47:22] <ottomata>	 Out[41]: 'hi | /usr/bin/env sed -e s@hi@wee@\n'
[15:47:28] <qchris>	 Right.
[15:47:37] <qchris>	 Looks correct to me.
[15:47:52] <qchris>	 Popen escapes the pipe.
[15:47:52] <ottomata>	 ok so, the answer is, that Popen does treat list and string commands differently
[15:48:22] <ottomata>	 the only way I could get Popen to do pipes the way I wanted was to use a string command and to set shell=True
[15:48:39] <qchris>	 That's why my review asks to not use pipes.
[15:48:52] <qchris>	 pipes in Popen are no good.
[15:49:01] <ottomata>	 i like most of those, i don't think I want to change how sh works, because i think you should be able to use pipes if you really want to
[15:49:07] <ottomata>	 but, i think for most of your comments we can avoid it
[15:49:09] <ottomata>	 gonna work on it
[15:49:44] <qchris>	 Let me boot the Google machine :-)
[17:41:34] <Ironholds>	 ottomata, ping
[17:42:40] <ottomata>	 hiyaa
[17:43:05] <Ironholds>	 ottomata, so, I'm getting claims that the JSON SerDE can't be found from hive.
[17:43:12] <Ironholds>	 "java.io.FileNotFoundException: File does not exist: /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar"
[17:43:21] <Ironholds>	 is it as simple as 'we upgraded to a newer version', or...?
[17:43:47] <ottomata>	 oooo i merged a change that qchris made today to makethat easier
[17:43:48] <ottomata>	 lemme see
[17:44:01] <Ironholds>	 aha
[17:44:34] <Ironholds>	 also, the heapsize increase + remembering to limit the partitions it's searching over worked like a charm. Would you like to send an email out about it or should I? I feel like people should probably know how to un-bork it when it borks.
[17:44:50] <ottomata>	 hmm, ok i'm not getting that error at all
[17:44:52] <ottomata>	 how are you getting it?
[17:45:07] <Ironholds>	 hive, ADD JAR /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar;
[17:45:24] <Ironholds>	 and then, well, any query I've tried, both one of my select-random-entries things and a very limited COUNT(*)
[17:45:24] <ottomata>	 Added resource: /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar
[17:45:32] <Ironholds>	 aha!
[17:45:36] <Ironholds>	 you have to run a query for it to bork ;p
[17:45:36] <ottomata>	 no i mean
[17:45:38] <ottomata>	 its working for me
[17:45:38] <ottomata>	 ok
[17:45:45] <Ironholds>	 try SELECT COUNT(*) FROM wmf.webrequest WHERE year = 2014 AND month = 05 AND day = 14 AND hour = 12;
[17:45:46] <ottomata>	 i'm running
[17:45:47] <ottomata>	 show create table
[17:45:48] <Ironholds>	 kk
[17:45:52] <ottomata>	 which usually borks if it isn't loaded
[17:45:53] <ottomata>	 try that
[17:45:57] <ottomata>	 show create table webrequest;
[17:46:10] <Ironholds>	 that works fine.
[17:46:13] <ottomata>	 ok running coutn
[17:46:34] <ottomata>	 i'm also limiting to webrequest_source='mobile'
[17:46:36] <ottomata>	 foir even smaller data
[17:46:40] <Ironholds>	 kk
[17:46:45] <Ironholds>	 and it runs?
[17:46:48] <ottomata>	 so far..
[17:46:55] <ottomata>	 yup runs fine
[17:47:02] <Ironholds>	 okay, that's just bloody weird.
[17:47:04] <Ironholds>	 on stat2?
[17:47:07] <ottomata>	 stat1002, ja
[17:47:08] <ottomata>	 real quick
[17:47:10] <ottomata>	 try it
[17:47:11] <Ironholds>	 (probably doesn't make a difference, but...)
[17:47:15] <ottomata>	 without adding any add jar command
[17:47:28] <ottomata>	 hmm, that failed for me
[17:47:29] <ottomata>	 !
[17:47:34] <ottomata>	 but it shoudlnt'
[17:47:53] <Ironholds>	 ottomata, what fail message?
[17:47:59] <ottomata>	 java.io.FileNotFoundException: File does not exist: /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar
[17:48:03] <ottomata>	 that's without adding the jar at all
[17:48:09] <Ironholds>	 yup, same error both times.
[17:48:12] <Ironholds>	 (for me)
[17:48:25] <ottomata>	 would be expected, but qchris's change is supposed to make that better
[17:48:25] <ottomata>	 hmmmm
[17:48:45] <ottomata>	 can you run your count query again?
[17:48:47] <ottomata>	 with jar added
[17:48:51] <Ironholds>	 did do, borks, same error.
[17:48:57] <ottomata>	 just now you did?
[17:49:03] <Ironholds>	 This is just silly. Sorry for producing something non-replicatable
[17:49:03] <Ironholds>	 yep.
[17:49:14] <ottomata>	 hmmm
[17:49:14] <Ironholds>	 third time fer testing, same problem.
[17:49:21] <ottomata>	 ok i'm going to su to you and try things
[17:49:31] <YuviPanda>	 suuuu tooo youuu
[17:50:52] <ottomata>	 hmmm
[17:50:58] <ottomata>	 ok Ironholds, not sure why add jar doesn't work
[17:50:59] <ottomata>	 that is weird
[17:51:00] <ottomata>	 but
[17:51:01] <ottomata>	 if you do
[17:51:07] <ottomata>	 --auxpath /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar
[17:51:09] <ottomata>	 on your hive cli
[17:51:11] <Ironholds>	 yup
[17:51:12] <ottomata>	 instead of doing add jar
[17:51:13] <ottomata>	 that works
[17:51:18] <ottomata>	 hive --auxpath /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar
[17:51:26] <Ironholds>	 da. and then -f yada yadda.
[17:51:27] <Ironholds>	 Thanks :)
[17:51:32] <ottomata>	 but still
[17:51:34] <ottomata>	 add jar shoudl work
[17:51:35] <ottomata>	 AND
[17:51:39] <ottomata>	 it shoudl now work without even specifying that!
[17:51:40] <ottomata>	 grrr
[17:52:10] <Ironholds>	 I should get a new set of business cards.
[17:52:15] <Ironholds>	 Oliver Keyes, Headacher of Ops
[17:52:36] <Ironholds>	 Breaks Things Faster than a Speeding Patch
[17:52:58] <Ironholds>	 Leaps Reproducible Bugs in a Single Bound
[17:53:12] <ottomata>	 ha
[17:53:22] <Ironholds>	 unrelated to the bug itself: thanks for the work with qchris on removing the ADD JAR need :)
[17:53:32] <Ironholds>	 the simpler we can make it to run hive queries the more people will use it.
[17:53:35] <ottomata>	 yeah
[17:53:36] <Ironholds>	 Wait, people will use it. Crap.
[17:53:41] <ottomata>	 haha
[17:53:42] <Ironholds>	 My stuff will run so slow!
[17:53:51] <ottomata>	 yeah, actually, hue in cdh5 might just actually work for us
[17:53:56] <ottomata>	 so there might be a web ui to hive...
[17:57:53] <Ironholds>	 well, the query is still failing, but for different reasons. progress!
[17:58:20] <Ironholds>	 and the task page 404s
[17:58:21] * Ironholds sighs
[18:42:35] <grrrit-wm>	 (CR) Ottomata: Add code to auto-drop old hive partitions and remove partition directories (40 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/135128 (owner: Ottomata)
[18:47:22] <grrrit-wm>	 (PS5) Ottomata: Add code to auto-drop old hive partitions and remove partition directories [analytics/refinery] - https://gerrit.wikimedia.org/r/135128
[19:06:13] <qchris>	 Ironholds: I just tested a simple query and it works both on stat1002 and analytics1010 without the ADD JAR, and without --auxpath
[19:06:23] <qchris>	 Which query are you using to make it misbehave?
[19:06:33] <qchris>	 (i.e.: not finding the jar)
[19:06:37] <Ironholds>	 qchris, SELECT COUNT(*) FROM wmf.webrequest WHERE year = 2014 AND month = 05 AND day = 14 AND hour = 12;
[19:06:45] <qchris>	 Thanks.
[19:06:46] <Ironholds>	 see above - otto couldn't replicate it, but su'd in as me and could.
[19:06:50] <Ironholds>	 it's very strange.
[19:07:00] <Ironholds>	 Still, the --auxpath trick worked, so it's resolved from a user POV.
[19:07:48] <qchris>	 Ironholds: That query fails for me as well.
[19:08:07] <qchris>	 Jould you try 'select * from wmf.webrequest where year=2014 and month=5 and day=30 and hour=1 limit 1;'
[19:08:13] <qchris>	 s/Jould/Could/
[19:08:29] <qchris>	 (I just want to rule out that something is broken with your account)
[19:08:31] <ottomata>	 that worked for m!
[19:08:33] <ottomata>	 me
[19:08:38] <ottomata>	 but coutn(*) didn't!
[19:09:06] <Ironholds>	 qchris, that works fine.
[19:09:10] <qchris>	 Ok.
[19:09:58] <qchris>	 ottomata: Could you but the jar into dfs?
[19:10:04] <qchris>	 (I lack permission to do so)
[19:10:20] <qchris>	 mkdir: Permission denied: user=qchris, access=WRITE, inode="/":hdfs:hadoop:drwxr-xr-x
[19:10:33] <qchris>	 s/but/put/
[19:11:26] <ottomata>	 you can put it anywhere,
[19:11:29] <ottomata>	 why put it at /?
[19:11:31] <ottomata>	 where do you want it?
[19:11:35] <ottomata>	 you can put it in /user/qchris
[19:11:41] <qchris>	 I wanted to put it at /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar
[19:11:53] <qchris>	 And /usr does not exist.
[19:11:58] <qchris>	 So I wanted to mkdir /usr
[19:12:14] <qchris>	 And I cannot mkdir, because I lack write on /
[19:14:06] <qchris>	 (My theory is that the plain 'select ... limit 1' does not spawn a map reduce job, hence can be done locally, but the 'select count(*) ...' does need a map reduce job, and wants the file in hdfs)
[19:18:00] <ottomata>	 hmmmm
[19:18:12] <ottomata>	 ok we can test, but we can't keep that file there
[19:18:14] <ottomata>	 ok qchris?
[19:18:33] <qchris>	 ok.
[19:19:05] <ottomata>	 ok its there
[19:19:06] <ottomata>	 try it
[19:19:18] <qchris>	 Trying...
[19:19:35] <qchris>	 Job is starting ...
[19:19:41] <qchris>	 Job is running...
[19:19:49] <qchris>	 So it looks like that really was the issue.
[19:19:55] <grrrit-wm>	 (PS1) Milimetric: Fix datetime parsing problem [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136429
[19:19:57] <grrrit-wm>	 (PS1) Milimetric: Clean up errant print [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136430
[19:19:59] <grrrit-wm>	 (PS1) Milimetric: Add Newly Registered Users metric [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136431 (https://bugzilla.wikimedia.org/65944)
[19:20:32] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Add Newly Registered Users metric [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136431 (https://bugzilla.wikimedia.org/65944) (owner: Milimetric)
[19:20:39] <qchris>	 Can we put the jar file in $some_well_known_place in hdfs?
[19:23:04] <grrrit-wm>	 (PS2) Milimetric: Add Newly Registered Users metric [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136431 (https://bugzilla.wikimedia.org/65944)
[19:24:17] <ottomata>	 ok qchrisy
[19:24:19] <ottomata>	 yes we can
[19:24:23] <ottomata>	 and actually need to for oozie stuff
[19:24:23] <ottomata>	 hmm
[19:24:42] <ottomata>	 heheh
[19:24:43] <ottomata>	  /wmf/kraken/artifacts/hcatalog-core-0.5.0-cdh4.3.1.jar
[19:24:45] <ottomata>	 its already there :)
[19:25:07] <ottomata>	 so the hive-site setting is for hdfs then?
[19:28:45] <qchris>	 It's for both I guess?
[19:28:53] <qchris>	 I just tried on my labs cluster.
[19:29:01] <qchris>	 and forcing local file system works.
[19:29:09] <qchris>	 So I'll prepare a patch for that.
[19:29:38] <qchris>	 Or do you prefer to force hdfs?
[19:29:48] <qchris>	 (I guess we have to force one of the two)
[19:30:20] <grrrit-wm>	 (PS1) Milimetric: Fix invalid users display for invalid cohort [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/136434
[19:33:32] <ottomata>	 ah, file:/// qchris?
[19:33:35] <ottomata>	 hmmmmm
[19:33:36] <qchris>	 Yes.
[19:33:41] <ottomata>	 i think forcing local is better, for now
[19:33:58] <ottomata>	 it will always be there with cloudera
[19:34:02] <ottomata>	 the hdfs path we ahve to put it there
[19:34:06] <ottomata>	 and make sure its there
[19:34:11] <ottomata>	 so in, labs, vagrant, etc.
[19:34:13] <ottomata>	 extra step
[19:34:36] <qchris>	 Ok: https://gerrit.wikimedia.org/r/136435
[19:54:36] <qchris>	 Ironholds: ottomata merged the fixup for the auxpath thing.
[19:54:50] <qchris>	 Could you try your query again (without auxpath or ADD JAR)
[19:55:08] <qchris>	 (It worked for me, but wanted to double check that it works for you as well)
[22:58:50] <jzerebecki>	 DarTar: hey. csteipp told me you were asked to do analytics regarding some tls chipher changes, which he talks about in https://gerrit.wikimedia.org/r/#/c/132393/1/templates/nginx/nginx.conf.erb in is 3rd comment.
[23:00:35] <jzerebecki>	 is the necessary data being collected and available for later analysis so it doesn't block deployment of that pach?
[23:04:40] <DarTar>	 jzerebecki: yes csteipp reached out, but this has been a pretty low priority for my team so far, I don’t have an answer to your question yet
[23:16:26] <jzerebecki>	 DarTar: are there any timings in the path until the user being able to start reading a page being continously monitored for regressions or at least observed from time to time?
[23:18:51] <DarTar>	 no, sorry – I don’t have an ETA yet, but I’ll bring this up with tnegrin so we can see how we can help
[23:20:51] <DarTar>	 jzerebecki: if I understand csteipp’s request we’re already collecting the data that is needed, so that should not affect the deployment in any way
[23:23:25] <jzerebecki>	 DarTar: nice. please comment to that effect on https://gerrit.wikimedia.org/r/#/c/132393/
[23:23:37] <DarTar>	 sure
[23:55:52] <jzerebecki>	 thank you :)