[02:28:51] <grrrit-wm>	 (PS1) Milimetric: Better information around the namespaces field [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/102871
[02:30:42] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Better information around the namespaces field [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/102871 (owner: Milimetric)
[13:21:41] <grrrit-wm>	 (CR) Erik Zachte: [C: 2 V: 2] Make squid tests time-independent [analytics/wikistats] - https://gerrit.wikimedia.org/r/102292 (owner: QChris)
[13:21:48] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Make squid tests time-independent [analytics/wikistats] - https://gerrit.wikimedia.org/r/102292 (owner: QChris)
[13:22:17] <grrrit-wm>	 (CR) Erik Zachte: [C: 2 V: 2] Use WORKSPACE variable to determine $__CODE_BASE in fallback [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[13:22:26] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Use WORKSPACE variable to determine $__CODE_BASE in fallback [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[13:22:27] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] use new wikivoyage logo on stats.wikimedia.org [analytics/wikistats] - https://gerrit.wikimedia.org/r/88978 (owner: Dzahn)
[13:22:28] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Make squid tests time-independent [analytics/wikistats] - https://gerrit.wikimedia.org/r/102292 (owner: QChris)
[14:18:08] <grrrit-wm>	 (CR) QChris: "Direct push, as Erik voted CR+2." [analytics/wikistats] - https://gerrit.wikimedia.org/r/102292 (owner: QChris)
[14:18:32] <grrrit-wm>	 (CR) QChris: "recheck" [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[14:54:32] <grrrit-wm>	 (PS4) Hashar: Use WORKSPACE variable to determine $__CODE_BASE in fallback [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[15:11:00] <grrrit-wm>	 (CR) QChris: "Erik CR+2 on the identicat PS3." [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[15:11:08] <grrrit-wm>	 (CR) QChris: [C: 2] Use WORKSPACE variable to determine $__CODE_BASE in fallback [analytics/wikistats] - https://gerrit.wikimedia.org/r/102299 (owner: QChris)
[15:14:29] <grrrit-wm>	 (PS5) QChris: use new wikivoyage logo on stats.wikimedia.org [analytics/wikistats] - https://gerrit.wikimedia.org/r/88978 (owner: Dzahn)
[15:14:35] <grrrit-wm>	 (PS4) QChris: Add some more author aliases [analytics/wikistats] - https://gerrit.wikimedia.org/r/92069 (owner: Nemo bis)
[15:14:38] <grrrit-wm>	 (PS3) QChris: Typofix in squids report [analytics/wikistats] - https://gerrit.wikimedia.org/r/100368 (owner: Nemo bis)
[15:14:47] <grrrit-wm>	 (PS5) QChris: Ignore sampled-1000 testdata generated when running tests [analytics/wikistats] - https://gerrit.wikimedia.org/r/102298
[15:15:30] <grrrit-wm>	 (CR) QChris: [C: 2] use new wikivoyage logo on stats.wikimedia.org [analytics/wikistats] - https://gerrit.wikimedia.org/r/88978 (owner: Dzahn)
[15:48:43] <ottomata>	 qchris_away:
[15:48:45] <ottomata>	 ah, away!
[16:36:14] <qchris>	 ottomata: chris_away back.
[16:41:52] <ottomata>	 ok i have discovered that I can indeed use camus to write snappy compressed sequence files and query them with external hive tables
[16:42:03] <ottomata>	 is this what we should do!?
[16:42:06] <ottomata>	 i do not know :;
[16:42:07] <ottomata>	 :p
[16:42:16] <qchris>	 :-)
[16:43:40] <qchris>	 Did you manage to add uncompressed files to hive and compress them afterwards upon need?
[16:45:03] <ottomata>	 no
[16:45:17] <qchris>	 So we have to compress right away :-(
[16:45:26] <ottomata>	 options are:
[16:45:45] <ottomata>	 uncompressed + external table
[16:45:45] <ottomata>	 compressed sequence file + external table
[16:45:45] <ottomata>	 compressed internal table
[16:45:57] <ottomata>	 compressed internal table would only work if we imported the data into hive later
[16:46:02] <ottomata>	 rather than using the raw imports
[16:46:29] <qchris>	 I also think that external table is the way to go.
[16:46:40] <qchris>	 If we cannot compress later,
[16:46:50] <qchris>	 we probably have to compress right away :-(
[16:47:04] <ottomata>	 yeah we want external tables for sure
[16:47:04] <ottomata>	 but, we will only keep 30 days of external table data
[16:47:18] <ottomata>	 i think the historical sanitized data will not be in external tables
[16:47:18] <ottomata>	 not sure though
[16:47:55] <qchris>	 If it's not in external tables ... can we use it like plain hdfs files?
[16:48:21] <qchris>	 (e.g.: consume from pig)
[16:52:04] <ottomata>	 sure, it is just stored in /user/hive/warehouse.../year=2013/month=12/...
[16:52:06] <ottomata>	 its just in files there too
[16:52:20] <qchris>	 ok.
[16:52:20] <ottomata>	 and other hadoop tools should be able to read snappy sequence files too
[16:52:29] <ottomata>	 seq files are well supported in hadoop
[16:52:34] <ottomata>	 http://mail-archives.apache.org/mod_mbox/hive-user/201211.mbox/%3CCAPEqew+BCPX28gyQ9ThsRPyXR38hJ9y5jz6jW5XT3m=4yyjOBg@mail.gmail.com%3E
[16:52:46] <qchris>	 Cool.
[17:52:01] <ottomata>	 qchris:
[17:52:03] <ottomata>	 java q for you
[17:52:16] <ottomata>	 [ERROR] /Users/otto/Projects/wm/analytics/camus/camus-etl-kafka/src/main/java/com/linkedin/camus/etl/kafka/common/SequenceFileRecordWriterProvider.java:[76,72] incompatible types
[17:52:16] <ottomata>	 [ERROR] found   : java.lang.Class<capture#254 of ? extends org.apache.hadoop.io.compress.CompressionCodec>
[17:52:16] <ottomata>	 [ERROR] required: org.apache.hadoop.io.compress.CompressionCodec
[17:52:29] <ottomata>	 I am doing
[17:52:31] <ottomata>	 FileOutputFormat.getOutputCompressorClass((JobConf)context, DefaultCodec.class);
[17:52:47] <ottomata>	 and here is the doc
[17:52:47] <ottomata>	 http://hadoop.apache.org/docs/current2/api/org/apache/hadoop/mapred/FileOutputFormat.html#setOutputCompressorClass(org.apache.hadoop.mapred.JobConf, java.lang.Class)
[17:53:02] <ottomata>	 not sure what i'm doing wrong there
[18:22:43] <qchris>	 ottomata: Which camus repo are you using?
[18:23:28] <qchris>	 Is everyithing on the same Hadoop version?
[18:35:45] <ottomata>	 i htink i got it
[18:35:54] <qchris>	 Ok.
[18:36:02] <qchris>	 What caused the problems?
[18:36:30] <ottomata>	 i was assigning th result of getOutputCompressorClass
[18:36:32] <ottomata>	 which is a class
[18:36:35] <ottomata>	 and I needed to use
[18:36:48] <qchris>	 Ah.
[18:36:50] <qchris>	 Ok.
[18:36:50] <ottomata>	 .getInstance()
[18:36:51] <ottomata>	 sorry
[18:36:51] <ottomata>	 .newInstance()
[18:37:08] <ottomata>	 it was confusing because the first error I was getting said I needed to pass use a Class
[18:37:10] <ottomata>	 and then I used a class
[18:37:16] <ottomata>	 and then the next error said it wanted an instance
[18:37:31] <ottomata>	 didn't realize I had actually solved the first error by  using the .class
[18:37:31] <qchris>	 :-D
[18:37:50] <qchris>	 Stupid java!
[19:10:10] <jeremyb>	 is rfaulkner on irc these days? nick?
[21:18:57] <halfak>	 jeremb: Just saw your Q.  I haven't seen rfaulkner on IRC recently.
[21:19:55] <halfak>	 ottomata: Can you tell me which machines have /dev/mapper/stat1-a "/a" mounted.  Is it just stat1?
[21:20:46] <ottomata>	 yes
[21:20:51] <ottomata>	 that's not nfs
[21:20:55] <ottomata>	 its just a lvm partition
[21:20:56] <halfak>	 Gotcha.
[21:21:01] <halfak>	 Fancy!
[21:21:02] <halfak>	 :)
[21:21:11] <halfak>	 I'm hoping to do a little cleanup.
[21:21:31] <halfak>	 I see there are some database directories there (e.g. mysql & mongo) are those still in use?
[21:26:40] <halfak>	 ottomata: ---^
[21:27:51] <ottomata>	 halfak, not sure, ummm
[21:27:54] <ottomata>	 check puppet?
[21:27:57] <ottomata>	 mysql almost certainly actually
[21:27:59] <ottomata>	 dunno about monog
[21:28:00] <ottomata>	 mongo
[21:28:17] <halfak>	 That mongo could have been the old one that I used for Snuggle.
[21:28:38] <halfak>	 Yeah.  There's only test databases in there.
[21:29:20] * halfak  goes to look through puppet config again. 

[21:29:32] <halfak>	 Thanks for your help.
[21:41:11] <ottomata>	 gotta run, latesr all!
[21:54:40] <jeremyb>	 gah, missed halfak
[21:54:45] * jeremyb  hands halfak a tab key

[22:59:34] <DarTar>	 halfak: you around?
[22:59:45] <halfak>	 Hey!  Yup
[22:59:47] <DarTar>	 hey
[22:59:55] <halfak>	 Feel'n better?
[23:00:14] <DarTar>	 meh, I think I've got some fever
[23:00:30] <DarTar>	 but not the end of the world
[23:00:48] <DarTar>	 do you have time for a quick check in with tnegrin and me?
[23:00:49] <ori-l>	 data fever?
[23:00:54] <halfak>	 lol
[23:00:56] <halfak>	 Sure.
[23:01:04] <halfak>	 I have Growth standup in... now
[23:01:09] <halfak>	 But i could meet in 15-20
[23:01:26] <ori-l>	 dartar's got chills, they're multiplyin'
[23:01:40] <DarTar>	 spreading some love
[23:02:04] <halfak>	 DarTar, will 3:20ish work?
[23:02:16] <DarTar>	 yes, 15-20 should be fine, I think tnegrin is also afk now
[23:06:57] <tnegrin>	 I'm available now
[23:07:15] <halfak>	 Still doing growth standup.  I should be ready soon
[23:07:52] <DarTar>	 tnegrin: I'm sending an invite and blocking Chambers if you need a room
[23:07:59] <tnegrin>	 thanks D
[23:14:01] <halfak>	 DarTar: OK.  I'm good to go.  I'll go hop into the hangout.
[23:14:14] <halfak>	 Bah!  I have no invite.
[23:16:53] <halfak>	 got it
[23:36:04] * jeremyb  hands halfak a tab key :P

[23:36:10] <jeremyb>	 (jer<tab>)