[16:36:06] <ottomata>	 phew
[16:36:07] <ottomata>	 sorry all
[16:36:11] <ottomata>	 internet troubles this morning :/
[16:45:38] <grrrit-wm>	 (PS1) Nuria: [WIP] Change db bindings for vagrant. [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109676
[16:51:47] <grrrit-wm>	 (CR) Edenhill: "test/inputgen.py and test/outputcheck.py are old and should be removed." [analytics/kafkatee] - https://gerrit.wikimedia.org/r/109505 (owner: Edenhill)
[16:56:34] <ottomata>	 Snaps:  :) I will get to that as soon as I can, but it it might not be til tomorrow!
[16:56:35] <ottomata>	 excited!
[17:01:54] <ottomata>	 ok through emails
[17:01:55] <ottomata>	 hmmm
[17:01:57] <ottomata>	 let's seeeee
[17:02:06] <ottomata>	 milimetric: wanna talk about wikimetrics in labs?
[17:05:24] <ottomata>	 or nuria, celery problems?
[17:06:02] <nuria>	 Maybe that is teh wya they are supposed to be .. so.. where are teh celery logs supposed to go?
[17:06:10] <nuria>	 Sorry
[17:06:15] <nuria>	 Missed one message
[17:06:25] <nuria>	 I could not find celery logs anywhere
[17:06:31] <nuria>	 on vagrant
[17:06:37] <ottomata>	 hm ok
[17:06:37] <ottomata>	 oh
[17:06:41] <nuria>	 But maybe that is on purpose?
[17:06:47] <ottomata>	  /var/log/upstart/wikimetrics.*
[17:08:16] <ottomata>	 is it there?
[17:08:25] <nuria>	 let me seee
[17:11:56] <grrrit-wm>	 (CR) Milimetric: [WIP] Change db bindings for vagrant. (2 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109676 (owner: Nuria)
[17:12:17] <nuria>	 ah yes, they are there!!!
[17:13:09] <nuria>	 sorry ... again
[17:13:18] <ottomata>	 yeah that isn't obvious though
[17:13:18] <ottomata>	 hm
[17:13:24] <ottomata>	 i wonder if we should symlink those somewhere
[17:13:39] <ottomata>	  /var/log/wikimetrics maybe
[17:13:44] <ottomata>	 we could symlink the apache logs there too
[17:14:51] <grrrit-wm>	 (PS1) Edenhill: OSX compilation fixes [analytics/kafkatee] - https://gerrit.wikimedia.org/r/109683
[17:15:15] <nuria>	 that would be nice
[17:15:27] <nuria>	 symlinking would make it more obvious
[17:15:54] <average>	 nuria: if you like, you can add a puppet rule to symlink it
[17:16:32] <average>	 nuria: for example like this https://gist.github.com/wsdookadr/35e6ade39e91c3f788a8
[17:16:57] <ottomata>	 yeah, average, we're working on the wikimetrics module as part of this
[17:17:01] <ottomata>	 so i might add it
[17:17:12] <average>	 ah :) ok
[17:18:23] <nuria>	 symlinking sounds great
[17:19:29] <Ironholds>	 yay, merged :D
[17:19:31] * Ironholds  high-fives ottomata 

[17:19:34] <Ironholds>	 thanks for getting this done :)
[17:19:45] <ottomata>	 running puppet now, i think it hsould just work
[17:19:46] <ottomata>	 we will see
[17:19:50] <ottomata>	 oof
[17:30:42] <ottomata>	 Ironholds: ,woooo
[17:30:44] <ottomata>	 it works! :)
[17:30:46] <ottomata>	 from stat1002
[17:31:14] <ottomata>	 this is cool, hmm, i think we should probably encourage everyone to use hadoop froms tat1002
[17:31:26] <ottomata>	 we'll still need accounts for users on namenodes so that user permissions for hdfs work properly
[17:31:34] <ottomata>	 but we don't really need to give folks accounts anywhere else I think
[17:31:36] <ottomata>	 hmm
[17:32:05] <Ironholds>	 excellent!
[17:32:20] <Ironholds>	 I shall start headscratchin' about rhdfs, too.
[17:32:39] <Ironholds>	 I think that sharding R tasks over the servers is the height of madness, but having proper integration betwixt the two for calling mapreduce tasks would be good.
[17:35:42] <ottomata>	 jaja
[17:35:44] <ottomata>	 fo sho
[17:35:49] <ottomata>	 nuria how goes it?
[17:35:52] <ottomata>	 milimetric: I summon thee!
[17:36:32] <ottomata>	 I require your powers for the conquering of OAuth in the virtual netherworlds
[17:37:29] <ottomata>	 nuria, btw, you have access to analytics cluster now
[17:39:30] <nuria>	 I think so! I just got e-mail. can you tell me of a machine that i can try?
[17:40:00] <milimetric>	 sorry ottomata
[17:40:02] <milimetric>	 in batcave
[17:40:04] <ottomata>	 welllllll, stat1002 now!
[17:40:05] <milimetric>	 join?
[17:40:07] <ottomata>	 which would be awesome
[17:40:13] <ottomata>	 kinda even better than an26
[17:40:47] <ottomata>	 nuria, run
[17:40:48] <ottomata>	  hive --auxpath /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar --database wmf
[17:40:50] <ottomata>	 and the
[17:40:51] <ottomata>	 then
[17:40:52] <ottomata>	 show tables;
[17:40:56] <ottomata>	 show create table webrequest_mobile;
[17:40:57] <ottomata>	 etc.
[17:40:59] <ottomata>	 play away!
[17:41:05] <milimetric>	 we definitely gotta make an ascii bat signal that pings us all in IRC
[17:41:07] <ottomata>	 ohh, gotta do one thing
[17:41:10] <ottomata>	 ha,
[17:41:10] <ottomata>	 uyea
[17:41:14] <nuria>	 in stat1002 ?
[17:41:22] <ottomata>	 milimetric: be yhere in one sec
[17:41:26] <ottomata>	 actually one sec nuria
[17:41:29] <ottomata>	 have to create your hdfs homedir
[17:42:06] <ottomata>	 ok go nuria!
[17:42:12] <ottomata>	 yeah, stat1002.eqiad.wmnet
[17:42:19] <ottomata>	 hive --auxpath /usr/lib/hcatalog/share/hcatalog/hcatalog-core-0.5.0-cdh4.3.1.jar --database wmf
[18:00:23] <grrrit-wm>	 (PS2) Milimetric: [WIP] Change db bindings for vagrant. [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109676 (owner: Nuria)
[18:02:06] <DarTar>	 tnegrin: can you add a video hangout to the sync up with Howie?
[18:02:18] <tnegrin>	 yes
[18:06:22] <tnegrin>	 can you ping me when howie arrives
[18:07:28] <DarTar>	 tnegrin: howie is here
[18:07:35] <tnegrin>	 coming
[18:46:43] <ottomata>	 ahh poop milimetric, non sefl hosted won't wrok
[18:46:43] <ottomata>	 duh
[18:46:43] <ottomata>	 because of that passwords issu
[18:46:46] <ottomata>	 e
[18:47:03] <milimetric>	 right
[18:47:22] <milimetric>	 hm, we can just leave it self hosted though right?
[18:47:25] <milimetric>	 at least the staging one
[18:48:12] <milimetric>	 is labs planning on a private puppet repo sometime soon?
[18:57:38] <ottomata>	 yeah
[18:57:39] <ottomata>	 i dunno
[18:57:43] <ottomata>	 not that I know of
[19:04:31] <DarTar>	 hey milimetric
[19:04:41] <milimetric>	 hey DarTar
[19:04:46] <DarTar>	 when is a good time to discuss the global VE activity request?
[19:04:50] <milimetric>	 now
[19:04:51] <milimetric>	 :)
[19:04:58] <DarTar>	 cool, quick hang out?
[19:05:03] <milimetric>	 sure, batcave
[19:05:15] <milimetric>	 https://plus.google.com/hangouts/_/calendar/d2lraW1lZGlhLm9yZ19jYjM3bXU0OGNuaHRkN2hybmE4czI3b25hb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29t.c6j7qidqs491nhi7ovk9pi4h14
[19:06:19] <average>	 https://github.com/wsdookadr/openddr-utils#current-results
[19:06:55] <average>	 I'm still working on the comparison, have to run outside to get some coffee
[19:07:22] <average>	 if anyone wants to try out this OpenDDR union thing see above ^^
[19:07:44] <average>	 there is a short explanation of the algorithm used here https://github.com/wsdookadr/openddr-utils/blob/master/OpenDDR.py#L100
[19:08:00] <average>	 I would be glad if I could get some feedback on it
[19:08:16] <average>	 it would help a lot
[19:10:21] <average>	 it's weird that from OpenDDR 1.14 to 1.15 there has been a big drop in number of devices
[19:10:27] <average>	 I'll look into their Changelog to see why that is
[19:10:48] <average>	 maybe they introduced something that would match more devices using less patterns (?, not sure)
[19:18:30] <ottomata>	 hmmm, milimetric
[19:18:47] <ottomata>	 [Mon Jan 27 19:18:19 2014] [error] [client 10.4.0.78] ImportError: No module named dateutil.relativedelta
[19:19:36] <milimetric>	 where's that from ottomata?
[19:20:03] <ottomata>	 https://gist.github.com/ottomata/8655535
[19:20:11] <ottomata>	 trying to get metrics-staging.wmflabs.org
[19:20:40] <milimetric>	 oh ok
[19:20:52] <milimetric>	 hm, that means dateutil is installed by default in some but not all places
[19:20:58] <ottomata>	 ImportError: No module named dateutil
[19:20:58] <ottomata>	 ?
[19:21:00] <milimetric>	 adding to requirements.txt, one sec
[19:21:03] <ottomata>	 hm ok
[19:21:20] <ottomata>	 pip install dateutil should do me for now?
[19:21:29] <drdee>	 average: openDDR dropped a whole bunch of patterns because there were licensing issues with those, what's left is fully open source but it means that temporarily fewer devices are recognized
[19:21:37] <ottomata>	 No distributions at all found for dateutil
[19:21:38] <ottomata>	 ?
[19:22:17] <average>	 drdee: ah, thank you :)
[19:22:47] <ottomata>	 hmm, wha?
[19:22:48] <ottomata>	 No distributions at all found for dateutil
[19:23:07] <ottomata>	 ah python-dateutil
[19:23:08] <ottomata>	 weird
[19:23:46] <average>	 drdee: so if someone's using OpenDDR pre-1.14 would he get in trouble because of the licensing issues ?
[19:24:23] <drdee>	 don't know -- but it would lock you into a library that is no longer maintained
[19:24:34] <ottomata>	 milimetric: i am in batcave
[19:24:52] <ottomata>	 more probs :D
[19:24:57] <milimetric>	 comin
[19:27:55] <grrrit-wm>	 (PS1) Milimetric: Adds explicit requirement for dateutil [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109707
[19:28:04] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Adds explicit requirement for dateutil [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109707 (owner: Milimetric)
[20:17:49] <grrrit-wm>	 (PS1) Terrrydactyl: Bug 58609 - Date and time revert creating a report [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/109725
[20:18:05] <terrrydactyl>	 milimetric ^
[21:17:56] <qchris>	 terrrydactyl: I've been told, you're suffering git?
[21:18:05] <terrrydactyl>	 yeah
[21:18:07] <terrrydactyl>	 :'(
[21:18:11] <qchris>	 What's the issue?
[21:18:33] <terrrydactyl>	 so i pushed this commit: https://gerrit.wikimedia.org/r/#/c/109725/ so that milimetric can take a look at an approach i did
[21:18:39] <terrrydactyl>	 but it was the wrong approach
[21:18:58] <qchris>	 ok.
[21:18:59] <terrrydactyl>	 so i want to go back to a clean copy and try a different way.
[21:19:05] <qchris>	 ok.
[21:19:15] <terrrydactyl>	 but i don't know what i should do with this commit. i want to keep it clean
[21:19:51] <qchris>	 Not sure what that means... you want to go to the state before the commit that you sent?
[21:19:53] <terrrydactyl>	 ideally, i want to revert back to before my commit and then do some changes and maybe amend the new changes to the commit i had already submitted
[21:20:03] <terrrydactyl>	 qchris: yes
[21:20:18] <qchris>	 terrrydactyl: ok.
[21:20:27] <qchris>	 Are you using gerrit-review or plain git?
[21:20:39] <qchris>	 s/gerrit-review/git review/
[21:20:40] <terrrydactyl>	 i have gerrit-review set up
[21:20:50] <qchris>	 What branch are you on?
[21:21:18] <terrrydactyl>	 my local master
[21:22:11] <terrrydactyl>	 i haven't been making branches to work on my code. something that i now realized that i forgotten to do
[21:22:15] <qchris>	 Mhmm maybe it's easier if I can look at your screen ... can you use hangouts or some screen sharing program?
[21:22:32] <terrrydactyl>	 i can use hangout
[21:22:49] <qchris>	 Ok. I'll just boot my hangout machine.
[21:23:01] <qchris>	 I'll invite you to a hangout.
[21:23:07] <terrrydactyl>	 great
[21:24:09] <ottomata>	 terrrydactyl: yeah a good gerrit workflow is to work with local tracking branches
[21:24:12] <ottomata>	 so, when you branch
[21:24:12] <ottomata>	 do
[21:24:22] <ottomata>	 git checkout -b <branch_name> origin/master
[21:24:32] <ottomata>	 that will create a local branch that tracks the origin master remote
[21:24:43] <ottomata>	 then you can switch between branches for different feature work
[21:24:53] <ottomata>	 and gerrit will not make the changesets depend on each other
[21:24:57] <terrrydactyl>	 thanks for the heads up ottomata
[21:25:20] <ottomata>	 if you are going to start over with this change, you might wnat to just do that and abandon the one  you've already submitted to gerrit
[21:25:33] <ottomata>	 also, if I ever mess up my local master this way
[21:25:43] <ottomata>	 (e.g. local master contains commits that haven't been merged via gerrit)
[21:25:48] <ottomata>	 i just blast my local master and recreate it
[21:26:06] <ottomata>	 git checkout <some other branch>
[21:26:06] <ottomata>	 git branch -D master
[21:26:06] <ottomata>	 git checkout -b master origin/master
[21:28:42] <ottomata>	 qchris, does CentralAuth == mediawiki oauth?
[21:28:46] <ottomata>	 do you know?
[21:29:49] <milimetric>	 no ottomata
[21:30:04] <milimetric>	 CentralAuth is like the system that gives global usernames and global ids
[21:30:19] <milimetric>	 mediawiki oauth uses the global ids when it responds to you
[21:30:30] <ottomata>	 hm ok
[21:31:50] <terrrydactyl>	 got the problem solved, thanks qchris
[21:31:55] <qchris>	 yw
[21:42:19] <grrrit-wm>	 (PS1) Jdlrobson: Prevent failures from blocking generation of other graphs [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109757
[21:44:42] <matanya>	 drdee: can the glam NARA udp2log move to oxygen? pipe 10 /usr/bin/udp-filter -F '\t' -p _NARA_ -g -b country >> <%= log_directory %>/glam_nara.tsv.log
[21:45:35] <drdee>	 ask ottomata
[21:46:20] <ottomata>	 haha
[21:46:22] <ottomata>	 ask drdee
[21:46:27] <matanya>	 nice loop here
[21:46:37] <ottomata>	 i can decide where it can move
[21:46:39] <drdee>	 i think it's ok :) but ottomata knows the load of the machine
[21:46:44] <ottomata>	 but we'd rather get rid of it if we can
[21:46:47] <ottomata>	 did we find out who we should ask?
[21:46:53] <matanya>	 i can just decide to remove all and see who is angry :)
[21:46:54] <drdee>	 maarten
[21:47:04] <ottomata>	 i don't thikn I know maarten
[21:47:10] <drdee>	 maarten dammers
[21:47:14] <drdee>	 multichill
[21:47:22] <drdee>	 i think you know him
[21:47:37] <matanya>	 i do
[21:47:58] <ottomata>	 i do?  hmm
[21:48:00] <matanya>	 drdee: same question goes for pipe 100 /usr/bin/udp-filter -F '\t' -p /w/api.php >> <%= log_directory %>/api-usage.tsv.log  and pipe 10 /usr/bin/udp-filter -F '\t' -d en.wikipedia.org -p /wiki/Wikipedia:Teahouse >> <%= log_directory %>/teahouse.tsv.log
[21:48:01] <ottomata>	 name is familiar
[21:48:24] <drdee>	 owner of that is Siko
[21:48:33] <matanya>	 both?
[21:48:40] <drdee>	 the teahouse one
[21:48:49] <drdee>	 the api is used for general debugging
[21:49:30] <matanya>	 drdee: i'll get siko (what is his mail?) can the API move?
[21:50:02] <grrrit-wm>	 (PS1) Milimetric: Fix union to have matching select clauses [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109760
[21:50:03] <drdee>	 siko: https://meta.wikimedia.org/wiki/User:Sbouterse_(WMF)
[21:50:30] <drdee>	 stuff can move -- i think load of the machine is the limiting factor and for that you have to check with ottomata
[21:51:07] <ottomata>	 yeah, we can move things at will, matanya, but i'd rather not have to think about extra load on the other log machines
[21:51:15] <ottomata>	 so the more we can destroy, the better :)
[21:51:55] <matanya>	 ok, i'll try detroying the other two, but what ever is left, i'll move over and shut down the host
[21:52:08] <matanya>	 thank you drdee and ottomata
[21:57:00] <grrrit-wm>	 (CR) Jdlrobson: [C: 2] Fix union to have matching select clauses [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109760 (owner: Milimetric)
[21:57:06] <grrrit-wm>	 (CR) Jdlrobson: [V: 2] Fix union to have matching select clauses [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109760 (owner: Milimetric)
[22:13:09] <grrrit-wm>	 (PS1) Jdlrobson: Remove useless print statement, add a new one [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109764
[22:24:36] <grrrit-wm>	 (PS1) Milimetric: Count only edits within 30 days of registration [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109770
[22:26:10] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Remove useless print statement, add a new one [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109764 (owner: Jdlrobson)
[22:29:02] <halfak>	 milimetric, FYI, got some code running now to gather VE data.  The speed seems reasonable.  I'll give you an update in a couple of hours.
[22:31:05] <grrrit-wm>	 (PS2) Milimetric: Count only edits within 30 days of registration [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109770
[22:31:37] <milimetric>	 awesome halfak, let me know
[22:31:49] <milimetric>	 and if anything turns sour, we got your back :)
[22:32:53] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Prevent failures from blocking generation of other graphs [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109757 (owner: Jdlrobson)
[22:33:19] <grrrit-wm>	 (CR) Jdlrobson: [C: 2 V: 2] Count only edits within 30 days of registration [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109770 (owner: Milimetric)
[23:32:00] <grrrit-wm>	 (PS1) Jdlrobson: Make sure we generate the current month as well as the older month [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109786
[23:32:02] <grrrit-wm>	 (PS1) Jdlrobson: Don't say generating graph twice when generating full graphs [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109787
[23:46:25] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Make sure we generate the current month as well as the older month [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109786 (owner: Jdlrobson)
[23:46:37] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Don't say generating graph twice when generating full graphs [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109787 (owner: Jdlrobson)
[23:48:27] <Ironholds>	 hrm; weird errors with the stat2 hive client.
[23:50:32] <Ironholds>	 nope, not stat2, just webrequest_mobile
[23:51:46] * Ironholds  files a bug

[23:59:02] <grrrit-wm>	 (PS1) Jdlrobson: Remove updates on graph edits-monthly-5plus-editors [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/109790