[01:12:46] <rdwrer>	 https://gitorious.org/analytics/multimedia/source/e98f1efd320c724d22de15fbcd279275de4c7d69:actions/commonswiki.sql is apparently not working...if anyone has insights
[01:12:50] <rdwrer>	 I get an empty file out of it
[01:21:29] <average>	 rdwrer: what does "not working" mean ?
[01:21:50] <average>	 rdwrer: have you filed a bug ? did you attach the error you got ?
[01:22:08] <average>	 rdwrer: oh so you get no results, I see
[01:23:29] <average>	 rdwrer: and since is that your repository, correct me if I'm wrong but that query belongs to you, it is your code
[01:24:05] <average>	 rdwrer: where does that code run ? on tools/labs/prod ?
[01:24:16] <Ironholds>	 average, I'm debugging for him now :)
[01:24:27] <Ironholds>	 or will be when the bugger gets back from afkness
[01:24:42] <average>	 Ironholds: ok, thanks for letting me know
[01:24:53] <Ironholds>	 np; sorry, just saw the discussion
[01:55:36] <Ironholds>	 average, query problem resolved :)
[01:59:50] <average>	 https://gitorious.org/analytics/multimedia/commit/e98f1efd320c724d22de15fbcd279275de4c7d69
[01:59:57] <average>	 this fixed it ?
[02:00:29] <average>	 anyway, I'm glad you were able to solve it
[02:00:30] <average>	 :)
[02:01:42] <Ironholds>	 average, naw, problem-existed-between-user-and-public-datasets
[02:02:02] <Ironholds>	 the query worked fine, there's just the delay before it gets transferred from the stat1 public datasets folder onwards, and mark was unaware of said delay.
[02:02:15] <Ironholds>	 so, it didn't plot. No data! Because the file wasn't populated yet
[10:07:11] <grrrit-wm>	 (CR) QChris: [C: 1] "Compiles for me (after adding cloudera repository, but that was needed" [analytics/camus] (wmf) - https://gerrit.wikimedia.org/r/117100 (owner: Ottomata)
[10:08:09] <hashar_>	 qchris: good morning!  Do you know whether Chad upgraded Gerrit yesterday?
[10:08:25] <qchris>	 I checked ... but it seems he didn't
[10:08:37] <hashar_>	 qchris: "good". Thank you :-]
[10:08:38] <qchris>	 We're still onn 2.8.1-1-g83098d0
[10:08:51] <qchris>	 Although it should be 2.8.1-2-...
[10:08:53] <hashar_>	 will send him a hate mail hehe
[10:08:57] <qchris>	 Sorry :-(
[10:09:17] <hashar>	 tis ok :]
[10:09:20] <qchris>	 Hehe.
[15:30:57] <milimetric>	 ottomata: can you help us task the labs migration
[16:32:19] <ottomata>	 man story tasking takes a long time!  milimetric, how about I go ahead and move wikimetrics-staging? :)
[16:32:28] <milimetric>	 sure
[16:34:08] <ottomata>	 i'll go ahead and create the instance for wikimetrics too, but not touch it yet
[17:19:53] <halfak>	 Hey milimetric, got a bit to talk indexes for EEVS stuff?
[17:20:07] <milimetric>	 sure
[17:20:12] <milimetric>	 i'm eating lunch though
[17:20:21] <milimetric>	 apologies in advance if i take long to type
[17:20:24] <milimetric>	 but yeah, let's talk
[17:20:28] <halfak>	 This should be quick.  No worries.
[17:20:38] <halfak>	 I filed this rt ticket a while back. https://rt.wikimedia.org/Ticket/Display.html?id=6806
[17:20:53] <halfak>	 It requests an index be created on *all* the wiki DBs.
[17:21:21] <halfak>	 This particular index will be very important for generating data about New editors, Productive new editors and Surviving new editors.
[17:21:54] <halfak>	 Originally, I thought the request was just for me and my analysis work, but I recently realized that WikiMetrics will probably benefit from it as well.
[17:21:55] <milimetric>	 uh, RT system is not letting me in
[17:22:09] <halfak>	 Boo.  Here's the index: CREATE INDEX user_timestamp ON archive (ar_user, ar_timestamp);
[17:22:14] <milimetric>	 ok, thx
[17:22:30] <milimetric>	 makes sense
[17:22:30] <halfak>	 Right now, there's no index on archive for "ar_user" which is a foreign key to user_id.
[17:22:42] <milimetric>	 yeah, agree that makes a lot of sense, but might be an expensive index
[17:22:51] <milimetric>	 as archive is some ginormous beast of a table right?
[17:23:04] <halfak>	 Indeed.  However, running the queries cross languages even once will take much longer than generating the index.
[17:23:15] <halfak>	 Na.  I built the index on enwiki in 2 min.
[17:23:16] <milimetric>	 not generating - that's fine
[17:23:23] <milimetric>	 but maintaining it
[17:23:38] <milimetric>	 like will it slow down inserts too much
[17:23:43] <halfak>	 Archive is rarely written to compared to Revision.
[17:23:59] <milimetric>	 ok, great
[17:24:02] <milimetric>	 then i'm all for it
[17:24:35] <halfak>	 Do you think we should file a MW bug requesting it so that it appears everywhere (labs, analytics slaves, etc.)?
[17:25:00] <milimetric>	 I'm not sure how fast of a turn-around time we'd get on that
[17:25:01] <halfak>	 Or just pursue the RT ticket for analytics slaves only?
[17:25:06] <milimetric>	 might be faster to just yeah, the slaves
[17:25:12] <halfak>	 Well, the RT ticket has been sitting since Feb. 10th.
[17:25:18] <milimetric>	 yeah, maybe tell Toby
[17:25:28] <milimetric>	 and I can raise it during scrum of scrums
[17:25:32] <milimetric>	 i'll make a card for it right now
[17:26:08] <halfak>	 the thing I'm worried about is whether it's worthwhile to have a DBA do the work on the slaves if we'll need it in labs as well.
[17:26:31] <halfak>	 Will WikiMetrics need it in labs?
[17:26:49] <milimetric>	 not right now
[17:26:54] <milimetric>	 because we're not using archive
[17:27:01] <milimetric>	 but we *will* right
[17:27:09] <milimetric>	 i mean to say, your new queries for metrics use it right?
[17:27:26] <halfak>	 Yes
[17:27:30] <milimetric>	 ok, then yes
[17:27:37] <milimetric>	 since I can't access RT, would you mind updating that in the ticket?
[17:27:47] <milimetric>	 say it would be useful in general on all the slaves, including labsdb
[17:27:52] <halfak>	 Sure.
[17:28:03] <milimetric>	 ok, and i'll raise it as important during scrum of scrums
[17:28:12] <halfak>	 Thanks!  :)
[17:30:01] <milimetric>	 https://wikimedia.mingle.thoughtworks.com/projects/scrum_of_scrums/cards/86
[17:30:20] <milimetric>	 cool, np, thanks for reminding me, I know you raised it before
[17:33:07] <halfak>	 np.  About time I raised it in a useful context.
[17:57:14] <ottomata>	 ha, milimetric instance proxy in eqiad labs doesn't work yet
[17:57:20] <ottomata>	 gotta hold off on the migration :p
[17:57:30] <milimetric>	 well, we can do the rest of the work
[17:57:36] <milimetric>	 and resume when instanceproxy is up
[17:57:40] <milimetric>	 but, when will that be up?
[17:57:41] <ottomata>	 yeah, well, i mean
[17:57:43] <milimetric>	 we might need to work around it
[17:57:44] <ottomata>	 yeah we'll see
[17:57:47] <ottomata>	 i started migrating staging
[17:57:50] <ottomata>	 and that's down now because of that
[17:57:51] <ottomata>	 is that ok?
[17:58:00] <milimetric>	 it's not great but it's ok
[17:58:03] <milimetric>	 hm....
[17:58:09] <milimetric>	 ok, interesting
[17:58:13] <ottomata>	 i think I successfully imported redis and mysql dbs though
[17:58:16] <ottomata>	 so that's cool!
[17:58:23] <milimetric>	 hey that'd be good
[17:58:29] <milimetric>	 but why'd you have to take down the original instance?
[17:58:43] <milimetric>	 it's not a big deal, just wondering
[17:59:46] <ottomata>	 well, i didn't i just moved the domain
[17:59:49] <ottomata>	 the proxy
[18:00:08] <ottomata>	 guess i could move it back…
[18:00:20] <ottomata>	 we'll just have to reimport  the dbs again
[18:00:20] <ottomata>	 but ok
[19:47:59] <grrrit-wm>	 (PS1) Terrrydactyl: Add TextAreaField for paste_username, reads input [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/117260
[19:48:20] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Add TextAreaField for paste_username, reads input [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/117260 (owner: Terrrydactyl)
[19:51:53] <grrrit-wm>	 (PS2) Terrrydactyl: Add TextAreaField for paste_username, reads input [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/117260
[19:52:03] <terrrydactyl>	 milimetric ^
[19:52:31] <milimetric>	 sweet, thank you!
[21:31:57] <grrrit-wm>	 (PS1) QChris: Switch UserAgentClassifier's returned schema to tuple [analytics/kraken] - https://gerrit.wikimedia.org/r/117319
[21:31:59] <grrrit-wm>	 (PS1) QChris: Add non_wmf_mobile_app handling to mobile_device_props.pig [analytics/kraken] - https://gerrit.wikimedia.org/r/117320
[21:32:01] <grrrit-wm>	 (PS1) QChris: Since mobile jobs have been turned off, remove now unneded pig scripts [analytics/kraken] - https://gerrit.wikimedia.org/r/117321
[21:32:03] <grrrit-wm>	 (PS1) QChris: Drop dclass dependency from zero_country.pig script [analytics/kraken] - https://gerrit.wikimedia.org/r/117322
[21:35:44] <grrrit-wm>	 (PS2) QChris: Since mobile jobs have been turned off, remove now unneded pig scripts [analytics/kraken] - https://gerrit.wikimedia.org/r/117321
[21:35:46] <grrrit-wm>	 (PS2) QChris: Add non_wmf_mobile_app handling to mobile_device_props.pig [analytics/kraken] - https://gerrit.wikimedia.org/r/117320
[21:35:48] <grrrit-wm>	 (PS2) QChris: Drop dclass dependency from zero_country.pig script [analytics/kraken] - https://gerrit.wikimedia.org/r/117322
[21:35:50] <grrrit-wm>	 (PS2) QChris: Switch UserAgentClassifier's returned schema to tuple [analytics/kraken] - https://gerrit.wikimedia.org/r/117319
[21:36:10] <grrrit-wm>	 (CR) Ottomata: [C: 2] Drop dclass dependency from zero_country.pig script [analytics/kraken] - https://gerrit.wikimedia.org/r/117322 (owner: QChris)
[21:36:18] <grrrit-wm>	 (CR) Ottomata: [C: 2] Since mobile jobs have been turned off, remove now unneded pig scripts [analytics/kraken] - https://gerrit.wikimedia.org/r/117321 (owner: QChris)
[21:38:43] <grrrit-wm>	 (CR) Ottomata: [C: 2] Switch UserAgentClassifier's returned schema to tuple [analytics/kraken] - https://gerrit.wikimedia.org/r/117319 (owner: QChris)
[21:39:05] <grrrit-wm>	 (CR) QChris: "This change was necessary to get the mobile jobs to run outside" [analytics/kraken] - https://gerrit.wikimedia.org/r/117319 (owner: QChris)
[21:39:32] <grrrit-wm>	 (CR) QChris: "This change was necessary to get the mobile jobs to run outside of Hadoop. I wanted to streamline it back before removing the mobile job's" [analytics/kraken] - https://gerrit.wikimedia.org/r/117320 (owner: QChris)
[21:44:52] <grrrit-wm>	 (CR) QChris: Add non_wmf_mobile_app handling to mobile_device_props.pig (1 comment) [analytics/kraken] - https://gerrit.wikimedia.org/r/117320 (owner: QChris)
[22:00:51] <grrrit-wm>	 (CR) Ottomata: Add non_wmf_mobile_app handling to mobile_device_props.pig (1 comment) [analytics/kraken] - https://gerrit.wikimedia.org/r/117320 (owner: QChris)
[22:48:09] <grrrit-wm>	 (CR) QChris: Add non_wmf_mobile_app handling to mobile_device_props.pig (1 comment) [analytics/kraken] - https://gerrit.wikimedia.org/r/117320 (owner: QChris)
[22:59:05] <grrrit-wm>	 (CR) Ottomata: [C: 2] Add non_wmf_mobile_app handling to mobile_device_props.pig (1 comment) [analytics/kraken] - https://gerrit.wikimedia.org/r/117320 (owner: QChris)
[22:59:17] <ottomata>	 qchris: , I"ve +2ed all of those I think
[22:59:20] <ottomata>	 merge at will :)
[22:59:38] <qchris>	 Thanks.
[22:59:47] <qchris>	 Should I merge?
[23:00:06] <qchris>	 I thought the one who does the review merges?
[23:00:22] <qchris>	 We need Jenkins to merge for us :-D
[23:01:21] <qchris>	 Thanks ottomata.
[23:01:35] <ottomata>	 either way
[23:01:36] <ottomata>	 you merge!