[09:33:59] New review: Yuvipanda; "Testing" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [09:34:30] New review: Yuvipanda; "TEstset" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [09:35:19] New review: Yuvipanda; "Spamming" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [09:35:46] New review: Yuvipanda; "More spamming" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [09:36:37] New review: Yuvipanda; "Hopefully final spamming" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [09:40:02] New patchset: QChris; "Allow countryCode lookups for IPv6 addresses" [analytics/kraken] (master) - https://gerrit.wikimedia.org/r/74595 [09:48:08] Woooooohaaaaaaaaaaaaaaaaaaaaa [09:48:22] Oh maaan, this must be the best day [09:48:38] Faidon merged one of my changes !!!! https://gerrit.wikimedia.org/r/#/c/73860/ [11:03:33] New review: Yuvipanda; "Hope. Meh." [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:03:55] New review: Yuvipanda; "Maybe?" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:07:25] New review: Yuvipanda; "It is raining here." [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:35:03] New review: Yuvipanda; "Testing echo to IRC" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:41:09] New review: Yuvipanda; "Testing comment formatting" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:41:46] New review: Yuvipanda; "Testing formatting some more" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:45:54] New review: Yuvipanda; "So if I do have multiple lines of changes" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:51:42] New review: Yuvipanda; "Setting both CR and this" [analytics/limn-mobile-data] (master) C: -2; - https://gerrit.wikimedia.org/r/59880 [11:55:26] New review: Yuvipanda; "Testing networkiness" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [11:58:43] New review: Yuvipanda; "Testing new Format" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:16:42] New review: Diederik; "Looks good!" [analytics/kraken] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/74595 [12:16:42] Change merged: Diederik; [analytics/kraken] (master) - https://gerrit.wikimedia.org/r/74595 [12:20:42] New review: Yuvipanda; "Testing" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:22:12] New review: Yuvipanda; "Testing again" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:23:03] New review: Yuvipanda; "TEsttt" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:24:00] New review: Yuvipanda; "Did I kill myself with flood?" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:24:52] New review: Yuvipanda; "Testing locally" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:26:07] New review: Yuvipanda; "Testttt" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:31:01] New review: Yuvipanda; "Test" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [12:33:00] New review: Yuvipanda; "Testtttt" [analytics/limn-mobile-data] (master) C: -2; - https://gerrit.wikimedia.org/r/59880 [13:08:09] New review: Yuvipanda; "Tada!" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [13:11:05] New review: Yuvipanda; "test" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [13:11:28] New review: Yuvipanda; "Empty Array is true. Grr Javascript" [analytics/limn-mobile-data] (master) - https://gerrit.wikimedia.org/r/59880 [13:48:15] hey milimetric, shouldn't we actually activate this card: https://mingle.corp.wikimedia.org/projects/analytics/cards/752 ? [13:52:21] morning qchris [13:56:13] (sorry about the spam, to people who were in the channel during the night) [13:57:48] yesss that was awful! [14:14:20] drdee: what's up with the 930PDT scrum? [14:14:44] we had to move it for one time because of scheduling conflict with the hadoop team [14:14:50] gotcha [14:14:51] cool [14:15:11] i should have mentioned that yesterday during scrum [14:16:32] not a big deal, but I think I'll be in transit at that time so I'll miss scrum [14:20:48] http://www.dodgycoder.net/2011/11/yoda-conditions-pokemon-exception.html [14:34:07] did I mention that paravoid merged #73860 ? [14:34:13] this came as a huge surprise [14:34:45] have to see about the last part, merging debian/* [14:42:18] that's great news average! [14:42:32] enjoy the sweetness of your victory [14:42:44] so one patch set is left? [14:46:59] yes [14:58:05] hey drdee [14:58:10] we need a public IP, I keep forgetting [14:58:23] or we can redirect from limn0... [14:58:47] for....... [14:58:49] I don't remember but it seemed like IPs needed manager approval or something special like that [14:58:57] to get a real address in labs [14:59:03] instead of the ugly instance one [14:59:36] drdee: as in wikimetrics.wmflabs.org [14:59:51] manager approval? [14:59:56] never heard of that beofre [15:00:00] i think it's just poke Ryan_Lane [15:01:11] ok [15:09:49] average: ^^ [15:22:38] average: yes ? [15:25:50] you saw my question? [15:27:12] drdee: https://metrics.wikimedia.org/ [15:27:14] Coren rules [15:27:31] I thought this was crazy hard from what dschoon was saying [15:27:54] no it's not hard; [15:28:14] you meant http://metrics.wmflabs.org/ [15:28:51] http://metrics.wmflabs.org/about/ gives a 404 :) [15:29:44] i also don't think we should link to the " all users who signed up via the Thank You campaign)" cohort on the frontpage [15:33:15] yeah, i didn't wanna mess with dario's wording [15:33:19] i'll think of something else [15:35:39] New patchset: Milimetric; "adapted dates for mediawiki timestamps" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74634 [15:35:39] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74634 [15:38:32] errr gerrit https is down [15:38:33] drdee: yeah, I answered, one patch left [15:40:10] ah, gitblit is down altogether [15:49:55] New patchset: Milimetric; "typo" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74638 [15:50:01] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74638 [16:14:22] New patchset: Milimetric; "decimal json fix" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74644 [16:14:26] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74644 [16:30:37] milimetric, average, erosen: scrum [16:59:26] damn, missed the scrum [16:59:42] I was taking a shower and then I saw it was at 18:30 [17:00:04] drdee: if you wanna talk with me I can join the hangout [17:03:05] * average just joined [17:29:05] New patchset: Stefan.petrea; "Debian metadata" [analytics/dclass] (debian) - https://gerrit.wikimedia.org/r/74651 [18:10:35] heya, milimetric, is instance-proxy.wmflabs.org working for you? [18:11:31] sorry, taking it here [18:11:45] yeah, so i'm smashing wikimetrics.instance-proxy with a big operation [18:11:48] so it'll be dead for a bit [18:11:54] is that the one you were hitting? [18:16:14] naw not that one [18:19:25] New patchset: Milimetric; "cohort upload fixes" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74657 [18:20:03] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74657 [18:31:19] New patchset: Milimetric; "hopefully the last cohorts upload fix" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74661 [18:31:32] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74661 [18:40:51] so last patchset for dclass was pushed [18:40:58] added Faidon as reviewer [18:43:04] awesome! average [18:52:55] changing locs, back in a bit [18:53:39] New patchset: Milimetric; "cohort upload working well, must add tests though" [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74665 [18:53:48] Change merged: Milimetric; [analytics/wikimetrics] (master) - https://gerrit.wikimedia.org/r/74665 [19:02:46] milimetric: been working on dealing with task errors; going to lunch now, but just checking in [19:03:01] milimetric: congrats on getting the upload working [19:03:51] hey erosen [19:03:53] ok [19:03:55] so tasks work! [19:03:56] woo! [19:03:59] really? [19:04:00] it was a json decoder thing [19:04:01] awesome [19:04:07] phew [19:04:14] that is seriously relieving, I was starting to get worried [19:04:18] it wasn't dumping decimal.Decimal properly [19:04:25] no idea why [19:04:37] but the solution is same as dates, just added it to the decoder [19:04:41] gotcha [19:04:52] cool, will you be around in a bit? [19:04:55] milimetric: [19:04:59] oh, one note of import: I changed wikimetrics-queue to wikimetricsqueue [19:05:04] i'm going to lunch now [19:05:05] but yes [19:05:10] we can talk after [19:05:12] awesome [19:05:15] bon apetit [19:05:15] see you in a bit [19:05:17] later [19:10:28] Where can I go to get a break-down of en.wiki editor demographics? [19:11:04] DarTar: ^ [19:11:24] drdee: ^ [19:11:25] demographics, being? [19:11:33] lifetime edits? [19:11:38] gender, race, age [19:11:42] sorry :) [19:11:46] :p [19:12:04] I imagine this is only from the editor surveys [19:12:29] yes, or from self-reported data on user profiles [19:12:49] or gender preference [19:14:52] Something like http://strategy.wikimedia.org/wiki/American_Wikipedia_reader_demographics but for editors [19:24:49] Actually, it looks like we've never collected racial information. That seems strange. [20:01:43] kaldari: we started collecting opt-in demographic data for gender http://meta.wikimedia.org/wiki/Research:Gender_micro-survey [20:01:53] the same approach could be used for other data [20:02:06] (that's currently targeting new registered users only) [20:04:32] DarTar: kaldari's not in this room [20:04:42] (anymore) [20:05:49] erosen: back [20:09:35] you're right, will drop him a line [21:12:26] (CR) Yuvipanda: "Final test, I promise!" [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/59880 [21:12:32] \o/ [21:45:57] (CR) Yuvipanda: "I swear I will not spam this again" [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/59880 [21:46:11] wanna bet YuviPanda? [21:46:35] (Abandoned) Yuvipanda: Generate data for Web too [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/59880 [21:46:37] drdee: ^ :P [21:46:43] i win :) [21:46:55] :P [21:46:59] * YuviPanda buys drdee a drink [21:47:30] * drdee opens a cool refreshing beer just passed down to him by YuviPanda, thanks! [21:49:34] drdee [21:49:34] woot! [21:49:35] https://kraken-namenode.wmflabs.org:8888/ [21:49:39] https and everythang! [21:50:05] hey nice ottomata [21:50:05] everythang???? [21:50:08] WOOT! [21:50:10] that's awesome [21:50:22] most def! [21:50:33] that's got hive, oozie, yeehaw [21:50:37] only one datanode [21:50:42] really only a 2 node cluster [21:50:46] but we can add and move things around if we wannt [21:50:53] but yeehaw! [21:51:20] very cool mr! [21:52:00] our logins don't work on it yet right? [21:52:01] oh they do! [21:52:03] holy crap! [21:53:32] yup [21:53:32] ldap! [21:54:32] have a good weekend all! [22:07:22] drdee: question about the webrequest stream and July zero numbers [22:07:37] drdee: do you know if htere is enough data in there from which to make reasonable estiamtes? [22:11:08] erosen: You're talking about the logs that ottomata produced? They are starting ~12 July. [22:11:24] qchris: awesome [22:11:50] qchris: do you know if they are high quality? [22:12:10] erosen: :-) I have no idea. [22:12:27] They are /a/squid/archive/zero/zero.tsv.log-201307* on stat1002 [22:12:30] no worries, i suspected you wouldn't know [22:12:54] ooo [22:13:01] qchris: is that the new filter? [22:13:24] I am just doing some plausibility checks on them to see if the pig scripts do their job, and except for the first file, they are at least somewhat consistent. [22:13:31] awesome [22:14:32] I have no idea by which filter ottomata produced the files. I am just mangling tham so the zero_{country,carrier} scripts work with them :-/ [22:14:49] awesome [22:14:51] check puppet [22:14:59] qchris: this is very useful; thanks [22:15:11] erosen: btw ... I wanted to ask you anyways where I should store the output of those scripts. [22:15:33] good question [22:15:41] you mean the zero_carrier.tsv [22:16:07] Now they are just the pig output directories with the part files, crc, success, ... but yes the output of zero_carrier.pig, so I guess that's zero_carrier.tsv [22:16:28] (Same for zero_country.{pig,tsv}) [22:18:37] drdee: do you have a second to chat about the urgent request from Amit? [22:18:48] at 4pm [22:18:53] kk [22:25:15] milimetric: around for hangout? sorry about all the delays [22:25:21] yo erosen [22:25:24] got your call [22:25:25] what's up? [22:25:41] basically amit wants to know if there is good webrequest data from july 9-14 [22:25:45] I figured you would konw [22:26:27] we turned on the x-analytics udp2log oxygen filter on july 10 [22:26:34] so that data should be good for sure [22:26:43] awesome [22:26:46] that's on stat1002 /a/squid/archive/zero/zer.tsv* [22:27:03] the data that is in kraken should be good again starting the 16th [22:27:09] k, so those unsampled for x-cs stream? [22:27:16] k, awesome [22:27:32] yup unsampled [22:27:41] ottomata: actually, looks like filter starts on the 12th [22:27:47] hmm [22:28:00] hmm, i just looked at my commit log [22:28:23] ottomata: no, you're good, i think [22:28:27] file was created on teh 12th [22:28:31] but the first request is from 2013-07-10T20:34:36 [22:28:39] aye ok, cool [22:28:40] which seems about right [22:28:41] then starting the 11th [22:28:42] yeah [22:29:00] cool [22:29:05] that was the question [22:29:08] sorry to keep you after hours [22:30:53] np i got a few [23:14:01] hey erosen, I'm around [23:14:09] heyo! [23:14:21] see my email? [23:14:24] hangout? [23:14:25] yup [23:14:26] yeah [23:15:12] i'm in the hangout [23:15:20] milimetric: ^^ [23:15:34] my hangouts is malfunctioning :(