[00:24:26] <dschoon>	 back
[00:24:46] <dschoon>	 back
[00:30:28] <drdee>	 cool milimetric, and thanks for using the bot!
[00:49:05] <erosen>	 dschoon: quick color brewer q
[00:49:12] <dschoon>	 sure
[00:49:15] <dschoon>	 shoot
[00:49:21] <erosen>	 do you know of a canonical js file somewhere on the interwebs
[00:49:22] <erosen>	 ?
[00:49:29] <erosen>	 so I can use it in a gist?
[00:49:33] <dschoon>	 i think d3 has one checked in
[00:49:38] <erosen>	 http://bl.ocks.org/4458705
[00:49:43] <erosen>	 aa
[00:50:47] <dschoon>	 i did one for python
[00:50:52] <erosen>	 yeah
[00:51:07] <erosen>	 i am trying to get it to used in a bl.ock
[00:51:08] <dschoon>	 nice
[00:51:16] <dschoon>	 are you not using that?
[00:51:27] <erosen>	 i'm not
[00:51:39] <dschoon>	 i mean, the one you have there
[00:51:55] <dschoon>	 https://github.com/mbostock/d3/blob/master/lib/colorbrewer/colorbrewer.js
[00:51:56] <erosen>	 i tried but I found it in some rando's github repo
[00:52:01] <dschoon>	 raw:
[00:52:01] <dschoon>	 https://raw.github.com/mbostock/d3/master/lib/colorbrewer/colorbrewer.js
[00:52:16] <erosen>	 excellent
[00:52:18] <erosen>	 I think that is what I need
[00:52:20] <erosen>	 thanks
[00:52:21] <dschoon>	 i get the feeling the one you're using is from me
[00:52:25] <erosen>	 hehe
[00:52:27] <dschoon>	 as it has exports stuff
[00:52:48] <erosen>	 yeah
[00:53:11] <erosen>	 i can't find the page where I found it but it was someone else
[00:53:39] <dschoon>	 the website is colorbrewer2.org
[00:53:49] <erosen>	 yeah
[00:53:49] <dschoon>	 but that JS impl is mike, iirc
[00:54:17] <erosen>	 i looked around but weirdly i didn't see any source
[00:54:26] <erosen>	 yay
[00:54:27] <erosen>	 works
[00:54:29] <dschoon>	 woo
[00:54:33] <erosen>	 http://bl.ocks.org/4458705
[00:54:47] <dschoon>	 (looks the same to me)
[00:55:00] <erosen>	 chrome doesn't deal well with blocks refreshes
[00:55:04] <erosen>	 i had to switch to firefox
[00:56:31] <dschoon>	 http://bl.ocks.org/d/4458705/
[00:56:37] <dschoon>	 that looks very red-blue to me
[00:57:10] <dschoon>	 you really ought to change those floats to %s
[00:57:11] <dschoon>	 :)
[00:57:16] <erosen>	 hehe
[00:57:19] <erosen>	 will do
[00:57:24] <dschoon>	 and make it clear that it's percent of traffic from that country
[00:57:31] <erosen>	 the colors are just a random set
[00:57:42] <erosen>	 i'll add a README.md
[00:58:00] <dschoon>	 er: make it clear that it's mobile as a fraction of all traffic from that country
[00:58:09] <dschoon>	 otherwise it looks like chad is 99% of our mobile traffic :)
[00:58:15] <erosen>	 hehe
[00:58:41] <dschoon>	 btw
[00:59:00] <dschoon>	 d3 has a string formatter which is basically the same as the python curly syntax
[00:59:43] <erosen>	 nice
[00:59:45] <erosen>	 i'll look into it
[00:59:51] <dschoon>	 d3.format(',.2p')(0.92134) == '92.13%'
[01:00:03] <erosen>	 that makes it easy
[01:00:29] <dschoon>	 sorry
[01:00:30] <dschoon>	 d3.format(',.2%')(0.92134) == '92.13%'
[01:00:40] <erosen>	 ah
[01:00:42] <dschoon>	 so you can just create formatter once
[01:00:45] <erosen>	 gotcha
[01:00:48] <dschoon>	 var formatter = d3.format(',.2%');
[01:00:51] <erosen>	 ya
[01:00:56] <erosen>	 it's function
[01:01:04] <dschoon>	 formatter(.92124) == '92.12%'
[03:13:11] <average_drifter>	 at the moment I'm getting 1minute/day_of_data
[03:13:22] <average_drifter>	 that's not bad, it could be better, still using just 1CPU..
[03:14:20] <average_drifter>	 http://stat1.wikimedia.org/spetrea/new_pageview_mobile_reports/r7/pageviews.html
[03:15:48] <average_drifter>	 this is another run, I'm getting data the same way (1st january, 1st february, 1st march , 1st april , 1st may ..)  and I ran the report on those
[03:18:48] <average_drifter>	 some of the percentages look off. that's because I still have as TODOs to select the period of the data to process ( currently it's getting the files mentioned above but those files don't contain data from that day only, they're just dumps from squid and they contain the previous day also, so there are problems with the first and the last month processed, but the fix is on the way)
[03:20:37] <average_drifter>	 as you can see the first month processed has very few entries, that's because the file for it (/a/squid/archive/sampled/sampled-log-20120101.gz probably contains just a few entries for january, and it contains more from the previous day which is 2011-12-31)
[03:23:23] <average_drifter>	 ottomata: would it be hard to customize the squid configuration to dump the data on disk according to the entries inside those files ?
[03:23:45] <drdee>	 looking now
[03:23:55] <average_drifter>	 ottomata: I mean, currently it just dumps the data at 03:00 in the morning so one day labeled 2012-10-02 contains entries from 2012-10-01 also
[03:24:19] <drdee>	 average_drifter: wikistats does contain some code to handle those issues
[03:24:26] <drdee>	 maybe you can use that?
[03:24:32] <average_drifter>	 ottomata: if we could do that, it would make my code more simple because I wouldn't have to check at which row the data I need starts
[03:24:51] <average_drifter>	 drdee: yeah, it's no problem to implement it
[03:25:21] <drdee>	 that's probably the fastest route
[03:25:26] <average_drifter>	 ok
[03:27:16] <average_drifter>	 1m / day => 30m / month (on average) => 6h / year => 12h to process 2010,2012, and january 2013
[03:33:36] <average_drifter>	 drdee: line 27 here =>   https://gerrit.wikimedia.org/r/#/c/41979/5/pageviews_reports/lib/PageViews/Model.pm
[03:33:44] <average_drifter>	 drdee: what do you think of that regex ?
[03:34:00] <average_drifter>	 drdee: my $re_valid_language = "([a-z\-]{1,8})";
[03:34:05] <drdee>	 looking
[03:34:18] <average_drifter>	 drdee: is this how we should verify if a language is valid ?
[03:34:31] <drdee>	 i don't think it can be longer than 3 maybe 4 characters