[13:41:14] (PS1) Milimetric: deduplicating by username and project [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/86851 [13:41:30] (CR) Milimetric: [C: 2 V: 2] deduplicating by username and project [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/86851 (owner: Milimetric) [13:52:41] * drdee is testing the new realtime page view counter for english wikipedia: 1, 2,3,4,5, stack overflow [13:53:28] ? [13:54:46] i was counting in my head [13:54:59] you need more coffee ;) [13:55:17] oh you are the realtime pageview counter [13:55:19] very good! [13:55:25] you will need a LOT more coffee if that is so [13:55:41] these days when I read 'stack overflow' i think of the website [13:55:47] true :) [13:56:14] can you have a look at https://mingle.corp.wikimedia.org/projects/analytics/cards/1195 [13:56:22] and tell me what's missing? [13:57:33] not sure, but we might want to just download from dumps, instead of rsync [13:57:45] less reliance on ssh keys and user perms [13:57:55] especially between labs and prod [13:58:20] yea, i tihnk you should take the tech details there out of the acceptance criterea [13:58:36] also, we decided to roll up daily, not hourly, right? [13:58:45] yes daily [13:58:46] so these would only be imported into mysql once a day [13:59:02] but if we download from dumps [13:59:08] it counts towards our bandwidth i believe [13:59:50] labs bandwidth? [13:59:54] there'a bandwidth cap [13:59:55] ? [13:59:59] not a cap [14:00:03] we pay for it [14:00:10] hmm [14:00:17] i dunno about that, hm [14:00:28] depends on how the request is routed [14:00:33] right [14:00:47] qchris: can you verify whether https://bugzilla.wikimedia.org/show_bug.cgi?id=54779 has been resolved? [14:04:18] qchris i think i know why we don't see a drop in traffic [14:04:21] it's because of this https://gerrit.wikimedia.org/r/#/c/86708/ [14:04:37] which means that all requests from a matching cidr range get tagged [14:04:52] it no longer looks at the language-code of the project [14:05:06] drdee: about bug 54779: No. That requires access to the raw stream which I do not have. I can do that only tomorrow ~noot UTC when the new file gets synced over. [14:05:20] so we are over now reporting page views for all carriers [14:05:29] ok regarding 54779 [14:06:54] drdee: About https://gerrit.wikimedia.org/r/#/c/86708/ I think we interpret that change differently [14:07:03] drdee: Do you think it changed behaviour? [14:07:20] yes it does [14:07:23] ? [14:07:24] How? [14:07:53] before a request would get tagged if both the ip address matched and the language code was whitelisted [14:08:04] now it will get tagged if the ip address matches [14:08:13] No, I do not think so. [14:08:30] It is still only tagged for the relevant of m, and zero. [14:08:45] Note the differency between X-CS and X-CS2. [14:09:04] X-CS2 gets set for all. [14:09:17] But X-CS only gets set for those where also m, or zero match. [14:09:30] are there now 2 X-CS's headers? [14:09:37] and the final "else" captures those that do not depend on m/zero. [14:09:52] There is X-CS and X-CS2. Yes. [14:09:57] That came with the netmapper changes. [14:10:58] See line 14 in templates/varnish/zero.inc.vcl.erb of the puppet repo [14:11:04] That sets X-CS2. [14:11:06] yup saw it [14:11:23] i thought they were trying to make it less complicated :) [14:11:42] intention vs. code :-) [14:11:49] But in some sense, it got simpler. [14:12:09] So they are right as well. [14:14:10] i think we need to make a decision tree to visualize what requests should get tagged and counted from an analytics PoV [14:14:40] i think the implementation in varnish is going to be less accurate as they optimize for performance [14:14:48] while we should be optimizing for accuracy [14:25:59] batcave anyone? [14:34:32] drdee: Coming. [15:14:22] drdee, hope my answer in the bug cleared stuff up [15:14:24] i'm off to the airport [15:14:25] can answer silly questions with silly answers over the phone :) [15:15:32] can you help me with creating a flow diagram on what should be counted as wikipedia zero traffic from an analytics pov? [15:15:48] try that on your silly phone! [15:16:38] ;) [16:37:10] (PS1) Stefan.petrea: Adding censored property [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/86878 [16:37:25] cool average! [16:37:40] cool average! [16:38:27] hi drdee_ [16:38:34] it's still in progress [16:38:49] but yeah, it's a step forward [16:38:52] hi milimetric [16:38:57] hi tnegrin , ottomata , qchris :) [16:38:57] aim just happy to see progress ;) [16:39:03] hey average [16:39:04] Hi average [16:39:11] hola [16:39:37] wanna hangout after the scrum average? [16:42:31] milimetric: sure [16:44:00] hii average [16:45:41] oh hi ottomata :) [16:45:44] are we using cassandra anywhere ? just asking [16:49:02] nope [16:52:05] ok [16:53:29] http://www.youtube.com/watch?v=KdxEAt91D7k -- The Math class [17:01:17] ottomata; scrum [17:05:09] ahhh sneaky! [17:47:47] DarTar: hey [17:47:59] DarTar: is censored calculation independent of the metric survival ? [17:48:02] hey average [17:48:09] DarTar: can you come to the batcave pls ? we're discussing it now [17:48:22] yes, I can join in 2 [17:49:47] no problem DarTar, I think we got this [17:52:22] I'm available now, ping me if you want to chat and I'll join the hang out [17:52:51] https://plus.google.com/hangouts/_/00e5ffe5580f0fecf619df2f0a27eae0c20e98e2