[01:11:53] 10Analytics, 10Analytics-Cluster, 10Wikimedia-General-or-Unknown: Browser and platform stats for logged-in vs. anon users for security and product support decisions - https://phabricator.wikimedia.org/T58575#2960075 (10Liuxinyu970226) [01:13:08] 10Analytics, 10Analytics-Cluster, 10Wikimedia-General-or-Unknown: Browser and platform stats for logged-in vs. anon users for security and product support decisions - https://phabricator.wikimedia.org/T58575#2960077 (10Liuxinyu970226) [01:13:24] 10Analytics, 10Analytics-EventLogging, 07Need-volunteer: Add sanitized User-Agent to default fields logged by EventLogging - https://phabricator.wikimedia.org/T54295#2960086 (10Liuxinyu970226) [06:17:10] 10Analytics: Add statistics and error logging to mediawiki history reconstruction job - https://phabricator.wikimedia.org/T155971#2960420 (10JAllemandou) [08:39:27] (03CR) 10Joal: "Quick comments for improvement inline." (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141) (owner: 10Mforns) [11:55:38] joal: just added the ACLs for reach dbproxy1010:3306 from the analytics vlan [11:55:52] Wouhou elukey ! [11:55:56] tested? [11:56:16] there is only one thing left that needs Jaime's comment, namely if we also need to whitelist dbproxy1011 (that might be a backup or something similar) [11:56:29] yep telnet an1044 -> dbproxy1010 works :) [11:56:43] awesome elukey :) [11:56:47] Thanks a mil mate [11:56:53] but we need to use the CNAME [11:57:05] Doesn't bother me too much [11:57:45] labsdb-analytics.eqiad.wmnet [11:57:59] because afaicu this one might point to dbproxy1010 or 1011 [11:58:33] k elukey [11:58:43] elukey: just tried with telnet, seems working :) [11:58:59] elukey: do you know if Jaime works today? [11:59:47] not sure, but I pinged also Manuel (in a meeting now) [11:59:53] k elukey [12:00:21] elukey: let me know when you have Manuel around, I'd like to ask him what process I should follow to ask for a user on that db [12:01:46] sure [12:01:52] also aqs1007 is alive! \o/ [12:01:58] no cassandra on top of course [12:02:09] awesome elukey :) [12:02:23] let me know if I can help [12:04:42] ah Eric answered! [12:05:06] so say we bootstrap aqs1007 with a new logical rack for cassandra [12:05:15] the first instance will stream from the whole cluster [12:05:24] the second one from the rack only [12:05:27] :/ [12:06:04] but the main issue is reasoning about a cluster with physical rack vs logical racks not in sync [12:06:38] from what I gathered, Restbases uses the Row number as rack (so groups of racks rather than single ones) [12:15:39] elukey: I now understand as well the lading (second instance for 1007 will only use same rack, so same machine ... meh) [12:15:57] elukey: however I don't understand the thing about physical / logical racks :S [12:17:01] me too [12:17:35] Eric says that theoretically having replication == racks helps when you have to reason about what happens in the cluster [12:18:46] elukey: I understand it makes things easier in the brain - He advise is that we use 3 racks ? (since we want rep factor = 3) [12:21:24] yeah something like that [12:22:18] elukey: if machines are physically in the same rack, I'm not opposed - if they are not, it'll get even messier in my opinion [12:46:16] joal: I have the same opinion but I feel that I am missing something [12:46:30] I'll try today to check restbase's config [12:46:48] in the meantime, I am installing the os on all the new aqs nodes :) [12:46:58] 40 cores and 128GB of ram !!! [12:47:13] Maaaaaan :) [12:47:19] huhuhu [12:47:35] * joal want to test clickhouse on those beasts :) [12:49:25] hahahah [13:07:29] 06Analytics-Kanban: Run a 1-off sqoop over the new labsdb servers - https://phabricator.wikimedia.org/T155658#2961083 (10elukey) >>! In T155658#2956313, @elukey wrote: > Are there any outbound rules for the Analytics VLAN by any chance? I can see the following on cr1: > > ``` > show configuration firewall famil... [13:56:02] elukey: heya, any news from Manuel? [14:00:22] ottomata: HelllLLLLOOOoooo :) Just in time :) [14:00:35] joal: he just finished the meeting and went to lunch :) [14:00:45] hiiiii [14:00:58] wasssuuup? [14:01:28] ottomata: kafkacat -C -b kafka1012.eqiad.wmnet:9092 -t test_banner_impressions_joal [14:01:46] OOoOOo! [14:02:13] nice! I know what to do... :) [14:03:17] ottomata: please set 'minute' query granularity - I aggregate and change timestamp for that [14:03:30] segment? [14:03:37] daily? [14:03:48] segment daily, query minute? [14:03:53] yessir please [14:03:59] k [14:04:12] optimal segment time is month, but we'll do that batch [14:04:17] thanks :) [14:04:55] oh him, but the streaming job only emits once per minute? [14:05:14] ottomata: once every 10 seconds, but timestamp is updated to be at minute [14:05:21] :) [14:05:28] oh ok [14:05:33] hm , did it stop then? [14:05:42] Yes, I stopped it [14:05:45] oh [14:05:45] k [14:06:00] Need to be a bit constructed than in a shell ;) [14:08:51] ottomata: I solved the inbound/outbound mistery for the analytics VLAN [14:09:01] also, o/ [14:09:11] as always it was me confusing things [14:09:29] the network ACLs are for each router's port [14:10:15] so from the router's point of view, *inbound traffic* is whatever flows *to* the interface from the host/switch attached [14:10:37] from the Analytics VLAN perspective, is traffic from others networks [14:11:10] sorry, traffic *to* other networks [14:16:06] 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2961220 (10mforns) @DStrine @AndyRussG Yes, I'm already on it :] [14:16:11] ahhh [14:16:17] k [14:16:21] right [14:16:43] so that is allowing traffic from analytics into the router's physical port destined to those IPs/ports? [14:16:44] elukey: ? [14:17:28] yes! [14:21:03] aye cool [14:21:04] good to konw [14:21:05] know [14:22:18] ottomata: do you like https://gerrit.wikimedia.org/r/#/c/332106/9/modules/role/manifests/analytics_cluster/client.pp ? The rest is only one liners [14:22:35] (I am also running pcc just in case) [14:23:13] mornin [14:23:57] o/ [14:24:18] (https://puppet-compiler.wmflabs.org/5181/ looks good) [14:25:08] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961246 (10Ottomata) Wow. Gilles asked in an email if someone from Analytics could help. I can! I don't know much... [14:25:21] 06Analytics-Kanban, 13Patch-For-Review: Productionize loading of edit data into Druid (contingent on success of research spike) - https://phabricator.wikimedia.org/T141473#2961247 (10Milimetric) Awesome, thanks @JAllemandou, can mark this done then. [14:25:33] elukey: sure [14:25:35] that's better? [14:25:47] oh this is the :: thing [14:25:48] sure [14:25:50] i don't miwd [14:25:51] midn [14:25:52] mind [14:25:55] super, merging [14:27:29] (03CR) 10Mforns: [WIP] Add banner impressions jobs (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141) (owner: 10Mforns) [14:29:34] hey hmm, nuria, milimetric, am reading quarterly goals [14:29:40] i don't think the frontend stuff should be called 'data lake' [14:29:46] eh? [14:29:54] I should read that, doing so now [14:29:56] sorry quarterly review [14:32:02] also, nuria, should the public eventstreams slide read 'productionize/deploy public eventstreams?' instead of 'launch'? [14:32:04] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2780126 (10Gilles) No, it's time it takes to generate the JS on the backend, which happens more frequently than one... [14:32:10] 'launch' is what we were doing this quarter [14:32:13] (ohh too early for nuria) [14:33:53] milimetric: slides say https://phabricator.wikimedia.org/T125854 is done? [14:33:56] its not, is it? [14:34:14] no [14:35:05] ok, commented [14:35:49] I'm kind of ok with calling it the Data Lake *front-end* [14:36:48] but yeah, we should talk about it [14:40:23] i thought data lake was more about the backend, and our name for data warehouse, etc. [14:41:34] https://en.wikipedia.org/wiki/Data_lake [14:43:55] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961273 (10Ottomata) Ok, I looked in the extension to see where it might reference those schemas, but I don't see an... [14:45:51] ott [14:45:55] doh [14:46:07] ottomata: yeah, it's what the place where all the data is called [14:46:15] but she's saying this is the data lake _front-end_ [14:46:39] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961277 (10Gilles) Well, since EventLogging users can create any number of schemas on the fly, I'm not surprised tha... [14:46:53] it's good because it explains why we're building a data lake (to make different front-ends to it, and get data published) [14:46:57] 10Analytics, 10EventBus: log-events topic emitted in EventBus - https://phabricator.wikimedia.org/T155804#2955138 (10Ottomata) Hm, interesting. It'd be really helpful to design a schema for log events then. Can extensions emit custom log events in any format, or do they just set some 'log type' field to a cu... [14:47:18] it's bad because it's really just called "wikistats" and is just one of the many ways to access data in the data lake [14:52:44] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961293 (10Gilles) Here's an example ResourceLoader request for the NavigationTiming schema's JS: https://en.wikipe... [14:52:47] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961294 (10Ottomata) Oh, ok. I think I'm getting it. So a user requests a page (say maybe after a recent deploy) t... [14:54:13] ottomata: back on ;) [14:54:30] joal: aye, just finished email checking :) [14:54:44] np ottomata, just letting you know ;) [14:55:33] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961311 (10Gilles) Yes, that's the chain of events. Anything's possible, the issue probably is in the mechanism used... [15:00:24] (03PS1) 10Joal: Add spark streaming job for banner impressions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/333655 [15:00:28] ottomata: --^ [15:02:28] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961344 (10Gilles) There is some very suspicious locking code with a 20s timeout in RemoteSchema.php My hunch would... [15:02:50] ottomata: I'm sorry you're my main go-to person this afternoon, I have another question for you when you'll have a minute (on mediawiki history scala CR) [15:03:02] joal: hit me! [15:04:19] (03CR) 10Ottomata: "Cool, I'd rather not merge this though, as for now we are just in PoC mode and trying stuff out." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/333655 (owner: 10Joal) [15:04:28] ottomata: About the page and user data extractor files / objects - changing their names: I have picked [User|Page]EventFromLoggingBuilder [15:04:38] ottomata: is that okey? [15:04:47] have to remember my comment... [15:04:48] :) [15:05:19] I'm not found of "From", but "EventLogging" without From means too much in any of my contexts :) [15:05:54] ottomata: link : https://gerrit.wikimedia.org/r/#/c/325312/12/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/mediawikihistory/page/PageHistoryDataExtractors.scala [15:07:45] joal: you are talking about renaming the classes? [15:07:51] objects [15:07:51] ? [15:07:52] classes + files [15:07:55] yes [15:07:57] so [15:08:14] PageHIstoryDataExtractor -> PageEventFromLoggingBuilder? [15:08:18] I have already renamed functions inside (made a lot of sense) [15:08:23] correct [15:08:25] hm [15:08:40] Or even just: PageEventBuilder ? [15:08:50] joal: Manuel told me that the accounts are probably managed by Chase.. [15:08:53] hmm, that sounds a little better [15:08:57] did you have anything specific to ask? [15:09:04] PageEventBuilder [15:09:04] PageEventFromLog(Builder) [15:09:26] joal: are all of these Event Extractor/Builders using log rows? [15:09:37] elukey: nope, I'll talk to chase tomorrow, meeting planned :) [15:10:10] gooood [15:10:21] I am going to add the last network ACL [15:10:22] ottomata: builders use log row yes, but some functions are used in other places [15:10:30] Thanks a lot elukey :) [15:10:43] I'll be all set up for tomorrow elukey, that's really awesome :) [15:11:00] aye that's fine. Ja joal i think PageEventBuilder is good. PageEvent____ [15:11:13] PageEventParser, PageLogEventParser [15:11:13] ottomata: Great :) [15:11:22] PageLogEventBuilder [15:11:28] naw [15:11:30] PageEventBuilder [15:11:50] i think its fine not to explicitly refer to 'log' in the obj name, especially if all of these behave about the same way [15:11:50] I used: buildMovePageEvent [15:11:56] ok [15:11:58] PageLogEventBuilder [15:12:00] sounds consistent [15:12:05] sorry [15:12:05] and buildSimplePageEvent [15:12:07] PageEventBuilder [15:12:11] cool [15:12:13] as you suggested :) [15:12:13] +1 [15:12:16] great [15:12:50] another one on naming: you suggested MediawikiEvent instead of HistoryEvent - makes sense but involves a lot of change, so triple checking going :) [15:12:56] ottomata: --^ [15:13:47] yeah, joal, i think that makes more sense to me. i'm not sure what a history event is...everything is history [15:13:51] except the future [15:14:00] 10Analytics, 10Pageviews-API: Yearly endpoint for the /pageviews/top API - https://phabricator.wikimedia.org/T154381#2909045 (10Milimetric) This is different from the per-article monthly endpoint. The storage needs would indeed be very modest, but with the current pipeline the job to compute the top article f... [15:14:22] ottomata: MediawikiEvent for the global object is cool - now for sub-objects: MediawikiEventPageInfo ? MediawikiEventPagePortion ? MediawikiEventPagePart [15:14:38] vs what? [15:14:41] ottomata: I can't make them subclasses, so it kinda need to be in the name [15:14:56] vs https://gerrit.wikimedia.org/r/#/c/325312/12/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/mediawikihistory/page/PageEvent.scala [15:14:56] ? [15:14:58] oops [15:15:01] vs PageEVent? [15:15:15] nope: https://gerrit.wikimedia.org/r/#/c/325312/12/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/mediawikihistory/denormalized/HistoryEvent.scala [15:15:22] sorry [15:15:22] eyah [15:15:22] i mean [15:15:26] you have these sub parts [15:15:29] but what are they sub parts of? [15:15:30] e.g. [15:15:33] MediawikiEventPageInfo [15:15:36] is a subpart of? [15:15:39] PageEvent? [15:15:44] MediawikiEvent [15:15:47] only [15:15:56] AH [15:15:57] ah [15:15:57] right [15:15:58] got it [15:16:03] In the same file [15:16:05] MediawikiPageEvent [15:16:11] but yeah, explicit is better [15:16:38] hmmm .. knowing we have PageEvent as well, I kinda don't like it :( [15:16:54] MediawikiEventPageDetails [15:16:56] k [15:17:00] yes ! [15:17:02] detials works [15:17:10] i dunno [15:17:12] info is fine too [15:17:28] fields? [15:17:42] these seem to be simple case classes with params/fields only? [15:17:44] Could do [15:17:50] correct [15:17:54] MediawikiEventPageFields [15:17:54] ? [15:17:59] To overcome the 22-limit parameter [15:18:04] works for me ! [15:18:12] OH that's the only reason they exist? [15:18:12] hahah [15:18:17] man, scala. [15:18:30] i kinda like MediawikiEventPageFields [15:18:34] MediawikiEventPageFields, MediawikiEventUserFields, MediawikiEventRevisionFields and MediawikiEvent [15:18:35] hmmm [15:18:36] although [15:18:39] you could have an instance of MediawikiEventPageFields [15:18:43] does that makes sense? [15:18:52] o: MediawikiEventPageFields = ... [15:19:03] pageFields.pageTitle [15:19:04] hmmm [15:20:05] for usage, I kinda prefer pageInfo.pageTitle, but it's almost never used [15:20:11] yeah.... [15:20:18] i'm leaning towards Info or Detail now [15:20:24] uhuh [15:20:33] which do you prefer? [15:20:47] I think Details is clearer [15:20:50] ok [15:20:53] i'm into it [15:20:54] info is very far too braod :) [15:21:00] cool awesome [15:21:00] Details it is. [15:21:16] I'll stil confirm with mforns, but it's on the way :) [15:21:17] I think Detail (not plural is better) [15:21:18] ja? [15:21:25] MediawikiEventPageDetail [15:21:26] hmmm [15:21:27] yessir [15:21:35] what what? [15:21:36] or Details? [15:21:37] hmmm [15:21:38] hahah [15:21:48] mforns: you can backlog the last 10 minutes ;) [15:21:55] hehe ok, reading [15:23:45] ottomata: that eventlogging resource loader issue might be really tricky [15:23:53] yeah [15:24:01] I remember ori, timo, and others wracking their brains about it [15:24:20] never made much sense to me, but caution is needed [15:25:16] joal, ottomata, names look good to me :] [15:25:34] 06Analytics-Kanban: Run a 1-off sqoop over the new labsdb servers - https://phabricator.wikimedia.org/T155658#2961478 (10elukey) >>! In T155658#2961083, @elukey wrote: > Last step is to verify with Jaime or Manuel if we need to whitelist dbproxy1011 too, since it might be dbproxy1010's backup if it fails (rememb... [15:25:38] mforns: so, MediawikiEventPageDetail, or MediawikiEventPageDetailS ? [15:25:45] ottomata too --^ [15:25:50] milimetric: this is def somethign i don't know much about [15:25:59] timo is involved in this ticket, i guess [15:26:04] joal, maybe plural, but both are fine [15:26:31] would we ever want to distinguish a single param from that case class as a 'detail'? [15:26:33] probably not... [15:27:18] joal: network work done, all set [15:27:31] * joal hugs elukey :) [15:27:58] hm ottomata - this argument is favor of singular, right? [15:28:03] i think so? [15:28:03] haha [15:28:07] :) [15:28:13] yeah [15:28:15] no [15:28:18] i'm into plural now. [15:28:21] thinking abou tit [15:28:24] if we called this [15:28:27] 'params' [15:28:29] we would never call it [15:28:33] MediawikiEventPageParam [15:28:38] agreed [15:28:40] we would do MediawikiEventPageParams [15:28:43] so, let's do details [15:28:46] Ok ! [15:28:50] ok! [15:28:52] :) [15:29:02] It's a wrap, now back to updating the code :) [15:29:07] Thanks a lot ottomata and mforns :) [15:29:08] thanks for name bouncing! [15:29:09] :) [15:29:51] haha [15:31:28] joal: qq [15:31:33] can you call "ts" "dt" [15:31:37] in banner impressions? [15:31:44] we usually use ts for integer timestamps [15:32:02] also, what are metricsSpec? [15:32:11] request_count, normalized_impressions [15:32:11] ? [15:32:12] mforns, ottomata: when talking about a MediawikiEvent in comments, I shall use mw_event - OK ? [15:32:15] and count? [15:32:32] joal: if in comment and you want to abbrev, i thikn don't make it look like code [15:32:35] MW Event is fine [15:32:47] ottomata: ok ! [15:32:58] MWEvent even, without space [15:33:01] either i guess :) [15:33:26] ottomata: back to druid: no count, see https://gerrit.wikimedia.org/r/#/c/331794/4/oozie/banner_impressions/druid/daily/load_banner_impressions.json [15:33:40] ottomata: the reason for ts is for druid convention :( [15:33:41] ah! [15:33:44] oh? [15:33:46] druid convention? [15:34:25] well, so far we've named druid timestamps ts without thinking of the ts/dt convention (sorry, my bad, didn't realise) [15:35:15] ottomata: Because druid examples name them ts :( [15:35:52] * joal cries in sadness for not having a brain big enough for conventions :( [15:35:55] haha [15:36:13] ja, dt are string ISO 8601s, and ts are integer unix epochs (either seconds or milliseconds) [15:36:26] joal: can we fix? how many places are we using ts incorrectly? [15:36:31] ottomata: Let's use dt for banners then [15:36:38] ok [15:37:09] k [15:38:03] joal, qq as well :] is pageview_data_directory used at all? here: https://github.com/wikimedia/analytics-refinery/blob/master/oozie/pageview/druid/daily/coordinator.xml#L19 [15:38:31] I havent found any use of it in the pageview druid load workflow [15:39:05] mforns: needs to defined for pageview/datasets.xml ;) [15:39:14] aaaaaah [15:39:51] joal: https://gist.github.com/ottomata/20cd0d9292f0be45c41edb80b41cf8fb [15:39:53] look ok? [15:39:58] ok ok got it [15:41:04] ottomata: we have pageviews in druid, that's all, the rest is not prod [15:41:10] ok - reading [15:41:16] ottomata: reading [15:46:54] ottomata: https://gist.github.com/jobar/8ff98301298cf1b74795cf8eccad0615 [15:47:00] ottomata: 2 minot changes [15:47:38] doubleSum and? [15:47:49] oh, status_code? [15:47:54] correct [15:47:59] rest seems very much ok [15:48:05] i don't see status_code in your data [15:48:10] hm [15:48:24] possibly cause it doesn't appear on url [15:48:38] now my wonder is: will druid accept null values? [15:48:53] ottomata: currently updating job [15:49:18] git st [15:49:21] oops [15:55:15] ottomata: back up [15:58:07] k [15:58:38] joal: still don't see status_code [16:00:33] ottomata: Aaaaah ! [16:01:34] joal: standup! [16:01:34] :) [16:01:37] joal: standdupppp [16:01:48] get up standup, standup for your rights [16:05:27] (03PS1) 10Joal: Update oozie job loading pageview in druid [analytics/refinery] - 10https://gerrit.wikimedia.org/r/333668 [16:11:02] joal: status_code":"2.2" ? [16:11:08] yep :) [16:11:13] what is status code? [16:11:18] trying again new stuff (off and on) [16:11:21] a banner impression thing? [16:11:24] I actually don't know ottomata [16:11:26] yes [16:11:26] haha ok [16:11:31] cool [16:36:06] (03Abandoned) 10Milimetric: [WIP] Script loading of edit history [analytics/refinery] - 10https://gerrit.wikimedia.org/r/301293 (owner: 10Milimetric) [16:36:33] 10Analytics-EventLogging, 06Analytics-Kanban, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2961773 (10Nuria) [16:47:15] 06Analytics-Kanban, 10Pageviews-API, 06Reading-analysis: Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016 - https://phabricator.wikimedia.org/T153699#2961818 (10Nuria) [16:52:04] 06Analytics-Kanban, 10MediaWiki-API: Copy cached API requests from raw webrequests table to ApiAction - https://phabricator.wikimedia.org/T155478#2944303 (10Nuria) [16:52:21] 06Analytics-Kanban, 10MediaWiki-API: Copy cached API requests from raw webrequests table to ApiAction - https://phabricator.wikimedia.org/T155478#2944303 (10Nuria) Need to spend some calendar time talking to reserach [16:52:33] 06Analytics-Kanban, 10MediaWiki-API, 06Research-and-Data: Copy cached API requests from raw webrequests table to ApiAction - https://phabricator.wikimedia.org/T155478#2961858 (10Nuria) [16:52:39] joal: ok, tranquility running [16:52:43] feel free to keep your spark job going [16:53:02] 06Analytics-Kanban, 06Research-and-Data: Coordinate with research to vet metrics calculated from edit data lake - https://phabricator.wikimedia.org/T153923#2961859 (10Nuria) [16:53:18] 10Analytics, 10RESTBase, 06Services, 15User-mobrovac: configure RESTBase pageview proxy to Analytics' cluster on wiki-specific domains - https://phabricator.wikimedia.org/T119094#2961862 (10Milimetric) [16:53:21] 10Analytics, 06Labs, 10Pageviews-API, 10wikitech.wikimedia.org: wikitech.wikimedia.org missing from pageviews API - https://phabricator.wikimedia.org/T153821#2961864 (10Milimetric) [16:57:06] 06Analytics-Kanban, 06Operations: Periodic 500s from piwik.wikimedia.org - https://phabricator.wikimedia.org/T154558#2961899 (10Nuria) [17:01:20] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2961906 (10Nuria) [17:01:23] 10Analytics, 10Analytics-EventLogging, 07Need-volunteer: Add sanitized User-Agent to default fields logged by EventLogging - https://phabricator.wikimedia.org/T54295#2961908 (10Nuria) [17:03:13] 10Analytics, 10Analytics-Cluster, 10Wikimedia-General-or-Unknown: Browser and platform stats for logged-in vs. anon users for security and product support decisions - https://phabricator.wikimedia.org/T58575#2961911 (10Nuria) General Browser stats available at http://analytics.wikimedia.org Resolving. [17:03:29] 10Analytics, 10Analytics-Cluster, 10Wikimedia-General-or-Unknown: Browser and platform stats for logged-in vs. anon users for security and product support decisions - https://phabricator.wikimedia.org/T58575#2961912 (10Nuria) 05Open>03Resolved a:03Nuria [17:04:16] 10Analytics, 10Analytics-EventLogging, 10MediaWiki-Vagrant: EventLogging vagrant role fails to provision - https://phabricator.wikimedia.org/T131085#2961914 (10Nuria) 05Open>03Resolved [17:09:23] 10Analytics, 06Operations, 10netops, 13Patch-For-Review: Open temporary access from analytics vlan to new-labsdb one - https://phabricator.wikimedia.org/T155487#2944637 (10Nuria) ping @elukey is this a duplicate [17:09:34] 06Analytics-Kanban, 06Operations, 10netops, 13Patch-For-Review: Open temporary access from analytics vlan to new-labsdb one - https://phabricator.wikimedia.org/T155487#2961934 (10Nuria) [17:18:27] 10Analytics, 10Wikimedia-General-or-Unknown: Disable queries for recent data on stats.grok.se - https://phabricator.wikimedia.org/T155785#2954564 (10Nuria) We are planning on backfilling old pageview data (likely not in pageview APi but other endpoint) WMF Analytics team does not have access to stats.grok.se [17:23:04] 10Analytics: Meta-statistics on MediaWiki history reconstruction process - https://phabricator.wikimedia.org/T155507#2961974 (10Nuria) [17:23:07] 10Analytics: Add statistics and error logging to mediawiki history reconstruction job - https://phabricator.wikimedia.org/T155971#2961976 (10Nuria) [17:24:19]