[00:05:39] (03PS12) 10Mforns: Allow for custom transforms in DataFrameToDruid [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) [00:08:27] (03CR) 10Mforns: "@Ottomata I think this is what we discussed, tested and works!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [00:09:10] !log restarted Turnilo to clear deleted datasource [00:09:14] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [00:10:31] byeeeeeee! [00:56:00] 10Analytics, 10Growth-Team, 10Product-Analytics, 10Patch-For-Review: Add EditAttemptStep properties to the schema whitelist - https://phabricator.wikimedia.org/T208332 (10Nuria) @Neil_P._Quinn_WMF indeed, i do not see that table, looking into this. [00:56:13] 10Analytics, 10Analytics-Data-Quality, 10Growth-Team, 10Product-Analytics, 10Patch-For-Review: Add EditAttemptStep properties to the schema whitelist - https://phabricator.wikimedia.org/T208332 (10Nuria) [04:43:56] (03CR) 10Ottomata: "Looks great! one nit one q :)" (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [09:17:54] joal: o/ - as FYI I am rolling restart cassandra on aqs for jvm upgrades [10:33:33] (03CR) 10Gilles: Add ServerTiming to EventLogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476841 (https://phabricator.wikimedia.org/T207862) (owner: 10Gilles) [11:58:15] (03CR) 10Addshore: [C: 03+2] Add build for deployment [analytics/wmde/toolkit-analyzer-build] - 10https://gerrit.wikimedia.org/r/480036 (https://phabricator.wikimedia.org/T209399) (owner: 10Michael Große) [11:58:21] (03Merged) 10jenkins-bot: Add build for deployment [analytics/wmde/toolkit-analyzer-build] - 10https://gerrit.wikimedia.org/r/480036 (https://phabricator.wikimedia.org/T209399) (owner: 10Michael Große) [12:01:07] joal: very interesting read http://johnjianfang.blogspot.com/2015/02/quorum-journal-manager-part-i-protocol.html [12:04:01] I was reading https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration#Adding_a_new_JournalNode_to_a_running_HA_Hadoop_Cluster and I was puzzled by the fact that in theory journal node should not be restarted when you add them [12:04:17] but only restart the namenodes [12:04:47] and this now makes sense - it is the QJM on the NameNode that does everything, IIUC the journalnodes are not aware of each other [12:05:00] (netstat also confirms that, I don't see TCP conns between them) [12:08:15] so in theory we could add/remove the nodes without HDFS Safe Mode [12:08:28] but I am still a bit reluctant :D [12:29:09] Wow - Thanks for the explanations elukey! I am completly ignorant around journal nodes :) [12:49:08] joal: maybe later on we can try adding two nodes? [12:49:20] Andrew should be around in ~1h, I'll also wait for him [12:49:32] the next steps are not hard but take time [12:49:37] 1) add the new journal nodes [12:49:43] 2) remove 1028/1035 [12:49:51] 3) slowly decom 14 worker nodes [12:49:59] elukey: Why not, but today is kids day and with a special afternoon for Lino making me miss standup - It'll need to be late for me [12:50:15] joal: ah! All right then, I'll break the cluster with Andrew :D [12:50:23] Have fun ;_) [12:55:09] Just for fun - `SELECT event_entity, COUNT(1) from mediawiki_history where snapshot = '2018-11' group by event_entity;` in hive and presto in cloud-analytics [12:55:29] Hive - 298 seconds ... Preso - 21 seconds :D [12:55:38] Mwahaha [13:01:56] o/ [13:02:32] does eventcapsule for eventlogging provide a way to get back to the actual request that the event came from / web request that loaded the page [13:02:35] * addshore guesses not [13:08:58] not that I know, buuut let's ask to more expert people :) [13:27:13] First tests with presto: it works great but could do with some optimizations, the simpler ones being on the dataset formatting [13:46:49] Ok I think I have understood the problem about comments - Related to old-comment field not being nullified event if comment_id is present [13:51:11] 10Analytics: Clean up home dirs for users jamesur and nithum - https://phabricator.wikimedia.org/T212127 (10elukey) Since this is the first time we do this, let's also document the process in https://wikitech.wikimedia.org/wiki/Analytics/Team/Oncall before closing the task. [14:43:34] ottomata: o/ [14:44:19] sooo journal nodes - today I've documented a bit and I am almost convinced that we could proceed without safe mode on [14:45:03] in theory, IIUC, each journal node doesn't care about the rest of the cluster, since it is the QueryJournalManager on the HDFS namenode that does everything [14:45:31] I was convinced, for example, that the journal nodes would have needed a restart once their cluster config was changed [14:45:45] but IIUC the namenode is the only one that we care about [14:45:59] for some reason I am still scared about these maintenance ops [14:49:57] o/ [14:50:18] elukey: that makes sense! [14:51:29] ottomata: if you have time we can do it after the meetings? or before ops sync (I have a meeting in a bit) [14:52:03] oo i have lots of meetings...until 13:15 here [14:52:08] 1:15 after our standup [14:52:12] so i could do it then! [14:52:26] acutally i'd need some lunch then, so maybe more like 1:45 after standup starts [14:52:27] ? [14:53:58] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: [BUG] userAgent missing from all EventLogging analytics Hive tables between 2018-11-29 and 2018-11-14 - https://phabricator.wikimedia.org/T211833 (10Ottomata) @chelsyx the useragent data should be back. Can you confirm? [14:55:00] ottomata: then we have grooming I think.. after it? [14:55:32] hm, no grooming today i think [14:55:33] no? [14:58:13] ah yes tomorrow! [14:58:14] nice :) [14:58:28] all right hten [15:02:42] 10Analytics, 10Operations, 10Performance-Team, 10Traffic: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) After brainstorming this more, since Nginx TLS termination is going to remain for the foreseeable future, even after we move backe... [15:03:16] 10Analytics, 10Operations, 10Performance-Team, 10Traffic: Only serve debug HTTP headers when x-wikimedia-debug is present - https://phabricator.wikimedia.org/T210484 (10Gilles) p:05Normal→03Low [15:24:05] 10Analytics, 10Analytics-Data-Quality, 10Contributors-Analysis, 10Product-Analytics: mediawiki_history missing page events - https://phabricator.wikimedia.org/T205594 (10Milimetric) @Neil_P._Quinn_WMF this is something we wanted to work on this quarter, but it was derailed by the actor/comment refactor. I... [15:24:11] elukey: should I override linter here? https://gerrit.wikimedia.org/r/c/operations/puppet/+/480759 or what's your pref [15:24:24] note using package_require did not trigger linter but using package{absent} does [15:24:27] so that's a fun thing [15:27:58] 10Analytics, 10Contributors-Analysis, 10Product-Analytics, 10Tracking: Product Analytics Data Lake needs - https://phabricator.wikimedia.org/T212172 (10elukey) [15:35:02] elukey: I'm going to go w/ what moritz suggested there in stead [15:35:30] 10Analytics, 10Contributors-Analysis, 10Product-Analytics, 10Tracking: Support all Product Analytics data needs in the Data Lake - https://phabricator.wikimedia.org/T212172 (10Neil_P._Quinn_WMF) [15:44:46] (03CR) 10Nuria: [C: 03+2] Add ServerTiming to EventLogging whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476841 (https://phabricator.wikimedia.org/T207862) (owner: 10Gilles) [15:45:03] chasemp: sorry was in a meeting! [15:45:33] totally fine with anything, it is just temporary then we'll clean up the code [15:49:40] ottomata: qq - the feedback from https://gerrit.wikimedia.org/r/#/c/operations/puppet/cdh/+/480433/ seems to be that a kerberos::exec would be preferred, but then I am wondering if we should keep cdh as separate module or not? [15:52:56] elukey: ehhhh [15:53:08] i think even if we moved it to ops/puppet repo [15:53:15] we'd break some code guidelines about modules using modules, no? [15:56:09] 10Analytics, 10Analytics-Data-Quality, 10Growth-Team, 10Product-Analytics, 10Patch-For-Review: Add EditAttemptStep properties to the schema whitelist - https://phabricator.wikimedia.org/T208332 (10Nuria) Ok, whitelist has a typo, that's why: edit-attempt-step should be edit-attempt-S-step As I said you... [15:57:11] ottomata: yep [15:57:23] this is why I preferred the cdh::exec [15:57:28] seemed more self contained [15:58:01] in theory we could try cdh::exec first, test it and see how it goes [15:58:55] elukey: i'd prefer to just stick with this cdh::exec, my comments were mostly aesthetic brain dump :p [15:59:09] moving it outside of cdh module is too muich, at least for now [16:03:17] all right :) [16:03:27] so IIUC this is +1 from you? [16:19:17] yup! [16:19:21] lemme put it on there [16:28:32] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: [BUG] userAgent missing from all EventLogging analytics Hive tables between 2018-11-29 and 2018-11-14 - https://phabricator.wikimedia.org/T211833 (10mforns) @Ottomata Coool, will do. [16:36:00] 10Analytics, 10Analytics-Data-Quality, 10Growth-Team, 10Product-Analytics, 10Patch-For-Review: Add EditAttemptStep properties to the schema whitelist - https://phabricator.wikimedia.org/T208332 (10Neil_P._Quinn_WMF) >>! In T208332#4834514, @Nuria wrote: > Ok, whitelist has a typo, that's why: edit-attemp... [16:42:38] 10Analytics, 10Analytics-Kanban, 10Datasets-General-or-Unknown, 10Patch-For-Review: cron job rsyncing dumps webserver logs to stat1005 is broken - https://phabricator.wikimedia.org/T211330 (10Ottomata) Ya is fine with me! [16:54:24] (03CR) 10HaeB: [V: 03+2 C: 03+2] "Thank you!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/480472 (https://phabricator.wikimedia.org/T209051) (owner: 10Fdans) [16:56:13] (03CR) 10HaeB: [V: 03+2 C: 03+2] "Thank you!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/480471 (https://phabricator.wikimedia.org/T209050) (owner: 10Fdans) [16:58:30] (03CR) 10HaeB: [V: 03+2 C: 03+2] "Thank you!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/480470 (https://phabricator.wikimedia.org/T209049) (owner: 10Fdans) [17:00:50] nuria: I’ll be a couple minutes late [17:00:59] long line for the bathroom [17:01:01] :) [17:01:09] milimetric: k [17:01:39] milimetric: classic [17:01:43] oh damn it's 6:01 [17:01:45] ping fdans mforns [17:10:48] (03PS8) 10Joal: Join to new actor and comment tables [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476553 (https://phabricator.wikimedia.org/T210543) (owner: 10Milimetric) [17:10:50] (03PS1) 10Joal: Update mediawiki-history comment and actor joins [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/480796 (https://phabricator.wikimedia.org/T210543) [17:36:02] nuria kzeta : having a browser issue with hangouts, should be fixed in a minute [17:37:52] 10Analytics, 10Analytics-Data-Quality, 10Growth-Team, 10Product-Analytics, 10Patch-For-Review: Add EditAttemptStep properties to the schema whitelist - https://phabricator.wikimedia.org/T208332 (10Nuria) Sorry, corrected comment but still my mistake, i saw the newest whitelist is deployed to stat1007:... [17:49:01] * elukey off! [18:01:30] 10Analytics, 10Analytics-Data-Quality, 10Contributors-Analysis, 10Product-Analytics: mediawiki_history missing page events - https://phabricator.wikimedia.org/T205594 (10Milimetric) Interesting findings so far. @fdans and I found https://www.mediawiki.org/wiki/Manual:Log_search_table which means logging a... [18:05:28] 10Analytics, 10Analytics-Kanban, 10Readers-Web-Backlog, 10Patch-For-Review: Print schema is whitelisting both session ids and page ids - https://phabricator.wikimedia.org/T209050 (10Nuria) [18:05:52] 10Analytics, 10Analytics-Kanban: Remove sessionId, pageId pairs from whitelist - https://phabricator.wikimedia.org/T205458 (10Nuria) [18:33:26] milimetric, fdans : can we have a meeting with joal to coordinate who is doing what in quality so we do not duplicate efforts (meeting can be early morning east coast) but let's make sure to coordinate [18:47:35] (03PS2) 10Joal: Update mediawiki-history comment and actor joins [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/480796 (https://phabricator.wikimedia.org/T210543) [18:47:50] milimetric, fdans - Shall we meet tomorrow after lunch CEST time? [18:48:21] joal: yea [18:48:27] I'll be on [18:51:50] milimetric, fdans , joal sounds good, you can let me know in standup tommorow, super thanks [18:52:17] (03CR) 10Mforns: Allow for custom transforms in DataFrameToDruid (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [19:16:52] (03CR) 10Ottomata: Allow for custom transforms in DataFrameToDruid (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [19:26:40] o/ which repo would include Spark jobs? [19:27:06] I'm trying to write a job to transform one Hadoop table into another, and looking for sample code to start with. [19:33:32] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Core Platform Team Backlog (Watching / External), 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation: Deployment Pipeline - https://phabricator.wikimedia.org/T211247 (10Pchelolo) One thing that came up during a meeting... [19:34:41] I can probably do this with a single query, to tell the truth. I'll start with that. [19:50:48] awight, the repo that holds Spark jobs is refinery-source: https://github.com/wikimedia/analytics-refinery-source [19:51:02] mforns: Thanks! [19:51:46] awight, np! [19:52:43] Brace yourself for reviewing an amateur Oozie job :-) [19:58:36] heh, I think we all (except for joal) write amateur oozie jobs [20:01:47] (03CR) 10Mforns: Allow for custom transforms in DataFrameToDruid (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [20:06:46] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10hashar) p:05Tria... [20:08:30] 10Analytics, 10Analytics-Kanban, 10DBA, 10Data-Services, and 3 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10MusikAnimal) If you need a use case, https://xtools.wmflabs.org/autoedits in particular used to be //much// fast... [20:13:32] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10hashar) ` php > p... [20:23:02] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10hashar) Poked @dc... [20:30:44] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10hashar) p:05Unbr... [20:31:11] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10hashar) [20:31:23] (03CR) 10Ottomata: Allow for custom transforms in DataFrameToDruid (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/477295 (https://phabricator.wikimedia.org/T210099) (owner: 10Mforns) [20:38:48] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10dcausse) wiki: me... [20:40:26] 10Analytics, 10ORES, 10Scoring-platform-team: Wire ORES scoring events into Hadoop - https://phabricator.wikimedia.org/T209732 (10JAllemandou) If I understand correctly, you're willing to process ORES events coming from EventLogging and explode them by model - Am I nearly Correct @awight ? [20:44:19] ottomata, thanks for the comments, I'm wondering if sth like this would prune?: WHERE year=2018 AND month=12 OR year=2019 AND month=1 [20:47:52] that would prune ya [20:47:58] mforns: ^ [20:48:24] you might need to paren them [20:48:25] but ya [20:51:35] ok, so I guess I can do some fancy WHERE clause that can solve that [20:52:36] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10Smalyshev) Looks... [20:53:36] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10Smalyshev) [20:54:13] 10Analytics, 10CirrusSearch, 10Discovery-Search, 10EventBus, 10Wikimedia-production-error: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10Smalyshev) [21:06:02] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Support all Product Analytics data needs in the Data Lake - https://phabricator.wikimedia.org/T212172 (10Neil_P._Quinn_WMF) [21:28:40] 10Analytics, 10ORES, 10Scoring-platform-team: Wire ORES scoring events into Hadoop - https://phabricator.wikimedia.org/T209732 (10awight) Bookmark to self—this query splits out the mediawiki_revision_score event by model: ` lang=sql select `database`, page_id, page_namespace, page_title, rev_id,... [21:29:21] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Support all Product Analytics data needs in the Data Lake - https://phabricator.wikimedia.org/T212172 (10Neil_P._Quinn_WMF) [21:29:23] 10Analytics, 10Product-Analytics: Sqoop more tables for mediawiki in monthly schedule - https://phabricator.wikimedia.org/T198983 (10Neil_P._Quinn_WMF) [21:38:22] 10Analytics, 10ORES, 10Scoring-platform-team: Wire ORES scoring events into Hadoop - https://phabricator.wikimedia.org/T209732 (10awight) >>! In T209732#4835444, @JAllemandou wrote: > If I understand correctly, you're willing to process ORES events coming from EventLogging and explode them by model - Am I ne... [21:44:11] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Support all Product Analytics data needs in the Data Lake - https://phabricator.wikimedia.org/T212172 (10Neil_P._Quinn_WMF) [21:59:14] ottomata, would it be interesting to have a function somewhere in refinery-source (refinery-hive maybe?) that given a since and an until, returns a WHERE clause condition that encloses those using year[, month[, day[, hour]]] partitions? [22:16:55] (03CR) 10Nuria: "While I understand the problem I cannot say I understand the solution." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/480796 (https://phabricator.wikimedia.org/T210543) (owner: 10Joal) [22:22:35] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Core Platform Team Backlog (Watching / External), 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation: Deployment Pipeline - https://phabricator.wikimedia.org/T211247 (10Nuria) Having two "intake services" running makes... [22:28:02] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Start refining all blacklisted EventLogging streams - https://phabricator.wikimedia.org/T212355 (10Neil_P._Quinn_WMF) p:05Triage→03Normal [22:30:05] 10Analytics, 10Analytics-Data-Quality, 10Contributors-Analysis, 10Product-Analytics, 10Growth-Team (Current Sprint): Resume refinement of edit events in Data Lake - https://phabricator.wikimedia.org/T202348 (10Neil_P._Quinn_WMF) [22:30:07] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Start refining all blacklisted EventLogging streams - https://phabricator.wikimedia.org/T212355 (10Neil_P._Quinn_WMF) [22:33:37] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Core Platform Team Backlog (Watching / External), 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation: Deployment Pipeline - https://phabricator.wikimedia.org/T211247 (10Pchelolo) > I assume they will be behind varnish s... [22:50:29] musikanimal : yt? [22:50:38] yo [22:54:25] musikanimal: i was thinking [22:54:45] musikanimal: that we could write a blogpost about fake top views in wikipedia [22:55:45] musikanimal: as in pages that are on our top list but we know are not top for non obvious reasons, our favorite page is a good exmaple cause ( i need to look at this again) there traffic is organic but non intentional [22:56:13] sure! I have a long list of other false positives that I can share [22:56:23] all those of "List of comedians" are fake, apparently [22:56:30] on enwiki [23:02:07] musikanimal: we can do it if you want , we can split the list and do some research and make a short post , it is kind of interesting cause what motivates teh false positives (seems to me) is different in each case [23:02:28] yeah I'm sure they are all different [23:04:19] here is a list of all the pages blacklisted from Topviews: [23:04:28] https://www.irccloud.com/pastebin/GZpN871U/ [23:05:21] then I have about 800 or so pages that are hidden from Topviews, but only for specific user agents/time periods [23:05:28] 10Analytics, 10Analytics-Kanban, 10DBA, 10Data-Services, and 3 others: Create materialized views on Wiki Replica hosts for better query performance - https://phabricator.wikimedia.org/T210693 (10Nuria) @MusikAnimal note that on the proposed scheme these views are not real time though, they are recreated mo... [23:07:06] musikanimal: ayayaya what is up with maximilian! [23:07:55] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Start refining ChangesListHighlights events - https://phabricator.wikimedia.org/T212367 (10Neil_P._Quinn_WMF) p:05Triage→03Normal [23:09:40] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Start refining InputDeviceDynamics events - https://phabricator.wikimedia.org/T212368 (10Neil_P._Quinn_WMF) p:05Triage→03Normal [23:10:34] 10Analytics, 10Analytics-Kanban, 10Fundraising-Backlog, 10User-Elukey: Return to real time banner impressions in Druid - https://phabricator.wikimedia.org/T203669 (10Nuria) Per our conversation in standup we are going to kill the job that imports from kafka directly and use eventlogging to druid, there wer... [23:22:38] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Start refining all blacklisted EventLogging streams - https://phabricator.wikimedia.org/T212355 (10Neil_P._Quinn_WMF) [23:22:44] 10Analytics, 10Analytics-Data-Quality, 10Contributors-Analysis, 10Product-Analytics, 10Growth-Team (Current Sprint): Resume refinement of edit events in Data Lake - https://phabricator.wikimedia.org/T202348 (10Neil_P._Quinn_WMF) 05Open→03Resolved Looks like this is all taken care of! ` select to... [23:46:31] 10Analytics, 10Research, 10WMDE-Analytics-Engineering, 10User-Addshore, 10User-Elukey: Phase out and replace analytics-store (multisource) - https://phabricator.wikimedia.org/T172410 (10Neil_P._Quinn_WMF) [23:47:26] This is strange. I'm trying to follow git-fat instructions for analytics-refinery and I get: [23:47:26] 10Analytics, 10Research, 10WMDE-Analytics-Engineering, 10User-Addshore, 10User-Elukey: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10Neil_P._Quinn_WMF) [23:47:29] > ERROR:git-fat: Error reading or parsing configfile: /Users/awight/work/analytics-refinery/.gitfat [23:47:56] git fat --version -> 0.5.0