[00:11:28] good morning! [00:15:46] Happy 6:15 PM MDT to you [00:41:17] 10Analytics, 10Product-Analytics, 10Growth-Team (Current Sprint): Homepage: specify purging strategy - https://phabricator.wikimedia.org/T219252 (10MMiller_WMF) [01:01:09] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10Tbayer) For the record: Information about how to work with the new setup was added to https://wikitech.wikimedia.org... [01:06:40] 10Analytics, 10MediaWiki-extensions-GrowthExperiments, 10Product-Analytics, 10Growth-Team (Current Sprint): Homepage: instrumentation - https://phabricator.wikimedia.org/T216586 (10MMiller_WMF) [01:21:02] fdans: hello [01:22:03] nuria: o/ [01:23:04] fdans: I did had time to CR your changes [01:23:11] * i did have [01:23:14] sorry [01:23:58] fdans: but one of the patches does not load at all for me after rebasing. [01:24:27] nuria: I broadly understand you are working on a proposal for a ML pipeline. As you make progress on this work I am interested in being apprised of the details, particularly as it relates to Scoring Platform's work. [01:24:35] nuria: yeah I was going through it now [01:24:51] I think we share a vision of converging our infrastructure more so I'm excited to see what comes of it. [01:26:35] hare: i can share current state of doc but it is very rough, one sec [01:28:21] Thank you! [02:02:27] nuria: the change works fine for me after your rebase, but I do need to correct a couple things you mention [02:06:36] (03CR) 10Fdans: [C: 04-1] "@nuria, I'm sure you've figured it out yourself by now, but this change has nothing to do with the all metrics one. I'll apply the comment" (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/494241 (https://phabricator.wikimedia.org/T187806) (owner: 10Fdans) [04:06:35] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10Pginer-WMF) [04:07:36] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10Pginer-WMF) 05Open→03Resolved Thanks for the extra details, @chelsyx and @dr0... [05:24:06] (03PS3) 10Fdans: Create metrics matrix component [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/498748 (https://phabricator.wikimedia.org/T187806) [05:32:14] (03CR) 10Fdans: [C: 04-1] "@Milimetric just created the two new metrics' meta-wiki page, so you can add them too in this change." (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/494241 (https://phabricator.wikimedia.org/T187806) (owner: 10Fdans) [06:59:22] 10Analytics, 10Performance-Team: ServerTiming EventLogging data getting nuked after 90 days despite whitelist - https://phabricator.wikimedia.org/T219262 (10Gilles) [06:59:29] 10Analytics, 10Performance-Team: ServerTiming EventLogging data getting nuked after 90 days despite whitelist - https://phabricator.wikimedia.org/T219262 (10Gilles) p:05Triage→03High [07:08:06] (03PS1) 10Gilles: Add CpuBenchmark to EventLogging whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) [08:11:30] Morning team [08:11:42] sqoop with 10 workers failed because of opne group [08:20:19] elukey: hello :) [08:20:45] elukey: would you mind confirming this cpu chart looks ok for us? https://grafana.wikimedia.org/d/000000377/host-overview?refresh=5m&panelId=3&fullscreen&orgId=1&var-server=an-coord1001&var-datasource=eqiad%20prometheus%2Fops&var-cluster=analytics&from=now-2d&to=now [08:27:10] bonjour! [08:27:10] opne group? [08:27:10] * elukey ignorant [08:27:10] joal: cpu usage looks acceptable, load avg is a bit high but there was a lot of I/O involved [08:27:10] it is only once a month for some hours so it should be acceptable [08:27:39] one group sorry - typo :S [08:28:38] elukey: we group wikis together based on their sizes 0 And one group (number 11 to be precise), contains too many wikis in relation to their sizes [08:28:46] Currently testing again having split the group in tow [08:29:10] also elukey - I think the solution we devised originaly using groups is not good now that we have the new host [08:29:46] I'd like the wiki list to provide the number of mappers to use for a given wiki, not a "group" [08:30:18] This would actually allow us to refresh the group-data (2 years old) - wikis have grown since that time :) [08:34:36] makes sense yes [08:52:38] (03CR) 10Joal: "> Q: will we really be able to use the fields as they are if we upgrade Hive and ALTER TABLES?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/497604 (https://phabricator.wikimedia.org/T218463) (owner: 10Joal) [09:01:34] (03PS3) 10Joal: Correct names in mediawiki-history sql package [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498861 [09:01:53] (03CR) 10Joal: Correct names in mediawiki-history sql package (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498861 (owner: 10Joal) [10:13:07] (03PS15) 10Elukey: Add artifacts for Debian Buster and upgrade to 0.31.0rc18-wikimedia1 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/495182 (https://phabricator.wikimedia.org/T212243) [11:00:03] 10Analytics, 10Operations, 10Product-Analytics, 10Patch-For-Review, 10User-Elukey: notebook/stat server(s) running out of memory - https://phabricator.wikimedia.org/T212824 (10aborrero) >>! In T212824#4967798, @elukey wrote: > Today I checked notebook1003 using the command `systemd-cgls memory`, that sho... [11:06:03] (03PS1) 10Mforns: Add alert for decommissioning of Wikimetrics [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/499168 (https://phabricator.wikimedia.org/T211835) [11:07:50] (03CR) 10Mforns: [V: 03+2 C: 03+2] "Self-merging, just documentation." [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/499168 (https://phabricator.wikimedia.org/T211835) (owner: 10Mforns) [11:16:14] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [12:14:22] (03CR) 10Bmansurov: WIP: Oozie: add article recommender (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) (owner: 10Bmansurov) [12:40:11] * elukey lunch! [12:56:46] hey elukey, do you have 10 mins to gimme some advice on wikimetrics opsy stuff? [12:57:03] oh!, when you' re back, of course [13:06:01] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [13:25:12] (03CR) 10Ottomata: "> I have not found `ts` fields in other places" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/497604 (https://phabricator.wikimedia.org/T218463) (owner: 10Joal) [13:25:44] (03CR) 10Ottomata: "> I don't mind going for _ts instead of longer _timestamp, but probably in another patch" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/497604 (https://phabricator.wikimedia.org/T218463) (owner: 10Joal) [13:30:47] 10Analytics, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Ottomata) In https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/497604/, Joal and I were considerin... [13:32:22] 10Analytics, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), 10Patch-For-Review: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Ottomata) [13:45:36] 10Analytics, 10EventBus, 10WMF-JobQueue, 10Core Platform Team Backlog (Later), and 2 others: Partition htmlCacheUpdate job topic - https://phabricator.wikimedia.org/T219159 (10Pchelolo) Actually, the existing topic need to be left alone, but 2 new topics 8 partitions each needs to be created: - `eqiad.cha... [13:50:54] 10Analytics, 10Analytics-Kanban, 10MediaWiki-Vagrant, 10Patch-For-Review: Vagrant initial provision fails on NodeJS version mismatch - https://phabricator.wikimedia.org/T218238 (10Ottomata) I've merged a patch that should avoid this problem. Let me know if it works for yall or not. Thanks! [13:52:08] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: Resurrect eventlogging_EventError logging to in logstash - https://phabricator.wikimedia.org/T205437 (10Ottomata) 05Open→03Resolved [13:58:11] 10Analytics, 10Analytics-Wikimetrics, 10I18n: Add i18n support to Wikimetrics - https://phabricator.wikimedia.org/T60634 (10mforns) 05Open→03Declined Wikimetrics is being discontinued. See: T211835. [13:58:38] (03PS1) 10Ladsgroup: Remove Google+ [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 [13:58:49] 10Analytics, 10Analytics-Wikimetrics: Nosetests / codecoverage packages missing from wikimetrics' setup - https://phabricator.wikimedia.org/T65002 (10mforns) 05Open→03Declined Wikimetrics is being discontinued. See: T211835. [14:00:53] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: WikimetricsUser monitors health of Wikimetrics in graphite - https://phabricator.wikimedia.org/T74138 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:01:50] 10Analytics, 10Analytics-Wikimetrics: Wikimetrics backup has no monitoring - https://phabricator.wikimedia.org/T71397 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:05:29] 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review: Wikimetrics docker build/test environment is broken - https://phabricator.wikimedia.org/T193780 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:05:51] 10Analytics, 10Analytics-Wikimetrics: Wikimetrics deploy is broken - https://phabricator.wikimedia.org/T193783 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:06:42] 10Analytics, 10Analytics-Wikimetrics: Wikimetrics backup should warn about symlinks pointing out of /var/lib/wikimetrics - https://phabricator.wikimedia.org/T69597 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:07:04] 10Analytics, 10Analytics-Wikimetrics: Wikimetrics needs to read config files /etc/wikimetrics/ rather that files on local repository by default when setting up the config - https://phabricator.wikimedia.org/T69542 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:07:11] 10Analytics, 10Analytics-Wikimetrics, 10Malayalam-Sites: Wikimetrics is not supporting mlwiki cohort - https://phabricator.wikimedia.org/T71462 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:07:18] (03CR) 10Ottomata: WIP: Oozie: add article recommender (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) (owner: 10Bmansurov) [14:07:24] 10Analytics, 10Analytics-Wikimetrics: Integrate wikimetrics with mediawiki-utilities - https://phabricator.wikimedia.org/T65203 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:07:39] 10Analytics, 10Analytics-Wikimetrics, 10MediaWiki-Vagrant: Wikimetrics-web startup fails on Ubuntu 14.04 MediaWiki-Vagarnt due to _winreg load failure - https://phabricator.wikimedia.org/T90017 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:07:53] 10Analytics, 10Analytics-Wikimetrics: Troubleshoot Wikimetrics RAE reports - https://phabricator.wikimedia.org/T93217 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:08:28] 10Analytics, 10Analytics-Wikimetrics, 10MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:09:32] 10Analytics, 10Analytics-Wikimetrics: wikimetrics puppet setup should not give rights to the server, only the wikimetrics database - https://phabricator.wikimedia.org/T65379 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:09:43] 10Analytics, 10Analytics-Wikimetrics: Wikimetrics queue should be restarted in development when files change - https://phabricator.wikimedia.org/T75671 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:09:59] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Can't edit list of users - https://phabricator.wikimedia.org/T67208 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:10:19] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Description of metrics includes link to on-wiki metrics documentation - https://phabricator.wikimedia.org/T93659 (10mforns) [14:10:28] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Plain language definitions of Wikimetrics metrics - https://phabricator.wikimedia.org/T93685 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:11:22] 10Analytics, 10Analytics-Wikimetrics: Make mechanism to deactivate recurrent reports in wikimetrics - https://phabricator.wikimedia.org/T74279 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:11:34] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Excessive failure messages trying to create cohorts and run messages on Wikimetrics - https://phabricator.wikimedia.org/T87235 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:11:44] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Uploading cohort or running a large report fails - https://phabricator.wikimedia.org/T87596 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:11:55] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: WikimetricsUser is able to chain metrics - https://phabricator.wikimedia.org/T73614 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:12:10] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10kchapman) [14:12:30] 10Analytics, 10Analytics-Wikimetrics: Cohort listing going against the database instead of using CohortService - https://phabricator.wikimedia.org/T68126 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:12:56] 10Analytics, 10ChangeProp, 10Community-Tech, 10Core Platform Team Backlog, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10kchapman) [14:13:07] 10Analytics, 10Analytics-Wikimetrics, 10Epic: Epic: AnalyticsEng has robust code to backfill metrics data - https://phabricator.wikimedia.org/T71252 (10mforns) [14:13:11] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: AnalyticsEng uses TimeSeries support to backfill data - https://phabricator.wikimedia.org/T71253 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:13:15] 10Analytics, 10Analytics-Wikimetrics: Add csrf token to all requests - https://phabricator.wikimedia.org/T68482 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:13:31] 10Analytics, 10ChangeProp, 10Community-Tech, 10Core Platform Team Backlog, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10EvanProdromou) Is this something that could be managed with k8s? https://kubernetes.io/docs/t... [14:13:47] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10kchapman) [14:15:09] (03CR) 10Lucas Werkmeister (WMDE): Remove Google+ (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 (owner: 10Ladsgroup) [14:15:23] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Make Wikimedia project titles more user-friendly - https://phabricator.wikimedia.org/T100692 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:15:46] 10Analytics, 10Analytics-Wikimetrics: Huge reports that could clog Wikimetrics may happen accidentally, add a warning - https://phabricator.wikimedia.org/T60754 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:15:56] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story:d WikimetricsUser runs 'Rolling Recurring old active editors' report - https://phabricator.wikimedia.org/T71569 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:16:09] 10Analytics, 10Analytics-Wikimetrics: delete useless wikimetrics.report or cohort records - https://phabricator.wikimedia.org/T120713 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:16:22] 10Analytics, 10Analytics-Visualization: Wikimetrics: Metric definition links are invisible - https://phabricator.wikimedia.org/T73568 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:16:29] (03PS2) 10Ladsgroup: Remove Google+ [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 [14:16:54] 10Analytics, 10Analytics-Wikimetrics, 10Epic: Epic: AnalyticsEng has robust code to backfill metrics data - https://phabricator.wikimedia.org/T71252 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:17:02] 10Analytics, 10Analytics-Wikimetrics: Some special characters break Wikimetrics' encoding {dove} - https://phabricator.wikimedia.org/T114884 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:17:19] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: WikimetricsUser generates report with Target Site breakdown - https://phabricator.wikimedia.org/T70475 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:18:02] (03CR) 10Ladsgroup: Remove Google+ (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 (owner: 10Ladsgroup) [14:18:33] hey... connection hiccup [14:18:41] heya elukey :] [14:18:59] :) [14:19:16] do you have 10 mins in da caif for qqs? [14:20:22] 10Analytics, 10Analytics-Wikimetrics: General javascript infrastructure cleanup - https://phabricator.wikimedia.org/T68484 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:21:12] (03CR) 10Lucas Werkmeister (WMDE): [C: 03+2] "Are" (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 (owner: 10Ladsgroup) [14:21:41] (03Merged) 10jenkins-bot: Remove Google+ [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/499198 (owner: 10Ladsgroup) [14:22:59] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: VSUser has data from 2008 to now - https://phabricator.wikimedia.org/T74133 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:23:54] 10Analytics, 10Analytics-Wikimetrics: queries in wikimetrics are not using the db indices optimally - https://phabricator.wikimedia.org/T65201 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:24:10] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: EEVSuser has agregate metrics - https://phabricator.wikimedia.org/T70193 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:24:23] 10Analytics, 10Analytics-Wikimetrics: Bot to call global metrics to event page {kudu} - https://phabricator.wikimedia.org/T120330 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:24:31] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics, 10Story: Story: Wikimetrics user views full list of cohort members - https://phabricator.wikimedia.org/T86335 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:25:07] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Inconsistent metrics results for 90 day rolling active editors calculation - https://phabricator.wikimedia.org/T147176 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:25:24] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Description of metrics includes link to on-wiki metrics documentation - https://phabricator.wikimedia.org/T93659 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:27:00] 10Analytics, 10Analytics-Wikimetrics: C_FORCE_ROOT is bad, change pickle as serialization format for celery - https://phabricator.wikimedia.org/T62289 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:27:13] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Ability to change bytes to characters - https://phabricator.wikimedia.org/T99962 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:28:24] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10aezell) @EvanProdromou I think that a solution like that is something closer to what @Joe was proposing than th... [14:28:38] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics, 10Story: Story: WikimetricsUser searches for cohort (filters) using tag name - https://phabricator.wikimedia.org/T75071 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:29:15] 10Analytics, 10Analytics-Wikimetrics: Run a spellchecker - https://phabricator.wikimedia.org/T60635 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:29:43] 10Analytics, 10Analytics-Wikimetrics, 10Epic: Epic: WikimetricsUser shares a cohort - https://phabricator.wikimedia.org/T69461 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:30:09] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: AnalyticsEng has Execution Pipeline to optimize/create resusable code for generating metrics - https://phabricator.wikimedia.org/T69501 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:30:42] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: AnalyticsEng has Execution Pipeline to optimize/create resusable code for generating metrics - https://phabricator.wikimedia.org/T71300 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:31:32] 10Analytics, 10Analytics-Wikimetrics: Include Tulu Wikipedia in Metrics and Quarry - https://phabricator.wikimedia.org/T148950 (10mforns) Note that Wikimetrics has been discontinued. See: T211835. [14:31:56] 10Analytics: Include Tulu Wikipedia in Quarry - https://phabricator.wikimedia.org/T148950 (10mforns) [14:32:24] 10Analytics, 10Analytics-Wikimetrics: Pick Metrics > Edits and others force to choose namespaces - https://phabricator.wikimedia.org/T60630 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:33:11] 10Analytics, 10Analytics-Wikimetrics, 10Epic: Epic: AnalyticsEng has robust code to backfill metrics data - https://phabricator.wikimedia.org/T71252 (10mforns) [14:33:14] 10Analytics, 10Analytics-Sprint-2014-11-13, 10Analytics-Wikimetrics, 10Story: Story: AnalyticsEng has editor_day table in labsdb - https://phabricator.wikimedia.org/T71145 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:33:32] 10Analytics, 10Analytics-Wikimetrics: Separate concurrent report queue and regular 'default' queue - https://phabricator.wikimedia.org/T71523 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:33:56] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics, 10Story: Story: Wikimetrics User uses tooltip to view all projects for an expanded cohort member - https://phabricator.wikimedia.org/T86332 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:34:23] 10Analytics, 10Analytics-Wikimetrics: Date and time revert when creating a report - https://phabricator.wikimedia.org/T60609 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:34:40] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: [BUG] Viewing ukwiki cohorts error - https://phabricator.wikimedia.org/T95320 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:34:42] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10Ottomata) FWIW, Analytics will (eventually?) be considering replacing our scheduler (Oozie) with Apache Airflow... [14:34:55] 10Analytics, 10Analytics-Engineering, 10Analytics-Wikimetrics: Can't correct names in a cohort - https://phabricator.wikimedia.org/T130715 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:35:16] 10Analytics, 10Analytics-Wikimetrics: Running tests requires to stop the queue - https://phabricator.wikimedia.org/T65001 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:35:35] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics, 10Need-volunteer, 10good first bug: "Create Report" button does not appear when uploading a new cohort - https://phabricator.wikimedia.org/T95456 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:36:59] 10Analytics, 10Analytics-Wikimetrics, 10Education-Program-Dashboard: I want WikiMetrics integration with the education dashboard that lets you easily pull reports about courses, institutions, etc. - https://phabricator.wikimedia.org/T92454 (10mforns) Just a heads up that Wikimetrics is being discontinued. Pl... [14:37:11] 10Analytics, 10Education-Program-Dashboard: I want WikiMetrics integration with the education dashboard that lets you easily pull reports about courses, institutions, etc. - https://phabricator.wikimedia.org/T92454 (10mforns) [14:37:23] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: Make timezone selector comprehensive and consistent - https://phabricator.wikimedia.org/T88604 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:37:38] 10Analytics, 10Analytics-Wikimetrics, 10good first bug: can't remove users from cohort in Iceweasel (aka Firefox, works fine in Chromium) - https://phabricator.wikimedia.org/T115160 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:37:50] 10Analytics, 10Analytics-Wikimetrics: Load test cohort uploading - https://phabricator.wikimedia.org/T69858 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:38:43] 10Analytics, 10Analytics-Wikimetrics: Puppet setup should not create wikimetrics_testing database in production - https://phabricator.wikimedia.org/T66295 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:38:53] mforns: sorry I missed your ping in the updates :D [14:39:00] sure! Do you want to bc now? [14:39:02] 10Analytics, 10Analytics-Wikimetrics: Pickle Serialization is hiding design flaws, change to JSON - https://phabricator.wikimedia.org/T63961 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:40:20] elukey, sure! omw [14:40:34] sorry for the big time spam [14:41:57] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics: some non-ascii characters do not show up in uploaded usernames and result in invalid usernames - https://phabricator.wikimedia.org/T93646 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:42:07] 10Analytics, 10Analytics-Wikimetrics, 10Community-Wikimetrics, 10Patch-For-Review, 10good first bug: User reads result of validation after creating a cohort - https://phabricator.wikimedia.org/T76914 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:42:40] 10Analytics, 10Analytics-Engineering, 10Analytics-Wikimetrics: Removing usernames from a cohort doesn't work - https://phabricator.wikimedia.org/T130714 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:42:51] mforns: bc doesn' work for me :( [14:43:13] 10Analytics, 10Analytics-Wikimetrics: Display global metrics report results on same page as report inputs {kudu} - https://phabricator.wikimedia.org/T121262 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:43:23] 10Analytics, 10Analytics-Wikimetrics: Central repository of global metrics reports {kudu} - https://phabricator.wikimedia.org/T121286 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:43:33] 10Analytics, 10Analytics-Wikimetrics, 10good first bug: Accented letters seem to be rejected in cohort names - https://phabricator.wikimedia.org/T111611 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:43:57] 10Analytics, 10Analytics-Wikimetrics: Recurrent runs for project cohorts should not block reports scheduled by users - https://phabricator.wikimedia.org/T71299 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:03] 10Analytics, 10Analytics-Wikimetrics: Deploy "rolling active editors" metric to production. - https://phabricator.wikimedia.org/T70447 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:10] 10Analytics, 10Analytics-Wikimetrics: Config override for queue.yaml from test.yaml is not working well - https://phabricator.wikimedia.org/T68770 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:18] 10Analytics, 10Analytics-Wikimetrics: configuration files are world readable - https://phabricator.wikimedia.org/T68730 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835.` [14:44:21] helloo milimetric you here? [14:44:29] 10Analytics, 10Analytics-Wikimetrics: Validation step on report creation is superfluous. We should remove it from the UI - https://phabricator.wikimedia.org/T68427 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:38] 10Analytics, 10Analytics-Wikimetrics: Alembic upgrade uses local config file db.yaml rather than the file deployed on /etc via puppet - https://phabricator.wikimedia.org/T66297 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:42] I’m here fdans [14:44:50] 10Analytics, 10Analytics-Wikimetrics: JSON output is not sorted - https://phabricator.wikimedia.org/T65920 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:44:58] milimetric: you want to catch up before standup? [14:44:58] welcome back! [14:45:01] sure [14:45:01] 10Analytics, 10Analytics-Wikimetrics: Limit what we serialize to celery via task.delay(...) - https://phabricator.wikimedia.org/T65737 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:45:05] omw [14:45:10] 10Analytics, 10Analytics-Wikimetrics: Notify on report completion - https://phabricator.wikimedia.org/T60632 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:46:03] sorry elukey I was in the hangout but forgot to click join [14:46:20] now it's busy, you want to go to a-batcave-2 ? [14:47:15] 10Analytics, 10Analytics-Wikimetrics: 1st run of recurrent report has empty resultset - https://phabricator.wikimedia.org/T73338 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:47:16] yeah! [14:47:25] 10Analytics, 10Analytics-Wikimetrics, 10Performance: Improve perf of rolling active editors with dba's suggestions - https://phabricator.wikimedia.org/T71610 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:47:33] 10Analytics, 10Analytics-Wikimetrics: Add MAILTO to backup crontab on puppet to be notified of failures - https://phabricator.wikimedia.org/T72853 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:47:43] 10Analytics, 10Analytics-Wikimetrics, 10Story: Story: AnalyticsEng uses connection pooling on database URL - https://phabricator.wikimedia.org/T73140 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:47:49] 10Analytics, 10Analytics-Wikimetrics: Tasks should clean up after themselves when they fail in STARTING phase - https://phabricator.wikimedia.org/T74563 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [14:51:14] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Operations, and 3 others: eventgate-analytics k8s pods occasionally can't produce to kafka - https://phabricator.wikimedia.org/T218268 (10akosiaris) >>! In T218268#5049090, @Ottomata wrote: > > `apt-get update` couldn't connect to the apt source IPv6 addresse... [14:53:42] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Operations, and 3 others: eventgate-analytics k8s pods occasionally can't produce to kafka - https://phabricator.wikimedia.org/T218268 (10Ottomata) OF COURSE! Thank you Alex. Since (IIUC), the preferred default is that all servers get IPv6 addies, we should... [14:54:38] (03CR) 10Nuria: [C: 03+2] Add alert for decommissioning of Wikimetrics [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/499168 (https://phabricator.wikimedia.org/T211835) (owner: 10Mforns) [15:08:51] 10Analytics, 10Analytics-Wikimetrics: Scheduling reports for the future - https://phabricator.wikimedia.org/T74119 (10mforns) 05Open→03Declined https://tools.wmflabs.org/nagf/?project=wikimetrics [15:09:19] 10Analytics, 10Analytics-Wikimetrics: slow report result and report list fetching - https://phabricator.wikimedia.org/T73603 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [15:09:30] 10Analytics, 10Analytics-Wikimetrics: Deal with scheduled recurrent reports when cohorts are deleted - https://phabricator.wikimedia.org/T70520 (10mforns) 05Open→03Declined Declining because Wikimetrics is being discontinued. See: T211835. [15:13:38] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10daniel) Wikidata also has a need for delayed and/or periodic tasks for change dispatching. That is currently ba... [15:13:54] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10daniel) [15:16:07] 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review: Wikimetrics docker build/test environment is broken - https://phabricator.wikimedia.org/T193780 (10sbassett) @mforns - Has Wikimetrics already been undeployed? It looks like it might have been, since metrics.wmflabs.org seems to point to a different... [15:29:08] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) These are the docs that have been added an alert to notify of Wikimetrics' sunsetting. `lines=10 WIKITECH https://wikitech.wikimedia.org/wiki/Wikimetrics h... [15:32:19] ottomata or elukey: wanna help us copy and chown some wikistats 1 files? [15:32:23] we're in the cave [15:32:34] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [15:35:08] (03CR) 10Nuria: WIP: Oozie: add article recommender (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) (owner: 10Bmansurov) [15:35:52] milimetric: sure [15:36:00] 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review: Wikimetrics docker build/test environment is broken - https://phabricator.wikimedia.org/T193780 (10mforns) @sbassett Yes, we have already redirected metrics.wmflabs.org to Event Metrics tool. And this week we'll release the cloud instances that run Wi... [15:37:13] 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review: Wikimetrics docker build/test environment is broken - https://phabricator.wikimedia.org/T193780 (10sbassett) @mforns Ok, I'll plan to resolve and make public T76782 this Friday afternoon. [15:37:15] (03CR) 10Nuria: Add CpuBenchmark to EventLogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [15:48:03] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Operations, and 3 others: eventgate-analytics k8s pods occasionally can't produce to kafka - https://phabricator.wikimedia.org/T218268 (10Ottomata) After ^: ` kafka-jumbo1003.eqiad.wmnet:9092/bootstrap: Connected to ipv6#[2620:0:861:102:10:64:16:99]:9092 ` L... [15:56:30] (03CR) 10Gilles: Add CpuBenchmark to EventLogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [15:56:41] (03CR) 10Gilles: "*Will it" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [15:59:54] (03CR) 10Nuria: Add CpuBenchmark to EventLogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [16:04:40] (03CR) 10Bmansurov: WIP: Oozie: add article recommender (034 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) (owner: 10Bmansurov) [16:18:22] (03PS2) 10Gilles: Add CpuBenchmark to EventLogging whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) [16:18:39] (03CR) 10Gilles: Add CpuBenchmark to EventLogging whitelist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [16:32:28] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [16:33:19] (03CR) 10Nuria: [C: 03+2] Add CpuBenchmark to EventLogging whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499053 (https://phabricator.wikimedia.org/T219263) (owner: 10Gilles) [16:33:28] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [17:01:09] sorry ateam wifi stopped working, I’ll see yall tomorrow [17:24:08] elukey: I finished auditing those files, so if you git add again and commit/push those changes, I can merge them and we can be done with the cleanup I think [17:24:24] if you prefer, I can open a task and leave it for you later, it's not urgent [17:24:43] nono it is ok [17:24:51] preferences about the commit msg ? [17:25:03] then I suppose I only have to do 'git push' right? [17:25:05] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10Niharika) >>! In T218812#5058017, @aezell wrote: > @EvanProdromou I think that a solution like that is somethin... [17:25:05] nothing fancy [17:25:35] milimetric: --^ [17:26:10] elukey: the git push refs/for blah thing yeah [17:26:27] no fanciness in the commit message - I guess it would be Erik's last commit to the project [17:26:31] so no pressure :) [17:26:32] ah not a direct push, you want to have a review [17:26:34] right? [17:26:46] oh, I don't think you can push, but if you can that's fine [17:26:59] I don't need to review it, I have no idea if it's good or not anyway :) [17:30:09] ah yes of course there is a password for Erik's ssh key [17:30:46] but in theory now you can copy it to your home and try to push it [17:33:27] ok, great, I’ll do that and push with my user. [17:35:09] let's also remove that key [17:37:24] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10Mooeypoo) I also think that making sure MediaWiki is installable and usable in shared hosting is a worthy goal,... [17:37:31] 10Analytics, 10EventBus: EventGate Helm chart should POST test event for readinessProbe - https://phabricator.wikimedia.org/T218680 (10Ottomata) Hm, I just noticed that there is a `eventgate-analytics-staging-service-checker` (pod/hook?) deployed along with eventgate-analytics. Perhaps I should just add a POS... [17:52:26] 10Analytics, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), 10Core Platform Team Backlog (Later), and 3 others: EventBusRCFeedFormatter should clean up events from nulls - https://phabricator.wikimedia.org/T216567 (10Pchelolo) After the patch was deployed we do not have nulls in recent ch... [17:58:26] (03PS1) 10Chelsyx: Add ExternalGuidance event logging table to whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/499270 (https://phabricator.wikimedia.org/T218838) [18:02:43] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Add ExternalGuidance event logging table to whitelist - https://phabricator.wikimedia.org/T218838 (10chelsyx) Thanks @Nuria ! @santhosh Could you please review the patch as well? Thanks! [18:02:55] 10Analytics, 10Product-Analytics, 10Patch-For-Review: Add ExternalGuidance event logging table to whitelist - https://phabricator.wikimedia.org/T218838 (10chelsyx) p:05Triage→03Normal [18:04:02] ottomata: let's talk about schemas in an hour or so [18:08:49] 10Analytics, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), 10Core Platform Team Backlog (Later), and 3 others: EventBusRCFeedFormatter should clean up events from nulls - https://phabricator.wikimedia.org/T216567 (10Ottomata) Since we'll be able to support map types with new schemas (onc... [18:10:18] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10MNeisler) Here's the updated [[ https://docs.google.com/document/d/1jzrE3xdyEHed4Ek5ORRedOlEeH-i111hdmG3tB... [18:15:36] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10mforns) Thanks a lot @MNeisler! @fdans took this task and is already working on this, will reassign the ta... [18:15:46] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10mforns) a:05MNeisler→03fdans [18:17:16] mforns: i do not think fdans is working on this https://phabricator.wikimedia.org/T211173 [18:19:42] mforns: he is working on this one rather: https://phabricator.wikimedia.org/T218964 [18:21:32] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Sunset Wikimetrics - https://phabricator.wikimedia.org/T211835 (10mforns) [18:28:19] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10Nuria) mmm, i think fdans is working on https://phabricator.wikimedia.org/T218964 FYI that a job to load... [18:32:07] nuria, oh, I got totally confused! [18:32:26] fff, will comment on task [18:32:43] mforns: fdans is working on eventlogging to druid, makes sense? [18:35:08] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10mforns) @Nuria @MNeisler Oh yea, I got totally confused. Sorry. Will assign this task to myself and start... [18:35:21] 10Analytics, 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10mforns) a:05fdans→03mforns [18:35:59] nuria, yes yes [18:36:25] nuria: did you wanna talk EL wiki jsonshcmea stuff? [18:36:32] just got confuuused, still need to warmup after vacation... [18:36:42] ottomata: on meeting can do that in a bit [18:37:20] sure anytime [18:47:01] hi milimetric [18:47:13] the dashboard we discussed yesterday [18:47:14] https://language-reportcard.wmflabs.org/cx2/#mt-engines [18:47:18] it still shows the same data [18:47:23] did you do a re-run? [18:47:40] * elukey off! [18:51:24] 10Analytics, 10Research, 10Wikidata: Copy Wikidata dumps to HDFs - https://phabricator.wikimedia.org/T209655 (10leila) >>! In T209655#4804046, @Nuria wrote: > Having missed most of goals this quarter due to our mw woes i think this might need to be moved to next quarter (q4?) @Nuria can your team help us wi... [18:54:28] aharoni: I scheduled it but it looks like it hasn’t kicked off, I’ll do that a bit later today [18:55:44] milimetric: ok [18:57:39] nuria: for context, in light of T102107 we (Research) will need to prioritize productionizing the recommendation API (the improved version). Besides a couple of tasks that Analytics is working on on this front, I'd like to talk with you about how to maintain it if it goes to production (in light of conversations about ML maintenance, scaling, etc.) Let me know if I can block a 15-min in your calendar one of these days. [18:57:40] T102107: Enable Content Translation in Catalan Wikipedia for all logged-in users - https://phabricator.wikimedia.org/T102107 [19:06:30] leila: sure, we will help with getting jobs running and hopefully with filipo's help on how to move binaries/data to prod once in prod probably you want to talk to faidon for everything outside the cluster, makes sense? [19:41:19] 10Analytics, 10Research, 10Wikidata: Copy Wikidata dumps to HDFs - https://phabricator.wikimedia.org/T209655 (10JAllemandou) Most of the complicated things already exist for this to work (equicalent of rsync for HDFS, spark job converting wikidata json dumps to parquet). I wanted for T216160 to be settled be... [20:08:59] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Run critical Analytics Hadoop jobs and make sure that they work with the new auth settings. - https://phabricator.wikimedia.org/T212259 (10Ottomata) Just saw this on the Confluent mailing list. Parking it here for future reference: https://groups.google.... [20:32:24] (03CR) 10Joal: [C: 03+1] "Great patch :) If you don't mind, can you add the following test at the end of the Pageview test class, for paranoid like me (please) :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498702 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [20:33:43] (03CR) 10Joal: [C: 03+1] "Again, great patch in collaboration with the one in refinery-source. Can you please add a link from this commit message to the commit on r" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/498700 (https://phabricator.wikimedia.org/T144100) (owner: 10Awight) [20:43:36] (03PS7) 10Joal: Update mw user-history timestamps [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/497604 (https://phabricator.wikimedia.org/T218463) [20:43:50] (03PS4) 10Joal: Correct names in mediawiki-history sql package [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498861 [20:44:20] 10Analytics, 10EventBus, 10Growth-Team, 10Internet-Archive, and 3 others: Edits to Flow pages result in a page-links-change event with no performer - https://phabricator.wikimedia.org/T216726 (10Milimetric) At some point someone should review https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/SpamBlac... [20:54:55] 10Analytics: Update grouped-wiki files for sqoop - https://phabricator.wikimedia.org/T219326 (10JAllemandou) [21:02:19] hello a-team : with the sunset of wikimetrics, shall we archive the repositories containing old code? [21:03:06] hauskatze: yes, we can do that, though will that mean it disappears from github? [21:03:56] milimetric: we normaly empty the repos and delete the mirrors - if there's a reason not to remove the github mirror it can remain there with an 'ARCHIVED' file [21:04:18] 10Analytics, 10ChangeProp, 10Community-Tech, 10EventBus, and 6 others: Provide the ability to have time-delayed or time-offset jobs in the job queue - https://phabricator.wikimedia.org/T218812 (10Krinkle) >>! Task description: > If a user adds an expiring item to their watchlist, when the expiration date/t... [21:04:27] hauskatze: I have no strong opinion, it has some useful code and examples that I reference from time to time, but I have a local copy [21:05:00] Let's keep the mirror. I see no harm. [21:18:54] milimetric: Heya - My last test of sqoop is running - A change is needed in the group we created originally (I also created https://phabricator.wikimedia.org/T219326 to better update the grouping file) [21:19:02] 10Analytics, 10ORES, 10Scoring-platform-team: Purge ORES scores from Hadoop and begin backfill when model version changes - https://phabricator.wikimedia.org/T209742 (10Harej) p:05Triage→03Lowest [21:19:14] 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team, and 3 others: [Epic] Make ORES scores for wikidata available as a dump - https://phabricator.wikimedia.org/T209611 (10Harej) p:05Low→03Lowest [21:19:22] joal: I just saw that in your e-standup, need any help? [21:19:36] 10Analytics, 10ORES, 10Scoring-platform-team: Include feature values in ORES changeprop stream - https://phabricator.wikimedia.org/T209734 (10Harej) p:05Triage→03Lowest [21:19:37] milimetric: if ok for you, I'll go triple check tomorrow morning that everything is fine, push my last patch after that, and it can go with tchu [21:19:46] 10Analytics, 10ORES, 10Scoring-platform-team: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 (10Harej) p:05Triage→03Lowest [21:19:52] ok, cool, I'll tchu tchu it 'morrow [21:19:54] 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team: Produce dump files for ORES scores - https://phabricator.wikimedia.org/T209739 (10Harej) p:05Triage→03Lowest [21:20:06] 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team, and 3 others: Decide whether we will include raw features - https://phabricator.wikimedia.org/T211069 (10Harej) p:05Triage→03Lowest [21:20:23] milimetric: if all goes well I'll have everything ready before you start :) [21:20:39] joal: sweeet [21:21:01] (03CR) 10Nuria: Correct names in mediawiki-history sql package (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/498861 (owner: 10Joal) [21:23:28] 10Analytics, 10MediaWiki-extensions-ORES, 10Scoring-platform-team, 10Core Platform Team Backlog (Designing), 10Services (designing): ORES hook integration with EventBus - https://phabricator.wikimedia.org/T201869 (10Harej) p:05Triage→03Normal [21:25:20] ottomata: still there? [21:32:15] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Active Editors metric per project family - https://phabricator.wikimedia.org/T188265 (10Nuria) [21:40:21] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Operations, and 3 others: eventgate-analytics k8s pods occasionally can't produce to kafka - https://phabricator.wikimedia.org/T218268 (10akosiaris) 05Open→03Resolved Resolving, feel free to reopen [21:56:28] 10Analytics, 10EventBus, 10Release Pipeline, 10serviceops, 10Services (watching): Modern Event Platform: Stream Intake Service: Documentation - https://phabricator.wikimedia.org/T219332 (10Ottomata) [21:56:34] 10Analytics, 10EventBus, 10Release Pipeline, 10serviceops, 10Services (watching): Modern Event Platform: Stream Intake Service: Documentation - https://phabricator.wikimedia.org/T219332 (10Ottomata) [21:57:39] 10Analytics, 10EventBus, 10Release Pipeline, 10serviceops, 10Services (watching): Modern Event Platform: Stream Intake Service: Documentation - https://phabricator.wikimedia.org/T219332 (10Ottomata) Starting on some great stuff here: https://wikitech.wikimedia.org/wiki/Event_Platform https://wikitech.w... [21:58:30] ottomata: want to talk schemas now [21:59:10] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Active Editors metric per project family - https://phabricator.wikimedia.org/T188265 (10Nuria) Of interest, agreed upon definition of active editor: https://meta.wikimedia.org/wiki/Research:Active_editor [22:02:23] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Sanitization should be run a second time - https://phabricator.wikimedia.org/T212014 (10Nuria) ping @mforns * i think* this ticket can now be closed right? [22:15:32] 10Analytics, 10Performance-Team: ServerTiming EventLogging data getting nuked after 90 days despite whitelist - https://phabricator.wikimedia.org/T219262 (10Nuria) @Gilles Data is on sanitized database , when incoming data is persisted to events and event_sanitized. hive (event_sanitized)> SELECT count(dt)... [22:16:33] 10Analytics, 10Performance-Team: ServerTiming EventLogging data getting nuked after 90 days despite whitelist - https://phabricator.wikimedia.org/T219262 (10Nuria) 05Open→03Resolved [22:16:53] nuria sorry gotta run to a potter class! [22:21:18] 10Analytics, 10Community-Tech, 10Education-Program-Dashboard, 10Event Metrics: I want WikiMetrics integration with the education dashboard that lets you easily pull reports about courses, institutions, etc. - https://phabricator.wikimedia.org/T92454 (10Nuria) [22:21:35] 10Analytics, 10Community-Tech, 10Education-Program-Dashboard, 10Event Metrics: I want WikiMetrics integration with the education dashboard that lets you easily pull reports about courses, institutions, etc. - https://phabricator.wikimedia.org/T92454 (10Nuria) Adding event-metrics here which where I think t... [22:24:05] 10Analytics, 10Community-Tech, 10Education-Program-Dashboard, 10Event Metrics: I want WikiMetrics integration with the education dashboard that lets you easily pull reports about courses, institutions, etc. - https://phabricator.wikimedia.org/T92454 (10Nuria) 05Open→03Resolved [22:28:43] 10Analytics, 10Analytics-Wikimetrics, 10Cleanup, 10GitHub-Mirrors, and 2 others: Archive WikiMetrics - https://phabricator.wikimedia.org/T219334 (10MarcoAurelio) [22:28:55] milimetric: filed T219334 so we can talk/investigate [22:28:57] T219334: Archive WikiMetrics - https://phabricator.wikimedia.org/T219334 [22:29:21] thx hauskatze [22:29:39] np [22:34:12] (03PS5) 10Bmansurov: WIP: Oozie: add article recommender [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) [22:35:58] 10Analytics, 10Analytics-Wikimetrics, 10Cleanup, 10GitHub-Mirrors, and 2 others: Archive WikiMetrics - https://phabricator.wikimedia.org/T219334 (10MarcoAurelio) [22:36:38] 10Analytics, 10Analytics-Wikimetrics, 10Cleanup, 10GitHub-Mirrors, and 2 others: Archive analytics-wikimetrics (deprecated by Event Metrics) - https://phabricator.wikimedia.org/T219334 (10MarcoAurelio) [22:37:56] 10Analytics, 10Analytics-Wikimetrics, 10Cleanup, 10GitHub-Mirrors, and 2 others: Archive analytics-wikimetrics (deprecated by Event Metrics) - https://phabricator.wikimedia.org/T219334 (10MarcoAurelio) 05Open→03Stalled p:05Triage→03Low [23:04:28] I know we had to split up analytics-store dbs [23:04:43] But any chance of making centralauth database be on all the segments? [23:04:56] It would make a lot of things a lot easier if I could join to stuff in that table [23:05:17] And i don't think its a very big db as far as those things go as it literally only has user information [23:05:45] bawolff: a question for our lovely DBA team of which we are mere replicators [23:06:00] Ah, so I'm complaining to the wrong people :) [23:06:21] I'll bring it up with them [23:06:50] * bawolff right now in the middle of typing GROUP_CONCAT( ... SEPERATOR '", "' ) so i can copy and paste into my ad-hoc queries [23:11:19] (03CR) 10Nuria: "Looking much better, does code run as is?" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/496885 (https://phabricator.wikimedia.org/T210844) (owner: 10Bmansurov)