[03:41:50] Analytics-Kanban, Team-Practices-This-Week: Get regular traffic reports on TPG pages - https://phabricator.wikimedia.org/T99815#1459600 (kevinator) [03:44:20] Analytics-Kanban, Team-Practices-This-Week: Get regular traffic reports on TPG pages - https://phabricator.wikimedia.org/T99815#1459609 (kevinator) I think this deserves some documentation on wikitech too. I'll document something on wikitech then I can take you through it @JAufrecht and then iterate on th... [05:31:33] Analytics, Traffic, operations: Provide summary of MediaWiki downloads - https://phabricator.wikimedia.org/T104010#1459671 (Legoktm) >>! In T104010#1456863, @MarkAHershberger wrote: > I got Kevin Luduc here at Wikimania to get us the information we needed. > > Thanks! Can these be published? [11:55:44] Analytics: Classification of Bing robots as spider traffic instead of user traffic - https://phabricator.wikimedia.org/T106134#1460080 (dr0ptp4kt) NEW [11:58:14] Analytics: Classification of Bing robots as spider traffic instead of user traffic - https://phabricator.wikimedia.org/T106134#1460089 (dr0ptp4kt) [13:18:46] Analytics-Tech-community-metrics: Jenkins-mwext-sync appears in "Who contributes code" - https://phabricator.wikimedia.org/T105983#1460195 (Aklapper) Confirming. Slightly related to {T100189} wrt blacklisting/excluding certain activity. [13:31:51] Analytics-Tech-community-metrics: Jenkins-mwext-sync appears in "Who contributes code" - https://phabricator.wikimedia.org/T105983#1460220 (Aklapper) p:Triage>Low [15:46:22] Analytics, Traffic, operations: Provide summary of MediaWiki downloads - https://phabricator.wikimedia.org/T104010#1460485 (Ckoerner) Thanks to @kevinator for providing these statistics of MediaWiki downloads. From Kevin, A couple of caveats around all the data: - I cannot filter out bots downloadin... [16:14:13] (PS1) Yuvipanda: Move slugify function into utils [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225326 [16:14:37] (CR) Yuvipanda: [C: 2] Move slugify function into utils [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225326 (owner: Yuvipanda) [16:19:36] Quarry: Allow filtering of the final report - https://phabricator.wikimedia.org/T61764#1460572 (yuvipanda) Wait you can already do this there is a search option in data tables... [16:28:18] Quarry: REPORTS-52 Add option to output report results to a wiki page - https://phabricator.wikimedia.org/T60833#1460588 (yuvipanda) Hmm, interesting. But this would require bot approval for all wikis? [16:28:31] Quarry: REPORTS-52 Add option to output report results to a wiki page - https://phabricator.wikimedia.org/T60833#1460590 (yuvipanda) p:Triage>Low [16:44:15] Analytics-Wikimetrics, Community-Wikimetrics: Make Wikimedia project titles more user-friendly - https://phabricator.wikimedia.org/T100692#1460660 (egalvezwmf) declined>Open [16:54:16] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [16:56:17] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [17:53:17] Analytics, MediaWiki-Stakeholders-Group, Traffic, Wikimania-Hackathon-2015, operations: Provide summary of MediaWiki downloads - https://phabricator.wikimedia.org/T104010#1460865 (Ckoerner) [17:53:36] Analytics, MediaWiki-Stakeholders-Group, Traffic, Wikimania-Hackathon-2015, operations: Provide summary of MediaWiki downloads - https://phabricator.wikimedia.org/T104010#1405390 (Ckoerner) [18:51:22] Quarry: Add list of query executions to the query page side-bar - https://phabricator.wikimedia.org/T100982#1460951 (Capt_Swing) Might want to put this in a "history" tab, rather than crowding the query window with a persistent sidebar. [18:51:54] Quarry: Add list of query executions to the query page side-bar - https://phabricator.wikimedia.org/T100982#1460953 (Capt_Swing) p:Triage>Normal [18:53:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [18:55:08] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [18:57:11] Analytics-Backlog, Quarry: it would be useful to run the same Quarry query conveniently in several database - https://phabricator.wikimedia.org/T95582#1460961 (Capt_Swing) p:Triage>High [18:58:40] Quarry: Allow published query titles to be searched or filtered by tag - https://phabricator.wikimedia.org/T90509#1460975 (Capt_Swing) Let's start with simple search within 'title' and description field? [18:58:49] Quarry: Allow published query titles to be searched or filtered by tag - https://phabricator.wikimedia.org/T90509#1460976 (Capt_Swing) p:Triage>High [19:03:26] Quarry, Tool-Labs-tools-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549#1460992 (Capt_Swing) possibly use Jinja? [19:04:07] Quarry, Tool-Labs-tools-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549#1460995 (Capt_Swing) p:Triage>Normal [19:05:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [19:06:35] Quarry: Recurring queries - https://phabricator.wikimedia.org/T101835#1460999 (Capt_Swing) p:Triage>Normal [19:07:08] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [19:07:27] Quarry: Recurring queries - https://phabricator.wikimedia.org/T101835#1348993 (Capt_Swing) Have recurring queries: to start, specify that a query can be run weekly or monthly, etc. [19:08:54] Quarry: Allow published query titles to be searched or filtered by tag - https://phabricator.wikimedia.org/T90509#1461012 (Capt_Swing) p:High>Normal [19:09:51] Quarry: Make "Home" navlink go to profile for logged-in users. - https://phabricator.wikimedia.org/T85175#1461013 (Capt_Swing) p:Triage>Normal [19:11:01] Quarry: Allow comments on queries - https://phabricator.wikimedia.org/T71543#1461016 (Capt_Swing) [19:11:21] Quarry: Add 'download as SQL' option - https://phabricator.wikimedia.org/T71191#1461021 (Capt_Swing) [19:11:48] Quarry: Build a schema explorer - https://phabricator.wikimedia.org/T71192#1461023 (Capt_Swing) [19:12:10] Quarry: Show the execution time in the table of queries - https://phabricator.wikimedia.org/T71264#1461025 (Capt_Swing) [19:13:08] Quarry: Provide a way to hyperlink texts - https://phabricator.wikimedia.org/T74874#1461043 (Capt_Swing) related to T78549 [19:14:28] Quarry: Add a list/table of popular queries - https://phabricator.wikimedia.org/T71266#1461053 (Capt_Swing) p:Triage>Normal [19:17:05] Quarry: Replace spaces with underscores for wiki usernames in URL - https://phabricator.wikimedia.org/T72166#1461064 (Capt_Swing) p:Triage>Low [19:18:35] Quarry: Make the table sortable - https://phabricator.wikimedia.org/T71265#1461067 (Capt_Swing) p:Triage>Lowest [19:19:32] Quarry: Make the table sortable - https://phabricator.wikimedia.org/T71265#1461072 (Capt_Swing) @yuvipanda and I think that T90509 will cover most of these use cases, plus some others. So we'll keep this as a lower priority for now? [19:21:14] Quarry: Categorizing the queries - https://phabricator.wikimedia.org/T76127#1461077 (Capt_Swing) p:Triage>Lowest [19:21:51] Quarry: Categorizing the queries - https://phabricator.wikimedia.org/T76127#1461087 (Capt_Swing) @yuvipanda and I think T90509 will solve this (user can put #tags in query description, and search for them in the search interface). [19:24:30] Quarry: Database dump for analysis - https://phabricator.wikimedia.org/T93907#1461092 (Capt_Swing) p:Triage>Normal [19:25:49] Quarry: Raise query limits - https://phabricator.wikimedia.org/T74342#1461098 (Capt_Swing) @halfak does the 20 minute limit solve this? [19:26:17] Quarry: Raise query limits - https://phabricator.wikimedia.org/T74342#1461103 (Capt_Swing) p:Triage>Normal [19:31:15] Quarry, Discovery, Labs-Other-Projects, Wikidata, and 2 others: Setup sparqly service at https://sparqly.wmflabs.org/ (like Quarry) - https://phabricator.wikimedia.org/T104762#1461107 (Capt_Swing) [19:40:59] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1461143 (Ottomata) [19:41:43] Analytics-Cluster, Ops-Access-Requests, operations: Sudo permissions for hdfs user on analytics-hadoop - https://phabricator.wikimedia.org/T104020#1461146 (Ottomata) @kevinator please approve. This will need to be discussed at an Ops meeting. The relevant group is analytics-admins, this grants sudo... [19:42:14] Quarry: Search or filter queries by title or summary - https://phabricator.wikimedia.org/T90509#1461148 (Capt_Swing) [19:44:46] Quarry, Tool-Labs-tools-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549#1461156 (Capt_Swing) Making this the 'Epic' ticket for all report-related tickets, which we @valhallasw can either close as duplicates, or they can just sit in the backlog for now. [19:47:23] Quarry, Tool-Labs-tools-tsreports: Quarry-TSreports feature parity - https://phabricator.wikimedia.org/T78549#1461158 (Capt_Swing) [19:47:27] Quarry: Provide a way to hyperlink texts - https://phabricator.wikimedia.org/T74874#1461157 (Capt_Swing) [19:48:28] Quarry: Add a stop button to halt the query - https://phabricator.wikimedia.org/T71037#1461161 (Capt_Swing) @yuvipanda does reloading the page stop the query? [19:48:53] Quarry: Add a stop button to halt the query - https://phabricator.wikimedia.org/T71037#1461164 (yuvipanda) Nope, does not. Submitting a new query stops the old one though. Am not sure if this is strictly necessary... [19:54:00] Quarry: Only load 'head' of result set - https://phabricator.wikimedia.org/T71076#1461173 (Capt_Swing) @halfak now that Quarry paginates results, with 100 per page, is this resolved? [20:03:23] Quarry: Raise query limits - https://phabricator.wikimedia.org/T74342#1461189 (Halfak) Yes. This was a badly specified bug. The query limit has been raised as requested, but I still can't really run this query. Let's mark this resolved and I'll look into why the query takes so long before I re-file. [20:03:31] Quarry: Raise query limits - https://phabricator.wikimedia.org/T74342#1461190 (Halfak) Open>Resolved [20:11:21] (PS1) Yuvipanda: Move user related code out into a separate blueprint [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225376 [20:11:23] (PS1) Yuvipanda: Make published field of query not null and default to False [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225377 [20:29:10] Quarry: Add a stop button to halt the query - https://phabricator.wikimedia.org/T71037#1461298 (Capt_Swing) If I start a long-running query, then realize I don't care about it and create a new one immediately, does the fact that the old one is still running cause any lag for my new query? [20:34:32] (CR) Yuvipanda: [C: 2 V: 2] Move user related code out into a separate blueprint [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225376 (owner: Yuvipanda) [20:34:46] (CR) Yuvipanda: [C: 2] Make published field of query not null and default to False [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225377 (owner: Yuvipanda) [20:34:52] (Merged) jenkins-bot: Make published field of query not null and default to False [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225377 (owner: Yuvipanda) [20:37:56] (PS1) Yuvipanda: Do not limit number of queries displayed on user pages [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225473 (https://phabricator.wikimedia.org/T77948) [20:38:44] (CR) Yuvipanda: [C: 2] Do not limit number of queries displayed on user pages [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225473 (https://phabricator.wikimedia.org/T77948) (owner: Yuvipanda) [20:38:50] (Merged) jenkins-bot: Do not limit number of queries displayed on user pages [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225473 (https://phabricator.wikimedia.org/T77948) (owner: Yuvipanda) [20:40:48] Analytics-Kanban, MediaWiki-extensions-ExtensionDistributor, Patch-For-Review: Set up graphs and dumps for ExtensionDistributor download statistics {frog} [3 pts] - https://phabricator.wikimedia.org/T101194#1461365 (Legoktm) >>! In T101194#1418119, @Milimetric wrote: > Also, James's query will continue... [20:42:32] Quarry, Patch-For-Review: Show all published queries in profile - https://phabricator.wikimedia.org/T77948#1461368 (yuvipanda) Open>Resolved a:yuvipanda It's a bit unweildy but works now! [21:21:10] Quarry: Number of queries shown in profile is wrong - https://phabricator.wikimedia.org/T86512#1461447 (yuvipanda) Check now? [21:34:20] (PS1) Ottomata: POC for Realtime Trending Pageviews [analytics/refinery/source] - https://gerrit.wikimedia.org/r/225485 [21:37:53] (PS1) Yuvipanda: Add timesince to published queries as well [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225489 [21:38:18] (CR) Yuvipanda: [C: 2] Add timesince to published queries as well [analytics/quarry/web] - https://gerrit.wikimedia.org/r/225489 (owner: Yuvipanda)