[08:24:07] Hi everybody, can I get pageviews per hour for whole project (not per day as https://tools.wmflabs.org/siteviews)? [09:20:38] Hi Urbanecm [09:21:17] Hi joal [09:21:31] For pageviews per hour per project, you need to go to dumps: https://dumps.wikimedia.org/other/pageviews/ [09:21:45] Urbanecm: --^ [09:22:13] Thanks a lot! [09:22:45] joal ^ [09:23:01] Urbanecm: There are 2 files types in the folders: page_title detailed (named pageviews-....gz), and projectviews (no page_title, projectviews-.....gz) [09:23:25] Urbanecm: You'll have to filter for the projects you're interested in though, since those files contain everything [09:24:15] Okay. I'll do it. Thanks again! [09:24:24] joal, ^ [09:24:26] Ohhhh Urbanecm, and as I recall, there also is project-view per hour in the API, but they seem not displayed by the tool [09:24:54] Urbanecm: Let me provide you with an example [09:25:09] joal, can you link me to that API? [09:25:56] Urbanecm: https://wikimedia.org/api/rest_v1/ [09:26:27] Urbanecm: In pageviews data, you'll see the /metrics/pageviews/aggregate endpoint - That's the one you're after I think :) [09:26:42] Urbanecm: Now I think that's all I have :) [09:26:43] I want projectviews, not pageviews. [09:26:59] Urbanecm: naming convention only: pageviews/aggregate = projectviews [09:27:13] Oh. Thanks joal . [09:27:40] np Urbanecm, enjoy data wrnagling :) [09:29:07] I will [09:29:38] Urbanecm: If you find funny/interesting results, don't hesitate to paste some link here, or to send an email to the analytics list :) [09:30:09] I will. [10:26:51] 10Analytics-Tech-community-metrics: Missing time units for percentile values - https://phabricator.wikimedia.org/T145425#2817299 (10Dicortazar) In that widget we're counting days. I've updated the change in upstream, so @Lcanasdiaz you should update the panel in the dashboard. [10:26:58] 10Analytics-Tech-community-metrics: Missing time units for percentile values - https://phabricator.wikimedia.org/T145425#2817300 (10Dicortazar) a:05Dicortazar>03Lcanasdiaz [10:27:11] 10Analytics-Tech-community-metrics: Kibana's Mailing List data sources are outdated - https://phabricator.wikimedia.org/T146632#2817301 (10Aklapper) 05Open>03Resolved [12:02:41] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Deployment of Maniphest panel - https://phabricator.wikimedia.org/T138002#2817504 (10Aklapper) --> December. [12:03:27] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Maniphest support to be added to GrimorieLab - https://phabricator.wikimedia.org/T138003#2817508 (10Aklapper) [12:11:40] 10Analytics-Tech-community-metrics: Kibana's Mailing List data sources are outdated - https://phabricator.wikimedia.org/T146632#2817560 (10Aklapper) 05Resolved>03Open @Lcanasdiaz: As https://wikimedia.biterg.io/app/kibana#/dashboard/Data-Status currently has indexing issues, I went to https://wikimedia.bite... [12:12:05] 10Analytics-Tech-community-metrics: Kibana's Mailing List data sources do not include recent activity on wikitech-l mailing list - https://phabricator.wikimedia.org/T146632#2817562 (10Aklapper) [13:17:26] (03PS39) 10Mforns: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [13:20:05] (03CR) 10jenkins-bot: [V: 04-1] [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [13:42:47] (03PS40) 10Mforns: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [13:48:14] (03CR) 10jenkins-bot: [V: 04-1] [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [14:01:00] o/ [14:01:20] Hi halfak|Mobile ! [14:01:23] Joining ! [14:01:33] Joal, milimetric:. Making coffee. Will join in a couple minutes [14:01:42] sure halfak|Mobile :) [14:28:09] (03PS41) 10Mforns: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [14:56:37] hey joal and milimetric, I know you're in meeting, ping me when it's good for you to start reviewing [15:03:48] 10Analytics-Tech-community-metrics: Mismatch between six names and certain email address in mediawiki-identities data - https://phabricator.wikimedia.org/T123643#2817932 (10Aklapper) p:05Low>03Lowest [15:04:10] bathroom break, joal + mforns we can hang soon? [15:04:15] 10Analytics-Tech-community-metrics, 07Regression: Git repo blacklist config not applied on wikimedia.biterg.io? - https://phabricator.wikimedia.org/T146135#2817933 (10Aklapper) p:05Normal>03High [15:04:18] works for me ! [15:04:22] milimetric, me too [15:04:24] :] [15:07:05] ok, back [15:07:07] going to cave [15:23:20] 10Quarry: Abuse Filter should be assign to someother person - https://phabricator.wikimedia.org/T151461#2817978 (10Sro43) [15:25:37] 10Quarry: Abuse Filter should be assign to someother person - https://phabricator.wikimedia.org/T151461#2817999 (10Samtar) 05Open>03Invalid p:05Triage>03Lowest [15:32:44] (03PS32) 10Mforns: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [15:34:29] (03PS33) 10Mforns: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [15:34:39] 10Analytics-Tech-community-metrics: "Last Attracted Developers" on Git-Demographics has incorrect date values for "First Commit Date" - https://phabricator.wikimedia.org/T151161#2818012 (10Aklapper) Ahem, Luis made me realize that Git-Demographics is about... **Git** (merged stuff). Not **Gerrit** (potentially n... [15:35:20] 10Analytics-Tech-community-metrics: Have "Last Attracted Developers" information for Gerrit (already exists for Git) - https://phabricator.wikimedia.org/T151161#2818013 (10Aklapper) p:05Normal>03Low [15:37:29] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2818019 (10Aklapper) [15:52:47] 10Analytics-Tech-community-metrics: "Backlog" widget on "Gerrit-Backlog" seems to cover only last two years, misses oldest open changesets - https://phabricator.wikimedia.org/T146893#2818053 (10Aklapper) (PEBKAC: By default the view **is** only about `Last 2 years`, Andre...) "Backlog" on https://wikimedia.bite... [15:58:20] (03CR) 10Milimetric: [WIP] Refactor Mediawiki History scala code (035 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [16:31:30] 06Analytics-Kanban, 06Operations, 10hardware-requests: stat1001 replacement box in eqiad - https://phabricator.wikimedia.org/T149911#2818146 (10mark) >>! In T149911#2772370, @RobH wrote: > I'd like to allocate spare pool system WMF4726 for this request. It has the following specs: > > * Dual Intel® Xeon® P... [16:48:10] 10Analytics-Dashiki, 06Analytics-Kanban: Add extension and category (ala Eventlogging) for DashikiConfigs - https://phabricator.wikimedia.org/T125403#2818188 (10Milimetric) a:03Milimetric [16:52:58] 10Analytics-Tech-community-metrics, 06Developer-Relations: Complete migration to new Bitergia's development dashboard (and then kill korma.wmflabs.org) - https://phabricator.wikimedia.org/T137997#2818195 (10Aklapper) [16:54:49] 10Analytics-Tech-community-metrics, 06Developer-Relations, 07Epic: Complete migration to new Bitergia's development dashboard (and then kill korma.wmflabs.org) - https://phabricator.wikimedia.org/T137997#2386336 (10Aklapper) p:05Normal>03Low I'm going to turn this task into an epic and use subtasks to li... [16:55:26] 10Analytics-Tech-community-metrics: Have "Last Attracted Developers" information for Gerrit (already exists for Git) - https://phabricator.wikimedia.org/T151161#2818203 (10Aklapper) [16:55:29] 10Analytics-Tech-community-metrics, 06Developer-Relations, 07Epic: Complete migration to new Bitergia's development dashboard (and then kill korma.wmflabs.org) - https://phabricator.wikimedia.org/T137997#2818202 (10Aklapper) [17:39:19] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Deployment of Maniphest panel - https://phabricator.wikimedia.org/T138002#2818325 (10Aklapper) [17:39:22] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Maniphest support to be added to GrimorieLab - https://phabricator.wikimedia.org/T138003#2818324 (10Aklapper) [17:39:23] 10Analytics-Tech-community-metrics: "Tickets" (defunct Bugzilla) vs "Maniphest" sections on korma are confusing - https://phabricator.wikimedia.org/T106037#2818326 (10Aklapper) [17:39:27] 10Analytics-Tech-community-metrics, 10Phabricator: Decide on wanted metrics for Maniphest in kibana - https://phabricator.wikimedia.org/T28#2818323 (10Aklapper) [17:52:39] (03CR) 10Nuria: [C: 032] WikidataArticlePlaceholderMetrics also send search referral data [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/305989 (https://phabricator.wikimedia.org/T142955) (owner: 10Addshore) [17:58:03] (03Merged) 10jenkins-bot: WikidataArticlePlaceholderMetrics also send search referral data [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/305989 (https://phabricator.wikimedia.org/T142955) (owner: 10Addshore) [18:05:29] 10Analytics-Tech-community-metrics: "Patchsets Statistics Per Review" widget on "Gerrit" is incomprehensible - https://phabricator.wikimedia.org/T151218#2818439 (10Aklapper) p:05Triage>03Low [18:06:12] 10Analytics-Tech-community-metrics: "Backlog" widget on "Gerrit-Backlog" has redundant "Changesets" column (always "1") - https://phabricator.wikimedia.org/T146891#2818442 (10Aklapper) p:05Low>03Lowest [18:13:29] 10Analytics-Tech-community-metrics: Consider removing IP address MediaWiki accounts from DB - https://phabricator.wikimedia.org/T149482#2818453 (10Aklapper) I'm wondering if there's interest in implementing this idea (and if it might improve performance or if that's rather unlikely), especially if other Bitergia... [18:37:29] 10Analytics-Tech-community-metrics, 10Differential, 06Developer-Relations (Jan-Mar-2017): Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#2818489 (10Aklapper) @Lcanasdiaz: [[ https://www.mediawiki.org/wiki/Wikimed... [18:52:12] 10Analytics-Tech-community-metrics, 07I18n: Add i18n/l10n support to korma.wmflabs.org - https://phabricator.wikimedia.org/T62073#656517 (10Aklapper) For korma.wmflabs.org this request is a WONTFIX/declined as korma.wmflabs.org will get superseded by wikimedia.biterg.io. Hence rephrasing this ticket and settin... [18:53:03] 10Analytics-Tech-community-metrics, 07I18n: Add i18n/l10n support to Technical Community Metrics - https://phabricator.wikimedia.org/T62073#2818558 (10Aklapper) 05Open>03stalled [19:11:46] 10Analytics-Tech-community-metrics, 10Possible-Tech-Projects, 07Epic: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#2818615 (10Aklapper) p:05Normal>03Low [19:50:20] Hey! Is there anyone who could review https://gerrit.wikimedia.org/r/#/c/323133/ This is about a discovery technical user having access to stats data. [19:50:29] * gehel is not really sure who's the owner here... [19:56:42] gehel: is this for access to analytics replicas? [19:56:56] nuria: yes, exactly [19:56:59] gehel: like the ones that host EL data? [19:57:17] nuria: EL ? Not sure what that means... [19:57:27] gehel: is there a reason why you cannot use already existing users & groups? [19:57:54] nuria: this is an already existing user and an already existing group, just the user is not yet in the group... [19:58:09] gehel: but this is a generic user, not a login [19:58:34] nuria: this is a technical user, yes, not a login shell [19:59:06] gehel: right, aim not sure that is the way we want things to be set up (i might be wrong) , need to consult with ops [19:59:23] * gehel is mostly part of Ops... [20:00:06] gehel: then .. you can probably point me to other members of stats group [20:00:23] We already have that user setup on stat* servers, running some crons to collect stats about maps and other discovery projects. We need that user to get some more access... [20:00:58] nuria: the current members of the "stats" group: ezachte,milimetric,joal,mforns,bearloga,nuria [20:01:13] gehel: right, i understand but i think we are undergoing changes to reduce the number of "umbrella" users [20:01:35] gehel: so it will be better if you used (i think) the currently existing stats user [20:01:53] gehel: that already runs analytics jobs, again, will need to consult with ops on this [20:02:24] gehel: we probably do not want to have two levels of access under stats group, but again, not 100% sure [20:02:55] gehel: is there a reason why you cannot use the current stats user? [20:04:02] nuria: there was some comments by ottomata on https://phabricator.wikimedia.org/T151063#2811510 which indicated this was the path to follow [20:04:34] gehel: an, i see, then otto can CR your patch [20:05:27] nuria: yep, probably, I was trying to see if there is anyone else around to kick this forward before everyone gets on weekend... [20:05:50] Ok, I'll try to grab Andrew... [20:05:56] 10Analytics-Tech-community-metrics: Check whether mailing list activity per person on korma is in sync with current "mlstats_mailing_lists.conf" - https://phabricator.wikimedia.org/T132907#2818735 (10Aklapper) a:05Lcanasdiaz>03Aklapper @Lcanasdiaz: Nothing to investigate anymore for you. I just need to fix m... [20:08:00] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Check whether mailing list activity per person on korma is in sync with current "mlstats_mailing_lists.conf" - https://phabricator.wikimedia.org/T132907#2818742 (10Aklapper) [20:08:38] nuria: Thanks a lot for the review! At least now it is clear that this isn't clear :) [20:21:48] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Fix incorrect mailing list activity of AKlapper (=Phabricator) in Technical Community Metrics user data - https://phabricator.wikimedia.org/T132907#2818796 (10Aklapper) [20:27:28] 10Analytics-Tech-community-metrics, 06Developer-Relations: Sudden rise of changesets in wikimedia.biterg.io metrics - https://phabricator.wikimedia.org/T145849#2818809 (10Aklapper) 05Open>03Resolved Indeed, looks more likely (correct) now, also when [[ https://wikimedia.biterg.io:443/goto/f900dad89bc71f02c... [20:59:33] 06Analytics-Kanban: unusual number of '-" pageviews in October - https://phabricator.wikimedia.org/T150990#2818937 (10Nuria) The bulk of the "-" pageviews increase is due to traffic coming from this user agent: "Blackboard Safeassign" that looks to be a wikipedia bot linked to this tool: http://www.safeassign.co... [21:12:21] (03PS1) 10Nuria: Adding self-identified bot to bot regex [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/323249 (https://phabricator.wikimedia.org/T150990) [21:14:22] 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2818984 (10Krinkle) [21:21:36] 10Analytics-Tech-community-metrics: Provide equivalent of korma's "Gerrit review queue" (listing the 'worst' SCR repositories) in Kibana - https://phabricator.wikimedia.org/T151488#2819021 (10Aklapper) [21:22:45] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2197987 (10Aklapper) [21:23:14] 10Analytics-Tech-community-metrics: Provide equivalent of korma's "Gerrit review queue" (listing the 'worst' SCR repositories) in Kibana - https://phabricator.wikimedia.org/T151488#2819021 (10Aklapper) [21:23:17] 10Analytics-Tech-community-metrics, 06Developer-Relations, 07Epic: Complete migration to new Bitergia's development dashboard (and then kill korma.wmflabs.org) - https://phabricator.wikimedia.org/T137997#2819039 (10Aklapper) [23:01:24] 10Analytics-Tech-community-metrics: "Changesets" on "Gerrit-Delays" should not list ABANDONED changesets? - https://phabricator.wikimedia.org/T151215#2819371 (10Aklapper) p:05Triage>03Lowest >>! In T138752#2816074, @Lcanasdiaz wrote: > The thing about this panel is to measure the amount of time need for a ti... [23:12:02] 10Analytics-Tech-community-metrics: Display attracted vs. retained contributors in Kibana - https://phabricator.wikimedia.org/T151501#2819403 (10Aklapper) [23:12:10] 10Analytics-Tech-community-metrics: Display attracted vs. retained contributors in Kibana - https://phabricator.wikimedia.org/T151501#2819403 (10Aklapper) p:05Triage>03Lowest [23:13:07] 10Analytics-Tech-community-metrics, 06Developer-Relations (Oct-Dec-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2819420 (10Aklapper) [23:28:13] 10Analytics-Tech-community-metrics: No korma data updates for last two weeks - https://phabricator.wikimedia.org/T151503#2819464 (10Aklapper) [23:28:24] 10Analytics-Tech-community-metrics: No korma data updates for last two weeks - https://phabricator.wikimedia.org/T151503#2819477 (10Aklapper) p:05Triage>03Unbreak!