[00:39:00] Hello analytics folks [00:39:14] For graphs like https://flow-reportcard.wmflabs.org/#usage-data/active-boards , how can I make it not render as percentages? [00:39:23] All our data is inflated by 100x because it's expressed as % now [01:25:24] I also can't figure out what software is rendering that graph, or where to find documentation for it [01:28:26] hi RoanKattouw, we only have one project that we do dashboarding with these days: Dashiki (https://wikitech.wikimedia.org/wiki/Analytics/Systems/Dashiki but don't read that it's too in-depth) [01:28:46] RoanKattouw: we *really* need to make that more clear on the dashboards themselves though, because everyone is confused, I'm sorry about thta [01:29:04] I'll make a ticket after I look at the percentage problem [01:29:11] Yeah, I even inspected the DOM to see if the CSS class names had prefixes [01:29:17] Nothing mentions Dashiki anywhere [01:29:30] Also is there any documentation for the format of config.yaml and generally about how to configure graphs? [01:29:37] I think I found something but wasn't sure if it was applicable [01:30:14] the docs are here https://wikitech.wikimedia.org/wiki/Analytics/Systems/Dashiki#On-wiki_configuration [01:30:32] the dashboards are always configured on-wiki, in the flow-reporcard case it's here... one sec.. [01:30:46] https://meta.wikimedia.org/wiki/Config:Dashiki:FlowReportcard [01:31:47] Oh, so then the config is for the report generator or whatever it's called? [01:32:05] config.yaml I mean, configures the software that generates the .tsv files from SQL queries [01:32:55] RoanKattouw: there's two things: reportupdater is what configures the queries/jobs that make the data and put it in the right place [01:32:56] Also on meta, some things are under Config:Dashiki: but some are under Dashiki: , so I only looked under the latter and missed that we had the former [01:33:06] and then Dashiki configures the dashboard to display the data [01:33:37] yeah, the ones under Dashiki: are old and haven't been moved yet, the Config:Dashiki ones are rendered by the custom Mediawiki/Dashiki extension [01:33:43] OK cool [01:34:05] And I just looked at the .tsv file and its numbers are not inflated 100x, so it must be Dashiki doing the inflation / interpretation as percentages [01:34:12] ok, but to answer your question, it might just be a bug, I gotta think for a second if there's a formatting option for that specific case [01:34:23] yeah, definitely dashiki is doing that [01:34:27] I mean maybe my config just doesn't specify a formatting option and it defaults to %? [01:34:59] Aha [01:35:01] "format": "percent" // number format [01:35:11] Now I just need to find docs on what the possible values for "format" are [01:35:16] Is there a schema for this JSON perhaps? [01:37:59] RoanKattouw: ok, I fixed https://flow-reportcard.wmflabs.org/#usage-data/unique-users by setting that format: kmb [01:38:44] Aha thanks [01:39:24] ok, got all of the other ones, and making a task right now to document this, will do it in the next week [01:40:04] Thanks [01:40:08] What does kmb stand for? [01:44:14] 10Analytics-Kanban: Document Dashiki - https://phabricator.wikimedia.org/T182477#3824562 (10Milimetric) [01:46:08] RoanKattouw: oh, kilo / mega / billion I think [01:46:48] Aah OK [01:47:10] yeah, I feel bad, I've neglected Dashiki for the past year while working on Wikistats and other stuff [01:47:19] thanks for pushing me to fix it up [01:51:52] 10Analytics-Kanban, 10Documentation: Document Dashiki - https://phabricator.wikimedia.org/T182477#3824575 (10Reedy) [02:40:07] (03PS1) 10Catrope: Correct SQL comments about moderation checks [analytics/limn-flow-data] - 10https://gerrit.wikimedia.org/r/396558 [03:59:14] (03PS1) 10Catrope: Fix definition of "active board", "active topic" [analytics/limn-flow-data] - 10https://gerrit.wikimedia.org/r/396566 [04:01:31] (03PS2) 10Catrope: Fix definition of "active board", "active topic" [analytics/limn-flow-data] - 10https://gerrit.wikimedia.org/r/396566 [05:26:11] 10Analytics, 10Research: Make HTML dumps available - https://phabricator.wikimedia.org/T182351#3824735 (10bd808) @leila the #cloud-services-team (specifically @madhuvishy) is working on {T168486} in concert with @ArielGlenn to migrate the public storage and delivery of Dumps (NFS, rsync, and HTTPS endpoints) t... [05:31:05] 10Analytics, 10Research: Make HTML dumps available - https://phabricator.wikimedia.org/T182351#3824740 (10bd808) See also: * {T17017} * {T133547} * #mediawiki-extensions-dumphtml [05:46:33] RoanKattouw: we will add a note to dashiki about where to find docs [09:27:58] 10Analytics, 10EventBus, 10MediaWiki-API, 10MediaWiki-JobQueue, and 3 others: Handling of structured data input in MediaWiki APIs - https://phabricator.wikimedia.org/T182475#3824833 (10mobrovac) We have the same need in the JobQueue migration project to be able to use Special:RunSignleJob, cf. {T182372} wh... [09:28:41] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Make Kafka JobQueue use Special:RunSingleJob - https://phabricator.wikimedia.org/T182372#3821757 (10mobrovac) [09:28:44] 10Analytics, 10EventBus, 10MediaWiki-API, 10MediaWiki-JobQueue, and 3 others: Handling of structured data input in MediaWiki APIs - https://phabricator.wikimedia.org/T182475#3824838 (10mobrovac) [09:31:34] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Make Kafka JobQueue use Special:RunSingleJob - https://phabricator.wikimedia.org/T182372#3824841 (10mobrovac) This is effectively blocked by {T182475}, since we need `Special:RunSignleJob` to be able to read structured data (which i... [10:57:50] 10Analytics-Kanban, 10Patch-For-Review: Productionize Superset - https://phabricator.wikimedia.org/T166689#3824946 (10elukey) Weird, when I try to access the website I get: ``` This page isn’t working superset.wikimedia.org redirected you too many times. Try clearing your cookies. ERR_TOO_MANY_REDIRECTS ```... [11:11:03] 10Analytics-Kanban, 10Patch-For-Review: Productionize Superset - https://phabricator.wikimedia.org/T166689#3824952 (10elukey) This is what I am seeing with tcpdump on thorium: ``` GET / HTTP/1.1 Host: localhost:9080 [..] X-Remote-User: Elukey X-Forwarded-Host: superset.wikimedia.org X-Forwarded-Server: supers... [11:12:28] anybody getting TOO_MANY_REDIRECTS with superset? [11:13:24] ahhh "its because superset is not auto-creating your superset account after auth" [11:14:05] 10Analytics-Kanban, 10Patch-For-Review: Productionize Superset - https://phabricator.wikimedia.org/T166689#3824956 (10elukey) Reading from the Analytics IRC chan it seems that "its because superset is not auto-creating your superset account after auth". So if anybody encounters the issue this is the problem :) [11:17:27] joal: o/ - I took a look to druid's metrics and they look sane, but it is interesting to see the differences between the segments count/size as seen by the coordinator and by the historicals [11:17:43] if you have time I'd like to double check that with you on Monday [11:17:51] elukey: We'll take time :) [11:18:10] hey! [11:18:29] sorry I thought you were not online, didn't mean to ping but just to leave a msg :) [11:18:38] banner data is also flowing [11:18:40] nice :) [11:19:21] elukey: I think I have an idea as o why it failed, but it's a bit strangwe [11:19:39] joal: is there anything that it is not in our analytics infra?? [11:19:51] huhu :D [23:25:33] 10Analytics-Tech-community-metrics, 10Developer-Relations: Go through default Kibana widgets; decide which ones are not relevant for us and remove them - https://phabricator.wikimedia.org/T147001#3825944 (10Aklapper) p:05Lowest>03Low