[00:21:02] Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2226075 (Tbayer) [00:25:44] Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2226083 (Tbayer) I have updated the Hive / Wikistats comparison in the task description with two more months' worth of data. Wikistats continues to... [01:41:26] HaeB: what we have done so far is add the word ARCHIVED to the schema description in the Schema page, and change status to inactive in the talk page [01:41:37] https://meta.wikimedia.org/wiki/Schema:Analytics [01:42:21] thanks, makes sense [02:40:21] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226202 (Yurik) Juts wrote in-depth implementation details at https://www.mediawiki.org/wiki/Extension:J... [02:53:25] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226211 (Yurik) @brion, could you think of use cases for partial data reads? I think it will be mostly "... [03:07:27] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2226245 (brion) Pulling individual data items out of large lists; pulling relevant columns in order to s... [04:07:43] (CR) Nuria: "I tested the happy case (regular report) for browser reports and sessions edit report." [analytics/reportupdater] - https://gerrit.wikimedia.org/r/284162 (https://phabricator.wikimedia.org/T117537) (owner: Mforns) [04:11:00] (CR) Nuria: [C: 2 V: 2] Add null values when query returns no results [analytics/reportupdater] - https://gerrit.wikimedia.org/r/284162 (https://phabricator.wikimedia.org/T117537) (owner: Mforns) [05:33:15] Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2017776 (Nemo_bis) I don't see why such a small (alleged) underreporting would be a problem. What I read in your table is that the numbers are dire... [06:13:43] Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2226397 (Tbayer) >>! In T126579#2226344, @Nemo_bis wrote: > I don't see why such a small (alleged) underreporting would be a problem. I don't cons... [07:17:32] Analytics, Pageviews-API, I18n: [[Wikimedia:Pageviews-hover-to-exclude/en]] typo - https://phabricator.wikimedia.org/T133232#2226444 (Liuxinyu970226) [08:29:44] Analytics-Cluster: Hadoop Yarn Nodemanagers occasionally die with OOM / GC errors - https://phabricator.wikimedia.org/T102954#2226522 (elukey) Open>Resolved The new heap size seems to have solved the problem completely, closing the task. [08:32:21] Analytics, Operations, Patch-For-Review: kafkatee cronspam from oxygen - https://phabricator.wikimedia.org/T132322#2226524 (elukey) Open>Resolved [09:38:41] Analytics-Cluster, Graphite: kafka statsd metrics rate calculation yields double underflow - https://phabricator.wikimedia.org/T97277#2226571 (fgiunchedi) opened https://github.com/jmxtrans/jmxtrans/issues/448 upstream to track this [10:32:56] * elukey lunch! [10:41:51] (CR) Mforns: [C: -1] "Looks good to me overall! And it's a tough change..." (2 comments) [analytics/dashiki] - https://gerrit.wikimedia.org/r/278395 (https://phabricator.wikimedia.org/T131547) (owner: Jdlrobson) [13:40:38] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227240 (matmarex) As I understand these are stored as regular MediaWiki pages now, so they have a maxim... [13:45:48] Analytics, MediaWiki-History-or-Diffs, Pageviews-API: Page view statistics link should be in page informations - https://phabricator.wikimedia.org/T133292#2227247 (Dvorapa) [14:03:32] Analytics, MediaWiki-History-or-Diffs, Pageviews-API: Page view statistics link should be in page informations - https://phabricator.wikimedia.org/T133292#2227247 (matej_suchanek) There is no such a link unless added locally via MediaWiki:Histlegend. Direct integration with either would be good idea. [15:10:41] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227437 (brion) >>! In T120452#2227240, @matmarex wrote: > As I understand these are stored as regular M... [15:12:04] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2227438 (brion) Side note: headers are rejected if they contain spaces. That seems odd? [15:49:09] madhuvishy: can we add "or just blame ottomata if services are down?" :P [15:49:26] :D [15:49:29] of course [16:01:41] joal, standup? [16:01:45] Yay! [16:01:49] :] [16:55:24] mforns: do you know if in wikimetrics, we have a cohort that expands to all user’s accounts across projects via CentralAuth, the metrics calculated on top of the cohort will also take into account wikidata? My guess is yes [16:55:47] madhuvishy, mmm [16:56:07] I can do a quick test [16:57:17] mforns: ah no worries - i'll check - was wondering if you know off the top of your head :) [17:05:53] going offline team! byyyeee [17:07:08] bye elukey ! [17:27:15] (CR) Nuria: "Let's create another task for 2) and will fix 1). Thank you." [analytics/dashiki] - https://gerrit.wikimedia.org/r/278395 (https://phabricator.wikimedia.org/T131547) (owner: Jdlrobson) [17:28:21] Analytics, Analytics-Dashiki: Dashiki breakdown layout problems. UI - https://phabricator.wikimedia.org/T133312#2227918 (Nuria) [17:29:09] mforns_: have created task for css issues on breakdown: https://phabricator.wikimedia.org/T133312 [17:29:26] nuria_, cool thanks! [17:29:34] nuria_, are you working today? [18:04:47] Analytics, Hovercards, Reading-Web-Sprint-71-m: Verify X-Analytics: preview=1 in stable - https://phabricator.wikimedia.org/T133067#2228083 (dr0ptp4kt) Update on this task. This may need to go back for re-work / result in a new task that changes the key-value for the header enrichment, per discussion... [19:29:40] Analytics: pagecounts-ez files are missing data - https://phabricator.wikimedia.org/T132761#2228374 (HYanWong) I can provide a list of missing title if need be, but I guess what I've said will be enough to trace the problem. [19:46:57] Analytics-Features: test #2 - https://phabricator.wikimedia.org/T75742#2228407 (kevinator) Open>Resolved [21:12:55] Analytics, MediaWiki-History-or-Diffs, Pageviews-API: Page view statistics link should be in page informations - https://phabricator.wikimedia.org/T133292#2228894 (Dvorapa) [21:14:55] Analytics, MediaWiki-History-or-Diffs, Pageviews-API: Page view statistics link should be in page informations - https://phabricator.wikimedia.org/T133292#2228923 (Dvorapa) @matej_suchanek Thank you, I updated the description [21:41:42] Analytics, MediaWiki-extensions-CentralNotice, Operations, Traffic: Generate a list of junk CN cookies being sent by clients - https://phabricator.wikimedia.org/T132374#2229057 (ori) Open>Resolved a:ori I captured about 20 minutes' worth of cookie names by running varnishlog on cp1066... [21:47:11] Analytics, MediaWiki-extensions-CentralNotice, Operations, Traffic: Generate a list of junk CN cookies being sent by clients - https://phabricator.wikimedia.org/T132374#2229103 (AndyRussG) Nice!! Thx much!! [21:49:21] Analytics: pagecounts-ez files are missing data - https://phabricator.wikimedia.org/T132761#2229104 (ezachte) The daily/monthly aggregates already use the newest data feed (aka Dan's webstatscollector 3.0, hadoop based) We should abandon the hourly file feeds, based on webstatscollector 1.0/2.0. Not the ag... [21:53:07] Analytics, MediaWiki-extensions-CentralNotice, Operations, Traffic: Generate a list of junk CN cookies being sent by clients - https://phabricator.wikimedia.org/T132374#2229110 (BBlack) Just noting here for posterity: since it sounds like we're potentially getting rid of cookies for future CN cam... [22:41:13] Analytics, Hovercards, Reading-Web-Sprint-71-m: Verify X-Analytics: preview=1 in stable - https://phabricator.wikimedia.org/T133067#2229375 (Nuria) @dr0ptp4kt it is unlikely that apps need to change. [22:42:46] madhuvishy: yt? seems that NOTHING happened on irc today [22:42:56] ha ha [22:42:57] yeah [22:43:09] Dan is out, Andrew was travelling [22:43:26] hiiii [22:43:28] there was a bit of post standup and then I guess europe folks leave early anyway [22:43:29] yo yo [22:43:49] ori! will be in office tomorrow and also next week most days [22:43:52] lunch some day? [22:47:17] Analytics-Kanban, Patch-For-Review: Add AQS endpoint for uniques - https://phabricator.wikimedia.org/T129518#2229399 (Nuria) [22:47:19] Analytics-Kanban: Document the new AQS Unique devices endpoint and launch it {bear} - https://phabricator.wikimedia.org/T129520#2229398 (Nuria) Open>Resolved [22:47:34] Analytics-Kanban: Document the new AQS Unique devices endpoint and launch it {bear} - https://phabricator.wikimedia.org/T129520#2108010 (Nuria) [22:47:36] Analytics-Kanban: Harvest unique devices endpoint request data like we do for pageviewAPI {bear} - https://phabricator.wikimedia.org/T132931#2229401 (Nuria) Open>Resolved [22:47:46] Analytics-Kanban: backfill pageviews for iOs app - https://phabricator.wikimedia.org/T132589#2229403 (Nuria) Open>Resolved [22:53:05] Analytics, DNS, Operations, Traffic: Create analytics.wikimedia.org - https://phabricator.wikimedia.org/T132407#2229410 (Nuria) >It will have to be new apache setup for prod ja, but since they will be hosted on a single domain, the puppetization doesn't need any knowledge of the subdirectories of... [22:53:37] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Add icinga process alerts for hive, oozie and mysql analytics-meta instance - https://phabricator.wikimedia.org/T133182#2229414 (Nuria) Open>Resolved [22:53:55] Analytics-Cluster, Analytics-Kanban, Operations, netops, Patch-For-Review: setup/deploy server analytics1003/WMF4541 - https://phabricator.wikimedia.org/T130840#2229427 (Nuria) Open>Resolved [22:53:59] Analytics-Cluster, Operations, hardware-requests: eqiad: New Hive / Oozie server node in eqiad Analytics VLAN - https://phabricator.wikimedia.org/T124945#2229428 (Nuria) [22:54:08] Analytics-Kanban, Patch-For-Review: Make reportupdater output emtpy values when query returns no results. - https://phabricator.wikimedia.org/T117537#2229429 (Nuria) Open>Resolved [22:54:22] Analytics-Kanban, Patch-For-Review: Make beeline easier to use as a Hive client {hawk} - https://phabricator.wikimedia.org/T116123#2229430 (Nuria) Open>Resolved [22:55:15] madhuvishy: how are you doing with mvn release plugin and jenkins? [22:56:34] nuria_: not too bad - i have some doubts which I'm going to ping hashar for in the task today - i'm also a bit sick with cold and fever so progress is a bit slower that I'd like [22:57:14] madhuvishy: ok, when I am done doing changes on VCL (hopefully later on today) i can be your rubber duck if you want [23:06:17] ottomata: yes totally [23:06:51] cool [23:30:15] Analytics, EventBus, Wikimedia-Stream: Public Event Streams - https://phabricator.wikimedia.org/T130651#2229503 (Krinkle) [23:44:23] (PS1) Krinkle: statsv: Start at offset LATEST instead of the default EARLIEST [analytics/statsv] - https://gerrit.wikimedia.org/r/284836 (https://phabricator.wikimedia.org/T132770) [23:44:59] (PS2) Krinkle: statsv: Start at offset LATEST instead of the default EARLIEST [analytics/statsv] - https://gerrit.wikimedia.org/r/284836 (https://phabricator.wikimedia.org/T132770) [23:45:17] (CR) Jforrester: "Thank you!" [analytics/statsv] - https://gerrit.wikimedia.org/r/284836 (https://phabricator.wikimedia.org/T132770) (owner: Krinkle) [23:45:26] (CR) Krinkle: "Tested on stat1002 to work as expected (first message is from around now). Also verified on stat1002 that without this, the first message " [analytics/statsv] - https://gerrit.wikimedia.org/r/284836 (https://phabricator.wikimedia.org/T132770) (owner: Krinkle) [23:45:38] Analytics-EventLogging, Operations, Performance-Team, Patch-For-Review: "Throughput of EventLogging NavigationTiming events" UNKNOWN - https://phabricator.wikimedia.org/T132770#2229550 (Krinkle) a:Krinkle [23:57:04] Analytics, Commons, Multimedia, Wikidata, and 3 others: Allow tabular datasets on Commons (or some similar central repository) (CSV, TSV, JSON, XML) - https://phabricator.wikimedia.org/T120452#2229568 (Jdforrester-WMF) >>! In T120452#2226245, @brion wrote: > Population of every US census place fo...