[00:34:00] (03PS1) 10Nuria: [WIP] Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [01:29:52] 10Analytics-Kanban, 10Analytics-Wikistats: Add link to new wikistats 2.0 to wikistats 1.0 pages - https://phabricator.wikimedia.org/T182001#3833100 (10Milimetric) Yes, we definitely need to rework that documentation. I have a bit too much on my plate now, I'll try to get to it soon. [01:31:09] 10Analytics-Kanban, 10Analytics-Wikistats: Privacy pageview threshold for map report - https://phabricator.wikimedia.org/T181508#3833101 (10Milimetric) I also notice that no country shows up that has fewer than 1.0K views. Is that another limit along with the 0.1%? [01:31:32] 10Analytics-Kanban: Load test druid backend via siege - https://phabricator.wikimedia.org/T182603#3833102 (10Milimetric) Docs are being updated here https://wikitech.wikimedia.org/wiki/Analytics/Systems/AQS/Scaling/LoadTesting#Druid_Info I still want to do a couple more tests, but I have an urgent issue to fix... [01:35:19] 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Software-Licensing: Add a license file to wikimetrics - https://phabricator.wikimedia.org/T60753#3833104 (10Milimetric) In talking this over, I noticed that a lot of repositories, such as the prominent puppet repository, do not have a LICENSE file. It seems to... [01:47:20] 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Software-Licensing: Add a license file to wikimetrics - https://phabricator.wikimedia.org/T60753#3833111 (10Legoktm) Yes and no. Anyone who works for the Wikimedia Foundation has a clause in their contract which states all of their code will be licensed under an... [02:23:59] 10Analytics-Kanban, 10Analytics-Wikistats: The laptop attempts a vertical take-off when loading ii.wikipedia.org - https://phabricator.wikimedia.org/T182700#3833205 (10Milimetric) p:05Triage>03High [02:24:17] (03PS1) 10Milimetric: Fix infinite loop [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397995 (https://phabricator.wikimedia.org/T182700) [06:33:30] 10Analytics, 10Discovery, 10EventBus, 10Wikidata, and 5 others: Create reliable change stream for specific wiki - https://phabricator.wikimedia.org/T161731#3833378 (10Smalyshev) @Nuria yes, consuming the data works. [09:22:56] heloooo [09:24:06] I just noticed that the top viewed articles in Wikistats2 is sorted by name desc, rather than by pageviews desc... [09:58:26] 10Analytics, 10Analytics-Wikistats: Sort "Top Viewed Articles" by views, rather than alphabetically - https://phabricator.wikimedia.org/T182757#3833579 (10mforns) [11:07:51] mforns: o/ [11:08:25] so I just noticed (since I didn't remember) that we already have the --no-whitelist parameter in the eventlogging_cleaner scritp [11:08:29] *script [11:08:44] so it is only a matter of triple checking on db1108 that the sanitization is doing fine [11:08:50] no data loss etc.. [11:09:27] and then we can deploy the data drop script to db1107 (EL master) [11:32:15] joal: o/ - mind to review https://gerrit.wikimedia.org/r/#/c/395923/2 ? [11:33:22] I think it is definitely worth to test [11:33:59] if you are ok I'd start deploying it on some nodemanagers now [11:36:04] mforns: omg wat [11:36:32] looking into it [11:37:06] this is happening both in dashboard and detail [11:40:29] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Make Wikistats 2's UI responsive - https://phabricator.wikimedia.org/T182764#3833808 (10fdans) [11:41:07] 10Analytics, 10Analytics-Wikistats: Make Wikistats 2's UI responsive - https://phabricator.wikimedia.org/T182764#3833822 (10fdans) [11:50:34] hey elukey I was away [11:54:49] elukey, cool, should I recheck db1108? [11:55:09] fdans, yes, it's probably a very simple thing? [11:56:30] mforns: if you have time that would be good, the cleaner has been running for a while so it is nice to check how it is doing :D [11:56:40] elukey, OK will do [11:57:53] elukey, good thing is nobody has complained about missing data :] [12:00:02] mforns: yet! :P [12:00:27] hehe, we're giving no reason to complain, that's why :] [12:45:02] Hi elukey - Sorry didn't notice the ping [12:45:33] elukey: I see no problem in updating the conf - I just don' know what effects it'll have on other things (seems no doc is available [12:52:26] fdans: question for you on WKS2 UI metrics organisation [12:53:25] elukey, I checked a good share of all tables that are not whitelisted, and they all either: are really old schemas (2013) and have no data at all, or are very new schemas (2017-11) and its min(ts) is closer than 90 days, or they are regular schemas and their min(ts) is sharp 90 days ago [12:53:33] in db1108 [12:53:40] I think we're good! [12:54:41] fdans: the "new pages" metric is under `contributing` section, while `edited-pages` is under `Content` section - It would make more sense to me to have new-pages in `content` - Any opinion? [12:56:14] elukey: have you seen my update for he script that checks for false-positive data-loss? [12:56:22] joal, +1 [12:56:41] Hi mforns_brb, thanks for the support ;) [13:00:36] mforns_brb: \o/ [13:00:44] joal: still in my todo, but it looks awesome! [13:01:00] not really awesome, but hopefully usefull :) [13:11:56] joal: I think there's even a case to be made for putting new-pages in content and edited-pages in contributing [13:12:16] interesting one :) [13:12:35] fdans: I'd like to actually regroup new-pages and edited-pages ... [13:13:16] joal to me edited pages speaks more about the activity of the community (hence contributing) [13:13:50] and new-pages speaks about the amount of stuff that has been created, which to me sounds like content [13:14:02] but it's a very blurry taxonomy I think joal [13:14:02] makes sense fdans [13:14:06] it is [13:18:50] * elukey lunch! [14:07:13] just restarted yarn nodemanagers on analytics102[8,9] to make them run with the new RSS calculation settings [14:09:44] k elukey [14:16:21] mornin yall [14:16:31] Yop milimetric [14:16:37] 10Analytics-Kanban: Top is ordering items by name and not by value - https://phabricator.wikimedia.org/T182772#3834138 (10fdans) [14:17:47] (03PS1) 10Fdans: Fix usage of _.sortBy in tops function [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/398049 (https://phabricator.wikimedia.org/T182772) [14:18:48] milimetric: mind taking a quick look at that? ˆ [14:19:05] (morning!) [14:19:23] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Enable more accurate smaps based RSS tracking by yarn nodemanager - https://phabricator.wikimedia.org/T182276#3834160 (10elukey) p:05Triage>03Normal [14:20:06] (03CR) 10Milimetric: [C: 032] Fix usage of _.sortBy in tops function [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/398049 (https://phabricator.wikimedia.org/T182772) (owner: 10Fdans) [14:20:30] thank youuuu milimetric! [14:20:44] fdans: take a look at my patch too, for the infinite loop https://gerrit.wikimedia.org/r/#/c/397995/ [14:20:51] yeah I was on it now milimetric [14:21:08] amusingly, it was happening for all wikis, not just "closed" ones, but the closed ones were repeating it forever [14:45:52] elukey: hiiiiii [14:45:57] fdans: heh, the top sorting bug causes funny double takes: https://stats.wikimedia.org/v2/#/en.wiktionary.org [14:45:58] did you want to do a thing with me this morning? kafka1023? [14:46:04] and/or canary? [14:46:41] ottomata: hiiiii [14:46:51] so no great news for this morning [14:47:22] there is still an issue in reimaging jessie hosts (https://phabricator.wikimedia.org/T182702) that we are debugging, so kafka1023 is not ready [14:47:37] and te canary code review is still pending the traffic's +1 [14:47:39] :( [14:49:23] haha ok :) [14:50:29] buut the kafka1023 rename is complete [14:50:43] so it is only a matter of getting jessie to run properly, format the partitions [14:50:49] and we are good to go [14:52:04] ayye k [14:52:52] the disks are configured as single RAID-0 Virtual Devices with 1 disk from the raid hw controller [14:53:10] I didn't find a way to configure them via the PERC interface with JBOD [14:53:56] haha, GIVE UP! LUCA this is only one node that will be around for a few months [14:54:00] hahah [14:54:01] OH [14:54:08] sorry, you can't give up? [14:54:11] this isn't partman. [14:54:17] milimetric: HAHAH love that omg [14:57:36] ottomata: nono I can give up, I already did, I was informing you :D [14:57:37] joal / elukey / fdans: wow, Druid serves daily results from its monthly caches. Check out that super tiny bump in cache misses in the last few minutes, that was the first time I pushed it with daily requests: https://grafana.wikimedia.org/dashboard/db/prometheus-druid?orgId=1&var-datasource=eqiad%20prometheus%2Fanalytics&var-cluster=druid_public&var-druid_datasource=All [14:58:03] haa :) [14:58:09] wow! [15:00:44] 10Analytics-Kanban: Load test druid backend via siege - https://phabricator.wikimedia.org/T182603#3834249 (10Milimetric) [15:01:48] oh coool milimetric [15:21:15] joal: do you have som eminutse to brain bounce a json refine thing with me before ops sync starts? [15:22:58] ottomata,joal - I'd use this 30min time to progress the kafka1023 issue, do you mind if I skip the meeting? [15:23:46] 10Analytics-Kanban, 10Analytics-Wikimetrics, 10Software-Licensing: Add a license file to wikimetrics - https://phabricator.wikimedia.org/T60753#3834471 (10Milimetric) Thanks very much for the explanation, @Legoktm [15:24:04] elukey: fine with me :) [15:25:07] Goran ran into a weird thing: any idea why R shelling out to hive would work from a cron, but R shelling out to beeline would not work? [15:29:30] (03PS2) 10Ricordisamoa: Load Open Sans from fontcdn [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/371514 [15:30:24] np elukey [15:30:36] ottomata: Let's brain-bounce now if you want :) [15:31:55] ok [15:35:56] actually milimetric, cache in druid are segment based - When your segment is cached, whether computing daily or monthly doesn't matter :) [15:36:37] oh yeah, makes sense joal [15:36:49] kind of obvious, but still very cool [15:37:00] not obvious to me, nice to know! [15:37:23] it means that as long as the caches aren't super cold, Druid will serve about 100 requests per second for at least 3 years for any metric on any project [15:37:25] that's a good result [15:37:38] milimetric: not obvisou for me either, I'm happy I had read this before to show off ;) [15:37:43] still slower than Cassandra, but also a much smaller cluster [15:37:48] it is milimetric [15:38:07] What I hope is that a full-time query won't unload the cache from previous values [15:38:29] let's try it :) Got a good idea for how to hammer it? [15:38:31] milimetric: not that smaller, we obtained very fast results with only 3 nodes [15:38:35] full time for one metric for all projects? [15:38:41] that's not very realistic [15:38:46] ottomata: do we have a task to use for the vk test instance? [15:38:50] but that's cassandra's point - superfast, but no computation [15:39:09] oh, that's true, I forgot the Cassandra load test was before the cluster increase [15:39:49] so joal, shall I try loading editors monthly since beginning for all projects? That should be more than it can cache [15:40:56] milimetric: I'd actually like if you could insert like 1 or 2 % urls that query druid for a very-long time period, in the middle of other more regular queries with revent times [15:41:18] milimetric: Idea is: how do those queries of very long-range affect ache [15:41:22] oh, sure, cool, will do [15:41:29] Thanks a lot mate: ) [15:42:32] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3834523 (10ggellerman) @Nuria: Here is a start to respond to request from @Aklapper for a description and a name for the space. Please review/edit or approve. Thanks! Descrip... [15:47:29] (03CR) 10Fdans: [V: 032 C: 032] Fix infinite loop (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397995 (https://phabricator.wikimedia.org/T182700) (owner: 10Milimetric) [15:52:43] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3834561 (10Nuria) One small amend: Description: Space used by the Analytics and Techops team for reviewing incoming requests from WMF-Legal that need to remain private. Name:... [15:55:45] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3834564 (10ggellerman) Thanks, @Nuria! @Aklapper do you have what you need to create this space? [15:57:36] 10Analytics, 10Discovery, 10EventBus, 10Wikidata, and 5 others: Create reliable change stream for specific wiki - https://phabricator.wikimedia.org/T161731#3834569 (10Nuria) @Smalyshev Ok, we aim to have the cluster handling all prod traffic by end of next quarter, until then it will be mirroing data whic... [16:00:17] ping elukey [16:00:41] joining, it is 17:00 :) [16:03:48] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3569915 (10greg) >>! In T174675#3834564, @ggellerman wrote: > @Aklapper do you have what you need to create this space? >>! In T174675#3830789, @Aklapper wrote: > Also, I assume... [16:03:50] 10Analytics, 10DC-Ops, 10Operations, 10ops-codfw, 10Patch-For-Review: Decomission eventlog2001 - https://phabricator.wikimedia.org/T182397#3834590 (10Papaul) [16:15:36] (03PS2) 10Nuria: [WIP] Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [16:17:07] (03PS3) 10Nuria: [WIP] Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [16:29:13] joal wanna go to batcave 2 and keep brain bouncing? i'm close i think... [16:39:37] 10Analytics-Kanban: Load test druid backend via siege - https://phabricator.wikimedia.org/T182603#3834727 (10Milimetric) Conclusion is, basically, the more you query, the better the caches get. And it seems we have room in the cache for the full history, so basically Druid should stand up to any abuse we can th... [16:40:03] Nice milimetric --^ [16:40:06] Yay [16:41:00] yeah, joal, it did 143 requests per second for the tricky test [16:41:09] which is the fastest yet [16:41:20] meaning the first time it hit a full-history thing, it probably just loaded all of it [16:41:37] so, hats off to you and elukey for configuring and setting up this monster [16:41:44] it is ready to meet the real world :) [16:42:02] I'm super surprised about that response time [16:42:07] But that's great :) [16:42:33] me too, it seems really awesome [16:42:36] milimetric: it means caching is handled correctly in druid - sounds reasonnable, good to have tested :) [16:42:54] milimetric: are the prometheus metrics consistent or did you find some weirdness? [16:42:57] elukey: And cache size is not even that big [16:43:07] elukey: like 200Mb [16:43:14] Not even at G yet [16:43:36] broker cache right? [16:43:44] no eviction yet - this means no need to remove from cache yet, sounds really good [16:44:08] woah! yeah, it's doing all this with 200MB!!! [16:44:10] how?! [16:44:16] well, 600MB spread over the cluster [16:44:19] milimetric: columnar ;) [16:44:31] yeah, a-w-e-some [16:44:38] super good indeed [16:44:48] I'm happy we went for that choice :) [16:45:03] all query response times under 500ms [16:45:09] this is worth a blog post for *sure* [16:45:12] also usually the broker end response is smaller than the data that needs to collect from historicals right? [16:45:36] (because it aggregates etc..) [16:45:49] yeah, and filters, you're right elukey [16:45:58] elukey: thing I wonder is how can we see historical cache [16:46:21] elukey: but I'm asking for every project and every metric [16:46:37] so that should offset any gains from filtering that the broker cache might have [16:46:50] milimetric: queries are always the same - no variability, result in cache :) [16:47:37] The last hists, there was p90 response time of 10ms :d [16:47:39] :D [16:47:44] Incredible [16:49:10] I checked the historical /var/log/historical-metrics.log metrics, and they seem consistent [16:49:26] I'd also check the query time ones just to be sure :) [16:49:34] elukey: no metrics in charts? I'm surprised [16:49:57] elukey: ;) [16:56:36] joal: ?? [16:57:18] elukey: you chart every thing, and you looked at the metrics in logs - making fun [16:57:27] * elukey cries in a corner [16:57:40] * joal pads elukey on the back [16:58:12] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3834783 (10ggellerman) I just had a loo at the [[ URL | WMF-Legal ]] board. I think that a separate board would make sense. The proj name could also be Analytics-Legal. [16:58:18] so if anybody wants to read the immense joy in reimaging servers, https://phabricator.wikimedia.org/T182702 [16:58:51] last reimage in progress, fingers crossed [16:59:10] 10Analytics-Kanban, 10Analytics-Wikistats: Add link to new wikistats 2.0 to wikistats 1.0 pages - https://phabricator.wikimedia.org/T182001#3834787 (10Erik_Zachte) Reports with new announcement are being generated. The English Wikipedia dump is still being parsed so I will update reports again later this week.... [17:16:39] kafka1023 finally installed [17:16:41] \o/ [17:16:48] running puppet now [17:21:47] * joal bows to master elukey [17:22:02] ottomata: other meeting cancelled - batcave? [17:24:52] ok milimetric - read about caching in druid again and realised i made a mistake when saying "semgent caching" [17:25:22] milimetric: query results are cached, but at segment level- meaning they can be reused as part of largber queries [17:25:33] milimetric: segments are not cached entirelly [17:27:59] oh joal ys [17:28:04] some moreinteresting findings [17:28:11] in da cave ottomata [17:30:15] 10Analytics, 10DC-Ops, 10Operations, 10ops-codfw, 10Patch-For-Review: Decomission eventlog2001 - https://phabricator.wikimedia.org/T182397#3834916 (10Papaul) [17:38:21] 10Analytics: Remove request for font.googleapis.com from analytics.wikimedia.org - https://phabricator.wikimedia.org/T182804#3834944 (10Nuria) [17:39:19] 10Analytics-Kanban, 10Analytics-Wikistats: Add link to new wikistats 2.0 to wikistats 1.0 pages - https://phabricator.wikimedia.org/T182001#3834957 (10Nuria) Super thanks, Erik. [17:42:24] ottomata: kafka1023 should be ready to go! (just created the spool partitions) [17:42:46] if you want to double check later on and let me know if everything looks good it would be great [17:42:48] need to go now :) [17:43:20] * elukey off! [17:48:05] (03PS2) 10Jforrester: Update for new Edit schema version 17520312 [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397626 (https://phabricator.wikimedia.org/T182620) [17:48:14] (03PS2) 10Jforrester: Drop support for the Edit_11448630 schema revision [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397627 [17:48:19] (03PS2) 10Jforrester: Swap archaïc code to use current tables [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397628 [17:54:41] nice elukey [17:57:00] 10Analytics, 10Discovery, 10EventBus, 10Wikidata, and 5 others: Create reliable change stream for specific wiki - https://phabricator.wikimedia.org/T161731#3835021 (10Smalyshev) @Nuria yes mostly, though I do have some questions, maybe we should set up a short meeting to discuss them? [18:02:04] 10Analytics, 10Analytics-Wikistats: Vital Signs: Please provide an "all languages" de-duplicated stream for the Community/Content groups of metrics - https://phabricator.wikimedia.org/T120037#3835062 (10Nuria) @Jdforrester-WMF Closing this ticket, consensus among research team is not to use rolling metrics but... [18:02:13] 10Analytics, 10Analytics-Wikistats: Vital Signs: Please provide an "all languages" de-duplicated stream for the Community/Content groups of metrics - https://phabricator.wikimedia.org/T120037#3835063 (10Nuria) 05stalled>03Resolved [18:03:33] 10Analytics, 10Analytics-Wikistats: Vital Signs: Please make the data for enwiki and other big wikis less sad, and not just be missing for most days - https://phabricator.wikimedia.org/T120036#3835080 (10Nuria) @Jdforrester-WMF Closing this ticket, consensus among research team is not to use rolling metrics bu... [18:03:45] 10Analytics, 10Analytics-Wikistats: Vital Signs: Please make the data for enwiki and other big wikis less sad, and not just be missing for most days - https://phabricator.wikimedia.org/T120036#3835081 (10Nuria) 05stalled>03Resolved [18:06:00] (03CR) 10Nuria: [C: 031] Update for new Edit schema version 17520312 [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397626 (https://phabricator.wikimedia.org/T182620) (owner: 10Jforrester) [18:07:10] 10Analytics, 10Discovery, 10EventBus, 10Wikidata, and 5 others: Create reliable change stream for specific wiki - https://phabricator.wikimedia.org/T161731#3835103 (10Nuria) @Smalyshev Please, 45 minutes with me and @Ottomata would do? [18:12:22] ottomata: i've updated the schema:popups timestamp field to be an integer, do we need to restart the validator processes? [18:14:36] 10Analytics-Kanban: Initial Launch of new Wikistats 2.0 website - https://phabricator.wikimedia.org/T160370#3835148 (10Nuria) [18:14:40] 10Analytics-Kanban, 10Analytics-Wikistats: Implement pageview metric in Wikistats UI - https://phabricator.wikimedia.org/T163817#3835147 (10Nuria) 05Open>03Resolved [18:15:16] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2.0 UI second deployment/iteration - https://phabricator.wikimedia.org/T170460#3835151 (10Nuria) 05Open>03Resolved [18:15:18] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2.0. - https://phabricator.wikimedia.org/T130256#3835152 (10Nuria) [18:16:46] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2.0. - https://phabricator.wikimedia.org/T130256#2131074 (10Nuria) [18:16:48] 10Analytics-Kanban: Initial Launch of new Wikistats 2.0 website - https://phabricator.wikimedia.org/T160370#3835155 (10Nuria) 05Open>03Resolved [18:17:59] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2.0. - https://phabricator.wikimedia.org/T130256#3835158 (10Nuria) [18:18:10] 10Analytics-Kanban: Initial Launch of new Wikistats 2.0 website - https://phabricator.wikimedia.org/T160370#3096561 (10Nuria) 05Resolved>03Open [18:30:15] phuedx: no, but you probably should update your clients so that the schema revision is your new revision [18:30:36] unfortunely eventlogging will use the revision in the data to validate, so events with decimals and the old schema revision will validate [18:30:38] i thikn. [18:30:49] i've got some ideas on how to make the JsonRefine (to hive) process more lenient [18:31:24] at the very least for long -> double conversion, pOSSibly for double -> long, or at the very least i think i can get spark to drop records that are invalid [18:31:27] 10Analytics, 10Discovery, 10EventBus, 10Wikidata, and 5 others: Create reliable change stream for specific wiki - https://phabricator.wikimedia.org/T161731#3835222 (10Smalyshev) yes, definitely [18:33:34] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3835277 (10Ottomata) I wonder if this description shouldn't explicitly call out analytics and ops; this space is more a general space for private WMFl-Legal related (tech?) tasks. [18:40:41] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3835349 (10Nuria) I do not think so, main usage will be managing tasks for legal holds which is of concern of analytics and ops, I do not think space should be super generic [18:40:53] ottomata: 👍 [18:43:27] fdans: did you guys deployed fix for infinite loop? [18:49:18] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3835389 (10Ottomata) Why though? Tilman is not on the 'analytics' team, but will need access. It isn't inconceivable that we might have to loop others teams (community folks?)... [18:56:25] nuria_: sorry my chrome notifications suck lately, I'm on it now [19:01:40] (03PS1) 10Fdans: Release 2.1.1 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/398097 [19:02:08] (03CR) 10Fdans: [V: 032 C: 032] Release 2.1.1 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/398097 (owner: 10Fdans) [19:02:59] fdans: ok, thank you [19:03:36] (03PS1) 10Fdans: Release 2.1.1 [analytics/wikistats2] (release) - 10https://gerrit.wikimedia.org/r/398100 [19:04:11] (03CR) 10Fdans: [V: 032 C: 032] Release 2.1.1 [analytics/wikistats2] (release) - 10https://gerrit.wikimedia.org/r/398100 (owner: 10Fdans) [19:04:36] nuria_: it'll be deployed in a few mins when the job runs [19:15:05] 10Analytics, 10DC-Ops, 10Operations, 10ops-codfw: Decomission eventlog2001 - https://phabricator.wikimedia.org/T182397#3835450 (10RobH) 05Open>03Resolved a:05Papaul>03None [19:15:28] (03CR) 10Zhuyifei1999: [C: 032] Load Open Sans from fontcdn [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/371514 (owner: 10Ricordisamoa) [19:15:46] (03Merged) 10jenkins-bot: Load Open Sans from fontcdn [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/371514 (owner: 10Ricordisamoa) [19:22:28] fdans: would you also be so kind as to test teh webpack patch? [19:22:40] fdans: i am testing it with safari and FF [19:23:09] fdans: but i think another pair of eyes could help [19:23:13] cc milimetric [19:23:41] nuria_: I’m off for the day but will do first thing tomorrow [19:23:54] fdans: sounds good thanks [19:26:32] I’ll take a look after I sqoop this data [19:42:20] (03CR) 10Jforrester: [C: 04-1] "Revise to 17541122." [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397626 (https://phabricator.wikimedia.org/T182620) (owner: 10Jforrester) [19:43:08] 10Analytics: Whilelist analytics.wikimedia.org and stats.wikimedia.org in add blockers - https://phabricator.wikimedia.org/T182816#3835518 (10Nuria) [19:43:38] 10Analytics: Whilelist analytics.wikimedia.org and stats.wikimedia.org in add blockers - https://phabricator.wikimedia.org/T182816#3835528 (10Nuria) Request it here: https://forums.lanik.us/ [19:44:51] (03PS3) 10Jforrester: Update for new Edit schema version 17541122 [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397626 (https://phabricator.wikimedia.org/T182620) [19:44:53] (03PS3) 10Jforrester: Drop support for the Edit_11448630 schema revision [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397627 [19:44:56] (03PS3) 10Jforrester: Swap archaïc code to use current tables [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397628 [19:45:23] nuria_: Do you mind if we move forward with publishing clickstream data? [19:45:46] milimetric: i think fdans deployed your fix but it seems to still be happening (browser freeze) [19:45:57] joal: no, as long as we have docs [19:46:15] nuria_: https://meta.wikimedia.org/wiki/Research:Wikipedia_clickstream#Releases [19:46:57] nuria_: analytics dumps HTML page has also been updated in puppet patch [19:47:13] joal: maybe a link on analytics/data that links to this? [19:47:32] nuria_: And I'm waiting for publishing to happen before asking Dario to update Figshare with a link to dumps [19:47:40] Sure nuria, I'll add that [19:48:55] hey folks, great stuff! I’m currently hosting our monthly showcase, drop me a line if/when I can help [19:48:57] Done [19:49:46] 10Analytics: Whilelist analytics.wikimedia.org and stats.wikimedia.org in add blockers - https://phabricator.wikimedia.org/T182816#3835538 (10Nuria) Looks like pageview api could also use whitelisting: https://forums.lanik.us/viewtopic.php?f=64&t=29643&p=90511&hilit=Wikipedia#p90511 [19:50:43] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3569915 (10Aklapper) * Regarding the project and its workboard: ** **Public** project #Analytics-Legal created: https://phabricator.wikimedia.org/project/view/3153/ ** Anyone can... [20:02:39] 10Analytics: Whitelist analytics.wikimedia.org and stats.wikimedia.org in add blockers - https://phabricator.wikimedia.org/T182816#3835566 (10Nuria) [20:04:10] 10Analytics-Kanban, 10Documentation: Document Dashiki - https://phabricator.wikimedia.org/T182477#3824562 (10Nuria) Also please add license "All data, charts, and other content is available under the Creative Commons CC0 dedication." [20:04:32] will do, nuria_ [20:04:39] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3835569 (10Aklapper) 05Open>03Resolved * Regarding the Space (a Space is **not** a project and hence there is no workboard): ** For access control, created [[ https://phabric... [20:05:28] 10Analytics, 10Analytics-Wikistats: wikistats rendering bug - https://phabricator.wikimedia.org/T182817#3835571 (10Nuria) [20:25:06] 10Analytics, 10Phabricator: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3835639 (10ggellerman) Thanks, @Aklapper ! [20:30:16] 10Analytics, 10Analytics-Wikistats: wikistats rendering bug - https://phabricator.wikimedia.org/T182817#3835665 (10kaldari) [20:30:19] 10Analytics, 10Analytics-Wikimetrics: Layout bug in Safari for detail page - https://phabricator.wikimedia.org/T182821#3835666 (10Nuria) [20:31:26] (03CR) 10Milimetric: [WIP] Splitting webpack bundle (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) (owner: 10Nuria) [20:36:29] 10Analytics, 10Analytics-Wikimetrics: Layout bug in Safari for detail page - https://phabricator.wikimedia.org/T182821#3835686 (10Nuria) https://stats.wikimedia.org/v2/#/es.wikipedia.org/reading/unique-devices Maybe version of safari too old? (7.*) [20:37:22] 10Analytics, 10Analytics-Wikimetrics: Layout bug in Safari for detail page - https://phabricator.wikimedia.org/T182821#3835687 (10Nuria) @fdans can you check link in safari? [20:40:06] (03PS4) 10Nuria: [WIP] Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [20:41:47] (03CR) 10Milimetric: "Feels smooth in Chrome and Firefox on Ubuntu 2016. One comment." (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) (owner: 10Nuria) [21:02:50] (03CR) 10Milimetric: [V: 032 C: 032] Drop support for the Edit_11448630 schema revision [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397627 (owner: 10Jforrester) [21:05:01] (03CR) 10Milimetric: [V: 032 C: 032] Update for new Edit schema version 17541122 [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397626 (https://phabricator.wikimedia.org/T182620) (owner: 10Jforrester) [21:05:38] (03CR) 10Milimetric: [V: 032 C: 032] Revert "Adding renamed tables to sql union statements" [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397625 (owner: 10Jforrester) [21:09:16] (03CR) 10Milimetric: "Jforrester, yes, if you want to help push this patch forward: https://gerrit.wikimedia.org/r/#/c/346915/" [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/301432 (https://phabricator.wikimedia.org/T115119) (owner: 10Milimetric) [21:28:17] milimetric: Question from Melody: On the dashboard, name is Wikimedia Statistics - Do we change that to Wikistats 2? [21:29:00] joal: hm... good question. stats.wikimedia.org says Wikimedia Statistics [21:29:21] I have no strong opinion, but it's interesting to think about it :) [21:29:22] but if we refer to it as Wikistats 2 everywhere, people might not know what the title means [21:29:39] either way we really need an About page where we can explain all this stuff [21:29:57] milimetric: as per nuria/mel talk, I updated every doc on wikitech to reference "wikistats 2" (or most of what I found) [21:30:02] in any case, I shouldn't make this decision, we should talk tomorrow [21:30:26] works for me milimetric :) [21:30:42] milimetric: trying to prepair context for tomorrows launch [21:32:12] joal: I can hang out with nuria later and we can look at it both ways, and pick one [21:32:21] sounds good :) [21:32:25] I'm kind of inclined to go for Wikimedia Statistics (which is what Wikistats is short for) [21:38:37] 10Analytics-Kanban, 10Analytics-Wikistats: Privacy pageview threshold for map report - https://phabricator.wikimedia.org/T181508#3835851 (10Erik_Zachte) - For a Wikipedia to be shown, it has to have a minimum of 0.1% of all traffic in pageviews. Yes. That happens in perl job that prepares csv files: Traffic... [22:25:57] joal: per latest milimetric suggestion Wiki(media) stats(istics) [22:26:10] i actually like that one although might be too geeky [22:30:42] It's a bit geeky :) [22:30:55] I have no strong opinion :) [22:31:00] joal: ya it is [22:31:28] we played a bit with graying out the media and istic parts of WIkimedia Statistics, we can look that over tomorrow [22:31:52] i agree with milimetric with "Wikimedia Statistics " on website and nickname wikistats [22:35:59] Good :) [22:36:41] nuria_, milimetric: I just uploaded that diagram: https://wikitech.wikimedia.org/wiki/File:Wikistats_2_Backend.png [22:37:07] joal: nice [22:37:36] nice diagram [22:37:44] Please review when you have time, I think of adding to https://wikitech.wikimedia.org/wiki/Analytics/Wikistats_2#Architecture [22:41:46] Done [22:46:39] ok, enough for me toniught [22:46:47] See you tomorrow a-team [22:47:18] nuria_, milimetric - If there is something you'd like me to do tomorrow morning about launch, please drop me an email :) [22:48:04] milimetric, joal: i was thinking i will write to lists later on today so it is really TOMORROW [22:48:10] in teh rest of the world [22:48:30] good with me, nuria [22:48:34] please [22:49:03] nuria_: just thinking of it - wan't tomorrow the launch of the Wikimedia Loves monument ? [22:49:28] joal: yes, but i do not think that should be a problem for e-mail lists [22:49:32] for the blog yes [22:49:45] ok sounds good :) [22:49:50] joal: but melody can put blogpost up at her leisure [22:49:55] joal: there is no rush [22:50:10] perfect [22:50:40] Gone to sleep then - will monitor Druid cache and AQS requests tomorrow morning :) [23:18:36] milimetric: You're awesome, BTW. :-) [23:19:01] James_F: donno what you're talkin' about, but I'll TAKE it! [23:19:02] :) [23:19:19] * James_F grins. [23:30:27] James_F: i closed those two tickets you opened a million years ago about vital-signs [23:55:24] (03PS5) 10Nuria: [WIP] Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [23:57:54] (03PS6) 10Nuria: Splitting webpack bundle [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/397989 (https://phabricator.wikimedia.org/T182601) [23:58:45] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Support multi DC statsv - https://phabricator.wikimedia.org/T179093#3836254 (10Krinkle) @Ottomata Replied at