[02:14:45] (03PS2) 10Milimetric: [WIP] Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) [02:15:00] nuria: you working? [02:15:42] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [02:47:47] (03PS3) 10Milimetric: Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) [03:04:12] (03CR) 10Milimetric: "Tested against an hour of webrequest. Looks good, no mismatches except the newly flagged wikimania and /page/mobile-sections/ URLs. One " (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [03:05:14] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Milimetric) Added a comment here for the Inuka folks, do ping me on (I don't know what the hell we're doing for chat these... [04:33:32] 10Analytics, 10Technical blog: Setup Matomo beacon for techblog.wikimedia.org - https://phabricator.wikimedia.org/T247918 (10bd808) [04:33:59] 10Analytics, 10Technical blog: Setup Matomo beacon for techblog.wikimedia.org - https://phabricator.wikimedia.org/T247918 (10bd808) p:05Triage→03High [05:32:15] (03PS3) 10Lex Nasser: Configure geoeditors monthly public Oozie job to work with geoeditors public monthly table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/576618 (https://phabricator.wikimedia.org/T244597) [05:38:51] 10Analytics, 10Technical blog: Setup Matomo beacon for techblog.wikimedia.org - https://phabricator.wikimedia.org/T247918 (10bd808) We will try using https://wordpress.org/plugins/wp-piwik/ to manage the tracker script integration with the blog. To do this we will apparently need [[https://matomo.org/blog/2015... [06:11:17] 10Quarry, 10DBA, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Marostegui) >>! In T246970#5977262, @Mike_Peel wrote: > Looking at CPU usage at https://grafana.wikimedia.org/d/000000607/cluster-overview?orgId=1&from=now-30d&to=now I can't... [07:18:12] 10Analytics, 10Product-Analytics (Kanban): SQL definition for wikidata metrics for tunning session - https://phabricator.wikimedia.org/T247099 (10jwang) @Isaac Thank you so much for sharing your ideas and suggestions! Regarding "SQL the number of total Wikidata entities", yes, I was trying to get an estimate... [07:57:45] 10Analytics, 10Event-Platform, 10serviceops, 10Patch-For-Review, 10Wikimedia-production-error: Lots of "EventBus: Unable to deliver all events" - https://phabricator.wikimedia.org/T247484 (10Joe) 05Open→03Resolved With the retries we're down to a background noise of ~ 10 events / hour, which should b... [09:27:18] 10Analytics, 10Event-Platform, 10serviceops, 10Patch-For-Review, 10Wikimedia-production-error: Lots of "EventBus: Unable to deliver all events" - https://phabricator.wikimedia.org/T247484 (10Joe) 05Resolved→03Open The numbers are higher than 10/hour, more like 60 / hour. I did some more digging and... [09:44:47] 10Analytics, 10Event-Platform, 10serviceops, 10Patch-For-Review, 10Wikimedia-production-error: Lots of "EventBus: Unable to deliver all events" - https://phabricator.wikimedia.org/T247484 (10Joe) https://nodejs.org/api/http.html#http_server_keepalivetimeout suggests the keepalive timeout is 5 seconds by... [10:36:18] joal: I’d appreciate a review on https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/580161/ and later maybe we can chat a bit about that whole part of our code. I see a bigger refactor in our future :) [10:44:52] 10Analytics, 10Operations, 10Research, 10Traffic, and 2 others: Enable layered data-access and sharing for a new form of collaboration - https://phabricator.wikimedia.org/T245833 (10Miriam) [11:31:51] * elukey lunch! [11:51:27] (03CR) 10Fdans: "@nuria I'm not sure how it happened, but the removal of the method got mixed with the bundling change and it's now on master so this can b" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577531 (https://phabricator.wikimedia.org/T212032) (owner: 10Fdans) [11:51:29] (03CR) 10Fdans: [C: 03+2] Remove graph trend from Wikistats detail page [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577531 (https://phabricator.wikimedia.org/T212032) (owner: 10Fdans) [13:13:01] 10Analytics, 10Pageviews-Anomaly: Topviews Analysis of the Hungarian Wikipedia is flooded with spam - https://phabricator.wikimedia.org/T237282 (10Bencemac) >>! In T237282#5976508, @Nuria wrote: > Update on this, we have deployed our identifying code to the pageview pipeline and it is being run on shadow mode... [13:17:35] (03Abandoned) 10Joal: Add dependency to Vegas (charts drawing) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580077 (owner: 10Joal) [13:53:55] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10SBisson) @Milimetric I would like to discuss here if you don't mind. From the patch in gerrit: > Android devices are acc... [13:55:10] (03CR) 10Nuria: [C: 03+2] "let's please do a follow up change to remove the unused method from the graph model" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/577531 (https://phabricator.wikimedia.org/T212032) (owner: 10Fdans) [13:55:13] (03CR) 10Joal: "I looked at technical aspects of the code, not functional ones. One not (import), but except from that looks great :)" (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [13:58:51] * elukey bbiab [13:59:08] elukey: today in ops sync let's talk about some newpyter stuff! [14:01:31] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-Focus-Sprint): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Rileych) [14:05:29] nuria: the method is already removed, i just put that change with a different one that’s already on master [14:05:51] fdans: I see, ok. [14:12:24] ottomata: sure! [14:13:25] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Nuria) @Milimetric the distinction is no different than the one it happens now across IOS and android, in pageview_hourly,... [14:15:25] (03CR) 10Nuria: "Deleted comment I did not mean to submit, sorry about that. Please add unit tests for the new methods." (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [14:15:30] (03CR) 10Nuria: [C: 04-1] Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [14:17:31] 10Analytics, 10Analytics-Kanban, 10Multimedia, 10Tool-Pageviews: Fix double encoding of urls on mediarequests api - https://phabricator.wikimedia.org/T244373 (10Nuria) There are a few outstanding issues with the mediarequests data that we need to solve for next quarter but this very one can be closed, tha... [14:18:58] nuria: the new methods are private, I added unit tests for the public methods to cover the new cases, and importantly the old unit tests should be covering everything [14:22:22] (03CR) 10Milimetric: Detect pageviews as requested by KaiOS (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [14:22:50] (03PS4) 10Milimetric: Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) [14:24:00] joal: any opinion on additional unit testing for this change, re: what nuria said? I didn't add tests for the private methods, though I suppose I could hack some in if that's the standard we use [14:24:20] (I'm just trying to merge this today so Fran can deploy source) [14:24:55] (03CR) 10Nuria: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [14:26:48] (03CR) 10Milimetric: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [14:30:31] milimetric: private methods should be tested as well with unit tests, unit tests are used to test individual code components and private functions are an example of those. [14:32:03] nuria: that’s not usual practice, as this and others explain: https://anthonysciamanna.com/2016/02/14/should-private-methods-be-tested.html [14:32:46] private functions are part of the black box, you generally test only the interfaces to the box as private methods can be refactored and that makes the tests brittle [14:35:56] 10Quarry, 10DBA, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Mike_Peel) >>! In T246970#5978191, @Marostegui wrote: > Not sure if you were looking at the right side, the host involved here is labsdb1011, which has a high CPU usage @Maro... [14:37:08] 10Quarry, 10DBA, 10Performance Issue: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10Marostegui) Keep in mind that you are using a shared resource, which is currently being used a lot: https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&from=now-12h&to=now&var-dc=eqia... [14:37:25] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10Marostegui) [14:37:32] haha nuria testing private methods was why I didn't want them to be private when we were debating SchemaLoader code long ago :) [14:38:39] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10MBH) May it be the same as T246970? [14:39:02] 10Quarry, 10DBA, 10Data-Services: Quarry: Lost connection to MySQL server during query - https://phabricator.wikimedia.org/T246970 (10Marostegui) >>! In T246970#5979745, @Mike_Peel wrote: >>>! In T246970#5978191, @Marostegui wrote: >> Not sure if you were looking at the right side, the host involved here is... [14:40:15] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10Marostegui) >>! In T247978#5979761, @MBH wrote: > May it be the same as T246970? Yes, most likely. This might be useful for your situation too: T246970#5977729 [14:43:31] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Definition of not text content metrics for tunning session - https://phabricator.wikimedia.org/T247417 (10Nuria) [14:43:56] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Definition of not text content metrics for tunning session (rich media,: images and the linke) - https://phabricator.wikimedia.org/T247417 (10Nuria) [14:46:56] 10Analytics, 10Discovery-Search (Current work): Copy English Wikipedia drafttopic scores to other wikis somewhere in the CirrusSearch pipeline - https://phabricator.wikimedia.org/T241015 (10TJones) 05Open→03Resolved [14:50:59] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10nshahquinn-wmf) > Android devices are accessing the /page/mobile-sections/ endpoint too @Milimetric will your patch cause... [14:55:06] 10Analytics, 10Operations, 10Traffic, 10Patch-For-Review: Test atskafka deployment - https://phabricator.wikimedia.org/T247497 (10elukey) ` elukey@kafka-jumbo1001:~$ kafka acls --add --allow-principal User:CN=varnishkafka --producer --topic atskafka_test_webrequest_text kafka-acls --authorizer-properties z... [15:00:16] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Nuria) >@Milimetric will your patch cause us to start classifying these Android app webrequests as pageviews? It would not... [15:00:57] elukey: still WIP https://docs.google.com/document/d/1r-oqMXViWvQCqsYz0qzezZBWpip8LvkvCGF6GivFB_8/edit# [15:01:04] if you have a chance look it over before our sync! [15:08:29] done :) [15:12:18] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10Marostegui) I have tried your query on the other hosts and it takes around 2-3 minutes to complete. But it is also true that they are way less loaded. Just a quick command: ` [15:11:29] maros... [15:30:54] nuria: FYI i've added both eventgate and flink k8s hw resource requests to our hw spreadsheet [15:31:03] (just talked with zbysko) [15:31:16] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10bd808) >>! In T247978#5979931, @Marostegui wrote: > The only thing I can think of to discard labsdb1011's specific host issues, would be to point quarry to another host for a few days (we'd n... [15:31:19] (03CR) 10Nuria: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [15:33:44] milimetric: ok, then I think you might you to add more tests cases for urls like /wiki/#bad [15:33:46] (03CR) 10Joal: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [15:35:33] nuria: ok, I can add more unit tests for the other paths, but that should be a separate change. To me, that’s kind of the point of testing the interface, that you can change the implementation [15:36:36] 10Quarry, 10DBA, 10Data-Services: Quarry query became work much slower - https://phabricator.wikimedia.org/T247978 (10Marostegui) >>! In T247978#5980018, @bd808 wrote: >>>! In T247978#5979931, @Marostegui wrote: >> The only thing I can think of to discard labsdb1011's specific host issues, would be to point... [15:58:03] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Nuria) >@Milimetric will your patch cause us to start classifying these Android app webrequests as pageviews? No, it would... [16:31:02] (03PS1) 10Milimetric: Use PageviewDefinition changes: KaiOS & wikimania [analytics/refinery] - 10https://gerrit.wikimedia.org/r/580999 [16:31:27] (03PS2) 10Milimetric: Use PageviewDefinition changes: KaiOS & wikimania [analytics/refinery] - 10https://gerrit.wikimedia.org/r/580999 (https://phabricator.wikimedia.org/T244547) [16:31:51] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Use PageviewDefinition changes: KaiOS & wikimania [analytics/refinery] - 10https://gerrit.wikimedia.org/r/580999 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [16:35:32] joal: so let me know if you see anything else with the pageview change and I'll fix it [16:36:08] milimetric: checking now [16:36:50] (03CR) 10Joal: "LGTM :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/580999 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [16:37:35] ok fdans, all aboard [16:37:57] (I kicked mforns' change to next week with his permission (ON SLACK!) [16:38:01] ) [16:38:04] 10Analytics, 10Analytics-Dashiki: Bookmarkable date filters for browser stats dashboard - https://phabricator.wikimedia.org/T143689 (10Nuria) 05Open→03Declined [16:38:10] 🚂🚃🚃🚃🚃🚃🚃🚃🚃 [16:38:21] hou hou hou [16:38:29] 10Analytics, 10Core Platform Team, 10MediaWiki-API, 10RESTBase-API: Top API user agents stats - https://phabricator.wikimedia.org/T142139 (10Nuria) 05Open→03Resolved [16:38:32] I was trying irc cloud :P [16:38:32] 10Analytics, 10Core Platform Team, 10MediaWiki-API: Copy cached API requests from raw webrequests table to ApiAction - https://phabricator.wikimedia.org/T155478 (10Nuria) [16:38:42] 10Analytics, 10Analytics-Dashiki: Dashiki should load stale data if new data is not available due to network/api conditions - https://phabricator.wikimedia.org/T138647 (10Nuria) 05Open→03Declined [16:38:48] this train is stupid because it both uses a steam locomotive and an electric catenary [16:38:53] (03CR) 10Joal: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [16:39:10] 10Analytics: Quantify false positives when filtering for number of distinct user agents per page in top pages computation - https://phabricator.wikimedia.org/T146911 (10Nuria) 05Open→03Declined [16:39:13] 10Analytics, 10Research-Backlog: [Open question] Improve bot identification at scale - https://phabricator.wikimedia.org/T138207 (10Nuria) [16:39:16] milimetric: I just realized I had completely misunderstood nuria's idea - Let's make it [16:39:27] 10Analytics: Quantify false positives when filtering for number of distinct user agents per page in top pages computation - https://phabricator.wikimedia.org/T146911 (10Nuria) Rather than declined, "superseeded by the work on bots" [16:39:44] of course joal, I didn't see it, you two have a parallel thread for commenting during meetings :) [16:39:54] huhuhu :) [16:40:02] fdans: hang on, we got someone running to catch the train [16:42:28] 10Analytics, 10Operations, 10Traffic: Publishing project anomaly data for censorship researchers. Evaluate privacy threats - https://phabricator.wikimedia.org/T183990 (10Nuria) [16:47:51] 10Analytics: Add druid load job for data quality table - https://phabricator.wikimedia.org/T244379 (10mforns) 05Open→03Declined Luca has made it possible to use Superset with Presto and Kerberos, so now we can plot all data quality metrics (and all other tables) with Superset. Declining! [16:48:45] 10Analytics, 10Analytics-Dashiki: Switch to fetch away from jquery - https://phabricator.wikimedia.org/T148053 (10mforns) a:05mforns→03None [16:53:27] nuria: that change is ninja, it cleaned up like 10 other lines [16:54:17] (03PS5) 10Milimetric: Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) [16:55:38] (03CR) 10Milimetric: Detect pageviews as requested by KaiOS (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [16:58:23] (03CR) 10Joal: [C: 03+2] "Great :) Thanks milimetric :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [17:03:49] (03Merged) 10jenkins-bot: Detect pageviews as requested by KaiOS [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/580161 (https://phabricator.wikimedia.org/T244547) (owner: 10Milimetric) [17:08:03] 10Analytics, 10ChangeProp, 10Core Platform Team, 10Event-Platform, and 2 others: Make Kafka JobQueue use Special:RunSingleJob - https://phabricator.wikimedia.org/T182372 (10Pchelolo) 05Open→03Declined In lights of T175146 we are no longer going to do this [17:08:06] 10Analytics, 10ChangeProp, 10Event-Platform, 10MediaWiki-JobQueue, and 5 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088 (10Pchelolo) [17:10:32] 10Analytics, 10Event-Platform, 10Multi-Content-Revisions, 10CPT Initiatives (Modern Event Platform (TEC2)): Redesign revision-related event schemas for MCR - https://phabricator.wikimedia.org/T186371 (10Pchelolo) a:05Pchelolo→03None [17:11:19] 10Analytics, 10ChangeProp, 10Event-Platform, 10RESTBase, and 3 others: Support change propagation for private wikis - https://phabricator.wikimedia.org/T137140 (10Pchelolo) 05Open→03Declined Don't think we will be doing that given the changed strategy regarding the future of RESTBase [17:12:57] 10Analytics, 10ChangeProp, 10Event-Platform, 10MediaWiki-JobQueue, 10Services (designing): Split ChangeProp metrics by wiki - https://phabricator.wikimedia.org/T175952 (10Pchelolo) 05Open→03Declined Don't think we will be doing this [17:13:00] 10Analytics, 10ChangeProp, 10Event-Platform, 10MediaWiki-JobQueue, 10Services (designing): Requests for new JobQueue monitoring capabilities - https://phabricator.wikimedia.org/T175780 (10Pchelolo) [17:13:24] 10Analytics, 10Event-Platform, 10Services (later): Document ChangeProp and EventBus monitoring - https://phabricator.wikimedia.org/T171533 (10Pchelolo) 05Open→03Resolved [17:13:27] 10Analytics, 10ChangeProp, 10Event-Platform, 10MediaWiki-JobQueue, and 5 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088 (10Pchelolo) [17:23:50] 10Analytics, 10MediaWiki-API: Copy cached API requests from raw webrequests table to ApiAction - https://phabricator.wikimedia.org/T155478 (10Pchelolo) Nothing for CPT to do here. [17:31:28] milimetric: how bout now [17:31:46] all good fdans [17:32:32] everything in the pad should be merged, webrequest load restarts with v0.0.119 and gepeditors yearly restarts [17:49:54] 10Analytics, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)), 10Core Platform Team Workboards (Clinic Duty Team): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Pchelolo) I finally got to loo... [17:57:12] * elukey brb [17:57:21] 10Analytics, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)), 10Core Platform Team Workboards (Clinic Duty Team): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Nuria) Queries can be run in h... [18:07:59] milimetric: super thnaks [18:08:02] *thanks [18:09:54] Pchelolo: ping me if you need help querying data [18:10:15] nuria: thank you, it all works. I didn't realize I needed events database [18:10:19] Pchelolo: we just installed presto that is loads better than hive (for not super big data) [18:10:27] I went to wmf database and was lost there :) [18:10:40] >presto --catalog analytics_hive [18:10:44] Pchelolo: >presto --catalog analytics_hive [18:10:53] Pchelolo: way faster queries, try it you'll see [18:17:34] (03PS1) 10Fdans: Update changelog for 0.0.119 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/581024 [18:17:54] !log beginning deploy of refinery-source 0.0.119 [18:17:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:19:20] (03CR) 10Fdans: [V: 03+2 C: 03+2] Update changelog for 0.0.119 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/581024 (owner: 10Fdans) [18:19:49] 10Analytics, 10Analytics-Wikistats, 10Chinese-Sites: X-axis is at odds with stated period in header of trend charts for 'total articles' for a wiki - https://phabricator.wikimedia.org/T180118 (10Erik_Zachte) a:05Erik_Zachte→03None removing myself, I'm no longer involved [18:22:36] 10Analytics, 10Analytics-Wikistats: German derivative of Wikistats report shows marked difference for new editors in Aug vs Sep - https://phabricator.wikimedia.org/T178891 (10Erik_Zachte) a:05Erik_Zachte→03None Removing myself, I'm no longer involved In fact this may have been resolved, I'm not sure. I ce... [18:24:24] (03Merged) 10jenkins-bot: Update changelog for 0.0.119 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/581024 (owner: 10Fdans) [18:24:56] Should I split my refinery patches between the dumps and Api changes or just lump them together? [18:25:54] lexnasser: spliting is always better [18:26:12] 10Analytics, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)), 10Core Platform Team Workboards (Clinic Duty Team): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Pchelolo) Here's an example of... [18:26:17] lexnasser: so the code reviever can more easily grasp what the changeset is about [18:29:30] 10Analytics, 10Product-Analytics, 10covid-19: Weekly updates on editors & readers - https://phabricator.wikimedia.org/T247873 (10kzimmerman) I'm going to circle back with Product Leads in the Product Leads meeting Tuesday, Mar 24. Pageviews are available on a weekly basis in Superset; here's a preliminary d... [18:34:37] 10Analytics, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)), 10Core Platform Team Workboards (Clinic Duty Team): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Pchelolo) So, I have establish... [18:45:33] 10Analytics, 10Event-Platform, 10Product-Analytics, 10CPT Initiatives (Modern Event Platform (TEC2)), 10Core Platform Team Workboards (Clinic Duty Team): Eventbus revisions are duplicated in event.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T218246 (10Pchelolo) 05Open→03Invalid... [18:51:28] !log refinery source 0.0.119 jars generated and symlinked [18:51:29] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:52:19] !log deploying refinery [18:52:20] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:56:10] * elukey off! [18:58:39] 10Analytics: Grant ingersol needs kerberos credentials - https://phabricator.wikimedia.org/T248014 (10Nuria) [19:00:15] 10Analytics: Grant needs kerberos credentials - https://phabricator.wikimedia.org/T248014 (10Nuria) [19:12:22] 10Analytics: Grant needs kerberos credentials - https://phabricator.wikimedia.org/T248014 (10Nuria) Once you get your credentials please ssh to stats1008.eqiad.wmnet and try > kinit to make sure things are working. Please see: https://wikitech.wikimedia.org/wiki/Analytics/Data_access#Hadoop We have just instal... [19:13:59] 10Analytics: Grant needs kerberos credentials - https://phabricator.wikimedia.org/T248014 (10Nuria) a:03elukey [19:39:18] !log refinery deployed [19:39:19] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:40:13] milimetric: geoeditors yearly has to be restarted from 2020 or 2021? [19:43:38] fdans: 2020 yeah, it timed out for that one [19:43:49] it does 2020 at the end of the year [19:45:56] wait milimetric, what start date should it have? [19:46:52] fdans: 2020-01, it's not special, you just use the last date that failed: https://hue.wikimedia.org/oozie/list_oozie_coordinator/0000480-191216160148723-oozie-oozi-C/ [19:47:16] そそそそそ [19:47:23] milimetric: sounds good [19:47:27] thx [19:47:35] what is that symbol lol [19:47:58] milimetric: in japanese, it's like sosososo [19:48:06] as in, yeah of course! [19:48:15] been watching a lot of terrace house lately [19:49:08] そそそそそ [19:50:20] milimetric: ok, webrequest bundle and geoeditors リスタート [19:50:46] (ristaato => restart) [19:50:48] bine bine, mulțumesc! [19:51:14] nice [19:51:18] nerds [19:51:40] we can probably carry out a passable conversation in 6 languages between the 7 of us [19:52:07] we should do that at standup once every week or something [19:52:30] Fruitless Fridays [19:53:26] yeah, it works, at least two of us have some experience in each of romanian, german, spanish, french, english japanese [19:54:38] Absolument ! [19:55:17] 10Analytics, 10Analytics-Wikistats, 10Chinese-Sites: X-axis is at odds with stated period in header of trend charts for 'total articles' for a wiki - https://phabricator.wikimedia.org/T180118 (10Dvorapa) Well, I think all old-wikistats related tasks should be closed as declined in favor of Wikistats 2. [19:55:22] 10Analytics: Grant needs kerberos credentials - https://phabricator.wikimedia.org/T248014 (10Isaac) Yep -- happy to help out if there are specific questions around referrers that I can help answer. Just let me know! [19:55:35] * joal is trying to manually reproduce the sosososo symbol [20:04:40] i tant [20:10:50] joal: https://en.wikipedia.org/wiki/So_(kana) [20:11:08] WAW ! [20:12:04] ありがとうございました [20:12:22] hehe [20:12:37] joal: remember that ソン [20:12:43] シツ [20:12:51] are all different characters [20:13:17] y'all talkin tha nee-hon-go? thought I was in the wrong channel [20:13:24] I can see that when having them close by - I probably wouldn't have noticed [20:13:44] hip: I'll take any opportunity to yap about japanese [20:13:50] especially japanese writing and kanji [20:15:18] fdans: really? you're into that? You take any JLPTs? [20:18:12] fdans: I'm going to try to do JLPT N3 or N4 this December, depending on how far along I can get. Give me all your tips [20:28:56] 10Analytics, 10Product-Analytics (Kanban): SQL definition for wikidata metrics for tunning session - https://phabricator.wikimedia.org/T247099 (10Isaac) > As for page_namespace, I want to keep all content pages. I checked page_namespace_is_content in table mediawiki_page_history. The list of namespace marked a... [21:08:20] (03CR) 10Milimetric: [V: 03+2] Increase yearly coordinator timeout to 370 days [analytics/refinery] - 10https://gerrit.wikimedia.org/r/579577 (https://phabricator.wikimedia.org/T246753) (owner: 10Milimetric) [21:08:43] (03CR) 10Milimetric: [V: 03+2] "oops, this never got merged, I'll have to put it on the next train" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/579577 (https://phabricator.wikimedia.org/T246753) (owner: 10Milimetric) [21:11:51] (03PS7) 10Mforns: Add dimensions to druid's pageview_hourly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) [21:12:34] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Inuka-Team (Kanban), 10Patch-For-Review: Set up pageview counting for KaiOS app - https://phabricator.wikimedia.org/T244547 (10Milimetric) >>! In T244547#5979835, @nshahquinn-wmf wrote: > @Milimetric will your patch cause us to start classifying the... [21:13:45] 10Analytics, 10Analytics-Kanban: geoeditors-yearly job times out - https://phabricator.wikimedia.org/T246753 (10Milimetric) this was accidentally not merged (just +2-ed on refinery which we forgot doesn't auto-merge) which caused Fran to restart for no reason. I put it on next week's train. [21:14:01] (03CR) 10Mforns: [V: 03+2] "I think these last changes do what @joal suggested." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/570681 (https://phabricator.wikimedia.org/T243090) (owner: 10Mforns) [21:33:20] 10Analytics, 10Product-Analytics, 10covid-19: Weekly updates on editors & readers - https://phabricator.wikimedia.org/T247873 (10kzimmerman) [21:40:57] 10Analytics, 10Product-Analytics, 10covid-19: Weekly updates on editors & readers - https://phabricator.wikimedia.org/T247873 (10Nuria) While data is not nicely denormalized like it is in the monthly dataset, the events database holds every edit/deletion/page creation (each event is its own table) that hap... [21:51:03] (03CR) 10Awight: Count disables of the TwoColumnConflict interface (033 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/577213 (https://phabricator.wikimedia.org/T246104) (owner: 10WMDE-Fisch) [21:51:09] 10Analytics, 10ContentTranslation, 10Language-Team (Language-2020-Focus-Sprint): Test Performance of Marian NMT translation in stat cluster - https://phabricator.wikimedia.org/T247245 (10Nuria) let us know if you need more help with this @santhosh