[00:36:22] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Technical contributors emerging communities metric definition, thick data - https://phabricator.wikimedia.org/T250284 (10jwang) @Nuria, @Bmueller Add edits to table "Editors_by_wikis_2020-04". And filtered out projects 1) no bots edits/editors 2) less... [01:17:54] 10Analytics, 10Product-Analytics: EventLogging data missing from event_sanitized schemas - https://phabricator.wikimedia.org/T253182 (10MMiller_WMF) [03:34:06] 10Analytics, 10Product-Analytics: EventLogging data missing from event_sanitized schemas - https://phabricator.wikimedia.org/T253182 (10Nuria) This seems like it could be related to the issues we had with kerberos-run-command when wrapping the refine-sanitize, kerberos was not surfacing the issue with snakeyam... [06:11:56] 10Analytics, 10Product-Analytics (Kanban): Create Druid tables for Druid datasources in Superset - https://phabricator.wikimedia.org/T251857 (10elukey) This is the puppet configuration for the job that pulls from Hive and index to Druid: ` profile::analytics::refinery::job::eventlogging_to_druid_job { 'pa... [06:18:39] 10Analytics, 10Product-Analytics (Kanban): Create Druid tables for Druid datasources in Superset - https://phabricator.wikimedia.org/T251857 (10elukey) I checked in the new version of Superset in staging (under testing) and the bug seems gone, weird.. [07:22:25] 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics, 10Patch-For-Review: Upgrade to Superset 0.36.0 - https://phabricator.wikimedia.org/T249495 (10elukey) The above patch was reverted for a little bug, and https://github.com/apache/incubator-superset/pull/9789 was sent and merged afterwards. I h... [08:32:08] (03CR) 10Andrew-WMDE: [C: 03+2] Remove known buggy conflicts [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596439 (owner: 10Awight) [08:35:24] 10Analytics, 10Operations, 10Traffic, 10Patch-For-Review: Create replacement for Varnishkafka - https://phabricator.wikimedia.org/T237993 (10ema) 05Open→03Resolved a:03ema Closing this task now given that an initial version of `atskafka` has been created and deployed. Further improvements such as T2... [08:39:55] (03CR) 10Andrew-WMDE: [C: 03+2] Count displayed rows [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596145 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [08:40:19] brb! [08:40:56] (03CR) 10Andrew-WMDE: [V: 03+2 C: 03+2] Remove known buggy conflicts [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596439 (owner: 10Awight) [08:44:43] (03CR) 10Andrew-WMDE: [V: 03+2 C: 03+2] Count displayed rows [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596145 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [09:01:50] (03CR) 10Andrew-WMDE: [C: 03+2] Refresh notebooks with April 2020 data [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596480 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [09:01:58] (03CR) 10Andrew-WMDE: [V: 03+2 C: 03+2] Refresh notebooks with April 2020 data [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596480 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [09:44:41] (03CR) 10Andrew-WMDE: [C: 03+1] "A little hacky, but does the trick. In the future, we might want to consider using a build tool (e.g. Maven) to manage the dependencies. T" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596421 (owner: 10Awight) [10:22:28] (03CR) 10Joal: "Some changes suggested inline :)" (038 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/596605 (https://phabricator.wikimedia.org/T252857) (owner: 10Fdans) [10:25:31] joal: nice socratic method in the CR :D [10:25:57] fdans: would you please teach me about socratic method? [10:26:14] * joal does stuff he doesn't even know had a name :S [10:26:49] joal: it's basically making a point through asking questions [10:27:01] https://en.wikipedia.org/wiki/Socratic_method [10:27:10] Ah! I do that often indeed - I hope if feels ok at reading time? [10:27:43] * elukey lunch! [10:28:50] joal: yesyesyes it's great :) [10:43:42] for number-miracle lovers: https://kottke.org/20/05/solving-the-the-miracle-sudoku [11:14:57] (03PS9) 10Andrew-WMDE: Analyze row count [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [11:38:45] (03CR) 10Andrew-WMDE: "In order to make the file less binary like and more palatable, I decided to embed the generated graphs as PNGs instead of using the defaul" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [11:48:47] 10Analytics, 10Performance-Team, 10Readers-Web-Backlog (Tracking): Review referer configuration of origin/origin-when-crossorigin/origin-when-cross-origin - https://phabricator.wikimedia.org/T248526 (10TheDJ) origin-when-crossorigin: Safari pre 11.1 (spec implementation error, 2017-2018) origin-when-cross-or... [11:50:05] (03PS1) 10Milimetric: Comment on join to page move events [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/597516 [11:50:12] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Comment on join to page move events [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/597516 (owner: 10Milimetric) [11:50:35] joal: I had missed your comment about my lack of comment, added here ^ [11:50:43] and with that... [11:50:46] CHOOO CHOOO [11:51:04] my change is the only passenger on the train, so I feel like I need to say: [11:51:12] AAAAAALLLL ABOOOOOOAAAAARD [11:51:24] anyone else? no changes? [11:52:19] all good milimetric :) [11:54:08] elukey: that's weird... should I be able to hdfs dfs -ls without kinit? [11:54:24] milimetric: if you have a ticket in cache [11:54:26] I forgot this morning, did it, got the usual kerberos error vomit, but then it did it anyway [11:54:33] really? [11:54:41] I'll copy the output here: [11:54:41] meh [11:55:08] https://www.irccloud.com/pastebin/QC0e0KBc/ [11:56:08] oh weird... but my ticket isn't expired: [11:56:10] https://www.irccloud.com/pastebin/pFNsSu9T/ [11:56:50] interesting milimetric - this is not the usual vomit :) It's a different one!P [11:56:52] and indeed doing a kinit makes that error go away. I think I found something completely insignificant! [11:57:02] :) [11:57:24] Let's review with elukey when he comes back [12:01:38] (03CR) 10Andrew-WMDE: [C: 03+1] "Looks good to me" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [12:02:31] (03CR) 10Awight: "> In order to make the file less binary like and more palatable, I" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [12:02:33] here I am :) [12:04:11] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10soworu) >>! In T252703#6142940, @Dzahn wrote: > @soworu Use the same user/password you used on https://wikitech.wikimed... [12:04:46] milimetric: ahhh interesting! I have never seen it, but I think that you made the request when your ticket was renewed by the KDC, or similar [12:05:46] it must be a corner case that we have never seen, but should happen very rarely and not repeat itself in a row [12:06:09] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) >>! In T252703#6152022, @soworu wrote: >>>! In T252703#6142940, @Dzahn wrote: >> @soworu Use the same user/passw... [12:06:59] (03CR) 10Awight: [V: 03+1 C: 03+2] "Locally confirmed working, thanks again for the tweaks." [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [12:08:18] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) Please try the following variations of the username including the capitalization: soworu-01 and SOworu [12:11:07] fdans: anything to merge for wikistats? I'm deploying to get https://phabricator.wikimedia.org/T213800 out [12:11:39] elukey: it did repeat itself, I tried twice and got the same error, then did kinit and the error went away [12:13:09] milimetric: strange, not sure what happened [12:13:17] but your creds were not expired [12:13:23] right? [12:14:12] right, they show expires at 15:01:59 UTC today so that's about 3 hours-ish after when I tried the command [12:14:28] is that right, am I UTC-4 right now? [12:14:46] yep [12:16:13] (03PS4) 10Fdans: Fix language dropdown for ios devices [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/589606 (https://phabricator.wikimedia.org/T246971) [12:16:30] milimetric: just rebased this thing so we can merge it [12:16:38] (03PS1) 10Milimetric: Update changelog for version 0.0.125 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/597526 [12:16:53] ok fdans, cool, will take a look and merge [12:17:22] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Update changelog for version 0.0.125 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/597526 (owner: 10Milimetric) [12:17:25] milimetric: thank youuu [12:17:50] Starting build #43 for job analytics-refinery-maven-release-docker [12:20:07] (03CR) 10Milimetric: [C: 03+2] Fix language dropdown for ios devices [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/589606 (https://phabricator.wikimedia.org/T246971) (owner: 10Fdans) [12:23:02] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10soworu) >>! In T252703#6152029, @Dzahn wrote: > Please try the following variations of the username including the capit... [12:28:57] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) @soworu Try again now, please. Looks like i forgot to add the -01 part when adding you the right group. Sorry ab... [12:29:13] Project analytics-refinery-maven-release-docker build #43: 09SUCCESS in 11 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release-docker/43/ [12:30:29] fdans: am I missing something in the build to get languages showing in the menu? I deployed to test and the language selector is just empty: (https://wikistats-canary.wmflabs.org/mobile/#/all-projects) sorry no time to look too much now doing other train stuff [12:31:19] milimetric: I'll take care of this, I might have messed something up in the rebase [12:31:41] Starting build #15 for job analytics-refinery-update-jars-docker [12:31:42] I'm going to get lunch now but I'll fix it and deploy it after [12:31:59] (03PS1) 10Maven-release-user: Add refinery-source jars for v0.0.125 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597534 [12:32:00] Project analytics-refinery-update-jars-docker build #15: 09SUCCESS in 19 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars-docker/15/ [12:34:56] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) @soworu Looks like it works now, i think i just saw you login in the log files, am i right? [12:34:59] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10soworu) >>! In T252703#6152084, @Dzahn wrote: > @soworu Try again now, please. Looks like i forgo... [12:43:40] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10soworu) >>! In T252703#6152094, @Dzahn wrote: > @soworu Looks like it works now, i think i just s... [12:43:46] (03CR) 10Awight: "> A little hacky, but does the trick. In the future, we might want to" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596421 (owner: 10Awight) [12:45:36] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) @soworu I found the following comment in the Apache config of superset: ` 31 #... [12:48:14] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10soworu) >>! In T252703#6152109, @Dzahn wrote: > @soworu I found the following comment in the Apac... [12:48:31] (03PS5) 10Awight: Embed Jupyter kernel template [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596421 [12:48:54] (03CR) 10Awight: "PS 5: updated readme" [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596421 (owner: 10Awight) [12:49:02] (03CR) 10Awight: [V: 03+2 C: 03+2] Analyze row count [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596188 (https://phabricator.wikimedia.org/T252507) (owner: 10Awight) [12:49:46] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations, 10Patch-For-Review: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Dzahn) >>! In T252703#6152113, @soworu wrote: > SOworu worked. Now in Turnilo. Thanks. Great! E... [12:51:43] 10Analytics, 10EventStreams: EventStreams socket stays connected without any traffic incoming - https://phabricator.wikimedia.org/T250912 (10stjn) >>! In T250912#6145819, @Ottomata wrote: > Hm, this is going to be hard to reproduce, especially given T179986. I'd expect this to be a client side problem, but it... [12:53:20] (03PS1) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [13:04:55] 10Analytics, 10MW-1.35-notes (1.35.0-wmf.31; 2020-05-05), 10User-Urbanecm, 10User-Zoranzoki21: Add awawiki to anlaytics whitelist - https://phabricator.wikimedia.org/T253225 (10Ladsgroup) [13:11:13] (03PS5) 10Awight: Move data to semi-permanent path [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596436 [13:11:33] 10Analytics, 10User-Urbanecm: Add gomwiktionary to analytics whitelist - https://phabricator.wikimedia.org/T253227 (10Ladsgroup) [13:16:07] Yo team - I updated the document on the analytics-edit-datasource, adding a 'totals' section to the number column to help grasping the datasize aspects, and also added a `pros and cons table` at the end. Please chime-in as you will :) [13:22:59] 10Analytics, 10EventStreams: EventStreams socket stays connected without any traffic incoming - https://phabricator.wikimedia.org/T250912 (10Ottomata) Oops I did mean what you said. THANK YOU [13:24:10] ottomata: o/ [13:24:24] I have just uploaded your refactoring for profile::java [13:24:35] let me know if it makes sense or if it misses stuff [13:24:37] 10Analytics, 10Analytics-Kanban, 10LDAP-Access-Requests, 10Operations: LDAP access to the wmf group for Segun Oworu (superset, turnilo, hue) - https://phabricator.wikimedia.org/T252703 (10Ottomata) Hue allows you to access Hive and files in HDFS, for which you need a shell account and membership in the ana... [13:24:42] (not urgent, anytime) [13:24:56] oh cool ok will look [13:46:29] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Creation of canonical pageview dumps for users to download - https://phabricator.wikimedia.org/T251777 (10Isaac) @fdans a clarification question: will the page IDs also be included in the [[https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews|pagevi... [13:47:40] (03CR) 10Andrew-WMDE: [C: 03+2] Move data to semi-permanent path [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596436 (owner: 10Awight) [13:47:44] (03CR) 10Andrew-WMDE: [V: 03+2 C: 03+2] Move data to semi-permanent path [analytics/wmde/TW/edit-conflicts] - 10https://gerrit.wikimedia.org/r/596436 (owner: 10Awight) [13:55:19] !log deployed refinery with refinery-source v0.0.125 [13:55:20] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:55:33] as soon as my kids stop screaming I'll go ahead and restart the jobs [13:55:52] and fdans lemme know when you're done with the bug so I can continue wikistats deploy, or if you want I can revert the dropdown fix [14:02:53] milimetric: nono don't worry I'll do the deploy [14:04:08] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Creation of canonical pageview dumps for users to download - https://phabricator.wikimedia.org/T251777 (10fdans) @Isaac you can open a task asking for it! But that's definitely not within the scope of this task. [14:06:39] (03PS2) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:12:30] (03PS3) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:14:46] (03PS4) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:16:12] ottomata: let's try to converge to one version of the code please, I am kinda becoming crazy in here :) [14:16:26] we can always refactor/add later if we see that it fits [14:17:26] (03PS5) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:17:41] elukey: ? [14:18:05] ottomata: the profile::java change :D [14:18:28] yes but in reference to what comment? [14:18:48] the last one [14:18:52] oh the default version? [14:18:54] yes yes [14:18:56] haha sure! just an idea! [14:18:59] ahhh ok [14:19:06] i don't event know if that is possible! [14:19:32] I will ask more people to chime in, like hasha*r and gehe*l [14:19:41] hopefully we'll have something good tomorrow [14:20:18] it is good elukey i like it a lot! [14:20:19] (03PS6) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:20:36] ottomata: yes it is really better now, I like it a lot, thanks for the code :) [14:21:37] I am going to open a task also to move systems to it once we are done [14:21:42] like cassandra etc.. [14:21:46] great [14:22:01] 10Analytics: Decomission notebook hosts - https://phabricator.wikimedia.org/T249752 (10MMiller_WMF) @nshahquinn-wmf helped me fix this. Things are working now. [14:30:53] 10Analytics, 10Product-Analytics: EventLogging data missing from event_sanitized schemas - https://phabricator.wikimedia.org/T253182 (10Nuria) a:03Milimetric [14:31:03] 10Analytics, 10Product-Analytics: EventLogging data missing from event_sanitized schemas - https://phabricator.wikimedia.org/T253182 (10Nuria) cc @mforns [14:40:35] 10Analytics, 10Product-Analytics: EventLogging data missing from event_sanitized schemas - https://phabricator.wikimedia.org/T253182 (10mforns) Oh! I forgot to back-fill sanitization when I fixed the problem with snakeyaml breaking upgrade. [facepalm] My bad... I will do the back-filling @Nuria & @Milimetric... [14:41:17] (03PS7) 10Fdans: Add Pageviews Complete dumps backfilling job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597541 (https://phabricator.wikimedia.org/T252857) [14:51:36] joal: helloooo do you have a minute pre-standup? [14:53:32] Hi fdans - here I am :0 [15:00:30] pin mforns standdduppp [15:00:40] oooops [15:03:49] (03PS1) 10Milimetric: Add new wikis to whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597566 [15:03:57] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Add new wikis to whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597566 (owner: 10Milimetric) [15:09:59] 10Analytics, 10Analytics-Kanban: Camus failing to import eqiad.mediawiki.(api|cirrussearch)-request from partitions leaders on kafka-jumbo1006 - https://phabricator.wikimedia.org/T252203 (10Milimetric) I still have those backups on hdfs, the cirrussearch one starts at 2020-05-02T06:58:54Z and ends at 2020-05-0... [15:12:40] 10Analytics, 10Analytics-Kanban: Camus failing to import eqiad.mediawiki.(api|cirrussearch)-request from partitions leaders on kafka-jumbo1006 - https://phabricator.wikimedia.org/T252203 (10Ottomata) OH RIGHT! Great. Will check that out and try to backfill it. [15:35:58] 10Analytics, 10Performance-Team, 10Readers-Web-Backlog (Tracking): Review referer configuration of origin/origin-when-crossorigin/origin-when-cross-origin - https://phabricator.wikimedia.org/T248526 (10Krinkle) 05Open→03Declined Thanks. [15:52:40] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Technical contributors emerging communities metric definition, thick data - https://phabricator.wikimedia.org/T250284 (10Nuria) Further exploration to see whether number of bots is a good proxy to identify emerging communities: - let's just consider 'wik... [15:55:06] 10Analytics, 10Better Use Of Data, 10Event-Platform: Produce an instrumentation event stream using new EPC and EventGate from client side browsers - https://phabricator.wikimedia.org/T241241 (10jlinehan) [15:55:50] 10Analytics, 10Better Use Of Data, 10Event-Platform, 10Product-Analytics: Explore sending batches of events from EPC libraries - https://phabricator.wikimedia.org/T239996 (10jlinehan) [15:59:09] 10Analytics: Make it easy to debug eventlogging instrumentation, add ability to send client canary events. - https://phabricator.wikimedia.org/T253239 (10Nuria) [16:05:28] 10Analytics, 10MW-1.35-notes (1.35.0-wmf.31; 2020-05-05), 10User-Urbanecm, 10User-Zoranzoki21: Add awawiki to anlaytics whitelist - https://phabricator.wikimedia.org/T253225 (10Nintendofan885) It's on line 52 [16:06:58] joal: are you already out or do you have a min for a code change? [16:08:27] elukey: will leave in minutes - show me :) [16:09:06] joal: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/597573/ :) [16:09:20] pcc is https://puppet-compiler.wmflabs.org/compiler1001/22642/ [16:10:03] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Patch-For-Review: Automate ingestion and refinement into Hive of event data from Kafka - https://phabricator.wikimedia.org/T251609 (10Ottomata) Had a great discussion with A-team today and got some great ideas. Kafka jumbo-... [16:10:11] 10Analytics: MEP: canary events so we know events are flowing through pipeline - https://phabricator.wikimedia.org/T250844 (10Ottomata) [16:10:13] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Patch-For-Review: Automate ingestion and refinement into Hive of event data from Kafka - https://phabricator.wikimedia.org/T251609 (10Ottomata) [16:11:14] nice elukey [16:11:29] 10Analytics, 10Analytics-Kanban: MEP: canary events so we know events are flowing through pipeline - https://phabricator.wikimedia.org/T250844 (10Ottomata) p:05Medium→03High a:03Ottomata [16:11:37] elukey: I'm leaving now but I'm eager to see if it helps (I'm sure it will) [16:12:24] 10Analytics, 10Analytics-Kanban: MEP: canary events so we know events are flowing through pipeline - https://phabricator.wikimedia.org/T250844 (10Ottomata) In https://phabricator.wikimedia.org/T251609#6152803 we figured out a mostly clean way to implement this. Am on it! [16:12:26] joal: also I am wondering if the issue in the public cluster (the broker lock up) might be because we don't have the "+10" in historical's thread size [16:12:36] or similar [16:12:45] could be alleviate the issue, who knows [16:12:57] anyway, will apply to analytics first! [16:12:58] thanks :) [16:14:42] going afk for a bit :) [16:35:13] nuria: mep sync? [16:35:24] ottomata: yes coming sorryy [16:37:11] 10Analytics: Add awawiki to anlaytics whitelist - https://phabricator.wikimedia.org/T253225 (10Zoranzoki21) Yes, I see that awawiki is on line 52 as @Nintendofan885 said. [17:14:15] 10Analytics: Make it easy to debug eventlogging instrumentation, add ability to send client canary events. - https://phabricator.wikimedia.org/T253239 (10Ottomata) [17:22:13] huh, check out this eventstreams based viz tool [17:22:14] https://www.tibco.com/products/tibco-spotfire/learn/demos/real-time-wikipedia-edits-demo [17:54:08] * elukey off! [18:01:21] milimetric: yt? [18:02:17] hey yep [18:02:44] i'm tryiing to write example doc on creating a new event stream with wikimedia events [18:02:45] ottomata: yep [18:02:51] sry :) [18:03:00] but im' not familiar with how to trigger anyting! [18:03:05] i want something very simple in mw [18:03:09] some UI hover or click [18:03:13] or page load something [18:03:15] or anything [18:03:27] that I can use to create instrumentation that emits a dummy event [18:03:39] i don't know where to look for docs either [18:03:47] i guess i want to know what are common values for mw.track [18:03:55] or trackSubscribe i guess [18:04:14] is there some reason you can't just send it on the console? [18:04:32] i could, but ni the demo docs i want to show adding it to instrumentation code too [18:04:37] oh! [18:04:40] sorry, docs, got it [18:05:16] you want to add it to an existing/popular extension like VisualEditor or something like that? [18:05:29] OH jquery.......... just found mabye an example in wm events [18:05:31] core doesn't have a whole lot of instrumentation I don't think, it's usually in some extension [18:05:34] citation usage [18:05:41] i want to use WikimediaEvents [18:05:44] yep, there's a bunch in WE [18:05:49] $( '#content' ).on( 'click', 'a.external', function () { [18:05:52] ??? [18:05:53] content.... [18:06:14] so when external links in the div id="content" are clicked [18:06:35] I'm assuming that's where the article's rendered [18:07:30] hmm ok, so i don't even care, a click is perfect [18:07:43] is maybe some jquery thing i can get and fire after click that is on every page? [18:07:49] yep, that would work fine, you can do .on( 'hover' also if you want [18:07:59] k... [18:08:11] you can do 'a' instead of 'a.external' and then it would be every link [18:08:17] ah ok [18:08:18] most pages have some links [18:08:20] ahhh [18:09:11] so [18:09:12] $( '#content' ).on( 'hover', 'a', function ( ) { console.log("woo hover"); } ); [18:09:22] shoudl get me logs when I hover over any link? [18:09:57] OH [18:10:00] hover is not thing [18:10:02] but mouseout is! [18:10:03] got it! [18:10:06] that'll do [18:10:07] thank you [18:11:41] sorry, yeah, mouseover and mouseout, and others here: https://api.jquery.com/category/events/mouse-events/ [18:12:48] milimetric: in a.external [18:12:51] is external a class? [18:12:53] css? [18:13:13] yes, dot something is a class [18:13:16] k greaet [18:13:21] And # something is an id [18:13:48] and the tag name is optional so ‘.external’ is anything with the external class [18:13:59] nice [18:14:14] cool so i'll make a fun extiw one for interwiki links hover event [18:14:22] And I’m sorry to inform you but now you’re a web developer [18:14:29] haha [18:52:35] 10Analytics, 10Analytics-Data-Quality, 10Datasets-Webstatscollector, 10Language-Team: Add alarms for high volume of views to pages with replacement characters - https://phabricator.wikimedia.org/T117945 (10mforns) a:05mforns→03None [18:54:22] 10Analytics, 10Operations, 10Traffic: Publishing project anomaly data for censorship researchers. Evaluate privacy threats - https://phabricator.wikimedia.org/T183990 (10Nuria) a:05Nuria→03None [18:54:32] 10Quarry: Quarry should refuse to save results that are way too large - https://phabricator.wikimedia.org/T188564 (10zhuyifei1999) a:05zhuyifei1999→03None [20:25:41] 10Analytics, 10Event-Platform, 10Product-Analytics: Review draft Modern Event Platform schema guidelines - https://phabricator.wikimedia.org/T233329 (10Ottomata) > Hm, that is a good idea. I think perhaps a 'analytics event platform tutorial' page will be better for this. How's that sound? Ok @nshahquinn-wmf... [20:26:51] 10Analytics, 10Event-Platform, 10Product-Analytics: Review Event Platform instrumentation documentation for Product teams - https://phabricator.wikimedia.org/T233329 (10Ottomata) [20:28:42] 10Analytics, 10Analytics-Kanban, 10Event-Platform, 10Product-Analytics: Review Event Platform instrumentation documentation for Product teams - https://phabricator.wikimedia.org/T233329 (10Ottomata) [20:29:19] 10Analytics, 10Analytics-Kanban, 10Event-Platform, 10Product-Analytics: Write and update Event Platform instrumentation documentation for Product teams - https://phabricator.wikimedia.org/T233329 (10Ottomata) [20:29:34] alright thanks milimetric [20:29:35] https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To [20:29:35] :) [20:29:39] would love any thoughts or comments or edits [20:29:47] also ping mforns ^ [20:30:08] ottomata: ok will look! [20:36:09] 10Analytics, 10Analytics-Kanban, 10Cloud-Services, 10Developer-Advocacy: Data missing on the hierarchical view on the wmcs-edits tool - https://phabricator.wikimedia.org/T252915 (10srishakatux) 05Open→03Resolved a:03srishakatux Thanks @Milimetric! Seems to have fixed now :) [20:36:20] 10Analytics, 10Analytics-Kanban, 10Cloud-Services, 10Developer-Advocacy: Data missing on the hierarchical view on the wmcs-edits tool - https://phabricator.wikimedia.org/T252915 (10srishakatux) a:05srishakatux→03Milimetric [21:04:28] (03PS1) 10Mforns: POC Airflow Refine [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597623 (https://phabricator.wikimedia.org/T241246) [21:05:22] (03CR) 10Mforns: [C: 04-2] "Not for merging :]" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/597623 (https://phabricator.wikimedia.org/T241246) (owner: 10Mforns) [21:10:41] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Spike: POC of refine with airflow - https://phabricator.wikimedia.org/T241246 (10mforns) I tested the code above in an-launcher1001 with the analytics user successfully! Note I used Airflow's sequentialExecutor, because I was using an Airflow sqlite instal... [22:45:56] 10Analytics, 10Event-Platform, 10Product-Analytics: Product Analytics to review & provide feedback for Event Platform Instrumentation How-To - https://phabricator.wikimedia.org/T253269 (10kzimmerman)