[00:01:23] <wikibugs>	 Analytics-EventLogging: Work with Ops to ensure understanding of how EL works - https://phabricator.wikimedia.org/T75811 (ggellerman) NEW p:Triage
[00:05:30] <grrrit-wm>	 (PS2) Mforns: [WIP] [analytics/mediawiki-storage] - https://gerrit.wikimedia.org/r/175467 (https://bugzilla.wikimedia.org/68448)
[00:07:59] <grrrit-wm>	 (CR) Nuria: "I will make a new commit here to address my comment regarding re-usability of the csv parser." (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/175458 (owner: Milimetric)
[01:25:45] <grrrit-wm>	 (CR) Nuria: "Sorry, one more thing." (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/175458 (owner: Milimetric)
[01:57:03] <springle>	 nuria__: still around?
[01:57:17] <nuria__>	 springle: yes
[01:57:27] <nuria__>	 for a while, actually
[01:57:31] <springle>	 http://dev.mysql.com/doc/refman/5.5/en/comparison-operators.html#operator_between
[01:58:05] <springle>	 BETWEEN range is inclusive of the limits. ie, greater then or equal to, less than or equal to
[01:58:27] <springle>	 can we check the sample queries definitely do the ranges you want?
[01:58:33] <nuria__>	 ok, i see we  are missing casting too, right?>
[01:59:27] <nuria__>	 springle: so we'll need to adjust dates in our vetting queries (for accuracy) and cast dates for speed , correct?
[01:59:37] <springle>	 i changed the newly-reigstered-user sample sql to really match september 1st. ld one did august 31 and sept 1
[01:59:53] <springle>	 don't worry about casting dates
[02:00:18] <springle>	 the underlying fields are proper timestamps, not mediawiki varbinary, so properly formatted date strings will be cast to dates regardless
[02:00:31] <springle>	 but do double check the date ranges
[02:00:54] <springle>	 seomtimes it is easier to use an INTERVAL
[02:01:11] <nuria__>	 ok, i see
[04:29:22] <wikibugs>	 Analytics-EventLogging: Establish support model for EL? (e.g. more ops involvement)  - https://phabricator.wikimedia.org/T75824 (ggellerman) NEW p:Triage
[04:33:00] <wikibugs>	 Analytics-EventLogging: Establish expectations for EL uptime - https://phabricator.wikimedia.org/T75826#783365 (ggellerman)
[04:35:13] <wikibugs>	 Analytics-EventLogging: Create single place to look at graphs indicating outages - https://phabricator.wikimedia.org/T75827 (ggellerman) NEW p:Triage
[04:36:27] <wikibugs>	 Analytics-EventLogging: Create monitoring checks to make sure events end up in DB - https://phabricator.wikimedia.org/T75828#783379 (ggellerman)
[04:37:35] <wikibugs>	 Analytics-EventLogging: Create deploy checklist - https://phabricator.wikimedia.org/T75829 (ggellerman) NEW p:Triage
[04:39:14] <wikibugs>	 Analytics-EventLogging: Dashboard of status of our systems (one stop shopping) - https://phabricator.wikimedia.org/T75830 (ggellerman) NEW p:Triage
[04:40:20] <wikibugs>	 Analytics-EventLogging: Consider idea of 1 sprint for EL operational love sprint - https://phabricator.wikimedia.org/T75831 (ggellerman) NEW p:Triage
[04:59:21] <grrrit-wm>	 (PS5) Nuria: Implement breakdowns for metrics with CSV data [analytics/dashiki] - https://gerrit.wikimedia.org/r/175458 (owner: Milimetric)
[05:02:45] <grrrit-wm>	 (CR) Nuria: "Patch #5 makes the cvs parser generic again and removes dependency on ko from objects on api layer." [analytics/dashiki] - https://gerrit.wikimedia.org/r/175458 (owner: Milimetric)
[06:41:36] <wikibugs>	 Analytics: Add central notice cookie to x-analytics field in web-request logs - https://phabricator.wikimedia.org/T75835 (kevinator) NEW p:Triage
[06:42:10] <wikibugs>	 Analytics: Add central notice cookie to x-analytics field in web-request logs - https://phabricator.wikimedia.org/T75835#783463 (kevinator) Apparently this patch should help implement this: https://gerrit.wikimedia.org/r/#/c/157841/
[07:03:54] <wikibugs>	 Analytics-Tech-community-metrics: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#783489 (Oxta28)
[07:34:15] <wikibugs>	 Analytics-EventLogging: Epic: Simplify dashboard deployment with EventLogging data - https://phabricator.wikimedia.org/T75836 (kevinator) NEW p:Triage
[07:35:25] <wikibugs>	 Analytics-EventLogging: Engineer runs script to test SQL for dashboard against a test database - https://phabricator.wikimedia.org/T75837 (kevinator) NEW p:Triage
[07:37:01] <wikibugs>	 Analytics-EventLogging: Engineer runs script to create a new dashboard - https://phabricator.wikimedia.org/T75838 (kevinator) NEW p:Triage
[07:37:50] <wikibugs>	 Analytics-EventLogging: Epic: Simplify dashboard deployment with EventLogging data - https://phabricator.wikimedia.org/T75836#783515 (kevinator)
[07:40:09] <wikibugs>	 Analytics-EventLogging: Epic: Simplify dashboard deployment with EventLogging data - https://phabricator.wikimedia.org/T75836#783516 (kevinator) The current pipeline to graph EL data is pretty much like this: Pipeline #0: * write EL schema * instrument code * data goes into EL * set up new repository, dashboa...
[09:48:17] <wikibugs>	 Analytics: Epic: Create pipeline EL -> DW -> Pentaho - https://phabricator.wikimedia.org/T75841#783779 (Qgil)
[09:48:21] <wikibugs>	 Analytics: Epic: Create pipeline EL -> DW -> Pentaho - https://phabricator.wikimedia.org/T75841#783537 (Qgil)
[09:48:42] <wikibugs>	 Analytics: Understand performance & monitor to current capacity- set up tests to identify performance bottleneck (alarm for no writes) - https://phabricator.wikimedia.org/T75825#783782 (Qgil)
[09:48:45] <wikibugs>	 Analytics: Understand performance & monitor to current capacity- set up tests to identify performance bottleneck (alarm for no writes) - https://phabricator.wikimedia.org/T75825#783359 (Qgil)
[12:57:47] <wikibugs>	 Analytics-EventLogging: Create deploy checklist - https://phabricator.wikimedia.org/T75829#784123 (QChris) Open>declined a:QChris I have not been able to get a word in yesterday when this brought up.  This item makes no sense.  The people who do the deploys know how to deploy. Some unmaintained list...
[13:19:29] <wikibugs>	 Analytics-EventLogging: Establish support model for EL? (e.g. more ops involvement) - https://phabricator.wikimedia.org/T75824#784135 (QChris)
[13:21:51] <wikibugs>	 Analytics-EventLogging: Establish expectations for EL uptime - https://phabricator.wikimedia.org/T75826#784141 (QChris)
[13:34:43] <wikibugs>	 Analytics-EventLogging: Work with Ops to ensure understanding of how EL works - https://phabricator.wikimedia.org/T75811#784161 (QChris) Open>declined a:QChris I don't think that this expectation is fair.  Ops should be able to restart EventLogging jobs and debug the Ops side of EventLogging. Ops can...
[13:50:45] <rtnpro>	 YuviPanda, ping
[13:51:10] <rtnpro>	 YuviPanda, did you have a chance to review my patch on paginating recent queries in quarry
[13:51:11] <milimetric>	 does anyone know how I can distinguish a redirect article from a "normal" article?  Are they in different namespaces?
[13:51:11] <rtnpro>	 ?
[14:04:18] <YuviPanda>	 rtnpro: doing now
[14:05:08] <rtnpro>	 YuviPanda, thanks, mate :)
[14:06:16] <YuviPanda>	 rtnpro: looks good! I'll test and merge in a bit.
[14:06:27] <YuviPanda>	 by a bit I mean now, I guess
[14:06:28] * YuviPanda does now
[14:07:41] <rtnpro>	 YuviPanda, okies
[14:08:44] <grrrit-wm>	 (CR) Yuvipanda: [C: 2] "w00t! Thanks for the patch, works well :)" [analytics/quarry/web] - https://gerrit.wikimedia.org/r/175400 (owner: Rtnpro)
[14:08:48] <YuviPanda>	 rtnpro: done :)
[14:08:54] <grrrit-wm>	 (Merged) jenkins-bot: Paginate recent queries page. [analytics/quarry/web] - https://gerrit.wikimedia.org/r/175400 (owner: Rtnpro)
[14:09:01] <YuviPanda>	 rtnpro: and deployed
[14:09:03] <rtnpro>	 YuviPanda, yay!
[14:09:29] <YuviPanda>	 rtnpro: wanna do https://phabricator.wikimedia.org/T72735 next?
[14:09:39] <YuviPanda>	 rtnpro: just a filter to the all queries view that lets you view only 'published' ones
[14:10:08] <rtnpro>	 YuviPanda, sure :D
[14:10:34] <YuviPanda>	 rtnpro: :D I wonder if we should just make that slightly more generic, and see 'failed' queries, 'running' queries, etc
[14:11:03] <YuviPanda>	 rtnpro: feel free to close the task
[14:11:11] <wikibugs>	 Analytics-Quarry: Provide a way to browse all "published" queries. - https://phabricator.wikimedia.org/T72735#784279 (rtnpro) a:rtnpro
[14:11:27] <rtnpro>	 YuviPanda, assigned the above bug to myself :)
[14:11:47] <YuviPanda>	 rtnpro: cool :)
[14:11:56] <YuviPanda>	 halfak: rtnpro has been bugfixing and feature adding to Quarry :)
[14:12:24] <halfak>	 o/ rtnpro  :)
[14:12:40] <rtnpro>	 :)
[14:12:46] <rtnpro>	 halfak, Hi
[14:12:47] <YuviPanda>	 rtnpro: I promise faster reviews next patch :)
[14:13:00] <rtnpro>	 YuviPanda, that's OK :)
[14:13:23] <rtnpro>	 YuviPanda, may be you can assign me a tasklist, so that I can multitask ;)
[14:13:29] <YuviPanda>	 rtnpro: halfak is one of our research folks, and one of the primary users/drivers of quarry
[14:13:31] <YuviPanda>	 rtnpro: sure, let me find more
[14:13:37] <YuviPanda>	 halfak: ^ feature requests!
[14:13:59] <halfak>	 As in you want some more?
[14:14:03] <YuviPanda>	 yes
[14:14:08] <YuviPanda>	 and file them
[14:14:11] <YuviPanda>	 I can do filing
[14:14:26] <rtnpro>	 :)
[14:14:30] <halfak>	 Hmm.  Let me observe with scrutiny.  I usually submit my ideas as soon as I have them.
[14:14:43] <YuviPanda>	 :)
[14:15:02] <halfak>	 Oh!  I want a browser for all of my queries.
[14:15:12] <halfak>	 You can see recent ones on my profile page, but not all of 'em.
[14:15:32] <halfak>	 It's often that I want to go back to one of mine.
[14:16:03] <halfak>	 Oh.  Another one is URL rewriting.  If you could append the url with some query title stuff, that would help keep track of 'em.
[14:16:17] <halfak>	 I'm imagining something after the ID -- like trello does.
[14:18:44] <wikibugs>	 Analytics-Quarry: Make it possible to link to latest resultset for a query - https://phabricator.wikimedia.org/T75883#784313 (yuvipanda)
[14:18:52] <YuviPanda>	 rtnpro: https://phabricator.wikimedia.org/T75883?workflow=create
[14:18:54] <halfak>	 OK. Back to other work.  You guys are awesome.  I will buy you a beer the first chance I get :)
[14:18:57] <YuviPanda>	 halfak: :D
[14:19:11] <halfak>	 YuviPanda, knows I make good on these promises :)
[14:19:31] <wikibugs>	 Analytics-Quarry: Add pagination to user queries on user pages - https://phabricator.wikimedia.org/T75884 (yuvipanda) NEW p:Triage
[14:19:35] <YuviPanda>	 rtnpro: https://phabricator.wikimedia.org/T75884?workflow=create
[14:19:38] <YuviPanda>	 indeed, he does!
[14:19:41] <YuviPanda>	 even if he is tired and it's 4AM
[14:19:46] <YuviPanda>	 halfak best ak
[14:21:33] <YuviPanda>	 rtnpro: so that's three :D let me file the other thing halfak just mentioned
[14:26:04] <wikibugs>	 Analytics-Quarry: Make query URLs have a sluggified version of the title in them - https://phabricator.wikimedia.org/T75885#784350 (yuvipanda)
[14:26:09] <YuviPanda>	 rtnpro: ^
[14:26:43] <rtnpro>	 YuviPanda, thanks :)
[14:27:16] <wikibugs>	 Analytics-Quarry: Add pagination to user queries on user pages - https://phabricator.wikimedia.org/T75884#784365 (rtnpro) a:rtnpro
[14:27:58] <wikibugs>	 Analytics-Quarry: Make query URLs have a sluggified version of the title in them - https://phabricator.wikimedia.org/T75885#784369 (rtnpro) a:rtnpro
[14:54:30] <qchris>	 milimetric: Are you looking for https://www.mediawiki.org/wiki/Manual:Page_table#page_is_redirect ?
[15:00:39] <milimetric>	 thanks qchris_meeting aaron pointed that out too
[15:00:58] <qchris_meeting>	 k
[15:13:37] <grrrit-wm>	 (PS5) Milimetric: [WIP] Add schema for edit fact table [analytics/data-warehouse] - https://gerrit.wikimedia.org/r/167839 (owner: QChris)
[15:15:03] <grrrit-wm>	 (CR) Milimetric: "Sean, I added the page_is_redirect field. This is something I just learned about and is an interesting example of a schema change we may " [analytics/data-warehouse] - https://gerrit.wikimedia.org/r/167839 (owner: QChris)
[15:29:15] <milimetric>	 qchris_away: do you have an example gerrit change that shows how you unit test UDFs?  I was looking for examples in the commit stream and on gerrit and couldn't find it easily
[15:31:01] <qchris_meeting>	 Depends a bit on the kind of test ... but let me fetch one ...
[15:31:33] <ggellerman>	 Is there a link to staff mtg hangout?
[15:31:57] <qchris_meeting>	 ggellerman: It's on the calender. Let me grab it
[15:32:11] <qchris_meeting>	 ggellerman:  https://plus.google.com/hangouts/_/wikimedia.org/analytics?authuser=0&hceid=d2lraW1lZGlhLm9yZ19yMWNvaHVib3JmYjlqcWMydHA0bmwxcXMxZ0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29t.l8cef6ek2tmq63fdab3j9tg97k
[15:32:40] <qchris_meeting>	 milimetric: Are you looking for something like https://gerrit.wikimedia.org/r/#/c/169346/2/refinery-hive/src/test/java/org/wikimedia/analytics/refinery/hive/TestIdentifyRefererUDF.java
[15:32:41] <ggellerman>	 thanks! I don't see it on my calendar
[15:33:24] <qchris_meeting>	 ggellerman: I tried readding you, but Google said "Guest already invited" :-/
[15:33:42] <ggellerman>	 I'm in!  Thanks :-)
[16:50:53] <milimetric>	 qchris_away: thanks! that's exactly what I was looking for
[16:51:04] <ggellerman>	 will head to office after Research standup
[16:51:53] <nuria__>	 kevinator, milimetric: i do not have showcase meeting on my calendar
[16:51:59] <milimetric>	 hm weird
[16:52:39] <milimetric>	 nuria__: that's especially weird since I see it on your calendar :)
[16:53:15] <milimetric>	 nuria__: what client do you use for google calendar?  I noticed this happens to you often
[17:03:23] <nuria__>	 milimetric: chrome
[17:03:45] <nuria__>	 milimetric: now i see it, you know, it's the rendering
[17:04:20] <nuria__>	 milimetric: sometimes my calendar pages look all bizarre css wise (i think they are too heavy for my poor connection)
[17:04:23] <milimetric>	 extra weird!
[17:04:31] <milimetric>	 yeah! that must be it
[17:04:40] <milimetric>	 i noticed the same when i do it over LTE with bad reception
[17:07:17] <milimetric>	 nuria__, kevinator: how do you guys feel about pageview breakdown?  Are we showing that today?
[17:07:41] <nuria__>	 It's ready to show, i leave that up to kevin
[17:07:42] <milimetric>	 maybe an idea is to show just the pageviews in dashiki, and then as a side note like "hey this is almost done" click the breakdown button?
[17:07:44] <kevinator>	 hmm, I have not looked at it yet
[17:08:16] <nuria__>	 milimetric: the breakdown still has ux issues with labels , i think we need to work on it a bit ore
[17:08:18] <nuria__>	 *more
[17:08:19] <kevinator>	 I think we need to preface it that it won’t be in this week’s deploy
[17:08:47] <nuria__>	 also it'll be great not show "0" counts are they are meaningless
[17:08:48] <kevinator>	 We would have to be clear that it’s a work in progress
[17:09:33] <nuria__>	 kevinator: but we said we showcase stuff ready to go , deployed and such and that was a 34 item that got half a sprint so it still needs some work
[17:09:35] <milimetric>	 hm.. ok, let's not show it then
[17:09:55] <milimetric>	 that feels like the right call to me
[17:10:08] <kevinator>	 yes, let’s not show then
[17:10:47] <kevinator>	 I asked qchris if he wanted to show his EventLogging work
[17:11:02] <kevinator>	 I love how his graph shows before and after batch inserts
[17:11:26] <kevinator>	 Nuria, can you talk about you batch inserts work?
[17:12:22] <kevinator>	 agh, I’m thinking of too many things and can’t type
[17:12:40] <nuria__>	 kevinator: Chris graph covers it, it's what is deployed now
[17:13:05] <kevinator>	 what’s the URL again for dashiki on staging?
[17:13:39] <nuria__>	 kevinator: https://metrics-staging.wmflabs.org/static/public/dash/#projects=ruwiki,itwiki,dewiki,frwiki,enwiki,eswiki,jawiki/metrics=DailyPageviews
[17:14:29] <nuria__>	 kevinator, milimetric: brb
[17:14:33] <kevinator>	 Thanks
[17:24:15] <wikibugs>	 Analytics-EventLogging: Establish support model for EL? (e.g. more ops involvement) - https://phabricator.wikimedia.org/T75824#784874 (ggellerman)
[17:43:07] <kevinator>	 milimetric, nuria__ , mforns_lunch , qchris_away , ggellerman: I have updated the slide deck for the showcase today.
[17:43:21] <nuria__>	 kevinator: Link?
[17:43:36] <kevinator>	 https://docs.google.com/presentation/d/1siaxV4CVzx-Rqbs9zrEC9lmfhijweaNuRz1Tm_oyNcE/edit?usp=sharing
[17:46:35] <nuria__>	 kevinator: Do you want to show the pageviews?
[17:47:30] <kevinator>	 hmm, Dario has strong opinion against showing a metric that has not been sanctionned by Research
[17:48:06] <kevinator>	 I’d like to, and put it in the context that this metric will not be deployed immediately
[17:51:29] <kevinator>	 I’ll talk to Dario when he comes out of the research standup.
[17:51:33] <nuria__>	 kevinator: well, let me know
[17:58:32] <wikibugs>	 Analytics-EventLogging: Establish support model for EL? (e.g. more ops involvement) - https://phabricator.wikimedia.org/T75824#784980 (QChris) >>! In T75824#784874, @ggellerman wrote: > I welcome your help with correcting language that seems unclear or unfair :-)  I asked questions on the parts that is unclea...
[18:02:08] <DarTar>	 kevinator: what’s up? done with standup, I was hoping to ride to the office so I can join the showcase from there
[18:03:03] <kevinator>	 DarTar: I’m discussing with the devs what to showcase at 11
[18:03:06] <DarTar>	 kevinator, nuria__ it’s totally fine to showcase the PV work (it’s not on VS yet)
[18:03:22] <kevinator>	 It’s on the staging server
[18:03:34] <kevinator>	 and it looks like vital signs
[18:03:34] <DarTar>	 that’s fine with me
[18:03:36] <kevinator>	 ok
[18:03:54] <kevinator>	 we’ll  say it won’t be deployed until we get the right definition of PV
[18:03:57] <DarTar>	 before we move it to prod and we make a public announcement I’d like to review the messaging
[18:04:05] <kevinator>	 ok
[18:04:09] <DarTar>	 but you guys should present it today
[18:04:16] <Ironholds>	 wait
[18:04:28] <Ironholds>	 if it won't be deployed until we have a better PV definition
[18:04:32] <Ironholds>	 why was time put into building it?
[18:04:54] <kevinator>	 https://metrics-staging.wmflabs.org/static/public/dash/#projects=ruwiki,itwiki,dewiki,frwiki,enwiki,eswiki,jawiki/metrics=DailyPageviews
[18:05:07] <DarTar>	 it looks like we need to sit down and work on a rollout + announcement plan together
[18:05:08] <kevinator>	 We have a goal to display page views this quarter
[18:05:31] <kevinator>	 this was prioritized before objections were raised about using the webstats collector definition
[18:06:08] <DarTar>	 alright folks, I’ll see you in the office in 30
[18:06:15] <kevinator>	 ciao
[18:20:32] <nuria__>	  Ironholds question for kevinator , toby
[18:20:43] <Ironholds>	 nuria__, indeed
[18:20:45] <nuria__>	 Ironholds: we'll talk about taht on teh retro
[18:20:48] <nuria__>	 *that
[18:20:51] <nuria__>	 *the
[18:22:22] <milimetric>	 Ironholds: as far as I can tell we're interested in showing the metrics side by side for comparison.  what I was trying to say today was that most of the work that went into building it is generic and not tied to the definition
[18:22:38] <Ironholds>	 aha. Cool.
[18:45:15] <kevinator>	 milimetric, nuria__, mforns, qchris: are you all set for the showcase?  who will present the “demos”
[18:45:34] <qchris>	 I'll present the graph.
[18:45:54] <qchris>	 But I am not set yet. We have 15 more minutes :-)
[18:46:22] <mforns>	 this time there's nothing for wikimetrics
[18:47:35] <mforns>	 kevinator, could you pass the link to the presentation, please?
[18:47:44] <nuria__>	 kevinator: I can present the pageviews
[18:48:04] <kevinator>	 https://docs.google.com/presentation/d/1siaxV4CVzx-Rqbs9zrEC9lmfhijweaNuRz1Tm_oyNcE/edit?usp=sharing
[18:48:04] <mforns>	 thx
[19:03:25] <wikibugs>	 Analytics-EventLogging: nail down what EL process looks like and understand impact on system - https://phabricator.wikimedia.org/T75912 (ggellerman) NEW p:Triage
[19:04:56] <wikibugs>	 Analytics-EventLogging: Talk to research about Mobile EL pipeline use case - https://phabricator.wikimedia.org/T75913 (ggellerman) NEW p:Triage
[19:50:11] <jgage>	 yess i made elasticsearch run out of memory with that query at the end of my demo
[19:51:10] <jgage>	 out of curiousity, was anyone else playing with kibana at that time?
[19:51:39] <jgage>	 (19:30 utc)
[20:16:43] <yurikR>	 qchris, around?
[20:16:53] <qchris>	 yurikR: Yup. I got your email.
[20:16:59] <yurikR>	 thougts?
[20:17:01] <qchris>	 But gimme a few more hours.
[20:17:10] <qchris>	 Have not looked at it in depth.
[20:17:12] <yurikR>	 np, thx )
[21:15:10] <travis-ci>	 wikimedia/mediawiki-extensions-EventLogging#272 (master - 8e9fd2c : Ori Livneh): The build was broken.
[21:15:10] <travis-ci>	 Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/596a0ea22967...8e9fd2ca4f64
[21:15:10] <travis-ci>	 Build details : http://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/42127014
[21:29:33] <Ironholds>	 James_F, who would I talk to about VisualEditor URL schemes?
[21:30:18] <Ironholds>	 specifically, are things like https://pt.wikipedia.org/w/index.php?title=Luise_Rainer&action=submit you lot?
[21:37:06] <mforns>	 milimetric, regarding your idea of virtual rooms: maybe we could even have a work-room NOT for meetings, just to be there while working...
[21:38:30] <milimetric>	 mforns: totally
[21:39:53] <mforns>	 so when you need to talk to someone you can just unmute and talk
[21:40:27] <mforns>	 may be nice to hear other's comments on random stuff
[22:10:44] <James_F>	 Ironholds: Me, yes but not in the VE context, see e-mail.
[22:11:35] <Ironholds>	 yep, see my reply
[22:12:25] <James_F>	 Yep, see my reply.
[22:15:58] <Ironholds>	 yep, see my reply
[22:41:56] <Deskana>	 Hello!
[22:42:27] <Deskana>	 We're sending events and we're seeing the network requests for the events coming in, but I don't actually see the relevant events in the table.
[22:42:35] <Deskana>	 Is there some big replication lag at the moment?
[23:01:28] <qchris>	 Deskana: Which schema are you interested in?
[23:02:20] <grrrit-wm>	 (PS3) Mforns: Implement mediawiki-storage lib [analytics/mediawiki-storage] - https://gerrit.wikimedia.org/r/175467 (https://bugzilla.wikimedia.org/68448)
[23:04:04] <grrrit-wm>	 (CR) Mforns: [C: -1] "README.md is still in progress. The rest can be reviewed." [analytics/mediawiki-storage] - https://gerrit.wikimedia.org/r/175467 (https://bugzilla.wikimedia.org/68448) (owner: Mforns)
[23:06:30] <grrrit-wm>	 (PS1) Milimetric: Switch to metadata inference for all graphs [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/175887
[23:07:15] <qchris>	 Deskana: Which schema are you interested in?
[23:07:46] <Deskana>	 qchris: MobileWikiAppSearch_10641988
[23:08:05] <qchris>	 Events should come in as we talk?
[23:08:24] <nuria__>	 deskana: is this for mobile app?
[23:08:30] <nuria__>	 Deskana: or mobile web?
[23:10:15] <Deskana>	 nuria__: Mobile apps.
[23:10:24] <nuria__>	 k
[23:10:45] <qchris>	 I cannot find them in server nor client side events.
[23:10:55] <qchris>	 Gonna check the request stream.
[23:11:00] <yurikR>	 qchris, hi, any updates? )
[23:11:08] <qchris>	 yurikR: Check emails :-)
[23:11:20] <qchris>	 You should have one from <10 minutes ago.
[23:11:27] <yurikR>	 \o/  looking
[23:11:51] <nuria__>	 deskana, qchris: need to step out to school for 30 mins but i can look at this when i am back as it is getting kind of late for european TZ
[23:12:25] <Deskana>	 nuria__: Okay, thanks. We're preparing to push this to production so I want to make sure the problem isn't on our end. :)
[23:12:54] <yurikR>	 qchris, thx! will work from there :)
[23:13:16] <qchris>	 yurikR: Cool. Just tell dfoy to ignore the difference. This is not worth your and my time :-D
[23:13:45] <yurikR>	 qchris, i did many times, doesn't work that way - they base their projections/perf reviews on numbers :)
[23:14:07] <yurikR>	 plus that's how they communicate with partners
[23:14:17] <yurikR>	 your users viewed X pages, etc.
[23:14:20] <qchris>	 yurikR: :-)
[23:15:50] <qchris>	 Deskana: I grabbed 10s of request stream and cannot find any events for that Schema either.
[23:15:59] <qchris>	 How many requests do you expect per second?
[23:18:18] <qchris>	 Deskana: ^
[23:20:24] <qchris>	 Also, you said that you see the EventLogging requests on the network. Can you send me a (non private data containing) URL that would show me what you are sending?
[23:25:10] <qchris>	 Deskana: You still there?
[23:27:32] <DarTar>	 qchris: go to bed
[23:27:53] <qchris>	 :-)
[23:27:56] * qchris is not listening.
[23:28:08] <wikibugs>	 Analytics-EventLogging: Epic: Engineer manages throughput of events for his schema - https://phabricator.wikimedia.org/T75941 (kevinator) NEW p:Triage
[23:28:19] <DarTar>	 folks, do we have a dedicated log accessible on stat1003:/a/ only logging events that fail client-side validation?
[23:28:27] <DarTar>	 I am supposed to know but, errr, I forgot
[23:28:48] <qchris>	 DarTar: No, we don't.
[23:29:06] <DarTar>	 or do you have any monitoring tool that we can use to find errors in validation per schema?
[23:29:20] <qchris>	 grep :-)
[23:29:24] <DarTar>	 right
[23:29:31] <DarTar>	 zgrep | jq
[23:29:38] <qchris>	 But no. I am not aware of other tools right now.
[23:29:54] <DarTar>	 I kind of remember a graphite plot visualizing failed events
[23:30:02] <DarTar>	 but I was probably drunk
[23:30:23] <DarTar>	 ok cool, thanks qchris
[23:30:34] <qchris>	 We have raw and validated there. That gives you a count for the failed requests.
[23:31:53] <DarTar>	 qchris: leila and I are doing manual QA for a specific subset of logs but it would be fantastic to have a standard way of sending notifications alerting the software devs when a certain volume of events past a specific threshold fail validation from a specific schema
[23:32:15] * DarTar filing a BZ ticket >:-)
[23:32:27] <qchris>	 s/BZ ticket/Phab task/
[23:32:37] <DarTar>	 right
[23:32:38] <qchris>	 But that sounds worthwhile.
[23:32:50] <DarTar>	 cool, thanks, and remember the painting
[23:33:20] <qchris>	 DarTar: I do :-)
[23:35:26] <Deskana>	 qchris: I was away for a bit.
[23:35:51] <Deskana>	 qchris: I was testing it myself before. The schema is only active on my developer test instance, so you shouldn't be seeing any events now.
[23:36:03] <Deskana>	 qchris: Let me fire it up and you can look.
[23:36:42] <qchris>	 I am ready to capture :-)
[23:36:51] <Deskana>	 qchris: You should be seeing events coming in now.
[23:37:02] <qchris>	 Yup. There was one.
[23:37:40] <Deskana>	 Okay, that's good! It works!
[23:37:44] <Deskana>	 So why are they not appearing in analytics-store? :-/
[23:39:10] <Deskana>	 I don't see a table for them.
[23:39:18] <Deskana>	 Let me check again.
[23:39:32] <Deskana>	 Yeah, I only see MobileWikiAppSearch_10593635
[23:39:38] <Deskana>	 I don't see MobileWikiAppSearch_10641988
[23:40:01] <Deskana>	 qchris: Any thoughts?
[23:40:23] <qchris>	 Still searcing the other logs for the event.
[23:40:34] <Deskana>	 Okay, thanks.
[23:44:43] <wikibugs>	 Analytics-EventLogging: database consumer could batch inserts (sometimes) - https://phabricator.wikimedia.org/T69450#786602 (kevinator)
[23:45:25] <wikibugs>	 Analytics-EventLogging: database consumer could batch inserts (sometimes) - https://phabricator.wikimedia.org/T69450#720228 (kevinator)
[23:45:29] <wikibugs>	 Analytics-EventLogging: database consumer could batch inserts (sometimes) - https://phabricator.wikimedia.org/T69450#720228 (kevinator)
[23:47:33] <qchris>	 Found the event in the logs. So it looks good from there. Gonna try to insert on a local instance and see if it works there.
[23:53:30] <mforns>	 milimetric, nuria__: I pushed mediawiki-storage for review, although I'm still working on the readme file
[23:53:52] <mforns>	 the rest is ready for review
[23:55:04] <mforns>	 if you want to wait until everything is pushed, that's perfectly ok for me
[23:56:58] <mforns>	 milimetric, I also saw your email with the cohort issues, I'll have a look on that tomorrow morning
[23:57:13] <qchris>	 Deskana: Sorry, for missing the elephant in the room. The database is missing events for all schemas.
[23:57:15] <mforns>	 leaving now, see you tomorrow!