[01:27:25] <grrrit-wm>	 (PS1) Milimetric: Refactor Compare layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/213967
[03:14:31] <wikibugs>	 Analytics-Tech-community-metrics, Engineering-Community: A new events/meet-ups extension - https://phabricator.wikimedia.org/T99809#1313707 (MZMcBride)
[03:18:28] <wikibugs>	 Analytics-Tech-community-metrics, Engineering-Community: A new events/meet-ups extension - https://phabricator.wikimedia.org/T99809#1313709 (MZMcBride) This is an interesting task. Thanks for filing it. :-)  I agree with @Qgil that we'd need to break this down a bit further, but it's definitely feasible.
[07:16:14] <wikibugs>	 Analytics-Tech-community-metrics, Engineering-Community: A new events/meet-ups extension - https://phabricator.wikimedia.org/T99809#1313930 (Qgil) #Possible-Tech-Projects?
[09:26:33] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314082 (JAllemandou) As per one of @ottomata previous comments, do we whitelist here or somewhere else ? Also, the definition  (langcode).(m|zer...
[09:39:15] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314095 (Yurik) This is exactly my point -- I don't know all the valid domains, and neither are most of researchers. The rule is basically: only...
[09:57:10] <wikibugs>	 Analytics-Tech-community-metrics: Median time to review for Gerrit Changesets, per month - https://phabricator.wikimedia.org/T97715#1314111 (Dicortazar) @Aklapper, ok, let's discuss about this, this Friday. I guess we have these data for Bugzilla (work in progress for Maniphest), but not for Gerrit, at least...
[11:32:31] <grrrit-wm1>	 (PS1) Mforns: Add stacked bars component to compare layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/214036 (https://phabricator.wikimedia.org/T91123)
[11:33:06] <grrrit-wm1>	 (CR) Mforns: [C: -1] "Still WIP" [analytics/dashiki] - https://gerrit.wikimedia.org/r/214036 (https://phabricator.wikimedia.org/T91123) (owner: Mforns)
[12:20:39] <ottomata>	 Morning!
[12:36:34] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314281 (Ottomata) Without a whitelist, @yurik's rule isn't codeable.  Maybe we should make something as simple as we can without a whitelist, bu...
[12:37:22] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314284 (Aklapper) >>! In T96371#1312109, @Dicortazar wrote: > * huggle (MISSING logs)  I can see them here: https://lists.wikimedia.org/pipermail/huggle/...
[12:38:35] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314290 (Aklapper) Uargh. Ignore my last comment; this is obviously about IRC logs and not mailing lists.
[12:40:20] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314299 (Dicortazar) @Aklapper, but those are mailing lists. I guess that in this ticket we're only interested in IRC channels.
[12:41:43] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314309 (Halfak) :(  Nope.   Hopefully @Ironholds has seen something like that.
[12:43:06] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: Code to write a new Camus consumer and store the data in two Hive tables [21 pts] {oryx} - https://phabricator.wikimedia.org/T98784#1314310 (Ottomata) As is, I'm not sure how this data could be mapped to a Hive Table.  We plan to currently only have one new Kafka...
[12:44:25] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314314 (Dicortazar) too later, sorry :(
[13:03:59] <halfak>	 o/ joal!
[13:04:07] <halfak>	 Just realized that I forgot to log into IRC
[13:04:50] <halfak>	 I should have said g'morning.  I don't get the chance all that often ^_^
[13:06:47] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314338 (Yurik) I would have no objections to making multiple fields instead of one -- language, sub domain, project. Than we can drop the www fo...
[13:09:56] <wikibugs>	 Analytics-Tech-community-metrics: Median time to review for Gerrit Changesets, per month - https://phabricator.wikimedia.org/T97715#1314344 (Aklapper) > Maybe I'm just too stupid to find this in korma, I think I once saw this?  This must have already existed at some time, seeing the screenshot in {T68265} (b...
[13:10:12] <joal>	 Hey halfak :)
[13:10:25] <joal>	 Don't worry, you still have a week ;)
[13:10:30] <halfak>	 :)  Yup
[13:10:43] <joal>	 Do we take3 some time today ?
[13:10:46] <joal>	 maybe now ?
[13:10:50] <joal>	 later ?
[13:10:51] <halfak>	 Sure.
[13:11:02] <halfak>	 Now is good.  Everyone else is sleeping.
[13:11:09] * halfak rubs hands together with EVIL intent
[13:11:15] <joal>	 Muhahaha :)
[13:12:05] <joal>	 batcave ?
[13:15:58] <halfak>	 Sure.  Will be in there in a monent
[13:16:01] <halfak>	 *moment!
[13:17:46] <joal>	 hmmm -> alternate batcave ?
[14:17:24] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314522 (Aklapper) Moved from petrb personal namespace to wm-bot namespace:  >>! In T96371#1312109, @Dicortazar wrote: > * huggle (MISSING logs)  http://b...
[14:55:44] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314616 (JAllemandou) Having dedicated fields instead of a generic one is the difficult part. For instance the language one is not present in eve...
[15:01:31] <joal>	 ottomata: Mooornihng :)
[15:01:38] <ottomata>	 morning!
[15:01:47] <joal>	 Quick question on kafka
[15:02:14] <joal>	 Can you explain why we don't have bytes out for event_logging-all this last 3 days ?
[15:02:32] <ottomata>	 oh, because the process to put data there isn't running?
[15:02:44] <joal>	 hm --> bytes out ?
[15:02:47] <ottomata>	 bytes out?
[15:02:59] <ottomata>	 what do you mean?
[15:03:17] <joal>	 The chart I am looking at is named BytesOutPerSec
[15:03:30] <ottomata>	 ay, sure, because there is nothing reading the data
[15:03:31] <joal>	 I assume it's about data that get out of kafka ?
[15:03:32] <ottomata>	 also there is no data so
[15:03:38] <ottomata>	 yeah that would be consumers
[15:04:01] <joal>	 Ok, weird though --> We are not supposed to produce events, or are we ?
[15:04:16] <ottomata>	 the thing I had producing events was just a hacky one off running in a screen
[15:04:31] <ottomata>	 if we want to produce events to kafka, we should really puppetize it on eventlog1001
[15:04:46] <ottomata>	 if you want some events to test though, i can restart my thing
[15:04:48] <joal>	 Yeah, but it's like it's been running for more than a month :)
[15:04:55] <ottomata>	 hm, no its not running
[15:05:27] <joal>	 http://ganglia.wikimedia.org/latest/graph_all_periods.php?hreg[]=analytics1012.eqiad.wmnet|analytics1018.eqiad.wmnet|analytics1021.eqiad.wmnet|analytics1022.eqiad.wmnet&mreg[]=kafka.server.BrokerTopicMetrics.%2B-BytesOutPerSec.OneMinuteRate&z=large&gtype=stack&title=kafka.server.BrokerTopicMetrics.%2B-BytesOutPerSec.OneMinuteRate&aggregate=1&r=hour
[15:05:32] <joal>	 Dark blue
[15:05:36] <joal>	 last chart
[15:05:46] <joal>	 I don't get it :)
[15:07:02] <ottomata>	 hm, yeah i would expect that to be 0
[15:07:17] <joal>	 Yeah, That's what I would have expected too !
[15:07:24] <joal>	 That's why I am asking :)
[15:11:58] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Community Metrics for IRC channels not updated since 09/2013 - https://phabricator.wikimedia.org/T96371#1314660 (Dicortazar) Thanks for the pointers!  The three channels are being updated just now.  Regarding to the list of them, we may move the discussion t...
[15:15:46] <qchris>	 joal: Are you refering to "analytics1012.eqiad.wmnet kafka.server.BrokerTopicMetrics.eventlogging-all-Byt..." in those graphs?
[15:16:00] <joal>	 indeed I do
[15:16:05] <joal>	 qchris: --^
[15:16:27] <qchris>	 They show "nan" for me right now. So no values are coming in.
[15:16:35] <qchris>	 But coloring in graphite is a bit weird ...
[15:16:44] <qchris>	 The blue in the graph is stemming from other metrics
[15:16:48] <qchris>	 like:
[15:17:07] <joal>	 Riiiiiight :)
[15:17:14] <joal>	 Get it qchris
[15:17:22] <qchris>	 analytics1021.eqiad.wmnet kafka.server.BrokerTopicMetricswebrequest_upload
[15:17:23] <joal>	 Tanks for that :)
[15:17:36] <qchris>	 yw. Ganglia is weird with colors :-/
[15:23:53] <ottomata>	 was about to say something similar:
[15:23:54] <ottomata>	 http://ganglia.wikimedia.org/latest/graph_all_periods.php?title=&vl=&x=&n=&hreg%5B%5D=analytics10(12%7C18%7C21%7C22)&mreg%5B%5D=kafka.server.BrokerTopicMetrics.eventlogging-all-BytesOutPerSec.OneMinuteRate&gtype=stack&glegend=show&aggregate=1
[15:23:58] <ottomata>	 only data for a short period
[15:45:53] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Analytics Team has a lager cluster {mule} - https://phabricator.wikimedia.org/T99952#1314767 (kevinator)
[15:45:55] <wikibugs>	 Analytics-Cluster, operations, procurement: Hadoop worker node procurement - 2015 - https://phabricator.wikimedia.org/T100442#1314768 (kevinator)
[15:53:12] <wikibugs>	 Analytics-Wikimetrics, MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631#1314794 (Memeht) @Millimetric, I'm trying to run wikimetrics locally. I've also created a google hangouts session and invited you to the event via your Phabricator calendar. Let me kno...
[16:01:16] <wikibugs>	 Analytics, MediaWiki-extensions-ContentTranslation: Limn language dashboard: eswiki graph is wrong/stuck - https://phabricator.wikimedia.org/T99074#1314819 (KartikMistry) @Milimetric Yes please, we need Sean now. Graph is still flat :/
[16:01:56] <wikibugs>	 Analytics, MediaWiki-extensions-ContentTranslation, ContentTranslation-Release5: Limn language dashboard: eswiki graph is wrong/stuck - https://phabricator.wikimedia.org/T99074#1284691 (KartikMistry)
[16:10:50] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Productionize reporting of app session metrics [8pts] {hawk} - https://phabricator.wikimedia.org/T97876#1314842 (kevinator) This was blocked on a bug using hive-context in spark. However, there is a workaround doing something similar to what is her...
[16:11:07] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314843 (Ironholds) I'm...pretty confused by all of this. I came up with the logic for this problem literally four months ago, while on the analy...
[16:13:04] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314844 (Ottomata) Ironholds, is that this?  https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/w...
[16:14:15] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314847 (Ironholds) I think that was adapted from my original code but not actually said original code (it might be, but I don't use switches ;p)
[16:15:22] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314854 (Ottomata) Did you actually implement?  This thing only returns project, but not project class.  There is more than just 'mobile' too.  W...
[16:15:29] <milimetric>	 madhuvishy: the three tasks I see for the Kafka EL pipeline are:
[16:15:30] <milimetric>	 https://gerrit.wikimedia.org/r/#/c/210701/9/server/eventlogging/handlers.py
[16:15:33] <milimetric>	 https://gerrit.wikimedia.org/r/#/c/210729/
[16:15:35] <milimetric>	 https://gerrit.wikimedia.org/r/#/c/210765/
[16:15:47] <madhuvishy>	 milimetric: right.
[16:15:56] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314855 (Ironholds) Then this was definitely _not_ my original code because I wouldn't have forgotten zero!
[16:16:02] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314856 (Ottomata) Sorry, take that back, it does return project class.  This is basically a normalized domain as is.  e.g. en.wikipedia, etc.
[16:16:08] <Ironholds>	 ottomata, suggest git blame for that UDF ;p
[16:17:26] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314861 (Ottomata) Well, I mean, it seems that this is attempting to return a normalized domain, independent of subdomains like m and zero.  Did...
[16:18:06] <ottomata>	 Ironholds: pretty sure joal has been working on that
[16:19:22] <ottomata>	 Ironholds: did you actually implement sucha  thing?
[16:22:34] <wikibugs>	 Analytics-Cluster, operations, Patch-For-Review: Turn off webrequest udp2log instances. - https://phabricator.wikimedia.org/T97294#1314869 (Ottomata)
[16:22:37] <wikibugs>	 Analytics-Cluster, operations, Patch-For-Review: Backport? and install kafkacat (on stat1002?) - https://phabricator.wikimedia.org/T97771#1314868 (Ottomata) Open>Resolved
[16:22:45] <Ironholds>	 ottomata, yes, and then I abandoned it because I was *assured* that nothing I did was of so much value that analytics didn't have the bandwidth to replicate it and take over my work with no loss of productivity
[16:22:52] <Ironholds>	 this seems to be working out...well, someone is happy somewhere ;p
[16:23:09] <ottomata>	 hahah
[16:23:10] <ottomata>	 uhhhhh
[16:23:11] <ottomata>	 ok
[16:23:25] <ottomata>	 dunno who you are talking to, i loved your patches
[16:24:27] <Ironholds>	 ottomata, management ;p
[16:31:04] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314883 (JAllemandou) I did write that, expecting to use is_zero and access_method to know whether zero or mobile.
[16:32:01] <joal>	 ottomata, Ironholds : Let's talk about blaming me ;)
[16:32:46] <Ironholds>	 joal, it's not a blame thing, dude
[16:32:54] <joal>	 Was joking :)
[16:32:55] <Ironholds>	 you did a UDF for a thing, we were wondering if it worked for a different thing ;p
[16:33:21] <joal>	 My thing is only a one thing thing, guys, sorry 'bout  the thing ...
[16:34:22] <joal>	 And, in comments, I suggested that it should only be used for pageviews --> we have no insurance it works when domain patterns haven't been double checked
[16:35:26] <joal>	 ottomata: Would you mind moving the analytics-ops checkpoint ?
[16:35:35] <ottomata>	 not at all
[16:35:38] <joal>	 I have a conflict :(
[16:36:08] <joal>	 also, what about domain stuff ?
[16:37:27] <joal>	 Thanks a lot dude
[16:37:32] <ottomata>	 wed same time
[16:37:44] <ottomata>	 we paused the convo to wait for you
[16:37:45] <joal>	 Any decision on how we are going to handle the domain splitting ?
[16:37:57] <joal>	 k
[16:38:04] <ottomata>	 i think we should have a meeting with you me yurik and madhu
[16:38:09] <ottomata>	 and dan
[16:38:09] <joal>	 Sounds a good idea
[16:38:29] <joal>	 Also, wanted to discuss with you having articles and project in the refined table
[16:38:37] <ottomata>	 articles?
[16:38:38] <joal>	 do we take some time ?
[16:38:40] <ottomata>	 page page name?
[16:38:41] <ottomata>	 like*
[16:38:46] <joal>	 yeah
[16:38:49] <ottomata>	 sure now is good
[16:38:57] <joal>	 Baaaaatcave !
[16:38:59] <ottomata>	 batcave!
[17:10:53] <wikibugs>	 Analytics-Cluster, Analytics-Kanban: Create new normalized uri_host field in refined webrequest table {hawk} [13 pts] - https://phabricator.wikimedia.org/T96044#1314981 (Yurik) Iszero is based on the presence of zero tag in xanalytics, not the zero sub domain in URL, from what I understood.
[17:53:24] <wikibugs>	 Analytics, MediaWiki-extensions-ContentTranslation, ContentTranslation-Release5: Limn language dashboard: eswiki graph is wrong/stuck - https://phabricator.wikimedia.org/T99074#1315063 (Milimetric) @Springle, It seems something is wrong with replication of eswiki, at least the user_properties table....
[18:52:48] <dr0ptp4kt_i_>	 milimetric: do you know if there are UAs outside of navigator.sendBeacon supporting ones that would succeed in sending an EL event from a browsers despite a GET operation being queued in the browser 'at the same time' as an attempted mw.track? i was looking at the Schema:Search, and have a query i think would be useful for Ironholds. but bmansurov noted i should see if maybe you know of the case where, for example, MSI
[18:53:22] <dr0ptp4kt_i_>	 milimetric: here's a query to see what i'm talking about SELECT count(*), (CASE when userAgent like '%MSIE%' then 'IE' else 'OTHER' END) as ua FROM Search_11670541 WHERE timestamp >= '20150526' AND timestamp < '20150527' AND event_action = 'click-result' group by ua;
[18:53:38] * milimetric reading
[18:53:41] <dr0ptp4kt_i_>	 milimetric: obviously UAs get spoofed and some UAs do it in an attempt to look compliant-ish
[18:54:17] <ottomata>	 oo hey dr0ptp4kt_i_, fyi, i packaged kafkacat for trusty and installed on stat1002
[18:54:25] <dr0ptp4kt_i_>	 the MSIE UAs for the click-result action constitute about 10%. the other ones *mostly* look like sendBeacon capable UAs (with a smattering of FF UAs that are supposedly too old, assuming they aren't forged)
[18:54:31] <ottomata>	 you wanted a tool to consume from webrequest logs, right?
[18:54:32] <dr0ptp4kt_i_>	 ottomata: weeeeee
[18:55:06] <dr0ptp4kt_i_>	 ottomata: yes, in lieue of the previous option
[18:55:11] <ottomata>	 cool, yup
[18:55:27] <ottomata>	 kafkatee is installed too, but kafkacat is a little easier to just use for one offs
[18:55:33] <ottomata>	 since it is not a daemon, doesn't need a config file, etc.
[18:55:40] <milimetric>	 dr0ptp4kt_i_: I don't know a list of such UAs from memory, I'd have to investigate
[18:56:27] <milimetric>	 dr0ptp4kt_i_:  the query you've got there seems ok, but if you want to run UA parser on it and then group by the output from that, I could help
[18:56:33] <dr0ptp4kt_i_>	 milimetric: of sendbeacon ones? https://developer.mozilla.org/it/docs/Web/API/Navigator/sendBeacon documents stuff. search engining for sendbeacon IE didn't seem to turn up anything
[18:56:51] <dr0ptp4kt_i_>	 is there a mysql stored proc for that in the el db?
[18:57:10] <milimetric>	 um... there's a UDF on hadoop and i realize that doesn't help you
[18:57:15] <dr0ptp4kt_i_>	 milimetric: or just get you the file?
[18:57:20] <milimetric>	 Ironholds: how'd you do UA parsing in mysql world?
[18:57:47] <milimetric>	 dr0ptp4kt_i_: what file? You're looking in the log db right?
[18:58:23] <dr0ptp4kt_i_>	 milimetric: sorry, meant to say you just want the dump of the distinct UAs? i guess you could run the query to get those, too :)
[18:58:53] <dr0ptp4kt_i_>	 milimetric: you know, SELECT DISTINCT userAgent, count(*) FROM Search_11670541 WHERE timestamp >= '20150526' AND timestamp < '20150527' AND event_action = 'click-result' group by userAgent;    or its ilk
[18:59:22] <milimetric>	 dr0ptp4kt_i_: yeah, so you're basically saying that if a UA is sending a 'click-result' event, then it's capable of either sendBeacon or "mw.track + GET at the same time"
[18:59:56] <milimetric>	 When you say "at the same time", wouldn't that be ... all browsers that support JS?
[19:02:49] <dr0ptp4kt_i_>	 milimetric: it seems like it. i couldn't grok https://github.com/wikimedia/mediawiki/search?utf8=%E2%9C%93&q=mw.track and https://github.com/wikimedia/mediawiki-extensions-EventLogging/search?utf8=%E2%9C%93&q=mw.track and https://github.com/wikimedia/mediawiki-extensions-EventLogging/search?utf8=%E2%9C%93&q=sendbeacon well enough to tell what may be going on there
[19:03:39] <dr0ptp4kt_i_>	 milimetric: i was thinking maybe there's some sort of race condition that avails itself. if the race condition would always succeed on certain classes of UAs, i guess that would be fine, but if it wouldn't avail itself in a predictable manner for given UAs, that would be nasty
[19:04:05] <dr0ptp4kt_i_>	 milimetric: as far as you know, though, does mw.track use sendBeacon exclusively when a UA supports it?
[19:05:21] <milimetric>	 dr0ptp4kt_i_: yes, it uses sendBeacon wherever it's supported
[19:05:24] <milimetric>	 or, if it doesn't, that's a bug and we should fix it
[19:05:54] <milimetric>	 dr0ptp4kt_i_: so are you seeing a problem, is that why you're looking into this?  Like, you're seeing fewer events of that kind than you'd expect?
[19:05:59] <dr0ptp4kt_i_>	 milimetric: ok...probably best if i add a caveat. we can always confine the analysis to sendBeacon supporting UAs.
[19:06:34] <dr0ptp4kt_i_>	 milimetric: well, here's the thing...the distinct UAs for click-result events is much lower than the distict UAs for all events in that schema for the same day. it seems like that isn't a coincidence
[19:06:40] <milimetric>	 dr0ptp4kt_i_: but even if sendBeacon isn't supported, I don't see any reason why mw.track would fail other than the browser tab is closed
[19:06:44] <dr0ptp4kt_i_>	 er, it seems like that's for a reason :)
[19:07:15] <milimetric>	 dr0ptp4kt_i_: I see, and that's the only situation where you're also issuing a GET around the same place in the code?
[19:07:23] <dr0ptp4kt_i_>	 milimetric: more specifically, there were 473 UAs for click-result
[19:07:33] <dr0ptp4kt_i_>	 milimetric: and there were 1201 total UAs
[19:07:34] <milimetric>	 what's "click-result"
[19:07:51] <milimetric>	 like, what's the feature do in the browser?
[19:08:06] <dr0ptp4kt_i_>	 milimetric: it's when a user clicks on one of the recommendations from the "suggest" field in the searchbar in the topright part of desktop
[19:08:48] <milimetric>	 dr0ptp4kt_i_: that makes sense, it would have to support sendBeacon then, no?  because you're navigating when you click, not just doing an ajax GEt
[19:09:00] <milimetric>	 oh that's what you were saying, you never said ajax  GET, I just assumed that
[19:09:15] <milimetric>	 dr0ptp4kt_i_: yes, so this instrumentation would only work with sendBeacon as far as I can tell
[19:09:36] <dr0ptp4kt_i_>	 milimetric: ok. the strange things we see in browsers.
[19:09:42] <dr0ptp4kt_i_>	 and with browsers
[19:11:03] <milimetric>	 hm, that's still weird though, it should still have already sent the event, it's not like the sending it self is async
[19:11:05] <milimetric>	 hm hm
[19:11:38] <milimetric>	 hey, the distinct UAs are up to 474 now! progress
[19:17:31] <dr0ptp4kt_i_>	 milimetric: ha!
[19:19:47] <milimetric>	 dr0ptp4kt_i_: so I got the distinct UAs that never show up in a click-result event, and I looked through them, and all of them seem like they support send beacon
[19:20:02] <milimetric>	 lots of Mozilla, IE 10, Safari, Chrome
[19:20:30] <milimetric>	 so my conclusion is that probably just fewer people are clicking the results and more people are just hitting enter
[19:22:46] <milimetric>	 so then I did "SELECT if(event_action = 'click-result', 'click-result', 'other'), count(*) FROM Search_11670541 WHERE timestamp >= '20150526' AND timestamp < '20150527' group by 1;" and I conclude that "click-result" just doesn't happen often enough to get you as high a variation as the other stuff
[19:23:12] <milimetric>	 dr0ptp4kt_i_: because you've got no more than 3000 events in click-result, but over 110,000 in the rest
[19:23:33] <milimetric>	 so, nothing sinister here, move along :)
[19:23:52] <milimetric>	 and, I take back what I said, I don't think sendBeacon is required for this
[19:24:02] <milimetric>	 because the navigation away from the page should come after the sending of the event
[19:24:19] <milimetric>	 the only thing that won't happen is maybe the acknowledging of the send back to mw.track, but that's not required
[19:26:10] <wikibugs>	 Analytics-Wikimetrics, MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631#1315351 (Memeht) [[ https://phabricator.wikimedia.org/p/Milimetric/ | @Milimetric ]] So sorry got the times screwed up, are you available later today still?
[19:26:53] <dr0ptp4kt_i_>	 milimetric: ok...yeah, i think for the non-sendBeacon UAs then, there's some possibility of click-result making it...or not making
[19:27:16] <wikibugs>	 Analytics-Wikimetrics, MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631#1315356 (Milimetric) @Memeht: yes.  I can grab some lunch and hang out now if you want.
[19:27:43] <dr0ptp4kt_i_>	 milimetric: remember, each keystroke for these 1/1000 inclusion (on a per page basis) can theoretically generate an event
[19:29:21] <dr0ptp4kt_i_>	 milimetric: i guess i shouldn't say 'remember', it's not like we've talked about this before! :)
[19:29:58] <milimetric>	 :)
[19:30:18] <wikibugs>	 Analytics-Wikimetrics, MediaWiki-Vagrant: Vagrant Setup alembic config errors - https://phabricator.wikimedia.org/T99631#1315362 (Memeht) [[ https://phabricator.wikimedia.org/p/Milimetric/ | @Milimetric ]] That would be wonderful! Say, 1p.m PST?
[19:31:04] <milimetric>	 dr0ptp4kt_i_: from an old Steven Wright joke: "I like reminiscing with complete strangers"
[19:31:53] <ottomata>	 joal: btw, i think i fixed AM proxy problem
[19:32:00] <ottomata>	 but nodemanagers will need restart to pick up the change
[19:32:04] <joal>	 ottomata: really ?
[19:32:06] <ottomata>	 ja
[19:32:13] <ottomata>	 it is the RM HA change that did it
[19:32:16] <joal>	 will try that tomorrow :)
[19:32:19] <ottomata>	 https://issues.apache.org/jira/browse/YARN-2713
[19:32:21] <joal>	 I had guessed so
[19:32:26] <ottomata>	 well, i haven't restarted them
[19:32:34] <ottomata>	 guessi could slowly one by one :/
[19:32:39] <ottomata>	 yeahhHHH i'll do it :)
[19:32:52] <joal>	 :)
[19:37:09] <joal>	 YEAH :)
[19:37:47] <joal>	 hmfff --> One time good, one time no good :)
[19:42:21] <ottomata>	 joal: yay!
[19:42:21] <ottomata>	 https://yarn.wikimedia.org/proxy/application_1430945266892_35235/
[19:42:35] <ottomata>	 this is just a workaround, fyi
[19:42:45] <ottomata>	 if 1002 gets promoted to active RM
[19:42:53] <ottomata>	 it will break again
[19:42:57] <ottomata>	 1001 is hardcoded as the url to use now
[19:42:57] <joal>	 ok, understood
[19:43:04] <ottomata>	 until we get that bugfix
[19:43:06] <ottomata>	 but ja, mostly will be fien
[19:43:06] <joal>	 Better than nothing
[19:43:11] <joal>	 Thanks a lot ottomata :D
[19:43:13] <ottomata>	 yup!
[19:43:17] <joal>	 Will go to bed now ;)
[19:43:25] <joal>	 Have a good end of day !
[19:44:17] <ottomata>	 k laaters!
[19:55:28] <dr0ptp4kt_i_>	 wrapping up. peace
[20:51:09] <wikibugs>	 Analytics-Cluster, Fundraising Sprint Kraftwerk, Fundraising Sprint L, operations: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1315643 (atgo)
[20:51:13] <wikibugs>	 Analytics-Cluster, Fundraising Sprint Kraftwerk, operations: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1315644 (AndyRussG)
[20:51:54] <wikibugs>	 Analytics-Cluster, Fundraising Sprint Kraftwerk, Fundraising Sprint L, operations: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1249877 (AndyRussG)
[21:39:34] <grrrit-wm>	 (PS2) Mforns: Add stacked bars component to compare layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/214036 (https://phabricator.wikimedia.org/T91123)
[21:39:56] <grrrit-wm>	 (CR) Mforns: [C: -1] "Still WIP" [analytics/dashiki] - https://gerrit.wikimedia.org/r/214036 (https://phabricator.wikimedia.org/T91123) (owner: Mforns)
[21:40:17] <ottomata>	 milimetric: python funtime brainbounce!
[21:40:20] <ottomata>	 ehhh?
[21:41:40] <milimetric>	 ottomata: ok, batcave
[22:04:51] <wikibugs>	 Analytics-Tech-community-metrics: Font used for "Organizations" header on contributors.html looks a bit out of place - https://phabricator.wikimedia.org/T100569#1315877 (Aklapper) NEW
[22:25:27] <mforns>	 milimetric, yt?
[22:36:17] <milimetric>	 mforns: hey, was out for a sec
[22:36:18] <milimetric>	 what's up
[22:36:35] <mforns>	 milimetric, hey
[22:36:51] <mforns>	 question on Dashiki
[22:37:00] <milimetric>	 shoot
[22:37:16] <mforns>	 I'm adding tests for the data converter I wrote, but require fails
[22:37:26] <mforns>	 do you have an idea of what is happening?
[22:37:55] <mforns>	 the first require works fine, the one that requires my converter
[22:38:07] <milimetric>	 which one fails?
[22:38:10] <mforns>	 but the require inside the converter that requires timeseriesData fails
[22:39:39] <milimetric>	 mforns: is it requiring it at the top level or inside one of the inner converter functions?
[22:39:51] <milimetric>	 they only work at the top level
[22:40:03] <milimetric>	 I think my latest patch is needed for what you're doing
[22:40:09] <milimetric>	 and I haven't been able to submit it yet, sorry :(
[22:40:21] <mforns>	 milimetric, no problem!
[22:40:25] <milimetric>	 maybe just push your code and we can hang out tomorrow and think over it together
[22:40:37] <milimetric>	 I'm having the *stupidest* problem ever
[22:40:40] <mforns>	 xD
[22:40:41] <mforns>	 ok
[22:40:45] <mforns>	 what is it?
[22:40:46] <milimetric>	 I added filters on the timeseries
[22:40:50] <milimetric>	 and I can't merge the filters
[22:41:01] <milimetric>	 I also added the ability to have multiple rows for each date
[22:41:13] <milimetric>	 and I'm merging that totally fine, it's a little  trickhy
[22:41:17] <mforns>	 oh!
[22:41:24] <milimetric>	 but these damn fromDate and toDate, I can't merge it!!
[22:41:24] <milimetric>	 ugh
[22:41:25] <mforns>	 cool
[22:41:39] <mforns>	 mmm
[22:41:40] <milimetric>	 anyway, I'll figure it out, we can talk tomorrow when I'm victorious instead of defeated :)
[22:42:10] <mforns>	 ok, and if you are still strugling, I'll try to help
[22:42:45] <mforns>	 milimetric, the latest code, except for the tests is already pushed [WIP]
[22:42:54] <milimetric>	 ok, cool
[22:43:25] <milimetric>	 this is gonna sound crazy, but I spray painted an iron fence  this weekend
[22:43:31] <milimetric>	 and I honestly think I huffed too much of the spray
[22:43:43] <milimetric>	 so I think I'm just struggling 'cause I'm still dizzy from that
[22:43:45] <milimetric>	 lol
[22:43:57] <mforns>	 milimetric, the only thing missing are the test, and ah! also, I noticed 15 mins ago that all the quotes in the stacked-bar code are mixed single and double, so I'll have to review that also
[22:44:08] <mforns>	 xDDDDD
[22:44:31] <milimetric>	 ok, sounds good.  Hopefully there's not too much rework after my patch, but we'll see
[22:45:17] <mforns>	 this remembers me of one day in brazil, in my former company, where some guys were using glue to stick a whiteboard to the wall
[22:45:22] <milimetric>	 omg i figured it out
[22:45:25] <milimetric>	 lol god i'm silly
[22:45:47] <milimetric>	 I have to remember - always bitch about my problems to Marcel, that fixes them
[22:45:51] <mforns>	 one of the guys went reeeeally crazy sniffing the glue, and started to make as if he was swimming in the air
[22:45:53] <milimetric>	 lol @glue stick
[22:45:59] <milimetric>	 ahahahaha
[22:46:18] <milimetric>	 I've seen similar stuff when I was a kid in school, that's really funny
[22:46:24] <milimetric>	 I hope he's ok?
[22:46:48] <mforns>	 xD, yes I suppose, didn't know the man
[22:47:23] <mforns>	 milimetric, so you found out the problem with the filter merging problem?
[22:47:56] <milimetric>	 yes
[22:47:59] <milimetric>	 pushing fix now
[22:48:00] <mforns>	 ok
[22:48:32] <milimetric>	 I did the merge / mergeAll functions two different ways before, and I was using my memory instead of my eyes
[22:48:48] <grrrit-wm>	 (PS2) Milimetric: Refactor Compare layout to use TimeseriesData [analytics/dashiki] - https://gerrit.wikimedia.org/r/213967
[22:49:25] <mforns>	 milimetric, tomorrow morning I'll merge your latest changes and try to test, can you push them even if they are not final?
[22:50:01] <milimetric>	 mforns: that last patch I just pushed is the second to last.  I'll push one more tonight and that should be it
[22:50:16] <milimetric>	 and it'll be final, I promise
[22:50:16] <mforns>	 oh! thanks!
[22:50:30] <mforns>	 :]
[22:51:20] <mforns>	 milimetric, I'll sign off then
[22:51:28] <milimetric>	 cool, have a nice night
[22:51:32] <mforns>	 see you tomorrow, g'night!
[22:51:38] <milimetric>	 'morro
[23:56:44] <grrrit-wm>	 (PS1) Milimetric: Use Dygraphs in Vital Signs [analytics/dashiki] - https://gerrit.wikimedia.org/r/214270
[23:57:31] <grrrit-wm>	 (PS2) Milimetric: Use Dygraphs in Vital Signs [analytics/dashiki] - https://gerrit.wikimedia.org/r/214270 (https://phabricator.wikimedia.org/T96339)