[00:46:06] Analytics-Backlog, Analytics-EventLogging, Privacy: Opt-out from logging some of the default EventLogging fields - https://phabricator.wikimedia.org/T108757#1530179 (Deskana) >>! In T108757#1529860, @Krenair wrote: > I think @tgr is talking about per-schema opt-out in the software, rather than a user c... [00:58:41] milimetric: are you still around? [01:17:04] Analytics-Backlog, Analytics-EventLogging, Privacy: Opt-out from logging some of the default EventLogging fields - https://phabricator.wikimedia.org/T108757#1530255 (Tgr) Indeed, I was thinking of a way to disable IP/useragent collection in the schema configuration (or `logEvent` call or whatever works... [01:18:00] milimetric: ok! 18 is looking good, its got replicas! [01:18:06] i'm going to install new version of kafka and restart [01:23:20] done [01:23:51] will do the same when 21 or 12 leaders have replicas [01:30:14] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1530270 (Tgr) Would it be difficult to change that? (Also, is it documented somewhere what those t... [01:33:54] ottomata: sweet, sorry I missed your ping [01:34:11] hey madhuvishy what's up [02:25:13] dashboard's looking good all around. Nice [03:35:41] sorry milimetric was having dinner. when i uncomment all the insert routes code, restbase doesn't start up fine [03:37:11] can you push the version you had last when we you were showing me the other day [05:52:00] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530596 (Tgr) ``` mysql:research@analytics-store.eqiad.wmnet [log]> select sum(event_xhrSupported and event_scriptAttributeSupported... [05:56:08] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530616 (Tgr) A sanity check where percentages are calculated over all browsers, not just CORS-supporting ones, does not show signifi... [05:58:01] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530631 (Tgr) [06:00:57] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530652 (Tgr) Open>Resolved [06:21:51] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530699 (TheDJ) @tgr, you should publish this on the blog or something. I'm sure there are other parties out there who would be inter... [06:29:41] Analytics, MediaWiki-extensions-ImageMetrics, Multimedia, Patch-For-Review: Measure how many users have CORS-hostile proxies - https://phabricator.wikimedia.org/T507#1530716 (Tgr) A bit technical for the blog though. Maybe a tweet? [08:45:07] Hi team [09:14:41] Hi joal [09:16:03] madhuvishy: let's do it together when you're at it again [09:17:50] graphs look good but there's a couple of weird spikes in bytes out and disk activity around 4 and 8, is camus running every 4 hours or something? I thought he turned it off [09:20:51] hey milimetric : Andrew switched camus after reastarting every node [09:21:15] probably around 4:30 [09:23:33] Log flush rate is back to flat, disk writes as well, still a bit of disk reads, but camus having to catch up is probably the cause of that one [09:26:31] Oh, i didn't know he did that. Cool, yeah saw all the brokers looking good so all that makes sense. We owe those guys on the list big! [09:26:46] Indeed ! [09:53:54] Analytics-Tech-community-metrics, ECT-August-2015: Ranking of repositories in Korma's code review page should update more often - https://phabricator.wikimedia.org/T102112#1531075 (Dicortazar) This task is already runnning daily, while the rest of the dashboard is updated every three days. Thus, all code... [11:28:28] milimetric: Camus doesn't catch up on missing hours it seems [11:38:48] hm... does it not try to? Or it tries and it's too far behind [11:39:15] It keeps loading everything up to now -2h, so I think it's an oozie issue more than camus [11:39:20] looking into it right now [11:40:12] oh interesting, somehow the offset it reads from isn't updated? [11:40:31] 2h sounds like the max it's allowed to consume [11:40:35] well that's weird: oozie says that some folders don't exist, while they do [11:40:44] liar! [11:40:46] :) [11:41:01] no no, was saying that basically camus is up-to-date, and it's a cluster issue :) [11:41:09] Exactly ! [11:41:28] I can't stand co;puters lying to me (for humans, it's different, they are humans after all) [11:41:47] :) [11:44:38] ok issue sorted out, some jobs needed a restart [11:46:02] nice, what was it? [11:46:46] It was oozie having tried to run jobs, but folders weren't there at the time [11:47:03] And I thought oozie would keep retrying to check for folders presence, but it seems it didn't [11:47:14] So I told it: retry, please :) [11:48:29] Only concern now is that it has slowed down the refine process (2 consecutive hours were missing, it blocked refinement) [11:48:42] And clauster will have a bit of work to catch back up [11:49:20] I should have double checked on folders before :S [12:03:46] Analytics-Tech-community-metrics, ECT-August-2015: Ranking of repositories in Korma's code review page should update more often - https://phabricator.wikimedia.org/T102112#1531338 (Qgil) One day for code review and three days for the rest looks is a very good frequency, thank you. Fell free closing this t... [12:06:35] how did you tell it to retry please [12:07:26] milimetric: cave for better explanation ? [13:00:43] sorry joal, doing some morning chores, we can talk later [13:01:08] maybe when you tell andrew so you don't have to do it twice [13:02:10] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [13:06:20] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [13:41:41] Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: EventLogging Icinga Alerts should look at a longer period of time to prevent false positives {stag} [5 pts] - https://phabricator.wikimedia.org/T108339#1531540 (mforns) Responding to Josephs CR comments here, because it's easier to quote his l... [13:57:22] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [13:59:51] Analytics-General-or-Unknown, Wikidata: Statistics for Wikidata API usage - https://phabricator.wikimedia.org/T64873#1531586 (Lydia_Pintscher) So I am a bit at a loss wrt the current state after these patches. Is there now a place where we can see stats for Wikidata's API usage? If so where? If not what... [14:03:06] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [14:04:06] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1531596 (Ottomata) I did a quick 5 minute googling, and it doesn't look like the varnishlog API ha... [14:05:04] helLOoOoOOOo [14:05:13] email checked, breakfast eatin. how dat der camus looking? [14:05:18] Good morning saviour of our kafkas ! [14:05:52] Everything back to normal on the kafka camus side [14:06:19] you said we missed some hours? [14:06:25] I took a few hours to react to jobs having to be restarted in oozie, so it's still backfilling [14:06:38] hadoop is busy [14:06:44] but looks good globally [14:06:48] did coordinators have to be restarted? or just some timed out workflows? [14:07:11] I reran some workflows [14:07:18] that is all [14:07:18] k [14:07:21] cool [14:07:27] thanks joal [14:07:39] But I thought oozie would have tried some of those by itself (hence my waiting) [14:07:42] np [14:08:30] I'm happily looking at flat flush rate and 0 replica lag :) [14:08:34] ottomata: --^ [14:08:35] indeed! [14:08:45] ok ok. so things to do today. I will not do anything in the migration plan to brokers today. i have lots of meetings, and should do some cleanup. [14:09:03] ok no problem [14:09:04] so, incident report, email about data loss (we should talk about that so we understand it and times), [14:09:09] gonna clean up some of the metrics stuff I hastily did. [14:09:23] oh, need to fix alerts, they will now be based on different metric names i thkn [14:09:26] maybe. [14:09:30] probably [14:09:33] i'll do the metrics alert stuff now then [14:09:58] joal: is camus still lagging? [14:10:06] or is it importing the last 10 minutes each run? [14:10:16] no lag in camus from what I can tell, but some in refinement [14:10:20] ok [14:10:33] hm. then we need to look at camus job. it is taking more than 10 minutes to run [14:10:35] maybe 30ish [14:10:36] or more [14:10:47] could be because there is more data in individual partitions [14:10:52] because of bits move [14:10:58] so, the text mappers have alot more each to do [14:11:10] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1531627 (BBlack) I don't think we'd want to even if we could, TBH. The privacy implications are b... [14:11:31] I was also removing bits from refinery oozie flows [14:11:34] we'll need to revisit webrequest_source with my vk change sometime soon. but in the meantime we may want to consider increasing partitions for webrequest partitions [14:11:38] topics* [14:11:44] cool, thanks joal. [14:11:50] joal, while you are at that, could you put in maps? [14:11:58] ottomata: surely [14:12:03] danke [14:12:04] hehe [14:12:09] s/bits/maps/ [14:12:17] almost ;) [14:12:30] shall we refine maps as well ? [14:12:33] eys [14:12:33] I guess yes [14:12:34] yes [14:12:36] yup [14:12:44] but it isn't part of pageviews or anything [14:12:49] so i think refine is all it needs [14:12:50] But no legacy_tsvs based on maps [14:12:53] aye [14:12:58] ok great [14:13:10] I'll do that and submit a patch later today [14:13:16] ok, gonna make some tickets! [14:13:49] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Build 0.8.2.1 Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1531633 (Ottomata) [14:14:05] ottomata: while I ma at oozie, I'll also add a sub-worflow for sending error emails [14:14:48] ottomata: can you send me the link to your CR about maps in camus please ? [14:16:12] https://gerrit.wikimedia.org/r/#/c/230535/ [14:16:14] ok cool! [14:16:32] btw, this bug joal: https://phabricator.wikimedia.org/T108717 [14:16:38] is the maps -> hadoop one [14:17:08] ok perfect, I'll assign it to myself [14:17:26] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [14:18:12] Analytics-Kanban, Discovery-Maps-Sprint, Patch-For-Review: Stream maps cluster requests into HDFS - https://phabricator.wikimedia.org/T108717#1531642 (JAllemandou) a:JAllemandou [14:18:50] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Build 0.8.2.1 Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1531644 (Ottomata) Phew, after much difficulty, the 4 original Precise brokers are now running 0.8.2.1. There was a bug in the version... [14:21:36] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [14:24:03] Analytics-Backlog, Analytics-Cluster: Reduce Camus runtimes - https://phabricator.wikimedia.org/T108830#1531655 (Ottomata) NEW a:Ottomata [14:35:29] Analytics-Kanban: Remove webrequest bits data from hadoop flows (camus, oozie) - https://phabricator.wikimedia.org/T108833#1531701 (JAllemandou) NEW a:JAllemandou [14:36:26] (PS1) Joal: Remove webrequest bits and add webrequest maps [analytics/refinery] - https://gerrit.wikimedia.org/r/231010 [14:37:51] Analytics-Kanban: Remove webrequest bits data from hadoop flows (camus, oozie) [3pts] {hawk} - https://phabricator.wikimedia.org/T108833#1531713 (JAllemandou) [14:38:20] Analytics-Kanban, Discovery-Maps-Sprint, Patch-For-Review: Stream maps cluster requests into HDFS [3pts] {hawk} - https://phabricator.wikimedia.org/T108717#1531715 (JAllemandou) [14:44:07] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [14:46:17] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [15:26:55] madhuvishy: what is the labs statsd host and port? [15:29:29] ottomata: labmon1001.eqiad.wmnet [15:29:31] 8125 [15:29:40] ahh k, danke, thanks [15:33:17] Analytics-Cluster, Analytics-Kanban: Read wiki dumps in Spark {hawk} - https://phabricator.wikimedia.org/T92385#1531854 (kevinator) Open>Resolved [15:41:37] (CR) Milimetric: [C: 2 V: 2] Add support for weekly frequency and granularity [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/230649 (https://phabricator.wikimedia.org/T108593) (owner: Mforns) [15:48:38] Analytics-Kanban, Patch-For-Review: Clean up mobile-reportcard dashboards {frog} [8 pts] - https://phabricator.wikimedia.org/T104379#1531881 (mforns) [15:49:01] Analytics-Kanban, Patch-For-Review: Date formatting bug on Vital Signs {crow} [1 pts] - https://phabricator.wikimedia.org/T108337#1531884 (ggellerman) [15:57:15] Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: EventLogging Icinga Alerts should look at a longer period of time to prevent false positives {stag} [5 pts] - https://phabricator.wikimedia.org/T108339#1531900 (JAllemandou) Thanks for vocal explanations :) Let's add the absolute function and... [15:59:35] hey joal, i'm thinking of canceling the ops checkpoint meetings now that gage is gone [15:59:41] i think you and I sync up enough [15:59:58] ottomata: I think we are in sync :) [16:00:21] sCouldn't easily place the sink there ... too bad ottomata [16:00:51] we in the sink?! [16:01:07] What are you sinking about ? [16:01:33] eh? [16:01:41] https://www.youtube.com/watch?v=gmOTpIVxji8 ottomata [16:01:47] Sorry, bad puns [16:02:20] hahahah [16:02:23] HAHAH [16:02:49] So please cancel that meeting :) [16:17:40] lol [16:30:29] (PS1) Joal: Update changelog.md before deployment. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/231042 [16:30:47] ottomata: if you have a second: --^ [16:48:33] (CR) Ottomata: [C: 2 V: 2] Update changelog.md before deployment. [analytics/refinery/source] - https://gerrit.wikimedia.org/r/231042 (owner: Joal) [16:48:42] thanks ottomata [16:51:16] Analytics-Backlog: Identify possible user identity reconstruction using location and user_agent_map pageview aggregated fields to try to link to IPs in webrequest - https://phabricator.wikimedia.org/T108843#1532043 (JAllemandou) NEW [16:51:31] git up [16:51:35] oops :) [17:03:44] (PS1) Milimetric: Fix annotations beyond last date [analytics/dashiki] - https://gerrit.wikimedia.org/r/231051 [17:03:57] mforns or madhuvishy, wanna do a quick review? ^ [17:04:36] I'll merge so I can have the dashboard fixed by scrum of scrums, but you can review and I'll address any problems in a later change [17:04:46] (CR) Milimetric: [C: 2 V: 2] Fix annotations beyond last date [analytics/dashiki] - https://gerrit.wikimedia.org/r/231051 (owner: Milimetric) [17:08:23] milimetric, saw your message now, will review [17:11:06] thx, sorry for the self-merge, it's deployed and working on vital-signs [17:11:40] milimetric, LGTM! [17:13:35] cool, th [17:13:36] x [18:11:02] Analytics-Backlog: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1532166 (mforns) NEW [18:13:35] Analytics-Backlog: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1532182 (mforns) [18:18:28] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532207 (Ottomata) We are eventually going to need a way to get more than 1014 bytes of data from... [18:20:21] (CR) Ottomata: [C: 2] "I'll let you merge and deploy Joseph." [analytics/refinery] - https://gerrit.wikimedia.org/r/231010 (owner: Joal) [18:21:45] joal: you wanna respond to Dario's page title normalization question? [18:22:02] I'm not familiar with the code and I figured the examples he gave shouldn't be happening [18:22:21] milimetric: I will do it [18:22:41] I have some emails to answer and I am procastinating a bit [18:24:11] np, just wondering if i should read up or move on [18:24:43] I think there might be a bug in the code [18:24:46] milimetric: --^ [18:25:00] makes sesne [18:25:09] I'll review [18:25:22] we could even re-do the existing page titles after you figure it out [18:25:31] but let's talk and let me know if you need any help [18:25:36] correct [18:26:12] I'll review the code, let you know of my findings, and we'll see [18:26:18] backfil page titles? is it worth it? page titles change anyway [18:29:00] ottomata: agreed [18:29:12] ottomata: could be possible though :) [18:30:10] ottomata: did the devops meeting move to now? [18:30:21] I have one for tomorrow I'm planning to attend ottomata. :D [18:30:58] the page titles change, but if we want to say "these page titles are denormalized all the way back in history", that would have to have a confusing caveat for the first couple months of history if we didn't fix it [18:31:04] leila, i think i was very confused [18:31:08] i think i had an old one on my cal for today [18:31:13] and was confused as to why no one else was on it [18:31:17] i believe tomorrow is correct [18:31:21] i deleted the one i had from today [18:31:47] I still see it ottomata in my calender as in invitation for today. Are you sure you deleted it? [18:32:39] I /think/ you've just created a monthly event for this time slot as well, ottomata. I see it in the following months as well. ;-) [18:35:06] i dunno, i deleted it from mine [18:35:12] not sure if it auto deletes from yours? [18:35:16] i am bad at google calendar [18:35:59] oh, leila how about now, did I do it? [18:36:15] lovely, it's fixed, ottomata. [18:38:35] Dan, I think I have found the bug [18:38:40] milimetric: --^ [18:39:49] joal: wanna talk about it? [18:39:57] cave ? [18:40:00] sure [18:40:37] btw, everyone, i'm about to deploy Event Logging, if anyone wants to join, I'll be in the batcave after I talk to jo [18:51:44] Analytics-Backlog: Set up bucketization of editCount fields {tick} - https://phabricator.wikimedia.org/T108856#1532299 (mforns) NEW [18:54:08] Analytics-Backlog: Set up bucketization of editCount fields {tick} - https://phabricator.wikimedia.org/T108856#1532311 (mforns) [18:56:39] Analytics-Backlog: Delete obsolete schemas {tick} - https://phabricator.wikimedia.org/T108857#1532313 (mforns) NEW [19:10:29] ottomata: I'm having trouble with archiva deployment :( [19:13:15] k, with you in 5 minutes. [19:18:35] hey joal, when you feel like oozie is all settled and cool [19:18:42] will you edit this comment and check the box? [19:18:42] https://phabricator.wikimedia.org/T106581#1531644 [19:18:46] (will it let you edit?) [19:19:28] ok joal waassuup with archiva? [19:19:46] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Build 0.8.2.1 Kafka package and upgrade Kafka brokers - https://phabricator.wikimedia.org/T106581#1532357 (JAllemandou) >>! In T106581#1531644, @Ottomata wrote: > Phew, after much difficulty, the 4 original Precise brokers are now ru... [19:20:03] issue when running javadoc :( [19:20:08] javadoC! [19:20:10] ? [19:20:18] yup [19:20:23] Weird, huh ? [19:20:26] whatcha mean? as part of deploy? [19:20:34] is that an artifact that gets deployed? [19:21:02] ottomata: statsd needs to be on eventlog1001 for my deploy to work [19:21:03] as part of deploy, yes [19:21:53] oo, python-statsd? [19:22:09] yeah, it's trying to get it from pypy otherwise [19:22:13] k on it... [19:22:26] hm, it is installed milimetric [19:22:26] i python-statsd - Python client for the statsd daemon [19:22:28] Searching for statsd>=3.1 [19:22:28] Reading https://pypi.python.org/simple/statsd/ [19:22:38] Version: 3.0.1-1 [19:22:41] do you need 3.1? [19:22:46] how about 3.0 ?: ) [19:22:47] madhuvishy: ? [19:22:48] donno, that's what madhuvishy put in there [19:22:52] ottomata: another issue in the mean time with hive [19:23:03] joal: heading to batcave [19:24:39] :) now it wants pykafka>=1.0.3 [19:24:58] oh! [19:25:00] ha, yes [19:25:13] you got it, will pupetize that [19:25:16] try now milimetric [19:25:25] all good, thx sir [19:30:27] madhuvishy: when you're back, I used statsd>=3.0 and it installed ok and is not throwing errors. I see the new metrics in graphite but I don't see any data yet [19:33:14] oh, of course, because we didn't configure it. But weird that the metrics are there in graphite :) [19:33:36] milimetric: maybe madhu was testing on an04? [19:33:57] that's true, then they'd be in there [19:34:23] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [19:36:23] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [20:02:59] milimetric: ottomata sorryyy i had to go somewhere in the morning [20:03:03] just got here [20:03:52] madhuvishy: np, do you think we can use statsd 3.0.1? [20:04:46] milimetric: yeah! i put that because it was the latest version, and i dint check what was on the server. my bad [20:04:56] cool, then just merge this and I'll deploy: https://gerrit.wikimedia.org/r/#/c/231140/ [20:05:51] also, though, we'll need to change the mysql url we configured in puppet to have the statsd host [20:06:02] if we want the new metrics to work [20:08:10] milimetric: I merged that [20:08:20] aah, right [20:08:31] yep, I deployed it [20:08:34] all is well [20:08:38] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532517 (Tgr) So how about bumping the limit @BBlack found in T91347#1249751 to 8192, raising the... [20:08:40] i'll submit a patch for the puppet stuff [20:11:15] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 20.00% of data above the critical threshold [30.0] [20:12:01] joal: all deployed :) [20:12:11] Thx ottomata [20:13:13] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 15.00% above the threshold [20.0] [20:19:53] Analytics-Backlog: Bug in pageview title extraction: change spaces to underscores after percent_decode (not only plus signs) - https://phabricator.wikimedia.org/T108866#1532543 (JAllemandou) NEW [20:19:56] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532544 (BBlack) >>! In T91347#1532207, @Ottomata wrote: > We are eventually going to need a way t... [20:20:11] Guys, I'll merge and deploy refinery tomorrow morning, and monitor [20:20:18] I'm done for tonight [20:20:22] See y'all ! [20:23:11] Analytics-Backlog: Double check Article Title normalization - https://phabricator.wikimedia.org/T108867#1532560 (Milimetric) NEW [20:24:09] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532567 (BBlack) Trying to answer this for myself, the original truncated URL from the top of this... [20:29:03] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532575 (Ottomata) > In a URL? What kind of data are considering eventlogging here that's so long?... [20:29:53] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532580 (Ottomata) > A lot of this seems like it could be compressed/reduced, and/or duplicates in... [20:33:24] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532587 (Tgr) >>! In T91347#1532544, @BBlack wrote: > Also, 2000-ish is a whole lot more palatable... [21:03:39] milimetric, yt? [21:03:53] hi mforns [21:04:00] i'm in a meeting [21:04:15] milimetric, ok, can you ping me when finished, please? [21:06:28] sure mforns but is it urgent? [21:06:31] it's late for you [21:06:39] no, not urgent [21:06:46] ok, we can talk tomorrow :] [21:07:26] yeah, sleep matters :) [21:07:27] nite [21:16:47] good night team, see ya tomorrow! [21:18:45] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532729 (BBlack) >>! In T91347#1532587, @Tgr wrote: > Actually all of that except the webHost come... [21:25:52] milimetric: should i just put in the prod statsd url here? https://github.com/wikimedia/operations-puppet/blob/production/manifests/role/eventlogging.pp#L172 [21:26:21] more like, does the url have to be configurable based on whether it's running on labs/prod [21:26:41] I see that it uses the prod url here - https://github.com/wikimedia/operations-puppet/blob/production/manifests/role/eventlogging.pp#L234 [21:26:57] and in the reporter role [21:39:58] https://gerrit.wikimedia.org/r/#/c/231170/ - made an initial patch [21:54:43] Analytics: Collect font support metrics - https://phabricator.wikimedia.org/T108879#1532874 (Tgr) NEW [21:58:36] Analytics, Design: Collect font support metrics - https://phabricator.wikimedia.org/T108879#1532891 (Tgr) [22:06:27] Analytics, Design: Collect font preference metrics - https://phabricator.wikimedia.org/T108884#1532933 (Tgr) NEW [22:06:54] Analytics, Design: Collect font preference metrics - https://phabricator.wikimedia.org/T108884#1532946 (Tgr) [22:07:07] Analytics, Design: Collect font support metrics - https://phabricator.wikimedia.org/T108879#1532874 (Tgr) [22:08:29] Analytics, Design: Collect font preference metrics - https://phabricator.wikimedia.org/T108884#1532933 (Tgr) A long time ago when this topic came up, @stevenwalling suggested [[ http://polarb.com/ | polar ]] as a source of UX inspiration. [22:18:52] Analytics-Backlog, Analytics-EventLogging, Traffic, operations: EventLogging query strings are truncated to 1014 bytes by ?(varnishncsa? or udp packet size?) - https://phabricator.wikimedia.org/T91347#1532990 (Tgr) >>! In T91347#1532729, @BBlack wrote: > Yeah, but they're there in the original requ... [22:21:37] leila: the query finished. it's on stat1002 at /home/madhuvishy/uniques-report/bot-detection [22:21:59] requests-per-user.hql is the query and requests-per-user.tsv has results [22:22:08] it's for July 1 2015. [22:22:13] hello madhuvishy. I'm going to a meeting in 10 min, and a hacking session after that until 5pm or so. Will look into it after that. [22:22:21] leila: sure [22:22:23] thank you! [22:22:31] np :) [22:27:51] milimetric: are we keeping our meeting in 15 min? [22:28:05] (wondering if I should remove the HOLD milimetric) [22:28:21] leila: yes, keeeping [22:28:30] great! :-) thanks milimetric. [22:39:05] Analytics-Backlog: Add referrer to pageviews_hourly - https://phabricator.wikimedia.org/T108886#1533055 (kevinator) NEW [22:39:21] milimetric: I'll be 15 min late, Ellery will be there on time though. you guys should start, I'll join at 7pm your time. [22:39:53] k, we shall gossip about you then [22:40:05] :D [23:02:02] Analytics, Analytics-Cluster, Fundraising Tech Backlog, Fundraising-Backlog, operations: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1533178 (atgo) p:Triage>Normal