[07:43:33] <wikibugs>	 Analytics-Backlog: Update maxmind database on Hadoop - https://phabricator.wikimedia.org/T109039#1538539 (JAllemandou) NEW
[08:34:58] <eranroz>	 hi, is there a dataset for getting what users are focusing *in specific* page? something like clicktale heatmaps?
[08:40:35] <wikibugs>	 Quarry: Every second attempt to use Quarry to do an SQL query fails - https://phabricator.wikimedia.org/T109014#1538688 (yuvipanda) Open>Resolved a:yuvipanda Oops, I had pooled in another runner host and forgot to do the manual provisioning of /etc/hosts (boo!). Fixed now.  Thanks for reporting it!
[09:01:14] <wikibugs>	 Analytics, Reading-Web: make MobileWebUIClickTracking schema usable - https://phabricator.wikimedia.org/T108723#1538751 (Jhernandez) Does this have to do with the recent emails about agregation and purging? (eg: removing the edit count to make it bucketed, etc)
[11:35:00] <wikibugs>	 Analytics, MobileFrontend, Reading-Web: Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1539217 (Jhernandez)
[12:04:13] <ottomata>	 mornin joal!
[12:04:17] <joal>	 Hi ottomata !
[12:04:23] <joal>	 how are you ?
[12:05:10] <ottomata>	 am gooOOoOd how are you?
[12:05:13] <ottomata>	 how dem der jobs doin?
[12:05:24] <joal>	 backfilling :)
[12:06:20] <joal>	 in the course of doing 2015-08-12 pageview and so on (load & refine are done)
[12:06:32] <joal>	 2015-08-13 ongoing for load and refine
[12:06:40] <joal>	 camus still catching up
[12:08:19] <ottomata>	 camus is about 10  hours behind
[12:09:03] <joal>	 should be ok tomorrow I think
[12:17:33] <joal>	 ottomata: load jobs look really better
[12:17:56] <joal>	 ottomata: still some (too much) failures, but many oks
[12:19:44] <joal>	 ottomata: last camus run: only 6 hours behind !
[12:19:48] <joal>	 Getting there !
[12:20:59] <joal>	 ottomata: question for you on kafka dashboards:partitions --> why is there a bump yesterday at about 20:00 ?
[12:21:09] <joal>	 Have you changed something ?
[12:22:42] <ottomata>	 noooo?
[12:22:50] <ottomata>	 great question
[12:22:50] <joal>	 I let you have a look
[12:22:51] <ottomata>	 looking
[12:23:16] <ottomata>	 oh, the __consumer_offsets topic now exists!
[12:23:23] <wikibugs>	 Analytics-Kanban, Research-and-Data: Bug in pageview title extraction: change spaces to underscores after percent_decode (not only plus signs) - https://phabricator.wikimedia.org/T108866#1539348 (JAllemandou) a:JAllemandou
[12:23:26] <ottomata>	 ha, ithink madhu might have been testing the python consumer stuff!
[12:23:37] <ottomata>	 she can't create real topics, but the kafka managed offset topics now exist
[12:23:37] <joal>	 Ok makes sense :)
[12:24:01] <ottomata>	 aye + 50 partitions
[12:24:35] <grrrit-wm>	 (PS1) Joal: Correct pageview title normalization bug [analytics/refinery/source] - https://gerrit.wikimedia.org/r/231545
[12:24:45] <joal>	 wow, that's a lot !
[12:29:42] <grrrit-wm>	 (CR) Ottomata: [C: 1] Correct pageview title normalization bug [analytics/refinery/source] - https://gerrit.wikimedia.org/r/231545 (owner: Joal)
[12:45:11] <ottomata>	 ok cool, joal, so load and refine done through 08-12?
[12:45:34] <ottomata>	 i will take some time today to look harder at data from 08-10 through 08-12 and try to estimate and document what has been lost
[12:53:58] <ottomata>	 running to post office, back in a bit, might head to a cafe soon tohugh too
[12:54:02] <joal>	 ottomata: indeed load and refine are done for 08-12
[12:54:11] <joal>	 thx for looking into that
[12:55:04] <joal>	 ottomata: if you find a good process to check how much data is lost, it would be reat to apply it on the 08-03/04 days as well
[13:31:12] <wikibugs>	 Analytics: Page view statistics looks not updated since 2015-08-13 at 18:00:00 - https://phabricator.wikimedia.org/T109070#1539558 (xcombelle) NEW
[13:40:59] <wikibugs>	 Analytics: Page view statistics looks not updated since 2015-08-13 at 18:00:00 - https://phabricator.wikimedia.org/T109070#1539591 (JAllemandou) This is a known problem. We have experience issues last weeks upgrading Kafka cluster (see [1] [2]), and now that this is solved we are cleaning data / rerunning jo...
[14:10:04] <ottomata>	 joal, just to reconfirm, load has been rerun for all of aug 11 and 12, right?
[14:10:07] <ottomata>	 seq stats recalculated?
[14:11:52] <joal>	 ottomata: confirmed
[14:12:34] <ottomata>	 danke
[14:12:39] <joal>	 np
[14:12:55] <joal>	 currently backfilling load/refine for the 13th
[14:13:12] <joal>	 and finishing regular jobs of the 12th
[14:14:29] <joal>	 I think this current finishing camus job could possibly the last long one (if not this one, then next with quasi certainty) ottomata
[14:14:58] <ottomata>	 aye cool!
[14:19:17] <wikibugs>	 Analytics-Backlog, Analytics-Dashiki, VisualEditor, Patch-For-Review: Improve the edit analysis dashboard {lion} - https://phabricator.wikimedia.org/T104261#1539654 (Milimetric) @Neil_P._Quinn_WMF, see the patch attached to probably get more clarity about the stuff we were talking about last week....
[14:19:40] <milimetric>	 ottomata: got a sec to help me brainstorm about the restbase puppet role?
[14:20:51] <ottomata>	 yes
[14:25:41] <ottomata>	 milimetric:  in batcave
[14:30:52] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Ranking of repositories in Korma's code review page should update more often - https://phabricator.wikimedia.org/T102112#1539667 (Dicortazar) Ok, so I'm closing this task since the server is already running and the dataset was updated.
[14:31:59] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Ranking of repositories in Korma's code review page should update more often - https://phabricator.wikimedia.org/T102112#1539669 (Dicortazar) Open>Resolved
[14:39:01] <wikibugs>	 Analytics-Tech-community-metrics, Engineering-Community, ECT-August-2015: Automated generation of repositories for Korma - https://phabricator.wikimedia.org/T104845#1539680 (Dicortazar) Some tasks done within this task: * [[ https://github.com/MetricsGrimoire/Octopus | Octopus ]], the tool to automatic...
[14:44:44] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1539685 (Dicortazar) When updating the list of repositories at {T104845} by Octopu (the tool in charge of retrieving the projects in Gerr...
[14:51:11] <milimetric>	 ottomata: in deployment-prep/common.yaml, why are some lines like cassandra::rpc_address: and others like "cassandra::seeds":
[14:51:16] <milimetric>	 (some have quotes and others don't)
[14:51:35] <ottomata>	 don't know
[14:51:59] <milimetric>	 :)
[15:00:16] <ottomata>	 joal: i want to rerun load for for some hours on 08-10
[15:00:18] <ottomata>	 sorry
[15:00:20] <ottomata>	 on 08-03
[15:00:23] <ottomata>	 and maybe on 08-10
[15:01:12] <ottomata>	 we have loss recorded in seq stats for 08-03 10-23, and i just checked a couple of those raw hours and saw 0 loss
[15:01:17] <ottomata>	 checking a few hours on 08-10 no
[15:01:18] <ottomata>	 now
[15:01:24] <ottomata>	 i expect the 08-10 to be real loss for sure
[15:01:39] <ottomata>	 mind if i start a load job for some hours on 08-03?
[15:01:51] <ottomata>	 just going to rerun the oozie coord actions, i think...
[15:02:49] <joal>	 ottomata: No prob
[15:03:10] <joal>	 I mean, there is stuff running, but it's not that bad ;)
[15:03:18] <ottomata>	 aye
[15:06:09] <wikibugs>	 Analytics-Backlog: Enforce policy for each schema: Sanitize {tick} [8 pts] - https://phabricator.wikimedia.org/T104877#1539750 (mforns)
[15:06:24] <wikibugs>	 Analytics-Kanban: Enforce policy for each schema: Sanitize {tick} [8 pts] - https://phabricator.wikimedia.org/T104877#1430750 (mforns)
[15:06:53] <wikibugs>	 Analytics-Kanban: Enforce policy for each schema: Sanitize {tick} [8 pts] - https://phabricator.wikimedia.org/T104877#1539762 (mforns) a:mforns
[15:09:28] <ottomata>	 yeah, the stuff on 08-10 is real loss, that's when vk couldn't produce
[15:09:56] <joal>	 hm ... how much ?
[15:09:59] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Jenkins-mwext-sync appears in "Who contributes code" - https://phabricator.wikimedia.org/T105983#1539766 (Dicortazar) I've updated the list of bots to be ignored. I leave this task in progress till the data are updated.
[15:12:39] <ottomata>	 joal, check otto.webrequest_sequence_stats_hourly
[15:12:51] <ottomata>	 8	10	15	139367362	154102899	-9.562141332591024
[15:12:51] <ottomata>	 8	10	17	117914040	154997863	-23.92537695826168
[15:12:51] <ottomata>	 8	10	18	111166215	149039984	-25.411817677060412
[15:12:51] <ottomata>	 8	10	19	109457476	151163094	-27.5898150113281
[15:12:51] <ottomata>	 8	10	20	108514393	147682600	-26.521883417545467
[15:12:51] <ottomata>	 8	10	21	106275424	132149737	-19.579541804158108
[15:12:58] <ottomata>	 but that's what you saw on monday too.  about 20% loss
[15:13:12] <ottomata>	 i just reran the query for a couple of those hours on raw data, and got the same results
[15:13:22] <joal>	 ok
[15:13:43] <joal>	 what about 08-03, same ?
[15:13:44] <ottomata>	 i got diffefrent result than the 08-03 loss when i reran a couple of hours there, hence why I am rerunnign load for those hours
[15:13:53] <ottomata>	 08-03 10-23
[15:14:02] <ottomata>	 will see what seq stats says after load for those
[15:14:06] <joal>	 k
[15:14:15] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1539778 (Qgil) Storing the data and keeping it available is fine and good. My aim creating this task was simply to remove them from the r...
[15:14:24] <ottomata>	 i only looked at hours 10 and 11 on 08-03
[15:14:27] <ottomata>	 and saw 0 loss now
[15:14:37] <ottomata>	 which maybe means we shoudl rerun those refine and other jobs too
[15:14:38] <ottomata>	 for those hours
[15:14:39] <ottomata>	 will see.
[15:14:56] <joal>	 ottomata: you let me know, I can do it :)
[15:19:34] <wikibugs>	 Analytics-Backlog: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1539790 (mforns) @Tgr  You probably are looking at **m4-master.eqiad.wmnet** and have seen that some tables have been purged. This was an error from our side couple weeks ago, some data was indeed deleted e...
[15:19:51] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Clean up mobile-reportcard dashboards {frog} [8 pts] - https://phabricator.wikimedia.org/T104379#1539791 (kevinator) Open>Resolved
[15:20:08] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: {tick} Schema Audit - https://phabricator.wikimedia.org/T102224#1539796 (kevinator)
[15:20:10] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: Reach out to half the schema owners [8 pts] {tick} - https://phabricator.wikimedia.org/T102517#1539795 (kevinator) Open>Resolved
[15:20:19] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Solve aggregator bug {wren} [5 pts] - https://phabricator.wikimedia.org/T107764#1539798 (kevinator) Open>Resolved
[15:20:45] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Date formatting bug on Vital Signs {crow} [1 pts] - https://phabricator.wikimedia.org/T108337#1539806 (kevinator) Open>Resolved
[15:21:03] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Make reportupdater support weekly granularity {tick} [8 pts] - https://phabricator.wikimedia.org/T108593#1539808 (kevinator) Open>Resolved
[15:21:33] <wikibugs>	 Analytics-Kanban: Remove webrequest bits data from hadoop flows (camus, oozie) [3pts] {hawk} - https://phabricator.wikimedia.org/T108833#1539811 (kevinator) Open>Resolved
[15:23:25] <wikibugs>	 Analytics-Kanban, Discovery-Maps-Sprint, Patch-For-Review: Stream maps cluster requests into HDFS [3pts] {hawk} - https://phabricator.wikimedia.org/T108717#1539829 (kevinator) Open>Resolved
[15:29:21] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1539853 (Dicortazar) I see the point. However metrics are provided for the full list of repositories. So, if there are open tickets from...
[15:33:39] <wikibugs>	 Analytics-Tech-community-metrics, ECT-August-2015: Remove deprecated repositories from korma.wmflabs.org code review metrics - https://phabricator.wikimedia.org/T101777#1539856 (Qgil) I was not able to access those tickets to do anything about them, so yes, let's just remove the repos. If someone would ha...
[15:35:25] <wikibugs>	 Analytics-Kanban, Patch-For-Review, WMF-deploy-2015-08-11_(1.26wmf18): Event Logging sends mysql consumer stats to statsd [8 pts] {oryx} - https://phabricator.wikimedia.org/T105935#1539858 (ggellerman)
[15:45:06] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Correct pageview title normalization bug [analytics/refinery/source] - https://gerrit.wikimedia.org/r/231545 (owner: Joal)
[15:45:14] <joal>	 thx milimetric
[15:45:45] <wikibugs>	 Analytics-Kanban, Research-and-Data: Bug in pageview title extraction: change spaces to underscores after percent_decode (not only plus signs) - https://phabricator.wikimedia.org/T108866#1539884 (Milimetric) Oops, merged this and accidentally missed the mis-link to the Gerrit patch:  https://gerrit.wikimed...
[15:46:50] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Permit custom db param in report config [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/231438 (https://phabricator.wikimedia.org/T107502) (owner: Mforns)
[15:47:03] <mforns>	 milimetric, :]
[15:51:34] <mforns>	 milimetric, you want to talk about the flow?
[15:51:48] <milimetric>	 yes
[15:52:01] <mforns>	 ok
[15:52:02] <milimetric>	 let's go to a-batcave-1?
[15:52:12] <madhuvishy>	 milimetric: I'd like to join in when you talk to andrew about cassandra puppet stuff :)
[15:52:20] <madhuvishy>	 i'll be around, working from home
[15:52:20] <milimetric>	 madhuvishy: yeah!
[15:52:21] <mforns>	 milimetric, ok
[15:52:22] <milimetric>	 k
[15:52:36] <milimetric>	 madhuvishy: meanwhile i'll add you to my patch so your eyes can bleed
[15:53:30] <joal>	 milimetric: could you add me as well, just for reading ?
[15:54:24] <milimetric>	 joal: sorry i accidentally added you to a merged patch as well, disregard that
[15:54:33] <joal>	 k, np milimetric
[15:54:36] <milimetric>	 also, marcel's on the puppet one too now, it's officially a party! :)
[15:54:43] <mforns>	 ;]
[15:54:50] <joal>	 Awesome ! a puppet party :)
[15:56:10] <wikibugs>	 Analytics, Analytics-Backlog, Research-and-Data, Research consulting: Too few page views for June/July 2015 - https://phabricator.wikimedia.org/T106034#1539917 (ezachte) {F1494815}  Some time ago I captured all requests for article London Bridge on English Wikipedia in June.  Alas I did not capture...
[15:56:51] <chasemp>	 I read that as puppy at first
[15:56:56] <chasemp>	 my disappointment followed shortly
[15:58:40] <wikibugs>	 Analytics, Analytics-Backlog, Research-and-Data, Research consulting: Too few page views for June/July 2015 - https://phabricator.wikimedia.org/T106034#1539920 (ezachte) {F1494850}  In both charts 30 June is an exceptional peak in traffic (Bridge was opened on June 30, many years ago)  Extra traffic...
[16:08:51] <grrrit-wm>	 (CR) Mforns: Timebox reports and move to reportupdater [analytics/limn-flow-data] - https://gerrit.wikimedia.org/r/231446 (https://phabricator.wikimedia.org/T107502) (owner: Mforns)
[16:08:57] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Timebox reports and move to reportupdater [analytics/limn-flow-data] - https://gerrit.wikimedia.org/r/231446 (https://phabricator.wikimedia.org/T107502) (owner: Mforns)
[16:12:09] <milimetric>	 chasemp: here you go, consolation prize: https://www.youtube.com/watch?v=LEehXbK9ws8
[16:12:13] <milimetric>	 sfw
[16:16:07] <chasemp>	 :) not unappreciated on a friday
[16:18:31] <madhuvishy>	 these puppies are sooo adorable
[16:59:35] <wikibugs>	 Analytics, MobileFrontend, Reading-Web: Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1540194 (bmansurov) @Jhernandez, yes I think so. This task was linked in the email if I'm not mistaken.
[17:06:28] <wikibugs>	 Analytics, MobileFrontend, Reading-Web: Make MobileWebUIClickTracking schema usable (too big) - https://phabricator.wikimedia.org/T108723#1540223 (JKatzWMF) no.  separate concerns.  this was instigated by my running the 'beta header' analysis.
[17:43:24] <madhuvishy>	 mforns: something's come up, and i won't be able to join the meeting today at 11, you carry on, i will ask you for an update on monday :) sorry!
[17:43:41] <mforns>	 madhuvishy, ok, don't worry
[17:43:57] <mforns>	 I think I'll write an email to internal about it
[17:44:15] <mforns>	 or at least to you and Ke-vin
[17:46:17] <madhuvishy>	 great thanks mforns!
[17:46:29] <mforns>	 np :]
[17:47:53] <wikibugs>	 Analytics-Backlog, Reading-Admin, Research-and-Data, Research consulting: Request for data: sites traffic by topics/ subject areas and geographies - https://phabricator.wikimedia.org/T107613#1540479 (SVentura) Open>Resolved @qgil  We have a short list of topics we can work with for the pilot pr...
[19:01:21] <milimetric>	 madhuvishy: when are you free to talk about this patch?  I know it's lunch for you now
[19:04:03] <milimetric>	 mforns: I'm not seeing alerts... hm ... :)
[19:04:08] <wikibugs>	 Analytics: Page view statistics looks not updated since 2015-08-13 at 18:00:00 - https://phabricator.wikimedia.org/T109070#1540799 (xcombelle) thanks
[19:04:18] <milimetric>	 let's see what crazy graphite says
[19:04:26] <mforns>	 milimetric, it's still just 10 invalid events per second
[19:04:38] <mforns>	 I'm going to raise it to 20 in short
[19:04:55] <mforns>	 hehehe
[19:05:19] <milimetric>	 it's showing up in the graph, the valid is about 10 behind raw
[19:05:41] <milimetric>	 and in some places, valid is above raw!! :P
[19:06:08] <mforns>	 milimetric, hehehe crazyphite
[19:07:55] <mforns>	 milimetric, now 20/sec
[19:12:49] <milimetric>	 mforns: isn't the threshold 30%?
[19:13:01] <milimetric>	 we'd need something like 50-60 per second to hit that
[19:13:05] <mforns>	 milimetric, warning: 20, critical: 30
[19:13:12] <milimetric>	 oh ok
[19:13:25] <milimetric>	 still, for 20, we'd need 34
[19:13:27] <mforns>	 but it is not a percentage I think
[19:13:35] <mforns>	 do you think it is?
[19:13:43] <milimetric>	 oh, no idea
[19:13:47] <mforns>	 I thought it was an absolute value
[19:13:52] <milimetric>	 I thought that's what the alerts say
[19:13:59] <mforns>	 oh!
[19:14:03] <mforns>	 ok
[19:14:11] <milimetric>	 wait, lemme look at one
[19:15:05] <mforns>	 CRITICAL: 20.00% of data above the critical threshold [30.0]
[19:15:16] <milimetric>	 that's very confusing, what's that mean?
[19:15:18] <mforns>	 don't know what to think
[19:15:20] <milimetric>	 20% of what data
[19:15:51] <mforns>	 maybe that percentage, 3 data points of 15
[19:16:25] <mforns>	 now it should be 25% of data above [20]
[19:17:35] <mforns>	 mmm, it doesn't seem to be triggering :/
[19:18:22] <mforns>	 OK, 30/sec now
[19:23:47] <milimetric>	 yeah, it should definitely trigger, did it get deployed properly?
[19:23:53] <milimetric>	 like does something need to be restarted?
[19:26:22] <mforns>	 milimetric, that's what I asked ottomata, but if the alerts stopped, I guess it took effect, no?
[19:27:31] <ottomata>	 i can double check if it got deployed
[19:27:41] <ottomata>	 what's the alert called?
[19:28:44] <milimetric>	 not sure, how do you tell from this: https://gerrit.wikimedia.org/r/#/c/230825/3/modules/eventlogging/manifests/monitoring/graphite.pp
[19:28:56] <milimetric>	 oh, eventlogging_difference_raw_validated
[19:29:07] <ottomata>	 got it
[19:29:08] <ottomata>	 check_graphite_threshold!http://graphite.wikimedia.org!10!movingAverage(absolute(diffSeries(eventlogging.overall.raw.rate,eventlogging.overall.valid.rate)),10)!20!30!15min!25!--over
[19:29:10] <ottomata>	 is the command it runs
[19:30:12] <icinga-wm>	 PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 33.33% of data above the critical threshold [30.0]
[19:30:18] <milimetric>	 yay!
[19:30:18] <milimetric>	 :)
[19:30:23] <milimetric>	 what'd you do?
[19:30:26] <ottomata>	 i did nothing!
[19:30:29] <milimetric>	 liar
[19:30:41] <milimetric>	 mforns: you can stop the slaughter now :)
[19:31:18] <milimetric>	 maybe 20/s wasn't enough to hit the warning
[19:31:24] <mforns>	 it worked!?
[19:31:52] <ottomata>	 hey, moving locations, back shortly.
[19:31:56] <mforns>	 it did not catch the warning level though...
[19:31:59] <mforns>	 ok
[19:32:14] <mforns>	 stopped
[19:32:46] <milimetric>	 yeah, i think it worked, there it is in the chat ^
[19:33:06] <mforns>	 :]
[19:33:39] <mforns>	 let's see how much time it takes to recover
[19:33:57] <milimetric>	 if you want i can watch for that and record it on the bug
[19:34:02] <milimetric>	 you should go have a nice friday man
[19:34:03] <milimetric>	 it's late
[19:36:52] <mforns>	 no, np, I started late today also
[19:37:05] <mforns>	 will stay a bit
[19:38:10] <mforns>	 thanks :]
[19:46:30] <AndyRussG>	 Hi! Mmmm wondering if anyone has a quick-n-dirty MW client-side way of creating a unique ID? anything typical or standard out there for the MW environment?
[19:47:31] <icinga-wm>	 RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK Less than 25.00% above the threshold [20.0]
[19:47:48] <mforns>	 :]
[20:09:59] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: EventLogging Icinga Alerts should look at a longer period of time to prevent false positives {stag} [5 pts] - https://phabricator.wikimedia.org/T108339#1541018 (mforns) The false alerts have ceased and the true alert test was successful, see t...
[20:16:05] <mforns>	 hey team, going to sign off
[20:16:15] <mforns>	 nice weekend everyone!
[20:24:30] <wikibugs>	 Analytics-Wikimetrics, Software-Licensing: Add a license file to wikimetrics - https://phabricator.wikimedia.org/T60753#1541101 (Ricordisamoa)
[20:25:09] <wikibugs>	 Analytics-Cluster, Software-Licensing: Change license to Apache license, Version 2.0 - https://phabricator.wikimedia.org/T65084#1541107 (Ricordisamoa)
[20:34:08] <milimetric>	 AndyRussG: yes, there's a function in mw.user.  It's trickier than you'd expect because there's a crypto library that's poorly implemented in some numbers so you can get a lot of repetitions.  Here's the code:
[20:34:09] <milimetric>	 https://github.com/wikimedia/mediawiki-extensions-VisualEditor/blob/master/modules/ve-mw/init/ve.init.mw.trackSubscriber.js#L18
[20:35:23] <AndyRussG>	 milimetric: fantasmic, thanks! Yeah I was peeking at this SO page, indeed it looks like there are complications: http://stackoverflow.com/questions/105034/create-guid-uuid-in-javascript
[21:05:12] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Add hourly aggregate sequence stats creation to webrequest load job - https://phabricator.wikimedia.org/T109136#1541283 (Ottomata) NEW a:Ottomata
[21:26:50] <ottomata>	 hey madhuvishy, are you working on last access uniques in hive?  or do you know who is?
[21:40:34] <Ironholds>	 ottomata, ping?
[21:41:37] <ottomata>	 heya
[21:42:27] <DragonFire1024>	 	Hello anyone might answer some questions? Why these stats haven't updated? http://stats.wikimedia.org/wikinews/EN/Sitemap.htm I was told in #wikimedia-tech to ping milimetric
[21:43:42] <ottomata>	 Ironholds:  hiii
[21:44:50] <Ironholds>	 ottomata, so, the search UDF I wrote; I can't load it.
[21:49:32] <ottomata>	 search udf?
[21:49:37] <ottomata>	 milimetric: , you still around?
[21:51:54] <Ironholds>	 ottomata, IsSearchRequestUDF
[21:52:03] <Ironholds>	 it was several months ago so I assume you've deployed to the cluster since then ;p
[21:52:07] <ottomata>	 whats the problem?
[21:52:49] <ottomata>	 jar -tvf refinery-core.jar | grep Search
[21:52:49] <ottomata>	   1845 Wed Aug 12 15:58:20 UTC 2015 org/wikimedia/analytics/refinery/core/SearchRequest.class
[21:53:14] <ottomata>	  ± jar -tvf refinery-hive.jar | grep Search
[21:53:15] <ottomata>	    806 Wed Aug 12 16:05:16 UTC 2015 org/wikimedia/analytics/refinery/hive/IsSearchRequestUDF.class
[21:53:15] <ottomata>	    832 Wed Aug 12 16:05:16 UTC 2015 org/wikimedia/analytics/refinery/hive/SearchClassifierUDF.class
[21:53:15] <ottomata>	   1845 Wed Aug 12 16:05:16 UTC 2015 org/wikimedia/analytics/refinery/core/SearchRequest.class
[21:55:05] <wikibugs>	 Analytics-Backlog: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#1541422 (Tgr) Ugh, I am sorry about this. A few days ago I made [[https://meta.wikimedia.org/w/index.php?title=Schema_talk:ImageMetricsCorsSupport&diff=prev&oldid=13095868|this edit]], and soon afterwards m...
[22:02:53] <milimetric>	 DragonFire1024: I'm not sure, if it's abnormal you could file a ticket in phabricator and tag "Analytics-Backlog" and we'll prioritize it and take a look
[22:03:09] <milimetric>	 ottomata: I'm taking a cake out of the oven in a bit, but yeah, around, why
[22:03:47] <DragonFire1024>	 How do I do that?
[22:03:50] <DragonFire1024>	 milimetric	
[22:05:17] <milimetric>	 DragonFire1024: https://phabricator.wikimedia.org/maniphest/task/create/
[22:05:30] <ottomata>	 i think we are having more camus lagging problems and I wanted a buddy, but no worries
[22:05:31] <milimetric>	 and you'll have to make an account there if you don't have one, but we promise not to change issue trackers any more for a long time
[22:05:40] <milimetric>	 ottomata: I'll brb and we can hang?
[22:05:42] <ottomata>	 j
[22:05:43] <ottomata>	 ja
[22:06:05] <Ironholds>	 ottomata, CREATE TEMPORARY FUNCTION is_search AS 'org.wikimedia.analytics.refinery.hive.IsSearchRequestUDF';
[22:06:14] <Ironholds>	 FAILED: Class org.wikimedia.analytics.refinery.hive.IsSearchRequestUDF not found
[22:07:01] <ottomata>	 did you ADD JAR
[22:07:02] <ottomata>	 ?
[22:07:27] <ottomata>	 (not sure if you have to, but that's what i'd do)
[22:07:32] <milimetric>	 k, in batcave
[22:08:37] <wikibugs>	 Analytics-Backlog: Stats for en.wikinews.org not working - https://phabricator.wikimedia.org/T109146#1541481 (DragonFire1024) NEW
[22:08:52] <DragonFire1024>	 done milimetric
[22:08:59] <DragonFire1024>	 oh as you can see :)
[22:10:05] <DragonFire1024>	 oops wrong tab
[22:12:14] <Ironholds>	 ottomata, I didn't; what JAR should I be adding?
[22:17:47] <ottomata>	 Ironholds:
[22:18:03] <ottomata>	  /srv/deployment/analytics/refinery/artifacts/refinery-hive.jar
[22:18:08] <Ironholds>	 ta
[22:41:56] <milimetric>	 thanks DragonFire1024, we'll look at it next week
[22:42:06] <DragonFire1024>	 ok
[22:54:28] <ottomata>	 crap milimetric not fixed, camus is still exceptioning
[22:56:22] <ottomata>	 milimetric: all exceptions are from analytics1012
[22:56:27] <ottomata>	 i'm going to bounce it.
[23:15:36] <ottomata>	 looking better...i hope