[00:00:59] neilpquinn: actually,no, all data will come from labs so we will have just public data, in this case not all deleted pages are "private" in nature [00:01:51] neilpquinn: data is slightly redacted when it moves to labs but pretty much all info we use is available (let us know if think otherwise) [00:02:11] all info we use to build metrics that is [00:08:21] 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1003 replacement - https://phabricator.wikimedia.org/T159839#3104654 (10RobH) Raid5 is being used in production on these boxes? That seems, non-ideal.... I'll start pulling together a quotes for this shortly. While this sys... [00:09:11] nuria: it depends on what data exactly is redacted. do you know where I can find I description? https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database#Databases doesn't fully say. [00:11:31] nuria: also, in that case how will I access the Data Lake from PAWS internal? They'll be on different clusters. [01:56:58] 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#3104881 (10MusikAnimal) So for March 14 we had this: https://wikimedia.org/api/rest_v1/metrics/pageviews/top/ru.wikipedia/all-access/2017/03/14 On the Russian Wikipedia, roug... [06:45:08] 06Analytics-Kanban: Enable Pageviews API for test.wikipedia.org - https://phabricator.wikimedia.org/T160484#3100464 (10Nemo_bis) > Currently filtered out of pageviews by definition As I said elsewhere including https://meta.wikimedia.org/wiki/Research_talk:Active_editor , the definition should be about how we c... [08:12:10] elukey, yt? :] [08:12:24] o/ [08:12:29] helloooo [08:12:33] holaaa [08:12:36] :] [08:12:46] problems with aqs beta? [08:12:52] do you know if francisco is done with beta aqs testing? [08:13:07] it's just his deploy removed my keyspace [08:13:14] so I have to deploy my code back [08:13:44] or I could actually, rebase on top of his change and deploy both [08:14:07] mmmm do you mean https://gerrit.wikimedia.org/r/#/c/342882/ ? [08:14:20] how come your keyspace was wiped? [08:14:23] looking [08:14:50] yesterday I had to wipe aqs beta though, do you mean the keyspace that you created afterwards? [08:14:57] dunno [08:15:03] but it's not there [08:15:19] mmmmm [08:15:31] no we created it the day before last thing [08:15:57] ok, so I have to deploy anyway [08:16:38] mforns: "we created it the day before last thing" - was it before yesterday's standup ? [08:16:47] (sorry I am a bit slow this morning , be patient :) [08:16:50] yes [08:17:18] xD no worries, IRC's like this [08:17:28] ok so I re-created the cassandra cluster AFTER standup [08:17:29] :) [08:17:40] wiping the cassandra sstables [08:17:41] ok ok [08:17:46] I will rebase on top of all that is deployed there [08:17:53] super [08:17:55] and deploy [08:18:11] test data is already in deployment-aqs01 [08:18:20] \o/ [08:18:22] all working fine? [08:19:24] let's see... for now it's only a csv :] [08:19:54] mforns: I am going afk for ~30/40 mins (cat to the vet, hopefully this week will be the last one). Ping me on IRC and I'll read back asap if you need me :) [08:20:19] no prob elukey take your time :] see you in a while [09:06:04] (03PS3) 10Mforns: Add new endpoint for legacy pageviews [analytics/aqs] - 10https://gerrit.wikimedia.org/r/342417 (https://phabricator.wikimedia.org/T156391) [09:16:37] elukey, I'm not sure if I'm doing something wrong, but the code I checked-out in tin, does not transfer to the nodes with scap deploy [09:17:05] I run scap deploy and all log messages are fine, but no new code in beta aqs nodes [09:28:46] mforns: back :) [09:29:00] hi elukey :] is cat ok? [09:29:23] recovering, daily injections because of an infection (pancreas it seems) [09:29:30] looooong loooong recovery [09:29:31] :) [09:29:34] mmm :[ [09:31:12] back to the change.. https://gerrit.wikimedia.org/r/342417 is the one that you are working on right? [09:31:33] elukey, yes [09:31:47] how did you check it out on tin? [09:31:52] I rebased it on top of what has been merged, which is the same that was deployed to aqs beta [09:31:59] and checked it out normallu [09:32:03] *normally [09:32:12] do you mean manually on tin? [09:32:16] yes [09:32:19] mmmmmm [09:32:30] no, not manually, [09:32:37] I mean, I rebased it on gerrit [09:32:59] ahhh okok! [09:33:03] and then sshd into tin, and git fetch + checkout [09:33:15] and nothing came down right? [09:33:18] then scap deploy [09:33:31] but no code was pushed to the nodes [09:34:01] same that happened yesterday with Fran [09:34:40] so aqs/src is a submodule of aqs-deploy, so we need to follow https://wikitech.wikimedia.org/wiki/Services/Deployment#Regular_Deployment to update the submodule sha [09:34:42] yesterday we fixed that by manually deploying to depl-aqs01, but today, with the 3 nodes, I didn't want to screw it [09:34:58] reading [09:35:10] we pull/scap-deploy aqs-deploy, not aqs/src. [09:35:40] it is a bit convoluted in the beginning but then it makes sense :D [09:36:25] basically the aqs-deploy repo holds the nodejs dependencies (frozen, no need to use npm-install/update) AND a git submodule, that is aqs/src [09:36:29] I see... [09:36:46] Hi lads - Let me know if there's anything I can help with [09:36:57] so when you update aqs/src, you merge and then you need to create a new code review for aqs-deploy to update the submodule sha [09:37:11] elukey, would it be too much to ask for you to pair with me in da cave for 5 mins? [09:37:25] hi joal :] [09:37:38] mforns: this is what Nuria did yesterday to fix Francisco's issue - https://gerrit.wikimedia.org/r/#/c/342882/ [09:37:52] mforns: can I disturb you a minute? [09:37:58] joal, sure [09:38:06] mforns: I can but Joseph is usually the master in aqs-deploy, if you want to pair with him it is way better than me :) [09:38:08] elukey, I see, that's the one you pasted before [09:38:18] ok, will ask joal [09:38:29] but I am here if you guys need my help [09:38:33] joal, here or cave? [09:38:37] thx a lot elukey :] [09:38:58] I don't mastering anything, but I can try to help :) [09:39:02] Let's batcave :) [09:39:06] k [09:48:58] elukey yesterday my deployment to beta failed like this: [09:49:00] https://www.irccloud.com/pastebin/7878qH2j/ [09:51:28] fdans: ah yes let me give you a handy command [09:51:46] SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-aqs01.deployment-prep.eqiad.wmflabs [09:52:03] you run it on tin, accept the ssh key and retry [09:52:10] scap deploy should work fine [09:52:26] ah also SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-aqs02.deployment-prep.eqiad.wmflabs [09:52:29] and [09:52:30] SSH_AUTH_SOCK=/run/keyholder/proxy.sock ssh mwdeploy@deployment-aqs03.deployment-prep.eqiad.wmflabs [09:52:51] let me try on tin to deploy [09:53:08] I'm getting Permission denied (publickey). [09:53:57] ok should be good now, I am deploying btw [09:54:14] done [09:54:53] fdans: to have better info for deployment errors, use scap deploy-log [10:33:10] neilpquinn: yes, we are just now scheduling regular jobs to keep it updated, and we'll have an announcement of how the partitioning will work and all that. [10:33:14] 10Analytics, 10DBA, 06Labs: Discuss labsdb visibility of rev_text_id and ar_comment - https://phabricator.wikimedia.org/T158166#3105483 (10ArielGlenn) After my initial ping, I dropped this ball. Picking it back up again. [10:36:34] (03CR) 10Milimetric: [V: 032 C: 032] Add beta feature graph for RCFilters [analytics/limn-ee-data] - 10https://gerrit.wikimedia.org/r/342946 (owner: 10Catrope) [10:45:18] 10Analytics, 13Patch-For-Review: Sort inconsistency in AQS timestamp behavior - https://phabricator.wikimedia.org/T160311#3105499 (10JAllemandou) Lasdt update after daily bug correction: There is inconsistency in monthly behavior: - Pattern is consistent in every endpoint except monthly ones: we include last... [10:51:52] !log deploying aqs to production [10:51:53] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:52:34] elukey: aqs tested in beta, ok to deploy in production? [10:53:17] fdans: sure [10:54:03] fdans: are marcel's changes going to be deployed separately? [10:54:11] elukey: yes [10:54:14] okok [10:54:35] fdans: you probably can't deploy to aqs though [10:55:10] how so elukey ? [10:55:28] because of permissions [10:55:28] 10Analytics, 13Patch-For-Review: Sort inconsistency in AQS timestamp behavior - https://phabricator.wikimedia.org/T160311#3105515 (10JAllemandou) Another comment: My check was for **END** boundaries --> Our pattern is to include them (due to alphabetical sorting of timestamps). For **START** boundaries, we act... [10:55:30] 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#3105516 (10Milimetric) The problem I'd be worried about is when traffic from a specific city makes sense, like there is local news about that city that isn't relevant to the r... [10:55:56]  [10:56:16] elukey, fdans: I'll run scap deploy [10:56:27] scap needs to access to the key-holder, a daemon that basically gives to scap the possibility to ssh to the hosts as the deploy user [10:56:35] joal: sure [10:56:45] Doing NOW ! [10:56:45] 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#3105519 (10Milimetric) p:05Normal>03High [11:00:18] (03CR) 10Joal: [V: 032 C: 032] Change keyspace name to project_v2 in fake data script [analytics/aqs/deploy] - 10https://gerrit.wikimedia.org/r/342876 (https://phabricator.wikimedia.org/T156312) (owner: 10Fdans) [11:08:25] 06Analytics-Kanban, 10Pageviews-API, 13Patch-For-Review: Monthly aggregate endpoint returns unexpected results and invalid timestamp - https://phabricator.wikimedia.org/T156312#3105573 (10fdans) a:05fdans>03JAllemandou [11:09:08] aqs metrics are good [11:09:09] :) [11:15:27] awesome elukey :) [11:19:01] a-team - going for a break, will be back later [11:42:30] * elukey lunch! [12:29:02] 10Analytics: Please update Tulu Language(tcy)in Wikipedia Statistics. - https://phabricator.wikimedia.org/T160630#3105788 (10Aklapper) Please describe which exact statistics you would like to see. stats.wikimedia.org is pretty dead and [[ https://phabricator.wikimedia.org/T152033 | will get replaced ]]. You ma... [12:42:19] 10Analytics: Please update Tulu Language(tcy)in Wikipedia Statistics. - https://phabricator.wikimedia.org/T160630#3105833 (10BHARATHESHAALASANDEMAJALU) https://stats.wikimedia.org/EN/TablesWikipediaKN.htm#wikipedians Recently active wikipedians, excl. bots, ordered by number of contributions such as contribut... [13:17:39] 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1003 replacement - https://phabricator.wikimedia.org/T159839#3105902 (10Ottomata) We want a RAID that gives us the most space with a little bit of redundancy. Is RAID 5 not the best choice? [14:09:15] hello joal, fdans, can I merge my patch? :] [14:09:52] mforns: having a quick look at last patch set :) [14:10:14] I mean, did you deploy already to prod? [14:10:23] mforns: also, thanks for the review! I responded and submitted new patch [14:10:42] ottomata, k, will look at it [14:11:03] nuria: doh; i'm just seeing your (now very old) ping [14:11:22] mforns oh sorry, yes, we have deployed, you're all good :) [14:11:34] fdans, ok :] thx [14:33:10] brb! [14:33:23] (03CR) 10Mforns: [V: 032 C: 032] "Self merge for testing purposes." [analytics/aqs] - 10https://gerrit.wikimedia.org/r/342417 (https://phabricator.wikimedia.org/T156391) (owner: 10Mforns) [14:34:07] oh nm, not brb [14:34:07] b [14:42:17] joal, yt? [14:42:38] neilpquinn: hola, no data is on the same place, getting docs and code merged this week, let's touch base on this with a meeting. [14:44:16] neilpquinn: do ping us on irc when you are back cc joal [14:44:25] 10Analytics, 10ChangeProp, 10EventBus, 06Revision-Scoring-As-A-Service, and 3 others: Create generalized "precache" endpoint for ORES - https://phabricator.wikimedia.org/T148714#2730864 (10Halfak) [14:48:43] (03PS1) 10Mforns: Update aqs to 3f52830 [analytics/aqs/deploy] - 10https://gerrit.wikimedia.org/r/343055 [15:01:16] joal: standddupppp [15:08:53] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3106338 (10Nuria) ping @jcrespo. Could you give us an ETA on when we could start doing these changes? [15:11:13] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3106339 (10Marostegui) >>! In T160454#3102530, @Nuria wrote: > Excellent, let us know what you think is a good time on your end to do this and we will take an outage accordingly. For us, the s... [15:14:15] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3106348 (10Nuria) >Could you give us the exact list of tables that would need to be renamed? Yes, will compile today and post here. [15:15:06] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3106349 (10Marostegui) >>! In T160454#3106348, @Nuria wrote: >>Could you give us the exact list of tables that would need to be renamed? > Yes, will compile today and post here. Great - thanks! [15:19:54] 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1003 replacement - https://phabricator.wikimedia.org/T159839#3106376 (10RobH) Raid5 has very slow write (same as raid6), due to the calculations on the parity striping (redundancy) across the disks. The fastest raid for writes... [15:24:20] ottomata: what if we request raid10 for --^ [15:24:35] ashgrigas: are you going to export stuff to zeplin? I don't have mac but I can also get others on the team to do it [15:25:46] elukey: we could, but that would give us less space, no? [15:26:35] milimetric: yes I can do that [15:26:41] If you think it's helpful [15:28:51] ottomata: sure but we can have bigger disks :) [15:29:19] elukey: but what do we gain? a little more redundancy i guess? [15:29:21] maybe a little more io? [15:30:07] ottomata: and also probably less headaches for ops people reviewing configs :) [15:30:10] ashgrigas: yeah, it'd be good if you could do it and take a quick look at the output to make sense it's sane [15:30:27] why is raid 5 more headache? [15:30:28] Sure [15:31:30] ottomata: this is only a feeling, I haven't found any partman config with raid 5/6 up to now and I thought it wasn't the reccomended choice.. but we can try it, I am not against it.. probably less io power and a bit more space available [15:32:35] 10Analytics: Add desktop only tab for browser reports on analytics.wikimedia.org - https://phabricator.wikimedia.org/T160642#3106427 (10Nuria) [15:32:41] elukey: raid5-gpt-lvm.cfg [15:32:42] ? [15:33:23] also, these hosts are 'one offs', so i have no objection to bypassing partman [15:33:45] there isn't that much benefit to having a complete partman if you only use it 'once' [15:33:52] ottomata: is it applied to any host? [15:34:07] anyhow, I would be really in favor of not having special hosts without partman configs [15:34:10] if possible [15:34:35] elukey: stat1003 [15:34:38] again, I am not against it (totally ignorant about best choices for RAID), just throwing some ideas :) [15:35:31] ahhh nice sorry I didn't know it, I thought you were proposing raid5 and Rob wasn't happy [15:35:35] okok makes sense [15:35:38] my mistake [15:35:39] :) [15:37:15] (03PS1) 10Milimetric: Add queries for desktop-only tab [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343073 (https://phabricator.wikimedia.org/T160642) [15:37:44] (03CR) 10Milimetric: [V: 032 C: 032] Add queries for desktop-only tab [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343073 (https://phabricator.wikimedia.org/T160642) (owner: 10Milimetric) [15:37:57] 10Analytics, 13Patch-For-Review: Add desktop only tab for browser reports on analytics.wikimedia.org - https://phabricator.wikimedia.org/T160642#3106454 (10Nuria) p:05Triage>03Normal [15:38:10] 06Analytics-Kanban, 13Patch-For-Review: Add desktop only tab for browser reports on analytics.wikimedia.org - https://phabricator.wikimedia.org/T160642#3106427 (10Nuria) [15:38:22] 06Analytics-Kanban, 13Patch-For-Review: Add desktop only tab for browser reports on analytics.wikimedia.org - https://phabricator.wikimedia.org/T160642#3106427 (10Nuria) a:03Milimetric [15:38:32] elukey: You're gonna hate me -- Do you think we could spend some time investigating another technology for fast SQL queries? [15:39:03] joal --verbose [15:39:15] * elukey is scared [15:39:31] elukey: I'm sure you recall we heard of prestodb.io at Apache conf [15:39:41] ahhh yes [15:40:08] I wonder about perf differences against druid/clickhouse [15:40:09] 10Analytics, 10Analytics-Cluster: Kafkatee generated files in /a/log/webrequest not updating since 2014-09-18 - https://phabricator.wikimedia.org/T73290#3106469 (10Ottomata) [15:40:11] 10Analytics, 10Analytics-General-or-Unknown: kafkatee not consuming for some partitions - https://phabricator.wikimedia.org/T73056#3106467 (10Ottomata) 05Open>03Resolved a:03Ottomata [15:40:39] 10Analytics: Please update Tulu Language(tcy)in Wikipedia Statistics. - https://phabricator.wikimedia.org/T160630#3106471 (10Nuria) p:05Triage>03Normal [15:41:27] 10Analytics, 10Analytics-EventLogging, 10DBA, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#2497306 (10Nuria) Moving to radar as i think is taken care of by dbas [15:41:42] joal: we can definitely work on it [15:41:54] (03PS1) 10Milimetric: Configure desktop-only queries to run from 2017-01-01 [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343076 [15:42:37] (03CR) 10Milimetric: [V: 032 C: 032] Configure desktop-only queries to run from 2017-01-01 [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343076 (owner: 10Milimetric) [15:42:54] 10Analytics, 10Analytics-EventLogging: Support third-party use by eliminating hard dependency on Varnish - https://phabricator.wikimedia.org/T45601#3106491 (10Nuria) Declining, For usage on vagarnt there is teh eventlogging dev server, beyond that we have not had any requests for EL support outside WMF. [15:43:03] 10Analytics, 10Analytics-EventLogging: Support third-party use by eliminating hard dependency on Varnish - https://phabricator.wikimedia.org/T45601#3106492 (10Nuria) 05Open>03declined [15:43:05] 10Analytics, 10Analytics-EventLogging, 10DBA, 10ImageMetrics: Drop EventLogging tables for ImageMetricsLoadingTime and ImageMetricsCorsSupport - https://phabricator.wikimedia.org/T141407#3106493 (10Marostegui) >>! In T141407#3106478, @Nuria wrote: > Moving to radar as i think is taken care of by dbas Not... [15:43:32] elukey: my concern is: where? [15:43:43] elukey: meaning, which machines? [15:44:58] joal: we could use druid machines without clickhouse [15:45:03] it shouldn't be a big deal [15:45:10] super good would be Labs [15:45:44] elukey: given that it's for perf test, labs is not really an option (for setup yes, not for perf) [15:45:50] yep yep [15:46:01] we can work on setting it up in a bare minimum way [15:46:05] and then we test it on druid [15:46:19] elukey: works for me :) [15:46:51] elukey: Could we stop clickhouse without disabling it completely (like that, it allows to still be able to test if needed) [15:49:25] joal: oh yes we just need to stop it and then re-start it if needed [15:49:51] elukey: That'd be great :) [15:50:30] joal: wanna give me some quick spark ideas? [15:50:39] https://hangouts.google.com/hangouts/_/wikimedia.org/a-batcave-2 [15:53:52] ottomata: You left me !!! [15:56:53] 10Analytics: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3106537 (10fgiunchedi) [15:57:06] 10Analytics: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3106550 (10fgiunchedi) [16:15:13] mforns: I made trouble, I got impatient and did "sudo -u hdfs git pull" on the repo on stat1002 [16:15:26] and now it says it can't find the files, I think because they're owned by hdfs:hdfs instead of hdfs:root? [16:15:37] that seems like a bug, but have you seen that before? [16:15:52] milimetric, reading [16:16:04] apologies [16:16:26] milimetric, the reportupdater repo? [16:16:46] yes, on stat1002 [16:16:53] looking [16:17:56] milimetric, who says that can't find the files? RU? [16:18:11] yeah, I tried running it and it says it can't find the new files [16:18:34] maybe it's a path problem? what command did you use? [16:18:39] ERROR - Report "desktop_site_by_os_family_and_major" could not be executed because of error: [Errno 2] No such file or directory [16:18:51] sudo -u hdfs python /a/reportupdater/reportupdater/update_reports.py -l info /a/reportupdater/jobs/reportupdater-queries/browser /a/reportupdater/output/metrics/browser [16:19:03] 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#3106734 (10Nuria) This is on our rather for Q1/Q2 (July 2017/september 2017) we will not be able to tackle this problem any sooner. [16:20:17] milimetric, I see what you mean [16:21:22] 10Analytics, 13Patch-For-Review: Sort inconsistency in AQS timestamp behavior - https://phabricator.wikimedia.org/T160311#3106742 (10Nuria) >Pattern is consistent in every endpoint except monthly ones: we include last date boundary, for instance, specifying a non-full day in daily request still provides you wi... [16:22:19] milimetric, maybe that's the reason, but I'm not sure [16:22:47] anyway, that is not a problem at all no? [16:23:00] i'm wondering if it's in a bad state now [16:23:03] yeah, probably no problem [16:23:57] milimetric, you only git pulled reportupdater-queries right? [16:24:01] yea [16:27:45] the jobs dir is empty now [16:28:55] when puppet runs it will fix it I guess [16:29:57] 06Analytics-Kanban, 13Patch-For-Review: Create AQS endpoint to serve legacy pageviews - https://phabricator.wikimedia.org/T156391#3106778 (10Nuria) [16:33:18] 06Analytics-Kanban, 15User-Elukey: Bump replication factor of system.auth table in cassandra when new nodes have finished bootstrap - https://phabricator.wikimedia.org/T157354#3106785 (10elukey) After a long investigation with Eric we didn't find any good root cause but only a lot of conjectures that are not s... [16:34:04] mforns: yeah, I deleted it just in case, relying on puppet [16:34:25] nuria: the trick to log node stdout is in https://wikitech.wikimedia.org/wiki/Analytics/AQS#Troubleshooting_Deploy [16:45:11] 10Analytics, 13Patch-For-Review: Sort inconsistency in AQS timestamp behavior - https://phabricator.wikimedia.org/T160311#3106829 (10JAllemandou) I don't mind if we decide to include or exclude end boundary but I'd like us to be consistent accross granularities. For instance, if we decide to exclude end bounda... [16:45:40] (03PS1) 10Milimetric: Run full backfill per team [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343090 [16:45:50] (03CR) 10Milimetric: [V: 032 C: 032] Run full backfill per team [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/343090 (owner: 10Milimetric) [16:46:49] 10Analytics, 10Analytics-EventLogging: Research Spike: Better support for Eventlogging data on hive - https://phabricator.wikimedia.org/T153328#3106833 (10Ottomata) Just brainbounced with @JAllemandou. We some have ideas, am going to write them here. First of all, this works! (Tested with Spark 2.1.) ``` v... [16:49:53] 06Analytics-Kanban, 15User-Elukey: Give me permissions in LDAP [Please don't use this anymore] - https://phabricator.wikimedia.org/T150790#3106842 (10elukey) [16:49:59] 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1003 replacement - https://phabricator.wikimedia.org/T159839#3106843 (10Ottomata) There aren’t any ‘services’ hosted on these nodes. The drives are only used for local data storage, so we aren’t really concerned with write io... [16:51:21] 10Analytics: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3106844 (10Ottomata) Yar, this is because many of the metrics are per-client. I'd like to know if clients start lagging, and there's not a real way to aggregate that. But, we really don't need to keep history of this data... [17:04:03] bbiab [17:06:22] 10Analytics-Dashiki, 06Analytics-Kanban: Show pageviews prior to 2015 in dashiki - https://phabricator.wikimedia.org/T143906#3106926 (10Nuria) [17:07:04] milimetric, I saw puppet put the files in there with the correct user group, did that fix the thing? [17:08:04] joal, yt? :] [17:08:19] hey mforns [17:08:55] 10Analytics-Dashiki, 06Analytics-Kanban: Show pageviews prior to 2015 in dashiki - https://phabricator.wikimedia.org/T143906#3106930 (10Nuria) a:03fdans [17:09:03] mforns: What can I do for you? [17:09:10] help! [17:09:21] with scap deploy aqs, please [17:09:26] sure mforns :) [17:09:31] batcave? [17:09:37] OMW [17:11:11] 10Analytics: Add ability to query new AQS endpoint to node pageview API client - https://phabricator.wikimedia.org/T160655#3106971 (10Nuria) [17:12:01] 10Analytics: Add ability to query new AQS endpoint to node pageview API client - https://phabricator.wikimedia.org/T160655#3106764 (10Nuria) [17:12:04] 06Analytics-Kanban: Move reportcard to dashiki and new datasources - https://phabricator.wikimedia.org/T130117#3106972 (10Nuria) [17:12:21] 06Analytics-Kanban: Add ability to query new AQS endpoint to node pageview API client - https://phabricator.wikimedia.org/T160655#3106764 (10Nuria) [17:12:22] milimetric can you make me the project owner in zepelin? [17:12:38] it says only the project owner can export to that project [17:12:54] (on a free plan) [17:15:29] 06Analytics-Kanban, 15User-Elukey: Give me permissions in LDAP [Please don't use this anymore] - https://phabricator.wikimedia.org/T150790#3106994 (10elukey) Changed the Task title and the Analytics docs, from now on please open a task like https://phabricator.wikimedia.org/T160662, this task is not valid anym... [17:17:04] (03CR) 10Mforns: [V: 032 C: 032] Update aqs to 3f52830 [analytics/aqs/deploy] - 10https://gerrit.wikimedia.org/r/343055 (owner: 10Mforns) [17:18:19] 10Analytics, 06Operations, 06WMF-Legal, 07Privacy: Honor DNT header for access logs & varnish logs - https://phabricator.wikimedia.org/T98831#1278628 (10Nuria) >If we're striving to honor DNT in its purest ideological form, people who have it set shouldn't be tracked at all, even for our internal tracking... [17:19:08] 10Analytics: Revamp Eventlogging so anyone can use it - https://phabricator.wikimedia.org/T143794#3107004 (10Nuria) Closing, we do not have any tenants for this work. [17:19:16] 10Analytics: Revamp Eventlogging so anyone can use it - https://phabricator.wikimedia.org/T143794#3107005 (10Nuria) 05Open>03declined [17:19:58] 06Analytics-Kanban: Count global unique devices per top domain (like *.wikipedia.org) - https://phabricator.wikimedia.org/T143928#3107008 (10Nuria) p:05Triage>03High [17:21:26] 10Analytics, 06Fundraising-Analysis, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Provide performant query access to banner show/hide numbers - https://phabricator.wikimedia.org/T90649#3107015 (10Nuria) ping @awight @AndyRussG I think our job on pivot and banners takes care of this item, le... [17:23:01] 10Analytics-Tech-community-metrics, 10Phabricator, 06Developer-Relations (Jan-Mar-2017): Decide on wanted metrics for Maniphest in kibana - https://phabricator.wikimedia.org/T28#3107040 (10Aklapper) Sharing which fields is currently available in the `maniphest` index on https://wikimedia.biterg.io: {F6630885} [17:26:04] 10Analytics, 13Patch-For-Review: Sort inconsistency in AQS timestamp behavior - https://phabricator.wikimedia.org/T160311#3107065 (10Nuria) >In that specific example this query should return 2 days, not three (since the last day is not finished). Agreed. [17:27:44] elukey: thanks for documenting [17:31:22] elukey: Heya, are you around? [17:31:39] elukey: We'd need some hammer, please [17:33:50] joal: I am here [17:34:02] elukey: Hi hammer-man [17:34:15] elukey: scap is broken on aqs-labs :( [17:34:48] elukey: bizarre error on aqs03, problem of manual changes on aqs01 (and we didn't touch aqs02 yet) [17:35:01] manual changes on aqs01? [17:35:19] I think so yes [17:35:50] like sourcing a different piece of code (a file is owned by root instead of deploy-service in a cahced rev) [17:35:52] elukey, scap deploy can not delete deploy cache revision because of permit issues [17:36:00] yea [17:36:57] has anybody messed with aqs01? [17:38:12] * joal shows both empty hands [17:38:18] elukey, I think that was prior to your yesterday's fix [17:38:30] * mforns 's face goes red [17:39:12] I didn't touch anything after yesterday's fix! [17:39:19] :) [17:39:25] ok so aqs01 should be fixed [17:39:31] :] thx [17:39:33] what was the other issue with aqs03? [17:40:11] elukey: even weirder fgor 03 --> It doesn't find the commit ? [17:40:32] mmm trying to deploy to see [17:41:15] That's weird, heh? [17:42:27] milimetric nevermind got a one month paid subscription to make it easier - link is here: https://zpl.io/1FrCdW [17:42:30] let me know your thoughts [17:42:35] elukey: looks like it works now [17:42:45] elukey: I want to know your spells ! [17:43:43] :] [17:44:45] no no wait [17:44:52] I am still working on it [17:44:59] elukey: waiting (and reading scap deploy-log) [17:46:32] (03CR) 10Joal: [V: 032 C: 032] "Merging now for next deploy." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/342205 (https://phabricator.wikimedia.org/T156312) (owner: 10Fdans) [17:47:10] done now :) [17:47:14] :] [17:47:17] elukey: saw that yes [17:47:25] elukey: I still want to know you spels :) [17:47:27] so I chowned the root files on aqs01 and removed the repo on aqs03 and re-created [17:47:37] k [17:47:38] what happened, so that I do not break anything again? [17:47:40] ran puppet and re-deployed [17:47:48] mforns: I am not sure :) [17:48:07] elukey: mforns last patch should be deployed now, correct? [17:48:24] I can see my keyspace :D [17:48:25] boy... the joy [17:48:35] elukey: Thanks a lot :) [17:48:42] thanks!! [17:48:52] :) [17:49:20] not sure why scap behaved in that way, but let's investigate if re-happens [17:49:24] k [17:51:21] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3107208 (10Ottomata) Hm, just looked myself, but I don't see any non EventLogging `log` databases on db1046 or db1047. At least, the 'eventlog' user can't see them. Not sure... [17:51:57] "seeing your keyspace" a special moment in the life of every a-teamer [17:55:37] 10Analytics-Tech-community-metrics: Add remaining KPIs to Overview once available in kibana - https://phabricator.wikimedia.org/T116572#3107233 (10Aklapper) KPIs we've listed in past quarterly slides are: * Median age of open changesets waiting for review * ~~Median age of open changesets waiting for review (Med... [17:58:28] going afk! byyyee o/ [18:01:41] Bye elukey [18:03:01] milimetric: , yt? [18:03:15] byee elukey! :) [18:07:50] nuria: I'm back now! Btw, I used the Data Lake (and the internal Jupyter) to do some really useful new user retention calculations last night :) [18:08:25] nuria: I'm hoping they're scalable enough to make a new user retention dashboard, but I'll see :) [18:08:43] neilpquinn: can talk in abit [18:08:49] Hi neilpquinn [18:09:10] Hey joal! How're you doing? [18:09:25] neilpquinn: I'm doing good ! Thanks :) [18:09:30] neilpquinn: How about you? [18:10:05] 10Analytics: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3107299 (10fgiunchedi) >>! In T160644#3106844, @Ottomata wrote: > Yar, this is because many of the metrics are per-client. I'd like to know if clients start lagging, and there's not a real way to aggregate that. > > But,... [18:11:07] joal: good! I've been working hard on https://meta.wikimedia.org/wiki/New_Editor_Experiences so I haven't had much time for analytics recently, so I've been missing that :) [18:11:43] Interesting neilpquinn :) [18:11:49] joal: I also keep thinking I need to schedule that Data Lake tour we talked about at all hands. I am really interested, I promise! :) [18:12:30] neilpquinn: Your timing is good, we're about to have some more productionized stuff by the end of month :) [18:13:20] joal: haha, should I wait until then? I was about to look at your calendar for next week. [18:13:59] neilpquinn: I think nuria sent an invite already ;) [18:15:36] joal: ah, yes, I know. But I assumed that would be talking at a high level about the project direction. You had offered to give a tour of the code itself which is what I'm thinking about :) [18:15:55] neilpquinn: Didn't get the thing :) [18:16:21] neilpquinn: let's do that - One meeting for overview, one for more details :) [18:16:59] (03PS6) 10Joal: Add snapshot to sqoop and namespace_map scripts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341586 (https://phabricator.wikimedia.org/T160152) [18:17:01] neilpquinn: since you are here... [18:17:11] do you use EventLogging MySQL databases? [18:18:33] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3107357 (10Ottomata) @DarTar @leila, @milimetric, @Tbayer: Q for yall. I need to figure out what we actually need to replace MySQL research slaves. I am unfamiliar with what... [18:19:21] (03PS7) 10Joal: Add snapshot to sqoop and namespace_map scripts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341586 (https://phabricator.wikimedia.org/T160152) [18:19:23] (03PS4) 10Joal: Update static wiki projects list [analytics/refinery] - 10https://gerrit.wikimedia.org/r/342030 (https://phabricator.wikimedia.org/T160153) [18:19:27] ottomata: in general, definitely yes, although because of priorities, not actually in the past couple of months. [18:19:53] k did you ever use db1047 to query? or just analytics-store [18:19:53] ? [18:20:27] db1047 is AKA s1-analytics-slave and s2-analytics-slave [18:20:32] oh, and analytics-slave [18:22:39] ottomata: I've almost exclusively used analytics-store. [18:23:13] ok thanks [18:23:17] I've used x1-analytics-slave a couple of times to do research on notifications, because that data isn't available on analytics-store. [18:24:07] oh [18:24:08] it isn't? [18:24:11] explain? [18:24:20] oh [18:24:22] that's a different host [18:24:22] ok [18:24:23] ottomata: the echo table aren't on analytics store. [18:24:34] i dunno what x1-... is but it isn't relevant for my problem :) [18:24:35] thanks [18:24:36] haha [18:24:43] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#2987618 (10Halfak) We can probably not replace db1047 but we'll want to take backups of the user created dbs there. [18:24:54] i'm mostly concerned with use differences between s[12]-analytics-slave, and analytics-store [18:25:03] i'm thikning we don't need s[12]-analytics-slave on a different box [18:25:06] but i could be wrong [18:27:17] ottomata: ah, I see. So yeah, I don't think I've ever used s[12]. And as an analyst I hate having things split across different servers because then I can't join them. I don't know the infrastructure headaches that causes, though :) [18:28:36] well, neilpquinn, all dbs are on analytics-store [18:28:44] bye team! see you tomorrow :] [18:28:48] i think db1047 exists as an extra place where s1 and s2 are as well [18:29:01] so if people want to only query things like enwiki, they have another box to do it on [18:29:05] i think it might be unnecessary though [18:29:17] and we have to refresh the EL analytisc mysql boxes v soon [18:29:28] so i'm hoping we can just decom db1047 (s1 and s2 slave) altogether [18:29:37] ottomata: fyi, x1-analytics-slave is db1031.eqiad.wmnet. only contains flow and notifications data :) [18:29:38] ottomata: all dbs, yes, but not quite all tables :) https://phabricator.wikimedia.org/T115275 [18:30:05] right [18:30:14] interesting [18:30:44] neilpquinn: what if that data was in hadoop instead? [18:30:47] along with mw history [18:30:50] and EL [18:31:48] ottomata: I remember talking about this a couple of months ago. I'd be fine with all data in Hadoop—would be easier for me than dealing two different systems. [18:33:28] ottomata: we also don't have many other people in editing who do direct analytics queries, and as long as a GUI like Hue is available, I think they'd be willing (if not exactly eager) to move :) [18:34:10] neilpquinn: do you use a local SQL GUI when you connect to mysql dbs? [18:34:48] neilpquinn: back, let me read backscroll [18:35:33] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#2987618 (10Neil_P._Quinn_WMF) I've never used either s1 or s2—only analytics-store and (once or twice) x1. As an extra data point, the [data access docs](https://wikitech.wiki... [18:35:35] neilpquinn: no high level talk in our meeting, rather how-to-access new data [18:36:46] nuria: well, I meant higher level than a code tour. But still, I'm sure if I ask some high level questions, you'd be willing to try answering? :) [18:36:57] neilpquinn: jaja [18:37:47] neilpquinn: yes, the main thing i want to communicate is that teh part of teh data lake project that i think will most benefit you is going to be done by end of this month [18:37:59] neilpquinn: jaja -> spanish laugh [18:38:26] nuria: si, lo entiendo bien :) [18:38:30] juas [18:39:28] neilpquinn: just want to make sure you are using it and letting us know if you find anything odd, data is new and we still need to throughly vet it, that will happen next quarter (erik zachte is helping us do that) [18:39:46] nuria: pero 'juas' tenía que buscar :) [18:40:23] ay ay [18:41:07] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3107446 (10Halfak) I've just backed up the `halfak` DB. I can also be responsible for backing up the `staeiou` DB. [18:41:28] neilpquinn: can you share your notebooks with https://meta.wikimedia.org/wiki/New_Editor_Experiences stuff? [18:41:52] nuria: yep, I've used it for a couple things, although I have to invest a good amount of extra time just because there a lot of new things I need to learn (how to use date functions, how to use the job browsers, etc.) [18:42:21] neilpquinn: ya, those will pay off [18:42:34] nuria: but I will keep at it. I have some suggestions which I can share at our meeting next week :) [18:42:45] nuria: yes, definitely a good investment :) [18:43:15] 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#3107448 (10MusikAnimal) >>! In T123442#3105516, @Milimetric wrote: > The problem I'd be worried about is when traffic from a specific city makes sense, like there is local new... [18:44:27] ottomata: I use Sequel Pro a fair amount for quick queries, since it's a really nice interface for writing SQL (syntax highlighting, ability to get an easy overview of the table layout, etc.). But of course there's the problem of the connection timing out if the computer goes to sleep, so for longer running things I used to use screen sessions/tsv files/scp/etc., which worked but was overall a huge pain. [18:45:14] aye [18:46:25] ottomata: now I pretty much use the Jupyter hub for everything. writing SQL in python strings isn't as nice (e.g. no syntax highlighting), but it's 200% worth it for the ability to keep everything (SQL and Hive queries, Python scripts, graphing, even uploading MassMessage target pages to Meta) in one place :) [18:47:43] nice :) [18:47:52] ya, it'd be awesome if we had a direct hive sql interface there [18:48:01] rather than having to go through python [18:48:03] actually... [18:48:08] well, it'd be cli i guess [18:48:14] on the terminal in jupyter [18:48:19] i guess hue kinda has that [18:49:31] ottomata: well, overall, going through python is a feature rather than a bug for me. Because I do my analysis in Python. [18:50:07] (03PS16) 10Joal: Port standard metrics to reconstructed history [analytics/refinery] - 10https://gerrit.wikimedia.org/r/322103 (https://phabricator.wikimedia.org/T160155) (owner: 10Milimetric) [18:50:25] :) [18:50:58] 10Analytics, 10Analytics-EventLogging: Test pushing EventLogging through the ELK stack - https://phabricator.wikimedia.org/T160679#3107460 (10EBernhardson) [18:51:27] ottomata: a nice Hive GUI (haven't tried the one in Hue yet, so I don't know how it is) would be nice for the simpler projects where I just want to write an HQL query, get the output, and give it someone, though. [18:52:29] neilpquinn: its kinda nice, when it works (which is more often than not these days) [18:52:33] you can get graphs right there! [18:52:36] and maps! [18:53:42] neilpquinn: https://hue.wikimedia.org/notebook/editor?editor=15 [18:53:44] can you see that? [18:53:47] i think you should be able to [19:08:56] ottomata: yep, I can see it! I'll check it out and let you know what you think. [19:10:13] nuria: yeah, I'll share it! Busy at this exact moment, but I'll let you know. [19:11:41] (03PS17) 10Joal: Port standard metrics to reconstructed history [analytics/refinery] - 10https://gerrit.wikimedia.org/r/322103 (https://phabricator.wikimedia.org/T160155) (owner: 10Milimetric) [19:15:06] (03PS11) 10Joal: Add oozie jobs for mw history denormalized [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341030 (https://phabricator.wikimedia.org/T160074) [19:15:26] 10Analytics, 10Analytics-EventLogging: Find an alternative query interface for eventlogging Mariadb storage - https://phabricator.wikimedia.org/T159170#3107567 (10Ottomata) Quick off the top of my head list of alternative techs to explore: - ClickHouse - CitusDB - Apache Drill - Impala - Presto - Kylin [19:32:17] (03PS5) 10Joal: Add oozie job for standard metrics computation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/342197 (https://phabricator.wikimedia.org/T160151) [19:38:25] (03PS12) 10Joal: Add oozie jobs for mw history denormalized [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341030 (https://phabricator.wikimedia.org/T160074) [19:39:20] (03PS6) 10Joal: Add oozie job for standard metrics computation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/342197 (https://phabricator.wikimedia.org/T160151) [19:39:41] 06Analytics-Kanban, 10Pageviews-API, 13Patch-For-Review: Monthly aggregate endpoint returns unexpected results and invalid timestamp - https://phabricator.wikimedia.org/T156312#3107649 (10Nuria) @MusikAnimal : we are still doing some code on hive to load data with appropriate dates but on your end you should... [19:40:11] git st [19:40:14] oops :) [19:53:07] (03PS24) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T144717) [19:53:27] Leaving for tonight a-team, I have a big bunch of stuff to test tomorrow morning :) [19:53:36] laters joal! [20:34:18] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3107852 (10Nuria) List of tables: CentralAuth_5690875 ChangesListFilters_16174591 ChangesListFilters_16403617 CommandInvocation_15243810 ContentTranslation_11628043 ContentTranslationCTA_16017... [20:35:45] 06Analytics-Kanban, 10DBA: Change length of userAgent column on EL tables - https://phabricator.wikimedia.org/T160454#3107859 (10Nuria) We will need to: - send notice to users - update automated scripts [21:02:17] 10Analytics, 06Operations, 06WMF-Legal, 07Privacy: Honor DNT header for access logs & varnish logs - https://phabricator.wikimedia.org/T98831#3107961 (10Gilles) Aggregate counts isn't problematic, but the data we store is. If it's recorded, it can be compromised. I know we have retention policies, etc. but... [21:04:34] ottomata: updated my drop hourly partitions puppet patch, but jenkins is complaining about lines i didn't even touch being > 140 characters ... any opposition to me making a new puppet define that generates the crons for refinery-drop-*-partitions ? [21:05:12] only thing i'm not sure is where that define properly fits...seems inside the role is inappropriate [21:05:26] > 140 for puppet? [21:05:28] that is a new req [21:05:29] i guess [21:05:54] yea must be, 140 doesn't seem that insane to require < less [21:05:56] ebernhardson: not opposed to that, but ya, would be hard to find a right place...maybe we need a refinery module [21:05:56] < than even [21:06:00] meh [21:06:09] i'd just deal with it, leave it long for now [21:06:13] i'll push it trhough :) [21:06:16] ok [21:14:16] 10Analytics, 10Analytics-EventLogging: Test pushing EventLogging through the ELK stack - https://phabricator.wikimedia.org/T160679#3108023 (10EBernhardson) Did some quick checks, it looks like the kafka 0.8 protocol is only supported in logstash 2.x.x. We are currently running 1.5 in prod, and we will soon be... [21:35:03] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3108227 (10leila) @Ottomata: I'm only using dbstore1002 these days. For my work, I'm fine if db1047 is gone. [22:00:32] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3108353 (10Halfak) I've confirmed that all tables in `staeiou` are cleared for deletion. @staeiou can come here to confirm if he gets the ping. [22:02:48] 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#3108355 (10Staeiou) >>! In T156844#3108353, @Halfak wrote: > I've confirmed that all tables in `staeiou` are cleared for deletion. > > @staeiou can come here to confirm if... [22:11:15] milimetric: omg that desktop tab is almost done, Magic! [22:13:13] 10Analytics, 10Analytics-EventLogging: Test pushing EventLogging through the ELK stack - https://phabricator.wikimedia.org/T160679#3107460 (10Nuria) @EBernhardson thanks for this work, I would be forever grateful if for agreggation we can use something that is not logstash [22:14:51] nuria: do you mean not mysql? :) [22:16:13] ebernhardson: no, that kibana ui that looks all nice but it is not useful at all [22:17:50] hmm, interesting. There really arn't other options in the elasticsearch world for these kinds of things. I'm not really sure what other orgs are doing exactly but plenty are using kibana for analytics. [22:18:31] ebernhardson: for counters, kind of like graphana? [22:19:06] ebernhardson: as in 'measures' rather that mutidimensional cube analisys, right? [22:19:16] nuria: it can aggregate to counters, but it can also do multidimensional analysis [22:19:25] ebernhardson: is there any public one you can point me to that you like? [22:19:38] i'm not sure what the interface for those kinds of things works out as though, i've not really used it for that purpose i just know they heavily promote it as such [22:20:06] ebernhardson: when i say multidimensional i mean something like http://pivot.wikimedia.org [22:20:30] ebernhardson: counters i can see, ya, for metrics, sure, let me see if i find one public [22:22:01] ebernhardson: i think kibana and superset are about same no? https://github.com/airbnb/superset [22:22:18] ebernhardson: seems that kibana is better than i remember it though [22:23:01] kibana was entirely re-written ~ 2 years ago, so depending on age you might be remembering the old app [22:23:08] ebernhardson: me -> old [22:23:27] ebernhardson: we casually have in our goals for next year a better visual exploration of EL data [22:23:35] :) the superset images at least look quite a bit like what kibana provides [22:24:39] ebernhardson: right, neither are cube-exploring like pivot but rather regular analytics dashboarding, yes [22:25:02] i think a big difference you will see though is that druid supports roll-up at ingestion time, which elasticsearch will not. For eventlogging sized data probably not a problem though [22:25:38] if eventlogging could go into druid instead that might be useful, i dunno. I just thought of this this morning as dba was concerned about eventlogging -> mysql issues [22:27:07] ebernhardson: I think the issue is more what types of queries you want to consume, for EL dashboarding is only useful for schemas like the ones used by perf team (counters rather than dimensions) [22:28:14] ebernhardson: for schemas with different dimensions you need a storage optimized to query in such a way not just the ui, i imagine elastic search does that too [22:28:49] ebernhardson: our plan with EL is to do away with mysql consumer [22:31:06] es offers two different storage methods, column based and inverted-indexes (can use both at same time and it will choose which to use), but not any kind of pre-rollups. you can certainly do multi dimensional aggregations in elasticsearch like i've done in pivot, but for the size of data we currently use in pivot (webrequests) elasticsearch would probably choke. eventlogging data probably small enough thats not a concern [22:31:34] but, if you already have plans that seems just fine. I just threw this out as an idea this morning and ottomatta thought it was worth at least doing a basic test of [22:36:27] ebernhardson: i totally agree that it is worth a test. please do not be shy suggesting , i just do not love kibana but that was the old kibana as you mentioned [22:36:40] :)