[07:24:49] hello people, db1107 has processed 157/304 alter tables up to now, looking good [07:25:01] one last 5:30h :D [07:25:14] err 5.3h [07:26:49] (afk, bbl!) [08:27:34] heyaaaa :] [08:28:39] o/ [09:02:52] PROBLEM - Check status of defined EventLogging jobs on eventlog1001 is CRITICAL: CRITICAL: Stopped EventLogging jobs: consumer/mysql-m4-master-00 consumer/mysql-eventbus [09:05:12] downtime expired --^ [09:08:30] so good news for the kernel updates - it seems that a cpu feature called PCID (Process context identifier) mitigates the performance impact of the new security features, and our analytics hosts support it afaics [09:08:43] joal: --^ [09:16:45] elukey: This is super good news :) [09:17:12] Good morning elukey :) [09:19:39] morning :) [09:20:33] elukey: Something else I missed last year: http://hadoop.apache.org/docs/r3.0.0/index.html [09:21:42] nice! [10:34:38] !log re-run mediacounts-archive-wf-2018-01-03 [10:34:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:35:54] elukey: Man, I wanted to do that --^ but got stuck in hue-joining wait process, then forgot - Many thanks ! [10:36:07] :) [10:36:16] it was a java oom, checking now logs to see if it re-happens [10:36:27] was hue acting weirly? [10:36:31] *werdly [10:36:38] ufffffffffffffffffffffff can't write [10:36:41] you got it :D [10:36:49] hehe [10:37:18] elukey: nothing weirder than usual: hue login is super long due to hdfs-http not being open [11:31:09] * elukey lunch! [12:02:55] (03PS1) 10Mforns: Replace setTimeout with Vue.nextTick [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/402026 (https://phabricator.wikimedia.org/T180412) [12:03:31] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Replace any debouncing with Vue.nextTick - https://phabricator.wikimedia.org/T180412#3756944 (10mforns) [12:05:01] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats Bug: Menu to select projects doesn't work (sometimes?) - https://phabricator.wikimedia.org/T179530#3874933 (10mforns) a:05fdans>03mforns [12:11:24] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats Bug: Menu to select projects doesn't work (sometimes?) - https://phabricator.wikimedia.org/T179530#3874962 (10mforns) I can reproduce the error in Ubuntu with both Chrome and Firefox. Will look into that right now. [12:33:05] aaand we are polling prometheus metrics for hadoop on the prometheus masters!! [12:33:08] \o/ [12:36:13] * joal dances the metrics dance for elukey [12:53:12] (03CR) 10Mforns: [C: 04-1] "Wait, I think I need some more changes." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/402026 (https://phabricator.wikimedia.org/T180412) (owner: 10Mforns) [12:56:35] (03CR) 10Mforns: "No, no. It's fine I think :]" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/402026 (https://phabricator.wikimedia.org/T180412) (owner: 10Mforns) [12:59:53] elukey: Just saw your message about the mediacount job [13:01:51] elukey: doing some tests to suggest a solution [13:02:05] super :) [13:24:21] (03PS1) 10Joal: Update oozie mediacounts archive job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/402041 [13:25:14] elukey: --^ [13:25:28] makes sense :) [13:51:29] taking a break a-team [14:03:46] (03CR) 10Milimetric: [C: 032] Replace setTimeout with Vue.nextTick [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/402026 (https://phabricator.wikimedia.org/T180412) (owner: 10Mforns) [14:03:57] thx [14:31:15] (03CR) 10Ottomata: [C: 031] Update oozie mediacounts archive job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/402041 (owner: 10Joal) [14:42:40] ottomata: o/ - thanks for the reviews! [14:42:47] :) [14:49:59] ottomata: I know that you'll probably going to kill me but I think I broke superset [14:50:43] I didn't check the private repo [14:51:02] the hieradata/role/common/statistics/web.yaml private config needs to be moved [14:51:20] ooo [14:51:35] yeah and it tried to init again, but admin/admin prevented it :D [14:51:45] going to fix it sorry [14:52:32] no prob! [14:52:35] probably no one willl noice [14:53:29] so content should migrate from hieradata/role/common/statistics/web.yaml to hieradata/role/common/analytics_cluster/webserver.yaml right? [14:55:05] sounds right yeah [15:00:03] all back online [15:10:56] ottomata: whenever you have a min I'd need to ask you one thing about a puppet error that I am seeing [15:13:43] yaaa go head elukey [15:14:12] so I am getting the following [15:14:15] Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Class[Standard] is already declared; cannot redeclare at /etc/puppet/modules/role/manifests/analytics_cluster/hadoop/worker.pp:11 at /etc/puppet/modules/role/m [15:14:21] anifests/analytics_cluster/hadoop/worker.pp:11:5 on node hadoop-worker-2.analytics.eqiad.wmflabs [15:15:02] I applied the analytics_cluster::hadoop::worker role, and it indeed list class { 'standard': } but I don't see other occurrences in profiles etc.. [15:15:09] plus this doesn't happen in prod [15:15:57] so, not sure where else it i is declared, but [15:16:06] there is a difference between class { 'standard': } and include standard [15:16:20] include can happen multiple times and puppet will only do it once [15:16:37] class {} expects that there might be paramaters, which could be declared differently in differnet places, so puppet won't allow it [15:17:44] yep yep, but I thought that any problem with this setting should have arose after the first puppet run in prod [15:17:59] that's the only class you are including on the node in labs? [15:18:01] role::analytics_cluster::hadoop::worker [15:18:02] ? [15:18:29] maybe labs is tricky? maybe there is a base node definition that declares standard somehow? [15:18:40] but, i betcha if you change the class { 'standard' to include ::standard [15:18:42] it'll work [15:19:23] ah yes, I put class { 'standard': } because of those guidelines, maybe in this case we can avoid it [15:19:29] going to study it a bit more ,thanks :) [15:19:51] the only places i see class { 'standard' in puppet is in your newly refactored role classes [15:19:57] everywhere else it is include [15:22:11] "All roles should include the standard profile" [15:22:16] but doesn't exists [15:22:19] uffff [15:23:15] doesn't exists? [15:24:15] I can only see the standard class, not profile [15:25:05] in theory if I do include ::standard in a role I get a -1 due to a violation [15:29:12] yeahhh, i think i've seen that before [15:29:13] its crazy [15:29:17] these rules are crazy! [15:40:33] holaaa [15:42:42] o/ [16:01:03] !log killing json_refine_eventlogging_analytics job that started yesterday and has not completed (has no executors running?) application_1512469367986_81514. I think the cluster is just too busy? mw-history job running... [16:01:04] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:03:45] NO PING [16:03:48] OH Wait [16:03:50] it is not standup time [16:03:51] haha [16:03:52] yeahhhhh [16:05:18] haahhah [16:12:56] ottomata: I think we're not far with elukey of having the puppet-side of druid-oriented streaming jobs: cron for restarting the job if needed, and alerts based on druid ingestion of events [16:13:09] ooo really? [16:13:10] COOL [16:13:25] ottomata: Would you give me a +2 on the scala code of refinery-source for this to actually be deployed? [16:13:31] link? [16:13:33] sure [16:13:49] https://gerrit.wikimedia.org/r/#/c/373030/ [16:13:50] i think we should probably do streaming jobs like this in two parts, one that writes the derived stream to kafka, another that just consumes and posts to druid [16:13:56] then the job is not druid dependent [16:14:05] I am not fully convinced that we need to restart those jobs, I'd set up alarms for now and see if we can fix the issues.. now the banner impressions seem stable no? [16:14:30] (genuine question, not advocating for one solution) [16:14:46] elukey: banner impression job seems very stable - the restart is to automate the fact that it could crash for instance in case of hadoop restart [16:15:10] true true [16:15:20] ottomata: mixed feeling about double jobs: well first it means double jobs, therefore 2 jobs to maintain [16:15:59] ottomata: second, since we're doing lambda, data will be fed through batch later, so no real need to make resilient streams [16:16:01] hmm, joal, maybe not double jobs...but what about write to kafka and consume from kafka in the same job? [16:16:13] or, write to kafka and then also write to druid [16:16:17] ottomata: Very feasible [16:16:39] someone i suspect that the derived output of many jobs like this will be nice to have in kafka for doing other things [16:16:43] somehow* [16:16:55] joal you already have a +1 on that from me :) [16:16:57] here's your +2 [16:17:12] (03CR) 10Ottomata: [C: 032] Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [16:17:18] ottomata: I do agree on that one ottomata - but we're kinda far from it as of now [16:17:20] aye [16:17:24] yeah, fine with me as is [16:17:28] let's keep that in mind in the future though [16:17:28] THanks mate :) [16:17:35] very much agreed ottomata [16:18:00] would be really nice to have a standardized way of doing that stuff, then you could expect to see the derived job stream in kafka, and use it for debugging, historical analysis in hadoop, etc. [16:19:50] ottomata: yessir [16:23:23] Heya fdans - I'd like to deploy - Do you mind if I go for it? [16:23:51] (03CR) 10Joal: [V: 032 C: 032] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/402041 (owner: 10Joal) [16:29:03] (03CR) 10jerkins-bot: [V: 04-1] Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [16:29:34] Arf crap [16:30:08] joal: yesssss gogogo [16:31:20] (03PS16) 10Joal: Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) [16:31:23] Thanks fdans [16:43:21] (03CR) 10Joal: [C: 032] "Trying again after rebase." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [16:43:56] fdans: have you ever done a deployment? [16:45:00] (03CR) 10jerkins-bot: [V: 04-1] Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [16:46:28] (03CR) 10jerkins-bot: [V: 04-1] Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [16:54:30] of course !!! My bad - sorry for spams a-team [16:56:17] (03PS17) 10Joal: Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) [16:57:48] spams? [16:58:09] the -1s :) [16:58:32] elukey: you know my ways :) [16:58:52] sorry elukey I was having a super late lunch [16:58:59] almost dinner [16:59:13] I lose conscience of time here in galicia [17:00:07] fdans: lunch at 17:00 -- THIS feels like spain :) [17:01:19] ping ottomata fdans [17:01:26] ping joal [17:01:32] standdduppp [17:01:39] soooooorry [17:14:59] elukey: Linus way :) [17:15:07] elukey: https://lkml.org/lkml/2018/1/3/797 [17:15:42] joal: and he was really polite (compared to his standards :) [17:15:50] yeah :) [17:21:15] 10Analytics: When displaying a graph include metric total not only average - https://phabricator.wikimedia.org/T184139#3873650 (10Milimetric) Metrics are currently configured as additive (Pageviews, Edits) or non-additive (Uniques, Edited Pages). I think in this case the metric is just mis-configured. But do y... [17:21:41] 10Analytics, 10Analytics-Wikistats: Wrong y-axis labels on wikistats graph - https://phabricator.wikimedia.org/T184138#3873625 (10Milimetric) ah, it's getting cut off, cool [17:28:07] (03CR) 10Joal: [C: 032] Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [17:35:59] (03Merged) 10jenkins-bot: Add tranquility to the banner streaming job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/373030 (https://phabricator.wikimedia.org/T168550) (owner: 10Joal) [17:45:31] 10Analytics: Pivot "MediaWiki history" data lake: Feature request for "Time" dimension to sp\lit by calendar month / quarter / year -- needs druid 0.10 - https://phabricator.wikimedia.org/T161186#3875841 (10fdans) 05Open>03Resolved a:03fdans Pivot can't do this monthly (the new one does, but we can't upgra... [17:48:40] (03PS1) 10Joal: Update changelog to v0.0.57 before deploy [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/402088 [17:49:25] 10Analytics, 10Fundraising-Backlog: Storage for banner history data - https://phabricator.wikimedia.org/T161635#3875876 (10fdans) 05Open>03declined @DStrine closing this task since there are no new updates. Feel free to reopen and ping us if you get back to it. [17:50:39] 10Analytics-Kanban: Upgrade druid - https://phabricator.wikimedia.org/T157977#3875879 (10fdans) [17:54:15] 10Analytics, 10Analytics-Wikistats: When displaying a graph include metric total not only average - https://phabricator.wikimedia.org/T184139#3875885 (10Nuria) [17:56:02] 10Analytics-Kanban, 10Analytics-Wikistats: When displaying a graph include metric total not only average - https://phabricator.wikimedia.org/T184139#3875890 (10fdans) a:03Nuria [17:58:15] 10Analytics-Kanban, 10Analytics-Wikistats: Wrong y-axis labels on wikistats graph - https://phabricator.wikimedia.org/T184138#3875908 (10fdans) [17:59:13] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: When displaying a graph include metric total not only average - https://phabricator.wikimedia.org/T184139#3875915 (10fdans) [18:00:49] 10Analytics-Kanban: Replacing standard edit metrics in dashiki with data from new edit data depot - https://phabricator.wikimedia.org/T143924#3875921 (10Nuria) Please see http://stats.wikimedia.org/v2 for edit metrics [18:00:56] 10Analytics-Kanban: Replacing standard edit metrics in dashiki with data from new edit data depot - https://phabricator.wikimedia.org/T143924#3875924 (10Nuria) 05Open>03Resolved [18:00:59] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2.0. - https://phabricator.wikimedia.org/T130256#3875925 (10Nuria) [18:01:02] 10Analytics, 10Analytics-Wikistats: Questionable metrics from Wikistats 2.0 Alpha - https://phabricator.wikimedia.org/T184011#3875926 (10fdans) 05Open>03Invalid [18:01:53] 10Analytics: Publishing project anomaly data for censorship researchers. Evaluate privacy threats - https://phabricator.wikimedia.org/T183990#3875929 (10fdans) a:03Nuria [18:05:24] 10Analytics: Enhance mediawiki-history page reconstruction with best historical information possible - https://phabricator.wikimedia.org/T179692#3733080 (10JAllemandou) [18:05:26] 10Analytics: continue to improve computation for pages, deletion/restores - https://phabricator.wikimedia.org/T183975#3875951 (10JAllemandou) [18:07:48] 10Analytics, 10Analytics-Wikistats, 10Hindi-Sites: Hindi Wikiversity is not showing in Wikimedia Stats - https://phabricator.wikimedia.org/T183682#3860400 (10fdans) This project is not included in the sqoop list that we use to get data [18:08:51] 10Analytics-Kanban, 10Analytics-Wikistats, 10Hindi-Sites: Hindi Wikiversity is not showing in Wikimedia Stats - https://phabricator.wikimedia.org/T183682#3860400 (10fdans) p:05Triage>03High [18:11:47] (03CR) 10Milimetric: Replacing JSON download with CSV download (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/401814 (https://phabricator.wikimedia.org/T183192) (owner: 10Nuria) [18:15:13] 10Analytics, 10Discovery, 10Discovery-Analysis, 10Discovery-Search: UDF for language detection - https://phabricator.wikimedia.org/T182352#3875990 (10debt) Punting to later - it'd be cool, but not sure we have the time to really dig into this right now. [18:17:26] 10Analytics-Kanban: Remove sensitive fields from whitelist for QuickSurvey schemas (end of Q2) - https://phabricator.wikimedia.org/T174386#3875993 (10Nuria) a:03fdans [18:21:57] Hello analytics people [18:22:14] Is it expected for the SQL view on EventLogging data on analytics-slave to be lagged or be periodically updated? [18:23:30] select max(timestamp) from ChangesListFilters_16837986 has been stuck at 20180103085902 (~34h ago) for the last ~8h at least [18:23:33] Hi RoanKattouw - eventlogging insertion into mysql has been stopped yesterday and will probably last until tomorrow due Upgrade of tables [18:23:43] Aha OK that makes senes [18:24:13] RoanKattouw: Normally, when started again, insertions should catch up (after some lag of course) [18:24:41] OK cool [18:25:11] I'm aggregating 3 months of data so I can easily live without the last 24h, just wanted to make sure there wasn't some unreported breakage going on [18:25:24] sounds good RoanKattouw :) [18:25:29] thanks for pinging [18:25:45] RoanKattouw: data is in hive though if you like! [18:25:47] updated hourly [18:25:48] :D [18:26:37] Thanks :) [18:31:12] nuria_: re our earlier talk about server access, I looked into using the beta cluster for data collection, but eventlogging seems very dead there [18:31:46] tgr: it shouldnt' be, will look into in in a bit [18:31:48] the db server ran out of disk space and looking at the tiny number of event schemas it seems like it stopped working years ago [18:31:58] not years, but that seems to happen all the time [18:31:58] thanks! [18:32:20] fdans: do you mind looking at https://gerrit.wikimedia.org/r/402088? [18:32:21] beta seems to need a constant eye or it gets rsuty [18:32:55] maybe there is some filtering going on then, but there seem to be way too few tables in the log DB [18:33:44] (03CR) 10Fdans: [C: 032] Update changelog to v0.0.57 before deploy [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/402088 (owner: 10Joal) [18:33:56] oh tgr we don't try and keep data there, often if we run into problems we wipe it [18:34:00] or start a new node or something [18:35:00] hrmm, working on q3 procurement, i have a note for WMCS data lake project [18:35:11] do you folks have any kind of documentation on what kind of servers those will need to be? [18:35:30] (i figured id ask in irc before making a task asking the same thing ;) [18:35:45] WMCS? [18:35:48] CS [18:35:49] ah yes [18:35:49] yes [18:36:08] im just pasting the description cell from the google sheet ;] [18:36:13] uhhhhh robh hard to say actually, we can't decide on what datastore to use. druid maybe, buuut it isnt' really that great for hte CS use case [18:36:24] we need to do some tech evaulation [18:36:41] ottomata: Id go for druid-style machines [18:36:49] but, probably 3ish nodes wiht lots of ram and a fair amount of storage and disk io, so maybe SSDs [18:36:51] yeah [18:36:51] understood, just keep in mind its scheduled for q3 which means the harware ahs to be onsite before the end of march =] [18:36:53] something like that [18:37:09] welllLLLllll we might not do it q3 after all...nuria_? [18:37:21] says note of 'pushed to late Q3 by Nuria (Dec 4th)' [18:37:37] but im just touching base on all of the upcoming orders ;] [18:37:37] * elukey off! [18:38:02] ottomata: so if no answer today that is 100% ok [18:38:09] i just didnt wanna assume and ignore. [18:40:27] (03Merged) 10jenkins-bot: Update changelog to v0.0.57 before deploy [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/402088 (owner: 10Joal) [18:44:05] fdans: about deploy, now that changelog is merged, I'm gonna start a jenkins build to make it get the latest mvn version [18:44:38] right [18:46:03] tgr: is eventlogging beta working now? Data gets wiped out as we test on it for what ottomata said, we test a bunch of our mysql-related changes there [18:46:09] tgr: let me take a look [18:50:35] tgr: dropped some tables [18:51:12] thanks nuria_, seems to be working now [18:51:24] tgr: restarting everything [18:53:13] tgr: ok, we use it frequently to test but as ottomata said it requires a lot of hand holding to work continuosly due to how small disk wise those instances are [18:53:25] tgr: let me know if you find any other problems [19:06:07] milimetric: from CR: this function: https://github.com/wikimedia/analytics-wikistats2/blob/master/src/models/GraphModel.js#L116 does not flatten a generic json object right? it assumes two levels one of which is "total"? [19:06:22] milimetric: like {a;1, b:{total:2}} [19:10:39] nuria_: right, but it’s used by the tabular graph, I thought it might be easier to work with that output, sorry I rushed that comment during the meeting [19:11:01] feel free to ignore if you disagree it’s useful [19:14:13] milimetric: ah i see, it is what teh tabular graph uses to present data [19:14:20] milimetric: ok, ya, that should work then [19:38:01] !log Deploy refinery-source using jenkins [19:38:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:38:34] nuria_: as you noticed, I wrote some on the blog doc - I'm very happy for you to make it cleaner / simpler :) [19:38:56] joal: sounds good, let me know when you are done and i will give it apass [19:39:25] nuria_: done for today - I think Dario would be better than me about knowing how the dataset has been used and why it is usefu [19:40:06] also nuria_ - fell free to drop any part you think is not needed - I like to provide concrete info on data, so I wrote some on that, but it might be wass too much [19:40:07] joal: ok, will write an intro for everyone and reorganize a bit adding todos for dario [19:40:49] joal: i think that is good too but see the eventreams blogpost for example, the beginning can be read an understood by non tech folks, the rest is pretty technical [19:41:00] sure [19:41:30] joal: BOTH are needed cause otherwise it is too high level for people that are into it [19:41:31] https://blog.wikimedia.org/2017/03/20/eventstreams/ [19:42:23] nuria_: I did simple graph analysis on the data and results are laready super fun :) [19:42:48] nuria_: I've managed to build cliques of articles that are pretty consistent :) [19:42:50] joal: and the data is both desktop and mobile clickstream? [19:43:06] nuria_: yes [19:43:30] joal: and when you say "have internal type" means that is an internal wiki link , right? [19:43:33] nuria_: not looking at sessions here - really just referer -> article, so no problem with sessionisation [19:43:34] wiki to wiki [19:44:04] nuria_: you should read the blog post :) Internal means internal inside the same wiki [19:44:35] joal: argh sorry, yes, the 'same site you are in" [19:44:49] correct [19:45:01] since the clickstream is single-site oriented [19:45:08] !log Deploy refinery using scap [19:45:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:45:12] k [19:45:19] will do this later on today [19:45:51] nuria_: I actually think we should try to push people to read Ellery's blog post - It's all super well explained there [19:47:31] joal: taht too, but imagine the ED reads this blogpost , she should have an idea of what this is about and why this matters w/o knowing what is a graph [19:47:53] nuria_: For sure - Not saying we shouldn't make a good non-tech intro :) [19:47:54] joal: cause clicks on websites are actually really easy to understand intuitevely [19:48:23] joal: and of course, for the fans, knowing all the details of why this is insightful is ALSO great [19:48:54] joal: so both are needed, yeah yeah [19:54:40] !log Deploying refinery onto hadoop [19:54:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:56:00] ok, deploy done - Gone for diner, I'll care restarting oozie for mediacounts tomorrow [19:56:14] laters! [19:56:17] Have a good end of day a-team [20:06:20] joal: I've actually starting writing a post for wm blog about clickstreams :) [20:22:52] bearloga: you may want to have a chat with Dario as I know he's working on one, too. [20:24:07] leila: thanks! will do! [20:35:32] bearloga: wait.. i missed your initial question... [20:36:05] nuria_: I didn't have a question! :) [20:36:29] ah sorry, let me share you our blogpost for the upcoming release [20:37:21] bearloga: Dario is working on the one i just shred, we started it yesterday [20:37:29] nuria_: cool! thank you! [20:37:42] bearloga: let's not write 2 at teh same time but rather coordinate pleaseee [20:37:56] * bearloga agrees 100% [20:41:50] haha k [21:54:22] 10Analytics, 10Discovery, 10Wikidata, 10Wikidata-Query-Service: Data request for logs from SparQL interface at query.wikidata.org - https://phabricator.wikimedia.org/T143819#3876607 (10Smalyshev) I made a more formal full description of which data I'd like to be in the public dataset, so people don't have... [23:33:58] (03PS3) 10Nuria: Replacing JSON download with CSV download [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/401814 (https://phabricator.wikimedia.org/T183192) [23:37:05] (03PS4) 10Nuria: Replacing JSON download with CSV download [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/401814 (https://phabricator.wikimedia.org/T183192) [23:37:13] (03CR) 10Nuria: Replacing JSON download with CSV download (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/401814 (https://phabricator.wikimedia.org/T183192) (owner: 10Nuria)