[11:38:38] <mforns>	 hi a-team!
[11:43:13] <milimetric>	 hey marcel
[11:43:33] <milimetric>	 I'm not really awake yet, but didn't wanna leave you hangin :)
[11:43:41] <mforns>	 good morning milimetric! :]
[11:44:03] <mforns>	 i thought elukey was here :]
[11:44:36] <moritzm>	 mforns: he's back tomorrow
[11:49:15] <mforns>	 moritzm, oh! thanks
[14:13:08] <grrrit-wm>	 (PS24) Mforns: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: Joal)
[14:29:55] <grrrit-wm>	 (CR) Ottomata: "- Let's default to using underscores in filenames that are not executables, unless there is a reason to do so. E.g. create_project_namesp" (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476) (owner: Milimetric)
[14:30:43] <grrrit-wm>	 (CR) Ottomata: Script sqooping mediawiki tables into hdfs (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476) (owner: Milimetric)
[14:43:45] <nuria_>	 mforns: re:vital signs, pageviews are working fine cc Deskana (it is the editing metrics that we are going to remove)
[14:44:37] <nuria_>	 milimetric: let's talk about EL after standup. there are couple things it needs like throttling
[14:47:49] <milimetric>	 k, cool
[14:49:32] <wikibugs>	 Analytics: Stop vital signs metric creation on wikimetrics - https://phabricator.wikimedia.org/T143715#2579091 (Nuria)
[15:00:32] <nuria_>	 ottomata: standddupppp
[15:01:48] <wikibugs>	 Analytics-Kanban, Analytics-Wikimetrics: Can't run reports on Wikimetrics - https://phabricator.wikimedia.org/T143399#2579109 (mforns) a:mforns>Nuria
[15:18:17] <wikibugs>	 Analytics: Revamp Eventlogging so anyone can use it - https://phabricator.wikimedia.org/T143794#2579160 (Nuria)
[15:27:15] <wikibugs>	 Analytics: Revamp Eventlogging so anyone can use it - https://phabricator.wikimedia.org/T143794#2579160 (Ottomata) Anyone can use our installation of it?  Or anyone can install and use their own installation of it?
[15:49:05] <milimetric>	 mforns: you wanna chat in the batcave for a bit, about next work?
[15:49:14] <mforns>	 milimetric, sure
[15:49:16] <mforns>	 omw
[16:35:18] <elukey>	 hello people :)
[16:44:42] <elukey>	 I checked the AQS alarms and I feel really sad
[16:44:48] <elukey>	 latencies went up again with traffic
[16:44:52] <elukey>	 :/
[16:48:14] <elukey>	 plus https://grafana.wikimedia.org/dashboard/db/aqs-cassandra-compaction looks a bit weird for the 5th month
[16:48:45] <elukey>	 all right, will start looking into my dear AQS tomorrow morning
[16:49:03] <elukey>	 probably I worry too much :D
[16:49:20] <elukey>	 thanks a lot for having loaded the new cluster!
[16:55:34] <mforns>	 hi elukey! don't worry about the cassandra compaction, we've been naughty and loaded new months before the old ones finished compacting
[16:55:49] <mforns>	 to test, and it looks good for now
[16:57:37] <milimetric>	 hi Luca :)  We're excited to have you back but please don't worry until tomorrow
[16:59:14] <elukey>	 o/
[16:59:21] <elukey>	 sure sure I worry too much :)
[17:03:42] <elukey>	 mforns: will ping you tomorrow for the details! :P
[17:05:00] <mforns>	 ok
[17:10:30] <elukey>	 going afk, talk with you tomorrow!!
[17:42:39] <nuria_>	 elukey: do not worry, i can explain the compaction tomorrow
[17:42:52] <nuria_>	 elukey: i am .. *ahem* doing some experiments .. muahahahaha
[20:12:35] <wikibugs>	 Analytics: ResearchSpike:  Pivot UI: react, playwood - https://phabricator.wikimedia.org/T143828#2580389 (Nuria)
[20:14:22] <wikibugs>	 Analytics-Kanban, Analytics-Wikimetrics: Can't run reports on Wikimetrics - https://phabricator.wikimedia.org/T143399#2567093 (Nuria) Open>Resolved
[20:14:41] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Extract edit oriented data from MySQL for simplewiki - https://phabricator.wikimedia.org/T134790#2580410 (Nuria)
[20:14:42] <wikibugs>	 Analytics-Kanban, EventBus, MW-1.28-release-notes, Patch-For-Review, WMF-deploy-2016-08-16_(1.28.0-wmf.15): Propose evolution of Mediawiki EventBus schemas to match needed data for Analytics need - https://phabricator.wikimedia.org/T134502#2580408 (Nuria) Open>Resolved
[20:24:18] <milimetric>	 nuria_: ClickHouse looks pretty amazing
[20:24:18] <milimetric>	 "It also offers an opportunity to use external dictionaries, dimension tables loaded from an external source, for seamless joins."
[20:24:34] <nuria_>	 milimetric: ya, joal was really wanting to try it
[20:24:52] <milimetric>	 using that ^ we can load once and never re-load unless we change the history reconstruction algorithm
[20:25:05] <milimetric>	 the spike with Druid is about finding out if that's even possible
[20:25:07] <nuria_>	 milimetric: that is why on the goals might need to say "test whether data can be efficiently loaded and query in a columnary datastore"
[20:25:28] <milimetric>	 big data OLAP would be even more generic
[20:28:03] <milimetric>	 hahaha, the code comments are in russian.  sold
[20:29:28] <nuria_>	 milimetric: man ...
[20:58:05] <milimetric>	 nuria_: "Pivot can connect to multiple clusters, also Postgres and MySQL" is a recent improvement
[20:58:15] <milimetric>	 a bit of a downside is it uses TypeScript
[21:09:38] <mforns>	 milimetric, hey! do you know where the namespaces CSV file is in hadoop?
[21:09:54] <milimetric>	 yes, one sec
[21:09:57] <milimetric>	 but you should be sleeping!
[21:10:09] <mforns>	 xD
[21:10:34] <milimetric>	 mforns: /user/milimetric/mediawiki/namespace_db_mapping/namespace.dictionary.csv
[21:10:40] <mforns>	 thanks milimetric!
[21:10:41] <milimetric>	 that's the latest with the db names in it
[21:10:44] <mforns>	 ok
[21:11:16] <milimetric>	 the schema is here: https://gerrit.wikimedia.org/r/#/c/306292/1/hive/mediawiki/edit-history/create-project-namespace-map-table.hql
[21:28:52] <milimetric>	 nuria_: yeah, so pivot already supports loading files and querying them
[21:29:03] <milimetric>	 it's not super elegant, and it loads the whole file in memory
[21:30:20] <milimetric>	 so we could send a pull request with a proper file query module, but it would be very simple work from what I can tell.  It's just a matter of transforming the file and maybe querying it efficiently without loading it in memory
[21:46:35] <milimetric>	 wow... pivot looks like about 80k lines of code
[22:38:59] <nuria_>	 milimetric: ok, let's talk tomorrow about whether that looks like something we want to tackle