[08:18:50] Good morning a-team ! [08:18:54] * joal is back :) [08:19:28] Welcome baaaaaaack joal!!! [08:19:39] so glad you're here :) [08:19:50] I'm happy too :) [08:19:58] How are you fdans? [08:20:44] very well, working from Galicia for a couple weeks :D [08:20:50] Nice :) [08:22:22] fdans: let me know if there is anything I can do for you you :) [08:22:55] joal: I'm hoping to see a standup debut of Naé soon [08:23:04] fdans: my plan for today is to catch up on emails and phab, and discuss with you guys at standup what I should be working on [08:23:11] hehehe fdans :) [08:23:17] It'll come for sure [08:26:34] other than that, we probably need to talk marcel you and me about the new editors metric for wikistats [08:27:28] ok :) new editors is the first editor-oriented wikistat-2 metrics we're putting together, right? [08:31:27] fdans: --^ [08:31:54] that is correct joal [12:11:08] halfak: Good morning ! Let me know if you have 1/2 hour this morning, I'd like to catch up on some tasks I have noticed :) [12:55:47] HELLOOOOO a-team! [12:56:03] Good morning Master ottomata :) [12:56:14] my joal! [12:56:16] hallo [12:56:20] long time no see ;) [12:56:24] indeed! [12:57:01] how long does it take to check email after paternity leave? :) [12:57:14] hehe, good questions :) [12:57:40] I'm done quick-scanning, now need to let my brain process, and talk to people to prioritize :) [13:00:27] heloooo team :] [13:00:33] Hi mforns ! [13:00:39] joaaaaaaaal :D [13:00:43] how are you? [13:01:05] I'm good mforns :) [13:01:08] Thanks :) [13:01:11] How are you? [13:01:15] good! [13:01:18] welcome back [13:26:19] !log adding index on (database, rev_timestamp) on mediawiki_page_create_2 table on dbstore1002: T170990 [13:26:21] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:26:21] T170990: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990 [13:32:32] 10Analytics-Kanban, 10Operations, 10Patch-For-Review, 10User-Elukey: Analytics Kafka cluster causing timeouts to Varnishkafka since July 28th - https://phabricator.wikimedia.org/T172681#3537706 (10faidon) >>! In T172681#3526998, @elukey wrote: > 1) Make sure to install/deploy pmacct 1.6.2 (follow up wi Deb... [13:39:13] 10Analytics-Kanban, 10User-Elukey: Calculate how much Popups events EL databases can host - https://phabricator.wikimedia.org/T172322#3537746 (10Ottomata) Hm, @nuria, I noticed that you imported this data into `/wmf/data/archive/eventlogging`, but created the table in the `wmf` database. Perhaps it would be b... [13:40:53] 10Analytics: Add pagecounts by article and top pagecounts to AQS - https://phabricator.wikimedia.org/T173720#3537749 (10mforns) [13:46:16] !log adding index on (database, rev_timestamp) on mediawiki_page_create_2 table on db1047: T170990 [13:46:18] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:46:18] T170990: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990 [13:50:49] 10Analytics-Kanban, 10User-Elukey: Calculate how much Popups events EL databases can host - https://phabricator.wikimedia.org/T172322#3537783 (10mforns) @Ottomata Nuria put the tables there, knowing that you would have an opinion, so she already mentioned that you could change them to wherever you saw fit.... [13:59:55] 10Analytics-Kanban, 10User-Elukey: Calculate how much Popups events EL databases can host - https://phabricator.wikimedia.org/T172322#3537799 (10Ottomata) Great :) [14:07:33] 10Analytics, 10Analytics-Wikistats: yearly folder in hdfs for mediacounts does not have write access for Wikistats by default (patched manually for earlier years) - https://phabricator.wikimedia.org/T173724#3537853 (10Erik_Zachte) [14:08:19] 10Analytics, 10Analytics-Wikistats: yearly folder in hdfs for mediacounts does not have write access for Wikistats by default (patched manually for earlier years) - https://phabricator.wikimedia.org/T173724#3537867 (10Erik_Zachte) [14:23:32] hey a-team! I gotta take my grandma to the hospital (nothing bad, just a checkup), so I can't make it to today's meetings but I'll be online later and will send an e-scrum then [14:24:46] ok! [15:29:20] 10Analytics, 10Analytics-Wikistats: yearly folder in hdfs for mediacounts does not have write access for Wikistats by default (patched manually for earlier years) - https://phabricator.wikimedia.org/T173724#3538077 (10Ottomata) Hm. The proper thing to do would be to move the wikistats aggregation logic into t... [15:32:23] 10Analytics-Kanban: Troubleshoot Wikimetrics "magic button" - https://phabricator.wikimedia.org/T173585#3538100 (10Ottomata) [15:34:57] 10Analytics, 10Research, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3530850 (10Ottomata) FYI, we put this on the Analytics 'Radar' column, as I don't think there is active work for the Analytics tea... [15:35:50] 10Analytics, 10Project-Admins, 10Research, 10cloud-services-team: Create a phabricator project called "wikireplica-datasets" - https://phabricator.wikimedia.org/T173512#3530877 (10bd808) @Halfak can we start with this just as a column on the #Data-Services workboard? I've been debating making those columns... [15:44:32] 10Analytics, 10Research, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3530850 (10chasemp) @halfak and I spoke a bit about this this morning. We talked about a SIG for wikireplica things and how this... [15:50:32] 10Analytics, 10Research, 10cloud-services-team: Create a database on the wikireplica servers called "datasets_p" - https://phabricator.wikimedia.org/T173513#3538175 (10bd808) The ask here as I understand is to provide a database that is co-located with the wiki replicas available via the `wikireplica-web.eqi... [15:53:33] 10Analytics, 10Analytics-Wikistats: yearly folder in hdfs for mediacounts does not have write access for Wikistats by default (patched manually for earlier years) - https://phabricator.wikimedia.org/T173724#3538184 (10Erik_Zachte) that's fine with me, thanks right now I am updating all Wikistats scripts to mak... [15:57:20] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: Implement EventLogging Hive refinement - https://phabricator.wikimedia.org/T162610#3538223 (10Ottomata) @joal, FYI: I've been testing with the following commands: ====== eventlogging ``` spark-submit \ --master yarn \ --deploy-mode client \ -... [16:00:00] 10Analytics, 10Analytics-Wikistats: yearly folder in hdfs for mediacounts does not have write access for Wikistats by default (patched manually for earlier years) - https://phabricator.wikimedia.org/T173724#3538260 (10Erik_Zachte) 05Open>03Resolved [16:07:53] Can't hear you anymore mforns_ :( [16:08:46] mwarf :( [16:10:03] mforns_: Let's postpone and setup another meeting with fdans :) [17:26:35] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 2 others: Visualize page create events for all wikis - https://phabricator.wikimedia.org/T170850#3538933 (10kaldari) [17:26:35] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, 10Community-Tech-Sprint: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3538931 (10kaldari) 05Open>03Resolved @Ottomata: Awesome! Thank you! [18:07:30] 10Analytics-Kanban: Troubleshoot Wikimetrics "magic button" - https://phabricator.wikimedia.org/T173585#3539018 (10mforns) a:03mforns [18:07:58] 10Analytics-Kanban: Troubleshoot Wikimetrics "magic button" - https://phabricator.wikimedia.org/T173585#3533876 (10mforns) p:05Triage>03High [18:10:10] 10Analytics, 10Operations, 10Ops-Access-Requests, 10Research, 10Research-collaborations: NDA, MOU and LDAP (analytics cluster) for Shilad Sen - https://phabricator.wikimedia.org/T171988#3539028 (10RobH) a:05RobH>03herron So @herron is on clinic duty this week, and expressed an interest in taking care... [18:22:07] ottomata: Hi [18:33:35] hiii [18:33:43] getting close to eclipse time! :o [18:33:55] hehe :) [18:34:07] two questions for you ottomata [18:34:23] 1 - Can I try to restart tranquility in my screen for banners? [18:34:42] 2 - Should I expect to see new druid historical hosts soon ( I have seen your merged patches :) [18:35:34] you can try tranquility, but you can't keep it running [18:35:37] we had to turn it off [18:35:40] since it wasn't productionized [18:35:49] joal: yes soon [18:35:58] luca has refactored some puppet stuff, and a couple of things are weird [18:36:00] so getting it together [18:36:03] :) [18:36:07] great ottomata :) [18:36:36] ottomata: another interesting info: the spark banner job is running without failure since 8 weeks - I think we have better settings now ;) [18:37:05] ottomata: for the 3 new nodes, I assume we don't need coordinator ? [18:38:44] ottomata: I'm gonna try to get tranquility in spark working (I got ping by Jo Seddon about faster data) [18:48:01] joal: aweseooome :) [18:59:45] joal: is there a reason to not run coordinators? [18:59:47] easier to just do so [19:00:09] ottomata: overkill I think, but nothing major [19:00:32] ottomata: you can go with everything everywhere (even if it's a bit overkill) [19:01:16] joal: i'll just do everything everywhere for now, if we ever add more nodes, then we'll consider limiting [19:01:31] ottomata: sounds good [19:11:31] ottomata: enjoy the eclipse :) [19:11:39] leaving for today a-team, see you tomorrow ! [19:11:51] byeeee joal [19:12:07] ottomata: We haz 6 druidz :D [19:13:32] ohhh yaaaa [19:15:52] 10Analytics-Cluster, 10Analytics-Kanban, 10Operations, 10Patch-For-Review: rack/setup/install druid100[456].eqiad.wmnet - https://phabricator.wikimedia.org/T171626#3539353 (10Ottomata) [20:08:17] 10Analytics, 10Research, 10cloud-services-team (Kanban): Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3530850 (10bd808) @Halfak can we close the 3 sub-tasks you forked from this and have the discussion before we jump to a specific s... [21:08:44] 10Analytics-Kanban: Troubleshoot Wikimetrics "magic button" - https://phabricator.wikimedia.org/T173585#3539637 (10mforns) @Marostegui Hi! I'm looking into this problem and it looks very much like the one Dan was fixing a couple days ago in: https://phabricator.wikimedia.org/T173062 Wikimetrics raises the same... [22:13:27] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad, and 2 others: rack/setup/install new kafka nodes kafka-jumbo100[1-6] - https://phabricator.wikimedia.org/T167992#3539838 (10Cmjohnson)