[01:38:54] <wikibugs_>	 (03PS1) 10Shilad Sen: Spark job to create page ids viewed in each session [analytics/refinery/source] (nav-vectors) - 10https://gerrit.wikimedia.org/r/383761 (https://phabricator.wikimedia.org/T174796)
[01:52:56] <wikibugs_>	 (03Abandoned) 10Shilad Sen: WIP: Spark job to create page ids viewed in each session [analytics/refinery/source] (nav-vectors) - 10https://gerrit.wikimedia.org/r/381169 (https://phabricator.wikimedia.org/T174796) (owner: 10Shilad Sen)
[07:10:12] <fdans>	 moooorning a-team!
[07:48:19] <wikibugs_>	 (03PS2) 10Fdans: Add "report a bug" link to the footer [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/383597 (https://phabricator.wikimedia.org/T177642)
[07:48:52] <wikibugs_>	 (03CR) 10Fdans: [V: 032 C: 032] Add "report a bug" link to the footer [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/383597 (https://phabricator.wikimedia.org/T177642) (owner: 10Fdans)
[07:56:27] <wikibugs_>	 (03PS6) 10Fdans: Add stub of new contributing and content metrics [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/382659 (https://phabricator.wikimedia.org/T175268)
[08:32:15] <elukey>	 hola people!
[08:32:56] <elukey>	 joal: o/ - morning! I have been reviewing the code review to create the druid public cluster and I'd like to discuss with you some details about firewalling
[08:33:08] <elukey>	 just to make sure that we are not missing anything
[08:48:37] <phuedx>	 o/ you lovely analytics folk
[08:48:50] <phuedx>	 i have an hiveql question, if y'all are up for it ;)
[08:49:26] <phuedx>	 (it might boil down to plain sql question, if i'm just not looking at the problem correctly)
[08:53:50] <elukey>	 phuedx: I am not an expert so probably you'll need to wait a bit for others but you can write down your doubts in the meantime :)
[08:54:04] <phuedx>	 elukey: ta
[08:55:07] <phuedx>	 original query: https://phabricator.wikimedia.org/P6054
[08:56:26] <phuedx>	 i'd like to alter it to include just os_family, browser_family, count, and percentage of the total
[08:56:40] <phuedx>	 i'd usually do this with a subquery
[08:57:15] <phuedx>	 but hiveql doesn't seem to be able to handle using maps in group by expressions very well
[08:58:33] <phuedx>	 for instance: select query.user_agent_map["os_family"] from ( ) as query; won't execute as the column "os_family" apparently isn't included in the group by clause in the sub query
[08:58:42] <phuedx>	 (i'll put this in a paste to clarify what i mean)
[09:01:12] <phuedx>	 y'know what
[09:01:15] <phuedx>	 ...
[09:01:20] <phuedx>	 ignore the last 5 minutes
[09:01:25] <phuedx>	 *facepalm*
[09:06:59] <elukey>	 ???
[09:07:00] <elukey>	 :D
[09:13:41] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Alert user about adblocker preventing AQS requests - https://phabricator.wikimedia.org/T177491#3678910 (10fdans)
[09:13:55] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Alert user about adblocker preventing AQS requests - https://phabricator.wikimedia.org/T177491#3660684 (10fdans) a:03fdans
[09:15:02] <wikibugs_>	 (03PS1) 10Fdans: Add central notice component and detect adblock [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/383798 (https://phabricator.wikimedia.org/T177491)
[09:18:11] <phuedx>	 elukey: it might be an issue with my use of the round function ;)
[09:18:30] <phuedx>	 will comment w/ more detail after i run a couple more tiny queries
[09:18:59] <elukey>	 oook!
[09:55:47] * fdans goes out to get roast chicken for lunch 🐔🔥
[10:18:59] * elukey lunch too
[11:48:11] <wikibugs_>	 10Analytics-EventLogging, 10Analytics-Kanban, 10Page-Previews, 10Readers-Web-Backlog, and 5 others: EventLogging subscriber module in ready state but not sending tracked events - https://phabricator.wikimedia.org/T175918#3679229 (10phuedx) >>! In T175918#3665782, @Tbayer wrote: > It might also be valuable...
[12:05:34] <elukey>	 all right going to merge https://gerrit.wikimedia.org/r/#/c/380804 to create the druid public cluster
[12:05:50] <elukey>	 I disabled puppet on most of our nodes as precaution
[12:19:31] <elukey>	 all good, running puppet on the druid100[123] nodes now
[12:32:21] <elukey>	 Exception in thread "main" com.metamx.common.ISE: Extension [/usr/share/druid/extensions/druid-hdfs-storage-cdh] specified in "druid.extensions.loadList" didn't exist!?
[12:32:27] <elukey>	 this is on the new druid nodes
[12:32:30] <elukey>	 lemme silence them
[12:34:15] <elukey>	 probably /usr/local/bin/druid-hdfs-storage-cdh-link to blame
[12:35:30] <elukey>	 yesss
[12:37:47] <elukey>	 all daemons up on druid100[456]
[12:55:46] <elukey>	 the coordinators are weird, their http endpoint is not available and this is in the error log Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure
[12:56:24] <elukey>	 that is the one holding up info about segments
[12:56:26] <elukey>	 mmmm
[12:56:37] <elukey>	 that one is probably on analytics1003
[12:56:49] <elukey>	 that is firewalled :D
[12:57:08] <elukey>	 druid.metadata.storage.connector.host: analytics1003.eqiad.wmnet
[12:57:11] <elukey>	 yesss
[12:59:01] * elukey coffee
[13:22:54] * elukey tries to move role::analytics_cluster::database::meta to a profile
[13:25:47] <wikibugs_>	 10Analytics, 10Analytics-EventLogging, 10Cognate, 10Collaboration-Team-Triage, and 14 others: Possible WMF deployed extension PHP 7 issues - https://phabricator.wikimedia.org/T173850#3679584 (10Anomie)
[13:29:40] <elukey>	 shortcut to https://gerrit.wikimedia.org/r/#/c/383833/
[13:29:54] <elukey>	 question though - are multiple druid clusters able to share the same DB?
[13:30:21] <elukey>	 double checking on analytics1003 just to be sure
[13:31:08] <elukey>	 MariaDB [(none)]> show databases;
[13:31:08] <elukey>	 +--------------------+
[13:31:08] <elukey>	 | Database           |
[13:31:08] <elukey>	 +--------------------+
[13:31:08] <elukey>	 | druid              |
[13:31:10] <elukey>	 | druid_public_eqiad |
[13:31:13] <elukey>	 erc..
[13:31:47] <elukey>	 all right all good
[13:37:08] <elukey>	 just connected to the Druid coordinator's ui, the cluster looks good
[13:37:23] <elukey>	 no datasources, nodes druid100[456]
[13:48:25] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Create Druid public cluster such AQS can query druid public data - https://phabricator.wikimedia.org/T176223#3679670 (10elukey)
[14:00:13] <milimetric>	 phuedx: helping with SQL's my favorite, let me know if the query's still not working
[14:05:19] <phuedx>	 milimetric: i'll be back in 20 minutes (just picking up my son from school)
[14:05:30] <phuedx>	 then i'll wrack your brain!
[14:05:36] <milimetric>	 sweet :)
[14:23:36] <ottomata>	 o/  elukey is druid public all up and kickin?
[14:24:48] <elukey>	 yesssss
[14:25:29] <ottomata>	 OHH elukey i misunderstood the comment about the networks for druid
[14:25:29] <ottomata>	 +1 looks good
[14:25:40] <elukey>	 \o/
[14:25:51] <ottomata>	 yeehaw thank you!
[14:36:12] <phuedx>	 milimetric: yt?
[14:36:35] <phuedx>	 also, would you prefer your brain to be picked via hangouts or irc?
[14:36:43] <milimetric>	 phuedx: yep, here
[14:36:49] <phuedx>	 👍
[14:36:52] <milimetric>	 anyway you like phuedx 
[14:48:11] <elukey>	 !log disable httpfs access on analytics1001
[14:48:12] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[14:49:00] <elukey>	 for anybody reading this --^, it means that query via Hue are not working from now on.. We are going to work on it asap :)
[14:52:19] <ottomata>	 not query, just filebrowsing
[14:52:35] <elukey>	 ahhhh
[14:58:29] <elukey>	 yep just confirmed, I can't read tcpdumps
[15:00:17] <joal>	 Hi a-team, I'm so happy I've managed to make it it on time to standup !
[15:01:08] <nuria_>	 ping milimetric fdans 
[15:01:37] <joal>	 My apologizes elukey for not answering your pings :(
[15:01:48] <fdans>	 nuria_: not making it to standup! see escrum :)
[15:01:50] <elukey>	 joal: I am deeply offended Joseph :D
[15:01:57] <joal>	 :D
[15:40:26] <elukey>	 !log run kafka preferred-replica-election to allow kafka1013 to re-join the topic leaders
[15:40:28] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[15:40:29] <elukey>	 ottomata: --^
[15:40:38] <elukey>	 something happened a couple of days ago
[15:40:47] <ottomata>	 hm k
[15:41:32] <elukey>	 some interesting logs from kafka1013 (even not recent ones)
[15:41:32] <elukey>	 [2017-06-20 10:10:23,463] ERROR [Replica Manager on Broker 13]: Error processing fetch operation on partition [eventlogging-client-side,6] offset 516688223 (kafka.server.ReplicaManager)
[15:41:37] <elukey>	 java.lang.IllegalArgumentException: Attempt to read with a maximum offset (516688222) less than the start offset (516688223).
[15:41:52] <elukey>	 [2017-10-12 14:53:28,984] ERROR Processor got uncaught exception. (kafka.network.Processor)
[15:41:55] <elukey>	 java.lang.ArrayIndexOutOfBoundsException: 18
[15:42:10] <elukey>	 the second one might be a recent librdkafka doing weird things
[15:42:20] <ottomata>	 huh i wonder if that is what triggered those burrow alarms
[15:42:29] <elukey>	 I was about to say the same!
[15:45:56] <Shilad>	 joal / ottomata: I've updated my CL with a job to prune down sessions and eliminate rare pageviews.
[15:46:45] <joal>	 Thanks for letting us know Shilad
[15:47:00] <joal>	 Shilad: I must apologize, I have been sick last week, and now I'm trying to catch up
[15:47:04] <Shilad>	 I'm moving on to linking to Wikidata concepts now. For the moment I am using a script that will extract sitelinks and import them into HDFS.
[15:47:06] <joal>	 Shilad: I've not forgotten
[15:47:16] <Shilad>	 joal: No worries! I'm not blocking on anything.
[15:55:22] <halfak>	 ottomata, I'm looking at https://www.confluent.io/blog/messaging-single-source-truth/ and it looks like the author advocates using Kafka as primary event store. 
[15:55:40] <halfak>	 Seemed to me you didn't like the idea of Kafka keeping a complete, replayable history of events. 
[15:55:53] <halfak>	 (no worries if you don't want to think about this now) 
[15:56:47] <ottomata>	 halfak:  in meeting, but ya, we've never done anything like that
[15:56:55] <ottomata>	 if the data is small enough, keyed well, and using compacted topics
[15:56:59] <ottomata>	 i have no problem with it though
[15:57:27] <halfak>	 Seems like that's something considered here.  In the future, I want to talk to you about applying suppression events to an event log. 
[15:57:37] <halfak>	 I'll keep reading 
[15:58:16] <wikibugs_>	 (03CR) 10Nuria: Spark job to create page ids viewed in each session (033 comments) [analytics/refinery/source] (nav-vectors) - 10https://gerrit.wikimedia.org/r/383761 (https://phabricator.wikimedia.org/T174796) (owner: 10Shilad Sen)
[15:58:16] <ottomata>	 suppression events?  tombstones?   i've never used compacted topics so i feel like i understand them at about a 75% level
[15:58:56] <halfak>	 gotcha.  Suppression events as in an event comes through to suppress information from a past event.  We'll use these when writing out public dumps. 
[15:59:34] <ottomata>	 aye, yeah, i think that is called a 'tombstone' in kafka compated topics, but not sure
[15:59:57] <halfak>	 Gotcha.  
[16:00:51] <ottomata>	 "Kafka log compaction also allows for deletes. A message with a key and a null payload acts like a tombstone, a delete marker for that key. Tombstones get cleared after a period."
[16:00:54] <ottomata>	 http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html
[16:11:22] <wikibugs_>	 10Analytics, 10Analytics-Wikistats: Alpha release: Label breakdown categories and other non-obvious concepts - https://phabricator.wikimedia.org/T177950#3680111 (10Ottomata)
[16:11:50] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Alpha release: Label breakdown categories and other non-obvious concepts - https://phabricator.wikimedia.org/T177950#3675829 (10Ottomata)
[16:23:53] <wikibugs_>	 10Analytics, 10Analytics-Wikistats: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680164 (10Erik_Zachte)
[16:25:52] <wikibugs_>	 10Analytics, 10Analytics-Wikistats: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680179 (10Erik_Zachte)
[16:27:40] <wikibugs_>	 10Analytics, 10Analytics-Wikistats: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680164 (10Erik_Zachte) a:03Nuria
[16:29:39] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Alpha release: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680184 (10Ottomata)
[16:31:57] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680191 (10Erik_Zachte)
[16:34:46] <nuria_>	 milimetric: looking at erik's ticket
[16:34:55] <milimetric>	 nuria_: hangout?
[16:35:02] <nuria_>	 milimetric: k
[16:35:43] <nuria_>	 milimetric: omw to batcave
[16:36:35] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Alpha release: Label breakdown categories and other non-obvious concepts - https://phabricator.wikimedia.org/T177950#3680195 (10Nuria) a:03fdans
[16:37:10] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback - https://phabricator.wikimedia.org/T178084#3680198 (10Erik_Zachte) {F10163152} In general: can we have numbers right-align? E.g. https://stats.wikimedia.org/v2/#/am.wiktionary.org/reading/total-pageviews but also elsewhere.
[16:41:26] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback From Erik Z - https://phabricator.wikimedia.org/T178084#3680210 (10Nuria)
[16:41:38] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback From Erik Z - https://phabricator.wikimedia.org/T178084#3680211 (10Erik_Zachte) On some places I see K/M/B for thousand/million/billion. On other places k/m/b. Maybe make a general formatting routine for this (preferedly language sensitive).  C...
[16:46:52] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback From Erik Z - https://phabricator.wikimedia.org/T178084#3680235 (10Nuria) We should aim to fix all these and i leave up to @Milimetric whether he wants to tag this Alpha (1st release) or Beta (2nd release)  >That page is almost useless for a wi...
[16:49:17] <wikibugs_>	 10Analytics, 10Operations: rack/setup/install furud.codfw.wmnet - https://phabricator.wikimedia.org/T176506#3680268 (10herron)
[16:58:37] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats 2 UI feedback From Erik Z - https://phabricator.wikimedia.org/T178084#3680280 (10Erik_Zachte) {F10163223} No nitpicking this time. An y-axis that doesn't start at zero, in combination with no numbers along the y-axis makes it anybody's guess what the chart t...
[17:25:24] <elukey>	 halfak: hey there!
[17:25:44] <wikibugs_>	 10Analytics, 10Operations: rack/setup/install furud.codfw.wmnet - https://phabricator.wikimedia.org/T176506#3627866 (10faidon) 05Open>03Resolved This is all installed and in production for about a week now.
[17:25:56] <elukey>	 I am wondering if the data that you have to copy is on server that is available on the public internet (meaning public ipv4)
[17:36:45] <elukey>	 already solved, phab task updated :)
[17:36:53] * elukey off!
[18:28:33] <wikibugs_>	 (03PS10) 10Joal: Update mediawiki-history-reduced oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174)
[18:32:27] <joal>	 ok team, trying to index mediawiki_history_reduced onto our brand new prod cluster :)
[18:35:12] <wikibugs_>	 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2017-09-26 (1.31.0-wmf.1)), 10Patch-For-Review: Stop collecting Data for outdated schemas PageCreation, PageDeletion, PageMove,  PageRestoration. Archive tables on hdfs - https://phabricator.wikimedia.org/T171629#3680695 (10Nuria) 05Open>03Resolved
[18:35:29] <wikibugs_>	 10Analytics-Kanban, 10Patch-For-Review: Cleaning scheme for banner data _SUCCESS files - https://phabricator.wikimedia.org/T164497#3680696 (10Nuria) 05Open>03Resolved
[18:35:42] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Implement pageview metric  in Wikistats UI - https://phabricator.wikimedia.org/T163817#3680701 (10Nuria)
[18:35:44] <wikibugs_>	 10Analytics-Kanban, 10Analytics-Wikistats: Re-read Round 2 feedback on wikistats on mediawiki and make any critical items into tasks - https://phabricator.wikimedia.org/T167674#3680700 (10Nuria) 05Open>03Resolved
[19:16:42] <wikibugs_>	 10Analytics-Kanban: Archive tables to hadoop:  MobileWikiAppToCInteraction_10375484_15423246 and Edit_13457736_15423246 - https://phabricator.wikimedia.org/T177960#3680833 (10Nuria) 05Resolved>03Open
[19:20:49] <wikibugs_>	 10Analytics-Kanban: Archive tables to hadoop:  MobileWikiAppToCInteraction_10375484_15423246 and Edit_13457736_15423246 - https://phabricator.wikimedia.org/T177960#3680857 (10Nuria) a:03Nuria
[20:01:35] <nuria_>	 joal: can you pass me one of your superset urls
[20:01:42] <nuria_>	 joal: with monthly data? 
[20:01:58] <nuria_>	 joal: I think i am going to add a screenshot to QR
[20:02:31] <joal>	 nuria_: The one with bizarre projects, or other?
[20:03:03] <nuria_>	 joal: any really
[20:04:14] <joal>	 nuria_: 
[20:04:14] <joal>	 http://localhost:8088/superset/explore/druid/12/?form_data=%7B%22datasource%22%3A%2212__druid%22%2C%22viz_type%22%3A%22line%22%2C%22granularity%22%3A%22month%22%2C%22druid_time_origin%22%3Anull%2C%22since%22%3A%222+years+ago%22%2C%22until%22%3A%22now%22%2C%22metrics%22%3A%5B%22sum__uniques_estimate%22%5D%2C%22groupby%22%3A%5B%22project_family%22%5D%2C%22limit%22%3A25%2C%22timeseries_limit_metric%
[20:04:20] <joal>	 22%3Anull%2C%22order_desc%22%3Atrue%2C%22color_scheme%22%3A%22bnbColors%22%2C%22show_brush%22%3Afalse%2C%22show_legend%22%3Atrue%2C%22rich_tooltip%22%3Atrue%2C%22show_markers%22%3Afalse%2C%22line_interpolation%22%3A%22linear%22%2C%22contribution%22%3Afalse%2C%22x_axis_label%22%3A%22%22%2C%22bottom_margin%22%3A%22auto%22%2C%22x_axis_showminmax%22%3Atrue%2C%22x_axis_format%22%3A%22smart_date%22%2C%
[20:04:26] <joal>	 22y_axis_label%22%3A%22%22%2C%22left_margin%22%3A%22auto%22%2C%22y_axis_showminmax%22%3Atrue%2C%22y_log_scale%22%3Afalse%2C%22y_axis_format%22%3A%22.3s%22%2C%22y_axis_bounds%22%3A%5Bnull%2Cnull%5D%2C%22rolling_type%22%3A%22None%22%2C%22time_compare%22%3Anull%2C%22num_period_compare%22%3A%22%22%2C%22period_ratio_type%22%3A%22growth%22%2C%22resample_how%22%3Anull%2C%22resample_rule%22%3Anull%2C%22r
[20:04:32] <joal>	 esample_fillmethod%22%3Anull%2C%22annotation_layers%22%3A%5B%5D%2C%22filters%22%3A%5B%5D%2C%22having_filters%22%3A%5B%5D%7D
[20:04:35] <joal>	 Arf
[20:04:38] <joal>	 nuria_: https://gist.github.com/jobar/f129cd6360f10e95ec8e7fa2d0d6d83a
[20:04:47] <joal>	 nuria_: let me know if you want something else
[20:06:09] <nuria_>	 joal: is it still on druid1001?
[20:06:10] <nuria_>	 ssh -N druid1001.eqiad.wmnet -L 8088:druid1001.eqiad.wmnet:8088
[20:06:11] <nuria_>	 ?
[20:06:36] <joal>	 stat1004 nuria
[20:19:12] <wikibugs_>	 (03PS13) 10Joal: Add mediawiki-history-metrics endpoints [analytics/aqs] - 10https://gerrit.wikimedia.org/r/379227 (https://phabricator.wikimedia.org/T175805)
[20:19:16] <joal>	 Heya milimetric :)
[20:19:24] <joal>	 Is now a correct time for you?
[20:21:53] <nuria_>	 joal: added slide, always better to talk with visuals
[20:22:16] <joal>	 it definitely is nuria_ :)
[22:03:43] <wikibugs_>	 (03PS14) 10Joal: Add mediawiki-history-metrics endpoints [analytics/aqs] - 10https://gerrit.wikimedia.org/r/379227 (https://phabricator.wikimedia.org/T175805)
[22:03:55] <joal>	 Hey milimetric, I'm satisfied with that one :) --^
[22:06:04] <milimetric>	 ok, joal, will +2 after one more look
[22:07:56] <joal>	 milimetric: it took me longer because I realised there was a bug in test that preventing some tests to fail ...
[22:08:00] <joal>	 fixed now :)
[22:08:26] <milimetric>	 cool, ok
[22:23:47] <leila>	 ottomata: for tasks related to Operations (for example, T177889) should I ping anyone separately?
[22:23:47] <stashbot>	 T177889: Request public key change for a research fellow - https://phabricator.wikimedia.org/T177889
[23:15:13] <ottomata>	 leila:  tag with access-requests
[23:15:20] <ottomata>	 operations-access-requests maybe
[23:16:48] <leila>	 yuuuuup. thanks, ottomata
[23:54:39] <wikibugs_>	 10Analytics, 10cloud-services-team (Kanban): Remove logging from labs for schema https://meta.wikimedia.org/wiki/Schema:CommandInvocation - https://phabricator.wikimedia.org/T166712#3681708 (10bd808) a:05bd808>03None The code has been rolled out everywhere to stop sending events to the eventlogging beacon...