[00:21:48] 06Analytics-Kanban, 06Performance-Team, 13Patch-For-Review: Update webperf EventLogging consumers for userAgent schema change - https://phabricator.wikimedia.org/T156760#3082525 (10Krinkle) a:05Krinkle>03Nuria [00:32:34] ottomata: nuria: I'm trying to understand the current eventlogging consumer in context of https://phabricator.wikimedia.org/T131977 [00:32:50] it seems both navtiming and ve use the same --endpoint, but one zmq and one eventlogging.connect(). [00:33:16] I assumed the latter would be kafka since that is recommended, but I can't quite tell from looking through https://github.com/wikimedia/eventlogging what connect() actually does. [00:33:37] EventConsumer doesn't seem to actually connect to anything directly. Not sure.. [00:36:16] 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 13Patch-For-Review, and 4 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3082571 (10mobrovac) 05Open>03Resolved Deployed, resolving [00:43:04] * Krinkle replies on ticket :) [00:43:52] 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 13Patch-For-Review, and 4 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3082619 (10Jdlrobson) Note, until the period parameter is available this is likely to render only things e... [00:44:31] 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 13Patch-For-Review, and 4 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3082622 (10Pchelolo) Yup, making a PR as we speak. [00:49:26] 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 13Patch-For-Review, and 4 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3082652 (10Jdlrobson) If all works correctly I'd expect to see `2017 BNP Paribas Open – Men's Singles` in... [00:52:29] 10Analytics, 10Analytics-EventLogging, 06Performance-Team: Stop using global eventlogging install on hafnium (and any other eventlogging lib user) - https://phabricator.wikimedia.org/T131977#3082671 (10Krinkle) >>! In T131977, @Ottomata wrote: > Currently hafnium runs /srv/webperf/ve.py which includes eventl... [00:58:31] 10Analytics, 10Analytics-EventLogging, 06Performance-Team: Make webperf eventlogging consumers use eventlogging on Kafka - https://phabricator.wikimedia.org/T110903#3082684 (10Krinkle) [00:59:30] 10Analytics, 10Analytics-EventLogging, 06Performance-Team: Make webperf eventlogging consumers use eventlogging on Kafka - https://phabricator.wikimedia.org/T110903#1589603 (10Krinkle) a:05ori>03Krinkle [01:45:43] 06Analytics-Kanban, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3082890 (10Nuria) [01:47:15] 06Analytics-Kanban, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3080590 (10Nuria) 05Resolved>03Open [04:38:12] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 13Patch-For-Review: Stop using global eventlogging install on hafnium (and any other eventlogging lib user) - https://phabricator.wikimedia.org/T131977#2184882 (10Nuria) >The puppet config for ve.py (uses global eventlogging) and navtiming.py (uses... [08:36:43] Hi a-team [08:37:02] I realized I made a mistake while merging patches for aqs yesterday [08:37:30] One patch got merged that was 1- not needed yet, 2- not ready yet [08:38:11] As long as we don't restart cassandra oozie job, there'll be no impact, so I suggest not rollbacking, but I'd like to have team opinion [08:39:56] hello! [08:40:07] what happens if we need to restart the cassandra oozie job? [08:40:32] I can see myself in the morning checking the jobs, seeing an oozia alarm and hitting "Restart" without thinking about it :D [08:56:22] (03PS10) 10Joal: [WIP] Port standard metrics to reconstructed history [analytics/refinery] - 10https://gerrit.wikimedia.org/r/322103 (owner: 10Milimetric) [08:56:49] Hi elukey :) [08:57:00] elukey: Re-running will not trigger any error [08:58:23] elukey: code has not been deplpoyed (in refinery), and re-run (even if deployed) involves versionned hdfs refinery folder - So the only problem that could happen is: on a newly deployed refinery, kill cassandra bundle and restart it [08:58:53] elukey: makes sense? [09:00:39] joal: ahhhh okok now I feel ok [09:00:45] +1 to keep it [09:01:02] (thanks for the explanation) [09:03:37] np elukey :) [09:15:25] now I just realized that yesterday I forgot to check an important thing [09:15:44] namely if partman does not touch disks during partitioning that are not listed [09:15:48] namely using keep [09:15:50] I believe it does [09:15:58] so this would allow us to save data [09:17:21] now that I think about it, yesterday on disk had like ~2TB of disk used and I thought it was weird.. because I was following the install guide, namely re-creating partitions [09:17:35] but I am pretty sure those were ok [09:17:36] sigh [09:17:49] I am tempted to reimage another noe [09:17:51] *node [09:20:35] joal: anything against me re-imaging an1041/ [09:20:36] ? [09:22:10] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 15User-Elukey: Reimage a Trusty Hadoop worker to Debian jessie - https://phabricator.wikimedia.org/T159530#3083379 (10elukey) [09:27:07] (03PS3) 10Joal: [WIP] Add oozie jobs for mw history denormalized [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341030 [09:27:21] elukey: nothing against, please do :) [09:27:37] elukey: I have not monitored an1040 - everything good I assume [09:27:52] joal: yep metrics are fine, and I haven't see issues in the logs for the moment [09:29:56] elukey: just checked quickly, llok good :) [09:30:25] elukey: just checked as well hadoop board - Have we done something to namenode yesterday? [09:30:52] elukey: Ah - Actually, normal regular pattern, old GC [09:33:59] elukey: When looking at 30 days in hadoop mtrics, the only one that really changes are nodemanager ones - Looks like bug is indeed corrected :D [09:34:22] * joal claps for not having to do regular restarts except for java upgrade :) [09:35:14] a-team - need to go for doctor appointment - Should be back beginning afternoon [09:42:10] o/ [09:58:50] PROBLEM - Hadoop NodeManager on analytics1041 is CRITICAL: PROCS CRITICAL: 0 processes with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager [10:01:00] argh forgot to silence [10:02:29] SO is this the closest channel to google analytics on freenode? [10:07:08] erasmus: Hi, what do you mean? [10:07:25] I mean there's no google analytics channel. [10:07:43] this is all I came up with when searching with alis. [10:08:43] erasmus: This is the Wikimedia Analytics Team channel, so probably far from what you are looking for :) [10:08:54] I know :) [10:12:31] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review, 15User-Elukey: Reimage a Trusty Hadoop worker to Debian jessie - https://phabricator.wikimedia.org/T159530#3083512 (10ops-monitoring-bot) Script wmf_auto_reimage was launched by elukey on neodymium.eqiad.wmnet for hosts: ``` ['ana... [10:15:30] 10Analytics-Cluster, 06Analytics-Kanban, 15User-Elukey: Audit fstabs on Kafka and Hadoop nodes to use UUIDs instead of /dev paths - https://phabricator.wikimedia.org/T147879#3083519 (10elukey) Fixed some little mistakes and added documentation to https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/A... [10:35:36] 06Analytics-Kanban, 06Operations, 10netops, 15User-Elukey: Review ACLs for the Analytics VLAN - https://phabricator.wikimedia.org/T157435#3083678 (10elukey) [10:38:18] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review, 15User-Elukey: Reimage a Trusty Hadoop worker to Debian jessie - https://phabricator.wikimedia.org/T159530#3083687 (10ops-monitoring-bot) Completed auto-reimage of hosts: ``` ['analytics1041.eqiad.wmnet'] ``` and were **ALL** suc... [11:01:26] hi team :] [11:03:18] o/ [11:04:00] o/ [11:06:21] (03PS3) 10Mforns: Add oozie workflow to load projectcounts to AQS [analytics/refinery] - 10https://gerrit.wikimedia.org/r/339421 (https://phabricator.wikimedia.org/T156388) [11:25:10] RECOVERY - Hadoop NodeManager on analytics1041 is OK: PROCS OK: 1 process with command name java, args org.apache.hadoop.yarn.server.nodemanager.NodeManager [11:46:47] analytics1041 up and running with debian! [11:46:54] I think I have a procedure to preserve data [11:46:59] but it is a bit hacky [11:47:09] I just started the datanode on 1041 [11:47:25] and it is working fine, atm deleting a lot of files (probably old/stale entries) [11:47:28] let's see how it goes [11:49:03] now I have entries like 2017-03-08 11:48:27,298 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving BP-1552854784-10.64.21.110-1405114489661:blk_1193849903_120181085 src: /10.64.53.25:49103 dest: /10.64.53.20:50010 [11:49:10] :) [12:03:46] * elukey lunch! [13:11:29] finally back, with again electrical issue at home [13:11:47] mforns: I have seen you have tried to rerun mediacount job [13:11:53] joal o/ [13:11:58] mforns: any idea what went wrong [13:12:00] ? [13:12:03] Hi joal [13:12:05] hi fdans [13:12:42] joal I'm running one last test query, are you ok with launching the beginning of time - march 1st job if that one goes well? [13:13:28] joal: I haven't investigated yet but it could have been me reimaging an1041 [13:14:14] elukey: I think mforns tried to rerun it, and it failed again (not cool) [13:14:31] joal, elukey, yes [13:14:43] fdans: If you're confident with your query and data, please go :) [13:14:53] fdans: have you tested loading some portins? [13:15:02] hmmm [13:16:40] do you have a couple mins to go about doing that with an oozie job, when the query finishes? [13:16:44] joal ^ [13:17:02] fdans: have looked at existing loading jobs? [13:17:09] have YOU looked sorry ? [13:17:23] elukey, mforns: Tried to rerun it once [13:17:51] yes, I was with that yesterday night [13:18:20] the wikitech doc about oozie jobs, I don't know who wrote it but it's FANTASTIC [13:18:48] fdans: Have you look at oozie loading jobs? [13:19:21] joal: you mean like this https://github.com/wikimedia/analytics-refinery/blob/master/oozie/cassandra/monthly/workflow.xml [13:24:54] joal, did you rerun it from hue button? [13:26:25] looks like it's fixed [13:26:50] mforns: yes, that's what I did [13:26:54] bizarre [13:27:12] fdans: I meqnt that indeed [13:27:33] ok, I wasn't sure we could use that button, so I run it with the usual oozie -run command, and it failed... dunno [13:28:27] mforns: hmm - do you have an history of that command you used? [13:30:13] joal, http://pastebin.com/ym0eWFEk [13:30:44] multiple things mforns [13:31:00] yes [13:31:04] oh this is clearing some things for me _watching_ [13:31:14] mforns: you should change the 2016 part of the subcommand extracting refinery âth [13:31:32] We now deploy in 2017 folders :) [13:31:39] oh yes, I knew that, but forgot [13:32:10] mforns: Second and more importantly, the command you launched runs a new coordinator, it doesn't RErun the existing failing job [13:32:32] of course [13:33:22] rerun button in hue does that, and in CLI, oozie job -rerun JOBID [13:33:31] mforns: --^ [13:33:38] sorry joal, internet hiccup [13:33:48] np mforns, was saying: [13:33:50] rerun button in hue does that, and in CLI, oozie job -rerun JOBID [13:33:53] last I read was " We now deploy in 2017 folders :)" [13:34:06] aha! [13:34:11] ah, so: mforns: Second and more importantly, the command you launched runs a new coordinator, it doesn't RErun the existing failing job [13:34:23] then the line pasted first ;° [13:34:34] yes, I expected it run a new temporaruy coordinator [13:34:48] fdans: so, having found existing loading code, what are your plans? [13:35:20] mforns: For failures like that we prefer to rerun them - easier to keep track of what has been succesfull [13:35:30] ok, makes sense joal thanks :] [13:35:37] np mforns :) [13:36:36] btw joal, if you have 5 mins now or later, can you have a look at the msck repair table error with me, see if it rings a bell, if not I'll continue looking into it [13:37:56] joal my take would be to copy the monthly job to a new directory and alter the properties so that it loads data from my location into the new keyspace [13:38:54] fdans: not bad :) There is a bit more to modify than just output keyspace (I suggest you review fields for instance), but this is the overall idea :) [13:39:01] mforns: sure, let's take a minute [13:39:31] ok [13:39:35] batcave? [13:40:52] sure mforns [13:41:22] (anyone has any idea of why my ssh session freezes every time a hive job finishes?) [13:42:06] fdans: are you using tmux/screen ? [13:42:15] nope [13:42:37] although this time it seems a connection hiccup, so never mind :) [13:42:57] yeah this is why I was suggesting tmux, it is great for all sort of ssh problems [13:54:06] elukey, heyyyy, can you help me with the metastore problem? [13:54:35] sure! [13:55:58] joal, one more thing, I saw there's this "Use v2 table in Cassandra, switch to padded day timestamp" change in refinery [13:55:59] should that be deployed this week? [13:55:59] if so, I can do it today [13:55:59] elukey, I can explain a bit: [13:57:35] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review, 15User-Elukey: Reimage a Trusty Hadoop worker to Debian jessie - https://phabricator.wikimedia.org/T159530#3084091 (10elukey) Summary of today. I reimaged analytics1041 with the analytics-flex.cfg partman recipe, that does not men... [13:57:39] we have this projectcounts-raw data in hdfs, that is divided in yearly folders, each folder is named like "year=2007", "year=2008", etc. [13:57:39] each folder has just 1 file (tsv, gzipped) [13:57:39] plus the _SUCCESS file [13:57:39] this structure is the same that hive uses for partitioned tables [13:57:40] now [13:57:41] I create an external table on top of that data and seems to work [13:59:39] but then I need to tell the metastore that the data is organized in yearly partitions, so I execute: [13:59:39] msck repair table projectcounts_raw [13:59:39] which is supposed to update the metastore recognizing the yearly partitions and enabling queries to the table [13:59:39] but it throws the following error: [13:59:40] FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask [13:59:41] the thing is... [13:59:41] I executed the same exact commands before my short vacation, and they worked [13:59:44] it looks like a metastore problem [14:00:07] mforns: deploy not needed, actually it's a mistake from me to have merged that patch that early [14:00:18] ok ok np [14:00:41] mforns: do you have the whole error message somewhere? [14:00:56] elukey, that's the whole error message :] [14:00:59] sigh [14:01:35] but maybe there's more logs I can look at, I looked for them but didn't find anything [14:02:02] maybe in metastore machine? analytics1003? [14:04:15] mforns: when was the last time that you executed it? [14:06:01] elukey, a couple minutes ago [14:06:01] like 15 minutes ago [14:06:01] I can execute it now [14:08:18] can't find anything relevant in the logs :/ [14:08:52] sorry elukey lost last 2 minutes (switched to other internet) [14:09:12] didn't write anything [14:09:16] k [14:09:53] mforns: is it possible to re-create the table? [14:10:06] elukey, sure [14:10:10] will do now [14:10:13] super [14:11:17] elukey, dropping... [14:11:23] done [14:11:46] elukey, creating... [14:11:52] done [14:12:24] elukey, now trying to msck repair... [14:12:34] done, with same error [14:14:44] fdans: are you doing ok? [14:17:29] sorry joal I continued working on EL while the query was running, so haven't done anything yet [14:17:38] fdans: no prob :) [14:18:15] fdans: While I'm pushing you to read the code base and try by yourself, I'm also trying to keep an eye on you in case ;) [14:18:42] ha, I appreciate that joal [14:19:24] fdans: If you feel the balance is not good in either way, please let me know :) [14:21:51] mforns: still nothing found.. [14:22:10] elukey: https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cm_mc_upgrade_to_cdh52_using_parcels.html#concept_r5x_wfx_jq__section_bk3_1d2_lq [14:22:29] section Upgrade the hive metastore database [14:22:49] it doesn't load [14:22:57] now it is [14:23:12] but we have 5.10 now [14:23:22] ? [14:23:52] elukey, didn't we upgrade to 5.5 last week? [14:24:36] nope 5.10 [14:24:44] and from https://etherpad.wikimedia.org/p/analytics-cdh5.10 Hive haven't changed [14:24:48] oh, but it says: CDH 5.3 to 5.4 or higher [14:24:55] *hasn't changed [14:25:09] I see [14:25:47] plus I think that the guide is for cloudera enterprise [14:25:53] oh.. [14:25:53] where they offer all the magic buttons [14:25:57] yes [14:26:00] :D [14:26:09] o.o [14:26:32] really weird though [14:26:38] we shouldn't have changed anything [14:27:19] did you see anything in the logs? [14:27:47] nope [14:27:48] I think I don't have permits to read them [14:27:51] k [14:28:30] mforns: I think that you can see them [14:28:41] shoud I sudo -u? [14:28:41] the other flag has read perms [14:28:44] no no [14:29:09] try less /var/log/hive/hive-server2.log [14:29:20] and /var/log/hive/hive-metastore.log [14:29:31] mforns: can you try to run again the command? [14:29:35] I am live tailing the logs [14:29:38] elukey, yes I can read [14:29:42] sure, one sec [14:29:53] done [14:31:56] grrr no logs at all [14:32:27] yep [14:32:29] goodmorninggg [14:32:41] hello! [14:32:50] maybe nothing to do with metastore! [14:33:08] I thought it was something related to the table [14:33:21] but I expected some logs :) [14:33:41] is there a place for hive query logs? [14:34:00] an1003 should be the place [14:35:10] mforns: can I run "msck repair table projectcounts_raw" ? [14:35:18] (I guess I need to be hdfs right?) [14:36:00] elukey, yes I'm running it as hdfs [14:36:15] sure, go ahead [14:36:18] on wmf database [14:39:23] AH! Found moar logs! [14:39:41] o////// [14:39:55] 2017-03-08 14:37:24,168 INFO ql.Driver (Driver.java:execute(1600)) - Executing command(queryId=hive_20170308143737_f0aeb879-3248-45cc-bd3a-7c0d4e9924ec): msck repair table projectcounts_raw [14:39:59] 2017-03-08 14:37:24,169 INFO ql.Driver (Driver.java:launchTask(1974)) - Starting task [Stage-0:DDL] in serial mode [14:40:02] 2017-03-08 14:37:24,181 WARN exec.DDLTask (DDLTask.java:msck(1779)) - Failed to run metacheck: [14:40:05] org.apache.hadoop.hive.ql.parse.SemanticException: No partition predicate found for Alias "org.apache.hadoop.hive.ql.metadata.HiveMetaStoreChecker@203f4429" Table "projectcounts_raw" [14:40:14] cool [14:40:18] mforns: I looked for the hive query id [14:40:32] in hive-server.log [14:40:38] ok [14:40:50] does the above error mean anything to you? [14:41:30] yes, sort of, but it's weird [14:41:36] ahahahah [14:41:47] because the table HAS partition predicate [14:42:01] you can check it by executing a show create table on it [14:42:59] but at least I have something to continue digging [14:44:06] No partition predicate found for Alias [14:44:08] it worked! [14:44:12] is it saying that it found a directory named alias? [14:44:17] in the partition hierarchy path? [14:44:22] it was because hive.mapred.mode=strict [14:44:54] ah... sigh.. sorry for bothering you ops with sth so non-ops [14:45:00] but the log message helped a lot [14:45:28] mforns: glad that you are unblocked :) [14:45:31] ottomata: hiiiiiiiiiiiiiiiiiiiiii [14:45:40] :] [14:45:43] I found a super hacky way to keep the partions when reimaging a node [14:46:29] oh ya? [14:46:34] elukey: i just thought of something too [14:46:41] we don't want to delete existing journalnode partiotions ! :o [14:46:53] i guess we could copy the content over to a datanode disk before we reinstall [14:47:05] or, move the journalnodes to jessie boxes once they are up, before we reinstall the trusty ones [14:48:07] good point, I didn't think about it! [14:48:18] ottomata: or we could grab a LVM snapshot and restore? [14:48:41] but I like the move journalnodes part [14:48:58] we could move the journal nodes as last step [14:49:03] before the hdfs master [14:49:12] we move them to jessie hosts [14:49:17] ensure that nothing explodes [14:49:21] reimage the old ones [14:49:56] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 13Patch-For-Review: Stop using global eventlogging install on hafnium (and any other eventlogging lib user) - https://phabricator.wikimedia.org/T131977#3084283 (10Ottomata) We maintain a Kafka -> ZMQ endpoint just for these processes :) If we can... [14:50:16] mforns: (whenever you have time) what is the use case for hive.mapred.mode=strict ? [14:52:52] elukey, strict mode doesn't let you do some things like cartesian joins or in this case... let the table access the respective partition folder in hdfs, second: http://stackoverflow.com/questions/39049620/no-partition-predicate-found-for-alias-even-when-the-partition-predicate-in-pres [14:53:02] 10Analytics, 10Analytics-EventLogging, 06Performance-Team, 13Patch-For-Review: Stop using global eventlogging install on hafnium (and any other eventlogging lib user) - https://phabricator.wikimedia.org/T131977#3084290 (10Ottomata) > Although PYTHONPATH must be /srv/deployment/eventlogging T118772 should... [14:53:13] so I set it to unstrict [14:53:20] just for this command [14:55:42] elukey: yeah, +1 [14:55:50] elukey: what's your hacky way to keep partitions? [14:57:02] joal: hmm, datasets.xml seems to expect a frequency, but my exported data has no such thing, it's just numbered [14:57:27] fdans: You probably don't need dataset.xml in your use case [14:57:56] riiight [14:58:26] fdans: you want to batcave for a minute? [14:58:38] that would be great joal [15:00:07] ottomata: https://phabricator.wikimedia.org/T159530#3084091 [15:02:54] 10Analytics, 10Analytics-EventLogging, 13Patch-For-Review, 10Scap (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#3084302 (10Ottomata) First create an eventlogging/scap/webperf repository in gerrit, and fill it with scap config informatio... [15:04:03] elukey: that sounds annoying but sane [15:04:04] we can do that [15:04:05] :) [15:37:14] joal: hi ops sync ( you can skip if you like) [15:40:00] ottomata: joining ! [15:52:07] fdans, if you are working on the new endpoint for legacy pageviews, can I pair? :] [15:52:35] mforns: of course, but I'm not currently with it [15:53:30] fdans, ok! let me know then when you're going to work on it [15:55:56] a-team: exterminator's here, sorry, will miss standup on short notice [15:56:19] np milimetric -- good luck with the exterminators ! [15:56:21] ok milimetric, good luck [15:56:48] milimetric'll be backh [15:56:56] my update: continuing to work on prototype, learning interesting things about Vue.js (check out their search function on their guide, it's awesome). Also met with language team for a while this morning, they have tons of data they're collecting to inform their work and I'll be working with them to port it to reportupdater. [15:57:09] lol [15:57:40] cool :] [16:01:40] a-team: standduppp [16:02:01] ottomata, milimetric , fdans: hola! [16:03:22] 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream: Create /schema/:schema endpoint in eventbus service to serve schemas by schema_uri - https://phabricator.wikimedia.org/T159179#3084477 (10Ottomata) [16:24:07] 10Analytics: Document and publicize AQS legacy pageviews - https://phabricator.wikimedia.org/T159959#3084517 (10mforns) [16:27:02] 06Analytics-Kanban, 10ChangeProp, 06Operations, 10Reading-Web-Trending-Service, 06Services (watching): Build and Install librdkafka 0.9.4 on SCB - https://phabricator.wikimedia.org/T159379#3084544 (10elukey) Yes I'd like to have the same version everywhere, we can coordinate with Traffic to roll out 0.9.... [16:31:07] meeting? cc elukey fdans [16:31:36] having problems with hangouts.. [16:31:39] Sorry, maybe i should not have included link,we are here: https://hangouts.google.com/hangouts/_/wikimedia.org/goals-checkup [16:31:52] mforns: batcave? [17:10:49] nuria: I just got back from working with pest control guy [17:10:57] apologies for missing the meetings [17:11:06] milimetric: np. [17:11:07] it was last minute so I sent an IRC ping [17:11:17] we are here: https://hangouts.google.com/hangouts/_/wikimedia.org/goals-checkup when ever you can [17:35:42] mforns: I'm going to SoS [17:35:50] (just re-confirming) [17:45:39] 10Analytics: Unlock Spark with Oozie - https://phabricator.wikimedia.org/T159961#3084689 (10JAllemandou) [17:47:26] nuria: do you have a minute? [17:47:32] joal: yesss [17:47:34] FINALLY [17:47:36] 10Analytics: Spike: Spark 2.x as cluster default (working with oozie) - https://phabricator.wikimedia.org/T159962#3084705 (10Ottomata) [17:47:45] 10Analytics: Spike: Spark 2.x as cluster default (working with oozie) - https://phabricator.wikimedia.org/T159962#3084705 (10Ottomata) Can we do it? How hard is it? [17:47:54] joal:baticueva? [17:48:02] nuria: sure ! La grotte ! [18:01:56] ottomata: I ma on hangout!!! [18:02:00] milimetric, will you be able to attend SoS with all the exterminating? I can do it if necessary (sorry if repeated message) [18:02:44] mforns: no prob, pest control is done for the day [18:02:53] k, thanks! [18:03:13] np [18:07:47] poor bohrium, I might have found something useful [18:08:02] my current theory is that piwik is bombarded by health checks [18:08:10] /o\ [18:09:02] a-team, I'm leaving for tonight, see you all tomorrow ! [18:09:09] bye joal ! [18:09:15] nite! [18:09:38] going afk too, tomorrow I'll tackle piwik! [18:09:39] byeee [18:09:40] o/ [18:17:33] me too! byeeeeee [19:29:24] 06Analytics-Kanban: Spike: Split unique devices data for Asiacell and non-Asiacell traffic in Iraq - https://phabricator.wikimedia.org/T158237#3085151 (10Nuria) [19:32:08] (03PS1) 10Nuria: Parqued Code - Asiacell modfications on unqiue devices [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341835 (https://phabricator.wikimedia.org/T158237) [19:32:45] 06Analytics-Kanban, 13Patch-For-Review: Spike: Split unique devices data for Asiacell and non-Asiacell traffic in Iraq - https://phabricator.wikimedia.org/T158237#3085167 (10Nuria) {F6341114} Asiacell unqiue devices since 2017-01-07 [19:33:33] 06Analytics-Kanban, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3085189 (10Nuria) 05Open>03Resolved [19:58:15] 10Analytics, 10Analytics-Cluster, 06Operations: Reinstall Analytics Hadoop Cluster with Debian Jessie - https://phabricator.wikimedia.org/T157807#3085345 (10Nuria) [19:58:17] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review: Move cloudera packages to a separate archive section - https://phabricator.wikimedia.org/T155726#3085344 (10Nuria) 05Open>03Resolved [19:58:55] 06Analytics-Kanban: Clean up datasets.wikimedia.org - https://phabricator.wikimedia.org/T125854#3085357 (10Nuria) 05Open>03Resolved [19:58:57] 06Analytics-Kanban, 13Patch-For-Review: Move datasets.wikimedia.org to analytics.wikimedia.org/datasets - https://phabricator.wikimedia.org/T132594#3085358 (10Nuria) [19:59:15] 10Analytics-Cluster, 06Analytics-Kanban, 15User-Elukey: Audit fstabs on Kafka and Hadoop nodes to use UUIDs instead of /dev paths - https://phabricator.wikimedia.org/T147879#3085371 (10Nuria) 05Open>03Resolved [19:59:28] 06Analytics-Kanban, 13Patch-For-Review: Bump default oozie launcher memory usage - https://phabricator.wikimedia.org/T159324#3085372 (10Nuria) 05Open>03Resolved [19:59:43] 10Analytics-Dashiki, 06Analytics-Kanban, 13Patch-For-Review: Add extension and category (ala Eventlogging) for DashikiConfigs - https://phabricator.wikimedia.org/T125403#3085373 (10Nuria) 05Open>03Resolved [20:00:22] 06Analytics-Kanban, 13Patch-For-Review: Fix description of webrequest table - https://phabricator.wikimedia.org/T157951#3085374 (10Nuria) 05Open>03Resolved [20:00:40] 10Analytics, 10Analytics-Cluster, 06Operations: Reinstall Analytics Hadoop Cluster with Debian Jessie - https://phabricator.wikimedia.org/T157807#3085380 (10Nuria) [20:00:44] 06Analytics-Kanban, 13Patch-For-Review: CDH 5.10 upgrade - https://phabricator.wikimedia.org/T152714#3085379 (10Nuria) 05Open>03Resolved [20:00:51] 06Analytics-Kanban: Create AQS endpoint to serve legacy pageviews - https://phabricator.wikimedia.org/T156391#3085389 (10MusikAnimal) [20:01:41] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review, 15User-Elukey: Update Zookeeper heap usage configuration and set alarms - https://phabricator.wikimedia.org/T157968#3085393 (10Nuria) [20:03:23] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 13Patch-For-Review, 15User-Elukey: Update Zookeeper heap usage configuration and set alarms - https://phabricator.wikimedia.org/T157968#3021877 (10Nuria) 05Open>03Resolved [20:03:41] 06Analytics-Kanban, 13Patch-For-Review: Run a 1-off sqoop over the new labsdb servers - https://phabricator.wikimedia.org/T155658#3085401 (10Nuria) 05Open>03Resolved [20:03:54] 06Analytics-Kanban, 13Patch-For-Review: Create EventStreams swagger spec docs endpoint - https://phabricator.wikimedia.org/T158066#3085402 (10Nuria) 05Open>03Resolved [20:03:57] 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream, 06Services (watching), 15User-mobrovac: EventStreams - https://phabricator.wikimedia.org/T130651#3085403 (10Nuria) [20:22:14] 10Analytics, 06Research-and-Data: geowiki data for Global Innovation Index - https://phabricator.wikimedia.org/T131889#3085445 (10Rafaesrey) Dear Leila, Hope you are doing fine. I write you to follow up on the data collection process. I also want to let you know that we now have an official launch date for th... [20:48:15] milimetric: how's your regex foo these days? :D [20:48:51] it's ok ottomata, what needs regexing [20:49:19] batcave? [20:49:33] omw [21:16:16] ottomata: yt? [21:16:19] msg ottomata [21:26:43] yo sorry was talking with dan [21:28:22] 10Analytics, 10Analytics-Cluster, 10EventBus, 10MediaWiki-Vagrant, 06Services (done): Kafka logs are not pruned on vagrant - https://phabricator.wikimedia.org/T158451#3085618 (10Pchelolo) 05Open>03Resolved [22:51:43] (03CR) 10Krinkle: Service Worker to cache locally AQS data (032 comments) [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/302755 (https://phabricator.wikimedia.org/T138647) (owner: 10Nuria) [22:52:38] 06Analytics-Kanban, 10ChangeProp, 06Operations, 10Reading-Web-Trending-Service, 06Services (watching): Upgrade librdkafka 0.9.4 on SCB and Varnishes - https://phabricator.wikimedia.org/T159379#3085869 (10Pchelolo)