[06:00:56] <mforns>	 hellooooo
[06:44:22] <elukey>	 hello super early mforns! :D
[06:46:10] <mforns>	 hello for once later than me elukey! :]
[06:48:20] <elukey>	 how is the weather like in India??
[06:48:22] <elukey>	 super hot?
[06:48:24] <mforns>	 I have to go to india to be earlier than you... hehehe
[06:48:28] <mforns>	 It's hot
[06:48:29] <elukey>	 ahahah
[06:48:37] <elukey>	 away
[06:48:41] <mforns>	 but not super hot
[06:48:56] <madhuvishy>	 more yoga mforns? :D
[06:48:57] <elukey>	 will you have time to travel around a bit?
[06:49:22] <mforns>	 yes madhuvishy, this year I'm practicing as well, or should I say suffering :]
[06:49:34] <madhuvishy>	 oooh that's awesome :D
[06:49:36] <madhuvishy>	 ha ha
[06:49:51] <mforns>	 we might go to bangalore the last week
[06:51:03] <madhuvishy>	 nice :) i guess i wouldn't know what to do in bangalore except eat and shop :)
[06:51:38] <madhuvishy>	 when you aren't stuck in traffic, or going to tech meetups
[06:53:43] <mforns>	 madhuvishy, back in 2007 we were a couple days in bangalore and it was cool, we went to the cinema to watch "om shanti om" (really crazy movie...)
[06:54:01] <madhuvishy>	 mforns: ha ha yes i've seen it :D
[06:55:14] <madhuvishy>	 and there are lots of better movies, thankfully!
[06:55:21] <mforns>	 hehe, sure :]
[06:57:55] <mforns>	 madhuvishy, I usually like indian movies, even bollywood ones, my favourite one is lagaan :D
[07:09:44] <madhuvishy>	 mforns: yes that's definitely a better one
[07:39:38] * elukey afk a bit (cat to the vet)
[09:53:48] * elukey is testing partitioning schemes on analytics1040
[09:53:51] * elukey hates Partman
[09:58:40] <mforns>	 hey team, will leave for a couple hours now, and be back 1:30h before standup
[11:30:27] <elukey>	 I think that I found the correct recipe for partman (an1040)
[11:30:48] <elukey>	 now I am going to run puppet and re-create all the partitions
[11:31:38] <elukey>	 we talked with Andrew to avoid wiping the datanode partitions for the other reimages
[11:31:55] <elukey>	 still need to find a good way to do it :)
[11:58:31] * elukey lunch!
[12:26:59] <elukey>	 ok now I am a bit confused
[12:27:16] <elukey>	 root@analytics1040:/var/lib/hadoop/data/l/hdfs/dn/current# ls -l
[12:27:16] <elukey>	 total 8
[12:27:16] <elukey>	 drwx------ 4 jmxtrans prometheus 4096 Feb 28 16:36 BP-1552854784-10.64.21.110-1405114489661
[12:27:19] <elukey>	 -rw-rw-r-- 1 jmxtrans prometheus  229 Feb 28 16:36 VERSION
[12:27:31] <elukey>	 and hdfs datanode fails to start due to perm issues..
[12:50:01] <elukey>	 !log restarted webrequest-load-wf-text-2017-3-7-9 from Hue (oozie id: 0010151-170228165458841-oozie-oozi-W mapred that failed: job_1488294419903_24496)
[12:50:03] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[12:50:13] <elukey>	 this one might have been me playing with analytics1040
[12:50:18] <elukey>	 from the logs it is not super clear
[12:52:42] <elukey>	 !log analytics1040 back in service
[12:52:43] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[13:01:09] <elukey>	 joal: an1040 looks good, first host up and running with Debian!
[13:04:35] <elukey>	 now let's watch it for the next two weeks and see how it behaves
[13:52:57] <joal>	 awesome elukey
[13:53:02] <joal>	 elukey: my congratulations
[13:53:32] <joal>	 elukey: one little trick with partman, then thing is up and running without trouble - looks good :) 
[14:02:56] <elukey>	 I encountered only some issues that might be due to a race in the first puppet run, or just me doing the wrong things :D
[14:02:59] <elukey>	 not sure
[14:04:43] <joal>	 fdans: are you nearby?
[14:05:06] <fdans>	 joal yep!
[14:06:27] <joal>	 fdans, elukey: is it aqs merge/deploy day?
[14:07:02] <elukey>	 it could be yes
[14:07:16] <joal>	 fdans: you're confortable with us merging and dpeloying?
[14:07:19] <fdans>	 oh yeah, I should merge that CR
[14:07:31] <fdans>	 please go ahead, mind if I shadow though?
[14:07:48] <joal>	 Please, let's do that
[14:08:02] <fdans>	 awesome, batcaverna?
[14:08:02] <joal>	 elukey, fdans: Merging the 2 CRs first
[14:08:32] <wikibugs_>	 (03CR) 10Joal: [V: 032 C: 032] "LGTM. Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/340093 (https://phabricator.wikimedia.org/T156312) (owner: 10Fdans)
[14:08:38] * elukey coffee first
[14:09:12] <wikibugs_>	 (03CR) 10Joal: [V: 032 C: 032] "LGTM. Merging for deploy." [analytics/aqs] - 10https://gerrit.wikimedia.org/r/339419 (https://phabricator.wikimedia.org/T156312) (owner: 10Fdans)
[14:10:02] <joal>	 Done
[14:10:16] <joal>	 elukey: Do we let you deploy, or do you want me to go for it?
[14:10:30] <joal>	 elukey: Ah sorry, had not seen
[14:10:40] <joal>	 elukey: Waiting for to be back from coffee :)
[14:13:07] <joal>	 fdans: Oh yes sorry
[14:13:09] <joal>	 Batcave !
[14:18:03] <elukey>	 joal,fdans - it is better if you guys do it, I'll be in standby if you need me
[14:18:18] <elukey>	 would it be ok?
[14:18:25] <joal>	 elukey !
[14:18:29] <joal>	 ok :)
[14:22:56] <ottomata>	 :)
[14:23:49] <elukey>	 ottomata: o/
[14:24:07] <elukey>	 analytics1040 seems to be up and running
[14:24:21] <ottomata>	 great!
[14:24:26] <elukey>	 I managed to create the "journal" LVM volume without any partition mounted on it
[14:24:36] <elukey>	 but it takes ~160GB
[14:25:01] <elukey>	 let me know if it could work
[14:25:08] <elukey>	 otherwise we might need to find a different solution
[14:26:01] <ottomata>	 i mean, i think it is very very unlikely that journal will ever get close to that
[14:26:15] <ottomata>	 but, we don't really need the space on the flex bay drives
[14:26:17] <ottomata>	 for anything else
[14:27:14] <elukey>	 yep yep
[14:27:44] <elukey>	 and root is not 60GB, /boot 1GB and swap 10G
[14:27:56] <elukey>	 sorry root *is* 60GB
[14:32:57] <ottomata>	 i thought we were going to get rid of swap?
[14:34:51] <elukey>	 yes I was oriented to get rid of it, but then I found a task in which Moritz mentioned https://lwn.net/Articles/690079/
[14:34:57] <elukey>	 that is a really interesting view of it
[14:35:02] <ottomata>	 oo reading
[14:35:03] <elukey>	 so I thought to keep it
[14:35:32] <elukey>	 but I am pretty sure we will not ever use it
[14:35:53] <wikibugs_>	 (03PS1) 10Joal: Update aqs to dd519b8 [analytics/aqs/deploy] - 10https://gerrit.wikimedia.org/r/341536
[14:37:57] <ottomata>	 yeah
[14:38:11] <ottomata>	 that article seems like it saying: swapping isn't so bad if you use fast drives!
[14:38:13] <ottomata>	 but yeah
[14:38:16] <ottomata>	 i agree that practically it won't matter for us
[14:38:20] <ottomata>	 i'm fine with whatever here really :)
[14:38:34] <ottomata>	 elukey:  160G journal is the rest of the free space on the drive?
[14:39:00] <elukey>	 well we have fast drives :)
[14:39:09] <elukey>	 yes if should
[14:39:22] <elukey>	 lvdisplay should show the breakdown
[14:39:27] <ottomata>	 oh ya looking
[14:39:50] <elukey>	 and pvdisplay shows 0 Free space
[14:39:53] <elukey>	 so it should be it
[14:40:23] <elukey>	 this was what I merged today https://gerrit.wikimedia.org/r/#/c/341337/4/modules/install_server/files/autoinstall/partman/analytics-flex.cfg
[14:40:41] <ottomata>	 oh!
[14:40:49] <ottomata>	 ' This volume might be used if the node will end up
[14:40:49] <ottomata>	 #  ** being a Journal node, but the steps to configure it will be done manually.'
[14:40:50] <ottomata>	 hm
[14:41:11] <ottomata>	 elukey:  you know, if we are doing manual steps anyway, maybe we should just make it the size we want?
[14:41:23] <ottomata>	 or
[14:41:49] <ottomata>	 we could do this, and then for actual journalnodes reduce the size of the lv before we use it
[14:41:53] <wikibugs_>	 (03CR) 10Fdans: [V: 032 C: 032] Update aqs to dd519b8 [analytics/aqs/deploy] - 10https://gerrit.wikimedia.org/r/341536 (owner: 10Joal)
[14:41:54] <ottomata>	 dunno
[14:43:24] <elukey>	 ottomata: the main problem is that partman takes the rest of the PV for the last logical volume
[14:43:38] <elukey>	 indipendently of what you put in there
[14:43:49] <elukey>	 the idea was to maybe resize it manually
[14:43:52] <elukey>	 or something similar
[14:44:18] <joal>	 !log Deploy AQS on beta
[14:44:19] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[14:45:43] <elukey>	 ottomata: I am up for discussion, I just wanted to find a working solution :)
[14:45:46] <joal>	 elukey: deploy failed on beta :(
[14:46:35] <elukey>	 that's good! Better than prod :)
[14:47:33] <joal>	 huhu
[14:47:45] <ottomata>	 hm yeah, hm.  elukey if there are manual steps, and we have to resize anyway...maybe we shouldn't call it journal, and just do the hack thing you mentioned
[14:47:54] <joal>	 elukey: fdans experienced this error before
[14:47:57] <ottomata>	 just make an lv called 'delete-me' or 'free-space' or something
[14:48:59] <wikibugs_>	 (03CR) 10Nikerabbit: [C: 032] Add new error types and abuse filter details printout [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/340982 (https://phabricator.wikimedia.org/T158834) (owner: 10Amire80)
[14:49:05] <wikibugs_>	 (03Merged) 10jenkins-bot: Add new error types and abuse filter details printout [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/340982 (https://phabricator.wikimedia.org/T158834) (owner: 10Amire80)
[14:50:09] <elukey>	 ottomata: we can do what we want
[14:50:10] <elukey>	 :)
[14:50:27] <elukey>	 even delete me
[14:50:51] <elukey>	 or maybe a manual step that deletes the journal logical volume and re-creates it
[14:51:10] <joal>	 elukey: error I got: https://gist.github.com/anonymous/a2a837e2563184c1d087a9172aad0883
[14:51:12] <ottomata>	 yeah, but it seems weird to have the journal partition everywhere if it is indenteded to be deleted
[14:51:16] <ottomata>	 if it is going to be used
[14:51:25] <joal>	 elukey: should it prevent us to deploy prod?
[14:51:54] <joal>	 elukey: Code gotcopied succesfully, but aqs didn't restart properly
[14:51:56] <elukey>	 ottomata: ?
[14:52:36] <elukey>	 joal: what is the deploy-log?
[14:52:43] <elukey>	 is it a scap issue?
[14:53:17] <elukey>	 ottomata: anyhow, the label for the partman recipe is one command so we can change it in 5 mins
[14:53:53] <ottomata>	 elukey:  yeah, i think we shoudl do that
[14:53:58] <ottomata>	 keep what you got, just change the label so we don
[14:54:01] <ottomata>	 don't confuse folks
[14:54:14] <ottomata>	 if they log into a non journal node, and see a 160G unmounted 'journal' partition
[14:54:23] <joal>	 elukey: https://gist.github.com/anonymous/49c05e2d3cb06a39ee190f6407c37b23
[14:54:24] <ottomata>	 and on a real jounalnode, a 10G mounted journal partition
[14:54:47] <ottomata>	 what should we call it?!
[14:55:12] <ottomata>	 'allocated-free-space-delete-me'
[14:55:13] <ottomata>	 ?
[14:55:13] <ottomata>	 haha
[14:56:07] <elukey>	 'unused-volume' ?
[14:56:19] <elukey>	 it is not mounted
[14:56:24] <elukey>	 we can call it batman
[14:56:33] <elukey>	 it doesn't really matter :)
[14:57:09] <elukey>	 joal: let me try something
[14:57:15] <joal>	 elukey: 
[14:57:17] <joal>	 sure
[14:57:29] <elukey>	 joal,fdans - I am going to remove the scap repo on aqs01 and re-deploy
[14:57:37] <ottomata>	 unused-volume +1 elukey
[14:57:40] <joal>	 elukey: please
[14:58:17] <joal>	 ottomata: Stop stealing elukey's attention ;) He's more interested in the stuff you tell him, but we still need his help :-P
[14:58:33] <ottomata>	 haha
[14:58:44] <ottomata>	 elukey's hammer is getting some good use
[14:59:14] <joal>	 I think he should use the Thor nicknqme
[14:59:15] <mforns>	 joal, helloooo
[14:59:18] <joal>	 Hi mforns 
[14:59:23] <joal>	 What's up?
[14:59:26] <mforns>	 hey :]
[15:00:06] <mforns>	 I can not find the table I created on top of projectcounts-raw dataset, do you know something?
[15:00:19] <elukey>	 it fails as well doing 14:59:39 Executing check 'depool'
[15:00:21] <elukey>	 I think
[15:02:25] <joal>	 elukey: :(
[15:02:31] <joal>	 mforns: no idea
[15:02:44] <mforns>	 joal, ok :[
[15:03:01] <joal>	 mforns: $I don't think we've experienced this type of issues after upgrade :(
[15:03:38] <mforns>	 joal, also I'm having problems executing 
[15:03:40] <mforns>	 msck repair table
[15:03:47] <ottomata>	 elukey: i'm ready to make a new journalnode
[15:03:54] <ottomata>	 i can proceed, or wait for you if you like too
[15:04:07] <elukey>	 ottomata: I am fixing aqs01 but I can follow you! 
[15:04:20] <ottomata>	 ok
[15:04:57] <fdans>	 elukey is the hero we all need but don't deserve
[15:05:31] <elukey>	 ahahhaha
[15:05:31] <ottomata>	 elukey:  current status is an28 datanode and nodemanager are up, but jn is down, right?
[15:05:45] <elukey>	 ottomata: nope, only datanode is down
[15:06:09] <elukey>	 fdans, joal - now it works, I just deployed - no idea why
[15:06:14] <joal>	 elukey: can we help in any way?
[15:06:20] <joal>	 okey
[15:06:23] <joal>	 weirdooh
[15:06:46] <elukey>	 ottomata: /etc/init.d/hadoop-hdfs-datanode is exit 0
[15:06:50] <elukey>	 the hack that I did
[15:07:00] <elukey>	 we'll need to re-install the pkg
[15:07:01] <ottomata>	 elukey:  on an28?
[15:07:03] <elukey>	 yeah
[15:07:21] <elukey>	 afaics the journal node was not using hdfs
[15:07:24] <ottomata>	 oh because the disk that is broken is the datnode disk?
[15:07:28] <elukey>	 yes!
[15:07:29] <ottomata>	 oh
[15:07:34] <ottomata>	 we don't have to move journalnode then!
[15:07:35] <ottomata>	 :)
[15:07:35] <ottomata>	 haha
[15:07:41] <elukey>	 ahahaha
[15:07:43] <ottomata>	 its up and running
[15:07:44] <ottomata>	 fine
[15:07:50] <ottomata>	 we can wait for chris to fix a dataanode
[15:07:52] <elukey>	 this is what I was trying to tell you yesterday!
[15:07:52] <ottomata>	 cool
[15:07:54] <ottomata>	 haha
[15:07:55] <ottomata>	 i didn't get it!
[15:10:00] <joal>	 elukey: I'm sorry but we still have issues :(
[15:11:12] <elukey>	 joal: same error? I thought it could have re-appeared, but I think it is indipendent from your code.. we could ask to releng!
[15:11:42] <joal>	 elukey: the node processes running on deployment-aqs01 run the previous version of the code
[15:12:18] <joal>	 elukey: if you tell me to go and deploy to prod I'll do it, but I'm afraid, as it's not working as is in beta
[15:15:06] <elukey>	 well probably they don't get the restart functionality
[15:15:14] <elukey>	 or I haven't configured it in beta
[15:15:18] <elukey>	 let's restart aqs01
[15:15:29] <joal>	 elukey: sure !
[15:15:46] <elukey>	 now it is /srv/deployment/analytics/aqs/deploy-cache/revs/e0da1bded35e803731d0442a293300ef98c76a27/node_modules/service-runner/service-runner.js
[15:16:36] <joal>	 elukey: looks better !
[15:17:51] <joal>	 elukey: And now the manual test request works :)
[15:18:00] <joal>	 elukey: Thanks again super hammer man !
[15:18:17] <joal>	 elukey: You +1 us to deploy to prod?
[15:18:47] <elukey>	 I think that we are good
[15:18:55] <elukey>	 but
[15:18:58] <joal>	 Thanks mate, doing that
[15:19:04] <elukey>	 let's double check that we have canaries set up
[15:19:40] <elukey>	 yep aqs1004
[15:19:49] <elukey>	 so it will ask you confirmation after it
[15:19:52] <elukey>	 we can check on it
[15:20:25] <fdans>	 luca to aqs: "i won’t kill you, but i don’t have to save you."
[15:20:31] <fdans>	 cos he's still batman
[15:20:32] <joal>	 elukey: I'll surely wait to test canary before fully deploying
[15:20:56] <joal>	 !log deploying aqs in prod
[15:20:56] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[15:21:22] <elukey>	 fdans: ahahah
[15:21:30] <elukey>	 I am only learning by master ottomata
[15:25:34] <joal>	 elukey: deploy to canary successfull - AQS answers requests, cassandra new table is created
[15:25:41] <elukey>	 \o/
[15:25:43] <joal>	 elukey: Asking permission to continue full deploy
[15:25:54] <elukey>	 checking metrics 
[15:26:01] <joal>	 elukey: sure !
[15:26:35] <elukey>	 joal: all good!
[15:26:40] <joal>	 awesome
[15:26:44] <joal>	 moving forward
[15:29:59] <joal>	 Success elukey
[15:30:08] <joal>	 Thanks fdans for your support !!!
[15:30:44] <elukey>	 goooooood
[15:30:48] <elukey>	 congrats fdans!
[15:33:51] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1002 replacement - https://phabricator.wikimedia.org/T159838#3080331 (10Ottomata)
[15:34:15] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 06Research-and-Data, 10Research-management: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#2734568 (10Ottomata)
[15:34:18] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1002 replacement - https://phabricator.wikimedia.org/T159838#3080331 (10Ottomata)
[15:34:54] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 06Research-and-Data, 10Research-management: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#2734568 (10Ottomata)
[15:39:39] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1002 replacement - https://phabricator.wikimedia.org/T159838#3080355 (10Ottomata)
[15:40:48] <elukey>	 ottomata: https://gerrit.wikimedia.org/r/#/c/341553 - WDYT?
[15:41:09] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1002 replacement - https://phabricator.wikimedia.org/T159838#3080331 (10Ottomata)
[15:41:38] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1003 replacement - https://phabricator.wikimedia.org/T159839#3080357 (10Ottomata) p:05Triage>03Normal
[15:41:45] <wikibugs_>	 10Analytics, 10Analytics-Cluster, 06Operations, 10hardware-requests: EQIAD: stat1002 replacement - https://phabricator.wikimedia.org/T159838#3080375 (10Ottomata) p:05Triage>03Normal
[15:42:16] <ottomata>	 merged elukey :)
[15:42:17] <ottomata>	 thanks
[15:42:34] <elukey>	 super :)
[15:46:47] <moritzm>	 is there a timeline/plan/ticket for upgrading stat1002 to jessie? if so, could you add that to https://phabricator.wikimedia.org/T159392?
[15:47:41] <ottomata>	 moritzm: 
[15:47:45] <ottomata>	 i just made it!
[15:47:51] <ottomata>	 we are replacing stat1002
[15:48:04] <ottomata>	 and stat1003
[15:48:15] <moritzm>	 ok, thanks
[15:49:56] <wikibugs_>	 10Analytics, 06Operations, 06Performance-Team: Update jq to v1.4.0 or higher - https://phabricator.wikimedia.org/T159392#3080405 (10Ottomata) Oh prefect!  We will soon be replacing stat1002 (T159838) and stat1003 (T159839) with newer hardware.  When we do so, we will upgrade these to Jessie.
[15:50:10] <wikibugs_>	 10Analytics, 06Operations, 06Performance-Team: Update jq to v1.4.0 or higher - https://phabricator.wikimedia.org/T159392#3080410 (10Ottomata)
[15:50:34] <wikibugs_>	 10Analytics: Alarm on data quality issues - https://phabricator.wikimedia.org/T159840#3080411 (10Nuria)
[15:51:03] <elukey>	 ottomata: I create the task to move hue and camus away from an1027, not sure if you have seen it
[15:51:41] <ottomata>	 i did!
[16:00:37] <ottomata>	 trying to join...
[16:16:57] <wikibugs_>	 10Analytics: Global Unique Devices Counts - https://phabricator.wikimedia.org/T143927#3080526 (10ema)
[16:17:00] <wikibugs_>	 06Analytics-Kanban, 06Operations, 10Traffic, 13Patch-For-Review: Add global last-access cookie for top domain (*.wikipedia.org) - https://phabricator.wikimedia.org/T138027#3080524 (10ema) 05Open>03Resolved a:03ema
[16:34:07] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3080590 (10Niedzielski)
[17:10:04] <wikibugs_>	 (03PS21) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T141548)
[17:22:06] <joal>	 milimetric: a linute for me?
[17:22:46] <milimetric>	 joal: yes!
[17:22:53] <joal>	 batcave?
[17:23:42] <milimetric>	 fdans / mforns: ok, project plan
[17:24:01] <fdans>	 project plan!
[17:24:04] <milimetric>	 so I was saying: if I do the prototype myself, I can finish probably by the end of the week with the basic skeleton
[17:24:17] <milimetric>	 if I work on it with Marcel, probably a bit earlier
[17:24:42] <milimetric>	 so it's up to mforns really, I'm ok finishing it and syncing up on it next week
[17:24:49] <milimetric>	 that way mforns can finish up
[17:24:54] <milimetric>	 (with aqs work)
[17:25:39] <mforns>	 milimetric, I guess I have like 1 day to fix the problems I have with projectcounts, and also I'll need to sync with fdans (if he grabs the aqs new endpoint task)
[17:26:35] <fdans>	 mforns: let's talk after staff if you want about that (or tomorrow, of course, it's probably super late for you)
[17:26:37] <mforns>	 after those 2 things, I can spend some time with vue, also need to continue with reportcard migration
[17:26:56] <mforns>	 fdans, yes, after staff, bed awaits :]
[17:27:11] <fdans>	 haha
[17:27:20] * fdans braces for Japan
[17:28:03] <fdans>	 well actually if you have a min we can go to the baticueva now mforns 
[17:28:30] <mforns>	 fdans, yes, baticueva 2 though, 1 is used by dan and joseph
[17:28:40] <fdans>	 omw
[17:41:59] <ottomata>	 phewwweee https://wikileaks.org/ciav7p1/
[17:47:13] <joal>	 ottomata: They got a malware called HIVE !!!!!
[18:01:13] <milimetric>	 holy leaks
[18:01:15] <milimetric>	 that's insane
[18:35:25] <mforns>	 bye team, see you tomorrow!
[18:35:39] <milimetric>	 nite!
[18:35:56] <elukey>	 byeeee
[18:36:35] <fdans>	 elukey joal the invitation also extends to you both, each diverting at some point to their home country :D
[18:37:11] <fdans>	 joal is a dad so he has an excuse, but elukey I'm sure Marika would understand
[18:45:16] <joal>	 fdans: thanks for the invite :)
[18:49:21] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3080590 (10Nuria) This likely means events are not valid, see: https://wikitech.wikimedia.org/wiki/Analytics/EventLogging/TestingOnBetaCluster#Validated_events
[18:49:33] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081278 (10Nuria) Ping us if you need help.
[18:51:03] <nuria>	 joal: one question if you are there
[18:52:32] <wikibugs_>	 (03PS1) 10Joal: Update sqoop and namespace_map scripts for versioning [analytics/refinery] - 10https://gerrit.wikimedia.org/r/341586
[18:53:32] <joal>	 hey nuria 
[18:53:35] <joal>	 sure
[18:56:50] <joal>	 nuria: arf, look like I missed you
[19:00:32] <joal>	 gone for the moment a-team, will be back after diner for checking jobs
[19:00:42] <milimetric>	 bon apetit
[19:00:52] <joal>	 merci !
[19:27:59] <wikibugs_>	 10Analytics, 10ChangeProp, 06Operations, 10Reading-Web-Trending-Service, 06Services (watching): Build and Install librdkafka 0.9.4 on SCB - https://phabricator.wikimedia.org/T159379#3081445 (10Ottomata) Good news! I think we don't have to rebuild varnishkafka.  Quick test on MWV has varnishkafka working...
[19:34:22] <travis-ci>	 wikimedia/mediawiki-extensions-EventLogging#638 (wmf/1.29.0-wmf.15 - 838abb7 : Translation updater bot): The build has errored.
[19:34:22] <travis-ci>	 Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/wmf/1.29.0-wmf.15
[19:34:22] <travis-ci>	 Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/208713217
[19:34:38] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081498 (10Niedzielski) @nuria, events are not appearing in server-side-events.log. From the docs you mentioned:  > server-side-events.log: server side events...
[19:37:52] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081527 (10Nuria) @Niedzielski Take a closer look at docs, your events should appear in all-events if they are valid, I have edited a bit so they are more cle...
[19:37:56] <nuria>	 fdans: are you testing on EL beta?
[19:41:22] <fdans>	 nuria: not right now, I'm with aqs
[19:45:35] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081588 (10Niedzielski) @nuria, ah I was thinking that events must appear under client //and// server events before they appear under all events and that beca...
[19:55:04] <wikibugs_>	 06Analytics-Kanban, 10ChangeProp, 06Operations, 10Reading-Web-Trending-Service, 06Services (watching): Build and Install librdkafka 0.9.4 on SCB - https://phabricator.wikimedia.org/T159379#3081625 (10Ottomata)
[20:30:33] <wikibugs_>	 06Analytics-Kanban, 10ChangeProp, 06Operations, 10Reading-Web-Trending-Service, 06Services (watching): Build and Install librdkafka 0.9.4 on SCB - https://phabricator.wikimedia.org/T159379#3081776 (10Ottomata) @elukey, I've dpkg -i librdkafka 0.9.4 on cp1045 and restarted varnishkafka.  Let's let this ru...
[20:43:57] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081819 (10Nuria) @Niedzielski  working on this, there was some testing code deployed to eventlogging that i think it might have caused this.
[21:28:00] <nuria>	 msg madhuvishy 
[21:32:55] <wikibugs_>	 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 10Reading Epics (Trending Edits), and 3 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3081948 (10Fjalapeno) Thanks @mobrovac!
[21:42:12] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3081994 (10Nuria) I think things should be ok now.
[21:54:41] <wikibugs_>	 10Analytics, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Android development event logging broken - https://phabricator.wikimedia.org/T159845#3082015 (10Niedzielski) 05Open>03Resolved a:03Nuria {icon thumbs-up} @Nuria, thank you! Things appear to be working perfectly now! Assigning to you and...
[22:40:25] <Pchelolo>	 ottomata: discovered that our idea to get rid of `ensure_topics_exist` script wasn't such a bright idea :( https://phabricator.wikimedia.org/T159883
[23:57:30] <wikibugs_>	 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 10Reading Epics (Trending Edits), and 3 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#3082415 (10mobrovac)