[07:04:27] <moritzm>	 good morning, I'm merging now https://gerrit.wikimedia.org/r/#/c/308576/ now, if anyone of you runs into rsync problems, please tell me (don't expect that, but just being cautious)
[07:09:15] <elukey>	 sure :)
[08:23:18] <joal>	 Hi elukey
[08:23:42] <elukey>	 o/
[08:25:22] <elukey>	 I have just logged in to yarn.wikimedia.org \o/
[08:25:28] <joal>	 :D
[08:25:35] <elukey>	 puppet is still running and access is only for ops
[08:25:42] <elukey>	 I'll extend it today
[08:25:45] <elukey>	 but it woooorksss
[08:26:19] * joal is so happy with the idea he won't have to think about tunnels for hadoop :)
[08:26:45] <joal>	 elukey: If not too complicated, could you add hdfs.wikimedia.org?
[08:27:16] <elukey>	 this is new for me
[08:27:22] <joal>	 elukey: currently I look at it through analytics1001:50070
[08:27:25] <elukey>	 is it from the HDFS namenode?
[08:27:33] <joal>	 yup
[08:27:42] <joal>	 Gives stats about hdfs and allows browsing
[08:27:43] <elukey>	 I didn't know it! Of course I'll do it
[08:27:55] <elukey>	 but it is a bit more complicated since it is a new domain
[08:28:11] <elukey>	 so TLS termination + redirect, certs, etc..
[08:28:20] <elukey>	 I'll open a phab task
[08:28:37] <joal>	 elukey: no rush, just since you're at it, I'd rather mention it :)
[08:28:58] <elukey>	 nono we need these tools for our day to day ops
[08:29:06] <joal>	 elukey: indeed :)
[08:30:16] <joal>	 elukey: another camus hdfs error yesterday :(
[08:30:22] <joal>	 elukey: we really need that patch
[08:33:22] <elukey>	 yeah
[08:41:25] <elukey>	 so I just took a look to the namenode's ui, nice! It does not contain tons of things but worth to expose
[08:41:41] <joal>	 elukey: I think so as well
[08:43:37] <wikibugs>	 Analytics, Spike: Spike: Evaluate alternatives to varnishkafka: varnishevents - https://phabricator.wikimedia.org/T138426#2613873 (elukey) Had a chat with @ema today and you are correct @Ottomata, the use case of the developers was:  varnishevent -> named pipe -> kafkacat -> kafka topic -> ELK indexer
[08:47:17] * elukey brb in 15 min
[08:48:39] <grrrit-wm>	 (PS2) Joal: Make camus partition checker more resilient [analytics/refinery/source] - https://gerrit.wikimedia.org/r/308532 (https://bugzilla.wikimedia.org/144716)
[08:50:02] <grrrit-wm>	 (CR) Joal: "Editted comments and commit message." (4 comments) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/308532 (https://bugzilla.wikimedia.org/144716) (owner: Joal)
[09:03:34] <grrrit-wm>	 (PS6) Joal: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - https://gerrit.wikimedia.org/r/307903 (owner: Milimetric)
[09:04:15] <grrrit-wm>	 (CR) Joal: "@milimetric: I didn't know Option("Blah!")!" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/307903 (owner: Milimetric)
[09:10:54] <elukey>	 back
[09:12:21] <elukey>	 joal: if you have time, could you please check https://yarn.wikimedia.org? you shouldn't be able to login atm
[09:12:32] <elukey>	 (triple checking)
[09:13:14] <joal>	 elukey: no logging for me:
[09:13:20] <joal>	 elukey: :( but :)
[09:15:06] <elukey>	 gooood!
[09:30:41] <elukey>	 joal: https://wikitech.wikimedia.org/wiki/LDAP_Groups - do you think that wmf/nda should be allowed to see yarn?
[09:30:44] <elukey>	 I guess so
[09:30:57] <joal>	 elukey: hm ... I don't know
[09:31:10] <elukey>	 they already see tons of things :D
[10:12:36] <elukey>	 joal: afaik there is no way to mess up with the hadoop cluster from yarn.w.o right?
[10:12:42] <elukey>	 I mean, killing jobs, etc..
[10:13:21] <joal>	 elukey: I think you're right
[10:13:38] <joal>	 elukey: It just provides personnal info because you can read hive queries running
[10:14:50] <elukey>	 I think it is safe to be opened to wmf personel
[10:15:29] <joal>	 k
[10:27:20] <elukey>	 joal: would you mind to check yarn.w.o again?
[10:27:25] <elukey>	 you should be able to login now
[10:28:05] <joal>	 elukey: WORKS !
[10:28:17] <elukey>	 nice :)
[10:28:24] * joal is so happy a little tear is slowly running my cheek
[10:28:35] <elukey>	 you had to login right?
[10:28:39] * elukey paranoid
[10:28:40] <joal>	 yessir
[10:28:42] <elukey>	 gooooood
[10:28:53] <elukey>	 all right I am going to send an email to analytics-internal
[10:29:00] <elukey>	 to see if we want also nda
[10:29:07] <elukey>	 then maybe a mail to analyitcs@ ?
[10:29:14] <elukey>	 so everybody will be able to check their jobs
[10:29:19] <joal>	 sounds god
[10:29:23] <elukey>	 super
[11:07:17] <mforns>	 hi team!
[11:07:50] <joal>	 Hey mforns !
[11:07:59] <mforns>	 hey joal :]
[11:09:28] <joal>	 mforns: How was the conf?
[11:10:37] <mforns>	 joal, it was cool! a couple very interesting talks, some good ones and a couple so-so/terrible ones :]
[11:10:46] <joal>	 :)
[11:10:53] <mforns>	 I'll make a summary email
[11:12:50] <elukey>	 mforns: o/
[11:12:55] <mforns>	 hi elukey :]
[11:24:25] <joal>	 taking a break a-team,
[11:24:27] <joal>	 later
[11:24:30] <mforns>	 k
[11:51:36] <grrrit-wm>	 (PS1) Addshore: Use ^ and $ while spliting metric value and type [analytics/statsv] - https://gerrit.wikimedia.org/r/308959
[12:59:30] <grrrit-wm>	 (CR) Ottomata: "Seems fine to me, is there a reason or maybe even a phab ticket that explains the need for this?" [analytics/statsv] - https://gerrit.wikimedia.org/r/308959 (owner: Addshore)
[13:01:46] <grrrit-wm>	 (PS2) Addshore: Use ^ and $ while spliting metric value and type [analytics/statsv] - https://gerrit.wikimedia.org/r/308959
[13:01:51] <addshore>	 ottomata: ^^
[13:07:03] <grrrit-wm>	 (CR) Ottomata: [C: 1] "I can merge and deploy this, but it might be better to get a service owner from Performance to deploy." [analytics/statsv] - https://gerrit.wikimedia.org/r/308959 (owner: Addshore)
[13:14:12] <grrrit-wm>	 (PS1) Mforns: Add re-run script [analytics/reportupdater] - https://gerrit.wikimedia.org/r/308977 (https://phabricator.wikimedia.org/T117538)
[13:49:16] <grrrit-wm>	 (PS2) Mforns: Add re-run script [analytics/reportupdater] - https://gerrit.wikimedia.org/r/308977 (https://phabricator.wikimedia.org/T117538)
[13:59:46] <joal>	 hey ottomata, do you think you'd have time to merge/deploy the camus-checker patch?
[14:08:53] <ottomata>	 joal:  heyyaa, yeah let's check it out now...let's see, we are going to try to use the new archiva, so I have to cahnge the jenkins stuff, or just the DNS totally, hmm.>..>.>
[14:08:54] <ottomata>	 hm
[14:08:55] <ottomata>	 hm
[14:08:55] <ottomata>	 hm
[14:09:14] <ottomata>	  have lots of meetings today, so if we can't do it before standup, i prob won't have time til late afternoon
[14:09:16] <joal>	 ottomata: I'm here, let me know If ican help
[14:09:18] <ottomata>	 maybe we can prep today and do tomorrow?
[14:09:26] <joal>	 ottomata: Sounds good :)
[14:09:33] <ottomata>	 will spend the time before standup prepping...just a min
[14:15:25] <ottomata>	 joal:  so, this is the first i've come back to look at the new archiva
[14:15:31] <ottomata>	 i think it isn't working yet.  i haven't been able to log into it
[14:15:33] <ottomata>	 it takes too long...
[14:15:50] <joal>	 ottomata: I assume it won't work for jenkins either then
[14:17:32] <ottomata>	 yeah, well, hm.  maybe. logging in via the UI has always been realllly slow
[14:20:39] <ottomata>	 maybe we should revisit the ldap issue.  there are some new versions
[14:20:40] <ottomata>	 OH!
[14:20:43] <ottomata>	 it loged me in!
[14:20:44] <ottomata>	 succesfully!
[14:20:44] <ottomata>	 ok
[14:20:45] <ottomata>	 cool
[14:20:46] <ottomata>	 so that's good
[14:21:00] <joal>	 great :)
[14:21:29] <ottomata>	 ok, i say let's change full DNS now, if we do that, everything should be transparent to jenkins
[14:22:48] <joal>	 ottomata: Ok, I trust you :)
[14:23:03] <joal>	 ottomata: that also mean I can try to deploy myself :)
[14:23:09] <ottomata>	 ja but i've never done it
[14:23:12] <ottomata>	 with jenkins
[14:23:15] <ottomata>	 so it might be good for me to learn
[14:23:22] <joal>	 ottomata: ok, tomorrow together :)
[14:23:26] <ottomata>	 k
[14:33:22] <ottomata>	 joal:  gonna check with mutante later, and get him to do a final rsync before we switch DNS
[14:33:25] <ottomata>	 but all should be ready for tomorrow
[14:33:35] <joal>	 cool ottomata, thanks :)
[15:00:59] <nuria_>	 ottomata, joal: standdduppp
[15:01:08] <joal>	 joining
[15:33:18] <joal>	 ottomata: in scala meeting, do you need me in EventBus?
[15:33:31] <ottomata>	 joal:  naw prob ok, unless you want to talk about new schemas
[16:27:51] <grrrit-wm>	 (CR) Nuria: [C: 2] Make camus partition checker more resilient [analytics/refinery/source] - https://gerrit.wikimedia.org/r/308532 (https://bugzilla.wikimedia.org/144716) (owner: Joal)
[16:32:13] <grrrit-wm>	 (Merged) jenkins-bot: Make camus partition checker more resilient [analytics/refinery/source] - https://gerrit.wikimedia.org/r/308532 (https://bugzilla.wikimedia.org/144716) (owner: Joal)
[18:00:07] <joal>	 Logging off a-team, see you tomorrow
[18:01:37] <ottomata>	 seeyaa
[18:01:45] <ottomata>	 a-team if anyone wants to join credit demo
[18:01:47] <ottomata>	 details are here https://etherpad.wikimedia.org/p/CREDIT
[18:01:53] <ottomata>	 actually
[18:01:54] <ottomata>	 https://www.mediawiki.org/wiki/CREDIT_showcase
[18:01:54] <ottomata>	 better
[18:01:58] <milimetric>	 I'm watching ottomata
[18:02:09] <ottomata>	 i'm probably toward the end
[18:03:22] <mforns>	 thx ottomata
[18:49:15] <milimetric>	 nice ottomata
[18:51:13] <grrrit-wm>	 (PS3) Nuria: Reduce rate of events coming from datepicker [analytics/dashiki] - https://gerrit.wikimedia.org/r/308819 (https://phabricator.wikimedia.org/T144856)
[18:51:21] <ottomata>	 thanks!
[18:53:48] <ottomata>	 hm milimetric, mforns just making sure, do you know of this page?
[18:53:48] <ottomata>	 https://meta.wikimedia.org/wiki/Research:Wiki_archaeology
[18:53:53] <ottomata>	 maybe yall should add weird things you find there too
[18:54:02] <mforns>	 ottomata, oh!
[18:54:31] <milimetric>	 ottomata: yeah, aaron showed it to us
[18:54:34] <mforns>	 cool :]
[18:54:46] <milimetric>	 I did get some stuff out of there
[18:54:50] <mforns>	 I see
[18:55:15] <milimetric>	 I was planning on doing another run through when we have the vertical slice
[18:55:45] <grrrit-wm>	 (CR) Nuria: [C: -1] "Need to test throughly" [analytics/dashiki] - https://gerrit.wikimedia.org/r/308819 (https://phabricator.wikimedia.org/T144856) (owner: Nuria)
[18:55:47] <milimetric>	 my opinion is that this thing will never be perfect but it's built so improvements are easy
[19:06:17] <grrrit-wm>	 (PS4) Nuria: Reduce rate of events coming from datepicker [analytics/dashiki] - https://gerrit.wikimedia.org/r/308819 (https://phabricator.wikimedia.org/T144856)
[19:06:57] <grrrit-wm>	 (CR) Nuria: "Super sorry about my sloppiness with this patch" [analytics/dashiki] - https://gerrit.wikimedia.org/r/308819 (https://phabricator.wikimedia.org/T144856) (owner: Nuria)
[19:07:10] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Reduce rate of events coming from datepicker [analytics/dashiki] - https://gerrit.wikimedia.org/r/308819 (https://phabricator.wikimedia.org/T144856) (owner: Nuria)
[19:07:28] <milimetric>	 psh, np nuria_, I left it for you to submit when you want
[19:08:03] <nuria_>	 milimetric: will do now and deploy your patch, sorry again for my sloppiness with this patch
[19:08:33] <milimetric>	 all good
[19:11:32] <nuria_>	 milimetric: btw, deploying the analytics.wikimedia.org is done by deploying source as we used to deploy dashiki
[19:12:21] <milimetric>	 I've never deployed that, yeah, can I hang out with you while you do it?
[19:12:23] <nuria_>	 milimetric: with piwiki site id 8 , explained here: https://github.com/wikimedia/analytics.wikimedia.org/blob/master/README.md
[19:12:27] <milimetric>	 nuria_: ^
[19:12:49] <milimetric>	 oh you mean you build it and just copy it up?
[19:12:56] <milimetric>	 it doesn't use the dashiki-deploy?
[19:13:18] <nuria_>	 milimetric: yes, as there is no other way to deploy to prod (fab sshing directly to those machines does not work)
[19:13:28] <milimetric>	 aha, got it
[19:13:29] <milimetric>	 ok
[19:13:56] <nuria_>	 milimetric: no, it cannot as prod network doesn't let us access using our ssh keys which is what fab needs
[19:14:20] <nuria_>	 milimetric: other option will be scap which will do 5 more things and deploy from source anyways
[19:14:39] <milimetric>	 right, we can maybe look at scap later
[19:14:45] <milimetric>	 but this seems fine, it's not complicated
[19:20:22] <grrrit-wm>	 (PS1) Nuria: Fixes for browser dashboard [analytics/analytics.wikimedia.org] - https://gerrit.wikimedia.org/r/309076
[19:21:06] <grrrit-wm>	 (CR) Nuria: [C: 2 V: 2] Fixes for browser dashboard [analytics/analytics.wikimedia.org] - https://gerrit.wikimedia.org/r/309076 (owner: Nuria)
[19:22:38] <nuria_>	 milimetric: deployed: https://analytics.wikimedia.org/dashboards/browsers
[19:22:56] <milimetric>	 cool
[19:23:06] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Reduce rate of events coming from datepicker - https://phabricator.wikimedia.org/T144856#2612479 (Nuria) a:Nuria
[19:23:46] <wikibugs>	 Analytics-Kanban: Compile a request data set for caching research and tuning - https://phabricator.wikimedia.org/T128132#2616352 (Nuria) Ping @Danielsberger
[19:31:59] <wikibugs>	 Analytics-Kanban: Coalesce nulls to 0s in output - https://phabricator.wikimedia.org/T144521#2602510 (Nuria) a:Nuria
[21:02:56] <grrrit-wm>	 (PS9) Milimetric: Script sqooping mediawiki tables into hdfs [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476)
[21:10:01] <grrrit-wm>	 (PS10) Milimetric: Script sqooping mediawiki tables into hdfs [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476)
[21:28:32] <grrrit-wm>	 (PS11) Milimetric: Script sqooping mediawiki tables into hdfs [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476)
[21:39:20] <grrrit-wm>	 (PS12) Milimetric: Script sqooping mediawiki tables into hdfs [analytics/refinery] - https://gerrit.wikimedia.org/r/306292 (https://phabricator.wikimedia.org/T141476)
[21:47:52] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Productionize edit history extraction for all wikis using Sqoop - https://phabricator.wikimedia.org/T141476#2499954 (Milimetric) This is ready for review @Ottomata or @JAllemandou or @mforns
[23:08:38] <wikibugs>	 Analytics, Pageviews-API, WMUA-Tech: No page view stats for wikimedia chapters sites - https://phabricator.wikimedia.org/T145033#2617636 (intracer)
[23:22:46] <wikibugs>	 Analytics, Pageviews-API, Wikipedia-iOS-App-Backlog, iOS-app-v5.2.0-Honey: filter suspicious TV channels pageviews from Top Read - https://phabricator.wikimedia.org/T144333#2617810 (JMinor) a:JMinor Meant to create a PR but the GitHub web UI confused me, and I committed this straight to dev:...
[23:29:11] <wikibugs>	 Analytics: Top Pageview stats for August 27th doesn't look right - https://phabricator.wikimedia.org/T144715#2617831 (Tbayer) Here are the dates where the pageviews for the top three pages started to rise drastically, remaining on that high plateau since then:  [[https://tools.wmflabs.org/pageviews/?project=...
[23:38:37] <wikibugs>	 Analytics: Top Pageview stats for August 27th doesn't look right - https://phabricator.wikimedia.org/T144715#2617853 (JMinor) See T144333 for info on iOS take on this.