[14:01:22] <hashar>	 #startmeeting Weekly CI triage
[14:01:22] <wm-labs-meetbot`>	 Meeting started Tue Sep 15 14:01:22 2015 UTC and is due to finish in 60 minutes.  The chair is hashar. Information about MeetBot at http://wiki.debian.org/MeetBot.
[14:01:22] <wm-labs-meetbot`>	 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
[14:01:22] <wm-labs-meetbot`>	 The meeting name has been set to 'weekly_ci_triage'
[14:01:28] <hashar>	 #link https://plus.google.com/hangouts/_/wikimedia.org/ci-weekly Google hangouts
[14:01:32] <jzerebecki>	 o/
[14:01:34] <hashar>	 feel free to ask for an invite as needed
[14:02:00] <hashar>	 #link https://www.mediawiki.org/wiki/Continuous_integration_meetings/2015-09-15 Agenda
[14:02:55] <hashar>	 #link https://www.mediawiki.org/wiki/Continuous_integration_meetings/2015-09-08/Minutes Past week minutes
[14:03:08] <hashar>	 #topic Using composer for jobs
[14:03:56] <jzerebecki>	 #link https://phabricator.wikimedia.org/T90303
[14:04:56] <hashar>	 #link https://gerrit.wikimedia.org/r/#/c/238188/ related JJB config change
[14:04:57] <zeljkof-meeting>	 o/
[14:06:05] <hashar>	 mwext-qunit   --> uses vendor
[14:06:16] <hashar>	 mwext-qunit-composer -> use composer (obviously)
[14:06:28] <hashar>	 so you can add a branch filter for mwext-qunit to prevent it from running on wmf/* branches
[14:06:38] <hashar>	 and add a filter for mwext-qunit-composer to NOT run it on wmf/* branches
[14:15:49] <hashar>	 #action need a composer version of mediawiki-testextensions  which uses zuul-cloner and mediawiki/vendor
[14:16:38] <hashar>	 #agree on branch cut, the wmf branch jobs will start falling jobs  because vendor is not updated.  The composer deps there need to be updated and send for review against the wmf branch.   The jobs using vendor.git should then pass. Merge and all wmf branches are green again
[14:16:40] <hashar>	 #agreed on branch cut, the wmf branch jobs will start falling jobs  because vendor is not updated.  The composer deps there need to be updated and send for review against the wmf branch.   The jobs using vendor.git should then pass. Merge and all wmf branches are green again
[14:25:56] <hashar>	 point taken for the Releng weekly meeting today (a couple hours from now)
[14:25:57] <hashar>	 * Composer / mediawiki/vendor for Jenkins jobs
[14:25:57] <hashar>	 Jan is switching our jobs on master to use composer instead of vendor
[14:25:58] <hashar>	 The REL/wmf branches versions of the jobs use vendor ( https://phabricator.wikimedia.org/T90303 ).
[14:25:58] <hashar>	 Problem: when we cut the branch, the wmf jobs will fail until vendor.git is updated.
[14:26:00] <hashar>	 The branch cut needs a new step:
[14:26:00] <hashar>	     * update composer
[14:26:02] <hashar>	     * update vendor.git
[14:26:02] <hashar>	     * get tests to pass and +2
[14:26:04] <hashar>	     * REL/wmf branches based jobs pass again (cause vendor has been updated)
[14:26:04] <hashar>	 Q:
[14:26:06] <hashar>	 * Where is the branch cut doc/process?
[14:26:06] <hashar>	 * Gotta update the make-release  make-wmf-branches scripts
[14:26:08] <hashar>	 * automatically build and commit mediawiki/vendor (composer) https://phabricator.wikimedia.org/T101123
[14:26:08] <hashar>	 * can be done just when we cut the branch (Monday). Leave us a day to update vendor before first deployment (Tuesday)
[14:28:08] <hashar>	 #topic CI scaling
[14:28:20] <hashar>	 #info Nodepool booted it is first instance in prod
[14:32:53] <hashar>	 #info Nodepool pool instances with label ci-debian-jenkins
[14:36:55] <hashar>	 Given a job such as tox-py27-jessie which is triggered by multiple repos
[14:36:55] <hashar>	 Inside Zuul gearman server that defines several functions given the job has ci-jessie-debian and DebianJessie labels
[14:36:56] <hashar>	 tox-py27-jessie
[14:36:56] <hashar>	 tox-py27-jessie:ci-jessie-debian
[14:36:58] <hashar>	 tox-py27-jessie:DebianJessie
[14:36:58] <hashar>	 Then we can use a Zuul function to switch on a per project basis:
[14:37:00] <hashar>	 if project == integration/config
[14:37:00] <hashar>	 then point to ci-jessie-debian AND NODE_OFFLINE_WHEN_COMPLETE=1
[14:37:02] <hashar>	 else
[14:37:02] <hashar>	 point to DebianJessie
[14:37:48] <hashar>	 so in short: we are almost there
[14:50:33] <hashar>	 #agreed will tune nodepool instances flavor as time goes.  Probably ends up with 3 profiles:  light (1CPU/2GB ram),  performance (4CPU/8GB ram),  mediawiki (x cpu/xg
[14:50:46] <hashar>	 #topic Elastic search eating ram on slaves
[14:51:31] <hashar>	 Takes up to 1.5G / 2G ram on the current slaves
[14:51:49] <hashar>	 not sure which jobs rely on it as a backend, probably CirrusSearch extension
[14:52:22] <dcausse>	 hashar: we use a bot in labs and estest100[12] for our browser tests
[14:52:38] <hashar>	 just in time :-}
[14:52:44] <dcausse>	 :)
[14:53:08] <hashar>	 dcausse: I think the browser tests jobs have been moved to Jenkins now
[14:54:04] <hashar>	 or we can create dedicated slaves
[14:54:43] <dcausse>	 how can I check? from what I see in gerrit it's still cindy (our browser test bot) (https://gerrit.wikimedia.org/r/#/c/235149/)
[14:54:57] <hashar>	 with nodepool we will be able to `sudo apt-get install elasticsearch`  or start es on demand
[14:55:48] <dcausse>	 hashar: sometimes we need to manually clean up the indices, will it be possible?
[14:56:14] <hashar>	 dcausse: we will get a new fresh instance on each run
[14:56:22] <dcausse>	 ok
[14:56:24] <hashar>	 so there will be no indices at all on the job start
[14:56:29] <dcausse>	 nice
[14:56:35] <jzerebecki>	 #info for starting daemons on deman systemd supports starting one when a connection is opened on a port, then passes on the connection once it is started
[14:56:39] <hashar>	 jzerebecki found a killer feature we could use ^^^^
[14:57:51] <hashar>	 dcausse: I will ask Dan Duvall about the Cindy/Jenkins status :-}
[14:58:02] <jzerebecki>	 #info see socket-based activation in man systemd.socket
[14:58:06] <jzerebecki>	 #link http://www.freedesktop.org/software/systemd/man/systemd.socket.html
[14:58:31] <dcausse>	 so for the ram usage problem can you list the indices currently inside your es instances (curl localhost:9200/_cat/indices )
[14:59:09] <jzerebecki>	 integration-slave-precise-1011:~$ curl localhost:9200/_cat/indices
[14:59:10] <jzerebecki>	 red open build886_content_first 4 2 0 0 1.1kb 432b
[14:59:31] <hashar>	 yeah they are never cleaned up
[15:00:08] <hashar>	 #action Find a way to clean out ElasticSearch indices on job completion.
[15:00:12] <hashar>	 #endmeeting
[15:00:13] <wm-labs-meetbot`>	 Meeting ended Tue Sep 15 15:00:12 2015 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
[15:00:13] <wm-labs-meetbot`>	 Minutes:        https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-14.01.html
[15:00:13] <wm-labs-meetbot`>	 Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-14.01.txt
[15:00:13] <wm-labs-meetbot`>	 Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-14.01.wiki
[15:00:13] <wm-labs-meetbot`>	 Log:            https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-14.01.log.html
[15:00:35] <hashar>	 dcausse: thanks for showing up :-}   I took your point about Cindy and we will talk about it during the RelEng weekly meeting
[15:00:46] <dcausse>	 hashar: thanks! :)
[15:01:21] <hashar>	 dcausse: we are doing this CI discussion every week on tuesday at 2pm UTC / 4pm CET. So feel free to join next week session if you want to talk about cindy :-}
[15:01:37] <dcausse>	 ok :)
[15:10:11] <zeljkof-meeting>	 #startmeeting Weekly browser tests triage
[15:10:12] <wm-labs-meetbot`>	 Meeting started Tue Sep 15 15:10:11 2015 UTC and is due to finish in 60 minutes.  The chair is zeljkof-meeting. Information about MeetBot at http://wiki.debian.org/MeetBot.
[15:10:12] <wm-labs-meetbot`>	 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
[15:10:12] <wm-labs-meetbot`>	 The meeting name has been set to 'weekly_browser_tests_triage'
[15:10:46] <zeljkof-meeting>	 #topic triaging "doing" column
[15:10:53] <hashar>	 o/
[15:11:12] <hashar>	 #link https://phabricator.wikimedia.org/tag/browser-tests/
[15:11:17] <hashar>	 Triaging "Waiting for"
[15:15:14] <hashar>	 #link https://integration.wikimedia.org/ci/job/browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/743/consoleFull example job to investigate
[15:17:00] <zeljkof-meeting>	 #action create a subtask of T92613 to upgrade all repos to the latest watir-webdriver
[15:19:33] <hashar>	 #agreed Mark resolved: https://phabricator.wikimedia.org/T92613 It takes about 20 seconds just to start a Sauce Labs browser
[15:24:54] <zeljkof-meeting>	 #topic triaging "waiting for" column
[15:39:11] <zeljkof-meeting>	 #action ping Vikas about T90577
[15:48:27] <zeljkof-meeting>	 #topic triaging "todo" column
[15:50:14] <zeljkof-meeting>	 #endmeeting
[15:50:15] <wm-labs-meetbot`>	 Meeting ended Tue Sep 15 15:50:14 2015 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
[15:50:15] <wm-labs-meetbot`>	 Minutes:        https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-15.10.html
[15:50:15] <wm-labs-meetbot`>	 Minutes (text): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-15.10.txt
[15:50:15] <wm-labs-meetbot`>	 Minutes (wiki): https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-15.10.wiki
[15:50:15] <wm-labs-meetbot`>	 Log:            https://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-15-15.10.log.html
[21:58:51] <AaronSchulz>	 spagewmf: for http://tools.wmflabs.org/meetbot/wikimedia-office/2015/wikimedia-office.2015-09-02-21.01.html do you want to just schedule a "edit mw.org pages" meeting? Feel free to propose something in gcalender (I assume you have more meetings than I do).
[23:49:39] <brendan_c>	 Hey everyone, does anyone know if Science Tuesday happening tonight?
[23:51:18] <Krenair>	 brendan_c, -staff