[00:06:22] <paladox>	 James_F or legoktm hi would you be able to review and merge https://gerrit.wikimedia.org/r/#/c/291604/ please
[00:06:56] <paladox>	 There has been no qunit tests freezing over the past week.
[01:06:35] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2347859 (10GWicke) Following an IRC discussion, I'm wondering what it would take to get a very minimal canary process going, wh...
[01:21:42] <paladox>	 Dereckson could you update the link at the bottom of https://wikitech.wikimedia.org/wiki/Deployments
[01:21:45] <paladox>	 Since it says
[01:21:46] <paladox>	 Deployment trains
[01:21:52] <paladox>	 mw::MediaWiki 1.27/Roadmap
[01:21:53] <paladox>	 please
[01:21:57] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2347890 (10scfc) Has E179 or #Differential yielded "more tangile benefits"?  The graph at https://korma.wmflabs.org/browser/scr.ht...
[01:22:05] <paladox>	 Which should be changed to mw:MediaWiki 1.28/Roadmap
[01:22:07] <paladox>	 please
[01:23:04] <paladox>	 Never mind i did it.
[01:23:13] <Dereckson>	 Thanks for the update.
[01:25:15] <paladox>	 Your welcome.
[01:37:52] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2347930 (10mmodell) I think we should use a server that is receiving production traffic. That way we can just watch error rates...
[01:42:55] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2347933 (10GWicke) Looking at network traffic in Kibana, it looks like querying logstash should be possible with something like...
[01:54:16] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2347952 (10bd808) Opening up the ferm rules to allow direct port 9200 traffic from tin. terbium, mira, and fluorine would be pr...
[01:59:57] <wikibugs>	 05Gerrit-Migration, 03releng-201516-q4, 10Wikimedia-IEG-grant-review: Migrate wikimedia-iegreview to Differential - https://phabricator.wikimedia.org/T132174#2347974 (10bd808) >>! In T132174#2347361, @greg wrote: > @Luke081515 Thanks :) But that's actually an answer to a different question :) >  > I am more...
[02:09:49] <ebernhardson>	 any gerrit admins around? i seem to have lost the password for our Cindy-the-browser-test-bot account (but i still own the email address it uses)
[02:17:56] <wmf-insecte>	 Yippee, build fixed!
[02:17:57] <wmf-insecte>	 Project selenium-QuickSurveys » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #42: 09FIXED in 4 min 55 sec: https://integration.wikimedia.org/ci/job/selenium-QuickSurveys/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/42/
[02:49:11] <ori>	 ebernhardson: is there a way to reset the password through gerrit? It's an LDAP account at the end of the day, no?
[02:50:13] <ori>	 oh, it can be done via the SSH CLI interface
[02:59:32] <Krenair>	 don't think gerrit is set up to write to LDAP
[02:59:53] <Krenair>	 oh, the HTTP password built into gerrit
[03:00:55] <Krenair>	 ebernhardson, it's an LDAP account so you can use wikitech password reset
[03:11:37] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2348018 (10mmodell) Now we need to come up with some reasonable query, filter and a python algorithm to surface relevant errors...
[03:20:09] <wmf-insecte>	 Project beta-update-databases-eqiad build #8960: 04FAILURE in 8.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8960/
[04:18:40] <wmf-insecte>	 Yippee, build fixed!
[04:18:40] <wmf-insecte>	 Project selenium-MultimediaViewer » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #33: 09FIXED in 22 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/33/
[04:20:38] <wmf-insecte>	 Yippee, build fixed!
[04:20:38] <wmf-insecte>	 Project beta-update-databases-eqiad build #8961: 09FIXED in 37 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8961/
[05:01:11] <grrrit-wm>	 (03PS1) 10Lethexie: Fix single space expected on single line comment. [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/292313 (https://phabricator.wikimedia.org/T136791) 
[05:02:47] <grrrit-wm>	 (03PS2) 10Lethexie: Fix single space expected on single line comment. [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/292313 (https://phabricator.wikimedia.org/T136791) 
[06:13:39] <wikibugs>	 03Scap3, 10Mobile-Content-Service: Investigate why recent mobileapps prod deployment didn't restart services - https://phabricator.wikimedia.org/T136773#2348163 (10mobrovac) 05Open>03Resolved a:03mobrovac That was my fault. I had to disable the restart for yesterday's deployment, but forgot to bring it b...
[06:28:24] <wmf-insecte>	 Yippee, build fixed!
[06:28:24] <wmf-insecte>	 Project mediawiki-core-code-coverage build #2052: 09FIXED in 3 hr 28 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage/2052/
[08:25:21] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-JobRunner, 10MediaWiki-Vagrant, 10TimedMediaHandler-Transcode: Video transcode job runner on beta cluster runs 5 jobs even though configured for 2 - https://phabricator.wikimedia.org/T110916#2348312 (10hashar) @brion maybe there are some useful details in the jobr...
[08:39:32] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2348363 (10hashar)  `OFFLINE_NODE_WHEN_COMPLETE` is not recognized by the Jenkins Gearman plugin, which was the reason I have...
[08:45:40] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2348380 (10hashar) Following up: in our jobs definitions, the git definition is usually provided by the default (in the above...
[09:31:21] <grrrit-wm>	 (03PS3) 10Lethexie: Add function commenting sniff. [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/291489 
[10:06:29] <wikibugs>	 06Release-Engineering-Team, 10Monitoring, 10Phabricator: string Wikimedia and MediaWiki not found on https://phabricator.wikimedia.org:443 - https://phabricator.wikimedia.org/T136813#2348577 (10jcrespo)
[10:14:06] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2348601 (10Qgil)
[10:37:00] <wikibugs>	 06Release-Engineering-Team, 10MediaWiki-General-or-Unknown, 05MW-1.23-release: 1.23.14 is really 1.23.13 - https://phabricator.wikimedia.org/T136484#2348661 (10Aklapper) @WayneSallee: [[ https://en.wikipedia.org/wiki/Diff_utility | Running a diff ]] (explaining that is out of scope for this task).
[10:48:20] <hashar>	 I am going to stop Zuul / CI for half an hour or so to take traces
[10:48:20] <hashar>	 will recheck what has been skipped / missed
[10:56:06] <paladox>	 hashar is that because zuul wont upgrade to the newer version.
[10:56:29] <hashar>	 paladox: yeah
[10:56:44] <paladox>	 Oh ok thanks for working on it :)
[11:08:50] <paladox>	 hashar im wondering could you change https://phabricator.wikimedia.org/W749 to public instead of all users.
[11:09:03] <paladox>	 since going to phabricator.wikimedia.org results in the logged in screen
[11:09:04] <paladox>	 please
[11:11:09] <wikibugs>	 06Release-Engineering-Team, 10Monitoring, 10Phabricator: string Wikimedia and MediaWiki not found on https://phabricator.wikimedia.org:443 - https://phabricator.wikimedia.org/T136813#2348737 (10jcrespo) Probably related to: T136816
[11:11:24] <hashar>	 paladox: fill a task in phabricator ? :)
[11:12:51] <paladox>	 hashar: i did https://phabricator.wikimedia.org/T136816
[11:13:57] <wikibugs>	 06Release-Engineering-Team, 10Monitoring, 10Phabricator: string Wikimedia and MediaWiki not found on https://phabricator.wikimedia.org:443 - https://phabricator.wikimedia.org/T136813#2348577 (10Paladox) Yep it seems that all panels have to be publish since even one having different permission can lock it to...
[11:16:38] <wikibugs>	 06Release-Engineering-Team, 10Monitoring, 10Phabricator: string Wikimedia and MediaWiki not found on https://phabricator.wikimedia.org:443 - https://phabricator.wikimedia.org/T136813#2348746 (10Aklapper) T136816 was my fault. Should work again now?
[11:19:14] <wikibugs>	 06Release-Engineering-Team, 10Monitoring, 10Phabricator: string Wikimedia and MediaWiki not found on https://phabricator.wikimedia.org:443 - https://phabricator.wikimedia.org/T136813#2348751 (10jcrespo) 05Open>03Resolved a:03jcrespo Yes, it does.
[11:22:53] <paladox>	 hashar https://phabricator.wikimedia.org/D246 :)
[11:23:06] <paladox>	 the idea is for that to support zuul in phabricator
[11:23:07] <paladox>	 :)
[12:07:57] <grrrit-wm>	 (03PS2) 10Hashar: [ReplaceText] Add npm test [integration/config] - 10https://gerrit.wikimedia.org/r/291840 (owner: 10Paladox)
[12:10:44] <hashar>	 !log Upgraded Zuul upstream code being 66c8e52..30a433b package is  2.1.0-151-g30a433b-wmf1precise1
[12:10:46] <qa-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master
[12:11:05] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] [ReplaceText] Add npm test [integration/config] - 10https://gerrit.wikimedia.org/r/291840 (owner: 10Paladox)
[12:11:21] <paladox>	 hashar thanks :)
[12:11:51] <grrrit-wm>	 (03Merged) 10jenkins-bot: [ReplaceText] Add npm test [integration/config] - 10https://gerrit.wikimedia.org/r/291840 (owner: 10Paladox)
[12:16:48] <hashar>	 paladox: that is a good use case to test Zuul is working properly :)
[12:16:53] <hashar>	 and apparently it does
[12:17:01] <paladox>	 hashar: :)
[12:17:14] <hashar>	 paladox: mind rebasing https://gerrit.wikimedia.org/r/#/c/291838/ ? :)
[12:17:31] <paladox>	 hashar I think twentyafterfour is seeing if he can get zuul supported in phabricator with https://phabricator.wikimedia.org/D246
[12:17:34] <paladox>	 and yes i will rebase now
[12:17:44] <hashar>	 I usually handle trivial rebase but then I am monitoring zuul behavior :D
[12:19:15] <grrrit-wm>	 (03PS3) 10Paladox: [ReplaceText] Add composer-test test [integration/config] - 10https://gerrit.wikimedia.org/r/291838 
[12:19:17] <paladox>	 hashar yep.
[12:19:21] <paladox>	 hashar ^^
[12:20:42] <paladox>	 hashar could you review https://gerrit.wikimedia.org/r/#/c/291670/ please.
[12:20:51] <paladox>	 It updates some olds tests for a repo
[12:22:56] <wmf-insecte>	 Project selenium-GettingStarted » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #38: 04FAILURE in 55 sec: https://integration.wikimedia.org/ci/job/selenium-GettingStarted/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/38/
[12:26:38] <grrrit-wm>	 (03CR) 10Hashar: [C: 032] "Build and deployed on gallium" [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/291886 (https://phabricator.wikimedia.org/T129938) (owner: 10Hashar)
[12:26:56] <grrrit-wm>	 (03CR) 10Hashar: [V: 032] 2.1.0-151-g30a433b-wmf1precise1 [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/291886 (https://phabricator.wikimedia.org/T129938) (owner: 10Hashar)
[12:31:46] <grrrit-wm>	 (03PS1) 10Hashar: Revert "2.1.0-151-g30a433b-wmf1precise1" [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292341 
[12:31:54] <grrrit-wm>	 (03CR) 10Hashar: [C: 032 V: 032] Revert "2.1.0-151-g30a433b-wmf1precise1" [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292341 (owner: 10Hashar)
[12:32:46] <grrrit-wm>	 (03PS1) 10Hashar: 2.1.0-151-g30a433b-wmf1precise1 [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292343 (https://phabricator.wikimedia.org/T129938) 
[12:33:21] <grrrit-wm>	 (03CR) 10Hashar: [C: 032 V: 032] 2.1.0-151-g30a433b-wmf1precise1 [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292343 (https://phabricator.wikimedia.org/T129938) (owner: 10Hashar)
[12:39:14] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Zuul, 07Upstream: Circular dependencies break Zuul - https://phabricator.wikimedia.org/T129938#2348941 (10hashar) 05Open>03Resolved a:03hashar I have deployed 2.1.0-151-g30a433b-wmf1precise1 on gallium.wikimedia.org It includes upstream fix 92464a2291a032b9253...
[12:40:19] <grrrit-wm>	 (03PS1) 10Hashar: Revert "Pin Zuul version used for testing" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) 
[12:40:27] <grrrit-wm>	 (03PS2) 10Hashar: Revert "Pin Zuul version used for testing" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) 
[12:41:29] <grrrit-wm>	 (03PS3) 10Hashar: Revert "Pin Zuul version used for testing" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) 
[12:43:01] <grrrit-wm>	 (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[13:04:56] <wmf-insecte>	 Project selenium-Math » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #37: 04FAILURE in 56 sec: https://integration.wikimedia.org/ci/job/selenium-Math/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/37/
[13:05:05] <wmf-insecte>	 Project selenium-Math » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #37: 04FAILURE in 1 min 4 sec: https://integration.wikimedia.org/ci/job/selenium-Math/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/37/
[13:17:41] <grrrit-wm>	 (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[13:18:24] <grrrit-wm>	 (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[13:35:10] <grrrit-wm>	 (03CR) 10Hashar: [C: 04-2] Revert "Pin Zuul version used for testing" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[13:35:18] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Revert "Pin Zuul version used for testing" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[14:11:05] <wikibugs>	 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Allow to test a mediawiki-config change to the beta cluster - https://phabricator.wikimedia.org/T136828#2349165 (10Dereckson)
[14:27:29] <wikibugs>	 06Release-Engineering-Team, 10MediaWiki-General-or-Unknown, 05MW-1.23-release: 1.23.14 is really 1.23.13 - https://phabricator.wikimedia.org/T136484#2349238 (10WayneSallee) But that's too obvious. :-) Yes, I should have thought of running diff.  For anyone reading that is not familiar with diff, just run: di...
[14:44:08] <grrrit-wm>	 (03PS1) 10Hashar: 2.1.0-151-g30a433b-wmf2precise1 [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292363 
[14:44:24] <wikibugs>	 10Browser-Tests-Infrastructure, 03releng-201516-q4, 10Malu (Malu-Prototype): Prototype new JS framework (malu) - https://phabricator.wikimedia.org/T131121#2349320 (10zeljkofilipin)
[14:44:32] <grrrit-wm>	 (03CR) 10Hashar: [C: 032 V: 032] 2.1.0-151-g30a433b-wmf2precise1 [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/292363 (owner: 10Hashar)
[14:55:37] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2349359 (10thcipriani)
[15:25:44] <wikibugs>	 03releng-201516-q4, 10Malu (Malu-Prototype), 07Surveys, 15User-zeljkofilipin: Send out browser testing user satisfaction survey - https://phabricator.wikimedia.org/T131123#2349432 (10zeljkofilipin)
[15:27:05] <wikibugs>	 03releng-201516-q4, 10Malu (Malu-Prototype), 07Surveys, 15User-zeljkofilipin: Send out browser testing user satisfaction survey - https://phabricator.wikimedia.org/T131123#2156648 (10zeljkofilipin)
[15:31:48] <wikibugs>	 03releng-201516-q4, 10Malu (Malu-Prototype), 07Surveys, 15User-zeljkofilipin: Send out browser testing user satisfaction survey - https://phabricator.wikimedia.org/T131123#2349459 (10zeljkofilipin)
[15:33:29] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: MediaWiki simple canary checks on mw1017 - https://phabricator.wikimedia.org/T136839#2349461 (10thcipriani)
[15:34:28] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: MediaWiki simple canary checks on mw1017 - https://phabricator.wikimedia.org/T136839#2349477 (10thcipriani)
[15:34:30] <wikibugs>	 03releng-201617-q4, 03Scap3 (Scap3-MediaWiki-MVP): Use scap3's canary deploys for MediaWiki - https://phabricator.wikimedia.org/T131120#2349476 (10thcipriani)
[15:36:27] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2349478 (10Nemo_bis)
[15:39:41] <wmf-insecte>	 Yippee, build fixed!
[15:39:42] <wmf-insecte>	 Project selenium-MobileFrontend » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #36: 09FIXED in 17 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/36/
[15:43:09] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2349493 (10thcipriani) Meta comment:  >>! In T110068#2347859, @GWicke wrote: > 1) Deploy the new code to a test host, like mw10...
[15:43:45] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2349496 (10mmodell) @scfc:   I suspect that reviewing patches for mobile is a lot different than reviewing mediawiki c...
[15:45:03] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2349498 (10Nemo_bis) MobileFrontend *is* a MediaWiki extension.
[15:46:51] <wmf-insecte>	 Yippee, build fixed!
[15:46:52] <wmf-insecte>	 Project selenium-MobileFrontend » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #36: 09FIXED in 24 min: https://integration.wikimedia.org/ci/job/selenium-MobileFrontend/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/36/
[16:04:43] <Dereckson>	 ~/Q rama
[16:06:25] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2349542 (10mmodell) @hashar:  I've already been using the jenkins phabricator plugin.  And here's one job which handles vario...
[16:18:01] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2349589 (10mmodell)
[16:19:33] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2151799 (10mmodell) I will attempt to get it templatized in jjb
[16:20:08] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2349596 (10mmodell)
[16:21:04] <wikibugs>	 10Continuous-Integration-Infrastructure, 05Gerrit-Migration, 03releng-201617-q1, 10Differential: Build glue from Differential to Nodepool - https://phabricator.wikimedia.org/T130950#2151799 (10mmodell)
[16:29:34] <wikibugs>	 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Allow to test a mediawiki-config change to the beta cluster - https://phabricator.wikimedia.org/T136828#2349165 (10Luke081515) For example someone writes a change at gerrit, and we give some people access to git, that...
[16:36:29] <grrrit-wm>	 (03CR) 10Paladox: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/292346 (https://phabricator.wikimedia.org/T136610) (owner: 10Hashar)
[16:57:31] <wikibugs>	 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Allow to test a mediawiki-config change to the beta cluster - https://phabricator.wikimedia.org/T136828#2349165 (10Krenair) You need access to the deployment-prep labs project and then you can deploy config changes to...
[16:59:06] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2349699 (10scfc) @mmodell: I don't know if MediaWiki core and extensions have different requirements for code review,...
[17:12:36] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-JobRunner, 10MediaWiki-Vagrant, 10TimedMediaHandler-Transcode: Video transcode job runner on beta cluster runs 5 jobs even though configured for 2 - https://phabricator.wikimedia.org/T110916#2349800 (10brion) @hashar I get fun stuff like:  ``` 2016-06-01T06:41:32+...
[17:17:48] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2349847 (10GWicke) The Kibana histogram query looks like a good candidate for detecting error frequency trends. Here is a sampl...
[17:20:03] <wikibugs>	 10Deployment-Systems, 10scap, 10Analytics, 10Analytics-Cluster, 03Scap3 (Scap3-Adoption-Phase1): Deploy analytics-refinery with scap3 - https://phabricator.wikimedia.org/T129151#2096617 (10Milimetric) @greg does this block you all for this quarter?  We want to get it done but we can wait until next quart...
[17:36:24] <wikibugs>	 10Deployment-Systems, 10scap, 10Analytics, 10Analytics-Cluster, 03Scap3 (Scap3-Adoption-Phase1): Deploy analytics-refinery with scap3 - https://phabricator.wikimedia.org/T129151#2349979 (10greg) Our quarterly goal was to get all services migrated this quarter; I know it's a tough thing that means work fo...
[17:38:27] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2350016 (10mmodell) @gwicke: Indeed, that is also something that can be turned into a nice text-mode display like  `▁▂▃▅▂▇`
[17:52:18] <wikibugs>	 10Deployment-Systems, 10scap, 10Analytics, 10Analytics-Cluster, 03Scap3 (Scap3-Adoption-Phase1): Deploy analytics-refinery with scap3 - https://phabricator.wikimedia.org/T129151#2350096 (10thcipriani) FWIW, the migration process has become more streamlined than it was when the previous analytics service...
[18:23:28] <wikibugs>	 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T136040#2350293 (10matmarex)
[18:25:25] <grrrit-wm>	 (03PS1) 10Florianschmidtwelzow: Mark GoogleAPIClient as archived [integration/config] - 10https://gerrit.wikimedia.org/r/292400 (https://phabricator.wikimedia.org/T119905) 
[18:27:16] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-JobRunner, 10MediaWiki-Vagrant, 10TimedMediaHandler-Transcode: Video transcode job runner on beta cluster runs 5 jobs even though configured for 2 - https://phabricator.wikimedia.org/T110916#2350331 (10hashar) In prod that is with $wmgMonologChannels['runJobs'] =>...
[18:28:52] <grrrit-wm>	 (03CR) 10Hashar: "GoogleLogin depends on GoogleAPIClient, but I guess the dependency is now filled via composer isn't it?" [integration/config] - 10https://gerrit.wikimedia.org/r/292400 (https://phabricator.wikimedia.org/T119905) (owner: 10Florianschmidtwelzow)
[18:42:04] <gwicke>	 twentyafterfour: got a moment to chat about next steps for the canary deploy?
[18:45:02] <twentyafterfour>	 gwicke: I'm in a meeting right now but in a few minutes I can
[18:46:03] <gwicke>	 ok, ping me when you have time
[18:49:46] <grrrit-wm>	 (03CR) 10Florianschmidtwelzow: "Right :) That's why the dependency is removed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/292400 (https://phabricator.wikimedia.org/T119905) (owner: 10Florianschmidtwelzow)
[18:53:35] <wikibugs>	 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T136040#2350493 (10thcipriani)
[19:00:46] <wikibugs>	 06Release-Engineering-Team, 06Developer-Relations, 06Team-Practices: Decision on Wikimedia Foundation service-level agreement on code review times - https://phabricator.wikimedia.org/T113707#2350508 (10Aklapper) **General request:** Please discuss general code review process aspects in T78768 (also see T7876...
[19:20:56] <twentyafterfour>	 gwicke: meeting is ending. Is now an ok time?
[19:22:43] <twentyafterfour>	 gwicke: I have code review in 40 minutes, free until then
[19:25:13] <gwicke>	 twentyafterfour: okay, so I was wondering about how to structure the next steps
[19:26:32] <gwicke>	 one fairly stand-alone task seems to be creating a module that, given a host, wait period & some other options like a message type, waits for that period, checks logstash & returns a verdict along with perhaps some background info
[19:26:50] <gwicke>	 a second one is running check.py against a test host
[19:27:02] <gwicke>	 and a third one doing the deploys & calling those scripts
[19:27:05] <gwicke>	 or modules
[19:28:15] <gwicke>	 I haven't looked at the scap code itself yet, so don't really know how hard 3) is
[19:28:25] <twentyafterfour>	 gwicke: yeah, that sounds about right. I like the idea of a stand-alone module that can do those checks.  Though I would also like to have a real-time histogram in scap, that part may not be worth the trouble
[19:28:35] <gwicke>	 1) (the logstash check) sounds relatively straightforward & stand-alone
[19:29:22] <gwicke>	 2) is straightforward too- mostly a matter of creating a swagger spec with some request / response pairs
[19:29:39] <twentyafterfour>	 3 really isn't terribly hard. I should probably draw up a diagram of the workflow. The scap code is really straightforward and easy to work with
[19:29:48] <gwicke>	 mobrovac just created a wrapper for 2), which we might be able to adapt
[19:30:47] <gwicke>	 check-<service>
[19:30:52] <thcipriani>	 the workflow is something I'm a little concerned about: keeping in mind deployments should be automatable in the beta cluster i.e. not pause for checks (which could just be put behind a flag: scap sync --full, seems like the simplest solution)
[19:32:00] <gwicke>	 --force to override checks?
[19:32:01] <twentyafterfour>	 the way I was envisioning it was a full deployment to one node, followed by a full deployment to all nodes
[19:32:06] <thcipriani>	 but I would agree that pausing for checks in various machines, particularly if those checks are stand-alone scripts, would be fairly easy in scap.
[19:32:13] <twentyafterfour>	 on beta we could just skip the whole thing
[19:32:26] <thcipriani>	 or --force, that's probably more expected behavior
[19:33:14] <gwicke>	 so I could look into 1), and ask mobrovac about 2), since he knows that area well
[19:34:01] <gwicke>	 could one of you pick up 3) (the integration / targeted deploys)?
[19:34:11] <twentyafterfour>	 gwicke: unblocking the firewall on logstash seems like the first step for getting logstash check working. Writing the code won't be difficult and you already came up with a query that should be a great starting point
[19:35:03] <gwicke>	 for now, using the user/pass is okay as a work-around
[19:35:15] <thcipriani>	 3 is somewhat a dependent task, but we can definitely get run command X on mw1017, roll forward to canary, run command Y on cannaries built in short order, I feel. twentyafterfour does that sound reasonable?
[19:35:19] <gwicke>	 the firewall shouldn't be hard once we are ready for a deploy
[19:35:31] <twentyafterfour>	 thcipriani: yes that's reasonable
[19:36:12] <gwicke>	 even if that command is just 'exit 0' for now
[19:36:24] <thcipriani>	 that's probably a good way to implement it initially.
[19:37:17] <gwicke>	 great, sounds like we have a plan
[19:37:22] <thcipriani>	 kk, after train I can try to wrangle the tasks in phab :)
[19:37:47] <gwicke>	 I'll add a note detailing the likely parameters for 1)
[19:38:47] <thcipriani>	 gwicke: thanks for your help, hopefully this makes SWAT deploys a lot less of an adventure.
[19:40:02] <gwicke>	 yes, it should at least be a good start
[19:56:03] <wikibugs>	 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T136040#2350728 (10Bawolff)
[19:59:50] <Amir1>	 thcipriani: hey, Have you seen this? https://phabricator.wikimedia.org/T136253#2337078
[20:00:47] <thcipriani>	 Amir1: yes, I did see that. I hadn't had a chance to look at it too closely, sorry :(
[20:01:03] <Amir1>	 don't worry. I was super busy too
[20:01:03] <thcipriani>	 from a high-level, it seems like a good solution to me.
[20:01:25] <thcipriani>	 but I haven't taken the deep dive on it that I meant to.
[20:03:19] <Amir1>	 okay, there is no rush. I fixed ores issue in another way. I just want to fix scap now
[20:03:53] <thcipriani>	 I don't have any objections, don't let me hold you up :)
[20:04:07] <Amir1>	 great. I will pursue this
[20:04:08] <Amir1>	 one other thing that comes up in deployment. scap builds two or three file 
[20:04:18] <Amir1>	 Directoires
[20:04:30] <Amir1>	 each for every deployment 
[20:04:51] <thcipriani>	 you mean: scap/log and .git/config-files and .git/DEPLOY_HEAD, those?
[20:04:53] <Amir1>	 so /srv/deployment/ores/deploy-cache/revs/#hash
[20:05:01] <Amir1>	 and another for another commit
[20:05:04] <thcipriani>	 ah, on the targets, yes.
[20:05:16] <thcipriani>	 those *should* be cleaned every 5 deploys
[20:05:24] <Amir1>	 that's super storage consuming 
[20:05:39] <Amir1>	 we had two deploys in a node in prod. it took 3 GB
[20:06:05] <Amir1>	 do you let shallow cloning? specially for submodules
[20:06:33] <Amir1>	 thcipriani: and the cleaning can be done automatically? or we should do it
[20:06:50] <thcipriani>	 the cleaning will be done automatically.
[20:07:20] <Amir1>	 can we reduce the number to, for example, three?
[20:07:21] <thcipriani>	 local clones use hard-links, so it should just be the size of the checkout, the objects will only exist in the deploy-cache/cache
[20:07:44] <thcipriani>	 er, the size of the currently checked-out code
[20:08:06] <thcipriani>	 IIRC I did make a config var for this, but I'd have to dig through code/docs to remember.
[20:08:23] <thcipriani>	 lemme finish the train deploy then I'll take a look.
[20:08:24] <Amir1>	 okay. I will try to find it
[20:08:27] <Amir1>	 sure
[20:20:21] <wmf-insecte>	 Project beta-update-databases-eqiad build #8977: 04FAILURE in 19 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8977/
[20:34:39] <Amir1>	 thcipriani: tell me once you are done. I have some questions wrt releases :D
[20:34:57] <thcipriani>	 Amir1: just finished, what's up?
[20:35:24] <Amir1>	 okay. We deployed ORES in prod today. We are setting the public endpoints 
[20:35:50] <Amir1>	 so most probably ores.wikimedia.org will be live tomorrow or Monday
[20:36:10] <Amir1>	 soo, we are ready to deploy the ores extension in fawiki
[20:36:26] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2350857 (10GWicke) @thcipriani, @mmodell & myself talked about next steps on IRC, and identified three independent tasks:  1) C...
[20:36:55] <thcipriani>	 Amir1: nice, congrats :)
[20:36:58] <Amir1>	 thcipriani: security review passed, translations are done. Community agrees. We were only waiting on moving to prod 
[20:37:03] <Amir1>	 thanks :)
[20:37:08] <halfak>	 Amir1, we'll want to do some performance testing against ores.wikimedia.org before we point the extension at it. 
[20:37:18] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Run qunit tests in IE8 (and possibly other Grade A browsers) - https://phabricator.wikimedia.org/T96432#2350860 (10Krinkle) >>! In T96432#2337192, @Paladox wrote: > @krinkle can we close this task since ie8 is support was dropped in mw core p...
[20:37:24] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Run qunit tests in IE8 (and possibly other Grade A browsers) - https://phabricator.wikimedia.org/T96432#2350862 (10Krinkle)
[20:37:26] <wikibugs>	 10Continuous-Integration-Infrastructure: Run QUnit tests via SauceLabs - https://phabricator.wikimedia.org/T96919#1229336 (10Krinkle)
[20:37:28] <halfak>	 But that shouldn't take much.  Depending on what we find, troubleshooting could be complicated
[20:37:28] <Amir1>	 definitely 
[20:37:31] <wikibugs>	 10Continuous-Integration-Infrastructure: Run QUnit tests via SauceLabs - https://phabricator.wikimedia.org/T96919#1229336 (10Krinkle)
[20:37:33] <wikibugs>	 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Run qunit tests in IE8 (and possibly other Grade A browsers) - https://phabricator.wikimedia.org/T96432#1216912 (10Krinkle)
[20:37:48] <wikibugs>	 10Continuous-Integration-Infrastructure: Run MediaWiki QUnit tests via SauceLabs - https://phabricator.wikimedia.org/T96919#1229336 (10Krinkle)
[20:38:07] <Amir1>	 I'm trying to find out when is the closest deployment window
[20:38:47] <Amir1>	 not to reserve it 
[20:39:17] <Amir1>	 thcipriani: can you tell me when is the closest deployment window (and the next one, if any complications happened)
[20:39:23] <wikibugs>	 10Continuous-Integration-Infrastructure: Run MediaWiki QUnit tests via SauceLabs - https://phabricator.wikimedia.org/T96919#2350868 (10Krinkle)
[20:39:36] <thcipriani>	 so https://wikitech.wikimedia.org/wiki/Deployments stays fairly static week-to-week
[20:39:36] <wikibugs>	 10Continuous-Integration-Infrastructure, 10MediaWiki-Unit-tests: Run MediaWiki QUnit tests via SauceLabs - https://phabricator.wikimedia.org/T96919#1229336 (10Krinkle)
[20:39:44] <Amir1>	 halfak: is it okay?
[20:39:46] <halfak>	 Amir1, +1 
[20:39:53] <Amir1>	 so may phab notes here
[20:39:55] <halfak>	 :) 
[20:39:56] <thcipriani>	 Amir1: do you meant the closest deployment window to...what?
[20:40:07] <halfak>	 fawiki & wikidatawiki
[20:40:14] <Amir1>	 mediawiki deployment 
[20:40:27] <Amir1>	 for rolling out a beta feature, can we rely on SWAT?
[20:40:41] <Amir1>	 or we should have be in a deployment train
[20:40:48] <thcipriani>	 ah, you mean for the extension?
[20:40:53] <Amir1>	 yup
[20:41:05] <Amir1>	 or even a dedicated event
[20:41:11] <thcipriani>	 ah, ok, so yeah, generally new extensions roll with the train process.
[20:41:33] <thcipriani>	 and then folks enable the extension on a particular wiki in evening (or the following morning) SWAT.
[20:41:41] <Amir1>	 I think fawiki in group3 right?
[20:41:50] <wmf-insecte>	 Yippee, build fixed!
[20:41:51] <wmf-insecte>	 Project selenium-Echo » chrome,beta,Linux,contintLabsSlave && UbuntuTrusty build #43: 09FIXED in 50 sec: https://integration.wikimedia.org/ci/job/selenium-Echo/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/43/
[20:41:58] <wmf-insecte>	 Yippee, build fixed!
[20:41:58] <wmf-insecte>	 Project selenium-Echo » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #43: 09FIXED in 57 sec: https://integration.wikimedia.org/ci/job/selenium-Echo/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/43/
[20:42:01] <thcipriani>	 yeah, it'll roll forward on Thursdays in group3
[20:42:23] <Amir1>	 so we have ~a week
[20:42:28] <Amir1>	 that's plenty
[20:42:40] <Amir1>	 halfak: ^
[20:42:59] <thcipriani>	 so to get the extension on the train, you'd add it here: https://github.com/wikimedia/mediawiki-tools-release/blob/master/make-wmf-branch/config.json
[20:43:23] <thcipriani>	 oh, ORES is there :P
[20:44:04] <Amir1>	 yeah, we did some deployments in beta before and I think they added it here because of that
[20:44:06] <thcipriani>	 and php-1.28.0-wmf.4/extensions/ORES is everywhere :)
[20:44:46] <thcipriani>	 so you'll just have to enable it in mediawiki-config, likely. You could do that in whatever SWAT window.
[20:45:00] <Amir1>	 yess
[20:45:02] <Amir1>	 awesome
[20:46:10] <thcipriani>	 also, by-the-by, I realized we, at some point, hard-coded the number of revs to keep: https://github.com/wikimedia/scap/blob/master/scap/context.py#L161-L180
[20:46:48] <Amir1>	 okay, I think we can tolerate 5 revs, let me talk to ops about it
[20:47:12] <Amir1>	 (strangely, it's 3 in beta cluster. I deployed a lot there and only there can be found)
[20:47:19] <Amir1>	 *three
[20:47:32] <thcipriani>	 huh, that's...weird.
[20:48:09] <Amir1>	 nodes in beta are small, maybe storage ran out?
[20:49:42] <Amir1>	 thcipriani: one thing, once the extension is deployed we need someone to run two maintenance scripts. Is it okay?
[20:51:20] <thcipriani>	 I do run some maintenance scripts as part of SWAT, but if they are especially onerous or time-consuming it might be worth scheduling a deployment window especially for this extension.
[20:51:31] <thcipriani>	 but scripts that take a few minutes are fine and easy.
[20:51:56] <Amir1>	 yup, the first one should take less than a sec
[20:52:07] <Amir1>	 the second one should be about 30-60 secs
[20:52:27] <thcipriani>	 ah, then yeah, might make a note on the Deployments page and remind the SWATter, but that sounds like it should be fine.
[20:52:39] <Amir1>	 nice
[20:52:40] <Amir1>	 thanks
[20:53:02] <Amir1>	 oh, another thing: for rolling out extensions that are only beta features do we need community consensuses? 
[20:53:12] <Amir1>	 for fawiki we have
[20:53:36] <Amir1>	 for wikidatawiki it's being done with the team knowing about this
[20:54:07] <Amir1>	 but let's say if we want to deploy it in ptwiki. Do I need consensuses 
[20:54:26] <halfak>	 JohanJ, ^ 
[20:54:51] <Amir1>	 I think he's off now.
[20:55:52] <thcipriani>	 Dereckson may be able to better answer that question than I, actually. I can say that there is generally a task that has input from a portion of the community that comes along with most of my SWAT deploys, especially for new extensions.
[20:59:04] <Amir1>	 thanks, totally understandable 
[21:00:27] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#2350922 (10thcipriani)
[21:00:29] <wikibugs>	 03releng-201617-q4, 03Scap3 (Scap3-MediaWiki-MVP): Use scap3's canary deploys for MediaWiki - https://phabricator.wikimedia.org/T131120#2350920 (10thcipriani)
[21:00:31] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: MediaWiki simple canary checks on mw1017 - https://phabricator.wikimedia.org/T136839#2350921 (10thcipriani)
[21:02:37] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create canary deploy process for MediaWiki - https://phabricator.wikimedia.org/T136883#2350945 (10thcipriani)
[21:03:14] <Dereckson>	 Hello. To get input from the community at early stage is beneficial, yes. We've seen had friction in the past for some deployments like VE or the multimedia viewer. If you ask from the start commnities to collaborate, you'll make them product owners and more involved in the process, so they will be more supportive.
[21:04:09] <Amir1>	 Dereckson: yeah, I know but this on has no UI change, only a beta feature, disabled by default and once it's enabled it shows its own changes
[21:04:10] <Dereckson>	 So, if it's not strictly necessery, as beta offers an opt in, it's a very good idea to do so.
[21:04:45] <wikibugs>	 10Deployment-Systems, 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Create a script to run test requests for the MediaWiki service - https://phabricator.wikimedia.org/T136839#2350960 (10thcipriani)
[21:05:56] <Amir1>	 I understand. I do the community engagement part
[21:10:29] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create ability to deploy-to and run-checks-on canary for MediaWiki deploys - https://phabricator.wikimedia.org/T136886#2351009 (10thcipriani)
[21:11:49] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create ability to deploy-to and run-checks-on canary for MediaWiki deploys - https://phabricator.wikimedia.org/T136886#2351026 (10thcipriani)
[21:11:51] <wikibugs>	 10Deployment-Systems, 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Create a script to run test requests for the MediaWiki service - https://phabricator.wikimedia.org/T136839#2351027 (10thcipriani)
[21:11:53] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#1568286 (10thcipriani)
[21:11:55] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create canary deploy process for MediaWiki - https://phabricator.wikimedia.org/T136883#2351025 (10thcipriani)
[21:13:09] <thcipriani>	 gwicke: twentyafterfour tracking task created: https://phabricator.wikimedia.org/T136883
[21:13:41] <thcipriani>	 (in case all the email alerts weren't enough :)) 
[21:19:59] <gwicke>	 I'll look into creating the basic logstash check script in a bit
[21:20:43] <wmf-insecte>	 Yippee, build fixed!
[21:20:43] <wmf-insecte>	 Project beta-update-databases-eqiad build #8978: 09FIXED in 42 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/8978/
[21:29:20] <wikibugs>	 10Beta-Cluster-Infrastructure, 10DBA, 10Flow, 03Collab-Team-2016-Apr-Jun-Q4: Run Flow External Store migration in dry-run mode on Beta - https://phabricator.wikimedia.org/T119567#2351104 (10Mattflaschen-WMF)
[22:01:02] <bd808>	 twentyafterfour: yes, please merge it before I add more revisions to the chain
[22:01:33] <bd808>	 re: https://gerrit.wikimedia.org/r/290156
[22:02:59] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Wikidata, 05MW-1.28-release-notes, 13Patch-For-Review, and 2 others: qunit jobs have karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2351215 (10Krinkle) See also <https://github.com/karma-runner/karma-chrome-launcher/issues/68>  @h...
[22:05:07] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Wikidata, 05MW-1.28-release-notes, 13Patch-For-Review, and 2 others: qunit jobs have karma stall when chromium disconnect - https://phabricator.wikimedia.org/T136188#2351231 (10Paladox) @Krinkle I doint thinkso but we traced it to chromium 50 since chromium 49 is...
[22:08:49] <paladox>	 ostriches hi im wondering if you could create a repo in gerrit for me.
[22:09:00] <paladox>	 Im wondering if you could rename https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/timeline to https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/Timeline
[22:09:01] <paladox>	 please
[22:09:56] <paladox>	 or twentyafterfour ^^ or legoktm
[22:09:57] <paladox>	 please
[22:16:11] <wikibugs>	 10Deployment-Systems, 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Create a script to run test requests for the MediaWiki service - https://phabricator.wikimedia.org/T136839#2351289 (10Krinkle)
[22:21:59] <wmf-insecte>	 Yippee, build fixed!
[22:22:00] <wmf-insecte>	 Project selenium-CentralAuth » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #41: 09FIXED in 1 min 58 sec: https://integration.wikimedia.org/ci/job/selenium-CentralAuth/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/41/
[22:29:43] <wikibugs>	 06Release-Engineering-Team, 10Fundraising-Backlog, 10fundraising-tech-ops: Spike: Talk about moving payments to WMF production MediaWiki - https://phabricator.wikimedia.org/T130658#2351327 (10awight)
[22:34:30] <wikibugs>	 06Release-Engineering-Team, 10Fundraising-Backlog, 10fundraising-tech-ops: Spike: Talk about moving payments to WMF production MediaWiki - https://phabricator.wikimedia.org/T130658#2351353 (10awight) @Jgreen  We're talking about trying to get ourselves on the train of WMF-standardized deployments for MediaWi...
[22:35:58] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Scap3 check to monitor logstash and detect changes in error frequency - https://phabricator.wikimedia.org/T110068#1568286 (10Joe) @GWicke just a couple of suggestions:  # If we want to check MediaWiki with service_checker (which I do agree w...
[22:36:55] <wikibugs>	 06Release-Engineering-Team, 10Fundraising-Backlog, 10fundraising-tech-ops: Spike: Talk about moving payments to WMF production MediaWiki - https://phabricator.wikimedia.org/T130658#2351359 (10awight) @mmodell  Sure thing, thank you for noticing this task.  We're trying to identify ways in which our process m...
[22:48:49] <wikibugs>	 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Allow to test a mediawiki-config change to the beta cluster - https://phabricator.wikimedia.org/T136828#2351391 (10Dereckson) Any config change or only stuff already merged on wmf/master branches?
[23:09:50] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create ability to deploy-to and run-checks-on canary for MediaWiki deploys - https://phabricator.wikimedia.org/T136886#2351440 (10thcipriani)
[23:12:35] <wikibugs>	 10Deployment-Systems, 10scap, 07WorkType-NewFunctionality: Create ability to deploy-to and run-checks-on canary for MediaWiki deploys - https://phabricator.wikimedia.org/T136886#2351552 (10thcipriani) p:05Triage>03High
[23:51:18] <wikibugs>	 10Continuous-Integration-Infrastructure, 07Documentation, 13Patch-For-Review: Jenkins: Generate CSS docs from LESS and publish to doc.wikimedia.org - https://phabricator.wikimedia.org/T60620#2351633 (10Mattflaschen-WMF) 05declined>03Open kss is still in core.
[23:51:42] <wikibugs>	 10scap, 03Scap3 (Scap3-MediaWiki-MVP), 07WorkType-NewFunctionality: Basic scap{2,3} canary deployment process & checks - https://phabricator.wikimedia.org/T110068#2351635 (10GWicke)
[23:51:54] <wikibugs>	 10Continuous-Integration-Infrastructure, 07Documentation, 13Patch-For-Review: Jenkins: Generate CSS docs from LESS and publish to doc.wikimedia.org - https://phabricator.wikimedia.org/T60620#2351636 (10Mattflaschen-WMF)
[23:54:27] <wikibugs>	 06Release-Engineering-Team, 10Fundraising-Backlog, 10fundraising-tech-ops: Spike: Talk about moving payments to WMF production MediaWiki - https://phabricator.wikimedia.org/T130658#2351641 (10mmodell) @awight: scap3 deployment would be the best bet I think.  Regular mediawiki train deployments will eventuall...