[00:20:53] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/SpaceInsideParens [ruby/api] - 10https://gerrit.wikimedia.org/r/168286 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[00:23:19] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "There's no contention around this one. Looks good." [ruby/api] - 10https://gerrit.wikimedia.org/r/168286 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[00:23:45] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fixed RuboCop offense Style/SpaceInsideParens [ruby/api] - 10https://gerrit.wikimedia.org/r/168286 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[01:34:15] <grrrit-wm>	 (03PS2) 10Ori.livneh: Reënable Apache lint check [integration/config] - 10https://gerrit.wikimedia.org/r/166033 (https://bugzilla.wikimedia.org/70068) (owner: 10Hashar)
[03:47:09] <wmf-insecte>	 Yippee, build fixed!
[03:47:09] <wmf-insecte>	 Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #236: FIXED in 1 min 21 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/236/
[04:17:38] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #251: FAILURE in 59 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/251/
[04:28:18] <wmf-insecte>	 Yippee, build fixed!
[04:28:19] <wmf-insecte>	 Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #152: FIXED in 10 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/152/
[06:15:25] <wmf-insecte>	 Yippee, build fixed!
[06:15:25] <wmf-insecte>	 Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #94: FIXED in 1 hr 10 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/94/
[06:50:36] <icinga-wm>	 PROBLEM - BetaLabs: Puppet failure events on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-videoscaler01.puppetagent.failed_events.value (30.00%)  
[06:59:58] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #254: FAILURE in 31 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/254/
[07:20:28] <icinga-wm>	 RECOVERY - BetaLabs: Puppet failure events on labmon1001 is OK: OK: All targets OK  
[07:28:38] <wmf-insecte>	 Yippee, build fixed!
[07:28:38] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #97: FIXED in 43 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/97/
[08:07:35] <wmf-insecte>	 Project browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #248: FAILURE in 23 min: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/248/
[08:41:29] <wmf-insecte>	 Project browsertests-VisualEditor-language-screenshot-linux-firefox » hr,contintLabsSlave && UbuntuTrusty build #19: FAILURE in 12 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=hr,label=contintLabsSlave%20&&%20UbuntuTrusty/19/
[08:59:08] <wmf-insecte>	 Project browsertests-VisualEditor-language-screenshot-linux-firefox » en,contintLabsSlave && UbuntuTrusty build #20: FAILURE in 12 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=en,label=contintLabsSlave%20&&%20UbuntuTrusty/20/
[09:07:05] <grrrit-wm>	 (03CR) 10Zfilipin: "Dan, can you please review this and +2 if you think we should enable RuboCop?" [integration/config] - 10https://gerrit.wikimedia.org/r/168061 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin)
[10:26:43] <wmf-insecte>	 Project browsertests-VisualEditor-language-screenshot-linux-firefox » he,contintLabsSlave && UbuntuTrusty build #23: FAILURE in 10 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=he,label=contintLabsSlave%20&&%20UbuntuTrusty/23/
[11:16:59] <grrrit-wm>	 (03PS1) 10Zfilipin: WIP Trying to fix file upload for VisualEditor language screenshots [integration/config] - 10https://gerrit.wikimedia.org/r/169675 
[11:41:52] <wikibugs>	 3Wikimedia / 3Continuous integration: CI browser test dashboard takes 100 seconds to appear on first load - 10https://bugzilla.wikimedia.org/70671#c3 (10Antoine "hashar" Musso (WMF)) What we could do is discard old build results after a given time. I am not sure whether we need to keep them after say 15 days.
[11:48:48] <grrrit-wm>	 (03PS2) 10Zfilipin: Fix file upload for VisualEditor language screenshots [integration/config] - 10https://gerrit.wikimedia.org/r/169675 
[12:06:47] <grrrit-wm>	 (03PS3) 10Zfilipin: Fix file upload for VisualEditor language screenshots [integration/config] - 10https://gerrit.wikimedia.org/r/169675 
[12:09:30] <grrrit-wm>	 (03CR) 10Amire80: [C: 032] Fix file upload for VisualEditor language screenshots [integration/config] - 10https://gerrit.wikimedia.org/r/169675 (owner: 10Zfilipin)
[12:13:07] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fix file upload for VisualEditor language screenshots [integration/config] - 10https://gerrit.wikimedia.org/r/169675 (owner: 10Zfilipin)
[14:06:51] <wmf-insecte>	 Project browsertests-Wikidata-PerformanceTests-linux-firefox-sauce build #34: FAILURE in 51 sec: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-PerformanceTests-linux-firefox-sauce/34/
[14:32:58] <wmf-insecte>	 Yippee, build fixed!
[14:32:59] <wmf-insecte>	 Project browsertests-Wikidata-PerformanceTests-linux-firefox-sauce build #36: FIXED in 32 sec: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-PerformanceTests-linux-firefox-sauce/36/
[14:43:17] <Tobi_WMDE_SW>	 zeljkof: can you quickly tell me what should be the name of the tag for scenarios that should be run in internet-explorer?
[14:43:20] <Tobi_WMDE_SW>	 is it @ie
[14:43:27] <zeljkof>	 no
[14:43:32] <Tobi_WMDE_SW>	 ok
[14:43:45] <zeljkof>	 @internet_explorer_VERSION
[14:43:53] <zeljkof>	 replace VERSION with 6-11
[14:43:55] <zeljkof>	 so
[14:44:02] <zeljkof>	 @internet_explorer_6
[14:44:04] <Tobi_WMDE_SW>	 zeljkof: ok, thx, that heps
[14:44:07] <Tobi_WMDE_SW>	 *helps
[14:44:18] <zeljkof>	 great
[14:44:22] <Tobi_WMDE_SW>	 there is no version needed for @firefox and @chrome?
[14:44:27] <Tobi_WMDE_SW>	 right?
[14:54:00] <zeljkof>	 correct
[15:29:53] <wikibugs>	 3Wikimedia / 3Continuous integration: CI browser test dashboard takes 100 seconds to appear on first load - 10https://bugzilla.wikimedia.org/70671#c4 (10Antoine "hashar" Musso (WMF)) I have proposed the idea of reducing the browser tests history to the QA mailing list: https://lists.wikimedia.org/pipermail/q...
[15:42:23] <wikibugs>	 3Wikimedia / 3Continuous integration: CI browser test dashboard takes 100 seconds to appear on first load - 10https://bugzilla.wikimedia.org/70671#c5 (10Željko Filipin) (In reply to Antoine "hashar" Musso (WMF) from comment #3) > What we could do is discard old build results after a given time. I am not > su...
[16:35:39] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): sync articles from production wikis (css/gadgets) - 10https://bugzilla.wikimedia.org/49779 (10Greg Grossmeier) a:5Ariel T. Glenn>3None
[17:23:58] <chrismcmahon>	 marxarelli: could you give a look to https://gerrit.wikimedia.org/r/#/c/168420/ ? it would be nice to merge it before I disappear completely for the rest of the week
[17:24:29] <marxarelli>	 chrismcmahon: sure thing
[17:25:05] <chrismcmahon>	 marxarelli:  thanks !
[17:37:16] <grrrit-wm>	 (03PS3) 10Dduvall: Enable RuboCop jobs for all repositories where it was experimental [integration/config] - 10https://gerrit.wikimedia.org/r/168061 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin)
[17:43:26] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] Enable RuboCop jobs for all repositories where it was experimental [integration/config] - 10https://gerrit.wikimedia.org/r/168061 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin)
[17:51:32] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/SpecialGlobalVars [ruby/api] - 10https://gerrit.wikimedia.org/r/168287 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[17:51:39] <chrismcmahon>	 thanks marxarelli that's my last outstanding patch for now
[17:52:03] * chrismcmahon is listening to a guy talking about automated testing at Facebook
[17:52:17] <marxarelli>	 chrismcmahon: right on
[17:55:57] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "Looks good, no exceptions here. All rebased and ready to go." [ruby/api] - 10https://gerrit.wikimedia.org/r/168287 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[17:56:50] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fixed RuboCop offense Style/SpecialGlobalVars [ruby/api] - 10https://gerrit.wikimedia.org/r/168287 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[18:41:16] <James_F>	 Can someone make me +sysop on Beta Labs hewiki? I need to import some content…
[18:42:18] <JohnLewis>	 James_F: sure - username you want it on? (just for reminders :p)
[18:42:49] <James_F>	 JohnLewis: "Jdforrester (WMF)" please. :-)
[18:42:59] <James_F>	 JohnLewis: http://he.wikipedia.beta.wmflabs.org/wiki/%D7%9E%D7%99%D7%95%D7%97%D7%93:%D7%90%D7%A7%D7%A8%D7%90%D7%99/Template
[18:43:17] <James_F>	 JohnLewis: "There are no pages in the following namespace: תבנית."
[18:43:23] <James_F>	 Fun. :-)
[18:44:43] <JohnLewis>	 James_F: get my last message? (connection is urg right now)
[18:45:52] <James_F>	 Was your last message asking which username? If not, no.
[18:46:04] <JohnLewis>	 James_F: that was it :)
[18:46:26] <James_F>	 <James_F> JohnLewis: "Jdforrester (WMF)" please. :-) 
[18:47:11] <JohnLewis>	 James_F: {{done}}! :)
[18:47:22] <James_F>	 Thanks!
[18:47:33] <JohnLewis>	 poke me if you need anything else
[18:48:14] <wikibugs>	 3Wikimedia / 3Quality Assurance: pdfhandler and MMV builds are misconfigured - 10https://bugzilla.wikimedia.org/72435#c3 (10Chris McMahon) bump
[18:48:27] * James_F nods.
[18:49:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c2 (10Greg Grossmeier) Can we have timing information to help diagnose?  See also: https://tools.wmflabs.org/nagf/?project=deployment-prep
[18:49:56] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366 (10Greg Grossmeier) p:5Unprio>3High
[18:50:11] <James_F>	 JohnLewis: Hmm. Is file-based import a different right?
[18:50:32] <Q1918981>	 James_F: It is - I'll add that to your account if you need it
[18:50:43] <James_F>	 JohnLewis: Please.
[18:50:52] <grrrit-wm>	 (03CR) 10Dduvall: [C: 031] "This is probably the most contentious topic when it comes to Ruby style. The bbatsov guide offers two options for string literals: single " [ruby/api] - 10https://gerrit.wikimedia.org/r/168288 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[18:51:01] <JohnLewis>	 done
[18:55:02] <James_F>	 JohnLewis: Yay. Thanks.
[19:02:12] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/StringLiterals [ruby/api] - 10https://gerrit.wikimedia.org/r/168288 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:04:14] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Rename all occurences of "deployment-prep" to "beta-cluster" - 10https://bugzilla.wikimedia.org/72694 (10Greg Grossmeier) 3NEW p:3Unprio s:3minor a:3None "deployment-prep" no longer makes sense and it's not how anyone refers to the cluster of vms known as "B...
[19:04:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Rename all occurences of "deployment-prep" to "beta-cluster" - 10https://bugzilla.wikimedia.org/72694 (10Greg Grossmeier) p:5Unprio>3Low s:5minor>3enhanc
[19:06:03] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "All rebased and ready to go." [ruby/api] - 10https://gerrit.wikimedia.org/r/168288 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:06:17] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fixed RuboCop offense Style/StringLiterals [ruby/api] - 10https://gerrit.wikimedia.org/r/168288 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:15:28] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Rename all occurences of "deployment-prep" to "beta-cluster" - 10https://bugzilla.wikimedia.org/72694#c1 (10Andrew Bogott) It might be possible to create a new project and then yoink the instances out of deployment-prep and into the new one.  That will break a fair...
[19:18:14] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Rename all occurences of "deployment-prep" to "beta-cluster" - 10https://bugzilla.wikimedia.org/72694#c2 (10John F. Lewis) Hashar: Have any opinions regarding this? I spoke with Greg and offered to make good use of some of my spare time by basically doing the bulk o...
[19:23:03] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/UnneededPercentQ [ruby/api] - 10https://gerrit.wikimedia.org/r/168289 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:23:21] <wmf-insecte>	 Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #153: FAILURE in 10 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/153/
[19:25:13] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c3 (10Chris McMahon) example failure from just now: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/153/testReport/junit/(root)/...
[19:27:24] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "Rebased." [ruby/api] - 10https://gerrit.wikimedia.org/r/168289 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:27:37] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fixed RuboCop offense Style/UnneededPercentQ [ruby/api] - 10https://gerrit.wikimedia.org/r/168289 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:46:26] <bd808>	 !log Logging seems broken following merge of https://gerrit.wikimedia.org/r/#/c/119941/24. Investigating
[19:46:29] <qa-morebots>	 Logged the message, Master
[19:53:58] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c4 (10Greg Grossmeier) Reedy/Antoine: Can you two diagnose this one, please?  Chris: Having more than two data points would be helpful (there might be a pattern eg tests running while code/...
[19:53:59] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/WordArray [ruby/api] - 10https://gerrit.wikimedia.org/r/168290 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:56:39] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "Rebased." [ruby/api] - 10https://gerrit.wikimedia.org/r/168290 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[19:57:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c5 (10Chris McMahon) Greg: it happens so often that I hadn't tried to track it closely.
[20:02:47] <grrrit-wm>	 (03PS3) 10Dduvall: Fixed RuboCop offense Lint/UselessAssignment [ruby/api] - 10https://gerrit.wikimedia.org/r/168292 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[20:03:45] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "Rebased." [ruby/api] - 10https://gerrit.wikimedia.org/r/168292 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[20:03:56] <grrrit-wm>	 (03Merged) 10jenkins-bot: Fixed RuboCop offense Lint/UselessAssignment [ruby/api] - 10https://gerrit.wikimedia.org/r/168292 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[20:05:59] <chrismcmahon>	 marxarelli: I have to take off for an hour, but I am perplexed as to how this happened: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/95/testReport/(root)/Topic%20collapsing/Small_topics_view/
[20:06:51] <chrismcmahon>	 marxarelli: also https://integration.wikimedia.org/ci/view/BrowserTests/view/-All/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/lastCompletedBuild/testReport/(root)/Topic%20collapsing/Small_topics_view/
[20:06:59] <chrismcmahon>	 I'll futz with it this afternoon
[20:09:13] <wmf-insecte>	 Yippee, build fixed!
[20:09:14] <wmf-insecte>	 Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #136: FIXED in 9 min 27 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/136/
[20:36:41] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Rename all occurences of "deployment-prep" to "beta-cluster" - 10https://bugzilla.wikimedia.org/72694#c3 (10Andre Klapper) (If you also want to rename the corresponding Bugzilla component, please file a ticket under "Wikimedia > Bugzilla")
[20:48:17] <wmf-insecte>	 Yippee, build fixed!
[20:48:18] <wmf-insecte>	 Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #297: FIXED in 54 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/297/
[20:50:04] <grrrit-wm>	 (03PS1) 10Dduvall: Environment abstraction layer and browser factories [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169850 
[20:50:06] <grrrit-wm>	 (03PS1) 10Dduvall: Firefox environment bindings and specs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169851 
[20:50:08] <grrrit-wm>	 (03PS1) 10Dduvall: Chrome environment bindings and specs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169852 
[20:50:10] <grrrit-wm>	 (03PS1) 10Dduvall: Phantomjs environment bindings and specs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169853 
[20:50:12] <grrrit-wm>	 (03PS1) 10Dduvall: Remote browser factories and improved docs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169854 
[20:50:14] <grrrit-wm>	 (03PS1) 10Dduvall: Implemented env and browser teardown [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169855 
[20:50:16] <grrrit-wm>	 (03PS1) 10Dduvall: Implemented env/browser teardown and sauce update [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169856 
[20:50:18] <grrrit-wm>	 (03PS1) 10Dduvall: Removed deprecated global methods [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169857 
[20:52:04] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Chrome environment bindings and specs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169852 (owner: 10Dduvall)
[20:52:07] <grrrit-wm>	 (03CR) 10jenkins-bot: [V: 04-1] Remote browser factories and improved docs [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/169854 (owner: 10Dduvall)
[20:53:48] <grrrit-wm>	 (03CR) 10Dduvall: [C: 04-2] "Not sure why gerrit did this, but there's now a new change at https://gerrit.wikimedia.org/r/#/c/169850/ for this patch set that will merg" [selenium] - 10https://gerrit.wikimedia.org/r/159644 (owner: 10Dduvall)
[20:54:02] <wikibugs>	 3Wikimedia / 3Continuous integration: Various Jenkins jobs broken after mediawiki/vendor became required - 10https://bugzilla.wikimedia.org/72700 (10Bryan Davis) 3NEW p:3Unprio s:3major a:3None Several jobs are broken with "PHP Fatal error:  Interface 'Psr\Log\LoggerInterface' not found in ..." follo...
[20:55:12] <bd808>	 chrismcmahon, greg-g: ^ some stuff that hashar missed when preparing for my PSR-3 patch
[20:56:29] <wikibugs>	 3Wikimedia / 3Continuous integration: Various Jenkins jobs broken after mediawiki/vendor became required - 10https://bugzilla.wikimedia.org/72700 (10Greg Grossmeier) p:5Unprio>3Immedi a:3Antoine "hashar" Musso (WMF)
[20:57:00] <greg-g>	 Krinkle: can you see if that's a quick fix or if we should revert (is it possible) the patch which caused the breakage?
[20:57:08] <greg-g>	 Krinkle: that being https://bugzilla.wikimedia.org/show_bug.cgi?id=72700
[20:57:12] <Krinkle>	 greg-g: what breakage
[20:57:37] <Krinkle>	 k
[20:58:00] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Logs not being received by udp2log after PSR-3 patch - 10https://bugzilla.wikimedia.org/72701 (10Bryan Davis) 3NEW p:3Unprio s:3major a:3Bryan Davis After https://gerrit.wikimedia.org/r/#/c/119941/ landed I am not seeing logs in /data/project/logs or logstash.
[20:58:12] <Krinkle>	 greg-g: Please refer here or on the bug to a job that fails. It seems mediawiki-core is passing before and after that patch
[20:58:28] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Logs not being received by udp2log after PSR-3 patch - 10https://bugzilla.wikimedia.org/72701 (10Bryan Davis) p:5Unprio>3High
[20:58:42] <greg-g>	 bd808: ^
[20:59:42] <bd808>	 https://integration.wikimedia.org/ci/job/mwext-Wikibase-client-tests/7214/console is one
[21:00:02] <greg-g>	 Krinkle: ^
[21:00:02] <bd808>	 The wikidata tests are not using the new zuul-cloner setup that hashar made
[21:00:07] * greg-g stops playing #REDIRECT
[21:00:07] <Q170484ktm>	 :/
[21:00:15] <Krinkle>	 https://integration.wikimedia.org/ci/job/parsoidsvc-php-parsertests/2773/console
[21:00:36] * greg-g is going into another hour of 1:1s starting... now
[21:00:38] <bd808>	 And the parsoid tests do their own crazy thing to setup MW
[21:00:50] <Krinkle>	 Only does mw-core-get.sh mw-set-env.sh mw-install-sqlite.sh
[21:00:59] <Krinkle>	 bd808: Can you help refactor those three scripts to function with vendor?
[21:01:13] <Krinkle>	 We can't change which scripts are evaluated since those three script calls are harcoded in a millin jobs
[21:01:20] <greg-g>	 eek
[21:01:25] <Krinkle>	 but fortuantely we factored the script contents from a macro into a .sh file
[21:01:29] <Krinkle>	 so we can update them in one place
[21:01:37] <Krinkle>	 it's a lot better than I made it sound
[21:01:40] <grrrit-wm>	 (03PS2) 10Dduvall: Fixed RuboCop offense Style/ClassAndModuleChildren [ruby/api] - 10https://gerrit.wikimedia.org/r/168309 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[21:02:13] <bd808>	 I think we can just add the clone of mediawiki/vendor into mw-core-get.sh?
[21:02:26] <Krinkle>	 https://github.com/wikimedia/integration-config/blob/ac5f1d9c/jjb/parsoidsvc.yaml#L43-L45
[21:03:00] <Krinkle>	 bd808: yeah
[21:03:12] <Krinkle>	 The other uses of mw-install-sqlite are with vendor present
[21:03:14] <grrrit-wm>	 (03CR) 10Dduvall: [C: 032] "Rebased." [ruby/api] - 10https://gerrit.wikimedia.org/r/168309 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[21:03:18] <Krinkle>	 it's in https://github.com/wikimedia/integration-jenkins/tree/master/bin
[21:03:27] <Krinkle>	 https://github.com/wikimedia/integration-jenkins/blob/master/bin/mw-core-get.sh
[21:03:32] <bd808>	 The fix for wikidata is this I think -- https://github.com/wikimedia/integration-config/blob/master/jjb/macro.yaml#L444-L456
[21:04:11] <Krinkle>	 bd808: Moving from get-mw-core to zuul-cloner/vendor is too major. Has side-effects, requires testing and requires recompiling and deploying 100s of jobs. That's a day-task.
[21:04:54] <bd808>	 Yeah. Hashar thought the jobs were ready, but apparently not
[21:05:08] <bd808>	 revert may be easiest for today
[21:05:10] <Krinkle>	 all extensions will be broken as that commit to mw-core rolls out. Wikidata and parsoid are just the first to hit it I reckon.
[21:05:21] <Krinkle>	 bd808: either revert or fix mw-core-get
[21:05:38] <Krinkle>	 The latter should be pretty striaght forward. I can review/test/git-deploy it from tin in <1h
[21:06:09] <bd808>	 ok. lets take a shot at that first
[21:07:38] <bd808>	 so that script is doing a clone of mw/core to the job local path right?
[21:08:41] <bd808>	 that is sick looking :( It unpacks a tar made either from the local repo or outsourced to gitblit
[21:11:45] <Krinkle>	 Indeed. I have no idea why it is that way.
[21:12:21] <bd808>	 I think hashar was trying to make things fast
[21:12:21] <Krinkle>	 I suppose to avoid a .git conflict. Though it'd be trivial to do a 1-depth clone from local zuul replication and then trash .git
[21:12:50] <Krinkle>	 s/zuul/gerrit/ replication
[21:13:07] <Krinkle>	 In fact, that's what we do in most cases, but that doesn't work in labs
[21:13:12] <Krinkle>	 since there's no replag there
[21:13:44] <bd808>	 So we can add the same crazy dance for mediawiki/vendor I guess
[21:17:52] <bd808>	 Krinkle: Is $TREE_ISH a git commit hash most of the time? Or a branch name?
[21:21:17] <Krinkle>	 Never seen that env before in my life
[21:21:43] <Krinkle>	 Ah it's a param to get-mw-core
[21:21:55] <Krinkle>	 https://github.com/search?q=mw-core-get+@wikimedia&type=Code&utf8=%E2%9C%93
[21:22:01] <Krinkle>	 1:{branch}
[21:22:32] * bd808 nods
[21:22:58] <Krinkle>	 bd808: It's only ever passed 'master' or '{branch}', in the latter case the relevant macro is never used with a branch parameter so that'd be the empty string
[21:23:08] <Krinkle>	 so it's only ever given 'master' or "" from what I can see
[21:24:37] <grrrit-wm>	 (03PS1) 10BryanDavis: Add support for mediawiki/vendor to mw-core-get.sh [integration/jenkins] - 10https://gerrit.wikimedia.org/r/169925 (https://bugzilla.wikimedia.org/72700) 
[21:25:03] <bd808>	 Krinkle: How horrible does that look? ^
[21:25:42] <wmf-insecte>	 Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #95: FAILURE in 1 hr 16 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/95/
[21:27:54] <Krinkle>	 bd808: Hm.. why mkdir vendor?
[21:28:12] <Krinkle>	 Ah, interesting
[21:28:35] <bd808>	 Good question. It looked to me like that dir needed to exist, but totally untested
[21:29:02] <bd808>	 This is type and hope coding at best
[21:33:26] <bd808>	 grepping through the jjb config, I think that only the parsoid and wikidata jobs may be effected actually
[21:33:37] <aude>	 and this is when i'm glad we no longer maintain wikidata jenkins separately
[21:33:52] <bd808>	 aude: :)
[21:34:29] <aude>	 i dont' know what $TREE_ISH is
[21:34:46] <aude>	 but worth a try
[21:35:20] <bd808>	 comments in the yaml seem to indicate that it should be a branch name for core
[21:36:12] <bd808>	 I'm a bit confused why hashar didn't make a new marco for the multiple zuul-cloner steps
[21:36:19] <bd808>	 *macro
[21:37:18] <bd808>	 ah. he did sort of -- https://github.com/wikimedia/integration-config/blob/d9cdb263f3a4ad143f00ea90b78a1840970ddd42/jjb/mediawiki-extensions.yaml#L114-L123
[21:41:41] <Krinkle>	 bd808: would that dir ever exist already?
[21:41:48] <Krinkle>	 Might be stricter to always create it
[21:42:07] <bd808>	 It shouldn't exist, no
[21:42:18] <bd808>	 unless there is old stuff in the workdir
[21:42:30] * bd808 runs to watch rfc meeting in #-office
[21:43:28] <subbu>	 bd808, Krinkle maybe mkdir -p?
[21:44:31] <Krinkle>	 bd808: Yeah, so let's do plain mkdir, -p is not needed since parnet dir is already established
[21:44:39] <Krinkle>	 if it exists, that's an error
[21:44:45] <Krinkle>	 strict first, loosen up as needed.
[21:48:06] <subbu>	 k
[21:48:44] <grrrit-wm>	 (03PS2) 10BryanDavis: Add support for mediawiki/vendor to mw-core-get.sh [integration/jenkins] - 10https://gerrit.wikimedia.org/r/169925 (https://bugzilla.wikimedia.org/72700) 
[21:50:21] <bd808>	 It should never need to make nested directories so -p would be overkill just to ignore the existing dir failure
[22:02:48] <grrrit-wm>	 (03CR) 10Krinkle: [C: 032] "Let's try this." [integration/jenkins] - 10https://gerrit.wikimedia.org/r/169925 (https://bugzilla.wikimedia.org/72700) (owner: 10BryanDavis)
[22:02:50] <grrrit-wm>	 (03Merged) 10jenkins-bot: Add support for mediawiki/vendor to mw-core-get.sh [integration/jenkins] - 10https://gerrit.wikimedia.org/r/169925 (https://bugzilla.wikimedia.org/72700) (owner: 10BryanDavis)
[22:04:29] <aude>	 deployed yet?
[22:06:25] <greg-g>	 see -operations
[22:07:10] <aude>	 k
[22:12:42] <wikibugs>	 3Wikimedia / 3Continuous integration: Various Jenkins jobs broken after mediawiki/vendor became required - 10https://bugzilla.wikimedia.org/72700#c3 (10Krinkle) 5PATC>3RESO/FIX Deployed to prod via git-deploy.  before: https://integration.wikimedia.org/ci/job/parsoidsvc-php-parsertests/2773/console  afte...
[22:13:12] <wikibugs>	 3Wikimedia / 3Continuous integration: Jenkins: Various jobs broken after mediawiki/vendor became required (mw-get-core needs mediawiki-vendor) - 10https://bugzilla.wikimedia.org/72700 (10Krinkle)
[22:14:54] <wmf-insecte>	 Yippee, build fixed!
[22:14:55] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #255: FIXED in 31 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/255/
[22:23:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c6 (10Bryan Davis) 503s could very well could be caused by https://gerrit.wikimedia.org/r/#/c/163078/ which restarts the hhvm server as part of each scap. While the hhvm fcgi process is shu...
[22:23:54] <grrrit-wm>	 (03CR) 10Dduvall: "I'm going to increase the acceptable line length to 100 for now. See the coding conventions for my rational.[1]" [ruby/api] - 10https://gerrit.wikimedia.org/r/168310 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[22:29:29] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c7 (10Greg Grossmeier) That is a really strong culprit....
[22:30:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c8 (10Ori Livneh) > Greg: it happens so often that I hadn't tried to track it closely.  How often do we run scap?
[22:31:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c9 (10Bryan Davis) (In reply to Ori Livneh from comment #8) > > Greg: it happens so often that I hadn't tried to track it closely. >  > How often do we run scap?  On every commit to core or...
[22:41:57] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c10 (10Chris McMahon) 163078 was merged 14 October, the timing looks good for this being the culprit.  It seems to occur with increasing frequency, but that could be my bias.
[22:45:42] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c11 (10Bryan Davis) (In reply to Chris McMahon from comment #10) > 163078 was merged 14 October, the timing looks good for this being the > culprit. >  > It seems to occur with increasing f...
[22:46:13] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c12 (10Greg Grossmeier) (In reply to Bryan Davis from comment #9) > (In reply to Ori Livneh from comment #8) > > > Greg: it happens so often that I hadn't tried to track it closely. > >  >...
[22:47:27] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): beta labs goes 503 for a short time - 10https://bugzilla.wikimedia.org/72366#c13 (10Greg Grossmeier) (In reply to Bryan Davis from comment #11) > (In reply to Chris McMahon from comment #10) > > 163078 was merged 14 October, the timing looks good for this being the...
[22:48:22] <chrismcmahon>	 greg-g: smoking gun :-)
[22:50:27] <greg-g>	 :)
[22:50:51] <bd808>	 I guess it's good that I read the qa list posts :)
[22:51:01] <greg-g>	 we love you, bd808 
[22:51:35] <chrismcmahon>	 bd808: greg-g I briefly hassled Ori about it when I was in SF last week.
[22:52:04] * bd808 can see that conversation now
[22:52:52] * chrismcmahon has spent a good chunk of GTAC working on stuff during the presentations, but it's all good. 
[22:56:31] <bd808>	 now if I could just figure out ho w I killed beta's logs I could end my day :(
[22:57:56] <chrismcmahon>	 greg-g: the 503s have definitely raised the flake factor on beta labs. but seconds later everything is fine again.
[22:58:48] <bd808>	 !log Stopped udp2log and started udp2log-mw on deployment-bastion
[22:58:50] <qa-morebots>	 Logged the message, Master
[22:59:04] <bd808>	 and logs are back :)
[22:59:14] <bd808>	 stupid conflicting services
[23:03:57] <grrrit-wm>	 (03PS3) 10Dduvall: Fixed RuboCop offense Metrics/LineLength [ruby/api] - 10https://gerrit.wikimedia.org/r/168310 (https://bugzilla.wikimedia.org/63307) (owner: 10Zfilipin)
[23:04:41] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): Logs not being received by udp2log after PSR-3 patch - 10https://bugzilla.wikimedia.org/72701#c1 (10Bryan Davis) 5NEW>3RESO/FIX This turned out to be caused by an old and known problem. The udp2log service was running on deployment-bastion instead of the needed...
[23:04:45] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): [OPS] udp2log prevents udp2log-mw from starting - 10https://bugzilla.wikimedia.org/38995 (10Bryan Davis)
[23:04:46] <wikibugs>	 3Wikimedia / 3Quality Assurance: Jenkins tests shouldn't go red when it's not its fault - 10https://bugzilla.wikimedia.org/72722 (10Jon) 3NEW p:3Unprio s:3normal a:3None The MobileFrontend tests (in Firefox) have been flakey for some time. Take a look at the list on the left for examples of recurring...
[23:06:57] <wikibugs>	 3Wikimedia / 3Quality Assurance: Jenkins tests shouldn't go red when it's not its fault - 10https://bugzilla.wikimedia.org/72722#c1 (10Chris McMahon)  Sorry Jon, a big part of the problem is https://bugzilla.wikimedia.org/show_bug.cgi?id=72366  which should get fixed soonish
[23:09:59] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366 (10Greg Grossmeier) s:5normal>3major
[23:10:14] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366#c16 (10Greg Grossmeier) (In reply to Bryan Davis from comment #14) > Giuseppe is interested in adding etcd support to pybal that could make this > possible but...
[23:10:29] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366#c14 (10Bryan Davis) (In reply to Greg Grossmeier from comment #13) > (In reply to Bryan Davis from comment #11) > >  > > If restarting hhvm is the cause, the fr...
[23:11:28] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366 (10Greg Grossmeier)
[23:11:58] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366#c15 (10Chris McMahon) "a bit"?
[23:12:16] <greg-g>	 oh, it's delayed again
[23:13:03] <bd808>	 booooot laaaaagggg
[23:13:35] <bd808>	 I'm going to celebrate my PSR-3 patch landing with a beer and a baseball game. :)
[23:17:01] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366#c17 (10Ori Livneh) I think someone should actually do the legwork to investigate this issue by cross-referencing 503s against the log files before we get carrie...
[23:20:59] <greg-g>	 bd808|BUFFER: enjoy sir :)
[23:25:28] <chrismcmahon>	 it would be disappointing if the only timing info about 503s on beta comes from browser test failures
[23:29:40] <greg-g>	 yeah :/
[23:38:44] <wikibugs>	 3Wikimedia Labs / 3deployment-prep (beta): HHVM fcgi restart during scap runs cause 503s (and failed tests) - 10https://bugzilla.wikimedia.org/72366#c18 (10Greg Grossmeier) (In reply to Ori Livneh from comment #17) > I think someone should actually do the legwork to investigate this issue by > cross-referenc...