[00:34:32] Yippee, build fixed! [00:34:33] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #267: FIXED in 53 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/267/ [00:35:57] (03PS1) 10Awight: add Ruby lint for CentralNotice [integration/config] - 10https://gerrit.wikimedia.org/r/172460 [00:53:25] Project browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce build #156: SUCCESS in 18 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce/156/ [01:06:49] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #277: SUCCESS in 40 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/277/ [01:06:49] PROBLEM - BetaLabs: Low disk space on /var on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki01.diskspace._var.byte_avail.value (100.00%) [01:24:48] Project browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce build #264: STILL FAILING in 1 hr 11 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce/264/ [02:27:02] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #123: FAILURE in 45 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/123/ [02:45:25] Project beta-scap-eqiad build #29305: FAILURE in 1 min 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/29305/ [02:55:08] Yippee, build fixed! [02:55:08] Project beta-scap-eqiad build #29306: FIXED in 1 min 9 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/29306/ [03:48:02] Yippee, build fixed! [03:48:03] Project browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #273: FIXED in 30 min: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/273/ [04:46:22] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #348: FAILURE in 1 hr 7 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/348/ [07:10:20] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #317: FAILURE in 1 hr 2 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/317/ [08:02:40] Yippee, build fixed! [08:02:41] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #400: FIXED in 1 hr 18 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/400/ [08:33:14] (03CR) 10Zfilipin: "I have noticed (in another repository) that since rubocop is non-voting for now, an offense will sneak in without anybody noticing, and th" [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/171985 (owner: 10Dduvall) [08:34:43] (03CR) 10Zfilipin: [C: 031] Instance evaluate blocks for new environments [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/171985 (owner: 10Dduvall) [09:54:14] (03PS1) 10Zfilipin: Releasing patch version 0.4.1 [selenium] - 10https://gerrit.wikimedia.org/r/172513 [09:57:47] (03CR) 10Amire80: [C: 032] Releasing patch version 0.4.1 [selenium] - 10https://gerrit.wikimedia.org/r/172513 (owner: 10Zfilipin) [09:58:02] (03Merged) 10jenkins-bot: Releasing patch version 0.4.1 [selenium] - 10https://gerrit.wikimedia.org/r/172513 (owner: 10Zfilipin) [10:20:11] (03PS1) 10Zfilipin: Fixed gem build warnings [selenium] - 10https://gerrit.wikimedia.org/r/172518 [10:20:33] (03CR) 10jenkins-bot: [V: 04-1] Fixed gem build warnings [selenium] - 10https://gerrit.wikimedia.org/r/172518 (owner: 10Zfilipin) [10:21:34] (03PS2) 10Zfilipin: Fixed gem build warnings [selenium] - 10https://gerrit.wikimedia.org/r/172518 [10:21:55] (03CR) 10jenkins-bot: [V: 04-1] Fixed gem build warnings [selenium] - 10https://gerrit.wikimedia.org/r/172518 (owner: 10Zfilipin) [10:24:45] (03CR) 10Zfilipin: Fixed gem build warnings (031 comment) [selenium] - 10https://gerrit.wikimedia.org/r/172518 (owner: 10Zfilipin) [10:25:08] Yippee, build fixed! [10:25:08] Project browsertests-VisualEditor-language-screenshot-linux-firefox » nl,contintLabsSlave && UbuntuTrusty build #37: FIXED in 9 min 28 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=nl,label=contintLabsSlave%20&&%20UbuntuTrusty/37/ [10:27:58] (03CR) 10Zfilipin: "recheck" [selenium] - 10https://gerrit.wikimedia.org/r/172518 (owner: 10Zfilipin) [10:34:26] (03CR) 10Zfilipin: "Not sure why yard job fails. :(" [selenium] - 10https://gerrit.wikimedia.org/r/172518 (owner: 10Zfilipin) [10:45:01] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ilo,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 14 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ilo,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [10:55:58] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ru,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 25 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ru,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [11:01:02] aharoni: unable to obtain stable firefox connection in 60 seconds [11:01:05] https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ru,label=contintLabsSlave%20&&%20UbuntuTrusty/38/console [11:01:12] for russian, of course :) [11:01:23] as soon as the job is finished I will make the change and rerun it [11:01:28] and other languages? [11:02:24] ilo is green [11:02:24] https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ilo,label=contintLabsSlave%20&&%20UbuntuTrusty/38/console [11:02:53] fr is running, green for now [11:02:53] https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=fr,label=contintLabsSlave%20&&%20UbuntuTrusty/38/console [11:06:32] Project browsertests-VisualEditor-language-screenshot-linux-firefox » fr,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 35 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=fr,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [11:20:20] Project browsertests-VisualEditor-language-screenshot-linux-firefox » mk,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 49 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=mk,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [11:33:31] Project browsertests-VisualEditor-language-screenshot-linux-firefox » it,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 1 hr 2 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=it,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [11:46:52] Project browsertests-VisualEditor-language-screenshot-linux-firefox » de,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 1 hr 16 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=de,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [11:59:28] Project browsertests-VisualEditor-language-screenshot-linux-firefox » pl,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 1 hr 28 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=pl,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [12:12:50] Project browsertests-VisualEditor-language-screenshot-linux-firefox » zh-hans,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 1 hr 42 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=zh-hans,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [12:13:49] Yippee, build fixed! [12:13:49] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #124: FIXED in 42 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/124/ [12:22:56] Yippee, build fixed! [12:22:56] Project browsertests-VisualEditor-language-screenshot-linux-firefox » hr,contintLabsSlave && UbuntuTrusty build #38: FIXED in 1 hr 52 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=hr,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [12:34:48] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ms,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 2 hr 4 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ms,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [12:47:16] Project browsertests-VisualEditor-language-screenshot-linux-firefox » yue,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 2 hr 16 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=yue,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [12:55:53] PROBLEM - BetaLabs: Low disk space on /var on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki01.diskspace._var.byte_avail.value (100.00%) [12:58:16] Project browsertests-VisualEditor-language-screenshot-linux-firefox » el,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 2 hr 27 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=el,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [13:05:24] lubaochuan: sorry, the hangout crashed on my machine [13:05:42] it says "This webpage has a redirect loop" [13:05:44] strange [13:05:52] I have to go, see you next week [13:05:58] Than's fine. Lets talk next time. [13:06:06] if you have any questions, here is a good place [13:06:13] Thanks much for your help. [13:06:15] or qa/wikitech-l mailing list [13:06:15] Got it. [13:06:21] * zeljkof is out of lunch [13:12:12] Project browsertests-VisualEditor-language-screenshot-linux-firefox » fi,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 2 hr 41 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=fi,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [13:15:25] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c21 (10Antoine "hashar" Musso (WMF)) hhvm on beta cluster now dumps files to /var/tmp/hhvm which is a 2GB partition. Noticed on deployment-medi... [13:18:27] 3Wikimedia Labs / 3deployment-prep (beta): hhvm apache fills /var/log/apache2 with access logs - 10https://bugzilla.wikimedia.org/73262 (10Antoine "hashar" Musso (WMF)) 3NEW p:3Unprio s:3normal a:3None The Apache on beta cluster instances fill up /var/log/apache2 rather quickly. The local logging sh... [13:21:38] RECOVERY - BetaLabs: Low disk space on /var on labmon1001 is OK: OK: All targets OK [13:23:49] Project browsertests-VisualEditor-language-screenshot-linux-firefox » pt,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 2 hr 53 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=pt,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [13:34:11] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #280: FAILURE in 28 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/280/ [13:43:57] 3Wikimedia Labs / 3deployment-prep (beta): Puppet failure on deployment-cache-bits01 - 10https://bugzilla.wikimedia.org/73263 (10Yuvi Panda) 3NEW p:3Unprio s:3normal a:3None Has been failing for almost a month now, monitoring didn't catch it. Shinken just caught it... Error: Failed to apply catalog:... [13:45:38] Project browsertests-VisualEditor-language-screenshot-linux-firefox » sl,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 3 hr 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=sl,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [13:45:40] 3Wikimedia Labs / 3deployment-prep (beta): Puppet failure on deployment-cache-bits01 - 10https://bugzilla.wikimedia.org/73263#c1 (10Yuvi Panda) Also on deployment-cache-text02: Error: Failed to apply catalog: Could not find dependent Service[gmond] for File[/etc/ganglia/conf.d/vhtcpd.pyconf] at /etc/puppet/... [13:53:59] greg-g: did we ever figure out who is the point person for 'hey puppet is failing in betalabs, fix it'? [13:54:00] greg-g: me? [13:59:21] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ja,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 3 hr 28 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ja,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [14:04:33] 3Wikimedia Labs / 3deployment-prep (beta): Puppet failure on deployment-cache-bits01 - 10https://bugzilla.wikimedia.org/73263#c2 (10Yuvi Panda) Also on deployment-cache-mobile03 Error: Failed to apply catalog: Could not find dependency File[/usr/lib/ganglia/python_modules] for File[/usr/lib/ganglia/python_m... [14:04:36] 3Wikimedia Labs / 3deployment-prep (beta): Puppet failure on deployment-cache-bits01 - 10https://bugzilla.wikimedia.org/73263#c3 (10Yuvi Panda) Looks like they're all related to labs no longer running gmond, which was merged around the time this problem started happening. We killed gmond because labs has no... [14:22:47] Project browsertests-VisualEditor-language-screenshot-linux-firefox » uk,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 3 hr 52 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=uk,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [14:35:21] (03CR) 10Hashar: "check experimental" [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/171985 (owner: 10Dduvall) [14:35:33] Project browsertests-VisualEditor-language-screenshot-linux-firefox » gl,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 4 hr 4 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=gl,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [14:36:17] (03CR) 10Hashar: [C: 032] "\O/" [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/171985 (owner: 10Dduvall) [14:36:35] (03Merged) 10jenkins-bot: Instance evaluate blocks for new environments [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/171985 (owner: 10Dduvall) [14:36:39] (03CR) 10Hashar: "Zeljkof: I guess we can ignore rubocop in this branch for now." [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/172007 (owner: 10Dduvall) [14:36:49] (03CR) 10Hashar: "check experimental" [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/172007 (owner: 10Dduvall) [14:37:00] (03CR) 10Hashar: "check experimental" [selenium] (env-abstraction-layer) - 10https://gerrit.wikimedia.org/r/172009 (owner: 10Dduvall) [14:45:56] Project browsertests-VisualEditor-language-screenshot-linux-firefox » cs,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 4 hr 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=cs,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:00:18] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ro,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 4 hr 29 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ro,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:04:03] aharoni: VE screen shots status [15:04:04] https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/38/console [15:04:20] so far 9 green, 13 failed [15:13:22] Project browsertests-VisualEditor-language-screenshot-linux-firefox » kn,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 4 hr 42 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=kn,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:20:24] Project browsertests-VisualEditor-language-screenshot-linux-firefox » nl,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 4 hr 49 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=nl,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:30:54] Project browsertests-VisualEditor-language-screenshot-linux-firefox » fa,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 0 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=fa,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:43:57] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ko,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 13 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ko,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [15:54:46] Project browsertests-VisualEditor-language-screenshot-linux-firefox » lb,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 24 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=lb,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [16:00:30] PROBLEM - CI: Low disk space on /var on labmon1001 is CRITICAL: CRITICAL: integration.integration-puppetmaster.diskspace._var.byte_avail.value (11.11%) [16:04:56] Project browsertests-VisualEditor-language-screenshot-linux-firefox » nb,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 34 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=nb,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [16:15:12] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ar,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 44 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ar,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [16:24:58] Project browsertests-VisualEditor-language-screenshot-linux-firefox » zh-hant,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 5 hr 54 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=zh-hant,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [16:36:06] I'm gonna look at rebuilding the beta apache config ontop of https://gerrit.wikimedia.org/r/170300 [16:36:49] RECOVERY - CI: Low disk space on /var on labmon1001 is OK: OK: All targets OK [16:39:10] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c22 (10Bryan Davis) (In reply to Antoine "hashar" Musso (WMF) from comment #21) > hhvm on beta cluster now dumps files to /var/tmp/hhvm which is... [16:39:29] Project browsertests-VisualEditor-language-screenshot-linux-firefox » es,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 6 hr 8 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=es,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [16:55:38] Project browsertests-VisualEditor-language-screenshot-linux-firefox » sv,contintLabsSlave && UbuntuTrusty build #38: SUCCESS in 6 hr 25 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=sv,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [17:06:43] Project browsertests-VisualEditor-language-screenshot-linux-firefox » et,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 6 hr 36 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=et,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [17:18:09] Project browsertests-VisualEditor-language-screenshot-linux-firefox » ast,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 6 hr 47 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=ast,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [17:24:46] Project browsertests-VisualEditor-production-linux-firefox-sauce build #28: FAILURE in 3 hr 47 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-production-linux-firefox-sauce/28/ [17:29:46] Project browsertests-VisualEditor-language-screenshot-linux-firefox » vi,contintLabsSlave && UbuntuTrusty build #38: FAILURE in 6 hr 59 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=vi,label=contintLabsSlave%20&&%20UbuntuTrusty/38/ [19:58:21] Yippee, build fixed! [19:58:22] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #349: FIXED in 1 hr 6 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/349/ [21:05:31] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce build #124: FAILURE in 1 hr 7 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/124/ [21:49:17] Yippee, build fixed! [21:49:18] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #318: FIXED in 1 hr 0 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/318/ [22:10:23] Yippee, build fixed! [22:10:23] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #256: FIXED in 2 min 59 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/256/ [22:11:36] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #401: FAILURE in 1 hr 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/401/