[01:17:55] PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL 100.00% of data above the critical threshold [0.0] [02:46:09] PROBLEM - Puppet failure on deployment-bastion is CRITICAL 22.22% of data above the critical threshold [0.0] [02:47:37] PROBLEM - Puppet failure on deployment-memc02 is CRITICAL 30.00% of data above the critical threshold [0.0] [02:47:41] PROBLEM - Puppet failure on deployment-memc04 is CRITICAL 30.00% of data above the critical threshold [0.0] [02:47:41] PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL 20.00% of data above the critical threshold [0.0] [02:47:51] PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL 20.00% of data above the critical threshold [0.0] [02:48:07] PROBLEM - Puppet failure on deployment-logstash2 is CRITICAL 55.56% of data above the critical threshold [0.0] [02:49:41] PROBLEM - Puppet failure on deployment-zookeeper01 is CRITICAL 50.00% of data above the critical threshold [0.0] [02:49:43] PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL 30.00% of data above the critical threshold [0.0] [02:50:34] PROBLEM - Puppet failure on deployment-urldownloader is CRITICAL 60.00% of data above the critical threshold [0.0] [02:50:35] PROBLEM - Puppet failure on deployment-zotero01 is CRITICAL 40.00% of data above the critical threshold [0.0] [02:50:50] PROBLEM - Puppet failure on mira is CRITICAL 60.00% of data above the critical threshold [0.0] [02:50:53] PROBLEM - Puppet failure on deployment-kafka02 is CRITICAL 40.00% of data above the critical threshold [0.0] [02:51:39] PROBLEM - Puppet failure on deployment-memc03 is CRITICAL 40.00% of data above the critical threshold [0.0] [02:52:09] PROBLEM - Puppet failure on deployment-sentry2 is CRITICAL 22.22% of data above the critical threshold [0.0] [02:52:37] PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL 30.00% of data above the critical threshold [0.0] [02:53:11] PROBLEM - Puppet failure on deployment-db2 is CRITICAL 55.56% of data above the critical threshold [0.0] [02:53:39] PROBLEM - Puppet failure on deployment-cxserver03 is CRITICAL 40.00% of data above the critical threshold [0.0] [02:53:44] PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL 60.00% of data above the critical threshold [0.0] [02:55:20] PROBLEM - Puppet failure on deployment-sca01 is CRITICAL 55.56% of data above the critical threshold [0.0] [02:56:16] PROBLEM - Puppet failure on deployment-sca02 is CRITICAL 22.22% of data above the critical threshold [0.0] [02:57:28] PROBLEM - Puppet failure on deployment-stream is CRITICAL 40.00% of data above the critical threshold [0.0] [02:57:56] PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL 30.00% of data above the critical threshold [0.0] [02:58:53] PROBLEM - Puppet failure on deployment-redis01 is CRITICAL 20.00% of data above the critical threshold [0.0] [02:58:55] PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL 50.00% of data above the critical threshold [0.0] [02:58:55] PROBLEM - Puppet failure on deployment-fluorine is CRITICAL 20.00% of data above the critical threshold [0.0] [02:59:21] PROBLEM - Puppet failure on deployment-pdf01 is CRITICAL 60.00% of data above the critical threshold [0.0] [03:00:45] PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL 40.00% of data above the critical threshold [0.0] [03:01:07] PROBLEM - Puppet failure on deployment-test is CRITICAL 22.22% of data above the critical threshold [0.0] [03:05:41] PROBLEM - Puppet failure on deployment-salt is CRITICAL 75.00% of data above the critical threshold [0.0] [03:05:41] PROBLEM - Puppet failure on deployment-upload is CRITICAL 20.00% of data above the critical threshold [0.0] [03:07:33] PROBLEM - Puppet failure on deployment-elastic05 is CRITICAL 60.00% of data above the critical threshold [0.0] [03:08:18] PROBLEM - Puppet failure on deployment-db1 is CRITICAL 40.00% of data above the critical threshold [0.0] [03:09:20] PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL 100.00% of data above the critical threshold [0.0] [03:09:26] PROBLEM - Puppet failure on deployment-mathoid is CRITICAL 40.00% of data above the critical threshold [0.0] [03:09:28] PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL 70.00% of data above the critical threshold [0.0] [03:10:04] PROBLEM - Puppet failure on deployment-redis02 is CRITICAL 44.44% of data above the critical threshold [0.0] [03:10:20] PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL 66.67% of data above the critical threshold [0.0] [03:15:04] PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL 55.56% of data above the critical threshold [0.0] [04:05:52] RECOVERY - Puppet failure on deployment-kafka02 is OK Less than 1.00% above the threshold [0.0] [04:06:14] RECOVERY - Puppet failure on deployment-sca02 is OK Less than 1.00% above the threshold [0.0] [04:07:56] RECOVERY - Puppet failure on deployment-videoscaler01 is OK Less than 1.00% above the threshold [0.0] [04:08:12] RECOVERY - Puppet failure on deployment-db2 is OK Less than 1.00% above the threshold [0.0] [04:08:54] RECOVERY - Puppet failure on deployment-fluorine is OK Less than 1.00% above the threshold [0.0] [04:08:54] RECOVERY - Puppet failure on deployment-pdf02 is OK Less than 1.00% above the threshold [0.0] [04:09:24] RECOVERY - Puppet failure on deployment-pdf01 is OK Less than 1.00% above the threshold [0.0] [04:10:19] RECOVERY - Puppet failure on deployment-sca01 is OK Less than 1.00% above the threshold [0.0] [04:10:43] RECOVERY - Puppet failure on deployment-salt is OK Less than 1.00% above the threshold [0.0] [04:11:37] RECOVERY - Puppet failure on deployment-memc03 is OK Less than 1.00% above the threshold [0.0] [04:12:37] RECOVERY - Puppet failure on deployment-memc02 is OK Less than 1.00% above the threshold [0.0] [04:12:37] RECOVERY - Puppet failure on deployment-memc04 is OK Less than 1.00% above the threshold [0.0] [04:13:51] RECOVERY - Puppet failure on deployment-redis01 is OK Less than 1.00% above the threshold [0.0] [04:14:17] RECOVERY - Puppet failure on deployment-elastic07 is OK Less than 1.00% above the threshold [0.0] [04:15:47] RECOVERY - Puppet failure on deployment-eventlogging02 is OK Less than 1.00% above the threshold [0.0] [04:16:07] RECOVERY - Puppet failure on deployment-test is OK Less than 1.00% above the threshold [0.0] [04:17:08] RECOVERY - Puppet failure on deployment-sentry2 is OK Less than 1.00% above the threshold [0.0] [04:17:36] RECOVERY - Puppet failure on deployment-elastic05 is OK Less than 1.00% above the threshold [0.0] [04:20:06] RECOVERY - Puppet failure on deployment-mediawiki01 is OK Less than 1.00% above the threshold [0.0] [04:20:19] RECOVERY - Puppet failure on deployment-jobrunner01 is OK Less than 1.00% above the threshold [0.0] [04:20:39] RECOVERY - Puppet failure on deployment-upload is OK Less than 1.00% above the threshold [0.0] [04:23:07] RECOVERY - Puppet failure on deployment-logstash2 is OK Less than 1.00% above the threshold [0.0] [04:23:13] RECOVERY - Puppet failure on deployment-db1 is OK Less than 1.00% above the threshold [0.0] [04:24:29] RECOVERY - Puppet failure on deployment-elastic08 is OK Less than 1.00% above the threshold [0.0] [04:24:30] RECOVERY - Puppet failure on deployment-mathoid is OK Less than 1.00% above the threshold [0.0] [04:25:06] RECOVERY - Puppet failure on deployment-redis02 is OK Less than 1.00% above the threshold [0.0] [04:25:30] RECOVERY - Puppet failure on deployment-urldownloader is OK Less than 1.00% above the threshold [0.0] [04:26:10] RECOVERY - Puppet failure on deployment-bastion is OK Less than 1.00% above the threshold [0.0] [04:27:42] RECOVERY - Puppet failure on deployment-elastic06 is OK Less than 1.00% above the threshold [0.0] [04:27:52] RECOVERY - Puppet failure on deployment-apertium01 is OK Less than 1.00% above the threshold [0.0] [04:29:42] RECOVERY - Puppet failure on deployment-parsoid05 is OK Less than 1.00% above the threshold [0.0] [04:29:42] RECOVERY - Puppet failure on deployment-zookeeper01 is OK Less than 1.00% above the threshold [0.0] [04:30:11] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #503: FAILURE in 38 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/503/ [04:30:30] RECOVERY - Puppet failure on deployment-zotero01 is OK Less than 1.00% above the threshold [0.0] [04:30:50] RECOVERY - Puppet failure on mira is OK Less than 1.00% above the threshold [0.0] [04:32:34] RECOVERY - Puppet failure on deployment-mediawiki02 is OK Less than 1.00% above the threshold [0.0] [04:33:42] RECOVERY - Puppet failure on deployment-cxserver03 is OK Less than 1.00% above the threshold [0.0] [04:33:42] RECOVERY - Puppet failure on deployment-mediawiki03 is OK Less than 1.00% above the threshold [0.0] [04:37:28] RECOVERY - Puppet failure on deployment-stream is OK Less than 1.00% above the threshold [0.0] [06:41:43] PROBLEM - Puppet failure on deployment-upload is CRITICAL 20.00% of data above the critical threshold [0.0] [06:52:10] PROBLEM - Puppet failure on deployment-bastion is CRITICAL 66.67% of data above the critical threshold [0.0] [06:59:02] PROBLEM - Puppet failure on integration-zuul-server is CRITICAL 100.00% of data above the critical threshold [0.0] [07:03:28] RECOVERY - Free space - all mounts on deployment-videoscaler01 is OK All targets OK [07:21:40] RECOVERY - Puppet failure on deployment-upload is OK Less than 1.00% above the threshold [0.0] [07:27:14] RECOVERY - Puppet failure on deployment-bastion is OK Less than 1.00% above the threshold [0.0] [09:09:11] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #664: FAILURE in 59 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/664/ [10:21:22] (03PS5) 10Paladox: Update interwiki tests [integration/config] - 10https://gerrit.wikimedia.org/r/225313 [10:23:08] (03PS6) 10Paladox: Update interwiki tests [integration/config] - 10https://gerrit.wikimedia.org/r/225313 [10:26:35] (03PS3) 10Paladox: Check for json in BreadCrumbs [integration/config] - 10https://gerrit.wikimedia.org/r/225291 [10:57:20] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » en,contintLabsSlave && UbuntuTrusty build #61: FAILURE in 33 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=en,label=contintLabsSlave%20&&%20UbuntuTrusty/61/ [11:20:30] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » he,contintLabsSlave && UbuntuTrusty build #62: FAILURE in 17 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=he,label=contintLabsSlave%20&&%20UbuntuTrusty/62/ [11:50:01] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » es,contintLabsSlave && UbuntuTrusty build #63: FAILURE in 14 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=es,label=contintLabsSlave%20&&%20UbuntuTrusty/63/ [11:59:31] PROBLEM - Free space - all mounts on deployment-videoscaler01 is CRITICAL deployment-prep.deployment-videoscaler01.diskspace._var.byte_percentfree (<50.00%) [12:07:18] PROBLEM - Puppet failure on integration-slave-jessie-1001 is CRITICAL 100.00% of data above the critical threshold [0.0] [13:17:47] PROBLEM - Puppet staleness on deployment-restbase01 is CRITICAL 100.00% of data above the critical threshold [43200.0] [13:58:40] vikasyaligar: around? I am at tech track [13:59:05] I know amir is busy these days, but the two of us should at least get together and try the talk at least once [13:59:40] zeljkof-conferen: Yup ! coming down. I was working on https://gerrit.wikimedia.org/r/#/c/225320/1/modules/ve-mw/tests/browser/features/support/pages/visual_editor_page.rb,cm [14:22:23] zeljkof-conferen: I am in tech track [14:23:02] zeljkof-conferen: Are you off to 4rth floor ? [14:23:54] vikasyaligar: yes, tech track is on 4th today [14:30:39] zeljkof-conferen: hmm ! I am now in "Don Alberto 3" where "Video Production Tools and Community in the Wikiverse" talk is going on [14:31:17] I am in the second row! :) [14:33:39] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #568: FAILURE in 7 min 38 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/568/ [15:14:13] PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL 100.00% of data above the critical threshold [0.0] [17:03:38] 6Release-Engineering, 7Tracking: Fix easy problems reported by RuboCop - https://phabricator.wikimedia.org/T91485#1462167 (10zeljkofilipin) [17:25:18] PROBLEM - Puppet failure on nodepool-t105406 is CRITICAL 100.00% of data above the critical threshold [0.0] [20:13:46] (03CR) 10Hashar: [C: 032] Update TimedMediaHandler tests [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox) [20:15:00] (03PS3) 10Hashar: Update TimedMediaHandler tests [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox) [20:15:23] (03CR) 10Hashar: "The template was listed in the list of jobs... Was not going to work." [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox) [20:17:49] (03CR) 10Hashar: [C: 032] Update TimedMediaHandler tests [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox) [20:19:39] (03Merged) 10jenkins-bot: Update TimedMediaHandler tests [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox) [20:22:15] (03PS4) 10Hashar: tests: refactor test_repos_have_required_jobs [integration/config] - 10https://gerrit.wikimedia.org/r/224790 [20:26:09] (03CR) 10Hashar: [C: 032] "Lets unleash that. It has the same coverage albeit an order of magnitude faster." [integration/config] - 10https://gerrit.wikimedia.org/r/224790 (owner: 10Hashar) [20:26:17] (03PS7) 10Hashar: tests: consistency of Zuul pipelines declarations [integration/config] - 10https://gerrit.wikimedia.org/r/224517 [20:27:23] (03Merged) 10jenkins-bot: tests: refactor test_repos_have_required_jobs [integration/config] - 10https://gerrit.wikimedia.org/r/224790 (owner: 10Hashar) [20:27:55] (03CR) 10Hashar: "What do you guys think about this test? It caught a few inconsistencies that I fixed already." (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/224517 (owner: 10Hashar) [21:19:43] (03PS5) 10Thcipriani: Add service deploy via scap [tools/scap] - 10https://gerrit.wikimedia.org/r/224374 [21:20:39] 6Release-Engineering, 10Wikimedia-Language-setup, 10Wikimedia-Site-requests, 6operations: nb subdomain redirects - https://phabricator.wikimedia.org/T86924#1462694 (10Krenair) And also nb.wikinews.org -> no.wikinews.org? [23:31:44] PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL 100.00% of data above the critical threshold [0.0] [23:35:08] 6Release-Engineering, 10Wikimedia-Language-setup, 6operations, 5Patch-For-Review: nb subdomain redirects - https://phabricator.wikimedia.org/T86924#1462962 (10Krenair) [23:38:35] (03CR) 10Krinkle: "fixme: TimedMediaHandler builds are now broken. The extension is not yet compatible with the generic mwext jobs. See https://gerrit.wikime" [integration/config] - 10https://gerrit.wikimedia.org/r/225246 (owner: 10Paladox)