[00:03:57] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:07:14] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[00:07:48] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:14:03] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[00:14:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[00:16:28] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:18:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[00:20:10] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[00:21:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[00:25:38] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[00:32:19] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:42:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[00:45:09] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:48:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[00:49:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[00:54:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[00:56:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:04:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:06:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:07:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:10:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:11:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[01:15:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[01:16:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[01:18:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[01:19:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:24:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:30:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[01:35:56] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[01:36:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:41:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:45:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:48:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[01:57:23] <shinken-wm>	 PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[01:57:37] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[01:59:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[02:05:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[02:05:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:07:31] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[02:07:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[02:20:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:22:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:24:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:30:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:31:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[02:37:30] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:37:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:47:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[02:50:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[02:55:37] <wmf-insecte>	 Project beta-scap-eqiad build #36489: FAILURE in 1 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36489/
[02:56:56] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[02:58:38] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[02:59:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[03:01:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[03:01:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:05:45] <wmf-insecte>	 Yippee, build fixed!
[03:05:45] <wmf-insecte>	 Project beta-scap-eqiad build #36490: FIXED in 1 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36490/
[03:15:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:17:13] <wmf-insecte>	 Yippee, build fixed!
[03:17:14] <wmf-insecte>	 Project browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce build #262: FIXED in 11 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce/262/
[03:20:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[03:21:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:26:56] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:28:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:29:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:30:32] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[03:45:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[03:48:01] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[03:52:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[03:54:26] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[03:54:55] <wmf-insecte>	 Yippee, build fixed!
[03:54:56] <wmf-insecte>	 Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce build #392: FIXED in 7 min 25 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/392/
[03:57:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[04:00:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[04:04:04] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[04:08:45] <wmf-insecte>	 Yippee, build fixed!
[04:08:46] <wmf-insecte>	 Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #452: FIXED in 51 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/452/
[04:15:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:17:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:17:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:19:24] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:22:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:26:49] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mathoid is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[04:29:01] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:29:15] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[04:30:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:35:02] <wmf-insecte>	 Project beta-scap-eqiad build #36499: FAILURE in 1 min 7 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36499/
[04:38:27] <wmf-insecte>	 Yippee, build fixed!
[04:38:28] <wmf-insecte>	 Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #211: FIXED in 43 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/211/
[04:45:16] <wmf-insecte>	 Project beta-scap-eqiad build #36500: STILL FAILING in 1 min 20 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36500/
[04:54:16] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[04:55:10] <wmf-insecte>	 Yippee, build fixed!
[04:55:11] <wmf-insecte>	 Project beta-scap-eqiad build #36501: FIXED in 1 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36501/
[04:56:48] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]  
[05:07:57] <wmf-insecte>	 Yippee, build fixed!
[05:07:57] <wmf-insecte>	 Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #426: FIXED in 43 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/426/
[05:08:25] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[05:21:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[05:38:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[05:39:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cxserver03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[05:41:10] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[05:46:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[05:56:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[05:57:40] <wmf-insecte>	 Yippee, build fixed!
[05:57:41] <wmf-insecte>	 Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #370: FIXED in 48 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/370/
[05:57:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[06:06:13] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:06:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[06:09:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cxserver03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:12:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[06:17:11] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[06:22:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:26:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:35:03] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[06:36:35] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK  
[06:41:08] <wmf-insecte>	 Yippee, build fixed!
[06:41:08] <wmf-insecte>	 Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #369: FIXED in 43 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/369/
[06:47:14] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:48:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[06:48:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[06:49:49] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[06:51:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[06:51:58] <shinken-wm>	 PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[06:55:31] <wmf-insecte>	 Project beta-scap-eqiad build #36514: FAILURE in 1 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36514/
[06:57:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[06:57:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:05:01] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:05:52] <wmf-insecte>	 Project beta-scap-eqiad build #36515: STILL FAILING in 1 min 47 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36515/
[07:06:03] <wmf-insecte>	 Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #399: FAILURE in 54 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/399/
[07:06:27] <wmf-insecte>	 Yippee, build fixed!
[07:06:27] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #381: FIXED in 25 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/381/
[07:08:15] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[07:14:46] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:15:27] <wmf-insecte>	 Yippee, build fixed!
[07:15:28] <wmf-insecte>	 Project beta-scap-eqiad build #36516: FIXED in 1 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36516/
[07:18:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:18:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:21:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:27:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:33:13] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[07:46:11] <wmf-insecte>	 Project beta-scap-eqiad build #36519: FAILURE in 2 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36519/
[07:55:34] <wmf-insecte>	 Yippee, build fixed!
[07:55:34] <wmf-insecte>	 Project beta-scap-eqiad build #36520: FIXED in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36520/
[08:02:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[08:13:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[08:25:15] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[08:27:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[08:28:29] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[08:29:31] <shinken-wm>	 PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds  
[08:34:22] <shinken-wm>	 RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 49884 bytes in 1.299 second response time  
[08:43:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[08:53:30] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[08:55:16] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[08:55:51] <wmf-insecte>	 Project beta-scap-eqiad build #36526: FAILURE in 1 min 55 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36526/
[08:57:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[09:05:32] <wmf-insecte>	 Yippee, build fixed!
[09:05:33] <wmf-insecte>	 Project beta-scap-eqiad build #36527: FIXED in 1 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36527/
[09:07:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[09:09:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[09:13:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[09:17:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[09:22:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[09:29:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[09:30:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[09:32:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[09:34:33] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[09:38:22] <shinken-wm>	 PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[09:38:46] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[09:40:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[09:42:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[09:55:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:03:23] <shinken-wm>	 RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:05:37] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:06:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[10:09:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[10:09:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[10:15:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[10:16:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[10:18:14] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[10:19:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:21:37] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[10:28:21] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[10:31:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:39:27] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:39:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:40:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:41:03] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:46:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:48:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:50:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[10:51:50] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[10:52:22] <shinken-wm>	 PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[10:54:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[10:57:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[10:58:21] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[10:58:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[11:02:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[11:08:58] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[11:09:20] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[11:13:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[11:16:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:20:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:22:19] <shinken-wm>	 RECOVERY - Puppet failure on deployment-apertium01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:23:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:24:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:26:09] <wmf-insecte>	 Project beta-scap-eqiad build #36541: FAILURE in 1 min 49 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36541/
[11:27:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:27:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:28:09] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoidcache02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[11:30:27] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[11:30:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[11:34:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:35:55] <wmf-insecte>	 Yippee, build fixed!
[11:35:55] <wmf-insecte>	 Project beta-scap-eqiad build #36542: FIXED in 1 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36542/
[11:38:51] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:38:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[11:38:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:42:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[11:49:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[11:52:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[11:53:12] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoidcache02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:55:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[11:55:58] <wmf-insecte>	 Project beta-scap-eqiad build #36544: FAILURE in 1 min 58 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36544/
[11:56:27] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[11:58:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[12:00:31] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:03:56] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:04:10] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoidcache02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[12:05:38] <wmf-insecte>	 Yippee, build fixed!
[12:05:39] <wmf-insecte>	 Project beta-scap-eqiad build #36545: FIXED in 1 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36545/
[12:11:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[12:12:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:18:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[12:21:26] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:28:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:31:31] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[12:31:49] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[12:34:12] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoidcache02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:34:58] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[12:37:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:38:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[12:39:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[12:39:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:48:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:50:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[12:50:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[12:52:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[12:56:28] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[12:56:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:04:23] <shinken-wm>	 RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:04:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[13:06:19] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[13:07:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[13:08:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:12:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[13:15:00] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:15:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:16:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:20:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:22:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:24:35] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[13:24:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:25:31] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[13:25:38] <wmf-insecte>	 Project beta-scap-eqiad build #36553: FAILURE in 1 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36553/
[13:34:53] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:35:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[13:36:12] <wmf-insecte>	 Yippee, build fixed!
[13:36:12] <wmf-insecte>	 Project beta-scap-eqiad build #36554: FIXED in 2 min 18 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36554/
[13:36:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:37:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:37:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:39:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[13:39:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[13:43:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[13:48:58] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[13:49:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:51:25] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[13:52:27] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[13:52:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[13:53:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[13:53:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[13:53:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[13:55:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[13:56:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[14:00:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:05:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[14:06:30] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[14:08:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:09:13] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:09:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[14:13:57] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:16:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:17:26] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:17:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:18:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:19:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[14:19:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[14:23:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:23:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:26:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:28:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[14:28:47] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[14:29:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:29:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[14:29:49] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[14:30:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:34:14] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:36:31] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:37:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[14:40:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[14:43:32] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[14:44:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:44:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:53:35] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sentry2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[14:58:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:58:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:59:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[14:59:48] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:00:10] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[15:00:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[15:02:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[15:05:12] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[15:05:48] <wmf-insecte>	 Project beta-scap-eqiad build #36563: FAILURE in 1 min 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36563/
[15:07:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mathoid is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[15:07:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:09:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[15:10:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:16:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[15:16:58] <wmf-insecte>	 Yippee, build fixed!
[15:16:58] <wmf-insecte>	 Project beta-scap-eqiad build #36564: FIXED in 2 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36564/
[15:18:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[15:20:35] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[15:20:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cxserver03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[15:22:04] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:25:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[15:25:09] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:30:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:31:01] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[15:33:03] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[15:33:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:34:52] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[15:35:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[15:37:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:39:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[15:39:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:40:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:41:33] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:50:00] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:50:36] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[15:51:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[15:51:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[15:56:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[15:56:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[15:59:38] <YuviPanda>	 ysigh
[16:01:01] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:03:03] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:03:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:04:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[16:04:49] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:06:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[16:06:58] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[16:10:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cxserver03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:12:56] <shinken-wm>	 PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[16:13:16] <shinken-wm>	 PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[16:14:00] <wmf-insecte>	 Project beta-code-update-eqiad build #38933: FAILURE in 59 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/38933/
[16:14:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[16:15:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[16:16:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:16:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:19:57] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[16:21:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:25:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:26:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:27:30] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[16:30:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[16:31:16] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:34:03] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[16:34:34] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:36:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:37:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[16:37:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:38:16] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic05 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[16:39:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:40:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:41:35] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[16:41:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[16:44:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:44:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[16:47:34] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[16:48:00] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[16:52:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[16:53:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[16:55:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[16:56:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[16:57:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[16:58:17] <shinken-wm>	 RECOVERY - Puppet failure on deployment-apertium01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:00:51] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:01:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[17:01:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[17:02:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:03:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:03:55] <wmf-insecte>	 Yippee, build fixed!
[17:03:55] <wmf-insecte>	 Project beta-code-update-eqiad build #38938: FIXED in 54 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/38938/
[17:04:03] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:05:21] <wmf-insecte>	 Project beta-scap-eqiad build #36570: FAILURE in 1 min 26 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36570/
[17:06:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:07:20] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[17:11:57] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:15:58] <wmf-insecte>	 Yippee, build fixed!
[17:15:58] <wmf-insecte>	 Project beta-scap-eqiad build #36571: FIXED in 1 min 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36571/
[17:17:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:17:30] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:17:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:17:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:18:26] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:20:27] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:21:11] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[17:21:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:26:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:27:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:29:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[17:29:48] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[17:32:20] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:32:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[17:35:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[17:37:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[17:42:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[17:45:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[17:46:10] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[17:52:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[17:55:05] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[17:55:55] <wmf-insecte>	 Project beta-scap-eqiad build #36575: FAILURE in 1 min 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36575/
[17:56:26] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[17:59:26] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:02:14] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 77.78% of data above the critical threshold [0.0]  
[18:02:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:05:49] <wmf-insecte>	 Project beta-scap-eqiad build #36576: STILL FAILING in 1 min 56 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36576/
[18:07:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:08:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[18:10:36] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:13:59] <shinken-wm>	 PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[18:15:38] <wmf-insecte>	 Project beta-scap-eqiad build #36577: STILL FAILING in 1 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36577/
[18:15:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[18:17:25] <wmf-insecte>	 Project browsertests-VisualEditor-language-screenshot-linux-firefox » gl,contintLabsSlave && UbuntuTrusty build #3: FAILURE in 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox/LANGUAGE_SCREENSHOT_CODE=gl,label=contintLabsSlave%20&&%20UbuntuTrusty/3/
[18:18:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[18:19:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[18:19:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:20:05] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:21:05] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #174: FAILURE in 19 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/174/
[18:21:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:22:33] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:23:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[18:25:44] <wmf-insecte>	 Yippee, build fixed!
[18:25:45] <wmf-insecte>	 Project beta-scap-eqiad build #36578: FIXED in 1 min 50 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36578/
[18:25:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:26:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[18:26:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:27:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:27:15] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[18:27:37] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:29:01] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[18:33:22] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[18:33:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:36:05] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[18:37:23] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[18:38:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[18:38:57] <shinken-wm>	 RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:43:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:45:35] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #228: FAILURE in 38 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/228/
[18:45:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:45:55] <wmf-insecte>	 Project beta-scap-eqiad build #36580: FAILURE in 2 min 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36580/
[18:48:15] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[18:48:51] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mathoid is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[18:49:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:51:37] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:52:13] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:53:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:53:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[18:55:41] <wmf-insecte>	 Yippee, build fixed!
[18:55:42] <wmf-insecte>	 Project beta-scap-eqiad build #36581: FIXED in 1 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36581/
[18:58:24] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:58:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[18:58:59] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[18:59:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[19:01:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[19:02:24] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:02:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[19:07:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[19:08:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:09:31] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[19:10:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[19:10:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[19:13:15] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:13:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:14:18] <shinken-wm>	 PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[19:16:09] <wmf-insecte>	 Project beta-scap-eqiad build #36583: FAILURE in 2 min 7 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36583/
[19:19:09] <wmf-insecte>	 Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #453: FAILURE in 59 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/453/
[19:21:04] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:23:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:23:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:25:43] <wmf-insecte>	 Project beta-scap-eqiad build #36584: STILL FAILING in 1 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36584/
[19:28:34] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[19:32:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:34:33] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:34:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[19:35:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:35:59] <wmf-insecte>	 Yippee, build fixed!
[19:35:59] <wmf-insecte>	 Project beta-scap-eqiad build #36585: FIXED in 2 min 3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36585/
[19:39:17] <shinken-wm>	 RECOVERY - Puppet failure on deployment-apertium01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:39:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[19:40:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:41:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:41:57] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [0.0]  
[19:46:45] <wmf-insecte>	 Project beta-scap-eqiad build #36586: FAILURE in 2 min 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36586/
[19:47:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:50:00] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[19:50:34] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[19:57:01] <wmf-insecte>	 Yippee, build fixed!
[19:57:01] <wmf-insecte>	 Project beta-scap-eqiad build #36587: FIXED in 3 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36587/
[19:58:32] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]  
[19:59:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:02:03] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]  
[20:04:49] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:04:49] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:09:56] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:10:28] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[20:12:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:14:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[20:15:29] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:15:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:24:03] <wmf-insecte>	 Yippee, build fixed!
[20:24:03] <wmf-insecte>	 Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #506: FIXED in 58 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/506/
[20:26:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:29:43] <wmf-insecte>	 Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #427: FAILURE in 46 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/427/
[20:29:57] <shinken-wm>	 PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[20:30:19] <shinken-wm>	 PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[20:31:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:36:47] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:37:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[20:37:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:38:09] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[20:39:59] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:40:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:42:05] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:43:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[20:43:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[20:44:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:45:28] <wmf-insecte>	 Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #431: FAILURE in 21 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/431/
[20:46:31] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[20:53:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[20:54:14] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic05 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[20:54:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:56:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[20:57:53] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[20:58:01] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[20:59:57] <shinken-wm>	 RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:00:22] <shinken-wm>	 RECOVERY - Puppet failure on deployment-apertium01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:00:28] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:00:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[21:01:50] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:07:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[21:08:12] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:08:37] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:09:25] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[21:09:51] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mathoid is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[21:10:45] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:10:55] <shinken-wm>	 PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[21:10:59] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[21:11:26] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]  
[21:11:52] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:12:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[21:16:29] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:18:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:20:42] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:21:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-redis02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[21:22:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:22:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[21:22:56] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:23:06] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:24:14] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic05 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:25:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:28:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[21:29:33] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:34:25] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[21:35:23] <shinken-wm>	 PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[21:35:59] <shinken-wm>	 RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:35:59] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:36:09] <wmf-insecte>	 Project beta-scap-eqiad build #36597: FAILURE in 2 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36597/
[21:37:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:38:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:38:56] <shinken-wm>	 PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:39:18] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:39:26] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:39:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[21:40:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:40:46] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:41:27] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:41:49] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:48:38] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[21:48:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:49:11] <wmf-insecte>	 Yippee, build fixed!
[21:49:12] <wmf-insecte>	 Project beta-scap-eqiad build #36598: FIXED in 4 min 38 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36598/
[21:51:00] <shinken-wm>	 RECOVERY - Puppet failure on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:51:46] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[21:51:47] <shinken-wm>	 PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[21:52:27] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[21:52:50] <shinken-wm>	 PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[21:52:54] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:53:18] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:54:04] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[21:55:55] <wmf-insecte>	 Project beta-scap-eqiad build #36599: FAILURE in 1 min 56 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36599/
[21:59:23] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:59:35] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0]  
[21:59:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[22:03:42] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:03:54] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[22:05:48] <wmf-insecte>	 Yippee, build fixed!
[22:05:49] <wmf-insecte>	 Project beta-scap-eqiad build #36600: FIXED in 1 min 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36600/
[22:09:10] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:09:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:10:47] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:12:43] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[22:14:12] <wmf-insecte>	 Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #227: FAILURE in 36 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/227/
[22:14:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[22:14:51] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:17:31] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:17:51] <shinken-wm>	 RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:18:50] <wmf-insecte>	 Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #370: FAILURE in 48 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/370/
[22:20:12] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[22:20:36] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[22:21:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[22:22:48] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:24:02] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:25:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:25:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:30:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]  
[22:30:28] <wmf-insecte>	 Project beta-scap-eqiad build #36602: FAILURE in 1 min 24 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36602/
[22:33:56] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:35:29] <wmf-insecte>	 Yippee, build fixed!
[22:35:30] <wmf-insecte>	 Project beta-scap-eqiad build #36603: FIXED in 1 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36603/
[22:36:41] <wmf-insecte>	 Yippee, build fixed!
[22:36:41] <wmf-insecte>	 Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #400: FIXED in 57 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/400/
[22:36:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:42:43] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:44:39] <shinken-wm>	 RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:45:10] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:45:38] <shinken-wm>	 RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:46:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:47:00] <shinken-wm>	 PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]  
[22:49:40] <shinken-wm>	 RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:55:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[22:55:35] <wmf-insecte>	 Project beta-scap-eqiad build #36605: FAILURE in 1 min 41 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36605/
[22:58:41] <shinken-wm>	 RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:04:14] <shinken-wm>	 PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]  
[23:05:38] <wmf-insecte>	 Yippee, build fixed!
[23:05:39] <wmf-insecte>	 Project beta-scap-eqiad build #36606: FIXED in 1 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36606/
[23:08:26] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [0.0]  
[23:12:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[23:13:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[23:15:10] <wmf-insecte>	 Project beta-scap-eqiad build #36607: FAILURE in 1 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36607/
[23:16:44] <shinken-wm>	 RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:16:58] <shinken-wm>	 RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:23:55] <shinken-wm>	 RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:25:13] <wmf-insecte>	 Project beta-scap-eqiad build #36608: STILL FAILING in 1 min 18 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36608/
[23:34:18] <shinken-wm>	 RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:35:27] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[23:35:35] <wmf-insecte>	 Yippee, build fixed!
[23:35:35] <wmf-insecte>	 Project beta-scap-eqiad build #36609: FIXED in 1 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/36609/
[23:35:41] <shinken-wm>	 PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]  
[23:37:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:38:25] <shinken-wm>	 RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:38:45] <shinken-wm>	 RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0]  
[23:41:13] <shinken-wm>	 PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]  
[23:42:44] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0]  
[23:46:30] <shinken-wm>	 PROBLEM - Free space - all mounts on deployment-cache-upload02 is CRITICAL: CRITICAL: deployment-prep.deployment-cache-upload02.diskspace._srv_vdb.byte_percentfree.value (<100.00%)  
[23:49:40] <shinken-wm>	 PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0]  
[23:51:24] <shinken-wm>	 PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]  
[23:55:02] <shinken-wm>	 PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]  
[23:56:39] <shinken-wm>	 PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]