[00:31:16] <shinken-wm>	 PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[06:54:02] <shinken-wm>	 PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[07:08:30] <shinken-wm>	 RECOVERY - Free space - all mounts on deployment-kafka01 is OK: OK: All targets OK
[07:29:02] <shinken-wm>	 RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0]
[10:50:02] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10releng-201516-q3, 10Developer-Relations, and 3 others: [RfC]: Migrate code review / management from Gerrit to Phabricator - https://phabricator.wikimedia.org/T119908#3899243 (10demon) >>! In T119908#3898613, @HappyDog wrote: > As for the current version of Gerrit bei...
[10:52:55] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Customize Gerrit's login UI - https://phabricator.wikimedia.org/T184778#3899256 (10demon) >>! In T184778#3898351, @Peachey88 wrote: > Since this isn't the default/upstream themeing, is it a level of tech debt we should be adding to gerrit as it become...
[11:06:59] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10releng-201516-q3, 10Developer-Relations, and 3 others: [RfC]: Migrate code review / management from Gerrit to Phabricator - https://phabricator.wikimedia.org/T119908#3899271 (10Paladox) @HappyDog i've brought the discussion to https://phabricator.wikimedia.org/conphe...
[12:11:11] <shinken-wm>	 PROBLEM - Puppet errors on saucelabs-01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[12:28:54] <wikibugs>	 10Gerrit: @Eisenhaus335 probably needs some help over at Gerrit - https://phabricator.wikimedia.org/T183797#3899314 (10Eisenhaus335) i dont read email and this forum before. just notice when reading email. i am dont know how gerrit and especially git work. i am just know how to newly know how to use git review....
[12:46:11] <shinken-wm>	 RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0]
[13:36:37] <wikibugs>	 10Gerrit: @Eisenhaus335 probably needs some help over at Gerrit - https://phabricator.wikimedia.org/T183797#3899362 (10Aklapper) @Eisenhaus335: This "forum" is the bug tracker. See https://www.mediawiki.org/wiki/Phabricator/Help For using Gerrit, see https://www.mediawiki.org/wiki/Gerrit/Tutorial - thanks!
[14:06:44] <wikibugs>	 10Gerrit: @Eisenhaus335 probably needs some help over at Gerrit - https://phabricator.wikimedia.org/T183797#3863984 (10Zoranzoki21) >>! In T183797#3899314, @Eisenhaus335 wrote: > i dont read email and this forum before. just notice when reading email. > i am dont know how gerrit and especially git work. i am jus...
[14:53:16] <shinken-wm>	 PROBLEM - Host deployment-puppetdb01 is DOWN: CRITICAL - Host Unreachable (10.68.23.76)
[15:22:29] <wikibugs>	 (03PS1) 10Rafidaslam: Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155
[15:22:31] <wikibugs>	 (03PS1) 10Rafidaslam: Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905)
[15:22:39] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905) (owner: 10Rafidaslam)
[15:22:41] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155 (owner: 10Rafidaslam)
[16:05:31] <wikibugs>	 (03PS2) 10Rafidaslam: Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155
[16:05:33] <wikibugs>	 (03PS2) 10Rafidaslam: Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905)
[16:06:10] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905) (owner: 10Rafidaslam)
[16:06:21] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155 (owner: 10Rafidaslam)
[16:11:56] <wikibugs>	 (03PS3) 10Rafidaslam: Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155
[16:11:58] <wikibugs>	 (03PS3) 10Rafidaslam: Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905)
[16:12:33] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905) (owner: 10Rafidaslam)
[16:12:42] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155 (owner: 10Rafidaslam)
[16:28:26] <wikibugs>	 (03PS4) 10Rafidaslam: Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155
[16:28:28] <wikibugs>	 (03PS4) 10Rafidaslam: Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905)
[16:47:33] <wikibugs>	 (03PS5) 10Rafidaslam: Make MessageValidator an abstract class [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404155
[16:47:35] <wikibugs>	 (03PS5) 10Rafidaslam: Add GitHubMessageValidator and add logic to switch the MessageValidator [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404156 (https://phabricator.wikimedia.org/T179905)
[17:28:46] <shinken-wm>	 PROBLEM - Puppet errors on deployment-snapshot01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[18:05:22] <apergos>	 still waiting on wikidiff2 package
[18:05:42] <apergos>	 at least I have not yet seen an upload of it to the repo
[18:54:29] <shinken-wm>	 PROBLEM - Puppet errors on deployment-secureredirexperiment is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[20:07:03] <wikibugs>	 (03PS1) 10Zoranzoki21: Run phan for CologneBlue and MonoBook skins [integration/config] - 10https://gerrit.wikimedia.org/r/404172 (https://phabricator.wikimedia.org/T179554)
[20:10:17] <wikibugs>	 (03CR) 10Zoranzoki21: "check experimental" [integration/config] - 10https://gerrit.wikimedia.org/r/404172 (https://phabricator.wikimedia.org/T179554) (owner: 10Zoranzoki21)
[20:40:49] <wikibugs>	 (03CR) 10Zoranzoki21: "check experimental" [integration/config] - 10https://gerrit.wikimedia.org/r/404172 (https://phabricator.wikimedia.org/T179554) (owner: 10Zoranzoki21)
[21:08:22] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mx02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]
[21:41:44] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap: Scap: TypeError: error: (not all arguments converted during string formatting); format string: (Passed unrecognized git_binary_manager {}); arguments: ((u'fat',)) - https://phabricator.wikimedia.org/T184882#3899618 (10thcipriani)
[21:43:00] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap: Scap: TypeError: error: (not all arguments converted during string formatting); format string: (Passed unrecognized git_binary_manager {}); arguments: ((u'fat',)) - https://phabricator.wikimedia.org/T184882#3899627 (10thcipriani)
[21:44:30] <shinken-wm>	 PROBLEM - Free space - all mounts on deployment-kafka01 is CRITICAL: CRITICAL: deployment-prep.deployment-kafka01.diskspace.root.byte_percentfree (<44.44%)
[22:04:45] <wikibugs>	 10Scap, 10Patch-For-Review: Update Debian Package for Scap3 - https://phabricator.wikimedia.org/T127762#3899642 (10thcipriani) 05Resolved>03Open a:05fgiunchedi>03None Hello, I just pushed new scap tags for version `3.7.6`/`debian/3.7.6-1`. This has the fix for T184882.   Is there an opsen who can updat...
[22:11:52] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap: Scap: TypeError: error: (not all arguments converted during string formatting); format string: (Passed unrecognized git_binary_manager {}); arguments: ((u'fat',)) - https://phabricator.wikimedia.org/T184882#3899649 (10thcipriani) p:05Triage>03Unbreak! This error...
[22:13:47] <shinken-wm>	 PROBLEM - Puppet errors on deployment-aqs02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[22:14:02] <shinken-wm>	 PROBLEM - Puppet errors on deployment-tmh01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[22:14:20] <shinken-wm>	 PROBLEM - Puppet errors on deployment-cassandra3-02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[22:15:01] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:17:03] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mathoid is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0]
[22:17:34] <shinken-wm>	 PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]
[22:17:52] <shinken-wm>	 PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:18:29] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mediawiki06 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[22:18:47] <paladox>	 Hmm i wonder why those started failing ^^.
[22:19:04] <thcipriani>	 I would guess it's because of me pushing to the release branch of scap
[22:19:38] <thcipriani>	 T184118
[22:19:38] <stashbot>	 T184118: scap package installed by CI breaks apt and thus puppet - https://phabricator.wikimedia.org/T184118
[22:19:40] <shinken-wm>	 PROBLEM - Puppet errors on deployment-jobrunner02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:19:44] * thcipriani fixes
[22:20:02] <shinken-wm>	 PROBLEM - Puppet errors on deployment-eventlog02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]
[22:22:42] <paladox>	 oh
[22:24:25] <shinken-wm>	 PROBLEM - Puppet errors on deployment-restbase02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:24:35] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mediawiki05 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:25:04] <thcipriani>	 alright, just pushed a README change to scap master, should make puppet on beta happy-ish.
[22:26:10] <paladox>	 thanks.
[22:28:08] <shinken-wm>	 PROBLEM - Puppet errors on deployment-pdfrender02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:28:24] <shinken-wm>	 PROBLEM - Puppet errors on deployment-restbase01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0]
[22:28:30] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mediawiki04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[22:29:40] <shinken-wm>	 PROBLEM - Puppet errors on deployment-ores01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0]
[22:29:56] <shinken-wm>	 PROBLEM - Puppet errors on deployment-mcs01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0]
[22:30:51] <shinken-wm>	 PROBLEM - Puppet errors on deployment-zotero01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0]
[22:32:05] <shinken-wm>	 PROBLEM - Puppet errors on deployment-sca03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0]
[22:34:26] <thcipriani>	 double-checked on deployment-mediawiki05, looks to be fixed, should start seeing recoveries at some point.
[22:44:33] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mediawiki05 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:49:59] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0]
[22:52:33] <shinken-wm>	 RECOVERY - Puppet errors on deployment-imagescaler01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:47] <shinken-wm>	 RECOVERY - Puppet errors on deployment-aqs02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:53:59] <shinken-wm>	 RECOVERY - Puppet errors on deployment-tmh01 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:54:17] <shinken-wm>	 RECOVERY - Puppet errors on deployment-cassandra3-02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:04] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0]
[22:57:54] <shinken-wm>	 RECOVERY - Puppet errors on deployment-tin is OK: OK: Less than 1.00% above the threshold [0.0]
[22:58:26] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mediawiki06 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:59:26] <shinken-wm>	 RECOVERY - Puppet errors on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0]
[22:59:42] <shinken-wm>	 RECOVERY - Puppet errors on deployment-jobrunner02 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:00:02] <shinken-wm>	 RECOVERY - Puppet errors on deployment-eventlog02 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:03:25] <shinken-wm>	 RECOVERY - Puppet errors on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:03:29] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:05:51] <shinken-wm>	 RECOVERY - Puppet errors on deployment-zotero01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:07:05] <shinken-wm>	 RECOVERY - Puppet errors on deployment-sca03 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:08:05] <shinken-wm>	 RECOVERY - Puppet errors on deployment-pdfrender02 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:09:42] <shinken-wm>	 RECOVERY - Puppet errors on deployment-ores01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:09:58] <shinken-wm>	 RECOVERY - Puppet errors on deployment-mcs01 is OK: OK: Less than 1.00% above the threshold [0.0]
[23:52:43] <wikibugs>	 10Release-Engineering-Team (Kanban), 10Scap, 10Patch-For-Review: Scap: TypeError: error: (not all arguments converted during string formatting); format string: (Passed unrecognized git_binary_manager {}); arguments: ((u'fat',)) - https://phabricator.wikimedia.org/T184882#3899710 (10thcipriani) p:05Unbreak!...