[00:01:13] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 14 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [00:41:18] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 24 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [01:21:22] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [02:01:27] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [02:41:32] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 24 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [03:21:37] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [04:01:37] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [04:41:37] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [05:21:37] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [06:01:42] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [06:41:42] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [07:21:42] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [08:01:47] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [08:41:52] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [09:16:01] 10Scoring-platform-team, 10Wikilabels: Write deployment guide for Wikilabels - https://phabricator.wikimedia.org/T194809#4209527 (10awight) [09:21:57] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [09:29:30] 10Scoring-platform-team (Current), 10ORES: ores-beta lacking a firewall rule to allow port 8081 - https://phabricator.wikimedia.org/T194603#4209581 (10awight) Weird punchline... I added another of the production roles cos we think there are messy interdependencies, and after rebooting, the result was *no* ipt... [09:37:18] 10Scoring-platform-team, 10Wikimedia-Hackathon-2018, 10Documentation: Create more visual aids for ORES - https://phabricator.wikimedia.org/T194739#4209596 (10awight) [09:37:20] 10Scoring-platform-team, 10ORES: Model information UI (graphs and statistics) - https://phabricator.wikimedia.org/T140364#4209599 (10awight) [09:38:48] 10Scoring-platform-team, 10ORES: Model information UI (graphs and statistics) - https://phabricator.wikimedia.org/T140364#2462303 (10awight) Here's a tiny bit of code I put together for P-R graphs specifically, https://github.com/adamwight/thresholds_diagrams [09:39:01] 10Scoring-platform-team, 10ORES, 10Wikimedia-Hackathon-2018: Model information UI (graphs and statistics) - https://phabricator.wikimedia.org/T140364#4209612 (10awight) [10:02:02] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [10:42:07] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [11:05:05] There are non-english articles in the english-only version of wikipedia dumps from dumps.wikimedia.org/enwiki. [11:22:12] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [12:02:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [12:14:13] 10Scoring-platform-team, 10Wikidata, 10Wikimedia-Hackathon-2018, 10WMDE-Tech-Communication-Mentoring-And-Events: Finding items on Wikidata that should be merged - https://phabricator.wikimedia.org/T127467#4209992 (10Bmueller) [12:42:21] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [12:45:14] (03CR) 10Thiemo Kreuz (WMDE): [C: 032] Adjusting the punctuation. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/365544 (owner: 10Felipe L. Ewald) [12:53:09] (03CR) 10jerkins-bot: [V: 04-1] Adjusting the punctuation. [extensions/ORES] - 10https://gerrit.wikimedia.org/r/365544 (owner: 10Felipe L. Ewald) [13:01:09] o/ [13:01:17] We're connecting! [13:01:37] o? [13:01:45] *o/ [13:02:39] It's great that o? looks like someone scratching their head questioning something :) [13:11:55] (03CR) 10Ladsgroup: [C: 032] Don't expect objects by reference in hook handlers [extensions/ORES] - 10https://gerrit.wikimedia.org/r/433173 (owner: 10Thiemo Kreuz (WMDE)) [13:17:37] (03Merged) 10jenkins-bot: Don't expect objects by reference in hook handlers [extensions/ORES] - 10https://gerrit.wikimedia.org/r/433173 (owner: 10Thiemo Kreuz (WMDE)) [13:22:26] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [13:35:05] (03CR) 10jenkins-bot: Don't expect objects by reference in hook handlers [extensions/ORES] - 10https://gerrit.wikimedia.org/r/433173 (owner: 10Thiemo Kreuz (WMDE)) [14:02:31] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [14:42:36] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [16:02:01] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [16:42:01] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [16:42:34] PROBLEM - puppet on ORES-worker05.experimental is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [17:11:04] RECOVERY - puppet on ORES-worker05.experimental is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [17:22:01] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [18:02:06] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [18:42:06] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [19:22:11] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [20:02:17] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [20:42:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 25 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [21:22:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [22:02:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 15 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [22:42:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 26 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils] [23:22:16] PROBLEM - puppet on ORES-lb02.Experimental is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[ldap-utils]