[14:18:24] wikimedia/articlequality#171 (extract_scores_updates - 4ccdf24 : Aaron Halfaker): The build failed. https://travis-ci.org/wikimedia/articlequality/builds/545773864 [15:15:31] wikimedia/articlequality#173 (extract_scores_updates - 73e253c : Aaron Halfaker): The build was fixed. https://travis-ci.org/wikimedia/articlequality/builds/545796997 [15:32:45] o/ harej [15:32:48] Standup! [19:28:07] wikimedia/articlequality#176 (extract_scores_updates - 93a6577 : Aaron Halfaker): The build was broken. https://travis-ci.org/wikimedia/articlequality/builds/545896305 [21:11:36] wikimedia/articlequality#178 (extract_scores_updates - 6a90080 : Aaron Halfaker): The build was fixed. https://travis-ci.org/wikimedia/articlequality/builds/545935075 [22:01:37] PROBLEM - check disk on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [22:01:49] PROBLEM - check load on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [22:01:50] hmm [22:02:03] PROBLEM - check users on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [22:02:52] Oooh. Labs is getting hammered. :) [22:03:04] I don't know if this is related or not. [22:04:01] Looks like web-01 is online. [22:04:02] Hmm [22:04:30] Is icinga2 having an issue? [22:05:21] PROBLEM - puppet on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [22:05:39] paladox, any thoughts about ^ [22:05:54] I'm ssh'd into ores-web-01 now and I can make requests to it locally. Seems fine. [22:05:57] I think you need to restart nagios-nrpe-server [22:06:18] sudo service nagios-nrpe-server restart [22:06:21] should do it [22:06:47] OK done [22:06:51] * halfak waits [22:07:04] RECOVERY - puppet on ORES-web01.Experimental is OK: OK: Puppet is currently enabled, last run 13 minutes ago with 0 failures [22:07:09] RECOVERY - check users on ORES-web01.Experimental is OK: USERS OK - 2 users currently logged in [22:07:11] RECOVERY - check load on ORES-web01.Experimental is OK: OK - load average: 0.11, 0.11, 0.09 [22:07:14] RECOVERY - check disk on ORES-web01.Experimental is OK: DISK OK [22:10:42] I'm heading out early [22:10:53] realized I could have done everything with tags! [22:11:11] pthbth