[00:45:53] PROBLEM - Blazegraph process on wdqs1003 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 997 (blazegraph), regex args ^java .* blazegraph-service-.*war [00:46:14] PROBLEM - Blazegraph Port on wdqs1003 is CRITICAL: connect to address 127.0.0.1 and port 9999: Connection refused [00:46:14] PROBLEM - WDQS HTTP Port on wdqs1003 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.001 second response time [00:46:56] RECOVERY - Blazegraph process on wdqs1003 is OK: PROCS OK: 1 process with UID = 997 (blazegraph), regex args ^java .* blazegraph-service-.*war [00:47:14] RECOVERY - Blazegraph Port on wdqs1003 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 9999 [00:47:14] RECOVERY - WDQS HTTP Port on wdqs1003 is OK: HTTP OK: HTTP/1.1 200 OK - 434 bytes in 0.023 second response time [01:24:46] PROBLEM - High lag on wdqs1003 is CRITICAL: CRITICAL: 36.67% of data above the critical threshold [1800.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [02:17:03] RECOVERY - High lag on wdqs1003 is OK: OK: Less than 30.00% above the threshold [600.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [06:26:24] Change on 12www.mediawiki.org a page Extension:Wikibase Client was modified, changed by Octahedron80 link https://www.mediawiki.org/w/index.php?diff=2721213 edit summary: [+62] [06:26:57] Change on 12www.mediawiki.org a page Extension:Wikibase Client was modified, changed by Octahedron80 link https://www.mediawiki.org/w/index.php?diff=2721214 edit summary: [+17] [06:28:00] Change on 12www.mediawiki.org a page Extension:Wikibase Client was modified, changed by Octahedron80 link https://www.mediawiki.org/w/index.php?diff=2721215 edit summary: [+12] [06:29:44] Change on 12www.mediawiki.org a page Extension:Wikibase Client was modified, changed by Octahedron80 link https://www.mediawiki.org/w/index.php?diff=2721217 edit summary: [-8] [09:26:52] PROBLEM - wikidata.org dispatch lag is higher than 300s on www.wikidata.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - pattern not found - 1949 bytes in 0.165 second response time [09:31:53] RECOVERY - wikidata.org dispatch lag is higher than 300s on www.wikidata.org is OK: HTTP OK: HTTP/1.1 200 OK - 1943 bytes in 0.104 second response time [10:16:48] SELECT ?tag WHERE { wd:Q8502 wdt:P1282 ?tag } <-- This query should give a result, but it doesn't. I can't figure out why. Any ideas? [10:18:54] SELECT ?key ?value WHERE { wd:Q8502 ?key ?value } <-- wdt:P1282 isn't in these results [10:19:49] the statement is marked as deprecated, wdt: doesn't return those [10:21:34] Thanks nikki! [10:22:47] and if you want all of the statements, you can use p:P1282/ps:P1282 [10:23:59] Brilliant! [12:04:50] Hi [12:11:50] bye [16:09:29] Thiemo_WMDE: I've got the failures down to one persistent one, something about the lookup results not getting 'normalized', I can keep digging later today but if you have thoughts I'd love them... https://integration.wikimedia.org/ci/job/mwext-testextension-hhvm-composer-jessie/11586/console [16:09:49] (the past couple weeks have been bad for Getting Shit Done™ sadly) [20:44:21] PROBLEM - wikidata.org dispatch lag is higher than 300s on www.wikidata.org is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - pattern not found - 1948 bytes in 0.138 second response time [20:49:21] RECOVERY - wikidata.org dispatch lag is higher than 300s on www.wikidata.org is OK: HTTP OK: HTTP/1.1 200 OK - 1933 bytes in 0.082 second response time [20:57:51] PROBLEM - MD RAID on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:57:52] PROBLEM - Updater process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:58:01] PROBLEM - Blazegraph Port on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:58:02] PROBLEM - WDQS HTTP on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 80: Connection refused [20:58:03] PROBLEM - Blazegraph process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:58:11] PROBLEM - configured eth on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:58:13] PROBLEM - WDQS HTTP Port on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:58:22] PROBLEM - DPKG on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [20:59:49] Is this due to high editing rates? [21:00:23] PROBLEM - WDQS SPARQL on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 80: Connection refused [21:00:32] PROBLEM - Disk space on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:00:35] PROBLEM - dhclient process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:00:41] PROBLEM - Check systemd state on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:00:51] PROBLEM - SSH on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 22: Connection refused [21:03:32] PROBLEM - puppet last run on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:07:22] PROBLEM - Host wdqs1004 is DOWN: PING CRITICAL - Packet loss = 100% [21:13:32] RECOVERY - Host wdqs1004 is UP: PING WARNING - Packet loss = 64%, RTA = 0.22 ms [21:13:32] RECOVERY - DPKG on wdqs1004 is OK: All packages OK [21:13:33] RECOVERY - WDQS SPARQL on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 15990 bytes in 0.001 second response time [21:13:52] RECOVERY - Disk space on wdqs1004 is OK: DISK OK [21:13:52] RECOVERY - dhclient process on wdqs1004 is OK: PROCS OK: 0 processes with command name dhclient [21:13:52] RECOVERY - Check systemd state on wdqs1004 is OK: OK - running: The system is fully operational [21:13:54] RECOVERY - SSH on wdqs1004 is OK: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u4 (protocol 2.0) [21:14:01] RECOVERY - MD RAID on wdqs1004 is OK: OK: Active: 4, Working: 4, Failed: 0, Spare: 0 [21:14:02] RECOVERY - Updater process on wdqs1004 is OK: PROCS OK: 1 process with UID = 997 (blazegraph), regex args ^java .* org.wikidata.query.rdf.tool.Update [21:14:12] RECOVERY - Blazegraph Port on wdqs1004 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 9999 [21:14:13] RECOVERY - WDQS HTTP on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 15990 bytes in 0.001 second response time [21:14:21] RECOVERY - Blazegraph process on wdqs1004 is OK: PROCS OK: 1 process with UID = 997 (blazegraph), regex args ^java .* blazegraph-service-.*war [21:14:22] RECOVERY - configured eth on wdqs1004 is OK: OK - interfaces up [21:14:31] RECOVERY - WDQS HTTP Port on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 435 bytes in 0.031 second response time [21:34:34] PROBLEM - configured eth on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:34:36] PROBLEM - WDQS HTTP Port on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:34:44] PROBLEM - DPKG on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:35:54] PROBLEM - WDQS SPARQL on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 80: Connection refused [21:36:04] PROBLEM - Disk space on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:06] PROBLEM - dhclient process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:06] PROBLEM - Check systemd state on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:14] PROBLEM - SSH on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 22: Connection refused [21:36:15] PROBLEM - MD RAID on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:24] PROBLEM - Updater process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:25] PROBLEM - WDQS HTTP on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 80: Connection refused [21:36:26] PROBLEM - Blazegraph Port on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:35] PROBLEM - Blazegraph process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:36:45] PROBLEM - puppet last run on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:37:54] RECOVERY - DPKG on wdqs1004 is OK: All packages OK [21:37:56] RECOVERY - WDQS SPARQL on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 15990 bytes in 0.001 second response time [21:38:05] RECOVERY - Disk space on wdqs1004 is OK: DISK OK [21:38:05] RECOVERY - dhclient process on wdqs1004 is OK: PROCS OK: 0 processes with command name dhclient [21:38:06] RECOVERY - Check systemd state on wdqs1004 is OK: OK - running: The system is fully operational [21:38:15] RECOVERY - SSH on wdqs1004 is OK: SSH OK - OpenSSH_6.7p1 Debian-5+deb8u4 (protocol 2.0) [21:38:25] RECOVERY - MD RAID on wdqs1004 is OK: OK: Active: 4, Working: 4, Failed: 0, Spare: 0 [21:38:25] RECOVERY - Updater process on wdqs1004 is OK: PROCS OK: 1 process with UID = 997 (blazegraph), regex args ^java .* org.wikidata.query.rdf.tool.Update [21:38:26] RECOVERY - WDQS HTTP on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 15990 bytes in 0.001 second response time [21:38:34] RECOVERY - Blazegraph Port on wdqs1004 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 9999 [21:38:44] RECOVERY - Blazegraph process on wdqs1004 is OK: PROCS OK: 1 process with UID = 997 (blazegraph), regex args ^java .* blazegraph-service-.*war [21:38:45] RECOVERY - configured eth on wdqs1004 is OK: OK - interfaces up [21:38:54] RECOVERY - WDQS HTTP Port on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 435 bytes in 0.016 second response time [21:40:54] PROBLEM - DPKG on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:41:04] PROBLEM - WDQS SPARQL on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 80: Connection refused [21:41:14] PROBLEM - Disk space on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:41:15] PROBLEM - dhclient process on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:41:15] PROBLEM - Check systemd state on wdqs1004 is CRITICAL: Return code of 255 is out of bounds [21:41:17] PROBLEM - SSH on wdqs1004 is CRITICAL: connect to address 10.64.0.17 and port 22: Connection refused [22:16:38] oi da [23:04:29] question: why are there RFBOTs open from as far back as 2014?