[00:00:16] sDrewthedoff: those URLs are just blank pages? [00:00:36] yep, though not sure else is associated [00:00:59] it still seems to be an existing page [00:01:06] tools.wmflabs.org is not a wiki? [00:01:11] I'm really confused as to what you're asking [00:01:36] it is showing up as an existing page http://tools.wmflabs.org/guc/?user=83.149.9.198 [00:02:37] if it is an existing page of some sort, can we please kill it [00:02:49] I... still don't know what you're asking me [00:02:54] tools.wmflabs.org is not a wiki [00:03:23] sDrewthedoff: ah, I see [00:03:29] I understand it is not a wiki [00:03:35] sDrewthedoff: the guc tool seems to have a bug where it is pointing things to tools.wmflabs.org instead of somewhere else? [00:03:47] ok, that I can sort [00:03:51] sDrewthedoff: all links on http://tools.wmflabs.org/guc/?user=83.149.9.198 seem to point to tools.wmflabs.org instead of whatever wiki they need to point to [00:04:52] seems to be a broader problem [00:05:05] too early to be able to identify issues like! [00:10:54] sDrewthedoff YuviPanda: see https://phabricator.wikimedia.org/T106897 it needs someone to merge https://gerrit.wikimedia.org/r/#/c/226939/ [00:11:38] ah I see [00:14:26] thanks sitic [00:15:12] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1482915 (10Billinghurst) shake shake shake [01:02:01] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1482966 (10Ricordisamoa) "Unbreak Now!" ```lines=10 MariaDB [(none)]> SELECT dbname, lang, name, family, url FROM... [01:06:25] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1482968 (10Ricordisamoa) ``` MariaDB [(none)]> SELECT dbname, lang, name, family, url FROM meta_p.wiki WHERE (url I... [01:09:18] 6Labs, 10Tool-Labs, 5Patch-For-Review: Missing or wrong information in meta_p.wiki table - https://phabricator.wikimedia.org/T56962#1482969 (10Ricordisamoa) [01:11:19] 6Labs, 10Tool-Labs: Missing or wrong information in meta_p.wiki table - https://phabricator.wikimedia.org/T56962#1482973 (10Ricordisamoa) [02:27:32] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1482982 (10Ricordisamoa) The sitematrix API is also broken: ```lang=javascript new mw.Api().get( { action: 'sitem... [02:28:55] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1482983 (10Ricordisamoa) Ricordisamoa raised the priority of this task from "Unbreak Now!" to "Unbreak Before Now!". [03:24:30] have we ever thought about doing something with docker? [03:35:21] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483013 (10Sitic) The sitematrix bug is tracked in {T106963}. [03:40:17] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483019 (10scfc) [05:16:02] 6Labs, 7Tracking: New Labs project requests (Tracking) - https://phabricator.wikimedia.org/T76375#1483030 (10Negative24) [05:29:41] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1407 is CRITICAL 60.00% of data above the critical threshold [0.0] [05:42:49] does qsub not read .bash_profile the same way jsub does, if at all? [06:04:40] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1407 is OK Less than 1.00% above the threshold [0.0] [06:15:27] !log logstash Testing irc input on 1.5.3 [06:15:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Logstash/SAL, Master [09:25:39] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1407 is CRITICAL 16.67% of data above the critical threshold [0.0] [10:05:40] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1407 is OK Less than 1.00% above the threshold [0.0] [10:07:32] PROBLEM - Puppet failure on tools-static-01 is CRITICAL 100.00% of data above the critical threshold [0.0] [10:15:23] 6Labs, 10Labs-Infrastructure, 7Database: Entries in meta_p.wiki and meta_p.legacy broken - https://phabricator.wikimedia.org/T106972#1483184 (10Dapete) 3NEW [10:19:22] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483193 (10Glaisher) [10:19:55] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1481075 (10Glaisher) [10:20:28] 6Labs, 10Labs-Infrastructure, 7Database: Entries in meta_p.wiki and meta_p.legacy broken - https://phabricator.wikimedia.org/T106972#1483198 (10Glaisher) [10:20:30] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1481075 (10Glaisher) [10:21:15] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483204 (10Mjbmr) [11:21:38] 10Tool-Labs-tools-Global-user-contributions: Wrong links in GUC - https://phabricator.wikimedia.org/T106976#1483251 (10Vituzzu) 3NEW [14:06:00] 10Tool-Labs-tools-Global-user-contributions: Wrong links in GUC - https://phabricator.wikimedia.org/T106976#1483338 (10Sitic) [14:06:01] 6Labs, 10Tool-Labs, 10Labs-Infrastructure, 5Patch-For-Review, 7Regression: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483339 (10Sitic) [14:06:27] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Global-user-contributions, 10Labs-Infrastructure, and 2 others: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1481075 (10Sitic) [15:13:23] 6Labs, 10Tool-Labs: Grants for my Tools-db missing to insert new lines - https://phabricator.wikimedia.org/T98790#1483412 (10jcrespo) I repaired the table, made a backup on my home dir and dropped the table. The table may have originally corruption, as it had some strange behaviour. I can reimport the table if... [15:59:03] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Global-user-contributions, 10Labs-Infrastructure, and 2 others: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483426 (10Krenair) [16:04:34] 6Labs: Labs Docker Project - https://phabricator.wikimedia.org/T106969#1483430 (10Krenair) [16:22:50] Nemo_bis: thank you for taking care of the ts wiki [16:24:16] :) well, some brutal transfer for now ;) [16:34:42] 10Wikibugs: wikibugs: remove redis dependency - https://phabricator.wikimedia.org/T1207#1483446 (10valhallasw) [17:33:26] 6Labs, 10Tool-Labs: Migrate individual tools to trusty to relieve pressure on older precise nodes - https://phabricator.wikimedia.org/T88228#1483512 (10scfc) 5Open>3declined a:3scfc Ubuntu Precise is supported until [[https://wiki.ubuntu.com/Releases|April 2017]]; we plan to migrate to Debian Jessie in t... [18:37:15] !log logstash Test log message [18:37:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Logstash/SAL, Master [18:39:12] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Global-user-contributions, 10Labs-Infrastructure, and 2 others: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483550 (10scfc) (Filed T106989 for my ramblings in T106897#1481393.) [19:42:10] fyi labs gives me a gateway time out [21:57:41] PROBLEM - Puppet failure on tools-webgrid-lighttpd-1407 is CRITICAL 40.00% of data above the critical threshold [0.0] [22:22:50] (03PS1) 10Sitic: Add ORES support [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/227163 (https://phabricator.wikimedia.org/T106727) [22:23:38] YuviPanda: want to have a look at ^ ? [22:29:53] (03PS1) 10Sitic: Switch to python 3 [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/227164 [22:32:14] (03PS2) 10Sitic: Switch to python 3 [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/227164 [22:37:41] RECOVERY - Puppet failure on tools-webgrid-lighttpd-1407 is OK Less than 1.00% above the threshold [0.0] [22:59:34] 6Labs, 10Tool-Labs, 10Tool-Labs-tools-Global-user-contributions, 10Labs-Infrastructure, and 2 others: meta_p.wiki table corrupt (contains many NULL entries for 'url' field) - https://phabricator.wikimedia.org/T106897#1483754 (10MZMcBride) >>! In T106897#1482983, @Ricordisamoa wrote: > Ricordisamoa raised t...