[02:40:26] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL - Socket timeout after 10 seconds [02:45:18] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 790867 bytes in 3.040 second response time [07:34:16] What is the best way to find the latest revision of a page, that wasn't done by bot? is it joining page table with revisions and user_groups, selecting max revid? [07:34:43] (with group by for page)? [11:22:34] 6Labs, 10Datasets-General-or-Unknown, 10Labs-Infrastructure, 10Wikidata: Add Wikidata json dumps to labs - https://phabricator.wikimedia.org/T100885#1322765 (10Addshore) 3NEW [11:29:27] 6Labs, 10Datasets-General-or-Unknown, 10Labs-Infrastructure, 10Wikidata: Add Wikidata json dumps to labs - https://phabricator.wikimedia.org/T100885#1322800 (10Lydia_Pintscher) [11:41:37] 10Tool-Labs-tools-Other, 6Wikisource: New export-to-book tools for Wikisource - https://phabricator.wikimedia.org/T92287#1322815 (10Nemo_bis) [13:15:29] (03CR) 10Ricordisamoa: "recheck" [labs/tools/extreg-wos] - 10https://gerrit.wikimedia.org/r/211055 (owner: 10Ricordisamoa) [14:22:05] help [14:23:46] not being able to ssh login to lab server [15:03:59] 10Tool-Labs-tools-Other, 10Phragile: Deploy Phragile on tool-labs - https://phabricator.wikimedia.org/T100192#1323032 (10Aklapper) [16:16:46] 6Labs, 10Wikimedia-Labs-General: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#1323113 (10Krenair) [16:16:55] 6Labs, 6operations: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#751702 (10Krenair) [16:24:25] Helder, o/ [16:24:31] hey [16:24:57] So, we're getting interesting errors when Helder SSHs to some instances that the rest of us can connect to just fine. [16:25:13] "no matching mac found: client hmac-sha2-512" [16:25:27] Anyone have an idea what this might be? [16:25:44] Also, Helder can you connect to tool labs? [16:27:05] Do you mean $ssh tools-login.wmflabs.org? [16:28:22] yeah [16:28:25] yep [16:28:39] OK. So it's just instances in project 'revscoring' so far. [16:28:52] Can you try ores-main.eqiad.wmflabs? [16:28:52] any patterns related to puppet runs? [16:28:59] wait, am I supposed to run "ssh ores-compute.eqiad.wmflabs" from there? [16:29:06] Helder, nope [16:29:11] From your local machine. [16:29:12] hm... [16:29:13] what client are you using? [16:29:54] I'm on Ubuntu Studio, using ssh in the terminal [16:30:26] $ssh ores-main.eqiad.wmflabs [16:30:26] no matching mac found: client hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160 server hmac-sha2-512,hmac-sha2-256 [16:30:27] ssh_exchange_identification: Connection closed by remote host [16:30:33] have you seen the recent posts about ssh algorithms on the labs/ops lists? [16:30:55] I don't think so [16:31:08] or is it http://www.gossamer-threads.com/lists/wiki/wikitech/590424?page=last ? [16:31:30] https://lists.wikimedia.org/pipermail/labs-l/2015-May/003730.html [16:31:38] yes [16:31:40] 'we have had 1 report on IRC so far from [16:31:40] a user who got "no matching MAC found" but apparently was able to fix [16:31:40] it by adjusting the client config. ' [16:31:49] but it doesn't explain what is the adjustment needed [16:32:03] and the gerrit change doesn't help either [16:41:47] Helder, can you confirm you've installed recent security patches? [16:42:00] Otherwise, I think we should file a phab task about it. [16:42:01] nope [16:42:08] which ones? how? [16:42:09] You can't confirm? :P [16:42:19] Oh, just whatever Ubuntu wants you to upgrade [16:42:53] "sudo apt-get update && sudo apt-get upgrade" [16:43:31] * Helder is doing that [16:44:09] ok, samething [16:47:49] K [16:47:51] hmm [16:47:58] I think it's time for phab task. [16:53:36] 6Labs: no matching mac found - https://phabricator.wikimedia.org/T100901#1323180 (10He7d3r) 3NEW [16:53:49] halfak, ^ [16:54:41] Try removing the section about gerrit and trying again. [16:54:50] Helder: ^ [16:56:32] alread tried that [17:01:29] I doubt that would work [18:06:36] NFS upset again? [18:07:12] Copying a 548 bytes file in my homedir takes several seconds [18:10:45] (03PS1) 10Sitic: Fix Moment.js language setting [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214826 [18:11:04] (03CR) 10Sitic: [C: 032 V: 032] Fix Moment.js language setting [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214826 (owner: 10Sitic) [18:12:04] (03PS1) 10Sitic: Added option to set the watchlist timeperiod [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214827 (https://phabricator.wikimedia.org/T100340) [18:12:34] (03CR) 10Sitic: [C: 032 V: 032] Added option to set the watchlist timeperiod [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214827 (https://phabricator.wikimedia.org/T100340) (owner: 10Sitic) [18:14:46] (03PS1) 10Sitic: Added celery error mail handling [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214828 [18:15:01] (03CR) 10Sitic: [C: 032 V: 032] Added celery error mail handling [labs/tools/crosswatch] - 10https://gerrit.wikimedia.org/r/214828 (owner: 10Sitic) [18:22:42] 6Labs, 6operations: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#1323285 (10scfc) 5Open>3Resolved a:3scfc All view databases (e. g., `fawikivoyage_p`) seem to have been created and are replicated and the `fawikivoyage.labsdb` host aliases in `mani... [18:23:10] 6Labs, 6operations: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#1323288 (10scfc) a:5scfc>3None [18:26:18] 6Labs, 6operations: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#1323291 (10scfc) @Reedy, could you please check/update the status of https://rt.wikimedia.org/Ticket/Display.html?id=8877 if it duplicates this task? [18:29:41] 6Labs, 6operations: fawikivoyage, orwikisource. maiwiki's dbs missing from Labs - https://phabricator.wikimedia.org/T75480#1323294 (10Krenair) That would be T84757 - the restricted task which I duplicated into this earlier. [18:38:36] 6Labs, 10Labs-Infrastructure, 6operations: Make labs/private really private - https://phabricator.wikimedia.org/T89642#1323303 (10Krenair) 5Open>3declined a:3Krenair [19:02:02] 6Labs, 6operations, 7network: permit syslog from labs to lithium - https://phabricator.wikimedia.org/T90695#1323313 (10Krenair) I assume this is a firewall thing? [22:11:08] Is there a persistent database for tool data available? Couldn't find anything at https://wikitech.wikimedia.org/wiki/Help:Tool_Labs [22:13:20] lfaraone: you mean a database where your tool can store data? Yes: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Database#Creating_new_databases [22:13:37] yeah, awesome. thanks :) [23:27:17] hello? [23:27:36] Anybody out there????? [23:30:12] legoktm: around? does the default uwsgi Python configuration log the stdout of the WSGI processes anywhere? [23:37:00] lfaraone: hey. uhh, I have no idea. [23:37:16] lfaraone: I would assume not. [23:37:39] lfaraone: if it does, it would be in ~/uwsgi.log