[01:21:30] PROBLEM - Free space - all mounts on tools-web-static-02 is CRITICAL: CRITICAL: tools.tools-web-static-02.diskspace._srv.byte_percentfree (<100.00%) [02:16:48] PROBLEM - Puppet run on tools-services-02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [02:50:56] what's wrong with stream.wmflabs.org? [02:51:50] RECOVERY - Puppet run on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0] [06:58:05] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - string 'Magnus' not found on 'http://tools.wmflabs.org:80/' - 531 bytes in 0.020 second response time [07:13:06] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 3670 bytes in 0.047 second response time [07:40:49] PROBLEM - Free space - all mounts on tools-services-01 is CRITICAL: CRITICAL: tools.tools-services-01.diskspace.root.byte_percentfree (<11.11%) [09:09:39] valhallasw`vecto: is https://phabricator.wikimedia.org/T144388 something you can help with? [09:34:54] Nemo_bis: I'm unaware of the policies involved [09:35:43] valhallasw`vecto: ok, thanks [10:21:34] !log phabricator reboot phab-01 [10:21:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Phabricator/SAL, Master [10:43:52] hi there [10:57:52] freddy2001: hello! If you have any questions, feel free to ask them -- if anyone can help you, they will respond (eventually, might take a while) [11:05:56] 06Labs: Request creation of wmwcourse labs project - https://phabricator.wikimedia.org/T144388#2607416 (10Nikerabbit) [11:12:15] I am often behind firewalls, which block everything, that is not port 80 or 443. There I have because of that no access to tools via ssh. Is it possible to set up anything like shellinabox to provide ssh access to users, who are restricted by a firewall? [11:13:03] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Agaherbert was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819356 edit summary: [11:19:43] freddy2001: we don't have plans to support that use case, unfortunately. [11:20:07] freddy2001: the best option is to talk to your sysadmin (and ask to unblock other ports such as 22) [11:27:10] my sysadmin told me, that everything needs to run over 80/443 and exceptions aren't possible because of the size of the network [11:27:43] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/ԱշոտՏՆՂ was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819359 edit summary: [11:28:53] and i can use (web)-ssh, if i use one of those ports [11:29:56] freddy2001, I know your pain :( [11:32:54] The simple way to get around it would to have the bastion SSH listen on port 80 or port 443. [11:33:13] That's probably not going to happen though :/ [11:33:55] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/VolkerE was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819361 edit summary: [11:34:11] The reason exceptions can't be made (guessing) is that everything has to go through their web proxy, so they can track everything. [11:34:20] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Yasbot was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819365 edit summary: [11:36:53] PROBLEM - Puppet run on tools-services-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:37:57] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Jogo.obb was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819367 edit summary: [11:38:40] tom29739: the main reason that's not going to happen is that we are not in the business of second-guessing decisions of other admins [11:39:04] if other admins do not want outgoing ssh connections, we should not frustrate them by suddenly allowing ssh on port 443 [11:41:35] My admin told me it was for security, so that you couldn't attack their network remotely, by SSHing from within their network. Whatever the reason is, we shouldn't be making methods to try and get past their restrictions. [11:43:28] yes, tom29739 they want to run everything through their proxy because of security reasons. [11:43:58] that's, what my admin told me too. [11:49:42] and i need the vpn-connection, otherwise i have no access to almost everything on my computer (software and files on network storages)... [12:21:48] valhallasw`vecto We are allowed to do everything, while it communicates to the outside via 443/80 [12:56:07] freddy2001: set up the VPN to not route connections to labs via that firewall? [13:00:34] I can't edit the config of the vpn and everything - included firefox - is installed on the network storage. I can only disable the whole vpn, but then i can't use my pc… [14:36:11] http://login.tools.wmflabs.org/ does not work, how can I log in with ssh? [14:36:53] there is no http server running ther [14:36:54] there [14:37:42] just `ssh yourinstanceusername@login.tools.wmflabs.org` [14:42:59] how to do that in windows 8 [14:43:00] ? [14:43:58] I don't know [14:44:14] probably written on wikitech somewhere [14:44:17] most likely involved putty [14:44:20] involves* [14:44:34] ash__: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs#Quick_start , https://wikitech.wikimedia.org/wiki/Help:Access_to_ToolLabs_instances_with_PuTTY_and_WinSCP [14:49:35] thanks Krenair and valhallasw`cloud [15:25:27] PROBLEM - Free space - all mounts on tools-web-static-01 is CRITICAL: CRITICAL: tools.tools-web-static-01.diskspace._srv.byte_percentfree (<100.00%) [15:29:08] 06Labs, 10Tool-Labs: lighttpd does not correctly close connections (CLOSE_WAIT) - https://phabricator.wikimedia.org/T104799#2607633 (10valhallasw) That's great news! Unfortunately, debian is not bundling 1.4.40 yet, even in `sid` , so we will have to wait for that to happen, unless we find the time to build-an... [15:43:41] 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Install jq, sed, grep, sort in k8s images - https://phabricator.wikimedia.org/T141082#2607652 (10valhallasw) [15:44:17] 06Labs, 10Tool-Labs, 10MediaWiki-Interwiki, 10MediaWiki-extensions-Interwiki, 10Wikimedia-Interwiki-links: Toollabs interwiki doesn't work when url parameters needed - https://phabricator.wikimedia.org/T140981#2607654 (10valhallasw) 05Open>03declined That's just how links in mediawiki work. Use a dir... [15:54:43] valhallasw`cloud Hi for https://gerrit.wikimedia.org/r/#/c/308440/ we are trying to fix it so it dosent error with those warnnings. Plus i have no idea what to put for the description. We can always update it later but the main point is to fix it, not continue to ignore the rules. [18:58:13] Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Jogo.obb was modified, changed by Merlijn van Deen link https://wikitech.wikimedia.org/w/index.php?diff=819378 edit summary: [19:44:26] 10Tool-Labs-tools-stewardbots: Automatically start the irc bots - https://phabricator.wikimedia.org/T144461#2607756 (10MarcoAurelio) 05Open>03Resolved I think this is resolved. If you disagree, please reopen with your reasons. [19:46:31] 10Tool-Labs-tools-stewardbots, 10Continuous-Integration-Config, 13Patch-For-Review: Implement jenkins tests on labs/tools/stewardbots - https://phabricator.wikimedia.org/T128503#2607759 (10MarcoAurelio) Is it possible to expand the tests to gate-and-submit (I think they work already, not sure) and postmerge?... [20:13:08] PROBLEM - Host secgroup-lag-102 is DOWN: CRITICAL - Host Unreachable (10.68.17.218) [20:19:37] PROBLEM - Host tools-secgroup-test-102 is DOWN: CRITICAL - Host Unreachable (10.68.21.170) [20:38:31] Nemo_bis : i think xcontribs is done and somewhat overhauled for now - though i'd strongly recommend an even bigger overhaul at some point in the not too distant future, in order to switch from a static png generator to a dynamic js data visualization library [20:38:58] i'll grab a smoke and see if it matches expectations in a few minutes [22:27:16] Damianz: around? [23:57:14] PROBLEM - SSH on tools-exec-1219 is CRITICAL: Server answer