[00:08:16] hey folks. creating a labs account now -- can someone confirm whether 'twp' is available as a shell account name? [00:14:29] anyone awake? :-) [00:31:06] topaz, it's taken [00:31:40] maybe that account is yours? [00:32:52] note that labs login is shared with phabricator and wikitech [00:35:32] Platonides: I got impatient and went ahead and submitted the request :-) [00:36:55] I don't know if the account you're looking at now is the result of the request I submitted [01:56:19] 10Tool-Labs: Get rid of portgranter - https://phabricator.wikimedia.org/T93046#1130960 (10scfc) I tested putting: ``` # Bind to a free port and note the port number. s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('', 0)) port = s.getsockname()[1] s.close() ``` into `portgrabber` and it worked as... [03:35:42] nakon, ping [03:35:52] pong :D [05:56:01] PROBLEM - ToolLabs Home Page on toollabs is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:00:54] RECOVERY - ToolLabs Home Page on toollabs is OK: HTTP OK: HTTP/1.1 200 OK - 753278 bytes in 3.391 second response time [06:38:03] PROBLEM - Puppet failure on tools-webgrid-tomcat is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:00:39] PROBLEM - Puppet failure on tools-webgrid-03 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [07:08:00] RECOVERY - Puppet failure on tools-webgrid-tomcat is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:42] RECOVERY - Puppet failure on tools-webgrid-03 is OK: OK: Less than 1.00% above the threshold [0.0] [08:51:02] 6Labs, 10Tool-Labs: Set up sufficient monitoring for toollabs - https://phabricator.wikimedia.org/T90845#1131176 (10Nemo_bis) Do uptime/downtime stats fall under this task? [09:54:06] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate shell access request process to Phabricator - https://phabricator.wikimedia.org/T72627#1131297 (10Qgil) [09:54:42] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate shell access request process to Phabricator - https://phabricator.wikimedia.org/T72627#752014 (10Qgil) Just checking: does this have any relation with {T550}? [09:55:18] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate new Labs projects request process to Phabricator - https://phabricator.wikimedia.org/T72626#1131305 (10Qgil) [09:56:26] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate new Labs projects request process to Phabricator - https://phabricator.wikimedia.org/T72626#751945 (10Qgil) Why do you need an extension? Would a link to create a task with pre-filled description work as well? See {T91538} [10:09:09] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate new Labs projects request process to Phabricator - https://phabricator.wikimedia.org/T72626#1131335 (10yuvipanda) 5Open>3Resolved a:3yuvipanda This has already happened - T76375 [11:18:58] 6Labs, 10Tool-Labs: Set up sufficient monitoring for toollabs - https://phabricator.wikimedia.org/T90845#1131552 (10JanZerebecki) No, this is primarily about getting notified when something of tool labs is down. Keeping historic information and making stats from them would be nice in general for our monitoring... [11:40:29] andrewbogott_afk: Nice house you got. [12:21:27] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate shell access request process to Phabricator - https://phabricator.wikimedia.org/T72627#1131641 (10scfc) Hmmm. Shell access is in some level equivalent to being a member of the bastion project in LDAP, but IIRC there's some more magic happening with wi... [12:22:58] 10Wikimedia-Labs-wikitech-interface, 6Phabricator: Migrate new Labs projects request process to Phabricator - https://phabricator.wikimedia.org/T72626#1131646 (10scfc) @Qgil: "This would probably **not** need any extension […]" [13:00:46] 10Tool-Labs: Make list.php not rely on portgranter - https://phabricator.wikimedia.org/T93197#1131686 (10scfc) 3NEW a:3scfc [13:01:11] 10Tool-Labs: Make list.php not rely on portgranter - https://phabricator.wikimedia.org/T93197#1131686 (10scfc) [13:01:12] 10Tool-Labs: Provide a status page (list) of all active proxy definitions - https://phabricator.wikimedia.org/T88216#1131694 (10scfc) [13:16:11] 10Tool-Labs: Provide a status page (list) of all active proxy definitions - https://phabricator.wikimedia.org/T88216#1131711 (10yuvipanda) Yup, should be done in Lua on nginx, I think. [14:05:36] 10Tool-Labs: Provide a status page (list) of all active proxy definitions - https://phabricator.wikimedia.org/T88216#1131827 (10scfc) a:3scfc Forgot to claim. "?some-magic-url" is up for bikeshedding, of course. [14:37:58] 10Wikimedia-Labs-wikitech-interface, 6Project-Creators: Migrate shell access request process to Phabricator - https://phabricator.wikimedia.org/T72627#1131921 (10Aklapper) [15:07:28] hi, so proprietary closed-source content is forbidden on on labs, but what about utilizing monitoring services like New Relic? In that case we'd just be integrating it with our open-source application, and be able to do our monitoring at newrelic.com [15:07:36] would something like that be allowed? [15:29:37] YuviPanda: re cherry-picking on deployment-salt in deployment-prep [15:30:03] it seems there is a rebase or sth in progress (has been since yesterday) [15:30:18] am i free to git checkout prod and continue? [15:30:25] i.e. cherry-pick my stuff? [15:48:19] mobrovac: yeah, just log your actions on RelEng and inform someone on the team [15:48:41] kk [15:48:44] thnx YuviPanda [15:59:18] 6Labs, 10Tool-Labs: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1132173 (10scfc) 3NEW [15:59:40] 6Labs, 10Tool-Labs: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1132182 (10scfc) a:3scfc [16:18:16] 6Labs, 10Tool-Labs, 10Beta-Cluster, 5Patch-For-Review: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1132260 (10scfc) p:5Triage>3Normal [16:50:04] 6Labs, 10Tool-Labs, 10Beta-Cluster: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1132375 (10scfc) a:5scfc>3None [16:54:29] 6Labs, 10Tool-Labs, 10Beta-Cluster: Private SSL key got removed on tools-webproxy-01/tools-webproxy-02 and maybe other Labs instances as well - https://phabricator.wikimedia.org/T93212#1132386 (10scfc) I think I was too optimistic about the filebucket: ``` scfc@tools-webproxy-01:~$ sudo puppet filebucket re... [17:12:54] 6Labs: Allow wayback machine to crawl flow-tests.wmflabs.org - https://phabricator.wikimedia.org/T93221#1132450 (10Nemo_bis) 3NEW [18:22:30] 6Labs, 6Collaboration-Team: Allow wayback machine to crawl flow-tests.wmflabs.org - https://phabricator.wikimedia.org/T93221#1132806 (10Legoktm) [20:22:18] 10Tool-Labs-tools-Other: grrrit-wm should ignore jenkins-bot merging l10n-bot commits - https://phabricator.wikimedia.org/T93259#1133218 (10Legoktm) 3NEW a:3yuvipanda [21:22:42] Coren: can you log into wikitech right now? [23:15:10] 6Labs: Get Labs openstack service dbs on a proper db server - https://phabricator.wikimedia.org/T92693#1133929 (10Andrew) [23:15:11] 6Labs, 5Patch-For-Review: db servers for designate and labs pdns - https://phabricator.wikimedia.org/T92694#1133927 (10Andrew) 5Open>3Resolved This is all working! [23:15:44] 6Labs, 5Patch-For-Review: Copy old internal instance IPs to designate - https://phabricator.wikimedia.org/T93085#1133935 (10Andrew) This is now done, thanks to the attached script. [23:17:50] 6Labs, 5Patch-For-Review: Copy old internal instance IPs to designate - https://phabricator.wikimedia.org/T93085#1133939 (10Andrew) {F100993} [23:28:20] andrewbogott: I shall try [23:29:07] andrewbogott: Yep. No issue. [23:29:34] Coren: yeah, it was broken a few hours ago, for a few minutes. nutcracker crapped out for unknown reasons [23:31:29] andrewbogott: Ah, didn't notice the timestamps - was out for dinner. [23:35:04] no worries [23:39:58] 6Labs, 5Patch-For-Review: Investigate replacing our custom DNS code with Designate - https://phabricator.wikimedia.org/T87280#1134035 (10Andrew) [23:39:59] 6Labs, 5Patch-For-Review: Move to a new dns scheme for labs: hostname.projectname.eqiad.wmflabs - https://phabricator.wikimedia.org/T93087#1134034 (10Andrew) [23:40:00] 6Labs, 5Patch-For-Review: Copy old internal instance IPs to designate - https://phabricator.wikimedia.org/T93085#1134031 (10Andrew) 5Open>3Resolved a:3Andrew um, which, obviously the password in that script is no longer the actual password. *blush*