[12:40:16] <Jayprakash12345>	 Hi, I am writing the script to automate the deployment. I am able to SSH into login.toolforge.org in Python script. But when I am doing the "become <tool>" comand then It is getting stuck. This is true as it needs an interactive shell terminal to run. I am just curious whether we can directly login in to tool or not. Example: SSH to direct
[12:40:17] <Jayprakash12345>	 tools.<toolname>@tools-bastion-13.login.toolforge.org, I mean something like this to get rid of become command.
[12:44:45] <arturo>	 Jayprakash12345: you cannot get rid of the `become` step, unfortunately
[12:45:38] <Jayprakash12345>	 Thank arturo!
[12:50:04] <arturo>	 Jayprakash12345: well, you may execute a command directly as your user
[12:54:17] <arturo>	 Jayprakash12345: see this `ssh dev.toolforge.org sudo -niu "tools.arturo-test-tool" "bash -c 'echo hello \$LOGNAME \$PWD'"`
[12:54:42] <arturo>	 if I execute that from my laptop
[12:54:49] <arturo>	 I get: `hello tools.arturo-test-tool /mnt/nfs/labstore-secondary-tools-project/arturo-test-tool`
[12:59:56] <dcaro>	 Jayprakash12345: we are planning to be able to release a webhook-based deployment feature by the end of this fiscal year (that'd be july 2025, https://phabricator.wikimedia.org/T194332), if you are not in a rush that might simplify your setup
[13:00:19] <arturo>	 this also works
[13:00:21] <arturo>	 `ssh dev.toolforge.org become arturo-test-tool "bash -c 'echo hello \$LOGNAME \$PWD'"`
[13:02:28] <dcaro>	 probably this task is easier to follow... https://phabricator.wikimedia.org/T362051
[13:07:03] <Jayprakash12345>	 Thanks dcaro for sharing this info.
[13:07:40] <arturo>	 I documented this become trick here https://wikitech.wikimedia.org/wiki/Help:Toolforge/Tool_Accounts#Scripting in case is useful for others
[13:09:40] <Jayprakash12345>	 arturo, It seems I was looking for this. Montage tool is also using the `sudo -ni -p` (https://github.com/hatnote/montage/blob/master/fabfile.py#L13) to directly going into tool.
[13:09:59] <Jayprakash12345>	 I was not able to get how to use this.
[13:10:23] <Jayprakash12345>	 Things are now clear, thanks both of you.
[13:10:30] <arturo>	 👍
[13:12:54] <arturo>	 that fabric python library looks really nice
[13:18:06] <dcaro>	 I used that library extensively in the past for managing deployments and the equivalent of "cookbooks", it really good for not-so-big scale (<1000s of nodes)
[13:20:24] <dcaro>	 the main inconvenient was that it's not asynchronous (there was no support at least back then), so you had to figure out a way to spawn processes if you did not want to block the application for each connection
[13:24:54] <dcaro>	 trip down memory lane xd, just checked my repos, that was ~10y ago!
[13:28:17] <arturo>	 !log admin [codfw1dev] restart rabbitmq, openstack services logs show connection errors
[13:28:21] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[15:24:23] <wm-bb>	 <Yetkin> curl timeout parameter does not work on Toolforge jobs, I suppose. It works fine on the grid, though. Any possible reasons for such a behavior?
[16:22:07] <JJMC89>	 Toolforge admins: please add the TFSC's tool as a maintainer of the jawi tool for T379340
[16:22:07] <stashbot>	 T379340: Adoption request for jawi - https://phabricator.wikimedia.org/T379340
[16:29:34] <arturo>	 JJMC89: done
[16:29:56] <arturo>	 !log tools.jawi add toolforge-standards-comittee as maintainer (T379340)
[16:29:59] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.jawi/SAL
[16:30:00] <stashbot>	 T379340: Adoption request for jawi - https://phabricator.wikimedia.org/T379340
[16:30:43] <JJMC89>	 thanks
[16:30:50] <arturo>	 I think it was my first time doing this, so for reference, what I did was this
[16:31:06] <arturo>	 https://usercontent.irccloud-cdn.com/file/iPkt7wuK/image.png
[16:31:35] <JJMC89>	 that's correct - I have access now
[16:42:49] <arturo>	 !log admin [codfw1dev] restart all nova services and rabbitmq out of despair
[16:42:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[21:07:35] <wm-bb>	 <JeanFred> Hello hello. Users have reported that my wikiloves tool has been inacessible for a few days − indeed https://wikiloves.toolforge.org/ goes in either 502 or 404. (I have not changed anything in recent weeks to the application.). To try to fix it,
[21:07:36] <wm-bb>	 <JeanFred> * I have done several rounds of `webservice restart`, to no avail. 
[21:07:37] <wm-bb>	 <JeanFred> * I have recreated the venv from scratch (`toolforge webservice python3.9 shell ; webservice-python-bootstrap --fresh`)
[21:07:39] <wm-bb>	 <JeanFred> * checking uwsgi.log, it looks it’s endlessly restarting every 5min or so, but nothing else in there.
[21:07:40] <wm-bb>	 <JeanFred> Any suggestions?
[21:28:03] <JJMC89>	 try stop then start instead of restart
[22:06:55] <wm-bb>	 <lucaswerkmeister> @JeanFred that sounds like it could be https://phabricator.wikimedia.org/T362867#10292196
[22:06:58] <wm-bb>	 <lucaswerkmeister> if that’s true, then stop + start might fix it, as JJMC89 said
[22:07:02] <wm-bb>	 <lucaswerkmeister> let me try to dig up the worker first
[22:07:10] <wm-bb>	 <lucaswerkmeister> tools-k8s-worker-nfs-8
[22:08:06] <lucaswerkmeister>	 dhinus / dcaro: ^ tools-k8s-worker-nfs-8 might still have NFS issues
[22:28:07] <dhinus>	 I'm not close to my laptop, I can have a look in 1 hour
[22:36:41] <wm-bb>	 <lucaswerkmeister> FWIW, I sshed into tools-k8s-worker-nfs-8 and it looks okay to me so far… ls /data/project or /data/project/wikiloves works, htop shows no processes in D state
[22:36:53] <wm-bb>	 <lucaswerkmeister> I think it can wait for an hour :)
[22:37:13] <wm-bb>	 <lucaswerkmeister> (and I assume it’s okay for Jean-Fred to stop+start the webservice in the meantime, now that you have the node name)
[23:04:33] <wm-bb>	 <JeanFred> Should I stop/start then? Or is it needed for some investigation?