[09:33:28] !log multichill@tools-bastion-12 tools.noclaims Deployed jobs.yml with claim-templates hourly [09:33:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.noclaims/SAL [11:56:12] !log multichill@tools-bastion-12 tools.geograph T319765 Deployed jobs.yml with geograph-uploader-resumed hourly [11:56:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.geograph/SAL [12:01:26] !log multichill@tools-bastion-12 tools.integraality emails: all - [12:01:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.integraality/SAL [13:30:47] !log bsadowski1@tools-bastion-13 tools.stewardbots Restarted StewardBot/SULWatcher because of a connection loss [13:30:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [14:21:59] Can we connect to tools-redis.svc.eqiad.wmflabs from Cloud VPS instance? [14:21:59] Context: I recently migrated from toolforge to Cloud VPS for one of the projects and I am not able to access WMF-hosted Redis. I could use local Redis but there is a lot of job history that I would like to retain from the old Redis service. [14:24:41] @anmolwassan: I don't think we poke a hole in the Toolforge firewall to allow that, no. And honestly the redis.svc.eqiad.wmflabs instances are overloaded so I don't think we would like to add more usage from arbitrary Cloud VPS projects. [14:31:25] Got it, thanks. However, is there a way to get dump of the older jobs in .rdb? (re @wmtelegram_bot: @anmolwassan: I don't think we poke a hole in the Toolforge firewall to allow that, no. And honestly the redis.svc.eqiad...) [14:39:04] @anmolwassan: In theory the data could be dumped. In practice we would need to figure out how to limit or edit the dump to only have data for your keys. I would suggest starting with a Phabricator task explaining what data you would like and why and then we can try to figure out if it is reasonably possible to do. [14:40:30] * bd808 wonders if piles of historic data for tools using the shared redis for celery is part of the capacity problem we have been seeing... [16:12:11] getting a lot of "Lost connection to MySQL server during query" from the anticompositebot tool, trying to read commonswiki_p [17:08:27] !log lucaswerkmeister@tools-sgebastion-10 tools.lexeme-forms deployed 89c98da81f (upgrade dependencies for Python 3.12 compat; also upgraded pip{,-tools} and wheel while I’m at it) [17:08:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [17:15:03] !log lucaswerkmeister@tools-sgebastion-10 tools.wd-image-positions deployed 3204d71b37 (upgrade dependencies for Python 3.12 compat; also upgraded pip{,-tools} and wheel while I’m at it) [17:15:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL [17:30:22] Hey all - not sure if anybody has a second, but I can’t seem to ssh into sec-utils-bullseye.security-tools.eqiad1.wikimedia.cloud anymore and I have no idea why. My .ssh/config hasn’t changed, and I can get into login.toolforge.org just fine (similar config). Getting: “channel 0: open failed: connect failed: No route to host stdio forwarding failed kex_exchange_identification: Connection closed by remote host” [17:36:29] what license should I use for a library that I want other Toolforge tools to use (eventually ^^)? [17:36:35] sbassett: I can confirm I get the same issue but I can connect to other instances. are there any other instances in security-tools project? [17:36:37] LGPL? [17:36:55] (there’s no right answer but I’m interested in opinions ^^) [17:37:18] !log multichill@tools-bastion-12 tools.multichill Deployed jobs.yml with quality-image-add daily T319912 [17:37:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.multichill/SAL [17:38:24] sbassett: I don't have access to security-tools in Horizon, but I should have root via ssh globally. ideas: try if it works on another instance in the same project. reboot the instance via Horizon. [18:10:31] mutante: Thanks. I don’t think I have access to any other cloud instances. We had some old instances, but we retired them: adhoc-utils01.security-tools.eqiad1.wikimedia.cloud, logparse01.security-tools.eqiad1.wikimedia.cloud, gerrit-sizzle.security-tools.eqiad1.wikimedia.cloud. [18:15:13] https://openstack-browser.toolforge.org/project/security-tools also lists no other instances in the project FWIW [18:16:49] sbassett: try logging in on https://horizon.wikimedia.org , switch to "security-tools" with the drop-down menu, then "Instances" on the left, select the instance and "soft reboot" from the menu on the right [18:17:46] lucas: good point I can see that in public too [18:17:50] sbassett: https://openstack-browser.toolforge.org/user/sbassett claims you would also have access to deployment-prep [18:17:53] you could try e.g. deployment-mwlog02.deployment-prep.eqiad1.wikimedia.cloud [18:17:59] SSHing to that one works for me at the moment [18:19:07] Lucaswerkmeister: Ah yes, let me try that real quick. [18:19:43] (openstack-browser is pretty useful ^^) [18:20:59] now kinda changing my mind and thinking of going with 3BSD instead (following Flask’s example) (re @lucaswerkmeister: what license should I use for a library that I want other Toolforge tools to use (eventually ^^)?) [18:21:29] (though I’m not a fan of the pointless “copyright ” thing) [18:29:42] I’m trying to have one gitlab repo mirror to another (cc taavi who for some reason got an email about the related failure) and getting a weird error when adding a new mirror: [18:29:42] Remote mirrors url is blocked: Requests to localhost are not allowed [18:30:00] (the remote URL is not in fact localhost, it’s https://gitlab.wikimedia.org/lucaswerkmeister/tool-wd-image-positions.git) [18:30:09] any idea what’s going on there? ^^ [18:32:20] https://forum.gitlab.com/t/gitlab-webhook-configuration/36146/2 suggests there’s an admin setting to “Allow requests to the local network from web hooks and services”, did we recently change that by any chance? [18:34:20] (I’m not seeing anything that sounds relevant in puppet.git, but idk if gitlab admin settings are even tracked there) [19:49:18] taavi pointed me at the config, and https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/blob/9133f357c215e2615b71df64e69423e94a8a45d7/gitlab-prod-1002.yaml#L57 looks like `allow_local_requests_from_system_hooks` has been `false` for two years, so that’s probably not it [19:49:31] unless a newer gitlab version started to listen to that setting when mirroring :/ [21:01:13] mutante lucaswerkmeister: ftr, I can ssh into deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud just fine [21:01:39] So something seems off with sec-utils-bullseye.security-tools.eqiad1.wikimedia.cloud... [22:09:51] sbassett: I think rebooting it is still the next step then