[08:28:10] hmmm.... is there a way to use novaobserver credentials from outside cloudvps? (ex, from my laptop) [08:42:12] morning. I never tried myself, but it would be nice [08:56:17] I got it working so far by using a socks proxy :/ [08:56:26] I'm trying to fix some stuff in openstack-browser [08:56:30] I'll send a patch [08:56:55] gtg now for a bit do some chores, [08:57:00] I'll be back after lunch [13:08:44] dhinus, thanks for figuring out that swift issue before I got up to my neck in the swift API! [13:10:18] dcaro: the answer to where you can and can't use novobserver creds directly is... [13:10:21] https://www.irccloud.com/pastebin/lMaDezkx/ [13:10:51] There's no real security reason to limit it to internal traffic other than avoiding ddos [13:10:53] oh.... so I have to safelist my ip? xd [13:11:13] Yeah, it's roughly the same issue that taavi ran into when doing local horizon dev [13:11:51] that safelist is a custom WMCS thing, I'm still /pretty sure/ that it's useful but it's less meaningful right now when idp doesn't enforce 2fa anyway... [13:12:45] yep, not sure if it's worth it now that I figured out how to use the socks proxy xd [13:12:54] (for development) [13:14:34] not worth it to change, you mean, or not worth it to keep it? [13:15:35] not worth it to change xd [13:15:46] (as in, keep it as is and avoid potential ddos) [13:17:00] yep, ok [13:17:31] I'll add a note in the readme though for the next time :) [13:18:58] yep the only downside is that it's not obvious why it doesn't work if you try, and don't know about that list :) [13:19:30] but maybe it's a good thing to keep shared accounts like novaobserver more restricted? [13:21:06] does it apply to password auth only, or to app creds as well? [13:21:26] * dhinus is not even sure you can/should create app creds for novaobserver [13:22:22] I think it's only for password [13:22:47] though afaik app creds are per-project/tenant, so maybe they can't do things like list all projects? [13:22:48] can you? [13:25:28] yeah, specifically only password auth [13:26:07] it should be possible to do cloud-wide queries with app creds, at least the policies allow it [13:26:22] the APIs aren't totally consistent about how to request things like that though [14:59:11] We have like two toolforge member requests that I am a bit unsure what to do with. [14:59:30] https://toolsadmin.wikimedia.org/tools/membership/status/1971 doesn’t sound like a good usecase for toolforge [15:00:56] https://toolsadmin.wikimedia.org/tools/membership/status/1968 is likely not something we want to support, but it someone can take a look that’d be nice [15:00:57] Raymond_Ndibe: it sounds like a request for wiki replica access, but they might be sad when they find out that the replicas do not have wikitext [15:01:56] that second one reads like someone wrote "write a smart sounding answer" into an LLM [15:04:20] Yes I agree about the second one. Account was also created the same day the request was submitted so looks fishy [16:26:05] andrewbogott: T399596 is my attempt at writing up the user_data failure problem. [16:26:06] T399596: http://169.254.169.254/openstack/latest/user_data semi-regularly unavaliable during Magnum Kubernetes cluster builds - https://phabricator.wikimedia.org/T399596 [17:11:16] andrewbogott: is it at all helpful to leave broken instances around for that user_data issue? The one I have right now is blocking getting the k8s cluster back up for other work, but I can find different things to work on if leaving it will help you debug things in the near term. [17:11:36] if the curl still fails on the VM then it's definitely useful. [17:12:21] good question... I can look after the meeting I'm in now to see what's up. [17:17:40] i guess that assumes you have ssh on the broken VMs which you may not [17:34:24] Raymond_Ndibe: I agree with bd808 on the toolforge request, for the first you might want to ask them to try paws instead [17:48:45] andrewbogott: sshd doesn't seem to be runningt on the zuul-k8s-v128-j2bcize4geug-master-0.zuul.eqiad1.wikimedia.cloud instance. I have forgotten where to find the instructions on entering an instance via more direct means. [17:49:58] there's a console cookbook you can run from cloudcumin1001 [17:50:25] it might not work though for magnum VMs [17:50:53] iirc there was some ssh key that was configurable, it showed in the UI (optional though) [17:54:55] I tried to get in using the keypair that Magnum attaches to the instances for me. port 22 gave a Connection refused response. I used to know how to enter an instance from the hypervisor but I can't remember the right keywords to find that doc on wikitech. [17:55:32] * bd808 will try the cookbook [17:57:31] hmm... it looks like it connected but the console is not responsive to input [18:26:54] info dumped at https://phabricator.wikimedia.org/T399596#11006029 which is probably not helpful for debugging the suer_info service failure, but at least I learned a few things about the stack along the way. [19:23:58] thanks bd808! Unfortunately I'm still digging into ceph issues so it'll be a while before I can pay a lot of attention to that.