[15:42:24] !log libraryupgrader arm keyholder [15:42:25] taavi: Unknown project "libraryupgrader" [15:42:25] taavi: Did you mean to say "tools.libraryupgrader" instead? [15:42:35] !log library-upgrader arm keyholder [15:42:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Library-upgrader/SAL [15:43:15] oops [17:31:43] Hi folks, I'd like your help migrating labweb-ssl LB VIP to IPIP encapsulation and maglev as part of the ongoing migration to liberica (T387305) i've already submitted the CR https://gerrit.wikimedia.org/r/c/operations/puppet/+/1123005, I'd need your help reviewing the CR and coordinating the migration with me, I can happily take care of all the on-hands work. Thanks! [17:31:43] T387305: Migrate labweb-ssl LB VIPs to IPIP encapsulation - https://phabricator.wikimedia.org/T387305 [17:44:35] vgutierrez: do you need someone with network knowledge? (If so, maybe waiting for artu.ro to come back from PTO might be best) [17:46:45] at least somebody that's aware of the implications of disabling reverse path filtering in the instance and MSS clamping using iptables [17:47:27] clamping is pretty specific so it only affects the vip and the service port [22:12:55] hi, is it possible that the ipoidopensearch.ipoidopensearch.eqiad1.wikimedia.cloud instance (T359298) was deleted? [22:12:55] T359298: Request creation of ipoid-opensearch VPS project - https://phabricator.wikimedia.org/T359298 [22:14:08] kostajh: I see it in horizon, but it is marked as being in state "Error" [22:14:41] I can't see it in Horizon anymore [22:15:33] oh, I see it [22:15:35] ok, confusing UI [22:15:55] There is some data I have never actually seen before in the view I have. "Invalid volume: duplicate connectors detected on volume 3d271bfd-55c7-4c0e-8c9e-c14deefa49e3" [22:16:16] hmm. What are our options? [22:16:45] should I try "shut off" and "start" instances? [22:17:37] I'd guess that you need help from andrewbogott to figure out how to resurrect the instance. I only see options to edit, "lock" (not sure what that is) or delete. [22:17:54] eidt is jsut changing security groups [22:20:23] The"fault" section that is new to me on the instance detail screen is dated Feb. 6, 2025, 2:22 p.m. That would roughly line up with the reboots andrewbogott did for https://lists.wikimedia.org/hyperkitty/list/cloud@lists.wikimedia.org/thread/ZOCPVXX6BKLC76OHMIQW26YLBCKEBTGQ/ [22:20:36] I see "You are not allowed to shut off instance" when I try to switch it off [22:22:12] other than writing up a bug report and waiting for andrew's help your main option is to make a new instance to replace the busted one [22:26:19] d.hinus or d.caro might be able to help too, but they should both be gone for the day. a.ndrewbogott may be as well. I think he's working AST hours (UTC-4) right now. [22:31:54] thanks