[08:52:25] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005 (10ayounsi) 03NEW [08:52:38] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10186833 (10ayounsi) [09:02:59] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10186931 (10cmooney) > We should write a cookbook that iterate over all network devices and run the safe command request system configuration rescue save > Anot... [09:55:44] 10netops, 06cloud-services-team, 10Cloud-VPS, 06Infrastructure-Foundations, 06SRE: openstack: initial IPv6 support in neutron - https://phabricator.wikimedia.org/T375847#10187090 (10cmooney) >>! In T375847#10182667, @aborrero wrote: > I see the dhcp6 packets from my test VM arriving into neutron: > > `... [10:06:06] 10netops, 06cloud-services-team, 10Cloud-VPS, 06Infrastructure-Foundations, 06SRE: openstack: initial IPv6 support in neutron - https://phabricator.wikimedia.org/T375847#10187153 (10cmooney) @aborrero the network assignment is incorrect also. 2a02:ec80:a100::/56 is the entire public IPv6 allocation for... [10:34:57] 10netops, 06cloud-services-team, 10Cloud-VPS, 06Infrastructure-Foundations, 06SRE: openstack: work out IPv6 and designate integration - https://phabricator.wikimedia.org/T374715#10187215 (10cmooney) Guys I would propose the following: * We delegate the allocated 'public' and 'private' ranges to the codf... [11:49:40] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10187345 (10ayounsi) From JTAC : > You can periodically take a vmhost snapshot of the device to avoid losing configurations. > On the device, back up the snapsho... [12:58:58] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10187510 (10cmooney) Ok. Well that's not something we can realistically do after every homer run. Cookbook might be an idea? Although the interactive yes/no p... [14:04:02] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Move sretest2002 primary uplink to asw-d4-codfw - https://phabricator.wikimedia.org/T370475#10187693 (10Jhancock.wm) 05Open→03Resolved taken care of! [14:31:32] fyi, we're going to use sretest1002 for our UEFI testing [14:37:08] if you want you can also reset sretest2001 and use it (supermicro) [14:37:46] thanks elukey [14:39:57] in theory it should be uefi ready, right after reset [14:39:59] in theory :D [15:02:47] 10netops, 06Infrastructure-Foundations, 06SRE: Upgrade Management routers to 22.4R3-S2 - https://phabricator.wikimedia.org/T369504#10187945 (10Papaul) [15:16:39] 10netops, 06Infrastructure-Foundations, 06SRE: Upgrade Management routers to 22.4R3-S2 - https://phabricator.wikimedia.org/T369504#10188045 (10ssingh) I am assuming this means a site depool during this period? [15:20:57] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10188101 (10ssingh) Question: > Cookbook might be an idea? Although the interactive yes/no prompt could be annoying to deal with. The 'yes/no' prompts I am gu... [15:28:41] summary of our first day here: https://phabricator.wikimedia.org/T376014#10188117 [15:33:35] 10netops, 06Infrastructure-Foundations, 06SRE: Upgrade Management routers to 22.4R3-S2 - https://phabricator.wikimedia.org/T369504#10188140 (10Papaul) @ssingh no site depool only management will not be available during the maintenance [15:34:01] 10netops, 06Infrastructure-Foundations, 06SRE: Upgrade Management routers to 22.4R3-S2 - https://phabricator.wikimedia.org/T369504#10188142 (10ssingh) >>! In T369504#10188140, @Papaul wrote: > @ssingh no site depool only management will not be available during the maintenance Thanks Papaul! [16:29:25] 10netops, 06Infrastructure-Foundations: Juniper: regularly run `request system configuration rescue save` - https://phabricator.wikimedia.org/T376005#10188431 (10Dzahn) I am wondering if you can (one-time) configure the devices to "never boot rescue" config and then not have to worry about running the "save" c... [16:55:30] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, 06SRE: cr3-ulsfo incident 22 Sep 2024 - https://phabricator.wikimedia.org/T375345#10188560 (10RobH) Entered ticket 00981959 for this swap to take place today: > Support, > > We would like to request remote hands to assist in retriving a s... [17:15:08] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, 06SRE: cr3-ulsfo incident 22 Sep 2024 - https://phabricator.wikimedia.org/T375345#10188649 (10RobH) Ticket accepted and they've retrieved the replacement shipment, should get pinged from them shortly to start the swap. [18:06:53] 10netbox, 06Infrastructure-Foundations: Evaluate usage of Kubernetes/Wikikube Tags in netbox and replace them with something if possible - https://phabricator.wikimedia.org/T354169#10189014 (10CDanis) A tag or custom field that indicated the name of the k8s cluster associated with each prefix would be suuuuper... [18:27:35] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-ulsfo, 06SRE: cr3-ulsfo incident 22 Sep 2024 - https://phabricator.wikimedia.org/T375345#10189124 (10RobH) Swap completed and @papaul confirms they can attach via serial console. The onsite portion of this troubleshooting and repair should now be...