[07:44:31] XioNoX: thank you for the review re: T419996 I'll homer the change shortly, is homer '*' ok or I can use a stricter glob ? [07:44:31] T419996: cloudcumin not able to communicate with openstack.eqiad1.wikimediacloud.org:25000 anymore - https://phabricator.wikimedia.org/T419996 [07:47:52] godog: you can look for `cr-cloud-vrf` in homer's config/devices.yaml [07:48:21] here you can do 'cr*codfw*' then a 2nd run with 'cr*eqiad*' [07:48:44] XioNoX: nice, thank you and will do [07:57:09] re-running capirca gethosts because it is outdated [07:57:35] godog: cool, you can run it without checking commit to see what would change [07:57:42] ok will do! [07:57:56] stepping away for daycare, back in ~30min [07:58:00] ack [08:00:31] looks like netbox urls have changed or scripts names are not supported anymore, https://netbox-next.wikimedia.org/extras/scripts/capirca.GetHosts/ is linked from wikitech though it says script not found, using the script id works however https://netbox-next.wikimedia.org/extras/scripts/1/ [08:08:38] ok I got an unexpected diff here, https://phabricator.wikimedia.org/P89870 about removing dse-k8s-worker1012 as a peer, investigating [08:13:53] ok status planned vs active in netbox for that host [08:14:20] I'll ask Ben [08:32:19] morning! [10:37:47] dhinus: can we repool clouddb1013:s3 now that it has caught up? [10:53:26] taavi: I'm not working today but yes, you can repool it now or I'll do it tomorrow. I wanted to make sure it was not crashing again after restarting replication. [11:48:05] * dcaro lunch [13:54:20] I'll kick off a bunch of codfw reboots in T419948 [13:58:31] ack [14:34:50] XioNoX: topranks: o/ I'm getting "Multiple definitions found for network: openstack.codfw1dev.wikimediacloud.org" when trying to run homer for https://gerrit.wikimedia.org/r/c/operations/homer/public/+/1254211, any idea why or what needs changing? [14:36:02] taavi: can you share the whole output? [14:36:25] XioNoX: https://phabricator.wikimedia.org/P89875 [14:36:56] taavi: it's probably in the capirca script output [14:36:59] in netbox [14:37:03] ohhhhhh [14:37:17] so I don't need the definitions in definitions/static.net at all? [14:37:26] was just about to say [14:37:27] double check, but most likely, yeah [14:37:31] https://netbox.wikimedia.org/ipam/ip-addresses/13148/ [14:37:34] https://netbox.wikimedia.org/ipam/ip-addresses/20074/ [14:38:10] https://gerrit.wikimedia.org/r/c/operations/homer/public/+/1254215 [14:38:30] thanks! [14:38:46] taavi: make sure it includes v4/v6 in the capirca script output too [14:39:46] yep, all good: [14:39:54] https://www.irccloud.com/pastebin/3FSGFQpX/ [14:39:57] https://netbox.wikimedia.org/extras/scripts/results/318626/ [14:40:05] it appears to be, and I'll see the actual router diff in a moment anyway [14:43:16] yeah looks good https://phabricator.wikimedia.org/P89876 [14:43:34] nice! [14:45:39] topranks: now I get a diff on cr1-eqiad changing a bunch of VRRP priorities, are those something I should commit or leave for you for later? [14:46:02] taavi: if you can leave it I will push the change in about an hour when I am done with current maintenance [14:46:37] yeah that's fine, I am not in a hurry with this [14:46:42] ok, thanks [14:47:02] I've tested the firewall rules to work in codfw, feel free to push those out at any time [15:16:50] taavi: that change is now pushed in eqiad [15:16:57] thanks! [15:16:58] looks good, it added both sets of IPs [18:35:09] * dcaro off [18:35:13] cya tomorrow!