[10:20:43] <onimisionipe>	 Krenair: will merging https://gerrit.wikimedia.org/r/c/operations/puppet/+/511120 make cloudelastic happy?
[10:36:31] <wikibugs>	 10netops, 10Operations, 10Patch-For-Review, 10cloud-services-team (Kanban): CloudVPS: evaluate if we can make rsync use 10G in cloudvirts - https://phabricator.wikimedia.org/T223272 (10aborrero) 05Open→03Resolved We got a bit of speed increase. Closing for now.
[12:11:18] <Krenair>	 onimisionipe, it might work
[12:11:20] <Krenair>	 but I made that by looking at the error and taking an educated guess as to what exactly went wrong in puppet
[12:11:27] <Krenair>	 as a puppet.git reviewer I imagine you'll want to run PCC against it
[12:23:08] <onimisionipe>	 Krenair: not sure about your comment here: https://gerrit.wikimedia.org/r/c/operations/puppet/+/511381
[12:23:30] <onimisionipe>	 I changed the bug line to refer to the original task of exposing wmf to cloud
[12:23:45] <onimisionipe>	 and not the nginx failed task
[12:34:13] <vgutierrez>	 hmm why are you getting rid of ocsp_proxy? I think it could be useful to allow ocsp_proxy to be undef, so Optional[String] $ocsp_proxy
[12:34:25] <vgutierrez>	 instead of the existing String $ocsp_proxy
[12:37:16] <onimisionipe>	 vgutierrez: but its not being used
[12:38:00] <vgutierrez>	 meaning you're not using OCSP at all, right?
[12:39:24] <onimisionipe>	 no we are not. Eric didnt even set the $do_ocsp flag
[12:41:06] <vgutierrez>	 right
[12:41:36] <vgutierrez>	 Krenair: re https://gerrit.wikimedia.org/r/c/operations/puppet/+/511120 would you mind grouping both if $do_ocsp?
[13:50:21] <Krenair>	 done
[14:00:49] <vgutierrez>	 hmmm considering that the default value for $certs is [] I think we don't need the !empty() check anymore, do we?
[14:02:30] <Krenair>	 true
[14:02:57] <Krenair>	 you know there is a reason I usually avoid doing this type of cleanup in such commits until a +2er requests it
[14:35:32] <vgutierrez>	 bblack: now that we have the vlanID interface naming scheme, we could proceed with lvs2010
[14:43:40] <bblack>	 vgutierrez: I've lost all context on where we've left off with lvs2007-10
[14:43:42] <bblack>	 checking ticket
[14:44:00] <vgutierrez>	 so basically we were blocked by the vlan interface length issue
[14:44:29] <vgutierrez>	 + the NIC hangout issue
[14:44:41] <vgutierrez>	 both are solved now
[14:44:54] <bblack>	 the firmwares are all updated too? https://phabricator.wikimedia.org/T196560#4927737
[14:45:13] <bblack>	 yeah and looks like 2009 + 2010 should be ready for puppetization
[14:45:13] <vgutierrez>	 I don't have confirmation from DCops, but I can check it / do it myself
[14:45:19] <bblack>	 2007/8 still need some cabling
[14:45:42] <vgutierrez>	 yeah, I'm reimaging lvs2010 cause it was in a weird state after all the tests that para.void made there
[14:46:01] <bblack>	 I'm gonna take a look at the 1013-16 transition stuff today too and see if I can get it all switched around to the new stuff while depooled
[14:46:07] <vgutierrez>	 (basically not reachable via the production interface)
[14:46:11] <vgutierrez>	 ack
[14:46:12] <bblack>	 ok
[14:47:47] <vgutierrez>	 of course our lvs like 2010 will suffer another interface renaming process after upgrading to buster
[14:48:01] <vgutierrez>	 **predictable**
[14:50:39] <bblack>	 :)
[14:53:41] <vgutierrez>	 onimisionipe: so https://gerrit.wikimedia.org/r/c/operations/puppet/+/511120 looks good IMHO, but it will require some manual cleaning on your side
[14:54:13] <onimisionipe>	 vgutierrez: Ok
[14:55:04] <onimisionipe>	 so removing the ocsp_proxy should remove all ocsp related stuff on cloudelastic according to PCC
[14:55:15] <onimisionipe>	 dunno if that's what you mean by cleaning
[14:56:30] <vgutierrez>	 what's currently preventing nginx to re(start) on cloudelastic is /etc/systemd/system/nginx.service.d/ocsp.conf
[14:56:52] <vgutierrez>	 as you can see on pcc, that file is only on the old catalog https://puppet-compiler.wmflabs.org/compiler1001/16632/cloudelastic1003.wikimedia.org/
[14:57:16] <vgutierrez>	 but removing it from the catalog doesn't remove it from the filesystem
[14:57:41] <onimisionipe>	 Oh..
[14:57:45] <onimisionipe>	 ok then
[14:57:55] <onimisionipe>	 I can do that.
[14:58:23] <vgutierrez>	 check pcc, there is some stuff to be removed
[14:58:55] <onimisionipe>	 Ok
[15:00:55] <vgutierrez>	 so let's merge it
[15:02:19] <vgutierrez>	 onimisionipe: take into account the icinga checks regarding OCSP as well
[15:02:22] <vgutierrez>	 sorry for the mess :(
[15:02:37] <vgutierrez>	 let me know if you need help with the cleaning
[15:09:16] <onimisionipe>	 Ok. no p. Thanks!
[15:16:29] <bblack>	 I got held up on the lvs1013-15 stuff, ran into an odd mystery about /e/n/i randomly not working right for the primary interface on bootup
[15:16:43] <bblack>	 still digging into the wtf
[15:17:12] <bblack>	 somtimes I get this on bootup in the logs:
[15:17:13] <bblack>	 May 20 14:57:00 lvs1013 sh[736]: RTNETLINK answers: File exists
[15:17:13] <bblack>	 May 20 14:57:00 lvs1013 sh[736]: ifup: failed to bring up enp4s0f0
[15:17:14] <bblack>	 May 20 14:57:00 lvs1013 systemd[1]: ifup@enp4s0f0.service: Main process exited, code=exited, status=1/FAILURE
[15:17:38] <bblack>	 and then none of the interface_tweaks things that execute via /e/n/i are applied to just that interface (the primary one), whlie the rest are applied ok
[15:17:58] <bblack>	 may be some kind of race, since sometimes it does work
[15:22:32] <onimisionipe>	 vgutierrez: I'll confirm noop on search cluster 
[16:15:38] <wikibugs>	 10Traffic, 10DNS, 10Operations: GSuite Test Domain Verification - https://phabricator.wikimedia.org/T223921 (10HMarcus)
[16:21:06] <wikibugs>	 10Traffic, 10DNS, 10Operations: GSuite Test Domain Verification - https://phabricator.wikimedia.org/T223921 (10HMarcus)
[16:22:26] <wikibugs>	 10Traffic, 10DNS, 10Operations: GSuite Test Domain Verification - https://phabricator.wikimedia.org/T223921 (10HMarcus)
[17:11:57] <wikibugs>	 10netops, 10Operations, 10cloud-services-team (Kanban): network hardware: drop nova-network related configuration - https://phabricator.wikimedia.org/T223925 (10aborrero)
[20:38:57] <wikibugs>	 10netops, 10Operations, 10cloud-services-team (Kanban): network hardware: drop nova-network related configuration - https://phabricator.wikimedia.org/T223925 (10Volans) p:05Triage→03Normal
[20:47:59] <wikibugs>	 10Traffic, 10Cloud-Services, 10Operations, 10cloud-services-team, and 3 others: Deprecate `base::service_unit` in puppet - https://phabricator.wikimedia.org/T194724 (10jijiki)