[05:51:17] <marostegui>	 We are going to take over the mediawiki-config repo for the x1 failover. Please coordinate with us before merging/Deploying something
[06:05:06] <marostegui>	 The failover is done, deployments and merges on mediawiki-config can resume as usual
[06:09:39] <elukey>	 nginx is the only submodule left in operations/puppet :)
[06:12:14] <apergos>	 nice! (to both)
[06:12:22] <vgutierrez>	 \o/
[06:28:02] <elukey>	 should we also move nginx to operations/puppet? Shouldn't be difficult
[06:28:10] <elukey>	 so we'll be submodules free finally
[06:28:18] <apergos>	 I would love it, personally
[06:31:40] <elukey>	 I have tested the procedure that Joe proposed (rm submodule, copy to environments/prod, merge, copy to modules/etc.) 4 times, no glitch
[06:32:21] <apergos>	 just wondering if there is some reason outside my knowledge that we want nginx kept as a submodule but I can't imagine why that would be
[07:08:50] <moritzm>	 elukey: +1 on doing that, I doubt anyone is using that outside of us, maybe just put in on next SRE meeting's agenda to give anyone a chance to object
[07:24:21] <moritzm>	 I've just updated our buster d-i image to rc3, if anyone is installing a buster host today and runs into any issues, please let me know
[09:57:44] <elukey>	 FYI all the YARN NodeManager Node-State
[09:58:04] <elukey>	 unknowns are due to icinga still catching up with the removal of a per host check
[09:58:16] <elukey>	 should go away after the next puppet runs on icinga1001
[10:45:04] <jbond42>	 @all i plan to deploy the hira search order change today at 11:30 UTC.  please vocide any concerns now if you have them https://gerrit.wikimedia.org/r/c/operations/puppet/+/511686
[10:46:57] <moritzm>	 sounds good
[10:50:39] <apergos>	 I'm looking at the snapshot differences from the puppet compiler
[10:50:46] <apergos>	 https://puppet-compiler.wmflabs.org/compiler1002/17020/snapshot1006.eqiad.wmnet/
[10:51:04] <apergos>	 meh it doesn't matter, just the order
[10:51:12] <apergos>	 fine by me then
[10:51:31] <jbond42>	 ack
[11:00:03] <arturo>	 jbond42: +1 good for me
[11:30:13] <jbond42>	 thanks all pushing now
[20:35:07] <ottomata>	 andrewbogott: o/
[20:35:09] <ottomata>	 https://phabricator.wikimedia.org/T225128#5304749
[20:35:18] <ottomata>	 is there anything special I should know here?
[20:35:44] <ottomata>	 i want to role spare::system to cloudvirtan so we can move them back to analytics prod
[20:35:51] <ottomata>	 don't want to accidnetally trigger any alerts
[20:36:04] <andrewbogott>	 Those boxes currently have two networks hooked up, for control plane and virt plane.  You probably only need one if you're going to manage them as bare metal.
[20:36:32] <andrewbogott>	 I would just downtime all the existing hostnames for a year or two, assuming that they'll be renamed/rebuilt before the downtimes expire.
[20:36:45] <ottomata>	 ok, they will with new hostnames.
[20:36:48] <andrewbogott>	 I don't think they'll trigger any alerts that aren't associated with the hostnames directly.
[20:36:50] <ottomata>	 ok great.
[21:01:32] <cdanis>	 anyone able to look at the error from I think our Debian mirror on sodium?  Subject: [ftpsync@sodium] ERROR: rsync errors
[21:03:14] <mutante>	 cdanis: yea, i can look
[21:04:23] <mutante>	 too many deletions for a --max-delete-limit .. but what did it want to delete
[21:09:28] <mutante>	 it's various installers from sid that get removed
[21:11:04] <mutante>	 i think it will fix itself in <12h, next time we get pushed to. it deleted the max number of files it was allowed and the remaining ones should just be deleted next time
[21:12:15] <mutante>	 ftpsync = rsync but they are pushing to us (Debian mirror, as opposed to the Ubuntu mirror where we pull)
[21:16:27] <cdanis>	 ahh that makes sense
[21:16:39] <cdanis>	 I was thinking it was probably something having to do with it being close to buster release time
[21:16:47] <cdanis>	 thanks :)
[21:18:23] <mutante>	 yw. it deleted 40000 and skipped 2254 or so.. should be fine next run
[21:25:00] <apergos>	 busterday soon
[21:28:57] <mutante>	 https://disney.fandom.com/wiki/Buster_(Toy_Story)
[21:38:23] <apergos>	 heh