[07:21:41] I just got kicked out off cumin2003 and can't get back to it...? [07:23:50] Emperor: bad timing, I just rebooted the last bastion (2003) you might have gone through it, should be good now [07:26:18] ah, yes, back in OK. [07:26:52] Emperor: how are the reimages going? [07:43:49] elukey: early days thus far on ms backends, but no drama (yet) [07:57:46] goood [08:22:12] Argh, I accientally ctrl-c'd a puppet-merge run. Can I somehow re-do the push-to-all-pms bit without a new change? Some of it completed, but one run was interrupted, I think [08:29:06] in theory the next puppet-merge should fix it, lemme check if we can do anything [08:34:47] filed https://gerrit.wikimedia.org/r/c/operations/puppet/+/1311395 to clear the state, just to be sure [08:35:16] ty! [08:35:17] in theory there is a combination of puppet-merge.py that I can use to fix it, but I don't feel adventurous today [08:38:59] klausman: all good [08:39:46] grazie mille! [08:41:38] Emperor: nobody is kicking anybody out of cumin2003 [08:41:55] we would ask kindly you vacate the premises in an orderly fashion [09:41:27] hey folks I'd need to reboot netbox1003 and cumin1003 [09:41:43] so ideally I am looking for a time frame where we could stop temporarily to run cookbooks [09:42:33] I see that claime, Emperor, Raine, cezmunsta are running cookbooks [09:42:44] do you have a preference? [09:46:17] elukey: I am not running any right now, so I can hang on before running any more in case that suits others [09:46:54] one alternative is for me to do it tomorrow morning, I can send an email now so people can better organize their work [09:46:57] let's do that [09:47:02] +1 [09:47:58] elukey: sorry, my cookbook on cumin1003 is a doomed re-image of one of the old-style ms-be nodes. Could be done in half an hour, I might be at it for longer :-/ [09:48:25] elukey: I have post-oncall TOIL tomorrow, so tomorrow works for me :) [09:49:38] elukey: Wait why ? [09:49:55] elukey: cumin2003 is rebooting today for network maintenance so I switched my cookbook run to 1003 [09:50:14] Ah, yeah, tomorrow works [09:50:31] claime: don't worry I was just asking if it was ok to do it now, but I'll do it tomorrow morning instead so people will know it in advance [09:50:38] elukey: <3 [09:50:39] thanks for the feedback <3 [13:00:55] federico3 OK if I merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1311427 along with my change? [13:01:28] yes please inflatador [13:02:23] federico3 ACK, the Deed is Done [13:02:44] tnx [19:32:54] If I want to decommission some VMs, do I remove them from Puppet first or run decom cookbook first? [19:37:33] inflatador: the dilemma is usually that if you remove it from site.pp first the decom bookbook won't find it.. but if you don't remove it it will say "omg, still finding this host in code". I would recommend to remove them from everything except site, run the cookbook, when it says "still here" say "i know what I'm doing" and then as the last step remove from site. should be the same regardless if [19:37:39] VM or physical [19:41:26] mutante thanks for the advice, working on it now [20:15:44] ah, looks like I'm gonna need to do some LVS stuff first anyway