[12:30:58] <godog>	 hi, I've started tomorrow's pad
[12:59:33] <volans>	 godog: thanks!
[12:59:38] <volans>	 I was about to check :
[12:59:39] <volans>	 :)
[13:01:28] <godog>	 yw volans !
[15:20:55] <hashar>	 times goes too fast
[15:21:27] <hashar>	 godog: from before lunch, I found out beta logstash no more work.  The instance spurts errors such as: 
[15:21:27] <hashar>	 [logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:11514"}
[15:21:32] <hashar>	 :exception=>#<Errno::EADDRINUSE: Address already in use - bind - Address already in use>,
[15:21:43] <hashar>	 cause rsyslog is already listening ;]   Filled as T233134
[15:21:52] <hashar>	 I guess it is due to some migration 
[15:42:25] <hashar>	 hmm
[15:42:37] <hashar>	 I have killed rsyslogd,  logstash managed to listen on udp:11514
[15:42:50] <hashar>	 so at least that is no more spamming. I have no idea whether logs are digested though ;]
[16:22:41] <hashar>	 net exploded
[16:31:09] <cdanis>	 jbond42: hm, did you just change something involving icinga?
[16:31:12] <cdanis>	 /usr/lib/python3/dist-packages/conftool-1.1.4.egg-info/PKG-INFO
[16:31:13] <cdanis>	 err
[16:31:16] <cdanis>	 check_icinga@wikitech-static.wikimedia.org found Icinga CRITICAL on icinga2001.wikimedia.org
[16:31:17] <cdanis>	 Issues of attempt 1 of 3:
[16:31:20] <cdanis>	 hostname 'icinga2001.wikimedia.org' doesn't match 'cas-icinga.wikimedia.org'
[16:31:36] <cdanis>	 the check_icinga script that runs on the outside-our-infra wikitech-static host is upset
[16:33:17] <jbond42>	 hmm yes i did, i added a second vhost on the icinga serveres which has an ssl cert for cas-icinga.wikimedia.org
[16:33:28] <cdanis>	 probably the script needs updating
[16:33:31] <jbond42>	 however it dosn't have an sni for icinga2001
[16:33:42] <jbond42>	 at least shouldn't
[16:34:28] <jbond42>	 i wonder if simply changing the apache priority will fix it
[16:36:57] <cdanis>	 I bet it would
[16:37:43] <volans>	 thanks cdanis for this, I'm in th emiddle of other chats and was not looking at email
[16:38:27] <jbond42>	 cdanis: can you check https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/538918
[16:39:27] <cdanis>	 jbond42: did not dig too deep but +1 from me
[16:40:45] <jbond42>	 thanks 
[16:41:24] <cdanis>	 jbond42: aside from the missing trailing comma on line 30 of cas.pp ;)
[16:41:46] <jbond42>	 :)
[16:47:15] <jbond42>	 cdanis: that change should be deployed now, can you re-check?
[16:47:24] <cdanis>	 yep running cronjob by hand
[16:47:35] <cdanis>	 Sep 24 16:47:21 wikitech-static.wikimedia.org check_icinga[13431]: 2019-09-24 16:47:21,028 [INFO] Check for host icinga2001.wikimedia.org: OK
[16:47:37] <cdanis>	 👍
[16:47:39] <cdanis>	 thanks jbond42
[16:47:44] <jbond42>	 cool :D
[16:47:59] <jbond42>	 ok signing off for a bit, ping iof needed
[16:48:06] <cdanis>	 yep I am getting some food as well
[16:56:17] <volans>	 cdanis: re-running a 2min cronjob by hand... no comment :-p
[16:56:47] <cdanis>	 I was watching in journalctl -f and it had literally just ran when puppet finished
[16:56:59] <cdanis>	 it was going to be the full two minutes, that's far too long for a millennial like myself to wait
[16:57:26] <volans>	 ahahahah