[11:28:06] <elukey>	 !incidents
[11:28:06] <sirenbot>	 5692 (UNACKED)  [2x] ProbeDown sre (dse-k8s-ctrl1001:6443 probes/custom eqiad)
[11:28:12] <elukey>	 !ack 5692
[11:28:13] <sirenbot>	 5692 (ACKED)  [2x] ProbeDown sre (dse-k8s-ctrl1001:6443 probes/custom eqiad)
[11:35:20] <elukey>	 !incidents
[11:35:20] <sirenbot>	 5692 (RESOLVED)  [2x] ProbeDown sre (dse-k8s-ctrl1001:6443 probes/custom eqiad)
[11:35:24] <elukey>	 gooood
[11:37:00] <elukey>	 !incidents
[11:37:00] <sirenbot>	 5693 (UNACKED)  [2x] ProbeDown sre (dse-k8s-ctrl1002:6443 probes/custom eqiad)
[11:37:00] <sirenbot>	 5692 (RESOLVED)  [2x] ProbeDown sre (dse-k8s-ctrl1001:6443 probes/custom eqiad)
[11:37:06] <elukey>	 !ack 5693
[11:37:07] <sirenbot>	 5693 (ACKED)  [2x] ProbeDown sre (dse-k8s-ctrl1002:6443 probes/custom eqiad)
[11:37:12] <elukey>	 what the hell
[11:52:00] <elukey>	 !incidents
[11:52:01] <sirenbot>	 5693 (RESOLVED)  [2x] ProbeDown sre (dse-k8s-ctrl1002:6443 probes/custom eqiad)
[11:52:01] <sirenbot>	 5692 (RESOLVED)  [2x] ProbeDown sre (dse-k8s-ctrl1001:6443 probes/custom eqiad)
[11:54:02] <elukey>	 I have silenced the alerts for dse-k8s-ctrl100* since given the errors in the logs it may be something that depends on running pods/jobs, that shouldn't page over the weekend in my opinion
[11:54:13] <elukey>	 (also added a silence for the specific HTTP probe)
[11:54:33] <elukey>	 Cc: brouberol, btullis (as FYI for tomorrow, no action needed now!)