[03:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [03:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [03:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [03:20:35] 06Machine-Learning-Team, 13Patch-Needs-Improvement: Outlink returns 500 when EventGate returns 503 Service Unavailable - https://phabricator.wikimedia.org/T346136#11376877 (10Pppery) [07:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [07:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [07:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [11:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [11:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [11:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [15:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [15:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [15:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [16:35:44] (03PS1) 10Zabe: Drop support for filtering model for rc_type [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1205974 (https://phabricator.wikimedia.org/T74157) [16:38:15] (03PS2) 10Zabe: Drop support for filtering model by rc_type [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1205974 (https://phabricator.wikimedia.org/T74157) [19:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [19:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [19:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas [23:17:04] FIRING: KubernetesDeploymentUnavailableReplicas: ... [23:17:04] Deployment aya-llm-predictor-00003-deployment in llm at eqiad has persistently unavailable replicas - https://wikitech.wikimedia.org/wiki/Kubernetes/Troubleshooting#Troubleshooting_a_deployment - https://grafana.wikimedia.org/d/a260da06-259a-4ee4-9540-5cab01a246c8/kubernetes-deployment-details?var-site=eqiad&var-cluster=k8s-mlserve&var-namespace=llm&var-deployment=aya-llm-predictor-00003-deployment - ... [23:17:04] https://alerts.wikimedia.org/?q=alertname%3DKubernetesDeploymentUnavailableReplicas