[09:04:51] FIRING: [4x] PuppetFailure: Puppet has failed on ms-be2090:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [12:14:51] FIRING: [3x] PuppetFailure: Puppet has failed on ms-be2091:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [12:22:07] ^ This is my welcome pacakge, a silence I did for a week, you're welcome Mr. Swift owner [12:24:51] FIRING: [3x] PuppetFailure: Puppet has failed on ms-be2091:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [12:29:51] RESOLVED: [3x] PuppetFailure: Puppet has failed on ms-be2091:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [12:41:01] that's Dr Swift owner to you ;p [12:41:20] srs, though, thanks for silencing. [12:41:38] I checked it was WIP first [12:42:37] yeah, ms-be209[0-4] are still with dc team for installation. I'm fixing up some of them. [12:43:42] https://i.imgflip.com/acd0vf.jpg [12:45:44] marostegui: and federico3 https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/1202150?usp=email [12:46:33] Emperor: welcome back. The eqiad deletion script will finish by tomorrow [12:46:56] it's still deleting but it should be done really soon (it's on thumbor/ path now) [12:48:30] ugh, ms-be1067 needs vacuum: https://grafana.wikimedia.org/d/000000378/ladsgroup-test?forceLogin&from=now-7d&orgId=1&to=now&viewPanel=panel-28&timezone=utc [12:49:10] Amir1: I've noticed it but it's already +1ed, want me to add an LGTM? [12:49:29] federico3: mostly yes since I know that you're aware :D [12:49:35] ok [12:49:47] since this will impact you, and I don't want you to be surprised :D [12:52:01] I'll have to do some hacks to import cookbooks in unit tests but not an issue [15:47:35] marostegui: this is the panel addition I was talking about: https://grafana-rw.wikimedia.org/d/876bb257-6cc2-4898-b7de-c797e9d4bbf8/mysql-federico-s-copy?forceLogin=true&from=2025-11-04T11%3A47%3A21.346Z&orgId=1&refresh=1m&timezone=utc&to=2025-11-04T20%3A03%3A54.324Z&var-job=%24__all&var-port=9104&var-server=db2150 - second panel on the left "dbctl weight" [22:45:48] FIRING: PuppetFailure: Puppet has failed on ms-be2094:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure