[11:54:15] Raymond_Ndibe: thanks for the harbor upgrade yesterday [12:27:29] 🙏🙏🙏 [13:24:09] dhinus: can I get a review of https://gerrit.wikimedia.org/r/c/operations/alerts/+/1118798/1/team-wmcs/bastionless.yaml? The associated metric is defined in https://gerrit.wikimedia.org/r/c/operations/puppet/+/1118526 [13:34:09] andrewbogott: looks good, let me double check a couple things [13:43:11] andrewbogott: +1d [13:43:18] thank you! [13:43:35] it will fire immediately as I left one broken example in place for verification :) [13:45:51] ack [14:11:01] ok, there it's firing, now I'll fix it and we'll see if it recovers... [18:47:21] re:toolsdb recent crashes, I added more info to T385900 [18:47:21] T385900: [toolsdb] mariadb crashing repeatedly on primary host - https://phabricator.wikimedia.org/T385900 [21:17:58] bd808: could I get your opinion on https://phabricator.wikimedia.org/T380679? You can just scroll down to the last comment. I'm interested in your thoughts about whether to hack in a CNAME or to just let those two hostnames die. [21:18:06] and also if there are other cases that I should be testing for [21:18:51] taavi, same question to you if you happen to be lurking [21:29:40] andrewbogott: if CNAMES are easy then that lets you kick the can down the road some additional years before breaking the abandoned but functional tools that may possibly be using the older service names. [21:30:24] Or make a hard deprecation plan, communicate it well, and help folks deal with the fallout. [22:29:19] that makes me think you expect there to be fallout! So I will start with cnames. I was hoping you would say something like "no one ever really used those anyway" [23:11:21] andrewbogott: I honestly have no idea. I think in the past people have imagined that we could have some data from theDNS system on which things were being queried, but as I recall that was wishful thinking. [23:11:50] hmmm that might be possible actually [23:13:12] You would want data from at least a full calendar month to catch traffic from scripts that only run monthly. Things that run less often than that probably have to deal with unexpected changes unfortunately. [23:14:46] The resolver data would likely only tell you that something was using the service name, but probably not what was using it, or really even how many things were using it.