[08:08:44] Krinkle: I'm not opposed to raising it, no limit is not viable long-term because the scripts don't run on dedicated hosts [09:33:12] hey, would it be possible for someone to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1182603 please? [09:35:43] on it urbanecm [09:36:50] thank you! [09:37:06] (done) [09:37:17] thanks again [09:37:24] np [10:11:57] o/ do we log http requests made through the webproxies? Context here is having a better understanding of how wdqs interact with the outside world. [10:13:33] dcausse: I think that we do have squid logs on teh webproxies [10:14:49] dcausse: try https://logstash.wikimedia.org/app/dashboards#/view/58c908a0-a394-11ec-bf8e-43f1807d5bc2 [10:15:19] volans: oh great, looking [12:38:48] If anyone in Infrastructure(?) can reply on T393437 as to whether I can do the work or it needs an SRE, I'd be grateful. [12:38:48] T393437: Provide nodejs22 base images for production - https://phabricator.wikimedia.org/T393437 [13:12:59] James_F: o/ tried to reply! Lemme know [13:13:32] elukey: Thanks! But we explicitly don't use Debian's extremely lagging versions of Node, and haven't for many years now? That's why these images exist. [13:14:12] elukey: See https://gerrit.wikimedia.org/r/plugins/gitiles/operations/docker-images/production-images/+/refs/heads/master/images/nodejs20/Dockerfile.template etc. [13:14:33] And AFAIK mere mortals like me can't begin to package things for thirdparty/node22 etc.? [13:15:22] James_F: I misremembered then, I didn't recall about thirdparty/node20 [13:16:03] ah right wait, I was wrong, bookworm shipped with node 18, this is why we have thirdparty/node20 [13:16:04] To SRE, "EOL" means "Debian won't ship security updates for Node itself". To those of us using, it means "all our libraries aren't even thinking of releasing security updates for it any more". We're already at that point for Node 20, even though Trixie just shipped with it. [13:16:27] Staying on Node 20 at this point is not good, let alone staying there for years. [13:16:45] James_F: this is not my understanding, node 20 is marked as maintenance, not EOL [13:17:11] anyway, I need to verify how node20 was imported for bookworm at this point [13:17:23] Yes, until October, but as I said, many of the things we rely on have already dropped Node 20 support. [13:18:11] Oh, wait, no, not October, but April 2026, as it's an LTS. Ignore me on that point. [13:18:46] I'd be delighted to do the work, but not sure if I can. [13:18:55] yeah this was my point,in theory we are good security wise (last famous words) with node 20 until April 2026 [13:19:05] Yeah. [13:19:10] I just followed up on task before seeing this discussion here: https://phabricator.wikimedia.org/T393437#11128451 [13:19:34] moritzm: Oh! Sorry. That's awesome. I can do the prod image patches and be the contact point at that end, if that'd be helpful? [13:19:43] Want to reduce load on SRE as much as possible! [13:19:47] I am very happy to see a big traction in upgrading to newer versions, don't get me wrong, I just wanted to ask what if here is a use case that we should be aware of (like new features, performances, etc.. expected from 22/24) [13:20:11] James_F: let's do that. I'll merge the patch tomorrow and sync it to the external repo [13:20:19] moritzm: Excellent. Thanks! [13:20:24] moritzm: where do we get the newer node versions from? I don't remember the origin of thirdparty/node20 for Bookworm [13:20:38] it's the official debs from Node.js [13:20:59] they suck from a Debian packaging level, but they are consistently kept up-to-date [13:21:36] like they vendor in OpenSSL, but then the official node releases also track OpenSSL updates, so it doesn't incur much overhead [13:22:02] node in Debian is ofc maintained much longer, there's a pending update for 18.x coming out in the next days with Debian-internal backports [13:22:52] ok TIL, I totally forgot/never-learnt this part, thanks for the explanation [13:23:00] but if you want to indulge in all the fun of the node ecosystem with it's Yenga tower of NPMs there's a usecase for these debs for some services [13:23:06] I thought we backported versions from higher debian releases when really needed [13:23:29] the idea is very tempting but I'll say no :D [13:24:27] the problem is that these are also not maintained for longer, like when Debian moves unstable to 24, there will be no more updates of 22 in Debian testing/sid [13:24:41] and then we end up spending really a lot of effort of maintainging such a branch [13:26:12] if we had 2-3 times the headcount, we could certainyl aim for that, but in the absence of that using the nodesource debs for interim branches not in Debian is a goof middleground [13:27:17] definitely [13:27:34] If you think this is bad, don't look at rustc vs. rustc-web from Firefox ESR. [13:27:39] (Also one of mine, sorry.) [13:33:55] James_F: if you need Rust 1.85, the forthcoming Boomworm point update in a week will update rustc-web (to facilitate the next firefox ESR update to Firefox 140.x: https://release.debian.org/proposed-updates/oldstable.html [13:34:32] (and also recent Chromium builds started to require more modern Rust as well) [13:53:23] moritzm: Oh, awesome, in that case I think we're happy to wait. [14:21:57] for the on-callers: I have depooled eqiad for kartotherian, to push all the traffic to codfw (running the new stack). This is meant to test that we can run in a single DC with the new settings etc.. [14:22:03] if you see any issues, please ping me [14:22:07] cc: moritzm [14:22:49] \o/ [14:32:51] so far all good!