[07:00:23] Is https://wikitech.wikimedia.org/wiki/Incident_status still the right place to publish incidents? I'm surprised to only see 1 incident report for 2026. [07:00:36] Are we getting really good? Or just less transparent? [08:42:53] gehel: people start in gdocs and then it goes to rot [08:43:07] but ideally yes, it should be published on wikitech [12:09:19] gehel: coincidentally, there was a reply about this on wikitech-l earlier today: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/2KJ6E5ID5W5MUSGYIIUKXPYDSQXNE5ES/ [12:11:22] (oh wait, that reply was sent after your original message here, lol) [12:23:16] sukhe: federico3 I will deploy a change which in theory should not cause issues, I will let you know if it does [12:29:40] it is not going well, I am rolling back [12:36:24] I was wrong, it works [12:37:50] ok I am moving to eqiad [13:01:35] godog: probably-stupid pontoon q> how do I adjust the OS version of individual hosts? I can locally-patch specmap.yaml to arrange for an entire role to be, say, bookworm; but what if I want to test "re-image one pontoon host as trixie"? [13:13:21] Emperor: FYI he's out today [13:13:59] ah, cool, I'll drop him an email, thanks. I did check the SRE vacations calendar [13:15:09] yeah I'm checking why the event in his calendar is not in the vacation one [13:30:47] can someone help me understand why some hosts that have been in puppet for a few days now, they are not on CI ? https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler-puppet7-test/6719/console [13:32:41] effie_: try https://wikitech.wikimedia.org/wiki/Help:Puppet-compiler#Manually_update_production [13:32:42] I would check if the mechanism describe in https://wikitech.wikimedia.org/wiki/Help:Puppet-compiler#Updating_nodes has any issue [13:32:49] volans: :D [13:33:12] :D [13:37:57] Emperor: that's fixed now (I can't say exactly why that was though) [13:57:35] oncallers o/: Assuming no one objects, I'm going to upgrade the sessionstore service to a Trixie-based image (new Go version, new Cassandra driver, etc). I don't expect issues, but I never do! [14:17:59] and {{done}} [15:56:48] Does anyone here have experience building debian packages that contain rust crates? I could use a consult. [15:57:51] I decided to opt out of dependency hell and just do an online build, but that is /also/ failing for reasons I can't isolate. [15:58:37] andrewbogott: I tried in 2019 :) https://phabricator.wikimedia.org/T220836 [16:00:16] andrewbogott: haven't done this for rust but IMO for use @ WMF, just vendoring the deps in-repo is totally fine for similar languages like Go [16:00:33] hm... since rush uses a .lock file and checksums I'm not too worried about the online build aspect since it should be reproducible. [16:01:25] did https://crates.io/crates/cargo-deb work? [16:01:38] But the rust build command (that works on its own) fails when it's invoked by dpkg-buildpackage. it seems like dpkg-buildpackage is disabling online builds somehow... [16:02:03] cdanis: ah, yes, you are describing the 'dependency hell' approach [16:02:05] which is still an option [16:02:12] ehehe [16:03:10] I was imagining that there was some way of telling dpkg-buildpackage "just go ahead and allow online building" and that would get me there. But maybe my issue is more complicated than that. [16:03:17] andrewbogott: https://gitlab.wikimedia.org/repos/sre/CIDERGRINDER/-/blob/main/debian/rules for a very minimal/brutalist example of what I do in golang [16:03:56] did you do `cargo vendor` ? [16:04:59] I think that at the moment the makefile is ignoring existing files, but let me see if I can get the crates out of band and get it to use them... [16:05:03] thank you for the sample! [16:07:44] I am getting slightly further now! [16:13:35] now maturin is mad about the lack of Cargo.toml.orig files [16:34:57] ...which is apparently a maturin bug :( [21:10:40] Small CR for K8s services proxy config if anyone feels comfortable taking a look: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1286504