[13:04:37] o/ [13:20:43] \o [13:20:53] o/ [15:01:32] feel like i talk way too fast, but still came out ok [15:12:30] I thought it was pretty cool [15:15:09] thanks! hopefully that means it made sense [16:37:47] dinner [16:56:19] i don't understand what's going wrong with the cloudelastic backfiller, or why it worked in eqiad/codfw and not cloudelastic :S [16:56:58] i think it did run and backfill and just isn't shutting down, but not 100% sure [18:54:51] * ebernhardson wonders if there is any benefit to rearranging a subset of titles for autocomplete. For example `2026 spain wildfires` could, even without a redirect, be rearranged to `spain wildfires 2026` [18:55:13] not as a replacement, but to augment [20:00:30] do we have any kind of automation for loading data into relforge? I just merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/1311440 (ECS logs) and while I'm sure we'll get some logs, it might be more exciting if there were indices ;) [20:01:41] inflatador: not really, you could probably do a snapshot/restore and get the extra bonus of testing snapshots :) [20:02:05] otherwise, hmm importing a dump isn't too hard. I wonder if we have instructions anywhere though [20:02:15] That's actually a pretty good idea [20:02:50] I have some scripts to load stuff from dumps as well. I keep telling myself I'll write some automation...maybe today is the day [20:05:20] separately, thanks lenovo! They have 'Lenovo Diagnostics Linux Bootable Versions', and the instructions for making the bootable usb say download and run this .exe :P [20:05:41] not the generic instructions, but literally the instructions for the linux version [20:06:48] LOL, reminds me of the Mac DisplayLink drivers that are MacOS.dmg.exe [20:10:14] lol, nice [20:15:13] tried hand-installing. In theory UEFI needs nothing, just fat32 format the drive and put the files in place...lets see [20:30:34] yeah, I think that is how the Dell firmware updates work. Although they also offer bash scripts that work at the Linux level [20:44:42] OK, Codex helped me crank out a crappy playbook for snap/restore https://gitlab.wikimedia.org/repos/search-platform/sre/ansible-playbooks/cirrussearch/-/blob/main/cirrussearch_snapshots.yml?ref_type=heads [21:32:34] so far I'm not seeing any logs hit logstash, checking relforge1010 for hints [21:38:11] `Log4j2-TF-2-ConfigurationFileWatcher` `java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "setContextClassLoader")` [21:38:21] sounds like java security stuff [21:56:51] still not working. Do you remember if relforge ever worked with the logging pipeline? I ask b/c it is firewalled off from other servers like the deploy servers [21:57:13] I one-offed relforge1010 to add the java security stuff, the error messages are gone but no logs