[08:15:49] <gmodena>	 o/
[08:31:50] <dcausse>	 o/
[11:51:16] <gmodena>	 cirrus-streaming-updater ci is having a bad day
[11:51:18] <gmodena>	 https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/jobs/436965
[11:51:31] <gmodena>	 3.231 mkdir: cannot create directory ‘asprof’: No space left on device
[11:51:47] <dcausse>	 :/
[11:53:07] <dcausse>	 not we can do much? I can drop this asprofiler we don't use it but unsure that's going to help much
[11:53:14] <dcausse>	 s/not/not sure/
[11:53:39] <gmodena>	 i think we might have to wait a bit and see if we get better resource allocation
[11:53:52] <dcausse>	 sure
[11:54:03] <dcausse>	 I'll hit the retry button this afternoon
[11:54:16] <gmodena>	 however,  I tried to restart the failing task.. and it failed again (out of disk) even on another runnier
[11:55:33] <dcausse>	 :(
[11:57:06] <dcausse>	 need to pick-up my son and have lunch, hopefully this gets better afternoon
[11:58:38] <gmodena>	 dcausse ack. See you later.
[12:55:51] <gmodena>	 lunch
[13:50:58] <gmodena>	 dcausse cirrus-streaming-updater's ci fixed itself 
[14:01:24] <dcausse>	 \o/
[14:13:50] <inflatador>	 <o/
[14:14:36] <gmodena>	 o/
[14:15:00] <inflatador>	 ^^ if y'all are running out of disk space on runners in the future, it might be a good idea to ask for helm in releng IRC, they are pretty responsive
[14:26:59] <dcausse>	 o/
[14:27:59] <inflatador>	 dcausse do you know if all the airflow concerns here have been address? Just trying to clean up tickets https://phabricator.wikimedia.org/T380621#10448953
[14:28:21] <inflatador>	 see also https://docs.google.com/document/d/18YXTuVMAiaVwmxOktcIENrPCV_hHA_k4HEWdV2EWxvM/edit?tab=t.0#heading=h.ubfayg1k41mk
[14:32:24] <dcausse>	 inflatador: yes all issues are resolved, I still need to look into the data cleanups with Gabriele to re-enable it fully
[14:34:06] <inflatador>	 dcausse ACK, thanks for taking a look!
[15:25:41] <inflatador>	 dcausse when we build the new opensearch plugins pkgs, will we need to each individual plugin's repo as well, such as https://gitlab.wikimedia.org/repos/search-platform/opensearch-analysis-hebrew ?
[15:26:07] <dcausse>	 inflatador: yes
[15:27:56] <inflatador>	 dcausse ACK, just making sure. I'll start making patches based on Erik's work in https://phabricator.wikimedia.org/T372769 . No hurry on this, just trying to scope out the work
[15:28:09] <dcausse>	 sure
[15:42:28] <dcausse>	 need to pick up the other kid, might be late for retro
[16:04:01] <gehel>	 retro in https://meet.google.com/eki-rafx-cxi cc ryankemper, gmodena, Trey314159 
[16:07:19] <gehel>	 looks like only Brian and myself are available today (with David maybe later), so we're canceling the retro. We'll talk in BCN!
[16:07:57] <gmodena>	 gehel welp. Sorry! This one was not in my calendar 
[16:08:32] <gmodena>	 just finished a call, I'm aronud now if you'd like. Otherwise BCN is fine too :)
[16:08:41] <gehel>	 Oh, it looks like I forgot to invite you.
[16:08:57] <gehel>	 Nah, let's cancel for today. We'll have other opportunities.
[16:09:11] <dcausse>	 just came back, +1 to cancel
[16:09:39] <dcausse>	 gehel: we have a meeting with scholia today, will you be able to make it?
[16:09:42] <gmodena>	 gehel np. Got the invite now
[16:10:30] <gehel>	 dcausse: yes, I'll be there. Thanks for the reminder!
[16:10:37] <dcausse>	 cool, thanks!
[16:26:09] <gmodena>	 https://phabricator.wikimedia.org/T370153
[16:26:40] <gmodena>	 ^ might be relevant to us (Move kafka-mirror Prometheus-based alerts from Icinga to alerts.git)
[16:47:51] <inflatador>	 workout, back in 40
[17:27:08] <inflatador>	 back
[17:30:59] <inflatador>	 for updating maven-hosted plugins, is the process to bump the version number in the XML files ( https://gerrit.wikimedia.org/r/plugins/gitiles/search/highlighter/+/40da2efbcad679f3b7e0bfea2461283a709240a2 ) and then run the corresponding Jenkins job, such as "search-highlighter-maven-java11"?
[17:54:56] <dcausse>	 inflatador: no, if we haven't moved this one to push to gitlab you have to be a member of sonatype and be a member of the org.wikimedia.search project and build it on your laptop
[17:56:12] <inflatador>	 dcausse ACK, I eventually RTFM and found it here ;( https://wikitech.wikimedia.org/wiki/Search/Elasticsearch_Administration#Deploying_plugins
[19:01:21] <inflatador>	 lunch, back in ~40
[20:09:05] <inflatador>	 back
[22:42:49] <inflatador>	 looks like we aren't scraping metrics from wdqs-categories JMX exporter (port 9103)...at least, I can't find them in Grafana explorer. The exporter itself seems OK
[23:52:25] <inflatador>	 making some progress on the categories reload job, check it out if interested https://gitlab.wikimedia.org/repos/search-platform/sre/wdqs-categories-helm/-/blob/main/README.md?ref_type=heads
[23:53:47] <inflatador>	 It'd probably work well as an airflow job if we can get Kubernetes PodOperator to work