[07:40:22] o/ [08:05:19] o/ [09:22:25] dcausse FYI: I created a mediawiki docker recipe for OpenSearch https://www.mediawiki.org/wiki/MediaWiki-Docker/Configuration_recipes/OpenSearch [09:22:48] i left references to elasticsearch services/volumes to allow an easy drop in in existing setups [09:23:08] gmodena: awesome thanks! [09:23:17] np! [09:24:22] do you maybe have some time after the unmeeting (or lunch) to continue our chat on how to handle country lists for the articlecountry tag? [09:24:35] At somepoint I think it might make sense to rename the service&volumles to opensearch but I agree that it might be easier for now [09:24:39] gmodena: sure! [09:25:17] thanks! [11:26:43] lunch [12:37:03] https://airflow-search.wikimedia.org/dags/mjolnir_weekly/grid?dag_run_id=scheduled__2025-02-06T00%3A00%3A00%2B00%3A00 [12:37:19] mjolnir's 2025-02-06 runs is looking good [13:19:10] dcausse bear with me... If I add a profile config file, do I need to do all the wiring via SearchProfileServiceFactory ? [13:42:12] gmodena: no no, just a "require" should be sufficient [13:42:31] $array = require( path_to_php_file ) [13:50:59] $array = require __DIR__ . '/../../profiles/MyFile.php'; [13:52:48] dcausse ack! [14:11:52] o/ [14:15:01] o/ [14:35:54] gmodena: added some bullet points in the DPE slides, haven't added anything about the research you've done on easy/hard queries but this should definitely be in there [14:40:13] dcausse thanks! I'll add the MLR work [15:10:23] ryankemper FYI, we are cancelling the retro today [15:12:19] gmodena: I've added code reviews and your name on https://docs.google.com/document/d/1AZcFLBj9b8c6gnS5QQWN0YOJ9boXSSIGgpZbdD1vETU/edit?tab=t.3cwwjokoh89i [15:15:36] inflatador, ryankemper: when you have a moment could you deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/1113741 ? [15:18:15] gmodena: we're changing the structure of the staff meeting slides. (https://docs.google.com/presentation/d/1LQZEIliTtNgMJioIeVZGSTX5fAZQKrLjum7Qea2GQ6I/edit#slide=id.g3284dceeb5a_1_328). Instead of going back and forth between teams (goals + updates), we're moving to just have team updates. I've moved the slides around, but it might make sense to merge the Search Platform slides. [15:23:39] dcausse :eyes [15:43:05] dcausse I deployed ^^ [15:44:27] inflatador: thanks! in some time (tomorrow?) once puppet has run everywhere could you also deploy the followup: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1113461/2 ? [15:53:34] dcausse ACK, I added it Ryan and I's SRE session for US afternoon. If that is too soon LMK [15:53:49] today that is...in 6h [15:54:35] gehel ack on both [15:55:43] gehel merging makes sense to me. I'll work on the slides tomorrow. Let's touch base before the DPE meeting? [15:57:16] dcausse pushed some WIP articlecountry work with the changes we discussed this morning. Using require directives was a bit fiddly, so for now the mappings are still hardcoded [15:59:20] gmodena: thanks! will take a look soon [16:00:42] gmodena: put time in my calendar when you need me! [16:01:20] gehel will do! [16:03:25] \o [16:04:22] o/ [16:42:52] errand [16:44:01] system update on my laptop, back in ~30 [16:44:48] France's birthday, I'll be off early and for the rest of the evening! [16:56:12] gehel nice! Enjoy the celebration! [17:05:38] inflatador ryankemper would you be around during the late backport window on Monday? I need to deploy a change (https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1120534). It should be low impact (just disabling an A/B test), and I can deploy it myself, but I’d feel more comfortable knowing there's SREs around. :) [18:10:42] back [18:11:45] gmodena I'll be around, but I haven't done a deploy in years. Not sure if I'd be much help ;) [18:18:14] inflatador thanks! The mw deployment I'm not worried about, post deployment search metrics could always use a second pair of eyes :D [18:19:11] afk. See you later / tomorrow morning [18:25:35] ACK, see you then! [18:44:51] dinner [18:47:59] yup I can help out as well [19:17:19] ryankemper I'm working on updating the cookbooks for Opensearch and I'm getting this error. Do you remember seeing it before? It doesn't seem directly related to OS https://phabricator.wikimedia.org/P73498 [19:35:21] Out w dog but have you traces where it reads that from [19:35:49] It’s possible it reads from a config file and not cluster state [20:10:07] The spicerack library code above does look like it's reading from an API, but the URL for relforge might be hard-coded somewhere. We've seen that issue with DAGs already [21:39:06] d-causse for when you get back tomorrow, I was gonna ask about elasticsearch-madvise...do you know if we still need it for opensearch? ref T386281 [21:39:06] T386281: Decide if we still need opensearch-madvise - https://phabricator.wikimedia.org/T386281 [21:43:16] inflatador: it is unlikely that the disk access patterns are going to change much with OpenSearch. By nature, we're accessing indices in a mostly random access. [21:44:34] ACK, I'm mainly just wondering if they fixed this issue in Opensearch and/or if there is now a kernel tunable we can use instead [21:44:56] So we would most probably still waste memory with large readahead. We might be able to test that under load, but it's more likely than not that we will need it (or something similar) [21:48:47] This **sounds** like the same issue, but is it? https://github.com/elastic/elasticsearch/issues/27748 [21:55:40] Seems similar enough. But seems to require Java 21 [21:57:41] Yeah, I'm reading thru all the linked threads. It seems like a better idea to just go ahead and compile it as opensearch-madvise [21:58:06] we can always fix this later once we're sure we have the right JVM, lucene version or whatnot [22:19:34] also for d-causse, re: mixed-cluster behavior, Erik did some tests in T379938 we may want to repeat [22:19:35] T379938: Evaluate mixed-cluster behavior of elasticsearch + opensearch - https://phabricator.wikimedia.org/T379938 [22:24:50] Do we have a newer repo for the elasticsearch-madvise? All I can find is https://gitlab.com/legoktm/es-thingy , which is linked from T264053 [22:24:50] T264053: Unsustainable increases in Elasticsearch cluster disk IO - https://phabricator.wikimedia.org/T264053