[09:35:19] a prefix query with constant_score on very wide prefix like p31=q* is ~1s and returs 118M items, seems acceptable to me... [09:35:29] seems to honor the timeout param pretty well too [09:37:05] going to stick to prefix for now and not allow wildcards, the wildcard query has no ways to set a limit on the determinized states unlike query_string [09:37:20] might send a PR to add this [09:41:04] errand+lunch [12:25:58] \o [12:38:58] o/ [13:18:22] o/ [13:42:44] just found out our cookbooks don't work with OpenSearch 2. That's gonna be a ticket ;( [13:42:51] :( [13:43:48] ouch :/ [13:43:54] `spicerack.apiclient.APIClientResponseError: PUT https://cloudelastic.wikimedia.org:9243/_cluster/settings returned HTTP 400` [13:44:36] Guess it's a slight change in the shape of the settings. I wanna say we saw something similar with translate wiki? [13:44:36] weird, could be very old settings we set? [13:45:26] I'm not sure? last error from translate was mainly due to php introducing type checks not matching what was in mediawiki-config [13:45:31] hmm, i thought maybe it would be transient settings, but checking docs even os 3.x still allows those [13:46:11] opensearch usually gives pretty acceptable error messages in the content, maybe the cookbook could be updated to print the body of the error as well [13:46:24] +1 [13:47:03] Yeah, I'm looking at https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/spicerack/+/refs/heads/master/spicerack/elasticsearch_cluster.py now [13:50:03] I think we should probably move some of the logic back into the cookbook. Working with spicerack is "fun" [13:50:35] FYI APIClientResponseError has the response object available [13:51:08] I don't any calls to _cluster/settings? do we use a lib/client of some sorts? [13:51:12] *see [13:51:40] created T426862 to take a closer look [13:51:41] T426862: Ensure cookbooks work with OpenSearch 2.x - https://phabricator.wikimedia.org/T426862 [13:53:19] might be a good opportunity for an LLM-assisted refactor [14:00:34] perhaps related: https://github.com/opensearch-project/OpenSearch/issues/18515 [14:02:28] yes there are weird settings there like archived.plugins.index_state_management.template_migration.control [14:20:27] Closing the loop, it was exactly this bug ^^ [14:20:57] Following the steps in the issue got us unblocked, thanks David [15:15:06] sorry, missing the Weds mtg due to rebooting everything ;{ [15:36:13] Also, I created https://docs.google.com/document/d/1XJ6_9ZHJ7yluAOA8DMhxmdDhLJZn8HaA5ADokZLpbr8/edit?tab=t.0#heading=h.2nz6eg90x2jo as a high-level OpenSearch migration plan, feel free to look it over and add/change anything [17:23:06] dinner