[13:09:42] \o [13:15:12] i'm not really following, what about https://phabricator.wikimedia.org/T431145 might need a new cluster? See most recent reply [13:17:47] it seems highly plausible somehow we are not all thinking the same thing if they are thinking a dedicated cluster might be required [13:25:01] o/ [13:26:10] if "linked artifacts indexing" refers to weighted tags then it's about our main search cluster? [13:26:26] * ebernhardson sighs...don't like security system. Asking opensearch-semantic-search for the mapping gets a permission deniod [13:26:40] :/ [13:26:52] dcausse: thats what i would have thought, to me the whole SLO request seemed like a non-thing. Like sure we can do it but it's basicalgly a nothing burger [13:27:03] so something is off if i think its anothing burger, and they think they might need a new dedicated cluste [13:27:25] ebernhardson: I copied the mapping yesterday at https://gitlab.wikimedia.org/dcausse/opensearch_semantic_highlighter_poc/-/blob/main/opensearch/mappings_settings.json?ref_type=heads [13:27:37] thanks! [13:32:35] I think this "linked artifacts indexing" concept is way too generic and lacks precision on access patterns for us to answer anything [14:09:10] yea, perhaps it's all just too fuzzy. I don't really know how to respond to that comment so i'm just going to leave it hanging for now... [14:56:56] hmm, random idea: Instead of (in addition to?) building out an expected highlights dataset from the research data we already have, what if we have claude "keyword-ify" the existing datasets? Basically turn the well formed queries into something that more closely matches what we see [14:57:08] re: highlighting [15:01:24] the "existing datasets" you mean squad2? [15:02:11] dcausse: SQuADv2, TyDiQA, and MultiSpanQA [15:02:37] Also i was tempted to add: https://aclanthology.org/Q19-1026/ [15:03:39] I have no clue what this would tell us to ask to keyword-ify, is your worry that these datasets are too far from our actual search logs? [15:03:58] dcausse: yes the concern is that the models are trained on "How old is Tom Cruise?" and the user queries are "tom cruise age" [15:04:36] the keyword version would be "tom cruise age"? [15:04:50] yea [15:05:21] my understanding is they got rid of the idea of routing queries, so we get whatever they send [15:06:15] somehow i am failing to find the dataset martin shared...somehow docs.google.com has terrible search [15:06:45] ebernhardson: I can find something self-contained and easy to use, sec [15:06:53] oh, it shows them at docs.google.com/spreadsheets [15:06:58] but not at the root [15:07:21] oh if you have something that would be nice [15:08:42] ebernhardson: https://people.wikimedia.org/~dcausse/pure_knn_10.json (should have the query and the returned chunks, hopefully no need to index anything) [15:10:57] dcausse: nice! thakns [15:13:39] re keywordification I think we could try and have a look, not super clear if we could expect the dataset to still be valid for the expected labels in the dataset. But have a quick look at the keywordified could tell us if it's reasonable [15:17:59] hmm, perhaps i'm too optimistic :) I guess i'd also have to review what comes out of the keyword-ification. I suppose i was mostly wondering what things are going to do if users don't provide the question words since it seems the models were trained on well-formed questions [15:22:59] yes I think it's worth trying, esp. if the info loss on keywordification is very low [15:23:10] pushed more at https://people.wikimedia.org/~dcausse/knn_dataset/ [15:24:09] unsure if particularly helpful tho... miracl is pretty much a questions dataset. other pure_knn_top_10 are just the same with different embedding models [15:31:21] heading out, have a nice week-end [20:27:05] * ebernhardson realizes he also needs to prep at least a minor outline and some graphviz for the regex algo presentation on tuesday