[07:30:20] Trey314159: there should be no problems with lower weights as long as we're still above float precision, main question I guess is IIRC dwim is supposed to complement the main algo but turns out that the weigth/scoring approach is not working as it should, but if on the other hand we want DWIM to be more part of the ranking (i.e. a dwim match may outperform a fuzzy match) it's different [07:30:45] this was possibly less visible with other dwim algo [10:06:18] sigh... I probably broke pool counter assignment, unsure why yet, but most queries go to the default pool :( [10:18:20] due to extension merge strategy I think... null is not kept with provide_default but returns [] and that causes the new code to believe there was a set of rules (empty) [10:51:30] errand+lunch [14:44:59] \o [14:47:43] o/ [16:37:15] dcausse: I'm okay with keeping the DWIM suggestions secondary. There are definitely times when the DWIM results will be better than the main suggestions, and there will be times when they will be better but also not be shown because there are 10 main suggestions. That's not ideal, but it's better than mixing in false positive results that really confuse the searcher. [16:37:23] Two random thoughts for possible future projects: 1) some UI signal that allows searchers to know where suggestions come from. There are so many sources, though. I love the newer one that says when a non-obvious redirect matched. [16:37:26] 2) Incorporating token-aware edit distance or something similar into suggestions. I can see what the query "when did t" matches the title "when I think about you", but I would not be surprised if it confused a searcher. It wouldn't solve that particular example, but not allowing (or further discounting) two typos in a row seems like it might weed out (or down rank) lower quality matches. [16:37:51] I'll create a patch for the weight change later today [16:48:26] thanks! [16:59:23] * ebernhardson waffles on what repository things live in...cindy needs minor variations of the prod config we apply in cirrus-toolbox, but where does the variation live? [16:59:48] or i should say, where does applying that variation live. hmm [17:01:14] things like knn mappings, vector size or the like? [17:01:52] dcausse: just the connectors and such, like we have the parameters.endpoint in each of the config files. Right now i have something that reads in the config and writes it back out with parameters.endpoint replaced with the stub location [17:02:20] but that feels awkward [17:02:23] cindy would call cirrus-toolbox? [17:02:46] yes, the cirrus test matrix would name a cirrus-toolbox release version, and cindy would apply it to the cluster [17:02:59] only the clusters with the neural plugin (so only 3.x) [17:03:38] it seems like such a silly simple question, like it's just replacing the liftwing endpoint with a local one. perhaps i'm overcomplicating [17:03:45] ok, I guess naively you'd want cirrus-toolbox to be "cindy" compliant but I guess that requires adding some more config/cli flags to cirrus-toolbox? [17:04:55] yea, hmm [17:05:06] but I suppose cindy has a copy of the knn field mapping... so it's kind of already duplicating some of the prod config... [17:05:15] hard to tell [17:05:19] maybe if i add an overrides.yaml? It would have something like "ml_connector/liftwing-jina-embeddings.json": {"endpoint": "https://..."} [17:05:40] i dunno, that then ties it to the exact names though...i have to ponder [17:05:59] basically my main goal is that if we break highlighting, a simple end-to-end test should have shown that [17:06:03] sure [22:13:23] meh, even after clearing out docker images and letting it rebuild, the cindy instance doesn't have enough disk with multiple opensearch versions [22:13:33] figuring out if we can use the extra volumes documented on wikitech [22:37:39] mysterious...docker starts fine, but things are getting stuck. like mysql didn't start in 240s [22:50:53] * ebernhardson broke dns apparently [23:24:43] but it looks to be passing, have semantic search (with stub embedding/highlighting apis) working in cindy