[10:16:28] <dcausse>	 lunch
[14:01:50] <ebernhardson>	 \o
[14:26:45] <dcausse>	 o/
[15:05:56] <inflatador>	 dcausse would you like me to take T406656 and do reloads on the categories graph, or would you prefer to keep troubleshooting?
[15:05:56] <stashbot>	 T406656: Reimage failed after prompt...is prompt needed? - https://phabricator.wikimedia.org/T406656
[15:07:05] <ebernhardson>	 surprising to me...take the last 100k events from update_pipeline.update.v1, `grep -v TAGS_UPDATE`, how many should be left?  I have 2
[15:07:06] <dcausse>	 inflatador: o/ I'm done troubleshooting, if you could fix the puppet discrepancies and do a reload this would be great :)
[15:08:10] <dcausse>	 ebernhardson: this is totally unexpected? did you scan the codfw topic?
[15:08:41] <dcausse>	 eqiad topic should be almost empty
[15:08:48] <ebernhardson>	 dcausse: oh!! eqiad has 2, codfw has 99,390.  
[15:08:51] <ebernhardson>	 hmm
[15:09:22] <ebernhardson>	 oldest event in eqiad is 2025-10-17T00:14
[15:09:36] <ebernhardson>	 sorry, newest.  
[15:09:39] <ebernhardson>	 well, both
[15:10:07] <ebernhardson>	 hmm, i don't know this is related to the existing weighted_tags problem...but maybe another problem :P  I guess some batch update shipped through eqiad?
[15:10:09] <dcausse>	 mw primary db is in codfw so all writes should happen from mw@codfw
[15:10:39] <dcausse>	 yes, I think we forcibly push weighted_tags for image_rec to kafka-main@eqiad
[15:11:03] <dcausse>	 should happen weekly on fridays
[15:11:03] <ebernhardson>	 ahh, yea that would make sense
[15:11:14] <ebernhardson>	 and 17th was friday
[15:11:19] <dcausse>	 should be it
[15:13:13] <dcausse>	 hm.. the truncate filter we use to fix extremely long keyword fields can't be used as a normalize apparently... option would be to use ignore_above but that might throw away some data rather than keeping some of it...
[15:13:39] <ebernhardson>	 hmm, it ignores the whole thing?
[15:13:56] <ebernhardson>	 yea..." If a string’s length exceeds the specified threshold, the value is stored with the document but is not indexed."
[15:14:25] <dcausse>	 well I could check what the impact would be
[15:14:50] <ebernhardson>	 i guess a pattern replace char filter?  But stuffing regex everywhere seems meh
[15:15:26] <dcausse>	 actually I need to understand what they use to determine if a filter qualifies for as normalizer or not, I'm not super clear on this
[15:16:19] <dcausse>	 initially I thought they forced you to use char_filters but looks like you can use token_filter as you said, but not all of them apparently :/
[15:19:00] <ebernhardson>	 random claim: a normalizer accepts only filters that are instances of either NormalizingTokenFilterFactory or NormalizingCharFilterFactory
[15:21:35] <dcausse>	 sigh...
[15:22:43] <ebernhardson>	 as far as i can tell...not having an equivalent truncate is perhaps an oversight...if we don't like pattern_replace i suppose a custom normalizing char filter could work, but i dunno if necessary
[15:23:35] <dcausse>	 ah cool IcuFoldingTokenFilterFactory is NormalizingTokenFilterFactory was mainly afraid of this one
[15:24:34] <dcausse>	 I think I can live with a pattern_replace, but could add a quick custom truncate or upstream a change
[15:28:36] <dcausse>	 actually IIRC we have a fix from Trey about the regex highlighter to deploy, might be a good time to add that custom truncate?
[15:31:00] <ebernhardson>	 yea there is already a .deb update waiting in the wings, we can add to it
[15:31:15] <ebernhardson>	 i don't think it's made it to apt.wikimedia.org yet, just sitting in the gitlab CI
[15:32:24] <dcausse>	 ok, pushing a patch hopefully won't take long
[15:35:14] <dcausse>	 ah and the "length" as well we use to remove empty tokens will have to be supported
[15:35:59] <inflatador>	 dcausse ACK, have taken the ticket
[15:36:07] <dcausse>	 thanks!
[17:43:58] <dcausse>	 dinner
[20:08:39] <inflatador>	 I'm trying to load some test data into the new opensearch-on-k8s instance, specially enwikibooks. I was wondering, can I just create the index with the same mapping as the live enwikibooks, or is there stuff I need to add/remove first?
[20:10:25] <ebernhardson>	 sigh, finally...spent the last two hours trying to figure out why jsondiff.py was blowing up...and the answer is: It doesn't know namespaces and dislikes getting the same title from multiple namespaces
[20:10:47] <ebernhardson>	 on the upside, refactored parts so it's easier to test :P
[20:17:52] <inflatador>	 that's annoying. Although I will admit, I didn't know json did namespacing
[20:19:09] <ebernhardson>	 it's not json namespaceing, jsondiff is comparing search results from two json dumps.  the results themselves can have either the title or the page id as the primary key, if it's the title then two different namespaces look the same
[20:19:19] <ebernhardson>	 (we can fix by using the prefixed title, it just doesn't happen to do that)
[20:37:59] <inflatador>	 I got the mapping, asking ChatGPT to remove all the non-standard analyzers isn't too bad ;)
[23:10:43] <inflatador>	 First data in opensearch on k8s! https://phabricator.wikimedia.org/T404907#11296571