[09:46:58] <klausman>	 \o Morning. I am not getting any results on any Logstash dashboards.
[09:47:15] <klausman>	 Or rather: i get stats like messages/s or UA breakdowns, but no actual messages
[09:47:35] <klausman>	 e.g. https://logstash.wikimedia.org/app/dashboards#/view/138271f0-40ce-11ed-bb3e-0bc9ce387d88?_g=h@1be52a8&_a=h@0eb7d9a
[09:52:43] <godog>	 klausman: I'm checking shortly too, cc elukey 
[09:53:05] <godog>	 klausman: the link doesn't work for me, would you mind using 'share' ?
[09:53:11] <klausman>	 we were already talking on #-sre, Luca thinks it's broken visualization
[09:53:29] <klausman>	 https://logstash.wikimedia.org/goto/fabe47bbf64a4f898f7aadcadbf06fe1
[09:53:30] <godog>	 ok we talk there
[09:57:10] <elukey>	 see https://logstash.wikimedia.org/goto/0f5e9fdcafe2e3c5e42d88da4623865c for example
[09:57:43] <elukey>	 the saved search on the right panel says "ChartRendererErrors"
[09:58:05] <elukey>	 one that works, like https://logstash.wikimedia.org/goto/471cfc0cd0af07ff86838eb62a193e5f, says "logstash-*"
[09:58:27] <elukey>	 this is why I suspect somebody saved/ovverrode the wrong thing
[09:58:46] <elukey>	 but the nice opensearch ui is not really helpful in telling me a way about how to fix the saved search :D
[09:58:49] <elukey>	 any idea?
[09:59:48] <godog>	 yes the panel depends on the ChartRendererErrors saved search, which has stopped returning results
[09:59:58] <godog>	 I think the problem is the filter requested_server_name: chart*
[09:59:58] <godog>	 
[10:00:11] <godog>	 that field requested_server_name I don't see it anymore
[10:00:43] <godog>	 actually the path too
[10:00:53] <elukey>	 is there a way to switch the saved search that the panel uses?
[10:01:03] <elukey>	 or do we need to redo the panel?
[10:01:21] <elukey>	 no idea why that saved search is there, really new to me
[10:02:34] <elukey>	 My suspicion is that there was an istio saved search and somebody overrode it with the chartrenderer one
[10:04:26] <elukey>	 ah yes if you save you can change the name
[10:04:38] <elukey>	 godog: ok if I just rename and fix the search?
[10:04:45] <godog>	 elukey: I think so yes
[10:06:20] <elukey>	 all works now :)
[10:06:25] <elukey>	 cc: klausman: --^
[10:06:27] <godog>	 \o/ thank you elukey 
[10:06:52] <elukey>	 thanks as well for the support <3
[10:06:54] <klausman>	 can confirm! thanks, everyone