[10:14:03] hnowlan jayme JFYI I'm merging https://gerrit.wikimedia.org/r/c/operations/alerts/+/1214478 and no impact expected [10:14:43] godog: uh, nice! [10:15:49] I know right? I needed that functionality anyways for T411470, might as well implement it [10:15:49] T411470: Page on cloudweb/horizon down - https://phabricator.wikimedia.org/T411470 [11:28:13] anyone know if anything is tracking wdqs melting down for the last few days? [11:28:48] godog: very nice [11:30:52] hnowlan: yes, DPE SRE and Wikidata Platform teams are keeping an eye on it. We probably want to delay the alerts a little bit to not make everyone scared. [11:36:42] gehel: ack, thanks - let us know if there's anything we can help with. Are there any candidates for silences? just to stem the flow a bit [11:38:31] Feel free to silence anything from WDQS for 24h. Ideally we'd like those to keep alerting, but only if issues last for > a few hours. [12:39:08] hnowlan: T411772 [12:39:09] T411772: Relax alerting for WDQS - https://phabricator.wikimedia.org/T411772 [12:41:16] thanks! [18:25:53] so, I might be stupid, but why can't I update my SSH key at idm.w.o ? I'm getting a 500 when I try to upload the new FIDO key, but an RSA key works [19:12:48] I'm currently setting up OIDC logging for an 3rd party app we run in dse-k8s-eqiad, and it seems that the JWT I'm getting from idp-test does not contain an `email` field (cf https://phabricator.wikimedia.org/P86411). Who's knowledgeable enough about IDP that I could bother them tomorrow? Thank you! [19:15:22] When looking in the idp-test logs, I can clearly see [19:15:22] WHAT: {access_token=AT-66-****************dlvKJsB-idp-test1005, scope=email openid profile, id_token=****************..., token_type=Bearer, expires_in=28800} [19:15:39] Raine: can you please open a task and tag it with Bitu and SRE-IF, then we can have a closer look tomorrow at what's failing [19:16:32] sk key types are already covered in the config, but we may have missed on [19:31:29] ok, thank you moritzm, will do