[07:17:11] 06Machine-Learning-Team (Q2 FY2026-27): Migrate LLM services to the new vLLM image - https://phabricator.wikimedia.org/T444464#12553597 (10isarantopoulos) pasting here for visibility from the commit message of https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1369075 that switches the image for the... [09:02:07] 06Machine-Learning-Team (Q2 FY2026-27), 06Research: Wiki-Eval V0: Research support for Milestone 1 (Dataset, prompts) - https://phabricator.wikimedia.org/T444754 (10MGerlach) 03NEW [09:23:27] 06Machine-Learning-Team (Q2 FY2026-27), 05Goal, 07OKR-Work, 13Patch-For-Review: Add support for paragraph-level captions in TTS v2 - https://phabricator.wikimedia.org/T444576#12553837 (10kevinbazira) The TTS service now supports paragraph-level captions when `paragraphs_json` or `paragraphs_vtt` are set. A... [11:22:01] (03CR) 10Gkyziridis: [C:03+2] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:34:51] (03CR) 10CI reject: [V:04-1] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:38:42] (03CR) 10Gkyziridis: python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:39:21] (03CR) 10Gkyziridis: [C:03+2] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:52:39] (03CR) 10CI reject: [V:04-1] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:54:59] (03CR) 10Gkyziridis: python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [11:55:05] (03CR) 10Gkyziridis: [C:03+2] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [12:11:56] (03CR) 10CI reject: [V:04-1] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [12:16:33] (03CR) 10Gkyziridis: "Gate keeps failing for CI reasons, not because of this change. The jobs built on the ROCm PyTorch images (revertrisk-wikidata, revise-tone" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [13:13:41] (03CR) 10Gkyziridis: [V:03+2 C:03+2] python: upgrade aiohttp to 3.14.5 [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1368637 (owner: 10Gkyziridis) [14:40:33] 06Machine-Learning-Team (Q2 FY2026-27): Publish Pipeline Issue: publishing large images fails - https://phabricator.wikimedia.org/T444811 (10gkyziridis) 03NEW [14:48:29] 06Machine-Learning-Team (Q2 FY2026-27), 10Continuous-Integration-Infrastructure: Publish Pipeline Issue: publishing large images fails - https://phabricator.wikimedia.org/T444811#12555561 (10gkyziridis) [15:00:06] (03PS3) 10Gkyziridis: revertrisk_wikidata: Integrate LAC (on-demand) flow [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1359881 (https://phabricator.wikimedia.org/T433699) [15:03:03] (03CR) 10Gkyziridis: "Thanks for you comment @achou@wikimedia.org!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1359881 (https://phabricator.wikimedia.org/T433699) (owner: 10Gkyziridis) [15:03:09] (03CR) 10CI reject: [V:04-1] revertrisk_wikidata: Integrate LAC (on-demand) flow [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1359881 (https://phabricator.wikimedia.org/T433699) (owner: 10Gkyziridis) [15:04:36] (03CR) 10Gkyziridis: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1359881 (https://phabricator.wikimedia.org/T433699) (owner: 10Gkyziridis) [15:08:26] (03PS1) 10Gkyziridis: inference-services: Remove llm, qwen36 and gpt-oss-safeguard servers [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1369459 (https://phabricator.wikimedia.org/T444471) [15:09:41] 06Machine-Learning-Team (Q2 FY2026-27), 07Essential-Work, 13Patch-For-Review: Clean up unused inference-services code after LLMISVC migration - https://phabricator.wikimedia.org/T444471#12555669 (10gkyziridis) Follow-up patch: [[ https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/... [15:16:12] 06Machine-Learning-Team (Q2 FY2026-27), 05Goal, 07OKR-Work: Q2 FY2026-27 Text-to-Speech V2 Experiment - https://phabricator.wikimedia.org/T441938#12555685 (10kevinbazira) **Weekly Update:** **Updated TTS DAG to delay deleting old revisions by a day (T444092):** - The Readers Growth team pointed out that a r... [17:57:49] 06Machine-Learning-Team (Q2 FY2026-27), 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Publish Pipeline Issue: publishing large images fails - https://phabricator.wikimedia.org/T444811#12556378 (10dduvall) @gkyziridis this seems to be an edge case with very large SBOMs. I will investiga... [19:50:54] 06Machine-Learning-Team (Q2 FY2026-27), 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Publish Pipeline Issue: publishing large images fails - https://phabricator.wikimedia.org/T444811#12556625 (10dancy) After disabling SBOM by default, I reran the 5 failed jobs and they're happy now: h...