[04:17:24] 06Machine-Learning-Team (Q1 FY2026-27): Add Qwen3-14B-FP8 LLMInferenceService in LiftWing staging - https://phabricator.wikimedia.org/T439659 (10kevinbazira) 03NEW [04:34:36] 06Machine-Learning-Team (Q1 FY2026-27): Add Qwen3-14B-FP8 LLMInferenceService in LiftWing staging - https://phabricator.wikimedia.org/T439659#12377732 (10kevinbazira) [08:43:13] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE: Decide on the future shape of ML/LW SLOs, and how they are monitored - https://phabricator.wikimedia.org/T434525#12378248 (10klausman) a:03klausman [08:43:29] 06Machine-Learning-Team: pytorch image builds are failing - https://phabricator.wikimedia.org/T437427#12378249 (10klausman) a:03klausman [08:44:04] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work: Remove article-descriptions model from LiftWing - https://phabricator.wikimedia.org/T438033#12378250 (10klausman) [08:44:17] 06Machine-Learning-Team, 06Data-Platform-SRE (2026-09-18 - 2026-10-09): pytorch image builds are failing - https://phabricator.wikimedia.org/T437427#12378251 (10klausman) [08:45:21] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE, 06DC-Ops, 10ops-eqiad, 06SRE: eqiad row A&B host migration details request for Machine Learning - https://phabricator.wikimedia.org/T432649#12378252 (10klausman) [08:45:42] 06Machine-Learning-Team, 10Cassandra, 06Data-Platform-SRE, 05Goal, 07OKR-Work: Provision Cassandra + Data Gateway resources for Tone Check - https://phabricator.wikimedia.org/T408129#12378255 (10klausman) [08:46:11] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE, 07Essential-Work: Remove article-descriptions model from LiftWing - https://phabricator.wikimedia.org/T438033#12378256 (10klausman) [08:46:24] 06Machine-Learning-Team, 06Data-Platform-SRE: pytorch image builds are failing - https://phabricator.wikimedia.org/T437427#12378257 (10klausman) [08:47:44] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE, 10Prod-Kubernetes, 06ServiceOps, and 3 others: Update istio-cni to 1.29 - https://phabricator.wikimedia.org/T434198#12378258 (10klausman) [09:11:47] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE, 07Essential-Work: Remove article-descriptions model from LiftWing - https://phabricator.wikimedia.org/T438033#12378345 (10klausman) a:03klausman [09:43:44] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12378456 (10gkyziridis) a:03gkyziridis [09:52:07] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12378516 (10gkyziridis) Starting gradually to move the LLMs one by one to the suggested machines. 1. llm-qwen3-14b -> ml-serve1015 2. llm-qwen36-27b -> ml-serv... [10:11:49] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work, 13Patch-For-Review: ml-serve-eqiad: Repartition two MI300X nodes from 24GB to 96GB partitions - https://phabricator.wikimedia.org/T436928#12378626 (10achou) @DPogorzelski-WMF Could you also update t... [10:40:50] (03CR) 10Bartosz Wójtowicz: "Thank you for the huge amount of work here, those tools will be very useful! I left number of comments that I caught during the 1st scan o" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345929 (https://phabricator.wikimedia.org/T439386) (owner: 10Kevin Bazira) [12:53:47] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12379374 (10gkyziridis) ==== Qwen3-14b Deployed on ml-serve1015==== `lang=Shell ` [12:54:36] 06Machine-Learning-Team, 06Collaboration-Services, 10GitLab, 06Product Safety and Integrity, and 4 others: Set an identifiable User-Agent on WMF tools calling gitlab.wikimedia.org - https://phabricator.wikimedia.org/T437216#12379385 (10Isaac) Just making an acknowledgement from the Research side -- many of... [12:56:31] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12379395 (10gkyziridis) [14:07:31] 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Update httpbb tests and add a single script to run them - https://phabricator.wikimedia.org/T439385#12379932 (10OKarakaya-WMF) 05Open→03Resolved [14:08:32] 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Update httpbb tests and add a single script to run them - https://phabricator.wikimedia.org/T439385#12379933 (10OKarakaya-WMF) closing this ticket as we have implemented both of the requirements in the ticket below; https://ge... [14:24:05] 06Machine-Learning-Team (Q1 FY2026-27): Add Qwen3-14B-FP8 LLMInferenceService in LiftWing staging - https://phabricator.wikimedia.org/T439659#12380046 (10kevinbazira) Following the steps in T433973#12264511, we deployed the qwen3-14b-fp8 llmisvc in LiftWing staging. It's now up and running: `bash kevinbazira@dep... [14:32:33] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-09-18 - 2026-10-09): Update kserve helm chart to include LLMInferenceService - https://phabricator.wikimedia.org/T433973#12380094 (10kevinbazira) @DPogorzelski-WMF thank you for working on this. We've followed the steps in T433973#12264511 and d... [14:53:39] 06Machine-Learning-Team (Q1 FY2026-27): Evaluate LibreChat and Open WebUI for LiftWing Studio Productionization - https://phabricator.wikimedia.org/T438987#12380197 (10achou) We decided to move forward with **LibreChat** for LiftWing Studio. The main deciding factors were: - **MIT license**, with additional de... [15:32:55] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor Suggestion Mode, 07OKR-Work: Update LLM-generated MoS suggestion copy - https://phabricator.wikimedia.org/T432665#12380488 (10VPuffetMichel) [15:33:01] 06Machine-Learning-Team, 06Editing-team, 10VisualEditor Suggestion Mode, 07OKR-Work: [SPIKE] Define new experiment which gives suggestions to everyone and restricts only one suggestion - https://phabricator.wikimedia.org/T432631#12380489 (10VPuffetMichel) [19:38:39] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12381896 (10HNordeenWMF) [20:55:07] 06Machine-Learning-Team (Q1 FY2026-27), 06Reader Growth Team: [ML TTS] Expose structured sentence, paragraph, and revision metadata for Read-Along synchronization - https://phabricator.wikimedia.org/T438228#12382527 (10SherryYang-WMF) p:05Triage→03Medium