[04:57:32] (03PS4) 10Kevin Bazira: tool-server: Upgrade FastMCP and mark tools read-only [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345561 (https://phabricator.wikimedia.org/T439386) [05:02:24] (03CR) 10Kevin Bazira: tool-server: Upgrade FastMCP and mark tools read-only (034 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345561 (https://phabricator.wikimedia.org/T439386) (owner: 10Kevin Bazira) [06:33:46] (03CR) 10Bartosz Wójtowicz: [C:03+1] "Thank you, LGTM!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345561 (https://phabricator.wikimedia.org/T439386) (owner: 10Kevin Bazira) [06:43:37] (03CR) 10Kevin Bazira: [C:03+2] tool-server: Upgrade FastMCP and mark tools read-only [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345561 (https://phabricator.wikimedia.org/T439386) (owner: 10Kevin Bazira) [06:46:40] (03Merged) 10jenkins-bot: tool-server: Upgrade FastMCP and mark tools read-only [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345561 (https://phabricator.wikimedia.org/T439386) (owner: 10Kevin Bazira) [09:01:46] (03PS1) 10Kevin Bazira: tool-server: Add analytics tools for Wikipedia public data [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1345929 (https://phabricator.wikimedia.org/T439386) [09:13:47] 06Machine-Learning-Team (Q1 FY2026-27): Hummingbird: Evaluate prompting and tooling quality, and identify future improvements - https://phabricator.wikimedia.org/T439206#12373088 (10OKarakaya-WMF) I've added a new tool: - wikipedia article html: LLMs fetch the html content of an article whenever they need to rea... [10:59:23] (03PS2) 10Gkyziridis: revertrisk_wikidata: Develop KServe v2 and grpc inference protocol [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1338218 (https://phabricator.wikimedia.org/T433699) [11:00:37] (03CR) 10Gkyziridis: revertrisk_wikidata: Develop KServe v2 and grpc inference protocol (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1338218 (https://phabricator.wikimedia.org/T433699) (owner: 10Gkyziridis) [11:40:06] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Host Qwen3.8-27B-FP8 on LiftWing - https://phabricator.wikimedia.org/T436644#12373857 (10gkyziridis) 05Open→03Resolved [11:41:01] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 13Patch-For-Review: Benchmark Qwen3.8-27B-FP8 model - https://phabricator.wikimedia.org/T437034#12373871 (10gkyziridis) 05Open→03Resolved [11:41:18] 06Machine-Learning-Team (Q1 FY2026-27), 10Wikidata, 06Wikidata-Omega (Triaging): Threshold analysis for WMDE use cases of Wikidata RR model - https://phabricator.wikimedia.org/T429049#12373876 (10gkyziridis) 05Open→03Resolved [11:44:30] 06Machine-Learning-Team (Q1 FY2026-27): Design LLM hosting on LiftWing - https://phabricator.wikimedia.org/T433580#12373878 (10BWojtowicz-WMF) [11:44:31] 06Machine-Learning-Team (Q1 FY2026-27): Design LLM hosting on LiftWing - https://phabricator.wikimedia.org/T433580#12373881 (10BWojtowicz-WMF) Regarding the open questions raised above: > 1. How does the rest-gateway reach the new LLM ISVC Gateway API entrypoint (LVS/discovery, TLS, NetworkPolicies)? Which Gate... [12:46:03] 06Machine-Learning-Team, 06DC-Ops, 10ops-eqiad, 06SRE, and 2 others: Q2:rack/setup/install ml-serve1017 - https://phabricator.wikimedia.org/T438253#12374107 (10klausman) a:05klausman→03RobH [12:53:44] 06Machine-Learning-Team (Q1 FY2026-27), 06Data-Platform-SRE (2026-09-18 - 2026-10-09), 07Essential-Work: ml-serve: review documentation and apply AMD MI300X system-optimization settings to ml-serve1012-1015 - https://phabricator.wikimedia.org/T431553#12374151 (10ops-monitoring-bot) Icinga downtime and Alertm... [13:20:25] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555 (10isarantopoulos) 03NEW [13:20:51] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12374333 (10isarantopoulos) Note that gpt-oss-safeguard-model has been moved in the task {T439395} [13:21:54] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12374335 (10isarantopoulos) [13:22:26] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27): Move llm isvcs to new partitions - https://phabricator.wikimedia.org/T439555#12374338 (10isarantopoulos) [13:29:52] 06Machine-Learning-Team (Q1 FY2026-27): Evaluate LibreChat and Open WebUI for LiftWing Studio Productionization - https://phabricator.wikimedia.org/T438987#12374349 (10achou) Here are the main findings from comparing Open WebUI and LibreChat across a few areas: **user-facing evaluation, agent/workspace setups, a... [13:32:43] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work: Expose gpt-oss-safeguard-20b publicly - https://phabricator.wikimedia.org/T439395#12374373 (10isarantopoulos) a:03isarantopoulos [13:42:08] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Expose gpt-oss-safeguard-20b publicly - https://phabricator.wikimedia.org/T439395#12374430 (10isarantopoulos) ` curl -s https://api.wikimedia.org/service/lw/inference/v1/models/llm-gpt-oss-safeguard-20b:predict -H "C... [13:42:38] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Expose gpt-oss-safeguard-20b publicly - https://phabricator.wikimedia.org/T439395#12374431 (10isarantopoulos) [13:47:52] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12374456 (10HNordeenWMF) [15:48:36] 10Lift-Wing, 06Machine-Learning-Team (Q1 FY2026-27), 07Essential-Work, 13Patch-For-Review: Expose gpt-oss-safeguard-20b publicly - https://phabricator.wikimedia.org/T439395#12375302 (10isarantopoulos) 05Open→03Resolved [17:15:06] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 23 - Oct 06]): TTS MVP within "For you" Data Validation - https://phabricator.wikimedia.org/T438506#12375848 (10SNowick_WMF) [20:45:20] 06Machine-Learning-Team, 06DC-Ops, 10ops-eqiad, 06SRE, and 2 others: Q2:rack/setup/install ml-serve1017 - https://phabricator.wikimedia.org/T438253#12376938 (10RobH) a:05RobH→03None [21:58:46] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377216 (10HNordeenWMF) [21:59:16] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377217 (10HNordeenWMF) [22:07:05] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377237 (10HNordeenWMF) [22:10:19] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377245 (10HNordeenWMF) [22:15:16] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377254 (10HNordeenWMF) [22:17:38] 06Machine-Learning-Team (Q1 FY2026-27), 06App Experience (AppEx Sprint [FY2627 Sep 9 - Sep 22]), 07Epic, 13Patch-For-Review: [Sub-epic] A/B test text-to-speech MVP within "For you" - https://phabricator.wikimedia.org/T434720#12377267 (10HNordeenWMF)