[06:05:34] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26): Upgrade production vLLM image to use vLLM version >= 0.19 - https://phabricator.wikimedia.org/T426766#11939089 (10isarantopoulos) [06:28:44] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26): prediction_classification events rejected by EventGate due to extra schema properties - https://phabricator.wikimedia.org/T426807 (10isarantopoulos) 03NEW [06:33:06] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26): prediction_classification events rejected by EventGate due to extra schema properties - https://phabricator.wikimedia.org/T426807#11939139 (10isarantopoulos) [07:09:15] 06Machine-Learning-Team (Q4 FY2025-26), 05Goal, 07OKR-Work: Q1 FY2025-26 Goal: Make article topic data available at scale and within SLOs for Year in Review - https://phabricator.wikimedia.org/T392833#11939270 (10isarantopoulos) **Status Update** We are targeting the following timeline for integrating Horde... [08:37:57] 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Experiment with new kserve version on ml-staging-codfw - https://phabricator.wikimedia.org/T419722#11939475 (10DPogorzelski-WMF) 05Open→03Resolved [08:38:51] 06Machine-Learning-Team, 13Patch-For-Review: Experiment with new kserve version on ml-staging-codfw - https://phabricator.wikimedia.org/T419722#11939478 (10DPogorzelski-WMF) [08:40:56] 06Machine-Learning-Team: Update prod kserve/knative - https://phabricator.wikimedia.org/T426823 (10DPogorzelski-WMF) 03NEW [08:42:29] 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Experiment with new kserve version on ml-staging-codfw - https://phabricator.wikimedia.org/T419722#11939496 (10isarantopoulos) [10:22:27] (03PS3) 10AikoChou: events: construct new prediction classification event independently [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1191305 (https://phabricator.wikimedia.org/T426807) [10:29:43] 06Machine-Learning-Team (Q4 FY2025-26), 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11939878 (10Samwalton9-WMF) Thanks folks!! This will make our analysis... [10:58:30] 06Machine-Learning-Team (Q4 FY2025-26), 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11939962 (10gkyziridis) 05Open→03Resolved [10:59:49] 06Machine-Learning-Team, 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11939966 (10gkyziridis) [11:07:48] 06Machine-Learning-Team (Q4 FY2025-26), 06Data-Engineering, 06Data-Engineering-Radar, 10Event-Platform: Add Multilingual RevertRisk predictions to mediawiki.page_revert_risk_prediction_change - https://phabricator.wikimedia.org/T415892#11940008 (10isarantopoulos) [11:09:46] 06Machine-Learning-Team (Q4 FY2025-26): Update prod kserve/knative - https://phabricator.wikimedia.org/T426823#11940015 (10isarantopoulos) [11:14:20] (03CR) 10AikoChou: events: construct new prediction classification event independently (033 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1191305 (https://phabricator.wikimedia.org/T426807) (owner: 10AikoChou) [11:17:01] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: prediction_classification events rejected by EventGate due to extra schema properties - https://phabricator.wikimedia.org/T426807#11940101 (10achou) a:03achou [11:51:41] 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Update prod kserve/knative - https://phabricator.wikimedia.org/T426823#11940248 (10DPogorzelski-WMF) [12:05:55] 10Lift-Wing, 06Machine-Learning-Team (Q4 FY2025-26), 13Patch-For-Review: Host Qwen 3.6-27B as an inference service - https://phabricator.wikimedia.org/T425680#11940309 (10gkyziridis) === Update === The deployment of [[ https://huggingface.co/Qwen/Qwen3-14B | Qwen3-14B ]] model on experimental eqiad was succe... [12:36:17] 06Machine-Learning-Team (Q4 FY2025-26), 06ServiceOps new, 06Traffic, 13Patch-For-Review: k8s changes needed to allow article topic (and other future isvcs) to use the kserve v2 inference protocol (and gRPC) - https://phabricator.wikimedia.org/T424049#11940457 (10JMeybohm) I probably don't have all the deta... [12:51:28] (03CR) 10Gkyziridis: [C:03+1] "LGTM!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1191305 (https://phabricator.wikimedia.org/T426807) (owner: 10AikoChou) [14:30:14] 06Machine-Learning-Team (Q4 FY2025-26), 07OKR-Work, 13Patch-For-Review: Add 'iommu=pt' kernel parameter on MI300x nodes for direct GPU-to-GPU communication (PCIe P2P) - https://phabricator.wikimedia.org/T421461#11941109 (10elukey) The provision cookbook is updated, ml-serve1012 and ml-serve1015 already done.... [15:34:43] (03CR) 10Sbisson: Update section suggestion fetching to request multiple at once (031 comment) [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1217170 (owner: 10Nik Gkountas) [15:54:45] 06Machine-Learning-Team (Q4 FY2025-26): Fix text normalization edge cases in TTS prototype - https://phabricator.wikimedia.org/T426756#11941567 (10kevinbazira) [19:01:45] (03CR) 10Ottomata: [C:03+1] "Another idea?" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1191305 (https://phabricator.wikimedia.org/T426807) (owner: 10AikoChou) [22:19:49] (03PS8) 10Nik Gkountas: Update section suggestion fetching to request multiple at once [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1217170 [22:19:52] (03CR) 10Nik Gkountas: Update section suggestion fetching to request multiple at once (031 comment) [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1217170 (owner: 10Nik Gkountas)