[07:04:54] good morning [07:11:02] morning! [07:20:13] good morning [07:49:33] 06Machine-Learning-Team, 05Goal: Q1 FY2025-26 Goal: Scaling Add-a-link to more wikis via production (airflow) pipelines - https://phabricator.wikimedia.org/T398950#11022737 (10OKarakaya-WMF) ml-pipelines and airflow dag MRs: - initial set up for add-a-link in ml-pipelines. - refactored anchor generation step (... [08:28:25] morning! [09:18:09] 06Machine-Learning-Team, 05Goal: Q1 FY2025-26 Goal: Scaling Add-a-link to more wikis via production (airflow) pipelines - https://phabricator.wikimedia.org/T398950#11022914 (10OKarakaya-WMF) [09:21:51] 06Machine-Learning-Team, 05Goal: Q1 FY2025-26 Goal: Scaling Add-a-link to more wikis via production (airflow) pipelines - https://phabricator.wikimedia.org/T398950#11022925 (10OKarakaya-WMF) [09:41:10] (03PS3) 10Kevin Bazira: RR: Validate lang parameter against canonical wikis [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) [09:45:19] (03PS4) 10Kevin Bazira: RR: Validate lang parameter against canonical wikis [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) [09:46:02] hello! [09:51:34] (03CR) 10Kevin Bazira: "Thank you for clarifying on this, now also the RRML blubber file copies the data directory." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) (owner: 10Kevin Bazira) [11:21:31] Hey folks, regarding updating the kserve to 0.15 in huggingface modelserver: https://phabricator.wikimedia.org/T367048. I see that we are currently install kserve from a file, do we want to move into the direction of installing it directly from the official release? Or do want to keep the same logic and installing it from a local file? [11:50:28] 06Machine-Learning-Team, 10EditCheck: Retrain peacock detection model for production use - https://phabricator.wikimedia.org/T388211#11023489 (10isarantopoulos) 05Open→03Resolved a:03isarantopoulos Resolving this task as the work described here is covered by {T396495} and {T398937} [11:52:13] 06Machine-Learning-Team, 10EditCheck: Retrain peacock detection model for production use - https://phabricator.wikimedia.org/T388211#11023516 (10isarantopoulos) a:05isarantopoulos→03None [12:09:32] 10Lift-Wing, 06Machine-Learning-Team: Request to host kid-friendly-classifier on Lift Wing - https://phabricator.wikimedia.org/T399872#11023620 (10isarantopoulos) Hi Daniel, thanks for filing this request! >What use case is the model going to support/resolve?** >Was made to detect content which we do not to ex... [12:13:12] 06Machine-Learning-Team, 07Essential-Work: Upgrade remaining model servers from debian bullseye to bookworm - https://phabricator.wikimedia.org/T400144 (10isarantopoulos) 03NEW [12:35:36] 06Machine-Learning-Team, 06Research, 10Research-engineering: Share code between Research & ML teams - https://phabricator.wikimedia.org/T398974#11023891 (10isarantopoulos) [13:06:53] 06Machine-Learning-Team, 06DC-Ops, 10ops-eqiad, 06SRE: Q4:rack/setup/install ml-serve101[2345] - https://phabricator.wikimedia.org/T393948#11024001 (10elukey) @jhathaway you rock thanks a lot! I verified with a reimage that d-i now correctly handles the new partman recipe. I didn't really think about using... [13:15:46] 10Lift-Wing, 06Machine-Learning-Team, 10EditCheck, 10Editing-team (Tracking): Create SLO dashboard for tone (peacock) check model - https://phabricator.wikimedia.org/T390706#11024022 (10elukey) @isarantopoulos we have improved the Pyrra's default config for latency but after a chat with Valentin we believe... [13:27:31] 06Machine-Learning-Team, 07Essential-Work: Upgrade the AMD GPU plugin for k8s to support MI300 GPUs - https://phabricator.wikimedia.org/T398600#11024124 (10isarantopoulos) [13:27:33] 06Machine-Learning-Team, 05Goal: Q1 FY2025-26 Goal: Operational Excellence - LiftWing Platform Updates & Improvements - https://phabricator.wikimedia.org/T398948#11024125 (10isarantopoulos) [13:27:42] 06Machine-Learning-Team: Add the ML team to the POSIX group `docker` on the ML lab machines. - https://phabricator.wikimedia.org/T393566#11024126 (10isarantopoulos) [13:27:46] 06Machine-Learning-Team, 05Goal: Q1 FY2025-26 Goal: Operational Excellence - LiftWing Platform Updates & Improvements - https://phabricator.wikimedia.org/T398948#11024127 (10isarantopoulos) [13:30:53] (03CR) 10Bartosz Wójtowicz: "Looks great!😊" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) (owner: 10Kevin Bazira) [13:56:47] 06Machine-Learning-Team, 07Essential-Work: Upgrade remaining model servers from debian bullseye to bookworm - https://phabricator.wikimedia.org/T400144#11024183 (10isarantopoulos) [14:08:59] 06Machine-Learning-Team, 06DC-Ops, 10ops-eqiad, 06SRE: Q4:rack/setup/install ml-serve101[2345] - https://phabricator.wikimedia.org/T393948#11024208 (10elukey) >>! In T393948#11024001, @elukey wrote: > @jhathaway you rock thanks a lot! I verified with a reimage that d-i now correctly handles the new partman... [14:10:03] 06Machine-Learning-Team, 06DC-Ops, 10ops-eqiad, 06SRE: Q4:rack/setup/install ml-serve101[2345] - https://phabricator.wikimedia.org/T393948#11024210 (10jhathaway) >>! In T393948#11024001, @elukey wrote: > @jhathaway you rock thanks a lot! I verified with a reimage that d-i now correctly handles the new part... [14:19:00] 06Machine-Learning-Team: Configure autoscaling for tone-check model server - https://phabricator.wikimedia.org/T400162 (10isarantopoulos) 03NEW [14:26:22] (03PS5) 10Kevin Bazira: RR: Validate lang parameter against canonical wikis [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) [14:26:58] (03CR) 10Kevin Bazira: RR: Validate lang parameter against canonical wikis (032 comments) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1169195 (https://phabricator.wikimedia.org/T399437) (owner: 10Kevin Bazira) [14:29:42] 06Machine-Learning-Team: Configure autoscaling for tone-check model server - https://phabricator.wikimedia.org/T400162#11024314 (10isarantopoulos) a:03gkyziridis [14:35:53] 06Machine-Learning-Team, 07Essential-Work: Upgrade remaining model servers from debian bullseye to bookworm - https://phabricator.wikimedia.org/T400144#11024368 (10isarantopoulos) [15:02:55] 06Machine-Learning-Team, 07Essential-Work: Reimplement the model-upload script to take into consideration new use cases - https://phabricator.wikimedia.org/T394301#11024493 (10BWojtowicz-WMF) We've discussed the points above in our ML Team Meeting, which resulted in a following plan: Since the current `model_... [17:10:34] 06Machine-Learning-Team: Build model training pipeline for tone check using WMF ML Airflow instance - https://phabricator.wikimedia.org/T396495#11025158 (10BTullis) Thanks for your excellent summary @gkyziridis. >>! In T396495#10970710, @gkyziridis wrote: > #####Solutions & Brainstorming > - **Passing data/model... [17:23:20] (03PS1) 10Sbisson: Fix mostpopular api test [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1171608 [18:36:23] (03CR) 10Nik Gkountas: [C:03+2] Fix mostpopular api test [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1171608 (owner: 10Sbisson) [18:36:56] (03Merged) 10jenkins-bot: Fix mostpopular api test [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1171608 (owner: 10Sbisson) [19:47:22] (03PS9) 10Sbisson: Recommendations based on difficulty level [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1171315 (https://phabricator.wikimedia.org/T399117)