[05:06:33] Good morning o/ [06:59:31] (03CR) 10AikoChou: [C:03+1] "So with this change, if we want to add additional args, we will add a "command" in deployment-charts (https://github.com/wikimedia/operati" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055972 (https://phabricator.wikimedia.org/T370670) (owner: 10Ilias Sarantopoulos) [07:00:13] morning! [07:03:42] (03CR) 10Ilias Sarantopoulos: "Yes either specifying just the command or args would do the trick." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055972 (https://phabricator.wikimedia.org/T370670) (owner: 10Ilias Sarantopoulos) [07:03:53] hey! [07:11:11] aiko: no need to test the apple silicon patch yet. It does run but it is too slow. I need to build a image specific for arm and figure out where to set torch.device to mps as it is everywhere hardcoded as either cuda or cpu [08:05:51] (03CR) 10Klausman: [C:03+1] huggingface: accept cmd args in docker entrypoint [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055972 (https://phabricator.wikimedia.org/T370670) (owner: 10Ilias Sarantopoulos) [08:13:16] (03CR) 10Ilias Sarantopoulos: [V:03+2 C:03+2] huggingface: accept cmd args in docker entrypoint [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055972 (https://phabricator.wikimedia.org/T370670) (owner: 10Ilias Sarantopoulos) [08:54:31] ack [09:15:06] 早上好！ [09:44:21] 早~~Luca :D [10:15:06] 06Machine-Learning-Team, 10Structured-Data-Backlog (Current Work): Estimate the logo detection service's expected load - https://phabricator.wikimedia.org/T370756 (10mfossati) 03NEW [10:15:28] 06Machine-Learning-Team, 10Structured-Data-Backlog (Current Work): Estimate the logo detection service's expected load - https://phabricator.wikimedia.org/T370756#10006168 (10mfossati) [10:16:35] 06Machine-Learning-Team, 06Structured-Data-Backlog: Deploy logo-detection model-server to LiftWing production - https://phabricator.wikimedia.org/T370757 (10mfossati) 03NEW [10:17:05] 06Machine-Learning-Team, 06Structured-Data-Backlog: Deploy logo-detection model-server to LiftWing production - https://phabricator.wikimedia.org/T370757#10006186 (10mfossati) [10:21:42] 06Machine-Learning-Team, 10Structured-Data-Backlog (Current Work): Create the logo detection model card - https://phabricator.wikimedia.org/T370759 (10mfossati) 03NEW [10:22:28] 06Machine-Learning-Team, 10Structured-Data-Backlog (Current Work): Create the logo detection model card - https://phabricator.wikimedia.org/T370759#10006220 (10mfossati) [10:37:35] aiko: I'm rebuilding one last image and I'll push the updates [10:39:56] isaranto: ok o/ [10:40:18] * klausman lunch [10:48:22] * isaranto lunch and errand [11:29:42] (03CR) 10Kevin Bazira: [C:03+2] major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh) [11:31:07] (03CR) 10CI reject: [V:04-1] major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh) [11:36:52] isaranto: so I checked the application I mentioned, but it's not related to our case. It uses an ollama client [11:57:17] ok, thanks for checking! [12:54:15] Morning all [12:55:23] morning Chris! [13:01:44] (03PS4) 10Ilias Sarantopoulos: huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 [13:03:02] I'm going to sync the wikimedia/kserve fork with the latest changes from kserve/kserve if that's ok [13:03:08] sgtm [13:09:58] hi Chris o/ [13:11:43] ok! [13:12:21] done! [13:14:50] ok tested it, all fine! [13:30:46] (03CR) 10AikoChou: [C:03+1] "I tested on my Mac and it worked like a charm." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 (owner: 10Ilias Sarantopoulos) [13:31:07] aiko: I found https://github.com/pytorch/pytorch/issues/81224 [13:31:29] ---^ only the branch name is wrong [13:31:53] sorry about that [13:31:58] fixin it now! [13:33:15] it seems that using a different image we may be able to use the integrated gpu BUT I think that it isn't worth the hassle. I just remembered that some torch operations or model architectures (can't recall exactly what) aren't supported by default for mps but only for cuda/cpu [13:33:49] isaranto: yeah I saw it. the conclusion looks like docker doesn't support MPS :( [13:34:13] perhaps this may have changed already as my last check was many months ago. I'm willing to give it a try today otherwise I would just do things on cpu with smaller models if you agree [13:34:29] it does the job for verifying that things work at least [13:35:14] (03PS5) 10Ilias Sarantopoulos: huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 [13:35:30] agree, it is ok on cpu only [13:35:43] for testing locally [13:35:44] (03PS6) 10Ilias Sarantopoulos: huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 [13:36:23] (03CR) 10AikoChou: [C:03+1] huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 (owner: 10Ilias Sarantopoulos) [13:36:33] (03CR) 10Ilias Sarantopoulos: "Done! Sorry I forgot to push the latest changes." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 (owner: 10Ilias Sarantopoulos) [13:40:11] (03CR) 10Nik Gkountas: ""pageviews" returned by the "query" API can be null (e.g. https://en.wikipedia.org/w/api.php?action=query&format=json&prop=pageviews&meta=" [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh) [13:40:50] (03PS11) 10Nik Gkountas: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh) [13:41:38] (03CR) 10Nik Gkountas: "Fixed in the latest patchset." [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh) [13:41:56] (03PS5) 10Nik Gkountas: Recommend articles to translate based on topic [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052950 (https://phabricator.wikimedia.org/T367873) (owner: 10Santhosh) [13:49:38] you're right (so is the conclusion in the above GH issue), there doesn't seem to be an image that supports this. So all I'm going to do is try it locally in a virtual env without docker that uses mps [13:58:26] 06Machine-Learning-Team: [LLM] log input/output size per request - https://phabricator.wikimedia.org/T370775 (10isarantopoulos) 03NEW [13:59:51] 06Machine-Learning-Team: [LLM] Run LLMs locally in ml-testing - https://phabricator.wikimedia.org/T370656#10006908 (10isarantopoulos) a:03kevinbazira [14:22:08] 06Machine-Learning-Team, 10Structured-Data-Backlog (Current Work): Estimate the logo detection service's expected load - https://phabricator.wikimedia.org/T370756#10006977 (10mfossati) 05Open→03In progress [15:00:16] (03CR) 10Ilias Sarantopoulos: [C:03+2] huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 (owner: 10Ilias Sarantopoulos) [15:01:02] (03Merged) 10jenkins-bot: huggingface: add blubber image for cpu/apple silicon [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1055977 (owner: 10Ilias Sarantopoulos) [15:16:06] * isaranto afk [15:26:38] 07artificial-intelligence, 10research-ideas: Machine translation learned by comparing content across wikipedia languages - https://phabricator.wikimedia.org/T155847#10007275 (10Prototyperspective) I don't think it makes sense to develop something learning from Wikipedia translation when tools already do and wh... [15:28:55] 07artificial-intelligence: Use AI to automatically generate edit summaries - https://phabricator.wikimedia.org/T334598#10007292 (10Prototyperspective) https://arxiv.org/abs/2404.03428 Edisum: Summarizing and Explaining Wikipedia Edits at Scale [15:36:03] (03CR) 10Nik Gkountas: [C:04-1] "just a minor issue with a dummy model_config inside models.py" [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052950 (https://phabricator.wikimedia.org/T367873) (owner: 10Santhosh) [15:54:05] * isaranto back for a bit [16:20:55] Llama 3.1 is out today https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f [16:24:43] and the related transformers update is already there https://github.com/huggingface/transformers/releases/tag/v4.43.0 [16:26:39] but we wan't use it in our model server until the transformers requirement is relaxed which is what this PR does https://github.com/kserve/kserve/pull/3783 [16:35:33] going afk for the evening o/ [16:51:04] o/