[06:27:54] good morning everyone! ☀️ [06:32:56] Good morning [06:35:11] morning folks :) [06:42:16] (03PS4) 10Bartosz Wójtowicz: langid: Add locust load test for language identification model. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) [06:44:53] (03CR) 10Bartosz Wójtowicz: "Thanks for suggestion, I've removed the other csv fiels with the new patchset and kept only the `stats.csv`." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [06:55:48] good morning! [07:44:41] (03CR) 10Ilias Sarantopoulos: [C:03+1] "Done" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [07:50:24] (03CR) 10Bartosz Wójtowicz: [C:03+2] langid: Add locust load test for language identification model. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [08:17:23] I see that my patch is not getting merged after giving it +2 on code review. Is it because it did not trigger any CI jobs, which would give it the "Verified" vote? https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1152251 [08:20:44] That's odd. [08:21:09] Jenkins did add the G&S message, but the G&S queue is empty and the patch is unmerged [08:21:21] Could you try to remove and re-add the +2? [08:23:25] (03CR) 10Bartosz Wójtowicz: langid: Add locust load test for language identification model. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [08:23:44] (03CR) 10Bartosz Wójtowicz: [C:03+2] langid: Add locust load test for language identification model. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [08:24:01] Let's see what happens [08:25:08] I think that is expected. if no CI jobs are triggered Jenkins can't verify the patch. If I remember correctly we do it manually in these cases. The preferred way would be to add a ci job for the test/ directory with some basic precommit hooks (formatting etc) [08:26:29] 10Lift-Wing, 06Machine-Learning-Team, 13Patch-For-Review: Host an OpenVINO model in LiftWing - https://phabricator.wikimedia.org/T395012#10874407 (10isarantopoulos) **Docker images**: The images that we can use as base images in WMF's production clusters have to be in [[ https://gerrit.wikimedia.org/g/operat... [08:34:17] isaranto: I see, will give the “Verified” manually in the patch. Should I also look into the CI repo to add the basic job for test/ dir after finishing with the prod re-deployments? [08:34:37] that would be great! [08:35:02] (03CR) 10Bartosz Wójtowicz: [V:03+2 C:03+2] langid: Add locust load test for language identification model. [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1152251 (https://phabricator.wikimedia.org/T393865) (owner: 10Bartosz Wójtowicz) [08:43:35] ^ it seems that giving +2 to verified makes it stuck in "Ready to submit" state [08:48:06] And you don't have a "Submit" button at the top? [08:48:21] (I do) [08:48:39] Ahh I missed it, thank you! [08:48:43] np :) [08:48:57] Sometimes, hitting reload is needed to make it appear [08:49:10] 06Machine-Learning-Team, 10Add-Link, 06Growth-Team, 05Goal: Q4 24-25 Goal: Investigate Add-a-link model training and deployment - https://phabricator.wikimedia.org/T393474#10874483 (10OKarakaya-WMF) I've trained/evaluated model per language on top 50 wikis [here](https://en.wikipedia.org/wiki/List_of_Wikip... [09:24:32] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06Wikipedia-Android-App-Backlog, 05FY2024-25 WE4.2, 10Moderator-Tools-Team (Kanban): Ensure all ORES i18n messages are available for wikis to add revert risk language agnostic filters to - https://phabricator.wikimedia.org/T395481#10874596 (10isarantop... [10:09:50] Would someone be up to review a patch updating our ml-services production docker images? 🥺 https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1152643. I re-checked all image and tags and made sure they exist in our docker registry. Staging images were updated and re-deployed last week. [10:21:45] having a look [10:26:53] thank you <3 [10:27:50] I presume the model update for AQLA is intentional? [10:29:26] Yes, this one is intentional, when updating it on staging we've discovered that the new image requires the model to be updated as well as it expects different files [10:30:07] ack, I added a nit, but otherwise LGTM [10:31:38] thank you! adding the model update mention to commit message right away [10:33:03] +1 from me as well. Please update the messagew [10:36:19] 06Machine-Learning-Team, 10Add-Link, 06Growth-Team, 05Goal: Q4 24-25 Goal: Investigate Add-a-link model training and deployment - https://phabricator.wikimedia.org/T393474#10874790 (10Michael) I feel there is a lot here that I do not understand and cannot respond to, but there is one detail I can contribut... [10:39:02] klausman: georgekyz: I've updated the commit message and will merge the patch now, thank you both! [10:39:14] go go goooo [10:39:20] 🤞 [10:53:29] 06Machine-Learning-Team, 10Add-Link, 06Growth-Team, 05Goal: Q4 24-25 Goal: Investigate Add-a-link model training and deployment - https://phabricator.wikimedia.org/T393474#10874856 (10OKarakaya-WMF) thank you @Michael, I've add the query in the image. [11:37:22] (03CR) 10Máté Szabó: "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152267 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [11:37:30] (03CR) 10Máté Szabó: "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152268 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [11:37:46] (03CR) 10Máté Szabó: "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152269 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [11:39:03] (03PS3) 10Máté Szabó: Add revertrisk_score AbuseFilter variable [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152270 (https://phabricator.wikimedia.org/T364705) [11:51:35] (03CR) 10CI reject: [V:04-1] Add revertrisk_score AbuseFilter variable [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152270 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [11:57:18] (03CR) 10Gkyziridis: [C:03+2] improve logging logic for PopulateDatabase backfill script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151706 (https://phabricator.wikimedia.org/T395253) (owner: 10Gkyziridis) [11:57:36] (03CR) 10Gkyziridis: [C:03+2] "Done" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151706 (https://phabricator.wikimedia.org/T395253) (owner: 10Gkyziridis) [12:02:42] (03CR) 10Gkyziridis: [V:03+2 C:03+2] improve logging logic for PopulateDatabase backfill script [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151706 (https://phabricator.wikimedia.org/T395253) (owner: 10Gkyziridis) [12:11:22] I'm re-reploying some ml services on staging after docker image bumps and one of the pods have problems with scheduling as it complains about insufficient CPU on nodes or untolerated taints. Namely, it's `reference-need-predictor-...` pod in `revision-models` namespace. Is there a way we can check if we have enough resources on the nodes? Should I maybe terminate the previous deployment to free up its resources? [12:15:39] you don't have permissions to delete the previous deployment [12:16:24] there is an issue with that namespace as seems difficult to deploy if all replicas are up and running [12:17:04] bartosz: in which cluster is this happening?? [12:17:52] isaranto: it's on `ml-staging-codfw` [12:19:04] ok then yeah this is a different issue than what I was thinking. it is just the node [12:19:22] there is a grafana dashboard -- lemme look it up [12:21:43] https://grafana.wikimedia.org/goto/grcPzXBNg?orgId=1 [12:22:37] only SREs have access to view nodes etc from kubectl but you can use the grafana dashboard [12:23:30] what do you mean by "it is just the node"? [12:23:49] isaranto: sweet, thanks for the grafana link <3 [12:24:17] I mean that it is not a problem with resourcequotas on the namespace. It is that it is a scheduling issue related to the node [12:24:26] ``` [12:24:26] 0/5 nodes are available: 2 node(s) had taint {node-role.kubernetes.io/control-plane: }, that the pod didn't tolerate, 3 Insufficient cpu. [12:24:26] ``` [12:24:37] Well, no. [12:24:40] sorry my first explanation was pretty cryptic [12:25:05] It's not the node, the "taint" referes to ml-staging-ctrl*, which are k8s nodes, but never run workloads [12:25:31] oh ok [12:25:35] 3 nodes (ml-staging200{1,2,3}) don't have enough CPU to spare [12:25:47] (or it's a quota issue) [12:27:41] We still have CPU available on the cluster, so it's a quota issue [12:27:54] the grafana dashboard says that we have 41.9CPUs available, but if it's across all nodes then it's possible that there is not a single node having capacity for ~24CPUs requested by the deployment [12:28:20] true [12:28:48] resourcequotas and limitranges seem ok in that namespace, so it is probably what Bartosz is saying. [12:29:15] perhaps we can just use much much less cpu for this specific staging deployment [12:29:26] https://grafana.wikimedia.org/goto/604GiuBHg?orgId=1 [12:29:45] teh r-m namespace is close to quota [12:30:09] if we need to work on it again , do load testing etc we can deploy a specific instance with more cpu [12:34:39] klausman: How come we have the same quota for limits and requests, but our usage for limits is lower than for requests? Would it mean that we don't specify limits in some of the resources or set the limits lower than requests? [12:35:51] isaranto: Should I create a small patch lowering the limits/requests for the reference-need in staging? [12:36:26] bartosz: this is my suggestion https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1152675 [12:37:41] sorry should have allowed you to do it so that you can play around with the config [12:37:47] I hope this would work now [12:39:12] bartosz: the metrics on that dashboard are very confusing, I am trying to figure it out [12:43:59] isaranto: No worries, thanks for providing the fix 🙌 I've left 1 small comment before merging [13:01:42] bartosz: I merged it, you can try again in a bit and see if it works now [13:04:58] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 10Moderator-Tools-Team (Kanban), 13Patch-For-Review: Enable revertrisk filters in simplewiki & trwiki - https://phabricator.wikimedia.org/T395668#10875347 (10gkyziridis) [13:05:54] (03CR) 10Ilias Sarantopoulos: "Thanks for solving this! I have a question regarding the deleted js/less modules" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [13:07:55] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 10Moderator-Tools-Team (Kanban), 13Patch-For-Review: Enable revertrisk filters in simplewiki & trwiki - https://phabricator.wikimedia.org/T395668#10875351 (10isarantopoulos) [13:09:42] georgekyz: the checklist in the task https://phabricator.wikimedia.org/T395668 is a bit misleading and one could assume that the filters are already deployed. I should have phrased it a bit differently. I'm going to change it to "deploy mw-config change" [13:11:40] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 10Moderator-Tools-Team (Kanban), 13Patch-For-Review: Enable revertrisk filters in simplewiki & trwiki - https://phabricator.wikimedia.org/T395668#10875373 (10isarantopoulos) [13:12:53] isaranto: thanks! Unfortunately looking at the helm diff now, I see that it overwrote the entire `custom_env` and deleted the essential env variables like `STORAGE_URI`, `MODEL_TO_DEPLOY`. Here's a follow-up patch setting them back: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1152686 [13:14:30] 🤦‍♂️ sorry about that. I had updated the patch locally but never pushed before merging [13:15:06] +1'd [13:16:03] thank you! will try re-deploying in a second [13:21:15] isaranto: klausman: The new deployment with lower requests/limits is happily running now and previous pending deployment got terminated. thanks for the help! [13:21:38] \o/ [13:21:51] np :) [13:30:42] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06DBA, 10MediaWiki-Recent-changes, and 3 others: [Epic] Recent Changes ORES Enabled Revert Risk Powered Filters Rollout Plan - https://phabricator.wikimedia.org/T391964#10875432 (10isarantopoulos) [13:30:43] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06Wikipedia-Android-App-Backlog, 05FY2024-25 WE4.2, 10Moderator-Tools-Team (Kanban): Add revertrisk-language-agnostic to RecentChanges filters - https://phabricator.wikimedia.org/T348298#10875433 (10isarantopoulos) [13:30:45] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06Wikipedia-Android-App-Backlog, 05FY2024-25 WE4.2, 10Moderator-Tools-Team (Kanban): Ensure all ORES i18n messages are available for wikis to add revert risk language agnostic filters to - https://phabricator.wikimedia.org/T395481#10875431 (10isarantop... [14:14:02] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06DBA, 10MediaWiki-Recent-changes, and 3 others: [Epic] Recent Changes ORES Enabled Revert Risk Powered Filters Rollout Plan - https://phabricator.wikimedia.org/T391964#10875621 (10gkyziridis) Hey @Kgraessle and @isarantopoulos. I just submitted a patch... [14:16:17] 06Machine-Learning-Team, 10Add-Link, 06Growth-Team, 05Goal: Q4 24-25 Goal: Investigate Add-a-link model training and deployment - https://phabricator.wikimedia.org/T393474#10875639 (10OKarakaya-WMF) Meeting notes with Martin: - How do we query databases? What are the important tables to check? (e.g. growt... [14:17:27] Hello, I’ve shared my meeting notes with Martin above [14:20:02] o/ Thank youuu [14:48:10] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MW-1.45-notes (1.45.0-wmf.4; 2025-06-03): Improve ORES extension table backfill script - https://phabricator.wikimedia.org/T395253#10875825 (10jsn.sherman) [15:04:48] (03PS3) 10Máté Szabó: LiftWingService: Add method to evaluate pre-save revert risk [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152269 (https://phabricator.wikimedia.org/T364705) [15:04:49] (03PS4) 10Máté Szabó: Add revertrisk_score AbuseFilter variable [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152270 (https://phabricator.wikimedia.org/T364705) [15:11:57] 10Lift-Wing, 06Machine-Learning-Team, 13Patch-For-Review: Host an OpenVINO model in LiftWing - https://phabricator.wikimedia.org/T395012#10875924 (10kevinbazira) >>! In T395012#10867030, @santhosh wrote: > Regarding the KServe API and Openvino model server: > ... > Since you people tried VLLM inference, I am... [15:15:43] 06Machine-Learning-Team: Article Summary Generation and Evaluation Pipeline using vLLM image - https://phabricator.wikimedia.org/T395246#10875933 (10kevinbazira) >>! In T395246#10871778, @isarantopoulos wrote: > Could you rerun these next week using also prompt [[ https://gitlab.wikimedia.org/repos/research/simp... [15:18:04] (03CR) 10CI reject: [V:04-1] Add revertrisk_score AbuseFilter variable [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152270 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [15:18:44] (03CR) 10CI reject: [V:04-1] LiftWingService: Add method to evaluate pre-save revert risk [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152269 (https://phabricator.wikimedia.org/T364705) (owner: 10Máté Szabó) [15:45:20] (03PS4) 10Máté Szabó: LiftWingService: Add method to evaluate pre-save revert risk [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152269 (https://phabricator.wikimedia.org/T364705) [15:45:20] (03PS5) 10Máté Szabó: Add revertrisk_score AbuseFilter variable [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1152270 (https://phabricator.wikimedia.org/T364705) [15:50:31] 06Machine-Learning-Team: Article Summary Generation and Evaluation Pipeline using vLLM image - https://phabricator.wikimedia.org/T395246#10876040 (10isarantopoulos) Thank you! I see some of the filenames are enclosed in either single or double quotes, while the [[ https://gerrit.wikimedia.org/r/plugins/gitiles/m... [16:01:19] 06Machine-Learning-Team: Article Summary Generation and Evaluation Pipeline using vLLM image - https://phabricator.wikimedia.org/T395246#10876078 (10isarantopoulos) I think one is missing from the directory. If I'm not mistaken it is `...And Justice for All (album).json` [16:04:00] (03PS10) 10Jsn.sherman: Fix highlighting for revertrisklanguageagnostic [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [16:04:18] (03PS11) 10Jsn.sherman: Fix highlighting for revertrisklanguageagnostic [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [16:05:30] (03CR) 10Jsn.sherman: "In the interest of getting this out ahead of branch cuts, I reduced the change to the model name reference for the highlighting only" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [16:05:43] (03CR) 10Jsn.sherman: Fix highlighting for revertrisklanguageagnostic (031 comment) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [16:09:32] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 10Moderator-Tools-Team (Kanban), 13Patch-For-Review: Enable revertrisk filters in simplewiki & trwiki [batch #1] - https://phabricator.wikimedia.org/T395668#10876095 (10isarantopoulos) [16:25:20] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #2] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395823 (10isarantopoulos) 03NEW [16:27:59] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #2] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395823#10876249 (10isarantopoulos) [16:32:34] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #2] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395823#10876277 (10isarantopoulos) [16:32:45] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06DBA, 10MediaWiki-Recent-changes, and 2 others: [batch #3] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395824 (10isarantopoulos) 03NEW [16:33:07] (03CR) 10Scardenasmolinar: [C:03+2] "Thanks for making the changes!" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [16:33:40] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team, 07Epic: [batch #3] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395824#10876294 (10Marostegui) [16:34:35] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #3] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395824#10876300 (10isarantopoulos) [16:37:19] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #2] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395823#10876313 (10isarantopoulos) [16:39:24] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team: [batch #3] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395824#10876322 (10isarantopoulos) [16:51:09] phabricator is my friend now <3 [17:07:17] * isaranto afk! [17:12:33] 06Machine-Learning-Team, 10Add-Link, 06Growth-Team, 05Goal: Q4 24-25 Goal: Investigate Add-a-link model training and deployment - https://phabricator.wikimedia.org/T393474#10876487 (10fkaelin) > I see these wikis were above the threshold in [these](https://docs.google.com/spreadsheets/d/1wI2cqu9dPeEa_T7iM... [17:18:22] (03CR) 10CI reject: [V:04-1] Fix highlighting for revertrisklanguageagnostic [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [17:21:41] (03CR) 10Scardenasmolinar: [C:03+2] "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1151700 (https://phabricator.wikimedia.org/T395256) (owner: 10Kgraessle) [17:43:38] 06Machine-Learning-Team, 06Web-Team: Non-English articles show autogenerated English summaries - https://phabricator.wikimedia.org/T395596#10876700 (10Jdlrobson-WMF) p:05High→03Medium [18:31:53] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06DBA, 10MediaWiki-Recent-changes, and 3 others: [Epic] Recent Changes ORES Enabled Revert Risk Powered Filters Rollout Plan - https://phabricator.wikimedia.org/T391964#10876984 (10isarantopoulos) @gkyziridis I have separated the wikis in 2 groups after... [20:39:58] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 06DBA, 10MediaWiki-Recent-changes, and 2 others: [Epic] Recent Changes ORES Enabled Revert Risk Powered Filters Rollout Plan - https://phabricator.wikimedia.org/T391964#10877448 (10A_smart_kitten) [20:57:12] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team, and 3 others: [batch #2] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395823#10877536 (10A_smart_kitten) [20:57:47] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team, and 2 others: [batch #3] Enable revertrisk filters in recent changes in multiple wikis - https://phabricator.wikimedia.org/T395824#10877541 (10A_smart_kitten)