[00:08:08] (03update) 10habs: T434468: Add 'See also' lists and user-contributed related articles [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/32 (owner: 10ecoholic) [00:15:38] (03update) 10habs: T431879: Mark new articles with a seedling icon, format age in green; some caching changes; add local-testing mode [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/34 (owner: 10snowfire) [00:47:17] 10Toolforge, 06tools-platform-team: [functional-tests,deploy] Lately got ~50% errors due to not finding the branch - https://phabricator.wikimedia.org/T440224#12531551 (10Raymond_Ndibe) have you noticed this again @dcaro ? [00:57:14] (03update) 10habs: T434468: Add 'See also' lists and user-contributed related articles [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/32 (owner: 10ecoholic) [01:05:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [01:10:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [01:12:43] 10Toolforge, 06tools-platform-team, 13Patch-For-Review: [builds-api] returns incorrect parameter values for use_latest_versions & use_deprecated_versions causing [components-api] to not re-use existing builds - https://phabricator.wikimedia.org/T442364#12531577 (10Raymond_Ndibe) 05Open→03Resolved a:... [01:18:44] (03approved) 10samwilson: composer: Run bin/console through PHP so the scripts work on Windows [toolforge-repos/ocr] - 10https://gitlab.wikimedia.org/toolforge-repos/ocr/-/merge_requests/22 (https://phabricator.wikimedia.org/T420874) (owner: 10zaidusyy) [01:19:13] (03merge) 10samwilson: composer: Run bin/console through PHP so the scripts work on Windows [toolforge-repos/ocr] - 10https://gitlab.wikimedia.org/toolforge-repos/ocr/-/merge_requests/22 (https://phabricator.wikimedia.org/T420874) (owner: 10zaidusyy) [01:19:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [01:34:14] 10Toolforge, 06tools-platform-team: [harbor] Review default quota - https://phabricator.wikimedia.org/T444050#12531621 (10Raymond_Ndibe) > Doing some simply maths, we can never hit the 16 job limit with build pack as we would need more than double the quota. > Taking sample-complex-app as a generic 'supported'... [01:34:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [02:23:49] (03update) 10habs: T434468: Add 'See also' lists and user-contributed related articles [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/32 (owner: 10ecoholic) [02:26:00] FIRING: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [02:26:32] (03approved) 10habs: T434468: Add 'See also' lists and user-contributed related articles [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/32 (owner: 10ecoholic) [02:26:52] (03merge) 10habs: T434468: Add 'See also' lists and user-contributed related articles [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/32 (owner: 10ecoholic) [02:32:54] (03update) 10habs: T431879: Mark new articles with a seedling icon, format age in green; some caching changes; add local-testing mode [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/34 (owner: 10snowfire) [02:35:37] (03approved) 10habs: T431879: Mark new articles with a seedling icon, format age in green; some caching changes; add local-testing mode [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/34 (owner: 10snowfire) [02:36:00] FIRING: [2x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [02:36:38] (03merge) 10habs: T431879: Mark new articles with a seedling icon, format age in green; some caching changes; add local-testing mode [toolforge-repos/wikinewsie] - 10https://gitlab.wikimedia.org/toolforge-repos/wikinewsie/-/merge_requests/34 (owner: 10snowfire) [02:37:37] (03open) 10ibrahemqasim: Refactor citation template resolution [toolforge-repos/mdwiki] (zz) - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/34 [02:39:23] (03update) 10ibrahemqasim: Refactor citation template resolution [toolforge-repos/mdwiki] (zz) - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/34 [02:39:29] (03merge) 10ibrahemqasim: Refactor citation template resolution [toolforge-repos/mdwiki] (zz) - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/34 [02:39:34] (03update) 10ibrahemqasim: Add citation template collection dashboard [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/33 [02:39:43] (03update) 10ibrahemqasim: Add citation template collection dashboard [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/33 [02:40:13] (03approved) 10ibrahemqasim: Add citation template collection dashboard [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/33 [02:41:11] (03merge) 10ibrahemqasim: Add citation template collection dashboard [toolforge-repos/mdwiki] - 10https://gitlab.wikimedia.org/toolforge-repos/mdwiki/-/merge_requests/33 [02:55:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [03:10:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [03:21:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [06:18:56] (03merge) 10countcount: Update Rust crate replica-pools to v0.1.3 [toolforge-repos/multiuserinfo] - 10https://gitlab.wikimedia.org/toolforge-repos/multiuserinfo/-/merge_requests/94 (owner: 10renovatebot) [06:29:32] (03merge) 10countcount: Update pnpm to v12.18.0 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/67 (owner: 10renovatebot) [06:29:39] (03merge) 10countcount: Update pnpm to v12.18.0 [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/78 (owner: 10renovatebot) [06:29:41] (03merge) 10countcount: Update pnpm to v12.18.0 [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/48 (owner: 10renovatebot) [06:29:45] (03merge) 10countcount: Update pnpm to v12.18.0 [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/49 (owner: 10renovatebot) [06:29:49] (03merge) 10countcount: Update pnpm to v12.18.0 [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/41 (owner: 10renovatebot) [07:14:02] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39 [07:14:17] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48 [07:14:25] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46 [07:15:12] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76 [07:16:15] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/multiuserinfo] - 10https://gitlab.wikimedia.org/toolforge-repos/multiuserinfo/-/merge_requests/93 [07:16:56] (03update) 10renovatebot: Update dependency jsdom to v31 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/68 [07:17:08] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65 [07:21:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [08:38:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [08:39:18] 10Cloud-VPS, 06tools-infrastructure-team: Only 50 distinct volume objects are backed up on cloudbackup2004:9100 (codfw), below 90% of the level 24h ago. Some backups may have been lost. - https://phabricator.wikimedia.org/T443945#12532000 (10fgiunchedi) [08:42:57] 10Cloud-VPS, 06tools-infrastructure-team: BackupCoverageDrop: Only 50 distinct volume objects are backed up on cloudbackup2004:9100 (codfw), below 90% of the level 24h ago. Some backups may have been lost. - https://phabricator.wikimedia.org/T443945#12532024 (10fgiunchedi) [08:48:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [08:49:36] (03update) 10raymond-ndibe: docs: specify python version for data model generation [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/215 (https://phabricator.wikimedia.org/T443675) [08:53:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [08:57:57] (03update) 10raymond-ndibe: docs: specify python version for data model generation [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/215 (https://phabricator.wikimedia.org/T443675) [08:58:17] (03update) 10raymond-ndibe: utils: add a script to regenerate the toolforge models [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/215 (https://phabricator.wikimedia.org/T443675) [08:59:11] (03update) 10raymond-ndibe: utils: add a script to regenerate the toolforge models [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/215 (https://phabricator.wikimedia.org/T443675) [09:03:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:19:30] 10Toolforge, 06tools-platform-team: [harbor] Review default quota - https://phabricator.wikimedia.org/T444050#12532558 (10DamianZaremba) > The above figures are slightly wrong if we take into account that harbor counts each layer once per project. The figure is more like: > ` > Average tool usage: 322.5 MB > N... [11:21:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [11:42:21] !log tlepage@cloudcumin1001 tools START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry for image docker-registry.svc.toolforge.org/nginx:1.30.6 (T440736) [11:42:26] T440736: api-gateway: Support bearer-token auth for Striker - https://phabricator.wikimedia.org/T440736 [11:42:47] !log tlepage@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) for image docker-registry.svc.toolforge.org/nginx:1.30.6 (T440736) [11:58:54] 10Tools: New Toolforge tool: drag-and-drop visual page designer that publishes wikitext - https://phabricator.wikimedia.org/T444134 (10Punith.nyk) 03NEW [12:49:45] 10Tool-Attribution-Generator, 10Tool-dataweaver: Att - https://phabricator.wikimedia.org/T444028#12533248 (10Esanders) 05Open→03Invalid [13:00:30] (03open) 10r4356th: Stop normalizing attribute values [toolforge-repos/delintbot] - 10https://gitlab.wikimedia.org/toolforge-repos/delintbot/-/merge_requests/52 [13:04:40] (03open) 10dtorsani: updated icons used in signals, updated codex version to use the new icons,... [toolforge-repos/wikimedia-attribution] - 10https://gitlab.wikimedia.org/toolforge-repos/wikimedia-attribution/-/merge_requests/11 [13:05:06] (03merge) 10r4356th: Stop normalizing attribute values [toolforge-repos/delintbot] - 10https://gitlab.wikimedia.org/toolforge-repos/delintbot/-/merge_requests/52 [13:05:06] (03update) 10dtorsani: Added initial examples to adoption gallery [toolforge-repos/wikimedia-attribution] - 10https://gitlab.wikimedia.org/toolforge-repos/wikimedia-attribution/-/merge_requests/9 [13:33:07] 10Tools, 10Lift-Wing, 06Research, 06Machine-Learning-Team (Q2 FY2026-27): FactDiff: an editor worklist of conflicting dates and numbers across Wikipedia language editions (WIP, feedback welcome) - https://phabricator.wikimedia.org/T441279#12533505 (10achou) Hi @Devdoingdev, Aiko from the WMF ML team here.... [13:55:12] 10Cloud-VPS, 06tools-infrastructure-team: BackupCoverageDrop: Only 50 distinct volume objects are backed up on cloudbackup2004:9100 (codfw), below 90% of the level 24h ago. Some backups may have been lost. - https://phabricator.wikimedia.org/T443945#12533668 (10fgiunchedi) Indeed perhaps 90% is too high on < 1... [14:13:12] 10Tools: New Toolforge tool: drag-and-drop visual page designer that publishes wikitext - https://phabricator.wikimedia.org/T444134#12533781 (10Punith.nyk) 05Open→03Declined Closing this for now; I will revisit the approach (standalone tool vs. extension/gadget) before reopening. No matching wish found o... [14:15:54] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-trainer Toolforge tool - https://phabricator.wikimedia.org/T444055#12533788 (10Raymond_Ndibe) +1 [14:16:23] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-editsets Toolforge tool - https://phabricator.wikimedia.org/T444054#12533791 (10Raymond_Ndibe) +1 [14:16:45] 10Toolforge (Quota-requests): Request increased build quota for cluebot-documentation Toolforge tool - https://phabricator.wikimedia.org/T444053#12533793 (10Raymond_Ndibe) +1 [14:17:25] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-staging Toolforge tool - https://phabricator.wikimedia.org/T444052#12533798 (10Raymond_Ndibe) +1 [14:17:56] 10Toolforge (Quota-requests): Request increased build quota for cluebot3 Toolforge tool - https://phabricator.wikimedia.org/T444051#12533804 (10Raymond_Ndibe) +1 [14:18:35] 10Cloud-VPS (Project-requests): Request creation of deorph VPS project - https://phabricator.wikimedia.org/T442348#12533812 (10fgiunchedi) >>! In T442348#12529922, @fnegri wrote: > Next week, I will create a `deorph-db` project in Cloud VPS where the Trove database can be created. +1 to trove-only project [14:20:37] !log tools.cluebotng-trainer Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42505001795 (https://github.com/cluebotng/component-configs/commits/3d315d295fbd9ddb713c5f5abe3e6e921e02f904) [14:20:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-trainer/SAL [14:27:42] 10Toolforge (Quota-requests): Request increased build quota for cluebot3 Toolforge tool - https://phabricator.wikimedia.org/T444051#12533852 (10TLepage-WMF) +1 [14:27:52] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-staging Toolforge tool - https://phabricator.wikimedia.org/T444052#12533854 (10TLepage-WMF) +1 [14:28:01] 10Toolforge (Quota-requests): Request increased build quota for cluebot-documentation Toolforge tool - https://phabricator.wikimedia.org/T444053#12533856 (10TLepage-WMF) +1 [14:28:14] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-editsets Toolforge tool - https://phabricator.wikimedia.org/T444054#12533858 (10TLepage-WMF) +1 [14:28:29] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-trainer Toolforge tool - https://phabricator.wikimedia.org/T444055#12533863 (10TLepage-WMF) +1 [14:32:27] 10Toolforge, 06tools-platform-team: [functional tests,components-api] config generate tests load the CLI warnings as config - https://phabricator.wikimedia.org/T444170 (10Raymond_Ndibe) 03NEW [14:32:52] 10Toolforge, 06tools-platform-team: [functional tests,components-api] config generate tests load the CLI warnings as config - https://phabricator.wikimedia.org/T444170#12533945 (10Raymond_Ndibe) 05Open→03In progress [14:36:13] 10Toolforge, 06tools-platform-team: [functional tests,components-api] config generate tests load the CLI warnings as config - https://phabricator.wikimedia.org/T444170#12533961 (10Raymond_Ndibe) [14:37:52] (03open) 10raymond-ndibe: components-api: keep warnings out of the generated config [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1508 (https://phabricator.wikimedia.org/T444170) [14:38:16] (03update) 10raymond-ndibe: models: reject unknown fields in the tool config [repos/cloud/toolforge/components-api] (add_component_parsing_tests) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/216 (https://phabricator.wikimedia.org/T362077) [14:52:20] 10Striker, 06tools-platform-team, 10Charts, 06Language and Product Localization, and 11 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#12534075 (10karapayneWMDE) [15:06:42] (03open) 10pcendrer: Check function [toolforge-repos/pcendrerbot-nombres-inapropiados] - 10https://gitlab.wikimedia.org/toolforge-repos/pcendrerbot-nombres-inapropiados/-/merge_requests/1 [15:06:53] (03update) 10pcendrer: Check function [toolforge-repos/pcendrerbot-nombres-inapropiados] - 10https://gitlab.wikimedia.org/toolforge-repos/pcendrerbot-nombres-inapropiados/-/merge_requests/1 [15:07:09] (03approved) 10pcendrer: Check function [toolforge-repos/pcendrerbot-nombres-inapropiados] - 10https://gitlab.wikimedia.org/toolforge-repos/pcendrerbot-nombres-inapropiados/-/merge_requests/1 [15:21:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [15:31:01] 10Tools, 10Lift-Wing, 06Research, 06Machine-Learning-Team (Q2 FY2026-27): FactDiff: an editor worklist of conflicting dates and numbers across Wikipedia language editions (WIP, feedback welcome) - https://phabricator.wikimedia.org/T441279#12534448 (10Devdoingdev) Hi @achou, thanks a lot, great to hear that... [15:57:55] 10Tools, 10Lift-Wing, 06Research, 06Machine-Learning-Team (Q2 FY2026-27): FactDiff: an editor worklist of conflicting dates and numbers across Wikipedia language editions (WIP, feedback welcome) - https://phabricator.wikimedia.org/T441279#12534520 (10Devdoingdev) A quick follow-up, since six articles is a... [16:11:54] (03PS1) 10Bking: IDP: Add dummy secret for staging Matomo [labs/private] - 10https://gerrit.wikimedia.org/r/1367873 (https://phabricator.wikimedia.org/T436186) [16:13:19] (03CR) 10Bking: [V:03+2 C:03+2] IDP: Add dummy secret for staging Matomo [labs/private] - 10https://gerrit.wikimedia.org/r/1367873 (https://phabricator.wikimedia.org/T436186) (owner: 10Bking) [16:16:11] 06cloud-services-team, 10Toolforge, 06tools-platform-team, 10Pywikibot: unexpected root-owned files in /data/project/pywikibot/public_html - https://phabricator.wikimedia.org/T375279#12534709 (10Raymond_Ndibe) >>! In T375279#12530041, @Xqt wrote: > Could someone with root privileges remove these directorie... [16:26:22] 10Cloud-VPS, 06tools-infrastructure-team, 06Test Platform (San Miguel de Allende 31): Test/benchmark virtio multiqueue for integration-cache-proxy - https://phabricator.wikimedia.org/T440418#12534763 (10AMarkossyan-WMF) a:03Mhurd [16:27:11] 10Cloud-VPS, 06tools-infrastructure-team, 06Test Platform (San Miguel de Allende 31): Test/benchmark virtio multiqueue for integration-cache-proxy - https://phabricator.wikimedia.org/T440418#12534766 (10Mhurd) Will circle back to testing this once the setting is enabled: https://phabricator.wikimedia.org/T44... [16:34:33] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-trainer Toolforge tool - https://phabricator.wikimedia.org/T444055#12534827 (10fnegri) a:03fnegri [16:34:38] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-editsets Toolforge tool - https://phabricator.wikimedia.org/T444054#12534829 (10fnegri) a:03fnegri [16:34:43] 10Toolforge (Quota-requests): Request increased build quota for cluebot-documentation Toolforge tool - https://phabricator.wikimedia.org/T444053#12534832 (10fnegri) a:03fnegri [16:34:49] 10Toolforge (Quota-requests): Request increased build quota for cluebotng-staging Toolforge tool - https://phabricator.wikimedia.org/T444052#12534833 (10fnegri) a:03fnegri [16:46:48] 10Toolforge (Quota-requests): Request increased build quota for cluebot3 Toolforge tool - https://phabricator.wikimedia.org/T444051#12534980 (10fnegri) > Build Quota increase requested: 2Gi > Similar requests have previously been completed under T405645, T405644 & T405643. In the old ones the increase request... [16:48:48] 10Toolforge (Quota-requests): Request increased build quota for cluebot3 Toolforge tool - https://phabricator.wikimedia.org/T444051#12534997 (10DamianZaremba) > In the old ones the increase requested was +1Gi, can you clarify if you would like these new ones to also be set to a total 2Gi quota (+1Gi increase), o... [17:21:13] 06cloud-services-team, 10Toolforge, 06tools-platform-team, 10Pywikibot: unexpected root-owned files in /data/project/pywikibot/public_html - https://phabricator.wikimedia.org/T375279#12535129 (10Xqt) 05Open→03Stalled [17:42:25] 10Tool-wikinewsie: Add tooltip explanations for "Mix", "Editors", "Edits", etc - https://phabricator.wikimedia.org/T435732#12535199 (10Pharos) 05In progress→03Resolved [17:42:38] 10Tool-wikinewsie: New Wikipedia articles should be their own event type on Wikinewsie - https://phabricator.wikimedia.org/T431879#12535201 (10Pharos) 05In progress→03Resolved [17:43:14] 10Tool-wikinewsie: 'See also' mode for Wikinewsie - https://phabricator.wikimedia.org/T434468#12535203 (10Pharos) 05In progress→03Resolved [17:45:14] (03approved) 10tlepage: toolforge-cd,utils: auto-create phab tasks for the monthly upgrade MRs [repos/cloud/cicd/gitlab-ci] - 10https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/merge_requests/102 (https://phabricator.wikimedia.org/T443234) (owner: 10raymond-ndibe) [17:46:45] (03update) 10danyya: Draft: Resolve T434400 "Dumps ingester crashes because of too many prepared statements (2)" [toolforge-repos/wiki-gender-stats] (develop) - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-gender-stats/-/merge_requests/109 [17:50:10] (03approved) 10tlepage: components-api: keep warnings out of the generated config [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1508 (https://phabricator.wikimedia.org/T444170) (owner: 10raymond-ndibe) [17:51:54] 14cloud-services-team (Kanban), 06Data-Engineering, 10Dumps-Generation, 10Wikimedia-Portals: dumps.wikimedia.org/other/pageviews/ appears to be stalled at 20210106-140000 - https://phabricator.wikimedia.org/T271561#12535236 (10bd808) >>! In T271561#12530711, @Superyetkin wrote: > The problem has just reapp... [18:57:23] 14cloud-services-team (Kanban), 06Data-Engineering, 10Dumps-Generation, 10Wikimedia-Portals: dumps.wikimedia.org/other/pageviews/ appears to be stalled at 20210106-140000 - https://phabricator.wikimedia.org/T271561#12535466 (10Superyetkin) 05Open→03Resolved The dumps have arrived in the correct loc... [19:02:10] 10Cloud-VPS, 06tools-infrastructure-team, 10Stashbot: stashbot reconnects after 15 minutes on ping timeout - https://phabricator.wikimedia.org/T443568#12535473 (10bd808) Stashbot doesn't set any explicit server ping/keep-alive monitor, so it gets whatever the default service behavior it has here from https:/... [19:07:38] FIRING: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [19:12:38] RESOLVED: ProbeDown: Service toolsbeta-test-k8s-haproxy-7:443 has failed probes (http_admin_beta_toolforge_org_ip6) - https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Runbooks/k8s-haproxy - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [19:21:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [19:49:04] (03update) 10danyya: Draft: Resolve T434400 "Dumps ingester crashes because of too many prepared statements (2)" [toolforge-repos/wiki-gender-stats] (develop) - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-gender-stats/-/merge_requests/109 [19:56:55] (03approved) 10raymond-ndibe: toolforge-cd,utils: auto-create phab tasks for the monthly upgrade MRs [repos/cloud/cicd/gitlab-ci] - 10https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/merge_requests/102 (https://phabricator.wikimedia.org/T443234) [19:57:06] (03merge) 10raymond-ndibe: toolforge-cd,utils: auto-create phab tasks for the monthly upgrade MRs [repos/cloud/cicd/gitlab-ci] - 10https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/merge_requests/102 (https://phabricator.wikimedia.org/T443234) [19:58:23] (03approved) 10raymond-ndibe: components-api: keep warnings out of the generated config [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1508 (https://phabricator.wikimedia.org/T444170) [19:58:33] (03merge) 10raymond-ndibe: components-api: keep warnings out of the generated config [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1508 (https://phabricator.wikimedia.org/T444170) [20:02:23] 10Toolforge, 06tools-platform-team, 13Patch-For-Review: [functional tests,components-api] config generate tests load the CLI warnings as config - https://phabricator.wikimedia.org/T444170#12535673 (10Raymond_Ndibe) 05In progress→03Resolved [20:03:14] 10Toolforge, 06tools-platform-team, 13Patch-For-Review: [cicd] auto-create the monthly dependency upgrade task and link the MRs to it - https://phabricator.wikimedia.org/T443234#12535676 (10Raymond_Ndibe) 05In progress→03Resolved [20:31:28] 10Tool-wikinewsie: Extend time windows to 7d on Wikinewsie Front Page - https://phabricator.wikimedia.org/T444223 (10Pharos) 03NEW [20:31:41] 10Tool-wikinewsie: Extend time windows to 7d or longer on Wikinewsie Front Page - https://phabricator.wikimedia.org/T444223#12535801 (10Pharos) [20:43:55] 10Tool-wikinewsie: Wikinewsie history page for each article card - https://phabricator.wikimedia.org/T444224 (10Pharos) 03NEW [20:46:59] 10Tool-wikinewsie: Wikinewsie feeds should let you explore the long tail - https://phabricator.wikimedia.org/T440617#12535857 (10Pharos) >>! In T440617#12496724, @Sj wrote: > a quick classifier of new pages that look like news events, with a cross-check for being an orphan or having only one editor, would also b... [20:50:37] (03update) 10danyya: Draft: Resolve T434400 "Dumps ingester crashes because of too many prepared statements (2)" [toolforge-repos/wiki-gender-stats] (develop) - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-gender-stats/-/merge_requests/109 [20:56:00] FIRING: [3x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [21:14:33] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39 [21:14:45] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48 [21:14:57] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46 [21:15:57] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76 [21:16:00] FIRING: [2x] OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [21:17:03] (03update) 10renovatebot: Update dependency jsdom to v31 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/68 [21:18:04] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65 [21:18:07] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65 [21:26:52] (03update) 10danyya: Draft: Resolve T434400 "Dumps ingester crashes because of too many prepared statements (2)" [toolforge-repos/wiki-gender-stats] (develop) - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-gender-stats/-/merge_requests/109 [22:11:00] RESOLVED: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [22:13:51] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/namehistory] - 10https://gitlab.wikimedia.org/toolforge-repos/namehistory/-/merge_requests/39 [22:13:57] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/allblocksever] - 10https://gitlab.wikimedia.org/toolforge-repos/allblocksever/-/merge_requests/48 [22:14:04] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/checkusertools] - 10https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/merge_requests/46 [22:14:56] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/rangetree] - 10https://gitlab.wikimedia.org/toolforge-repos/rangetree/-/merge_requests/76 [22:15:42] (03update) 10renovatebot: Update dependency jsdom to v31 [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/68 [22:15:43] (03update) 10renovatebot: Lock file maintenance [toolforge-repos/wiki-mail-verify] - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-mail-verify/-/merge_requests/65 [22:29:57] (03update) 10danyya: Draft: Resolve T434400 "Dumps ingester crashes because of too many prepared statements (2)" [toolforge-repos/wiki-gender-stats] (develop) - 10https://gitlab.wikimedia.org/toolforge-repos/wiki-gender-stats/-/merge_requests/109 [22:44:03] !log tools.cluebot3 Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42714216923 (https://github.com/cluebotng/component-configs/commits/162553adae765260d7a2ab19fba838f5c2818b17) [22:44:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebot3/SAL [22:45:06] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/42714216808 (https://github.com/cluebotng/component-configs/commits/162553adae765260d7a2ab19fba838f5c2818b17) [22:45:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL