[06:33:31] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12446580 (10hashar) [06:33:46] 10Gerrit: Unable to git pull from gerrit - error 403 - https://phabricator.wikimedia.org/T441578#12446582 (10hashar) [06:33:54] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12446581 (10hashar) [06:48:03] 06Release-Engineering-Team, 06Collaboration-Services: Update sync-gitlab-group-with-ldap to handle syncing users who have no email address - https://phabricator.wikimedia.org/T414428#12446616 (10ABran-WMF) [06:54:33] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592 (10Count_Count) 03NEW [06:57:24] (03update) 10addshore: Draft: Secrets [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/725 [07:04:42] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12446654 (10Count_Count) Uploading artifacts from jobs also stopped working, see e.g. https://gitlab.wikimedia.org/toolforge-repos/checkusertools/-/jobs/1004449 (HTTP 500) [07:05:03] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12446655 (10Count_Count) [07:23:33] hey! would it be possible for someone to do some cleanup in the artifact bucket in Gitlab? the repo repos/releng/cli uses 81GB of our 350GB quota, we're currently unable to run CI jobs because it is at capacity [07:33:57] Oh it was me that broke it! I was just about to write a phab ticketr [07:34:04] that might also mean I have a button to fix it [07:35:08] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12446720 (10Addshore) I was also just about to create this ticket. https://gitlab.wikimedia.org/repos/releng/cli/-/jobs/1004424 ` Testing merged commit 5d6f45b14acb2ef0df8578904ef4e0055e6d68bd Uploading artifacts f... [07:39:08] (03open) 10phedenskog: Document the plan for Zuul data in develstats [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/101 [07:41:16] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12446724 (10Addshore) for the cli repo, Total artifacts size 71.22 GiB Total artifacts count 41 So with any luck things might start ticking along again? I'll continue cleanup (and likely make some of these artifacts... [07:43:47] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12446736 (10ArthurTaylor) a:05ArthurTaylor→03None [08:03:39] (03CR) 10Hashar: [C:03+2] "Awesome, thank you for the ping!" [integration/config] - 10https://gerrit.wikimedia.org/r/1356332 (owner: 10Phuedx) [08:05:00] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/ReaderExperiments] Add TestKitchen dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1356332 (owner: 10Phuedx) [08:05:33] 10GitLab (Project and group requests), 06Release-Engineering-Team: Create new GitLab project group: media-platform-team - https://phabricator.wikimedia.org/T441601 (10Jgiannelos) 03NEW [08:07:18] (03CR) 10Hashar: [C:03+2] zuul: [mediawiki/extensions/ReadingLists] Add CentralAuth for phan [integration/config] - 10https://gerrit.wikimedia.org/r/1356620 (owner: 10Umherirrender) [08:08:31] (03Merged) 10jenkins-bot: zuul: [mediawiki/extensions/ReadingLists] Add CentralAuth for phan [integration/config] - 10https://gerrit.wikimedia.org/r/1356620 (owner: 10Umherirrender) [08:21:44] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12446889 (10Addshore) I get `Something went wrong while deleting. Please refresh the page to try again.` lots while trying to cleanup the artifacts, so not sure how much more progress I can make beyond the 10GiB I ha... [08:25:14] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12446902 (10ABran-WMF) [08:26:27] neat :) I'll add you on the ticket ! [08:29:23] 06Release-Engineering-Team: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604 (10ABran-WMF) 03NEW [08:29:52] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12446929 (10Addshore) [08:33:06] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12446945 (10Addshore) >>! In T441592#12446889, @Addshore wrote: > I get `Something went wrong while deleting. Please refresh the page to try again.` lots while trying to cleanup the artifacts,... [09:00:03] (03update) 10phedenskog: Document the plan for Zuul data in develstats [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/101 [09:37:07] 10Phabricator, 10Library-Card-Platform, 06Moderator-Tools-Team, 10The-Wikipedia-Library: Move all tasks from Library-Card-Platform to The-Wikipedia-Library - https://phabricator.wikimedia.org/T441640 (10Samwalton9-WMF) 03NEW [09:43:12] 10GitLab (Project and group requests), 06Release-Engineering-Team: GitLab Private Repository Request for: product-safety-and-integrity/vulnhunt-record - https://phabricator.wikimedia.org/T441641 (10kostajh) 03NEW [09:44:08] 10Continuous-Integration-Config: phpunit-coverage-php83-publish always fails due to pcov not being available - https://phabricator.wikimedia.org/T441467#12447655 (10hashar) 05Open→03Resolved a:03hashar I am marking this one fixed, the immediate action has been to rollback Ie9fd840b3a8b81a59f8f5f1f90bc2... [09:46:57] (03PS1) 10Hashar: jjb: enable pcov by default in quibble-coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1356809 (https://phabricator.wikimedia.org/T420009) [09:52:42] 10Phabricator, 06Collaboration-Services, 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)), 13Patch-For-Review: data lake: phabricator events - https://phabricator.wikimedia.org/T434047#12447698 (10Antoine_Quhen) @Addshore Here are some code to review: * [[ https://gitlab.wikimedia.org/repos/dat... [10:30:16] (03update) 10addshore: Support custom development services in dashboard [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/674 (https://phabricator.wikimedia.org/T412006) [10:32:18] (03CR) 10Hashar: "For the `phpunit-coverage-{php}-publish` jobs, the `pcov` extension has previously been disabled by Timo. T395743 / 0410f4f175957c378982c" [integration/config] - 10https://gerrit.wikimedia.org/r/1356809 (https://phabricator.wikimedia.org/T420009) (owner: 10Hashar) [10:34:41] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12447871 (10ABran-WMF) thanks @Vgutierrez for the review and the rollout of [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/1351947 | this change... [10:39:09] (03PS1) 10Mszwarc: Zuul: [mediawiki/extensions/CheckUser] Add WikimediaAntiAbuse dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1356818 (https://phabricator.wikimedia.org/T440871) [10:46:30] 10Beta-Cluster-Infrastructure, 06MediaWiki-Engineering, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393#12447956 (10Raine) >>! In T435393#12442942, @So... [11:39:12] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12448136 (10Addshore) a:03Addshore [11:42:45] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12448159 (10Addshore) 05Open→03Resolved Wrote a little script, https://phabricator.wikimedia.org/P96796 that looks at the last 48h But I ended up running it with a 100MiB limit rather t... [11:56:49] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12448239 (10ABran-WMF) ` arnaudb@gitlab1004:~ $ sudo gitlab-psql -c "SELECT r.path AS project, pg_size_pretty(ps.packages_size) AS packages, pg_size_pretty(ps.build_artifacts_size) AS artif... [12:19:21] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12448333 (10ABran-WMF) [12:24:12] 10GitLab, 06Release-Engineering-Team (Radar), 06Collaboration-Services, 13Patch-For-Review: gitlab behind CDN - https://phabricator.wikimedia.org/T425441#12448383 (10ABran-WMF) [13:43:28] 10GitLab (Infrastructure), 06Collaboration-Services: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678 (10ABran-WMF) 03NEW [13:43:41] 06Release-Engineering-Team, 10a Wikimedia CLI: Cleanup CLI repo artifacts - https://phabricator.wikimedia.org/T441604#12448903 (10ABran-WMF) [13:43:42] 10GitLab (Infrastructure), 06Collaboration-Services: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12448902 (10ABran-WMF) [13:43:56] 10GitLab (Infrastructure), 06Collaboration-Services, 06Data-Persistence: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12448905 (10ABran-WMF) [13:45:02] 10GitLab (Infrastructure), 06Collaboration-Services, 06Data-Persistence: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12448938 (10ABran-WMF) [13:46:56] 10GitLab (Infrastructure), 10Ceph, 06Collaboration-Services, 06Data-Persistence, 10SRE-swift-storage: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12448945 (10MatthewVernon) I increased storage from 350G to 400G this morning; is that sufficient? [13:47:57] 10GitLab (Infrastructure), 10Ceph, 06Collaboration-Services, 06Data-Persistence, 10SRE-swift-storage: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12448970 (10ABran-WMF) as long as we address {https://phabricator.wikimedia.org/T316553} and/or {https://phabricator.wikim... [13:49:57] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12448999 (10Addshore) @Count_Count resolved for you? [13:52:28] 10Gerrit, 07Upstream: Gerrit Zuul plugin style mismatches related changes styling - https://phabricator.wikimedia.org/T441684 (10hashar) 03NEW [13:59:07] 10Continuous-Integration-Config: phpunit-coverage-php83-publish always fails due to pcov not being available - https://phabricator.wikimedia.org/T441467#12449060 (10Daimona) >>! In T441467#12442940, @hashar wrote: > That enables `pcov` in that container. At the end of the execution the container is dropped... [13:59:21] (03CR) 10Daimona Eaytoy: [C:03+1] jjb: enable pcov by default in quibble-coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1356809 (https://phabricator.wikimedia.org/T420009) (owner: 10Hashar) [14:04:52] (03CR) 10Daimona Eaytoy: [C:03+1] "> If we want to solve T395743 with built-in phpunit validation for `@covers`, despite upstream PHPUnit declining this use case, I guess we" [integration/config] - 10https://gerrit.wikimedia.org/r/1356809 (https://phabricator.wikimedia.org/T420009) (owner: 10Hashar) [14:06:51] (03CR) 10Daimona Eaytoy: [C:03+1] "(And to further clarify: I think when we previously had pcov enabled by default, we might also have had `pcov.enabled=1` by default, which" [integration/config] - 10https://gerrit.wikimedia.org/r/1356809 (https://phabricator.wikimedia.org/T420009) (owner: 10Hashar) [14:25:48] (03merge) 10phedenskog: Document the plan for Zuul data in develstats [repos/releng/develstats] - 10https://gitlab.wikimedia.org/repos/releng/develstats/-/merge_requests/101 [14:51:43] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06Test Platform, 07Epic: Implement CI dashboards (https://releng-data.wmcloud.org/-/dashboards) for the new Zuul setup - https://phabricator.wikimedia.org/T441692 (10Peter) 03NEW [14:59:03] Oh my, there may soon be a day where I can't complain about CI anymore [14:59:10] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1353208 [14:59:18] CI came back in 7min [14:59:41] used to be 15-20min [15:00:19] I can actually revise a patch and continue a conversation with a person about that patch, such that both of us are still able to remember what it was about and haven't goten distracted with something else yet [15:00:51] (this was a new revision so no result cache at play. it's a real run.) [15:01:42] Peter_W: phedenskog: nice work :) [15:02:38] 10GitLab (CI & Job Runners), 06Release-Engineering-Team: registry.cloud.releng.team returns 500 responses when pushing layers - https://phabricator.wikimedia.org/T427250#12449492 (10dancy) 05Open→03Resolved [15:17:35] 10WikimediaDebug, 07Essential-Work, 05MW-1.47-notes (1.47.0-wmf.23; 2026-10-06), 10Test Kitchen (Experiment Platform Sprint 31): Add Test Kitchen functionality to WikimediaDebug - https://phabricator.wikimedia.org/T437240#12449609 (10JVanderhoop-WMF) 05Open→03Resolved [15:17:44] 06Release-Engineering-Team (Radar), 10WikimediaDebug, 07Essential-Work, 10Test Kitchen (Experiment Platform Sprint 31): Update WikimediaDebug to fetch active experiment config and allow selection of experiments and groups - https://phabricator.wikimedia.org/T438283#12449614 (10JVanderhoop-WMF) 05Open... [15:21:40] (03update) 10dancy: deploy: Do not write deploy server info into DEPLOY_HEAD [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1357 (https://phabricator.wikimedia.org/T197470) [15:21:43] (03update) 10dancy: deploy: Do not write deploy server info into DEPLOY_HEAD [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1357 (https://phabricator.wikimedia.org/T197470) [15:23:43] (03CR) 10Phedenskog: [C:03+2] zuul: try integration-cache-proxy for UploadWizard's npm test job [integration/config] - 10https://gerrit.wikimedia.org/r/1353720 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [15:24:55] (03Merged) 10jenkins-bot: zuul: try integration-cache-proxy for UploadWizard's npm test job [integration/config] - 10https://gerrit.wikimedia.org/r/1353720 (https://phabricator.wikimedia.org/T439116) (owner: 10Mhurd) [15:25:14] (03merge) 10dancy: deploy: Do not write deploy server info into DEPLOY_HEAD [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1357 (https://phabricator.wikimedia.org/T197470) [15:27:48] 10Phabricator-Bot-Requests, 06Release-Engineering-Team (Doing 😎), 06Data-Engineering (Q2 FS26/27 October 1st - December 31th)): Create a Phabricator bot account for Data Engineering dev-metrics (developer-activity-pipelines) - https://phabricator.wikimedia.org/T441326#12449709 (10Antoine_Quhen) [15:43:23] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/1353720 [15:43:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:45:17] 10GitLab (Infrastructure), 10Ceph, 06Collaboration-Services, 06Data-Persistence, 10SRE-swift-storage: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12449951 (10LSobanski) p:05Triage→03Medium [15:46:48] 10GitLab (Infrastructure), 10Ceph, 06Collaboration-Services, 06Data-Persistence, 10SRE-swift-storage: Gitlab's storage quota needs a raise - https://phabricator.wikimedia.org/T441678#12449983 (10LSobanski) 05Open→03Resolved a:03MatthewVernon [15:47:51] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12449992 (10LSobanski) 05Open→03Resolved a:03ABran-WMF Resolving, please reopen or create a new task if this re... [15:52:54] 10Gerrit, 06Collaboration-Services, 07Upstream: Gerrit deployment initialize the site generating ssh key pairs - https://phabricator.wikimedia.org/T441113#12450049 (10Dzahn) a:03Dzahn [15:57:08] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12450091 (10Count_Count) >>! In T441592#12448999, @Addshore wrote: > @Count_Count resolved for you? Yes, scheduled pipelines are running again. [15:57:24] 10GitLab: Gitlab scheduled pipelines stopped running - https://phabricator.wikimedia.org/T441592#12450095 (10Count_Count) 05Open→03Resolved [15:58:33] 10Scap: scap backport/scap deploy-promote: Offer to revert changes - https://phabricator.wikimedia.org/T440666#12450098 (10dancy) 05Open→03Resolved Deployed. [16:09:59] (03PS1) 10Hashar: Point Zuul plugin to stable-3.10 and update it [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1356923 (https://phabricator.wikimedia.org/T441684) [16:10:05] (03CR) 10CI reject: [V:04-1] Point Zuul plugin to stable-3.10 and update it [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1356923 (https://phabricator.wikimedia.org/T441684) (owner: 10Hashar) [16:13:02] (03PS2) 10Hashar: Point Zuul plugin to stable-3.10 and update it [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1356923 (https://phabricator.wikimedia.org/T441684) [16:29:46] (03update) 10dancy: spiderpig: Export Prometheus metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1356 [16:31:11] (03update) 10dancy: spiderpig: Export Prometheus metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1356 [16:31:59] (03update) 10dancy: spiderpig: Export Prometheus metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1356 [16:34:09] (03merge) 10dancy: spiderpig: Export Prometheus metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1356 [16:36:25] (03open) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [16:36:25] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [16:44:34] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [16:45:33] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [16:45:41] (03update) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [16:45:46] (03open) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [16:45:48] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [16:45:49] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [16:45:53] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [16:46:51] (03open) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [16:58:32] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [16:58:39] (03update) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [16:58:39] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [16:59:02] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [17:07:16] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [17:09:03] 10Beta-Cluster-Infrastructure, 10Wikidata, 10Wikidata Lexicographical data, 06Wikidata-Omega: OutOfBoundsException: Attempt to get non-existing setting "tmpEnableMobileEditingUIBetaFeature" (Beta Commons down) - https://phabricator.wikimedia.org/T441718 (10Krinkle) 03NEW [17:09:41] 06Release-Engineering-Team (Priority Backlog 📥), 07Essential-Work, 05Release, 05Train Deployments: 1.48.0-wmf.1 deployment blockers - https://phabricator.wikimedia.org/T438222#12450561 (10Krinkle) [17:09:42] 10Beta-Cluster-Infrastructure, 10Wikidata, 10Wikidata Lexicographical data, 06Wikidata-Omega: OutOfBoundsException: Attempt to get non-existing setting "tmpEnableMobileEditingUIBetaFeature" (Beta Commons down) - https://phabricator.wikimedia.org/T441718#12450560 (10Krinkle) [17:12:30] 10Beta-Cluster-Infrastructure, 10Wikidata, 10Wikidata Lexicographical data, 06Wikidata-Omega: OutOfBoundsException: Attempt to get non-existing setting "tmpEnableMobileEditingUIBetaFeature" (Beta Commons down) - https://phabricator.wikimedia.org/T441718#12450574 (10Krinkle) [17:38:29] (03PS1) 10Subramanya Sastry: Set up user-agent with email id to ensure we aren't throttled [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356957 [17:38:29] (03PS1) 10Subramanya Sastry: differ.js: Ensure headers are always set [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356958 [17:38:29] (03PS1) 10Subramanya Sastry: Add a new config for php8.3 vs php8.5 testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356959 [17:39:02] (03CR) 10CI reject: [V:04-1] Add a new config for php8.3 vs php8.5 testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356959 (owner: 10Subramanya Sastry) [17:39:48] (03PS2) 10Subramanya Sastry: Add a new config for php8.3 vs php8.5 testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356959 [17:44:23] (03PS3) 10Subramanya Sastry: Add a new config for php8.3 vs php8.5 testing [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1356959 (https://phabricator.wikimedia.org/T440321) [18:03:02] 06Release-Engineering-Team (Doing 😎), 06WMF-NDA-Requests: Requesting access to WMF-NDA for Samwalton9-WMF - https://phabricator.wikimedia.org/T441714#12450782 (10Aklapper) 05Open→03Resolved a:03Aklapper Done. Papertrail: https://phabricator.wikimedia.org/p/Samwalton9-WMF/ is linked to https://meta.wi... [18:23:40] 10GitLab (Project and group requests), 06Release-Engineering-Team, 07Essential-Work: GitLab Private Repository Request for: product-safety-and-integrity/wmf-security-skills - https://phabricator.wikimedia.org/T441301#12450839 (10thcipriani) 05Open→03Resolved a:03thcipriani Done: https://gitlab.wiki... [18:26:16] 10GitLab (Project and group requests), 06Release-Engineering-Team, 07Essential-Work: GitLab Private Repository Request for: product-safety-and-integrity/vulnhunt-record - https://phabricator.wikimedia.org/T441641#12450856 (10thcipriani) 05Open→03Resolved a:03thcipriani Done! [18:41:38] 10Beta-Cluster-Infrastructure, 06MediaWiki-Engineering, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393#12450885 (10Southparkfan) >>! In T435393#124479... [18:51:29] 10Beta-Cluster-Infrastructure, 06MediaWiki-Engineering, 10Cloud-VPS (Debian Bullseye Deprecation), 13Patch-For-Review: Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393#12450909 (10Southparkfan) 05In progress→... [19:18:59] (03CR) 10Hashar: [C:03+2] Point Zuul plugin to stable-3.10 and update it [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1356923 (https://phabricator.wikimedia.org/T441684) (owner: 10Hashar) [19:26:50] 10Gerrit, 06Collaboration-Services, 07Upstream: Gerrit deployment initialize the site generating ssh key pairs - https://phabricator.wikimedia.org/T441113#12451084 (10hashar) >>! In T441113#12430201, @Paladox wrote: > There's this https://gerrit-review.googlesource.com/c/gerrit/+/643250 to resolve this Got... [19:28:03] (03Merged) 10jenkins-bot: Point Zuul plugin to stable-3.10 and update it [software/gerrit] (wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1356923 (https://phabricator.wikimedia.org/T441684) (owner: 10Hashar) [19:38:06] 10GitLab (Project and group requests), 06Release-Engineering-Team: Create new GitLab project group: media-platform-team - https://phabricator.wikimedia.org/T441601#12451198 (10brennen) a:03brennen Sure, one sec... [19:42:39] 10GitLab (Project and group requests), 06Release-Engineering-Team, 07Essential-Work: Create new GitLab project group: media-platform-team - https://phabricator.wikimedia.org/T441601#12451232 (10brennen) 05Open→03Resolved Created: https://gitlab.wikimedia.org/repos/media-platform-team Added you both... [19:44:39] 06Release-Engineering-Team, 06Collaboration-Services, 06NFDI-Germany, 10Wikidata, and 3 others: Github actions 403 errors cloning from Gerrit - https://phabricator.wikimedia.org/T441387#12451253 (10hashar) 05Resolved→03Open I found a new one which is LFS being blocked and git-lfs does not support setti... [19:47:38] 10GitLab (Project and group requests), 06Release-Engineering-Team, 06App Growth (AppG Sprint [FY2627 Oct 14 - Oct 28]), 07Essential-Work: Finding right group for microsite project in GitLab - https://phabricator.wikimedia.org/T439904#12451274 (10ABendall-WMF) [19:56:03] (03open) 10dancy: Release 4.301.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1368 [19:59:29] (03merge) 10dancy: Release 4.301.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1368 [20:05:22] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451336 (10sbassett) 05Open→03In progress p:05Triage→03Medium... [20:05:42] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451342 (10sbassett) [20:12:18] (03open) 10dancy: requirements.txt: Bump fastapi and starlette on Python 3.10 and newer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1369 [20:13:31] (03update) 10dancy: requirements.txt: Bump fastapi and starlette on Python 3.10 and newer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1369 [20:13:53] (03update) 10dancy: requirements.txt: Bump fastapi and starlette on Python 3.10 and newer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1369 [20:16:13] (03merge) 10dancy: requirements.txt: Bump fastapi and starlette on Python 3.10 and newer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1369 [20:18:06] (03update) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [20:18:37] (03update) 10dancy: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 [20:18:38] (03update) 10dancy: scap: Record more deployment metrics [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1366 [20:18:40] (03update) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [20:24:22] (03update) 10dancy: spiderpig: Export the HTTP metrics of the apiserver [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1367 [20:53:42] 10Gerrit, 06Collaboration-Services, 13Patch-For-Review, 07Upstream: Gerrit deployment initialize the site generating ssh key pairs - https://phabricator.wikimedia.org/T441113#12451577 (10Dzahn) I largely agree with the analysis here. It was the right call to delete those unmanaged new keys asap. I confir... [21:10:18] (03open) 10dancy: patches: Comment on the task of an already-applied security patch [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1370 (https://phabricator.wikimedia.org/T439619) [21:18:31] (03update) 10dancy: patches: Comment on the task of an already-applied security patch [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1370 (https://phabricator.wikimedia.org/T439619) [21:18:34] (03update) 10dancy: patches: Comment on the task of an already-applied security patch [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1370 (https://phabricator.wikimedia.org/T439619) [21:48:51] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451824 (10sbassett) [21:51:42] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451841 (10sbassett) [21:52:06] (03approved) 10thcipriani: scap: Record who ran each command [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1365 (owner: 10dancy) [21:52:56] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451856 (10sbassett) [22:00:32] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451885 (10sbassett) [22:02:08] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451894 (10sbassett) [22:03:26] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451899 (10sbassett) [22:04:17] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451902 (10sbassett) [22:05:21] 06Project-Admins, 06Product Safety and Integrity, 06Security-Team, 07SecTeam-Processed: Create user- boards for members of the Security Team (to primarily track security issue responsibilities) - https://phabricator.wikimedia.org/T441335#12451904 (10sbassett) 05In progress→03Resolved [22:14:52] (03open) 10addshore: Add developer workflow demo recordings [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/726 [22:39:08] (03open) 10addshore: Work on linti best practices for command docs and structure [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/727 [22:42:07] (03open) 10addshore: gateway auth for tunnels only, and don't always re tunnel [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/728 [23:23:22] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12452102 (10Dzahn) Here is some good news: We already have the `hoo... [23:25:55] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12452105 (10Dzahn) Beyond that it seems it first would require the l... [23:28:01] 10Gerrit, 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Develop a means for preventing developers from accidentally pushing embargoed security patches to gerrit - https://phabricator.wikimedia.org/T388247#12452109 (10Dzahn) > series of "security-ish" strings built into a...