[00:04:16] (03update) 10addbot: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2024-08-12-115310 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/577 [00:04:17] (03open) 10addbot: Bump postgres from 13.15 to 13.16 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/576 [00:04:20] (03open) 10addbot: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2024-08-12-115310 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/577 [04:08:08] 10GitLab: GitLab Private Repository Request for: Wiki.NYC-OWMF-config - https://phabricator.wikimedia.org/T371710#10065804 (10Tgr) IMO a `wikispore` namespace would be nice if it's not too much work. (It seems Cloud VPS projects have no supergroup like `toolforge-repos` for Toolforge projects.) [07:37:23] hashar: o/ whenever you get a minute, please help merge: https://gerrit.wikimedia.org/r/1062701 [07:37:24] thanks in advance [07:41:50] (03PS3) 10Arthur taylor: Revert^2 "Add Wikibase client-only jobs to EntitySchema checks" [integration/config] - 10https://gerrit.wikimedia.org/r/1062682 (https://phabricator.wikimedia.org/T367156) [07:42:20] (03CR) 10Arthur taylor: Revert^2 "Add Wikibase client-only jobs to EntitySchema checks" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1062682 (https://phabricator.wikimedia.org/T367156) (owner: 10Arthur taylor) [08:16:43] 10Release-Engineering-Team (Radar), 06collaboration-services, 10Data Pipelines, 06Data-Engineering, and 2 others: Upgrade Airflow to 2.9.3 - https://phabricator.wikimedia.org/T365449#10066074 (10Stevemunene) 05Open→03Resolved All hosts are currently on v2.9.3 now so I am marking this as resolved. {... [09:39:19] 10Phabricator: "Call to phutil_nonempty_string() expected null or a string, got: bool" rendering alt text of image file - https://phabricator.wikimedia.org/T352170#10066239 (10Aklapper) 05Open→03Declined Never seen again. Please reopen if this still happens. [09:42:22] 10GitLab: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538 (10TheDJ) 03NEW [10:22:02] 10GitLab, 10VPS-project-devtools: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538#10066439 (10Bugreporter) Also, we need a notice in that tool to indicate this is only a test instance, and provide a link to official Wikimedia... [10:55:43] 10Phabricator (Upstream), 07Upstream, 07Voice & Tone: Ambiguous user-visible string "owner" mashes concepts of Assignee and Author - https://phabricator.wikimedia.org/T279827#10066513 (10Aklapper) [11:00:52] 10Phabricator (Upstream), 07Upstream, 07Voice & Tone: Ambiguous user-visible string "owner" mashes concepts of Assignee and Author - https://phabricator.wikimedia.org/T279827#10066564 (10Aklapper) [11:04:58] (03open) 10samtar: Draft: Add CommunityRequests to wmf_core [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/106 (https://phabricator.wikimedia.org/T372527) [13:48:18] 10Deployments, 10logspam-watch: Make logspam-watch available on deployment.eqiad.wmnet - https://phabricator.wikimedia.org/T372563 (10Lucas_Werkmeister_WMDE) 03NEW [13:49:18] 10Deployments, 10logspam-watch: Make logspam-watch available on deployment.eqiad.wmnet - https://phabricator.wikimedia.org/T372563#10066920 (10Lucas_Werkmeister_WMDE) [14:14:27] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: (some) Gitlab builds hanging - https://phabricator.wikimedia.org/T371620#10066974 (10thcipriani) [14:21:10] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: (some) Gitlab builds hanging - https://phabricator.wikimedia.org/T371620#10067002 (10LSobanski) Related: {T372510} [14:41:04] 10GitLab: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568 (10mszabo) 03NEW [14:41:15] 10GitLab: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568#10067100 (10mszabo) https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/jobs/342362 is an example pipeline failure. Trying to pull the image locally reveals it's getting a 500: ` $ podm... [14:43:49] 10GitLab, 06Release-Engineering-Team: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568#10067115 (10kostajh) [14:49:15] 10GitLab, 06Release-Engineering-Team: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568#10067134 (10thcipriani) Can confirm, here's what I see in the logs: ` time="2024-08-15T14:40:45.958387552Z" level=error msg="response completed with error" err.code=... [15:22:06] 10GitLab, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538#10067210 (10brennen) > Have a robots.txt that makes it non-indexable Reasonable. I do... [15:22:12] 10GitLab, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538#10067212 (10brennen) [16:02:41] 10GitLab, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538#10067315 (10brennen) Here we go: https://docs.gitlab.com/omnibus/settings/nginx.html#... [16:34:04] 10Deployments, 06Release-Engineering-Team, 10logspam-watch: Make logspam-watch available on deployment.eqiad.wmnet - https://phabricator.wikimedia.org/T372563#10067401 (10brennen) Thanks for the detailed problem statement. I haven't found it //too// painful to shell into the mwlog box separately, but I think... [16:44:38] (03open) 10dduvall: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) [16:44:39] (03update) 10dduvall: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) [16:45:28] 10GitLab, 06Release-Engineering-Team: Enable Extensions Marketplace on GitLab Web IDE - https://phabricator.wikimedia.org/T372314#10067451 (10brennen) > I especially missed the vim mode Understandable. :) I've got basically zero experience of vscode or its marketplace. I'm guessing this would be fine, but we... [16:46:31] (03update) 10dduvall: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) [16:46:37] (03update) 10dduvall: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) [16:46:50] (03approved) 10jnuche: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [16:46:58] (03merge) 10dduvall: ci: Support TF_PLAN_APPLY_ARGS variable for additional arguments [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/397 (https://phabricator.wikimedia.org/T372568) [17:17:34] (03open) 10dduvall: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) [17:17:35] (03update) 10dduvall: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) [17:17:43] (03update) 10dduvall: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) [17:17:50] (03update) 10dduvall: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) [17:20:36] (03approved) 10jnuche: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [17:20:49] (03merge) 10dduvall: ci: Support TF_FORCE_UNLOCK variable [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/398 (https://phabricator.wikimedia.org/T372568) [17:30:27] (03open) 10dduvall: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) [17:30:28] (03update) 10dduvall: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) [17:30:35] (03update) 10dduvall: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) [17:30:40] (03update) 10dduvall: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) [17:31:16] (03approved) 10jnuche: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [17:31:29] (03merge) 10dduvall: ci: Fix the force-unlock job [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/399 (https://phabricator.wikimedia.org/T372568) [17:38:51] (03open) 10dduvall: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) [17:38:51] (03update) 10dduvall: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) [17:39:07] (03update) 10dduvall: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) [17:39:11] (03update) 10dduvall: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) [17:39:47] (03approved) 10jnuche: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [17:39:57] (03merge) 10dduvall: ci: Pass `-force` to bypass `force-unlock` confirmation [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/400 (https://phabricator.wikimedia.org/T372568) [18:01:48] (03open) 10dduvall: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) [18:01:49] (03update) 10dduvall: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) [18:01:57] (03update) 10dduvall: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) [18:02:02] (03update) 10dduvall: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) [18:03:45] (03approved) 10jnuche: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [18:04:15] (03merge) 10dduvall: ci: Split force-unlock job to work with both production and staging [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/401 (https://phabricator.wikimedia.org/T372568) [18:05:18] dduvall: is there a way I can try out a blubber + kokkuri idea without first getting my hands on a GitLab repo that is authorized to use trusted runners? I want to mess with some beta cluster ideas, but I'm not sure that they will actually go anywhere useful yet. [18:05:55] bd808: kokkuri+blubber work in DO yeah [18:07:46] cool! I will attempt to make my thing and see how far I can get then. :) [18:09:21] I basically want to build a thing like https://gitlab.wikimedia.org/repos/releng/gitlab-terraform-images/-/blob/wmf/stable/.pipeline/blubber.yaml except using bookworm + tofu and then use it like https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/blob/main/.gitlab-ci.yml [18:12:02] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568#10067600 (10thcipriani) p:05Triage→03High [18:15:27] (03open) 10dduvall: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) [18:15:27] (03update) 10dduvall: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) [18:15:40] (03update) 10dduvall: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) [18:15:45] (03update) 10dduvall: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) [18:16:01] (03approved) 10jnuche: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) (owner: 10dduvall) [18:16:35] (03merge) 10dduvall: ci: Fix force-unlock-production job name [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/402 (https://phabricator.wikimedia.org/T372568) [19:13:21] (03open) 10dduvall: README.md: Document forced resource replacement and unlocks [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/403 (https://phabricator.wikimedia.org/T372568) [19:13:22] (03update) 10dduvall: README.md: Document forced resource replacement and unlocks [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/403 (https://phabricator.wikimedia.org/T372568) [19:13:54] (03update) 10dduvall: README.md: Document forced resource replacement and unlocks [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/403 (https://phabricator.wikimedia.org/T372568) [19:14:01] (03update) 10dduvall: README.md: Document forced resource replacement and unlocks [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/403 (https://phabricator.wikimedia.org/T372568) [19:17:02] (03update) 10dduvall: README.md: Document forced resource replacement and unlocks [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/403 (https://phabricator.wikimedia.org/T372568) [19:43:29] 10GitLab, 10Release-Engineering-Team (Priority Backlog 📥), 13Patch-For-Review: Mirrored Docker Hub images are not working in GitLab pipelines - https://phabricator.wikimedia.org/T372568#10067819 (10thcipriani) 05Open→03Resolved a:03dduvall Should be fixed now and I see successful reads and writes i... [20:49:33] 10GitLab: GitLab Private Repository Request for: Wiki.NYC-OWMF-config - https://phabricator.wikimedia.org/T371710#10068048 (10bd808) >>! In T371710#10065804, @Tgr wrote: > IMO a `wikispore` namespace would be nice if it's not too much work. (It seems Cloud VPS projects have no supergroup like `toolforge-repos` f... [22:47:15] 10GitLab: GitLab Private Repository Request for: Wiki.NYC-OWMF-config - https://phabricator.wikimedia.org/T371710#10068268 (10Tgr) Thanks, I see now there is https://gitlab.wikimedia.org/explore/groups to list namespaces. So maybe https://gitlab.wikimedia.org/cloudvps-repos/wikispore/Wiki.NYC-OWMF-config then? [23:00:07] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T366963#10068290 (10thcipriani) 05Open→03Resolved [23:16:19] 10GitLab: GitLab Private Repository Request for: Wiki.NYC-OWMF-config - https://phabricator.wikimedia.org/T371710#10068309 (10thcipriani) 05Open→03Resolved >>! In T371710#10068268, @Tgr wrote: > Thanks, I see now there is https://gitlab.wikimedia.org/explore/groups to list namespaces. > > So maybe https... [23:29:42] 10GitLab, 06Release-Engineering-Team: Create new GitLab project group: sprinthackular2024 - https://phabricator.wikimedia.org/T372443#10068318 (10NBaca-WMF) >>! In T372443#10063167, @bd808 wrote: >> to contribute, limit to members of the #sprinthackular2024-all slack channel > > There is no connection between... [23:44:13] 10Deployments, 10Release-Engineering-Team (Priority Backlog 📥), 07Kubernetes: Build and publish multiple MediaWiki production images for a given set of PHP versions - https://phabricator.wikimedia.org/T370934#10068322 (10bd808) [23:47:38] 10GitLab, 06Release-Engineering-Team: Create new GitLab project group: sprinthackular2024 - https://phabricator.wikimedia.org/T372443#10068326 (10thcipriani) 05Open→03Invalid > We can close this request then and use topics for this purpose 👍