[00:00:38] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:05:38] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:30:29] 10Release-Engineering-Team (Priority Backlog 📥), 07Developer Productivity: Add link to gerrit repo settings in description of mirrored github repos - https://phabricator.wikimedia.org/T372791#10076002 (10Krinkle) Poem, this is what Gerrit is currently configured to set during the first push on a new GitHub mir... [00:32:47] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076004 (10Dzahn) - removed both users from Phabricator group WMF-NDA. done. - I have no rights to manage acl*securit... [00:33:29] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076008 (10Dzahn) [00:39:11] What tag do I need to add to my .gitlab-ci.yml job to pin it to the wmcs runners? https://www.mediawiki.org/wiki/GitLab/Workflows/CI doesn't mention the use of tags at all. [00:40:21] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076019 (10Dzahn) [00:40:37] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076020 (10Dzahn) [00:43:18] from https://phabricator.wikimedia.org/rOPUPfcb77ab5f90106de450b1af1f3907522b82b56f6 I think the answer to my question might be 'wmcs'. [01:05:24] 10GitLab, 07Documentation: https://www.mediawiki.org/wiki/GitLab/Workflows/CI doesn't mention using tags to select runners - https://phabricator.wikimedia.org/T372846 (10bd808) 03NEW [01:06:36] 10GitLab, 07Documentation: https://www.mediawiki.org/wiki/GitLab/Workflows/CI doesn't mention using tags to select runners - https://phabricator.wikimedia.org/T372846#10076073 (10bd808) I found lots of related stuff at https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner, but this wikitech doc is theoretic... [01:07:25] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076074 (10Dzahn) [01:09:12] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10076077 (10Dzahn) - removed from Gerrit group wmde-mediawiki @KFrancis Hi, the users should be re... [01:19:42] 10GitLab (CI & Job Runners): Push by kokkuri to registry.cloud.releng.team/bd808/deployment-prep-opentofu failing after working last week - https://phabricator.wikimedia.org/T372835#10076081 (10bd808) 05Open→03Invalid I'm going to close this as invalid because backing up and going forward again made thin... [01:28:31] 10GitLab: Images from registry.cloud.releng.team should be usable by the "wmcs" runners - https://phabricator.wikimedia.org/T372848 (10bd808) 03NEW [01:28:52] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Figure out how to provision a Kubernetes cluster using Magnum and OpenTofu - https://phabricator.wikimedia.org/T372498#10076110 (10bd808) [01:28:53] 10GitLab: Images from registry.cloud.releng.team should be usable by the "wmcs" runners - https://phabricator.wikimedia.org/T372848#10076111 (10bd808) [01:32:33] (03PS1) 10Subramanya Sastry: Add dag to the wikipedias list [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1063925 [02:30:45] 10GitLab, 06Release-Engineering-Team: Create new GitLab project group: Web team - https://phabricator.wikimedia.org/T372854 (10Jdrewniak) 03NEW [05:53:02] 10Phabricator: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857 (10Aklapper) 03NEW [06:25:33] (03CR) 10Jgiannelos: [C:03+2] Add dag to the wikipedias list [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1063925 (owner: 10Subramanya Sastry) [06:26:09] (03Merged) 10jenkins-bot: Add dag to the wikipedias list [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/1063925 (owner: 10Subramanya Sastry) [06:49:54] 10GitLab, 06Release-Engineering-Team, 06collaboration-services, 10VPS-project-devtools, 13Patch-For-Review: https://gitlab.devtools.wmcloud.org is being indexed by google (and scoring pretty high) - https://phabricator.wikimedia.org/T372538#10076347 (10Jelto) 05Open→03Resolved a:03Jelto @dzahn... [09:00:47] 10Phabricator: Spaces request for data-persistence-automation - https://phabricator.wikimedia.org/T372862 (10ABran-WMF) 03NEW [09:02:08] 10Phabricator: Spaces request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076549 (10ABran-WMF) [09:18:12] 10Phabricator: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076646 (10ABran-WMF) [09:24:44] 10Phabricator: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076652 (10ABran-WMF) [10:36:06] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076971 (10Peachey88) [10:38:15] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076975 (10Peachey88) Please provide a description you would like on the created project? [10:46:32] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10076980 (10ABran-WMF) sorry, I removed a bit too much on the description: This is a project to centralize tasks for data-persistence automation needs Is it verbose enough? [10:52:16] 10Phabricator: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#10076989 (10Aklapper) I do not manage to reproduce no matter which language settings I choose, but an easy reproducer is replacing `'d' => "{$start_yea... [10:59:12] 10Phabricator: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#10076992 (10Aklapper) There are two options to deal with this: One is to continue ignore that `DateTime` should not be null, as `catch (Exception $ex)... [11:08:30] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10077000 (10Aklapper) > Editable By: acl*phabricator Why does this need to be so restricted? Is there a related repository, or are there too many? https://gitlab.wikimedia.org/repos/data_persistence maybe? A... [11:08:42] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10077001 (10Aklapper) (Also, why is this assigned to me? :) [11:09:09] 10Phabricator: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#10077002 (10Aklapper) Docs upstream part (unrelated to a bug fix): https://we.phorge.it/D25793 [11:23:17] 10Beta-Cluster-Infrastructure, 10Add-Link, 10Growth-Team (FY2024-25 Q1 Sprint 4): refreshLinkRecommendation script fails in Beta cluster with FileNotFoundError - https://phabricator.wikimedia.org/T370792#10077061 (10Michael) [12:04:04] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 05MW-1.43-release: Configure CI to support REL1_43 - https://phabricator.wikimedia.org/T372317#10077163 (10Atieno) a:03Atieno [12:04:54] (03PS1) 10Atieno: Zuul: Configure the REL1_33 test and gate pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/1064006 (https://phabricator.wikimedia.org/T372317) [12:06:02] (03PS2) 10Atieno: Zuul: Configure the REL1_43 test and gate pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/1064006 (https://phabricator.wikimedia.org/T372317) [12:07:15] (03CR) 10CI reject: [V:04-1] Zuul: Configure the REL1_43 test and gate pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/1064006 (https://phabricator.wikimedia.org/T372317) (owner: 10Atieno) [12:12:57] (03CR) 10Atieno: Introduce extension_survey.py (032 comments) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054308 (owner: 10Daniel Kinzler) [12:13:04] (03CR) 10Atieno: extension_survey: add mode for collection service usage (031 comment) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054478 (owner: 10Daniel Kinzler) [12:17:36] (03CR) 10Daniel Kinzler: Introduce extension_survey.py (032 comments) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054308 (owner: 10Daniel Kinzler) [12:23:34] (03CR) 10Daniel Kinzler: extension_survey: add mode for collection service usage (031 comment) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054478 (owner: 10Daniel Kinzler) [12:24:17] 10Phabricator: Calendar: Call to a member function format() on null at $this->getQueryDateFrom($query)->getDateTime() - https://phabricator.wikimedia.org/T372857#10077232 (10Aklapper) Added debug output in https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/commit/5679ff7c5e6c3b421bcf2e7f7c82d74a61b85278 [12:30:19] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10077287 (10ABran-WMF) a:05Aklapper→03None >>! In T372862#10077000, @Aklapper wrote: > Why does this need to be so restricted? I thought it was one of the less restrictive ones > Is there a related reposit... [12:32:51] 10Continuous-Integration-Config, 10phan-taint-check-plugin: mw-tools-phan-demos-publish fails with: "configure: error: cannot run C compiled programs." - https://phabricator.wikimedia.org/T372887 (10Daimona) 03NEW [12:35:33] 10Beta-Cluster-Infrastructure, 10CheckUser, 10Temporary accounts, 06Trust and Safety Product Team, and 2 others: Install CheckUser in Beta Cluster - https://phabricator.wikimedia.org/T357959#10077300 (10kostajh) [13:17:55] (03CR) 10Atieno: Introduce extension_survey.py (032 comments) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054308 (owner: 10Daniel Kinzler) [13:18:28] (03CR) 10Atieno: extension_survey: add mode for collection service usage (031 comment) [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054478 (owner: 10Daniel Kinzler) [14:00:18] (03update) 10brennen: Exclude Phabricator Spaces from auto-linking [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/50 (https://phabricator.wikimedia.org/T371131) (owner: 10aklapper) [14:00:33] (03merge) 10brennen: Exclude Phabricator Spaces from auto-linking [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/50 (https://phabricator.wikimedia.org/T371131) (owner: 10aklapper) [14:13:08] (03PS1) 10Zfilipin: jjb: Update teams responsible for selenium-daily jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) [14:13:20] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥): Do not autolink shard names (e.g. s1) as Phabricator Spaces - https://phabricator.wikimedia.org/T371131#10077707 (10Aklapper) [14:18:11] (03CR) 10Jforrester: jjb: Update teams responsible for selenium-daily jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) (owner: 10Zfilipin) [14:19:55] (03merge) 10brennen: Create Herald task condition to have Projects include their subprojects [repos/phabricator/phabricator] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/62 (https://phabricator.wikimedia.org/T144041) (owner: 10aklapper) [14:20:44] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-08-20 - https://phabricator.wikimedia.org/T372898 (10brennen) 03NEW [14:23:44] (03open) 10brennen: update submodules for 2024-08-20 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/52 (https://phabricator.wikimedia.org/T372898) [14:29:23] 10Phabricator (phabricator-next): Avoid the need to update Herald exclusion rules everytime a new team sprint project is created - https://phabricator.wikimedia.org/T144041#10077792 (10Aklapper) [14:39:41] (03update) 10brennen: update submodules for 2024-08-20 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/52 (https://phabricator.wikimedia.org/T372898) [14:44:53] 06Release-Engineering-Team, 13Patch-For-Review: Buildkit 0.15.2 released - https://phabricator.wikimedia.org/T372820#10077864 (10dancy) 05In progress→03Resolved [14:54:52] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 [14:59:51] (03merge) 10brennen: update submodules for 2024-08-20 release [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/52 (https://phabricator.wikimedia.org/T372898) [15:03:40] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-08-20 - https://phabricator.wikimedia.org/T372898#10077957 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=f3acd713-6137-4779-817d-b90e7fe6f2ab) set... [15:05:54] 10Phabricator (2024-08-20), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-08-20 - https://phabricator.wikimedia.org/T372898#10077964 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=8a94f45e-b8d8-4716-9170-d3074ce61022) set by je... [15:06:09] 10Phabricator (2024-08-20), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Deploy Phabricator/Phorge 2024-08-20 - https://phabricator.wikimedia.org/T372898#10077966 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=c7c8e174-a6a7-4d24-94ec-d7689589b655) set by je... [15:19:18] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥): Do not autolink shard names (e.g. s1) as Phabricator Spaces - https://phabricator.wikimedia.org/T371131#10078015 (10Aklapper) Resetting milestone; this one didn't behave as we wanted [15:38:32] (03CR) 10Zfilipin: jjb: Update teams responsible for selenium-daily jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) (owner: 10Zfilipin) [15:39:21] (03open) 10brennen: local.json: remarkup.ignored-object-names: escape \d [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/53 (https://phabricator.wikimedia.org/T371131) [15:42:37] (03CR) 10Jforrester: jjb: Update teams responsible for selenium-daily jobs (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1064038 (https://phabricator.wikimedia.org/T372868) (owner: 10Zfilipin) [15:45:12] (03merge) 10brennen: local.json: remarkup.ignored-object-names: escape \d [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/53 (https://phabricator.wikimedia.org/T371131) [15:52:26] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥): Do not autolink shard names (e.g. s1) as Phabricator Spaces - https://phabricator.wikimedia.org/T371131#10078194 (10Aklapper) [16:12:35] 14Release-Engineering-Team (Doing), 10Scap, 06serviceops: RESTBase deployment fails with scap internal error - https://phabricator.wikimedia.org/T294936#10078303 (10dancy) 05Resolved→03Open [16:16:09] (03merge) 10dancy: README.md: Initial notes on handling new releases [repos/releng/buildkit] (wmf/v0.15) - 10https://gitlab.wikimedia.org/repos/releng/buildkit/-/merge_requests/69 [16:17:43] (03CR) 10Atieno: [V:03+2 C:03+2] extension_survey: add mode for collection service usage [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054478 (owner: 10Daniel Kinzler) [16:17:50] (03CR) 10Atieno: [V:03+2 C:03+2] Introduce extension_survey.py [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054308 (owner: 10Daniel Kinzler) [16:29:23] (03Merged) 10jenkins-bot: Introduce extension_survey.py [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054308 (owner: 10Daniel Kinzler) [16:29:23] (03Merged) 10jenkins-bot: extension_survey: add mode for collection service usage [tools/code-utils] - 10https://gerrit.wikimedia.org/r/1054478 (owner: 10Daniel Kinzler) [16:35:06] 10Phabricator (2024-08-20), 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions: Remove misleading warning from security escalation interface - https://phabricator.wikimedia.org/T372335#10078477 (10Aklapper) 05Open→03Resolved Deployed by brennen on 2024-08-20, thus res... [16:35:07] 10Phabricator (2024-08-20), 10Wikimedia-Phabricator-Extensions: Update phabricator-translations for 2024.19 release - https://phabricator.wikimedia.org/T363188#10078468 (10Aklapper) 05Open→03Resolved Deployed by brennen on 2024-08-20, thus resolving. [16:35:37] 10Phabricator (2024-08-20): Avoid the need to update Herald exclusion rules everytime a new team sprint project is created - https://phabricator.wikimedia.org/T144041#10078483 (10Aklapper) 05Open→03Resolved Deployed by brennen on 2024-08-20, thus resolving. [16:35:43] 10Phabricator (2024-08-20): Avoid the need to update Herald exclusion rules everytime a new team sprint project is created - https://phabricator.wikimedia.org/T144041#10078485 (10Aklapper) Followup note to myself: Now try with a first team that wants to be a guinea pig [16:35:53] 10Phabricator (2024-08-20), 10Release-Engineering-Team (Priority Backlog 📥), 07Voice & Tone: Error message when trying to archive a Phabricator project without permissions only talks about creating projects - https://phabricator.wikimedia.org/T256103#10078479 (10Aklapper) 05Stalled→03Resolved Deploye... [16:36:31] 10Diffusion, 10Phabricator (2024-08-20), 10Release-Engineering-Team (Priority Backlog 📥): Set new Diffusion repos URIs to No I/O by default - https://phabricator.wikimedia.org/T224753#10078481 (10Aklapper) 05Open→03Resolved Deployed by brennen on 2024-08-20, thus resolving. [16:36:40] 10Phabricator (2024-08-20), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-08-20 - https://phabricator.wikimedia.org/T372898#10078473 (10Aklapper) 05Open→03Resolved a:03brennen Deployed by brennen on 2024-08-20, thus resolving. [16:50:17] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 10Wikimedia-Apache-configuration, and 3 others: Apache 2.4.61 throws a 403 Forbidden for links containing %3F - https://phabricator.wikimedia.org/T370110#10078534 (10brennen) 05Open→03Resolved a:03brennen [16:55:10] 10Release-Engineering-Team (Radar), 10MediaWiki-Debug-Logger, 10observability, 07Developer Productivity, 10MediaWiki-Platform-Team (Radar): Consider enabling all MW log channels by default for WMF - https://phabricator.wikimedia.org/T228838#10078573 (10Krinkle) [16:55:11] 10Deployments, 10MediaWiki-libs-HTTP: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633#10078575 (10Krinkle) [17:08:48] 14Release-Engineering-Team (Doing), 10Scap, 06serviceops: RESTBase deployment fails with scap internal error - https://phabricator.wikimedia.org/T294936#10078634 (10dancy) 05Open→03Resolved [17:13:03] 06Release-Engineering-Team, 10Scap: RESTBase deployment fails with unhandled error - https://phabricator.wikimedia.org/T372921 (10dancy) 03NEW [17:13:13] 06Release-Engineering-Team, 10Scap: RESTBase deployment fails with unhandled error - https://phabricator.wikimedia.org/T372921#10078653 (10dancy) p:05Triage→03High [17:24:50] (03update) 10dancy: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [17:24:52] (03open) 10dancy: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [17:24:54] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10078697 (10KFrancis) Thanks, @Dzahn, the spreadsheet has been updated. [17:27:27] (03update) 10dancy: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [17:28:34] (03update) 10dancy: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [17:31:03] (03approved) 10dduvall: Fix: Remove TLS certs during `train-dev clean` [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/84 (owner: 10dancy) [17:31:28] (03merge) 10dancy: Fix: Remove TLS certs during `train-dev clean` [repos/releng/train-dev] - 10https://gitlab.wikimedia.org/repos/releng/train-dev/-/merge_requests/84 [18:23:48] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 2 others: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633#10078899 (10Krinkle) >>! @dancy wrote in the **task description**: > ` > CurlFactory:211 cURL error 56: Recv fail... [18:23:49] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 2 others: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633#10078904 (10Krinkle) [18:24:43] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 2 others: Burst of curlFactory cURL errors during mediawiki deployments - https://phabricator.wikimedia.org/T371633#10078905 (10Krinkle) [18:25:20] 10Deployments, 06serviceops, 10Shellbox, 10Wikibase-Quality-Constraints, and 2 others: Burst of GuzzleHttp Exception for http://localhost:6025/call/constraint-regex-checker - https://phabricator.wikimedia.org/T371633#10078911 (10Krinkle) [18:51:14] (03approved) 10thcipriani: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) (owner: 10dancy) [18:51:18] (03merge) 10thcipriani: checks.py: Discard blank checks [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/408 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) (owner: 10dancy) [19:10:36] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [19:11:49] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10079042 (10Dzahn) [19:14:21] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10079043 (10Dzahn) [19:15:45] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10079045 (10Dzahn) The actual LDAP UIDs are **gtzatchkova** (and ALSO: **guergana** but that isn't... [19:15:51] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [19:17:34] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10079050 (10Dzahn) [19:22:47] 06Gerrit-Privilege-Requests, 10LDAP-Access-Requests, 06Security-Team, 10SRE-Access-Requests: Offboard Guergana Tzatchkova (WMDE) and Frederik Ring from WMF systems - https://phabricator.wikimedia.org/T372767#10079066 (10Dzahn) @jhathaway Could you take over from here as part of clinic duty and since you ar... [19:30:06] 06Release-Engineering-Team, 10Scap, 13Patch-For-Review: RESTBase deployment fails with unhandled error when there are empty checks defined - https://phabricator.wikimedia.org/T372921#10079092 (10dancy) [19:48:31] 10GitLab (CI & Job Runners), 13Patch-For-Review: Images from registry.cloud.releng.team should be usable by the "wmcs" runners - https://phabricator.wikimedia.org/T372848#10079142 (10bd808) p:05Triage→03Medium a:03bd808 [19:50:46] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Figure out how to provision a Kubernetes cluster using Magnum and OpenTofu - https://phabricator.wikimedia.org/T372498#10079160 (10bd808) The GitLab CI pipeline at https://gitlab.wikimedia.org/bd808/deployment-prep-opentofu/-/pipelines/71739 shows a gitops aut... [19:52:19] 10GitLab, 06Release-Engineering-Team: Create new GitLab project group: Web team - https://phabricator.wikimedia.org/T372854#10079163 (10brennen) 05Open→03Resolved a:03brennen Created, invited all on the list as owners. [19:52:24] 10GitLab (CI & Job Runners), 07Documentation: https://www.mediawiki.org/wiki/GitLab/Workflows/CI doesn't mention using tags to select runners - https://phabricator.wikimedia.org/T372846#10079168 (10brennen) [20:08:40] 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: cloudvps-repos/deployment-prep - https://phabricator.wikimedia.org/T372937 (10bd808) 03NEW [20:26:18] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: Figure out how to provision a Kubernetes cluster using Magnum and OpenTofu - https://phabricator.wikimedia.org/T372498#10079247 (10bd808) [20:26:19] 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: cloudvps-repos/deployment-prep - https://phabricator.wikimedia.org/T372937#10079248 (10bd808) [20:26:57] 10Beta-Cluster-Infrastructure, 10GitLab (Project Migration), 06Release-Engineering-Team: Create new GitLab project group: cloudvps-repos/deployment-prep - https://phabricator.wikimedia.org/T372937#10079249 (10bd808) [20:31:53] 10GitLab (CI & Job Runners), 07Documentation: https://www.mediawiki.org/wiki/GitLab/Workflows/CI doesn't mention using tags to select runners - https://phabricator.wikimedia.org/T372846#10079254 (10bd808) https://www.mediawiki.org/wiki/GitLab/Workflows/Deploying_services_to_production#Runners has a good descri... [20:36:54] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS (Debian Buster Deprecation), 13Patch-For-Review: Remove or replace deployment-restbase04.deployment-prep.eqiad1.wikimedia.cloud (Buster deprecation) - https://phabricator.wikimedia.org/T370460#10079257 (10Eevans) >>! In T370460#10070859, @Ee... [21:00:10] 14Release-Engineering-Team (GitLab III: GitLab in LA 🪃): Provision Ingress with TLS for reggie to allow image reuse in pipelines - https://phabricator.wikimedia.org/T324361#10079303 (10bd808) > However we need to clearly communicate our policy (and best practice) that **even temporary containers should never... [21:19:19] 06Project-Admins: Create project tag and working board for WE5_2-KR - https://phabricator.wikimedia.org/T372942 (10Mooeypoo) 03NEW [21:32:19] 06Project-Admins: Create project tag and working board for WE5_2-KR - https://phabricator.wikimedia.org/T372942#10079434 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #fy2024-25_kr_5.2_simplify_feature_development has been created: https://phabricator.wikimedia.org/project/view/7... [21:33:59] 06Project-Admins: Request for tag "WBS - Extending" - https://phabricator.wikimedia.org/T369290#10079443 (10Aklapper) 05Open→03Declined @jon_amar-WMDE: Unfortunately closing this Phabricator task as no further information has been provided. After you have provided the information asked for and if this is... [21:34:01] 06Project-Admins: Request for tag "WBS - Migrating" - https://phabricator.wikimedia.org/T369291#10079441 (10Aklapper) 05Open→03Declined @jon_amar-WMDE: Unfortunately closing this Phabricator task as no further information has been provided. After you have provided the information asked for and if this is... [21:34:34] 06Project-Admins: Request for tag "WBS - Updating" - https://phabricator.wikimedia.org/T369289#10079445 (10Aklapper) 05Open→03Declined @jon_amar-WMDE: Unfortunately closing this Phabricator task as no further information has been provided. After you have provided the information asked for and if this is... [21:34:42] 06Project-Admins: Create project tag and working board for WE5_2-KR - https://phabricator.wikimedia.org/T372942#10079459 (10Mooeypoo) Thank you! [21:36:52] 06Project-Admins: Create project tag for Extension:Cookiecutter - https://phabricator.wikimedia.org/T372699#10079467 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #cookiecutter has been created: https://phabricator.wikimedia.org/project/view/7376/ (In case you need to edit the p... [21:54:53] (03open) 10dancy: test_deploy.py: Add check for proper handling of check overrides [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/409 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [21:54:54] (03update) 10dancy: test_deploy.py: Add check for proper handling of check overrides [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/409 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [21:56:36] (03merge) 10dancy: test_deploy.py: Add check for proper handling of check overrides [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/409 (https://phabricator.wikimedia.org/T149668 https://phabricator.wikimedia.org/T372921) [21:57:15] (03open) 10dancy: Release 4.99.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/410 [21:58:42] (03merge) 10dancy: Release 4.99.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/410 [22:15:16] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10079567 (10Aklapper) I assume this is a group - https://www.mediawiki.org/wiki/Phabricator/Project_management#Types_of_Projects [22:21:44] 06Project-Admins: Project request for data-persistence-sre - https://phabricator.wikimedia.org/T372862#10079579 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #data-persistence-sre has been created: https://phabricator.wikimedia.org/project/view/7377/ (In case you need to edit th... [22:25:15] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 [22:28:21] 06Release-Engineering-Team, 10Scap, 13Patch-For-Review: RESTBase deployment fails with unhandled error when there are empty checks defined - https://phabricator.wikimedia.org/T372921#10079585 (10dancy) The fix for the blank check problem has been merged and deployed to the production deploy server but I ran... [22:37:04] (03update) 10sandeeps: cli.py: add scap version to fatal error meessages for better logging [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/407 [22:42:32] 10Phabricator: Unhandled Exception ("PhutilAggregateException") - https://phabricator.wikimedia.org/T372227#10079617 (10Aklapper) 05Stalled→03Declined Unfortunately closing this Phabricator task as no further information has been provided. @Yann: After you have provided the information asked for and if... [23:06:41] (03open) 10kindrobot: Draft: PLEASE SUBMIT MRs TO NEW ACTIVE FORK [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/630 [23:07:33] (03close) 10kindrobot: Build the Codex docs again [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/622 (owner: 10catrope) [23:08:02] (03close) 10kindrobot: Create wikis in a background process [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/405 (owner: 10matmarex) [23:08:03] (03update) 10kindrobot: Create wikis in a background process [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/405 (owner: 10matmarex) [23:08:13] (03close) 10kindrobot: Add ElasticSearch service [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/380 (owner: 10matmarex) [23:08:14] (03update) 10kindrobot: Add ElasticSearch service [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/380 (owner: 10matmarex) [23:08:23] (03close) 10kindrobot: Allow patches to be updated [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/270 (owner: 10matmarex) [23:08:23] (03update) 10kindrobot: Allow patches to be updated [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/270 (owner: 10matmarex) [23:08:38] (03close) 10kindrobot: Allow wikis to be redirected when deleting [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/172 (owner: 10matmarex) [23:08:38] (03update) 10kindrobot: Allow wikis to be redirected when deleting [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/172 (owner: 10matmarex) [23:08:48] (03close) 10kindrobot: Allow site config to be overridden by trusted users [repos/ci-tools/patchdemo] - 10https://gitlab.wikimedia.org/repos/ci-tools/patchdemo/-/merge_requests/112 (owner: 10matmarex) [23:09:01] Sorry for the spam :o [23:30:53] 06Release-Engineering-Team, 10Scap, 13Patch-For-Review: RESTBase deployment fails with unhandled error when there are empty checks defined - https://phabricator.wikimedia.org/T372921#10079804 (10bd808)