[00:31:39] 06cloud-services-team, 10MediaWiki-extensions-OpenStackManager, 10wikitech.wikimedia.org, 13Patch-For-Review: Remove OpenStackManager from Wikitech - https://phabricator.wikimedia.org/T161553#9869649 (10bd808) >>! In T161553#9869599, @Stashbot wrote: > {nav icon=file, name=Mentioned in SAL (#wikimedia-oper... [00:37:41] FIRING: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [00:47:41] RESOLVED: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [01:50:11] (03open) 10bd808: add_item_to_window: handle existing section at end of list [toolforge-repos/schedule-deployment] - 10https://gitlab.wikimedia.org/toolforge-repos/schedule-deployment/-/merge_requests/7 [01:53:21] (03merge) 10bd808: add_item_to_window: handle existing section at end of list [toolforge-repos/schedule-deployment] - 10https://gitlab.wikimedia.org/toolforge-repos/schedule-deployment/-/merge_requests/7 [02:09:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [02:19:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [05:34:05] 10Toolforge (Toolforge iteration 11): [builds-cli, jobs-cli,envvars-cli,webservice] update package/version scripts to use bookworm - https://phabricator.wikimedia.org/T366674#9869874 (10Slst2020) [05:48:27] 10Toolforge: [builds-api, envvars-api] Use the aliases instead of ResponseMessage type - https://phabricator.wikimedia.org/T366870 (10Slst2020) 03NEW [05:49:30] (03update) 10sstefanova: openapi: refactor yaml [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/96 (https://phabricator.wikimedia.org/T366668) [05:49:44] (03merge) 10sstefanova: openapi: refactor yaml [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/96 (https://phabricator.wikimedia.org/T366668) [05:53:24] 10Toolforge (Toolforge iteration 11), 13Patch-For-Review: [jobs-api, builds-api, envvars-api] Make openapi yaml comply with OAPI spec 3.1.0 - https://phabricator.wikimedia.org/T366668#9869945 (10Slst2020) [05:55:22] (03merge) 10sstefanova: cli: use prefixed endpoints [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/43 [05:55:55] 10Toolforge (Toolforge iteration 11), 13Patch-For-Review: [jobs-api, builds-api, envvars-api] Make openapi yaml comply with OAPI spec 3.1.0 - https://phabricator.wikimedia.org/T366668#9869947 (10Slst2020) 05In progress→03Resolved [05:57:23] (03open) 10project_1317_bot_df3177307bed93c3f34e421e26c86e38: builds-api: bump to 0.0.155-20240607054956-c39a1ef1 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/319 (https://phabricator.wikimedia.org/T366668) [06:31:23] 10Tools: Tool User Task BOT is not functioning - https://phabricator.wikimedia.org/T366876 (10mounika123) 03NEW [06:32:52] 10Tools: User Task BOT is unavailable - https://phabricator.wikimedia.org/T366876#9870035 (10mounika123) [06:40:04] 10wikitech.wikimedia.org, 07Technical-Debt: Update Phabricator BlockIpComplete hook to use "user.edit" Conduit API - https://phabricator.wikimedia.org/T366587#9870040 (10Aklapper) Thanks everyone! [07:15:50] (03open) 10sstefanova: utils: update deb build and bump setup [repos/cloud/toolforge/envvars-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/envvars-cli/-/merge_requests/45 (https://phabricator.wikimedia.org/T366674) [07:21:50] (03open) 10sstefanova: utils: fix code deletion [repos/cloud/toolforge/builds-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-cli/-/merge_requests/74 [07:23:32] (03PS1) 10Brouberol: gitlab: iterate over all pages of results to index missing repositories [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1039977 (https://phabricator.wikimedia.org/T366878) [07:24:29] (03CR) 10CI reject: [V:04-1] gitlab: iterate over all pages of results to index missing repositories [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1039977 (https://phabricator.wikimedia.org/T366878) (owner: 10Brouberol) [08:05:22] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879 (10Kalyan321) 03NEW [08:06:10] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870168 (10Kalyan321) [08:08:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [08:09:05] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870169 (10Kalyan321) [08:10:09] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870170 (10Kalyan321) [08:11:12] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870174 (10Kalyan321) [08:18:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [08:21:33] (03open) 10sstefanova: utils: update deb build and bump setup [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/36 (https://phabricator.wikimedia.org/T366674) [08:25:31] 10Toolforge (Toolforge iteration 11), 13Patch-For-Review: [builds-cli, jobs-cli, envvars-cli] update package/version scripts to use bookworm - https://phabricator.wikimedia.org/T366674#9870228 (10Slst2020) [08:32:12] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete virt-star stub secret [labs/private] - 10https://gerrit.wikimedia.org/r/1039227 (owner: 10Muehlenhoff) [08:32:45] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove tendril stub cert [labs/private] - 10https://gerrit.wikimedia.org/r/1039189 (owner: 10Muehlenhoff) [08:32:53] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete LDAP stub secrets [labs/private] - 10https://gerrit.wikimedia.org/r/1039184 (owner: 10Muehlenhoff) [08:44:01] 10Tool-python-toolforge, 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870283 (10taavi) 05Open→03Invalid Please contact the tool maintainers first. #Toolforge is for reporting issues with the infrastrcture itself, which I don't see here, and #... [09:00:14] (03update) 10aborrero: kubernetes: introduce securityContext in the pod template [repos/cloud/toolforge/tools-webservice] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/37 (https://phabricator.wikimedia.org/T362050) [09:01:09] (03update) 10aborrero: kubernetes: introduce securityContext in the pod template [repos/cloud/toolforge/tools-webservice] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/37 (https://phabricator.wikimedia.org/T362050) [09:02:21] 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870327 (10Kalyan321) [09:03:30] 10Toolforge: updating the list of languages by editor gadget - https://phabricator.wikimedia.org/T366879#9870330 (10Kalyan321) [09:06:33] 10Toolforge: tools-static webserver shows directory listing instead of the index file - https://phabricator.wikimedia.org/T366482#9870333 (10Mormegil) Well, what difference is there between “static assets” and “full websites”? Back in my day, all websites were just static assets (…kids these days…). :-D The use... [09:22:41] (03open) 10aborrero: functional-tests: build-smoke-test: retry build show test [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/320 [09:38:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [09:42:31] (03PS1) 10Majavah: Add fake opentofu admin passwords and tokens [labs/private] - 10https://gerrit.wikimedia.org/r/1040099 (https://phabricator.wikimedia.org/T365696) [09:44:46] (03CR) 10Majavah: [V:03+2 C:03+2] Add fake opentofu admin passwords and tokens [labs/private] - 10https://gerrit.wikimedia.org/r/1040099 (https://phabricator.wikimedia.org/T365696) (owner: 10Majavah) [09:48:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [09:50:09] !log sstefanova@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api [09:50:19] !log sstefanova@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api [09:58:52] !log sstefanova@cloudcumin1001 tools START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api [09:59:05] !log sstefanova@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api [10:02:37] !log sstefanova@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api [10:02:47] !log sstefanova@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api [10:07:00] (03update) 10sstefanova: envvars-api: bump to 0.0.47-20240606142628-c69cfbb6 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/318 (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:07:24] (03approved) 10sstefanova: envvars-api: bump to 0.0.47-20240606142628-c69cfbb6 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/318 (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:07:28] (03merge) 10sstefanova: envvars-api: bump to 0.0.47-20240606142628-c69cfbb6 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/318 (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:09:28] FIRING: PuppetSyncFailure: Failed to update Puppet repository /srv/git/labs/private on instance tools-puppetserver-01 in project tools - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetSyncFailure [10:09:49] !log sstefanova@cloudcumin1001 tools START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api [10:10:00] !log sstefanova@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api [10:25:11] 10cloud-services-team (FY2023/2024-Q3-Q4), 10Puppet-Infrastructure, 13Patch-For-Review: Ownership confusion on cloud-local puppet servers - https://phabricator.wikimedia.org/T364492#9870555 (10pmiazga) 05Resolved→03Open I'm going to re-open this ticket as this is still an issue on `deployment-prep.deploy... [10:26:38] (03update) 10sstefanova: builds-api: bump to 0.0.155-20240607054956-c39a1ef1 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/319 (https://phabricator.wikimedia.org/T366668) (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:27:30] (03open) 10aborrero: lima-vm: dotfiles: add dotfiles for arturo [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/136 [10:27:33] (03update) 10sstefanova: builds-api: bump to 0.0.155-20240607054956-c39a1ef1 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/319 (https://phabricator.wikimedia.org/T366668) (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:27:42] (03merge) 10sstefanova: builds-api: bump to 0.0.155-20240607054956-c39a1ef1 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/319 (https://phabricator.wikimedia.org/T366668) (owner: 10project_1317_bot_df3177307bed93c3f34e421e26c86e38) [10:34:37] (03approved) 10sstefanova: lima-vm: dotfiles: add dotfiles for arturo [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/136 (owner: 10aborrero) [10:40:24] 10Toolforge, 10MediaWiki-extensions-Toolhub: wiki wanting to find such articles. - https://phabricator.wikimedia.org/T366895 (10Kalyan321) 03NEW [10:43:05] 10Toolforge, 10MediaWiki-extensions-Toolhub: wiki wanting to find such articles. - https://phabricator.wikimedia.org/T366895#9870624 (10taavi) 05Open→03Invalid As I said previously, #Toolforge is not the place to report bugs on individual tools. [10:46:32] 10Toolforge: .. - https://phabricator.wikimedia.org/T366879#9870629 (10KALYAN907) [10:48:04] (03merge) 10aborrero: lima-vm: dotfiles: add dotfiles for arturo [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/136 [10:56:09] (03update) 10aborrero: lima-vm: dotfiles: arturo: update bashrc [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/137 [10:56:12] (03open) 10aborrero: lima-vm: dotfiles: arturo: update bashrc [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/137 [10:57:09] (03merge) 10aborrero: lima-vm: dotfiles: arturo: update bashrc [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/137 [11:05:11] (03close) 10aborrero: k9s: add config files [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/128 [11:05:38] (03open) 10aborrero: lima-vm: dotfiles: arturo: add additional git aliases [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/138 [11:06:31] (03merge) 10aborrero: lima-vm: dotfiles: arturo: add additional git aliases [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/138 [11:09:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [11:19:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [11:27:50] !log aborrero@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers [11:27:58] !log aborrero@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers [11:28:10] !log aborrero@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno [11:28:29] !log aborrero@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno [11:39:24] 10Toolforge: Getting HTTP status code - https://phabricator.wikimedia.org/T366897 (10Kalyan899) 03NEW [11:42:01] 10Toolforge: Getting HTTP status code - https://phabricator.wikimedia.org/T366897#9870710 (10Kalyan899) [12:00:38] (03update) 10aborrero: components: add kyverno [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/238 (https://phabricator.wikimedia.org/T279110) [12:01:13] (03update) 10aborrero: components: add kyverno [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/238 (https://phabricator.wikimedia.org/T279110) [12:08:13] (03update) 10aborrero: maintain_kubeusers: add support for kyverno policies [repos/cloud/toolforge/maintain-kubeusers] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/18 (https://phabricator.wikimedia.org/T279110) [12:09:30] (03open) 10taavi: Manage new flavors [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/1 (https://phabricator.wikimedia.org/T364458) [12:19:12] (03open) 10sstefanova: Draft: dev: add pre-commit [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/37 [12:33:41] (03update) 10sstefanova: Draft: dev: add pre-commit [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/37 [12:37:41] FIRING: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [12:47:41] RESOLVED: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [12:47:55] (03update) 10aborrero: maintain_kubeusers: add support for kyverno policies [repos/cloud/toolforge/maintain-kubeusers] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/18 (https://phabricator.wikimedia.org/T279110) [13:06:56] (03PS7) 10Seb35: Make gitinfo.json identical to MediaWiki’s GitInfo [labs/tools/extdist] - 10https://gerrit.wikimedia.org/r/834708 (https://phabricator.wikimedia.org/T258537) [13:17:47] (03CR) 10Seb35: "@Legoktm: could you review it? See also the task to see an example." [labs/tools/extdist] - 10https://gerrit.wikimedia.org/r/834708 (https://phabricator.wikimedia.org/T258537) (owner: 10Seb35) [13:22:27] 10Quarry: query runs forever - https://phabricator.wikimedia.org/T366909 (10Wurgl) 03NEW [13:55:35] 10Cloud-VPS (Project-requests), 10Performance-Device-Lab, 06Quality-and-Test-Engineering-Team, 10Synthetic-Performance-Testing: Request creation of web performance test VPS project - https://phabricator.wikimedia.org/T366569#9871053 (10Peter) Thank you @Andrew ! I didn't thought about using the reverse... [14:07:41] FIRING: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [14:17:41] RESOLVED: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [15:07:45] 10Toolforge: Getting HTTP status code - https://phabricator.wikimedia.org/T366897#9871212 (10taavi) 05Open→03Invalid And you don't need to create a third account once your first two have been blocked. [15:38:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [15:42:50] (03open) 10aborrero: kind: add additional worker nodes to kubernetes [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/139 [15:48:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [15:55:58] 06cloud-services-team, 10Toolforge: toolforge: track PSP migration plan - https://phabricator.wikimedia.org/T364297#9871364 (10aborrero) on the latest tests I've conducted, I detected that the kyverno mutation rule somehow clashes with the PSP mutations. * kyverno mutation rule adding `hostIPC: false` * then... [16:13:03] 10Toolforge: [jobs] Allow configuration of Promethus scraping of a specific endpoint for publication in grafana.wmcloud.org - https://phabricator.wikimedia.org/T362012#9871400 (10taavi) [16:14:08] 10Toolforge: [jobs] Allow configuration of Promethus scraping of a specific endpoint for publication in grafana.wmcloud.org - https://phabricator.wikimedia.org/T362012#9871405 (10taavi) I'm assuming that by the time we get to this we have gotten around completing {T348755}. [16:15:33] 10Toolforge: Set up new Prometheus instance for user-created data - https://phabricator.wikimedia.org/T366923 (10taavi) 03NEW [17:14:54] 06cloud-services-team, 10MediaWiki-extensions-OpenStackManager, 10wikitech.wikimedia.org, 13Patch-For-Review: Remove OpenStackManager from Wikitech - https://phabricator.wikimedia.org/T161553#9871643 (10taavi) Well, that's embarrassing.. turns out the reason for that is a missing `connect()` call. Fixed in... [17:40:25] 06cloud-services-team, 10Cloud-VPS, 07Epic: [Epic] Provide logging/metrics/monitoring SaaS for Cloud VPS tenants - https://phabricator.wikimedia.org/T194333#9871723 (10taavi) [17:40:26] 06cloud-services-team, 10Toolforge, 10community-labs-monitoring: Establish an internal system or a recommended external system for monitoring user-created Toolforge web services - https://phabricator.wikimedia.org/T53434#9871722 (10taavi) [18:30:14] 10Cloud-VPS: Cloud VPS mail servers should drop mail sent from non-supported domains - https://phabricator.wikimedia.org/T366935 (10taavi) 03NEW [18:31:44] 10Cloud-VPS: Reject outbound traffic to port 25 (SMTP) from instances without public IPs - https://phabricator.wikimedia.org/T366936 (10taavi) 03NEW [18:47:06] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack [18:59:59] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [21:07:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [21:12:41] FIRING: [3x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [21:17:41] FIRING: [3x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [21:22:41] RESOLVED: [3x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:07:41] FIRING: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:17:41] RESOLVED: [2x] CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:29:28] RESOLVED: PuppetSyncFailure: Failed to update Puppet repository /srv/git/labs/private on instance tools-puppetserver-01 in project tools - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetSyncFailure [22:38:36] 10Wikibugs: Replace wikibugs2-znc container with tool-containers:bnc container - https://phabricator.wikimedia.org/T366966 (10bd808) 03NEW [22:39:46] 10Tool-containers, 10Wikibugs: Replace wikibugs2-znc container with tool-containers:bnc container - https://phabricator.wikimedia.org/T366966#9872562 (10bd808) [22:45:32] 10Tool-bridgebot: Add a webservice to display running config - https://phabricator.wikimedia.org/T261989#9872619 (10bd808) The config is now published at https://gitlab.wikimedia.org/toolforge-repos/bridgebot/-/blob/main/etc/bridgebot.toml [22:49:07] 10Tool-bridgebot, 10Tool-containers: Replace tool-bridgebot/znc container with tool-containers/bnc container - https://phabricator.wikimedia.org/T366970 (10bd808) 03NEW [22:49:36] 10Tool-containers, 10Wikibugs: Replace tool-wikibugs/znccontainer with tool-containers/bnc container - https://phabricator.wikimedia.org/T366966#9872642 (10bd808) [23:05:37] 10Tool-containers: Create a cookiecutter template for building new service containers - https://phabricator.wikimedia.org/T366972 (10bd808) 03NEW [23:06:00] 10Tool-containers: Create a cookiecutter template for building new service containers - https://phabricator.wikimedia.org/T366972#9872694 (10bd808) 05Open→03In progress p:05Triage→03High a:03bd808 [23:27:58] 10Tool-containers: Create a cookiecutter template for building new service containers - https://phabricator.wikimedia.org/T366972#9872711 (10bd808) See https://github.com/lucaswerkmeister/cookiecutter-toolforge for inspiration