[00:02:28] 10PAWS: New upstream release for OpenRefine - https://phabricator.wikimedia.org/T380658#10350282 (10LibUp-bot) [03:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [03:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [07:10:45] (03approved) 10raymond-ndibe: [builds-builder] create toolforge harbor project [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/63 (https://phabricator.wikimedia.org/T358225) [07:10:54] (03merge) 10raymond-ndibe: [builds-builder] create toolforge harbor project [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/63 (https://phabricator.wikimedia.org/T358225) [07:13:04] (03open) 10group_203_bot_4866fc124f4b41659f667468a6115cf3: builds-builder: bump to 0.0.123-20241123071107-d41b1e76 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/618 (https://phabricator.wikimedia.org/T358225) [07:14:39] !log raymond-ndibe@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-builder (T358225) [07:14:43] T358225: [maintain-harbor] Move to become a toolforge component - https://phabricator.wikimedia.org/T358225 [07:20:27] !log raymond-ndibe@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder (T358225) [07:20:32] T358225: [maintain-harbor] Move to become a toolforge component - https://phabricator.wikimedia.org/T358225 [07:21:34] !log raymond-ndibe@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component builds-builder (T358225) [07:25:14] (03update) 10raymond-ndibe: [lima-kilo] add harbor to toolforge-common.yaml [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/214 (https://phabricator.wikimedia.org/T358225) [07:27:35] !log raymond-ndibe@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder (T358225) [07:27:39] T358225: [maintain-harbor] Move to become a toolforge component - https://phabricator.wikimedia.org/T358225 [07:32:51] (03approved) 10raymond-ndibe: builds-builder: bump to 0.0.123-20241123071107-d41b1e76 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/618 (https://phabricator.wikimedia.org/T358225) (owner: 10group_203_bot_4866fc124f4b41659f667468a6115cf3) [07:33:06] (03merge) 10raymond-ndibe: builds-builder: bump to 0.0.123-20241123071107-d41b1e76 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/618 (https://phabricator.wikimedia.org/T358225) (owner: 10group_203_bot_4866fc124f4b41659f667468a6115cf3) [08:10:31] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [11:33:14] FIRING: Kernel error: Server cloudvirt1061 may have kernel errors - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Kernel_panic - https://grafana.wikimedia.org/d/b013af4c-d405-4d9f-85d4-985abb3dec0c/wmcs-kernel-panic-detector?orgId=1&var-instance=cloudvirt1061 - https://alerts.wikimedia.org/?q=alertname%3DKernel+error [11:33:20] 06cloud-services-team: Kernel error Server cloudvirt1061 may have kernel errors - https://phabricator.wikimedia.org/T380673 (10phaultfinder) 03NEW [11:50:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [12:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [12:33:46] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [12:42:07] (03update) 10raymond-ndibe: [toolforge-deploy] allow for running both admin and tools tests [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/605 (https://phabricator.wikimedia.org/T358225) [12:42:52] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [12:47:25] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [12:47:39] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [12:47:47] (03update) 10raymond-ndibe: [toolforge-deploy] deploy maintain-harbor [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/563 (https://phabricator.wikimedia.org/T358225) [13:39:13] 06cloud-services-team, 10Cloud-VPS, 06Infrastructure-Foundations, 13Patch-For-Review, 07Puppet: Puppet removed "nameserver" line from /etc/resolv.conf - https://phabricator.wikimedia.org/T379927#10350696 (10Andrew) Just found another VM where this happened: liwa3-2.linkwatcher.eqiad1.wikimedia.cloud [14:56:52] 06cloud-services-team, 10Cloud-VPS: Drop support for VMs with .wmflabs FQDNs - https://phabricator.wikimedia.org/T380679 (10taavi) 03NEW [14:57:00] 06cloud-services-team, 10Cloud-VPS: Drop support for VMs with .wmflabs FQDNs - https://phabricator.wikimedia.org/T380679#10350780 (10taavi) [14:59:57] 06cloud-services-team, 10Cloud-VPS: Drop support for VMs with .wmflabs FQDNs - https://phabricator.wikimedia.org/T380679#10350783 (10taavi) 05Open→03Stalled [15:33:14] FIRING: Kernel error: Server cloudvirt1061 may have kernel errors - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Kernel_panic - https://grafana.wikimedia.org/d/b013af4c-d405-4d9f-85d4-985abb3dec0c/wmcs-kernel-panic-detector?orgId=1&var-instance=cloudvirt1061 - https://alerts.wikimedia.org/?q=alertname%3DKernel+error [16:43:14] FIRING: [2x] Kernel error: Server cloudcontrol1005 may have kernel errors - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Kernel_panic - https://alerts.wikimedia.org/?q=alertname%3DKernel+error [18:51:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [19:01:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [20:43:14] FIRING: Kernel error: Server cloudvirt1061 may have kernel errors - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Kernel_panic - https://grafana.wikimedia.org/d/b013af4c-d405-4d9f-85d4-985abb3dec0c/wmcs-kernel-panic-detector?orgId=1&var-instance=cloudvirt1061 - https://alerts.wikimedia.org/?q=alertname%3DKernel+error [20:50:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [21:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:06:52] (03PS1) 10Urbanecm: newArticlesPortals: Link to the configuration page [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095539 [22:06:52] (03CR) 10Urbanecm: [C:03+2] newArticlesPortals: Link to the configuration page [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095539 (owner: 10Urbanecm) [22:06:53] (03PS1) 10Urbanecm: jobs: Disable wikidata-label-import [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095540 [22:06:53] (03CR) 10Urbanecm: [C:03+2] jobs: Disable wikidata-label-import [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095540 (owner: 10Urbanecm) [22:06:53] (03PS1) 10Urbanecm: deletedEverywhere: Add minor updates [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095541 [22:06:55] (03CR) 10Urbanecm: [C:03+2] deletedEverywhere: Add minor updates [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095541 (owner: 10Urbanecm) [22:07:14] (03Merged) 10jenkins-bot: newArticlesPortals: Link to the configuration page [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095539 (owner: 10Urbanecm) [22:07:16] (03Merged) 10jenkins-bot: jobs: Disable wikidata-label-import [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095540 (owner: 10Urbanecm) [22:07:19] (03Merged) 10jenkins-bot: deletedEverywhere: Add minor updates [labs/tools/urbanecmbot] - 10https://gerrit.wikimedia.org/r/1095541 (owner: 10Urbanecm)