[00:46:11] (03PS1) 10Samwilson: Add mediawiki/skins/SimpleText to layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 [05:53:00] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: Request mariadb database for Zuul - https://phabricator.wikimedia.org/T394844#10874196 (10LSobanski) p:05Triage→03High [05:53:04] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: allow new zuul executor VMs in prod to talk to cloud VPS - https://phabricator.wikimedia.org/T394838#10874198 (10LSobanski) p:05Triage→03High [05:53:08] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 18 - https://phabricator.wikimedia.org/T394382#10874200 (10LSobanski) p:05Triage→03High [05:54:25] 10Gerrit, 10Release-Engineering-Team (Radar), 06collaboration-services, 06Traffic, 13Patch-For-Review: Separate Gerrit https and ssh/git hostnames - https://phabricator.wikimedia.org/T394271#10874208 (10LSobanski) p:05Triage→03High [05:56:26] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Investigate moving from Digital Ocean to WMF infrastructure - https://phabricator.wikimedia.org/T389936#10874214 (10LSobanski) p:05Triage→03Medium [06:00:31] 10Phabricator, 10Release-Engineering-Team (Doing 😎): Disable "To:" and "CC:" Recipient Hint footers in mail notifications (metamta.recipients.show-hints) - https://phabricator.wikimedia.org/T391924#10874217 (10Aklapper) 05Open→03Resolved No replies/concerns/complaints I'm aware of. https://phabricator.... [07:21:35] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391434#10874274 (10Aklapper) 05Stalled→03Declined @Sirchanmp: Unfortunately closing this Phabricator task as no further information has been provided. After... [07:56:00] (03PS3) 10Hashar: jjb: allow castor to change MTIME on cache dir [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [07:57:23] (03CR) 10Hashar: "I have hihacked your change to `setfacl -m user:65534:w cache`. I have updated the `integration-config*` Jenkins job so we should see the" [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [07:57:41] (03CR) 10CI reject: [V:04-1] jjb: allow castor to change MTIME on cache dir [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [08:11:11] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions: Train blockers task throws exceptions on strange version numbers - https://phabricator.wikimedia.org/T304540#10874388 (10Aklapper) 05Open→03Stalled [08:26:37] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Check GitLab artifact retention time - https://phabricator.wikimedia.org/T395014#10874408 (10Jelto) Thanks for the feedback! I created a dedicated artifact backup on `gitlab2002` in case we need to restore anything: ` jelto@gi... [08:39:56] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Consider setting up an https://github.com/knyar/phalerts instance in metricsinfra - https://phabricator.wikimedia.org/T394446#10874423 (10taavi) @bd808 I'm guessing you're after a rule that sends all deployment-prep aler... [09:14:00] (03open) 10jelto: update artifact expiry to 6mo [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/69 (https://phabricator.wikimedia.org/T395014) [09:22:39] (03CR) 10Hashar: jjb: allow castor to change MTIME on cache dir (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [09:25:58] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Check GitLab artifact retention time - https://phabricator.wikimedia.org/T395014#10874598 (10Jelto) I updated the retention and cleanup settings on all GitLab instances and projects. I'll wait until the aut... [09:28:53] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Check GitLab artifact retention time - https://phabricator.wikimedia.org/T395014#10874602 (10Jelto) [09:32:39] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS: Consider setting up an https://github.com/knyar/phalerts instance in metricsinfra - https://phabricator.wikimedia.org/T394446#10874619 (10taavi) [09:47:42] (03PS4) 10Hashar: jjb: set cache directory ownership to nobody:nogroup [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [09:51:50] (03CR) 10Hashar: "The `mkdir` is changed by the equivalent of:" [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [09:52:13] Lucas_WMDE: I think I have finally fixed that rsync issue "failed to set times on "/cache/."" :) [09:52:25] by creating the cache directory with the same ownership as the container [09:52:39] and I think that is a tech debt we have borrowed since ~ 2017 or so [09:55:39] (03CR) 10Lucas Werkmeister (WMDE): jjb: set cache directory ownership to nobody:nogroup (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [09:56:57] \o/ [10:09:13] (03close) 10cgoubert: mediawiki-cli: Install fonts-freefont-ttf [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/178 [10:09:42] (03CR) 10Hashar: jjb: set cache directory ownership to nobody:nogroup (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:11:22] 06Project-Admins, 10Release-Engineering-Team (Doing 😎): Create milestone tag for Ultraviolet/Vue rewrite - https://phabricator.wikimedia.org/T395722#10874707 (10Aklapper) 05Open→03Resolved a:03Aklapper Done: https://phabricator.wikimedia.org/project/view/7954/ [10:11:40] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] jjb: set cache directory ownership to nobody:nogroup (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:12:08] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "(I meant to +1 with my last comment already but I guess it got lost somewhere)" [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:27:36] (03CR) 10Hashar: [C:03+2] "I am updating all jobs 😎 Thanks for having triggered this fix Lucas!" [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:28:57] (03CR) 10Lucas Werkmeister (WMDE): [C:03+1] "Thanks for actually doing everything :D" [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:29:24] (03Merged) 10jenkins-bot: jjb: set cache directory ownership to nobody:nogroup [integration/config] - 10https://gerrit.wikimedia.org/r/1152248 (https://phabricator.wikimedia.org/T188488) (owner: 10Lucas Werkmeister (WMDE)) [10:59:16] (03PS1) 10Hashar: jjb: gerrit: use standard "setup" macro [integration/config] - 10https://gerrit.wikimedia.org/r/1152656 [11:00:54] (03CR) 10CI reject: [V:04-1] jjb: gerrit: use standard "setup" macro [integration/config] - 10https://gerrit.wikimedia.org/r/1152656 (owner: 10Hashar) [11:07:41] (03PS2) 10Hashar: jjb: gerrit: use standard "setup" macro [integration/config] - 10https://gerrit.wikimedia.org/r/1152656 [11:14:09] (03CR) 10Hashar: [C:03+2] jjb: gerrit: use standard "setup" macro [integration/config] - 10https://gerrit.wikimedia.org/r/1152656 (owner: 10Hashar) [11:15:58] (03Merged) 10jenkins-bot: jjb: gerrit: use standard "setup" macro [integration/config] - 10https://gerrit.wikimedia.org/r/1152656 (owner: 10Hashar) [12:19:27] (03PS2) 10Samwilson: Add mediawiki/skins/SimpleText to layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 [12:38:34] (03CR) 10Hashar: [C:03+2] "Yet another skin! :]" [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 (owner: 10Samwilson) [12:40:20] (03Merged) 10jenkins-bot: Add mediawiki/skins/SimpleText to layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 (owner: 10Samwilson) [12:41:41] (03CR) 10Samwilson: "At least it doesn't have any PHP code. :-P" [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 (owner: 10Samwilson) [13:01:05] (03CR) 10Hashar: [C:03+2] "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/1152447 (owner: 10Samwilson) [13:14:56] 10Scap (SpiderPig 🕸️): “Show sensitive information” checkbox broken, suspends terminal - https://phabricator.wikimedia.org/T394411#10875380 (10Lucas_Werkmeister_WMDE) Seems to be fixed indeed, thanks! [13:23:08] 10Continuous-Integration-Infrastructure, 10Castor: Castor rsync causes: rsync: failed to set times on "/cache/.": Operation not permitted (1) - https://phabricator.wikimedia.org/T188488#10875404 (10hashar) 05Open→03Resolved a:03hashar That should be solved now after having changed the `cache` directo... [13:23:31] (03CR) 10Hashar: [C:03+2] Add Matrix to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1152391 (owner: 10Pppery) [13:24:07] (03PS2) 10Jforrester: Zuul: Add Matrix to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1152391 (owner: 10Pppery) [13:24:13] (03CR) 10Jforrester: [C:03+2] Zuul: Add Matrix to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1152391 (owner: 10Pppery) [13:25:27] (03Merged) 10jenkins-bot: Zuul: Add Matrix to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1152391 (owner: 10Pppery) [13:35:59] (03PS1) 10Hashar: jjb: drop obsolete creation of log & src dirs [integration/config] - 10https://gerrit.wikimedia.org/r/1152702 [13:40:32] (03PS2) 10Hashar: jjb: drop obsolete creation of log & src dirs [integration/config] - 10https://gerrit.wikimedia.org/r/1152702 [13:40:32] (03PS1) 10Hashar: jjb: change bazel jobs to use setup/teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1152703 [13:44:51] (03PS3) 10Hashar: jjb: drop obsolete creation of log & src dirs [integration/config] - 10https://gerrit.wikimedia.org/r/1152702 [13:44:52] (03PS2) 10Hashar: jjb: change bazel jobs to use setup/teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1152703 [13:44:52] (03PS1) 10Hashar: Zuul: [operations/software/gerrit/plugins/events-wikimedia] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1152704 (https://phabricator.wikimedia.org/T304947) [13:44:53] (03PS1) 10Hashar: jjb: remove unused Bazel job [integration/config] - 10https://gerrit.wikimedia.org/r/1152705 (https://phabricator.wikimedia.org/T304947) [13:45:58] !log Updating Jenkins jobs for "drop obsolete creation of log & src dirs" | https://gerrit.wikimedia.org/r/c/integration/config/+/1152702 [13:45:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:46:06] (03CR) 10Hashar: [C:03+2] jjb: drop obsolete creation of log & src dirs [integration/config] - 10https://gerrit.wikimedia.org/r/1152702 (owner: 10Hashar) [13:46:31] (03CR) 10Hashar: [C:03+2] "The job is no more used and is removed by I9dc97699433c2e5d7ef7332cdc8188ae8c65629b" [integration/config] - 10https://gerrit.wikimedia.org/r/1152703 (owner: 10Hashar) [13:46:40] (03CR) 10Hashar: [C:03+2] Zuul: [operations/software/gerrit/plugins/events-wikimedia] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1152704 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [13:46:48] (03CR) 10Hashar: [C:03+2] jjb: remove unused Bazel job [integration/config] - 10https://gerrit.wikimedia.org/r/1152705 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [13:47:16] (03Merged) 10jenkins-bot: jjb: drop obsolete creation of log & src dirs [integration/config] - 10https://gerrit.wikimedia.org/r/1152702 (owner: 10Hashar) [13:47:52] (03Merged) 10jenkins-bot: jjb: change bazel jobs to use setup/teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1152703 (owner: 10Hashar) [13:47:55] (03Merged) 10jenkins-bot: Zuul: [operations/software/gerrit/plugins/events-wikimedia] mark as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1152704 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [13:48:04] (03Merged) 10jenkins-bot: jjb: remove unused Bazel job [integration/config] - 10https://gerrit.wikimedia.org/r/1152705 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [13:53:58] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806 (10TheresNoTime) 03NEW [13:56:12] (03update) 10btullis: Fix the paths of some of the dumps scripts and config files [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/181 (https://phabricator.wikimedia.org/T394389) [13:56:45] If anyone could have a look at this, I'd be grateful. https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/181 Thanks. [14:08:38] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 13Patch-For-Review: move os-reports.wikimedia.org to kubernetes - https://phabricator.wikimedia.org/T350794#10875608 (10JMeybohm) I got a bit confused trying to find the deployment for this. It is currently managed via releases in `helmfile.d... [14:20:07] Project mediawiki-core-doxygen build #10915: 04FAILURE in 2 min 3 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/10915/ [14:30:29] RESOLVED: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance deployment-webperf21 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [14:34:49] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/CookieConsent] Add basic CI [integration/config] - 10https://gerrit.wikimedia.org/r/1152750 [14:35:10] (03CR) 10Jforrester: [C:03+2] Zuul: [mediawiki/extensions/CookieConsent] Add basic CI [integration/config] - 10https://gerrit.wikimedia.org/r/1152750 (owner: 10Jforrester) [14:36:37] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/CookieConsent] Add basic CI [integration/config] - 10https://gerrit.wikimedia.org/r/1152750 (owner: 10Jforrester) [14:36:48] !log Zuul: [mediawiki/extensions/CookieConsent] Add basic CI [14:36:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:37:16] !log Zuul: [operations/software/gerrit/plugins/events-wikimedia] mark as archived, for T304947 [14:37:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:37:18] T304947: Investigate sending Gerrit events to our data lake - https://phabricator.wikimedia.org/T304947 [14:37:30] !log Zuul: Add Matrix to CI allowlist [14:37:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:11:00] hi - anyone know what the deal is with these deployment chart CI failures? https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1152759 [15:15:00] hashar: ^ the CI output there (https://integration.wikimedia.org/ci/job/helm-lint/25274/console) mentions T282893 which is the task you hoped you’d partially solved earlier today IIUC [15:15:00] T282893: Various CI jobs failing after "mkdir: cannot create directory ‘log’: Permission denied" - https://phabricator.wikimedia.org/T282893 [15:16:45] maybe I broke something [15:20:02] Project mediawiki-core-doxygen build #10916: 04STILL FAILING in 1 min 59 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/10916/ [15:21:00] I will roll back [15:23:07] (03PS1) 10Hashar: Revert "jjb: drop obsolete creation of log & src dirs" [integration/config] - 10https://gerrit.wikimedia.org/r/1152761 (https://phabricator.wikimedia.org/T282893) [15:32:24] done [15:37:13] 06Project-Admins, 10Pywikibot, 10Pywikibot-Flow: Archive the #pywikibot-flow project - https://phabricator.wikimedia.org/T394710#10875968 (10Xqt) [15:41:17] Lucas_WMDE: fixed! Thank you for the report ( https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1152759 ) [15:42:20] thanks for fixing it! [15:42:33] and pity it didn’t work out :( [15:42:42] 10Beta-Cluster-Infrastructure, 06cloud-services-team, 10Cloud-VPS: Consider setting up an https://github.com/knyar/phalerts instance in metricsinfra - https://phabricator.wikimedia.org/T394446#10875992 (10bd808) >>! In T394446#10874423, @taavi wrote: > @bd808 I'm guessing you're after a rule that sends all d... [15:42:56] Lucas_WMDE: well I have deployed that config change without thoroughly checking the impact [15:43:10] I have literally thrown it over the wall hoping for it to not come back [15:43:15] and well.. OF COURSE it broke something :) [15:43:22] and it flew the over side of the wall back to me [15:44:13] 06Project-Admins, 10Pywikibot, 10Pywikibot-Flow: Archive the #pywikibot-flow project - https://phabricator.wikimedia.org/T394710#10875997 (10Xqt) I have no permission to archive this component [15:45:29] 10Gerrit (Gerrit 3.11), 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Upgrade to Gerrit 3.11 - https://phabricator.wikimedia.org/T379714#10876001 (10ABran-WMF) [15:49:58] 10Gerrit (Gerrit 3.12): Upgrade to Gerrit 3.12 - https://phabricator.wikimedia.org/T392448#10876032 (10ABran-WMF) [15:59:34] 06Project-Admins, 10Release-Engineering-Team (Doing 😎), 10Pywikibot: Archive the #pywikibot-flow project - https://phabricator.wikimedia.org/T394710#10876069 (10Aklapper) 05Open→03Resolved a:03Aklapper Thanks! Done. [15:59:38] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 06Data-Persistence: Request mariadb database for Zuul - https://phabricator.wikimedia.org/T394844#10876075 (10Dzahn) [16:03:30] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 06Data-Persistence: Request mariadb database for Zuul - https://phabricator.wikimedia.org/T394844#10876082 (10Dzahn) Hello Data-Persistence, we would like to request a new database on production database servers. It is to be... [16:05:47] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 06Data-Persistence: Request mariadb database for Zuul - https://phabricator.wikimedia.org/T394844#10876088 (10Marostegui) Could you follow the database creation template and fill out the following: ` Database creation template... [16:06:39] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806#10876091 (10Aklapper) Personally I am still not a fan of any temporary logo changes in Phabricator based on a random calendar system. [16:07:57] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services, 06Data-Persistence: Request mariadb database for Zuul - https://phabricator.wikimedia.org/T394844#10876093 (10Dzahn) a:03Dzahn Alright, will do. Thanks Manuel. I don't have answers for QPS and size but will try to get estima... [16:23:38] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806#10876195 (10brennen) 05Open→03In progress a:03brennen Set to the version I //think// we used for T337964. Happy Pride! [16:25:28] 10Phabricator, 06Security-Team, 07Security: Audit members of acl*security for more than 12 months of no activity (May 2025) - https://phabricator.wikimedia.org/T368224#10876225 (10Jly) @Aklapper would you be able to provide a list of those users, as I do not have DB access. I am happy to follow up with a tas... [16:29:25] (03approved) 10brennen: update artifact expiry to 6mo [repos/releng/gitlab-settings] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/69 (https://phabricator.wikimedia.org/T395014) (owner: 10jelto) [16:30:49] 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Dropped Security Patches in Wikimedia Production - https://phabricator.wikimedia.org/T395528#10876260 (10sbassett) 05Open→03Resolved a:03sbassett Everything related to this issue has been fixed, resolving for now. I th... [16:30:50] 06Release-Engineering-Team, 06Security-Team, 07SecTeam-Processed, 07Security: Dropped Security Patches in Wikimedia Production - https://phabricator.wikimedia.org/T395528#10876265 (10sbassett) p:05Triage→03High [16:31:59] Yippee, build fixed! [16:31:59] Project mediawiki-core-doxygen build #10917: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/10917/ [16:42:35] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806#10876339 (10Lucas_Werkmeister_WMDE) >>! In T395806#10876091, @Aklapper wrote: > Personally I am still not a fan of any temporary logo changes in Phabricator bas... [16:47:52] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 [16:47:52] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 [16:47:55] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 [16:48:48] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 [16:53:14] 10Continuous-Integration-Infrastructure (Zuul upgrade): Install a microK8s instance on WMCS project - https://phabricator.wikimedia.org/T395826 (10hashar) 03NEW [16:55:05] 10Phabricator, 06Release-Engineering-Team: Temporarily replace the Phabricator logo for Pride Month (2025) - https://phabricator.wikimedia.org/T395806#10876414 (10Aeyeu) >>! In T395806#10876339, @Lucas_Werkmeister_WMDE wrote: >>>! In T395806#10876091, @Aklapper wrote: >> Personally I am still not a fan of any... [16:56:09] 10Continuous-Integration-Infrastructure (Zuul upgrade): Install a microK8s instance on WMCS project - https://phabricator.wikimedia.org/T395826#10876421 (10hashar) [17:06:30] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [17:06:31] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [17:06:34] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [17:07:59] 10Continuous-Integration-Infrastructure (Zuul upgrade): Install a microK8s instance on WMCS project - https://phabricator.wikimedia.org/T395826#10876476 (10hashar) ` microk8s (1.26/stable) v1.26.15 from Canonical** installed ` I have no idea how it works though but it has something: ` $ sudo /snap/bin/microk8s... [17:26:20] 10Continuous-Integration-Infrastructure (Zuul upgrade): Install a microK8s instance on WMCS project - https://phabricator.wikimedia.org/T395826#10876587 (10hashar) On `microk8s.zuul3.eqiad1.wikimedia.cloud` I have retrieved the config including credentials using `sudo /snap/bin/microk8s kubectl config view --raw... [17:31:13] I am going to merge a change now.. that will make Gerrit replicate to a second replica. [17:31:21] to the new host gerrit2003 [17:31:41] before we never had more than one replica at a time. now the code allows for that. [17:32:03] not touching github replication in any way [17:32:37] replication config is the same for both and only repeated once: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1140520/6/hieradata/common/profile/gerrit.yaml [17:33:05] naming follows the gitlab standard with "replica-a" and "replica-b" style [17:40:11] 10Continuous-Integration-Infrastructure, 10Castor: Castor rsync causes: rsync: failed to set times on "/cache/.": Operation not permitted (1) - https://phabricator.wikimedia.org/T188488#10876686 (10matmarex) Thanks, this has been bugging me for years! [18:17:06] I'm getting an "remote: error: internal error" error when doing git review (for a very old patch). Is there useful debugging I can share to file this as a task? [18:17:07] 10GitLab (Upstream pit of despair 🕳️), 10Phabricator, 10Release-Engineering-Team (Doing 😎), 10Wikimedia-Phabricator-Extensions, 13Patch-For-Review: 'Related GitLab patches' box includes patches for unrelated tasks (that begin with the current task's ID) - https://phabricator.wikimedia.org/T395728#10876930... [18:18:45] Oh, interesting, looking in the git view it looks like my pushes worked, but returned an error? [18:20:05] (On https://gerrit.wikimedia.org/r/c/mediawiki/tools/codesniffer/+/279615 for reference.) [19:01:36] hrm, looks like a nullpointerexception [19:22:02] > Caused by: java.lang.NullPointerException: Cannot read field "value" because "anotherString" is null [19:22:24] the kind of error message that gives you a lot of confidence that everything is fine. [19:56:40] 10Release-Engineering-Team (Radar), 06serviceops-radar, 06SRE: deployment server - low disk space on /srv - https://phabricator.wikimedia.org/T387796#10877281 (10Dzahn) T394955 [20:01:08] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: DiskSpace: Disk space gerrit1003:9100:/ 5.694% free - https://phabricator.wikimedia.org/T371951#10877314 (10Dzahn) T394955 [21:19:53] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [21:19:55] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [21:19:57] (03update) 10dduvall: artifacts: Support `exclude` patterns for copies/requirements [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/123 [21:39:40] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Bitu, 06Infrastructure-Foundations: User corvus cannot activate ssh key in Bitu - https://phabricator.wikimedia.org/T395857 (10hashar) 03NEW [21:50:16] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Bitu, 06Infrastructure-Foundations: User corvus cannot activate ssh key in Bitu - https://phabricator.wikimedia.org/T395857#10877689 (10jhathaway) here is the first error in the logs, perhaps an invalid public key: ` DEBUG 2025-06-02 21:06:16,153 Exc... [21:52:10] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 (https://phabricator.wikimedia.org/T395645) [21:52:11] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 (https://phabricator.wikimedia.org/T395645) [21:52:15] (03update) 10dduvall: apt: Support `signed-by` field in apt configuration [repos/releng/blubber] - 10https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/124 (https://phabricator.wikimedia.org/T395645) [21:56:29] (03update) 10dduvall: spiderpig: Support MediaWiki train deployments [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/808 (https://phabricator.wikimedia.org/T392610) [21:56:36] (03update) 10dduvall: spiderpig: Support MediaWiki train deployments [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/808 (https://phabricator.wikimedia.org/T392610) [21:56:43] (03update) 10dduvall: spiderpig: Support MediaWiki train deployments [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/808 (https://phabricator.wikimedia.org/T392610) [22:24:50] (03update) 10dduvall: spiderpig: Support MediaWiki train deployments [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/808 (https://phabricator.wikimedia.org/T392610)