[00:00:37] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:04:19] (03open) 10addbot: Bump docker-registry.wikimedia.org/wikimedia/mediawiki-libs-shellbox image group to 2024-10-01-174300 [repos/releng/cli] - 10https://gitlab.wikimedia.org/repos/releng/cli/-/merge_requests/583 [00:05:37] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [06:27:54] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [kiyohan] - https://phabricator.wikimedia.org/T376194#10198651 (10abi_) >>! In T376194#10193604, @Aklapper wrote: > @Kiyohan: Hi and welcome to Phabricator! Could you maybe link to some previous contributions, or... [06:50:06] he,y a note to whoever is running the train log triage later today ( hashar?) , it's in conflict with the monthly staff meeting. could someone move it? [07:13:44] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [kiyohan] - https://phabricator.wikimedia.org/T376194#10198701 (10Aklapper) 05Open→03Resolved a:03Aklapper [07:32:20] apergos: I have conflicts before and after. I have moved it to 14:00 CEST / 12:00 UTC [07:32:26] which I guess is 15:00 in Greece? :) [07:33:04] hashar: crap, I have a meeting then I need to be at for our team. :-( [07:34:38] :D [07:35:05] it is probably going to be quiet, and I do the triage immediately after having run the train anyway [07:35:21] I think that log triage window is to prepare for promoting all wikis to the new version when the train runs during the US slot [07:35:26] at least that is why it is on thursday [07:35:44] but when we run it from Europe, the triage happens several hours after [07:35:50] anyway, maybe we should revisit :D [08:02:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198750 (10hashar) p:05Unb... [08:14:45] hashar: ok, well I will ask here after my conflicting meeting, what the highlights were and if there was anything for the mw eng team to be aware of. thanks for the info! [08:20:50] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198785 (10Joe) Please next... [08:26:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198804 (10Joe) Also, havin... [08:27:27] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198805 (10hashar) From the... [08:53:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198877 (10dcaro) Hmm, it's... [09:17:20] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198891 (10hashar) I went o... [09:36:32] 06Release-Engineering-Team, 06Data Products, 06Data-Platform-SRE, 10Dumps-Generation, and 2 others: Migrate current-generation dumps to run from our containerized images - https://phabricator.wikimedia.org/T352650#10198903 (10Joe) >>! In T352650#10179932, @dr0ptp4kt wrote: > @Joe checking - is Q2 FY 24-25... [10:03:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198973 (10dcaro) >>! In T3... [10:08:13] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10198979 (10aborrero) The cl... [10:31:30] Project beta-update-databases-eqiad build #79353: 04FAILURE in 11 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79353/ [10:36:13] Error: Class 'MediaWiki\Extension\CampaignEvents\Hooks\Handlers\SpecialPageInitListHandler' not found [10:39:04] https://phabricator.wikimedia.org/T373442#10199014 [10:40:30] 10Beta-Cluster-Infrastructure, 10CampaignEvents, 10Campaign-Tools (Campaign-Tools-Current-Sprint), 07ci-test-error, 13Patch-For-Review: Remove feature flag for hiding invitation list special pages - https://phabricator.wikimedia.org/T373442#10199015 (10RhinosF1) [11:00:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10199043 (10cmooney) As of n... [11:22:38] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10199083 (10aborrero) I noti... [11:32:14] Yippee, build fixed! [11:32:15] Project beta-update-databases-eqiad build #79354: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79354/ [11:44:06] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, and 2 others: Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10199126 (10cmooney) >>! In T374830#10199083, @aborrero wrote:... [12:36:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, and 2 others: Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10199225 (10aborrero) >>! In T374830#10199126, @cmooney wrote:... [12:57:39] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10199254 (10Aklapper) a:03Aklapper Thanks for the reminder about the existence of `epipe.py`. Running `more T356077.mbox | python .... [14:23:07] Project beta-update-databases-eqiad build #79357: 04FAILURE in 3 min 6 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79357/ [14:29:07] TypeError: Argument 3 passed to MediaWiki\JobQueue\JobQueueGroupFactory::__construct() must be an instance of Wikimedia\Stats\IBufferingStatsdDataFactory, instance of BufferingStatsdDataFactory given, called in /srv/mediawiki-staging/php-master/includes/ServiceWiring.php on line 1033 [14:30:20] Amir1: ^ [14:30:42] James_F: ^ [14:30:48] sounds like https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1076264 maybe? [14:31:50] yes, file a ticket, we take a look [14:34:17] Amir1: https://phabricator.wikimedia.org/T376392 [14:36:19] thanks [14:36:31] np [14:49:13] Reedy: that's fast (see review though) [14:50:52] Your review is wrong ;) [14:51:30] Reedy: +1'd [14:51:45] i should pay attention to less things at once [14:51:58] i'm in a uni workshop with a bunch of wittery cunts behind me [15:05:21] * hashar looks up "wittery cunts" [15:09:22] hashar: noisy people talking nonsense while i'm trying to concentrate [15:12:54] Thanks Reedy, RhinosF1. [15:13:06] I'm not guaranteeing that will fix it [15:13:08] But... [15:13:12] We'll see and have fun. [15:13:41] Reedy: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1076265 completes my stack for now, if you want to have more fun. ;-) [15:14:00] heh [15:14:01] Might do another 100 or so classes ahead of the REL1_43 branch point in two weeks. [15:14:07] Let's try and unbreak one thing first... :P [15:14:13] Psh. [15:14:20] Let's break all the things. [15:16:48] James_F: sounds perfect [15:16:55] Reedy: where's the fun in that [15:17:08] CI is saying no [15:17:17] It's no fun. [15:17:54] Project beta-scap-sync-world build #174764: 04FAILURE in 2 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/174764/ [15:18:25] rsync: getaddrinfo: deployment-deploy04.deployment-prep.eqiad1.wikimedia.cloud 873: Temporary failure in name resolution [15:18:31] i guess the dns issues are back [15:28:18] Yippee, build fixed! [15:28:18] Project beta-scap-sync-world build #174765: 09FIXED in 2 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/174765/ [15:32:20] Yippee, build fixed! [15:32:20] Project beta-update-databases-eqiad build #79358: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/79358/ [15:33:43] ... [15:33:46] Great that the patch isn't merged [15:44:15] 10Release-Engineering-Team (Priority Backlog 📥), 05Release, 05Train Deployments: 1.43.0-wmf.25 deployment blockers - https://phabricator.wikimedia.org/T375656#10199915 (10hashar) 05Open→03Resolved There are some error logs in the backend but nothing that is unusual or that seems to be a concern. I am... [15:55:06] Reedy: we had a fatal earlier too that resolved next run [15:58:02] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org - https://phabricator.wikimedia.org/T374830#10199971 (10hashar) > There... [16:34:06] ah hashar: anything I should take back to mw platform from the train log triage? I guess there might have been some wikitech-related stuff which we should know about but not necessarily act on [16:35:31] apergos: nothing no. I have closed the train blocker :] [16:35:56] there were some issues here and there related to the parser but they got addressed over night by content-transformers [16:36:03] and it lloks like the train is quiet! [16:36:28] ok, that's awesome news! [16:36:38] 🏆 [16:36:58] I am off for dinner! [16:37:16] I should think about dinner "soon" too, enjoy! [17:06:14] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200233 (10RhinosF1) This is a test commebt [17:25:25] 10Continuous-Integration-Config, 10MediaWiki-Vendor, 07Composer, 07PHP 8.4 support: Upgrade composer to 2.8.0+ - https://phabricator.wikimedia.org/T376409 (10Reedy) 03NEW [17:27:58] 10Continuous-Integration-Config, 10MediaWiki-Releasing, 10MediaWiki-Vendor, 07Composer, and 6 others: Upgrade composer from 2.6.4 to 2.7.x - https://phabricator.wikimedia.org/T360973#10200353 (10Reedy) [17:27:59] 10Continuous-Integration-Config, 10MediaWiki-Vendor, 07Composer, 07PHP 8.4 support: Upgrade composer to 2.8.0+ - https://phabricator.wikimedia.org/T376409#10200356 (10Reedy) p:05Triage→03Low [17:35:54] 10Deployments, 10Release-Engineering-Team (Priority Backlog 📥), 07Kubernetes, 13Patch-For-Review: Build and publish multiple MediaWiki production images for a given set of PHP versions - https://phabricator.wikimedia.org/T370934#10200417 (10Scott_French) Test deployment looks good. I'll wait at least a day... [17:42:28] 10Continuous-Integration-Config, 10MediaWiki-Vendor, 07Composer, 07PHP 8.4 support: Upgrade composer to 2.8.0+ - https://phabricator.wikimedia.org/T376409#10200438 (10Reedy) [18:03:16] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200516 (10Dzahn) Will it blend? [18:04:16] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200520 (10Dzahn) response via mail! [18:07:04] brennen: surprise, phab inbound mail is fixed [18:07:06] https://phabricator.wikimedia.org/T356077#10200520 [18:10:34] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200540 (10Dzahn) 05Open→03Resolved @Aklapper surprise, it works :) comment above was a response to my own comment. I had to - enable email in... [18:11:36] 10Phabricator, 06Release-Engineering-Team, 06collaboration-services: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200558 (10Dzahn) @eoghan The actual fix was https://gerrit.wikimedia.org/r/c/operations/puppet/+/1077709/2/modules/phabricator/files/phab_epipe.py [18:15:22] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200565 (10Aklapper) Uh nice, task creation in T376411 also worked out. "Even a blind chicken finds a grain at times", as they sa... [18:17:04] mutante: nice! [18:17:34] ;) [18:18:59] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Python3-Porting: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200582 (10Aklapper) [18:24:25] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Python3-Porting: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10200595 (10Dzahn) It's kind of confusing that the Phabricator emails come **from: (03update) 10dduvall: build-images: Skip JSON/MD5 l10n generation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/461 (https://phabricator.wikimedia.org/T369115) [18:43:55] (03merge) 10dduvall: build-images: Skip JSON/MD5 l10n generation [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/461 (https://phabricator.wikimedia.org/T369115) [18:53:53] (03open) 10dduvall: Release 4.109.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/469 [18:55:35] (03merge) 10dduvall: Release 4.109.0-1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/469 [19:02:05] Reedy: No "make a PHP 8.4 image for CI" task yet? :-) [19:02:33] it's only a week into RC1 ;p [19:02:44] Yes, hence why I'm building them now. [19:03:03] Missing a few things (excimer, redis, etc.) [19:03:12] Surprise surprise. [19:05:27] I filed a task about that ;) [19:09:18] And php-ast doesn't build for it yet. [19:09:40] (03PS1) 10Jforrester: [WIP] Docker: Provide PHP 8.4 images based on 8.4.0-rc1 [integration/config] - 10https://gerrit.wikimedia.org/r/1077780 [19:09:49] But I've pushed what I have, in case someone wants to have fun. [19:18:07] (03close) 10dancy: spiderpig WIP [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/327 [19:20:25] (03open) 10dancy: Drop verify-requirements.txt [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/470 [19:20:27] (03update) 10dancy: Drop verify-requirements.txt [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/470 [19:21:10] (03update) 10dancy: Drop verify-requirements.txt [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/470 [19:23:09] (03close) 10dancy: Drop verify-requirements.txt [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/470 [19:23:33] (03update) 10dancy: WIP: Spiderpig HTTP API [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [19:23:33] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [19:23:33] (03open) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [19:23:39] (03update) 10dancy: WIP: Spiderpig HTTP API [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [19:23:41] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [19:24:04] does whatever it does (tm) [19:25:07] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [19:35:34] (03update) 10dancy: WIP: Spiderpig HTTP API [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/451 [19:35:35] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [20:04:08] (03open) 10reedy: releases.json: Upgrade phpunit/phpunit to 9.6.21 [repos/ci-tools/libup-config] - 10https://gitlab.wikimedia.org/repos/ci-tools/libup-config/-/merge_requests/36 [21:56:59] 06Project-Admins, 07Tracking-Neverending: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706#10201041 (10Ahoelzl) Hello, taking over Sprint / Milestone management from @lbowmaker I need to be able to create milestones for the Data Engineering team. Thi... [23:16:09] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471