[01:21:04] 06Project-Admins: Create #Arabic-Sites tag - https://phabricator.wikimedia.org/T422567 (10Gerges) 03NEW [04:32:35] (03PS1) 10Phedenskog: changelog: Prepare changelog for 1.17.0 [integration/quibble] - 10https://gerrit.wikimedia.org/r/1268704 [04:35:43] (03CR) 10CI reject: [V:04-1] changelog: Prepare changelog for 1.17.0 [integration/quibble] - 10https://gerrit.wikimedia.org/r/1268704 (owner: 10Phedenskog) [05:01:12] (03PS2) 10Phedenskog: changelog: Prepare changelog for 1.17.0 [integration/quibble] - 10https://gerrit.wikimedia.org/r/1268704 [06:23:38] 10Continuous-Integration-Infrastructure, 10Fresh, 13Patch-For-Review: Rebuild integration-agent-qemu-1003 from Bullseye to Bookworm - https://phabricator.wikimedia.org/T422488#11797681 (10hashar) [06:24:57] !log Deleted integration-agent-qemu-1003 Bullseye image, replaced by integration-agent-qemu-1004 which is on Bookworm # T422488 [06:24:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:24:59] T422488: Rebuild integration-agent-qemu-1003 from Bullseye to Bookworm - https://phabricator.wikimedia.org/T422488 [06:27:11] 10Continuous-Integration-Infrastructure, 10Fresh, 13Patch-For-Review: Rebuild integration-agent-qemu-1003 from Bullseye to Bookworm - https://phabricator.wikimedia.org/T422488#11797690 (10hashar) 05Open→03Resolved @dancy you got subscribed because before deleting the integration-agent-qemu-1003 insta... [06:27:12] 06Project-Admins: Create #Arabic-Sites tag - https://phabricator.wikimedia.org/T422567#11797693 (10Aklapper) I am usually reluctant to add more such language tags as I am always worried about people spending time on categorizing for the sake of categorizing. This is a bit similar to Wikimedia sister project tags... [06:27:42] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Rebuild all Jenkins agents VM to Bookworm to support Java 21 - https://phabricator.wikimedia.org/T421114#11797694 (10hashar) [06:31:37] !log Deleted integration-agent-castor05 Bullseye instance, replaced by integration-agent-castor06 which is on Bookworm # T421114 [06:31:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:31:40] T421114: Rebuild all Jenkins agents VM to Bookworm to support Java 21 - https://phabricator.wikimedia.org/T421114 [06:38:51] (03update) 10viktoriahillerudwmse: Added textprocessor to trusted runners [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/162 [06:38:55] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Rebuild all Jenkins agents VM to Bookworm to support Java 21 - https://phabricator.wikimedia.org/T421114#11797705 (10hashar) [06:42:43] 10Continuous-Integration-Infrastructure, 06cloud-services-team, 10Cloud-VPS: Puppet fail to create volume group for ephemeral disk space when it is sda (instead of sdb) - https://phabricator.wikimedia.org/T422258#11797706 (10hashar) [07:15:24] 10Continuous-Integration-Infrastructure, 06cloud-services-team, 10Cloud-VPS: Puppet fail to create volume group for ephemeral disk space when it is sda (instead of sdb) - https://phabricator.wikimedia.org/T422258#11797761 (10hashar) I asked WMCS admins: ` lang=irc hashar: unfortunately the dri... [07:37:39] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, 13Patch-For-Review: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11797856 (10ABran-WMF) Connection reuse has been [[ https://gerrit.wikimedia.org/r/c/operations/puppet/+/1268557... [08:34:36] 10GitLab (Infrastructure), 06collaboration-services: Fix verbous gitlab logging on test instance - https://phabricator.wikimedia.org/T422589 (10Jelto) 03NEW [09:27:23] (03approved) 10jhuneidi: backport.py: Process scap backport --revert changes in right order [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1115 (https://phabricator.wikimedia.org/T388057) (owner: 10dancy) [09:34:40] 10Continuous-Integration-Infrastructure, 06cloud-services-team, 10Cloud-VPS: Puppet fail to create volume group for ephemeral disk space when it is sda (instead of sdb) - https://phabricator.wikimedia.org/T422258#11798108 (10hashar) > Stuff to find: where is `sdb` hardcoded? ;-] So we have in the labs_lvm mo... [10:27:47] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, 13Patch-For-Review: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11798247 (10ArthurTaylor) Got this just now: https://integration.wikimedia.org/ci/job/quibble-with-gated-extensi... [10:37:24] (03PS1) 10Zfilipin: WIP: devcontainer: Add git-review [integration/config] - 10https://gerrit.wikimedia.org/r/1268916 (https://phabricator.wikimedia.org/T422598) [11:28:55] 06Project-Admins: Create #Arabic-Sites tag - https://phabricator.wikimedia.org/T422567#11798533 (10Gerges) Hi @Aklapper, thanks for the thoughtful feedback. The goal of #Arabic-Sites is to make it easier to track tasks that directly impact Arabic Wikimedia projects, as they are currently scattered and hard to fo... [12:11:12] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Fix verbous gitlab logging on test instance - https://phabricator.wikimedia.org/T422589#11798723 (10Jelto) p:05Triage→03Medium a:03Jelto [12:11:40] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, 13Patch-For-Review: gerrit: Adapt timeouts to avoid 502 errors in CI jobs - https://phabricator.wikimedia.org/T421827#11798726 (10ABran-WMF) >>! In T421827#11798247, @ArthurTaylor wrote: > Got this just now: https://integration.wi... [12:42:17] (03PS3) 10Zfilipin: Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) [12:43:00] (03CR) 10CI reject: [V:04-1] Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [12:56:16] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Fix verbous gitlab logging on test instance - https://phabricator.wikimedia.org/T422589#11798983 (10Jelto) The disk on `gitlab-1002` test instance was full again. I disabled rsyslog on the host temporarily using: ` jelto@gitlab-1002:~$... [13:05:42] (03update) 10kamila: Draft: Change base image to use ICU 72 [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/239 (https://phabricator.wikimedia.org/T419049 https://phabricator.wikimedia.org/T421913) [13:08:01] (03PS4) 10Zfilipin: Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) [13:09:33] (03PS5) 10Hashar: Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:17:33] o/, I have a MR on the releng repo, https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/239 , which I will need to merge on a particular date (namely Apr 20). Could I please either get permissions to merge on the repo, or someone's time to push the buttons when I'm about to do the upgrade? [13:26:30] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work, 13Patch-For-Review: Disaster recovery for k8s upgrade - https://phabricator.wikimedia.org/T419580#11799365 (10jnuche) 05Open→03Resolved Tested on `catalyst-dev`: * Created a new pod "A" * Used the new servic... [13:29:34] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Upgrade K3s cluster to most recent stable version - https://phabricator.wikimedia.org/T400077#11799392 (10jnuche) [13:30:07] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Upgrade K3s cluster to most recent stable version - https://phabricator.wikimedia.org/T400077#11799395 (10jnuche) a:03jnuche [13:30:11] (03PS6) 10Hashar: Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:30:39] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: patchdemo staging: Creation of new demos with catalyst backend fails on connecting to db - https://phabricator.wikimedia.org/T421181#11799399 (10jnuche) [13:31:37] (03CR) 10Zfilipin: "The job is deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:40:14] (03PS7) 10Hashar: Use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:40:18] (03PS1) 10Hashar: jjb: add standalone job for GrowthExperiments Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268960 (https://phabricator.wikimedia.org/T419574) [13:40:54] (03PS8) 10Hashar: Zuul: use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:47:47] (03CR) 10Zfilipin: [C:03+1] jjb: add standalone job for GrowthExperiments Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268960 (https://phabricator.wikimedia.org/T419574) (owner: 10Hashar) [13:47:51] (03CR) 10Hashar: [C:03+2] jjb: add standalone job for GrowthExperiments Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268960 (https://phabricator.wikimedia.org/T419574) (owner: 10Hashar) [13:47:59] (03CR) 10Hashar: [C:03+1] "That worked on https://integration.wikimedia.org/ci/job/quibble-with-GrowthExperiments-extensions-browser-tests-only-vendor-php83/ ! So we" [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T419574) (owner: 10Zfilipin) [13:48:39] (03Merged) 10jenkins-bot: jjb: add standalone job for GrowthExperiments Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268960 (https://phabricator.wikimedia.org/T419574) (owner: 10Hashar) [13:51:49] (03PS3) 10Pwangai: phpunit: Support parallel execution for core default suite [integration/quibble] - 10https://gerrit.wikimedia.org/r/1268686 (https://phabricator.wikimedia.org/T422108) [14:12:09] 10GitLab (Infrastructure), 06collaboration-services, 13Patch-For-Review: Fix verbous gitlab logging on test instance - https://phabricator.wikimedia.org/T422589#11799793 (10Jelto) 05Open→03Resolved The patch above fixed the noisy rsyslog configuration on the devtools test instance. The syslog/journal... [14:21:57] 06Release-Engineering-Team, 10MediaWiki-Debug-Logger, 10Wikimedia-Logstash, 07Developer Productivity, 06MediaWiki-Platform-Team (Radar): Logstash dashboard mediawiki-errors lacks "error"-level messages from diagnostic channels - https://phabricator.wikimedia.org/T341815#11799857 (10Krinkle) 05Open→... [14:31:11] gerrit2003:~$ find /srv/gerrit/git/mediawiki/core.git -name '*.pack'|wc -l [14:31:11] 144 [14:31:14] that is quite a few :] [14:31:44] each of them have their own idx [14:31:55] so jgit has to crawls through the idx [14:33:16] the good news is GerritForge is working on adding multi-pack index [14:33:22] MIDX [14:33:23] https://git-scm.com/docs/multi-pack-index [14:34:17] (03merge) 10dancy: Added textprocessor to trusted runners [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/162 (owner: 10viktoriahillerudwmse) [14:34:33] (03update) 10dancy: Added textprocessor to trusted runners [repos/releng/gitlab-trusted-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/162 (owner: 10viktoriahillerudwmse) [14:36:38] next one in number of packfiles: operations/puppet 87, Wikibase 47, cloud/instance-puppet 38 ConfirmEdit 30 etc [14:37:47] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Fix up Gerrit sshd.idleTimeout - https://phabricator.wikimedia.org/T417996#11800009 (10hashar) [14:38:42] Raine: I added you to the project. [14:42:51] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Fix up Gerrit sshd.idleTimeout - https://phabricator.wikimedia.org/T417996#11800045 (10hashar) [14:50:56] dancy: thank you, awesome! [14:58:41] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Gerrit, 06Release-Engineering-Team, 06collaboration-services: Fix up Gerrit sshd.idleTimeout - https://phabricator.wikimedia.org/T417996#11800114 (10hashar) 05Stalled→03Open I have filed this task as a follow up to the incident T417497 (CI losts... [15:24:01] 10Continuous-Integration-Infrastructure (Zuul upgrade), 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 13Patch-For-Review: Fix up Gerrit sshd.idleTimeout - https://phabricator.wikimedia.org/T417996#11800214 (10hashar) [15:40:04] (03PS9) 10Zfilipin: Zuul: use standalone jobs for GrowthExperiments Cypress tests [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T417412) [15:40:36] (03CR) 10Zfilipin: "PS9 points to the correct task." [integration/config] - 10https://gerrit.wikimedia.org/r/1268559 (https://phabricator.wikimedia.org/T417412) (owner: 10Zfilipin) [15:45:42] (03PS1) 10Matthias Mullie: Add WikimediaEvents dependency for ReaderExperiments [integration/config] - 10https://gerrit.wikimedia.org/r/1269006 [15:49:47] (03CR) 10Hashar: [C:03+2] "WikimediaEvents extension.json requires `EventLogging` and `EventBus` which are already dependencies of `WikimediaEvents`." [integration/config] - 10https://gerrit.wikimedia.org/r/1269006 (owner: 10Matthias Mullie) [15:51:31] (03Merged) 10jenkins-bot: Add WikimediaEvents dependency for ReaderExperiments [integration/config] - 10https://gerrit.wikimedia.org/r/1269006 (owner: 10Matthias Mullie) [16:05:00] (03CR) 10Hashar: [C:03+1] "What I am wondering is: **can this change breaks a code path somewhere?** (instead of maybe emitting a warning)." [integration/docroot] - 10https://gerrit.wikimedia.org/r/1267982 (owner: 10Umherirrender) [16:15:56] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Release Pipeline (Blubber): Remove the "blubber" label from Jenkins agents - https://phabricator.wikimedia.org/T422234#11800444 (10hashar) Thank you @dduvall! I have missed PipelineLib treated labels as capabilities when the rest of t... [16:42:37] (03CR) 10Umherirrender: "It can break, as internal functions can now throw TypeError, in that case it the repo should stay as it." [integration/docroot] - 10https://gerrit.wikimedia.org/r/1267982 (owner: 10Umherirrender) [16:42:59] (03Abandoned) 10Umherirrender: Declare strict types on all php files [integration/docroot] - 10https://gerrit.wikimedia.org/r/1267982 (owner: 10Umherirrender) [16:46:22] cccccbukvgbcddrnugcinblnguljhdguebgnkunkdifi [16:46:54] spotted playing vimgolf! [16:53:13] https://bash.toolforge.org/search?q=yubikey [16:57:02] nice one Lucas_WMDE thanks! :) [17:49:13] i have got to get the one where you have to touch 2 contacts [17:52:41] (03PS1) 10Dreamrimmer: zuul: Add Codename Noreste to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1269030 [17:54:50] !log Unblock 167.56.0.0/13 (T422721) [17:54:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:06:18] 10Diffusion, 10Phabricator, 06collaboration-services, 10Wikidata, and 4 others: WikibaseLexeme submodule is not available - https://phabricator.wikimedia.org/T409519#11801200 (10dancy) @alex-mashin Can you retry whatever you were doing again? You'll need to be sure to update your checkout of the mediawiki... [18:08:54] 06Release-Engineering-Team (Priority Backlog 📥), 10Wikifeeds, 07Essential-Work: Wikifeeds CI is not working - https://phabricator.wikimedia.org/T422507#11801207 (10thcipriani) Thanks for jumping on this @dduvall 🏅 [18:43:14] (03update) 10dancy: backport.py: Process scap backport --revert changes in right order [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1115 (https://phabricator.wikimedia.org/T388057) [19:11:36] 10Phabricator, 06Infrastructure-Foundations, 10Mail, 06SRE: Replace Exim on phabricator servers with Postfix - https://phabricator.wikimedia.org/T378029#11801471 (10A_smart_kitten) [19:12:37] (03merge) 10brennen: Support Phabricator integration via "Closes T123456" [repos/releng/gitlab-webhooks] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/37 (owner: 10amastilovic) [19:13:27] 10Phabricator, 06collaboration-services, 06Infrastructure-Foundations, 10Mail, 06SRE: Replace Exim on phabricator servers with Postfix - https://phabricator.wikimedia.org/T378029#11801490 (10Dzahn) [19:25:00] 10GitLab (Integrations), 06Release-Engineering-Team: gitlab-webhooks build fails on "The runtime.txt file isn't supported" - https://phabricator.wikimedia.org/T422734 (10brennen) 03NEW [19:26:25] (03open) 10brennen: Replace runtime.txt with .python-version [repos/releng/gitlab-webhooks] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/39 (https://phabricator.wikimedia.org/T422734) [19:26:50] !log gitlab-webhooks: building & deploying https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/37 - hitting some build tooling stuff, trying a fix per instructions in the error log [19:26:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:27:51] (03merge) 10brennen: Replace runtime.txt with .python-version [repos/releng/gitlab-webhooks] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-webhooks/-/merge_requests/39 (https://phabricator.wikimedia.org/T422734) [19:39:21] 10GitLab (Integrations), 06Release-Engineering-Team: gitlab-webhooks build fails on "The runtime.txt file isn't supported" - https://phabricator.wikimedia.org/T422734#11801569 (10brennen) 05Open→03Resolved a:03brennen [20:06:47] (03update) 10reedy: settings.yaml: Remove StopForumSpam from WMF [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/240 (https://phabricator.wikimedia.org/T422185) [20:08:55] (03merge) 10reedy: settings.yaml: Remove StopForumSpam from WMF [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/240 (https://phabricator.wikimedia.org/T422185) [20:21:37] (03open) 10dancy: Release 4.245.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1122 [20:24:07] (03merge) 10dancy: Release 4.245.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1122 [21:31:49] Project mediawiki-core-doxygen build #19410: 04FAILURE in 13 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/19410/ [22:00:46] (03PS1) 10Ahmon Dancy: dockerfiles/quibble-coverage: Update mediawiki/phpunit-patch-coverage to 0.0.17 [integration/config] - 10https://gerrit.wikimedia.org/r/1269068 (https://phabricator.wikimedia.org/T421689) [22:01:34] !log Unblock 95.216.12.170/32 (T422751) [22:01:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:13:20] (03PS2) 10Ahmon Dancy: dockerfiles/quibble-coverage: Update mediawiki/phpunit-patch-coverage to 0.0.17 [integration/config] - 10https://gerrit.wikimedia.org/r/1269068 (https://phabricator.wikimedia.org/T421689) [22:14:03] 10GitLab (Integrations), 06Release-Engineering-Team, 06cloud-services-team, 10Toolforge: gitlab-webhooks build fails on "The runtime.txt file isn't supported" - https://phabricator.wikimedia.org/T422734#11802149 (10bd808) Adding the #toolforge tag in case someone searches for this general problem only... [22:15:28] (03CR) 10Ahmon Dancy: [C:03+2] dockerfiles/quibble-coverage: Update mediawiki/phpunit-patch-coverage to 0.0.17 [integration/config] - 10https://gerrit.wikimedia.org/r/1269068 (https://phabricator.wikimedia.org/T421689) (owner: 10Ahmon Dancy) [22:17:19] (03Merged) 10jenkins-bot: dockerfiles/quibble-coverage: Update mediawiki/phpunit-patch-coverage to 0.0.17 [integration/config] - 10https://gerrit.wikimedia.org/r/1269068 (https://phabricator.wikimedia.org/T421689) (owner: 10Ahmon Dancy) [22:19:02] !log Updating docker-pkg files on contint primary for https://gerrit.wikimedia.org/r/c/integration/config/+/1269068 [22:19:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:20:31] (03update) 10jhuneidi: backport.py: Process scap backport --revert changes in right order [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/1115 (https://phabricator.wikimedia.org/T388057) (owner: 10dancy) [22:25:05] (03PS1) 10Ahmon Dancy: jjb: Bump quibble-coverage to 1.16.0-s3 [integration/config] - 10https://gerrit.wikimedia.org/r/1269070 (https://phabricator.wikimedia.org/T421689) [22:32:04] Yippee, build fixed! [22:32:05] Project mediawiki-core-doxygen build #19411: 09FIXED in 14 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/19411/