[08:38:32] (03CR) 10Hashar: "I am not sure I see the point in upgrading the repo which is already using PHPUnit 9.6.x. I'd like to avoid useless commits spam :)" [integration/docroot] - 10https://gerrit.wikimedia.org/r/1099256 (owner: 10Libraryupgrader) [08:55:48] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs failing with: Could not resolve host: gerrit.wikimedia.org (2024-11-27) - https://phabricator.wikimedia.org/T380991#10370170 (10hashar... [08:57:32] (03update) 10jnuche: install-world: rework installer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/601 (https://phabricator.wikimedia.org/T380772) [08:57:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10370168 [09:14:34] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Quibble: Speed up git clone operation in wmf-quibble job - https://phabricator.wikimedia.org/T374717#10370221 (10hashar) [09:16:42] 10Continuous-Integration-Infrastructure, 10Quibble: Add support for multi-DB testing to MediaWiki - https://phabricator.wikimedia.org/T379925#10370225 (10kostajh) >>! In T379925#10362231, @hashar wrote: > For T342763, @Urbanecm proposed a patch to Quibble to have it spin up a second database ( https://gerrit.w... [09:28:12] 06Release-Engineering-Team, 10MediaWiki-Core-Tests, 10Wikidata, 10wmde-wikidata-tech, and 4 others: Create a daily job running core + extension PHPUnit tests serially - https://phabricator.wikimedia.org/T372618#10370239 (10karapayneWMDE) [09:32:46] !log deployment-prep: pruning branches and tags from the git repositories under /srv/mediawiki-staging [09:32:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:38:14] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Quibble: Speed up git clone operation in wmf-quibble job - https://phabricator.wikimedia.org/T374717#10370595 (10hashar) For mediawiki/extensions/WikimediaMessages taking 13.395, that is when resolving the deltas. It is ~ 50MB, I will add... [10:41:49] 06Release-Engineering-Team, 06serviceops: legalteam wiki reliably returns 500s - https://phabricator.wikimedia.org/T381252 (10Ladsgroup) 03NEW [10:42:44] 06Release-Engineering-Team, 06serviceops: legalteam wiki reliably returns 500s - https://phabricator.wikimedia.org/T381252#10370615 (10Ladsgroup) →14Duplicate dup:03T381250 [10:42:53] 06Release-Engineering-Team, 06serviceops: legalteam wiki reliably returns 500s - https://phabricator.wikimedia.org/T381252#10370618 (10Clement_Goubert) →14Duplicate dup:03T381250 [11:02:01] everything is slow this morning [11:02:02] :/ [11:07:27] !log integration: rebooted integration-puppetserver-01 , stall waiting for IO after having cherry picked a puppet.git patch [11:07:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:08:33] Failed to open TCP connection to integration-puppetserver-01.integration.eqiad1.wikimedia.cloud:8140 (Connection refused - connect(2) for "integration-puppetserver-01.integration.eqiad1.wikimedia.cloud" port 8140) [11:08:37] The Sigh ™ [11:21:47] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Quibble, 13Patch-For-Review: Speed up git clone operation in wmf-quibble job - https://phabricator.wikimedia.org/T374717#10370843 (10hashar) After having cherry picked the Puppet patch on the integration project, I gave it a try on `int... [11:48:50] 10Phabricator: Various Phabricator search options error out with PhabricatorTypeaheadInvalidTokenException - https://phabricator.wikimedia.org/T381196#10370953 (10Aklapper) →14Duplicate dup:03T217984 [11:48:51] 10Phabricator (Upstream), 07Upstream: Listing tickets "Closed By: Current Viewer" while logged out causes PhabricatorTypeaheadInvalidTokenException: This datasource ("PhabricatorPeopleUserFunctionDatasource") can not evaluate the function "viewer(...)". - https://phabricator.wikimedia.org/T217984#10370955 (10Ak... [12:08:29] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Quibble, 13Patch-For-Review: Speed up git clone operation in wmf-quibble job - https://phabricator.wikimedia.org/T374717#10371047 (10kostajh) >>! In T374717#10370843, @hashar wrote: > After having cherry picked the Puppet patch on the i... [12:31:56] 10Deployments, 06Release-Engineering-Team, 06serviceops, 07Wikimedia-production-error: httpb fails upon deployment of 1.44.0-wmf.5 - https://phabricator.wikimedia.org/T380958#10371128 (10Clement_Goubert) >>! In T381250#10370809, @Ladsgroup wrote: > Now erroring with: > ` > 11:13:16 Check 'check_testservers... [13:57:35] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move commons-query.wikimedia.org and query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10371358 (10Jelto) Great thanks, I sent you an invite for tomorrow 14:00 CET. [14:04:33] (03CR) 10Hashar: [C:03+2] Add NotifyMe as a dependency of BlueSpiceBookshelf & BlueSpicePageAssignments [integration/config] - 10https://gerrit.wikimedia.org/r/1098576 (owner: 10Hslater) [14:06:17] (03Merged) 10jenkins-bot: Add NotifyMe as a dependency of BlueSpiceBookshelf & BlueSpicePageAssignments [integration/config] - 10https://gerrit.wikimedia.org/r/1098576 (owner: 10Hslater) [14:06:40] (03CR) 10Hashar: [C:03+2] "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/1098576 (owner: 10Hslater) [14:17:15] (03open) 10jnuche: add a secondary deployment server [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/43 (https://phabricator.wikimedia.org/T380772) [14:18:45] (03merge) 10jnuche: add a secondary deployment server [repos/releng/scap3-dev] - 10https://gitlab.wikimedia.org/repos/releng/scap3-dev/-/merge_requests/43 (https://phabricator.wikimedia.org/T380772) [14:43:02] Project beta-code-update-eqiad build #524480: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524480/ [14:50:59] (03CR) 10Hashar: [C:03+2] Enable PHPUnit Parallel tests for WikiLambda [integration/config] - 10https://gerrit.wikimedia.org/r/1098921 (https://phabricator.wikimedia.org/T375551) (owner: 10Arthur taylor) [14:52:20] (03Merged) 10jenkins-bot: Enable PHPUnit Parallel tests for WikiLambda [integration/config] - 10https://gerrit.wikimedia.org/r/1098921 (https://phabricator.wikimedia.org/T375551) (owner: 10Arthur taylor) [14:53:03] Project beta-code-update-eqiad build #524481: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524481/ [14:57:32] (03PS2) 10Hashar: Remove zotero pipeline for archival [integration/config] - 10https://gerrit.wikimedia.org/r/1093879 (https://phabricator.wikimedia.org/T380259) (owner: 10Mvolz) [14:58:27] (03CR) 10Hashar: [C:03+2] "I have amended the change to mark the repository as archived in Zuul by using:" [integration/config] - 10https://gerrit.wikimedia.org/r/1093879 (https://phabricator.wikimedia.org/T380259) (owner: 10Mvolz) [15:00:22] (03Merged) 10jenkins-bot: Remove zotero pipeline for archival [integration/config] - 10https://gerrit.wikimedia.org/r/1093879 (https://phabricator.wikimedia.org/T380259) (owner: 10Mvolz) [15:01:08] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371648 (10Mvolz) [15:01:13] (03PS1) 10Hashar: zuul: mark zotero as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1099716 (https://phabricator.wikimedia.org/T380259) [15:01:24] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371651 (10Mvolz) [15:01:27] (03CR) 10Hashar: [C:03+2] zuul: mark zotero as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1099716 (https://phabricator.wikimedia.org/T380259) (owner: 10Hashar) [15:02:10] (03PS1) 10Hashar: Review access change [services/zotero] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1099717 [15:02:34] (03PS2) 10Hashar: Archive repository [services/zotero] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1099717 (https://phabricator.wikimedia.org/T380259) [15:02:45] (03CR) 10Hashar: [V:03+2 C:03+2] Archive repository [services/zotero] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1099717 (https://phabricator.wikimedia.org/T380259) (owner: 10Hashar) [15:03:04] Project beta-code-update-eqiad build #524482: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524482/ [15:03:09] (03Merged) 10jenkins-bot: zuul: mark zotero as archived [integration/config] - 10https://gerrit.wikimedia.org/r/1099716 (https://phabricator.wikimedia.org/T380259) (owner: 10Hashar) [15:13:06] Project beta-code-update-eqiad build #524483: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524483/ [15:15:52] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371791 (10Mvolz) [15:16:06] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371793 (10hashar) [15:17:55] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371841 (10Mvolz) [15:21:48] 10GitLab (Pipeline Services Migration🐤), 10Citoid, 06Editing-team, 10Projects-Cleanup, 13Patch-For-Review: Archive Zotero gerrit repo as it's been replaced with gitlab - https://phabricator.wikimedia.org/T380259#10371796 (10hashar) 05Open→03Resolved a:03Mvolz I have applied the change for CI, f... [15:23:07] Project beta-code-update-eqiad build #524484: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524484/ [15:30:39] Yippee, build fixed! [15:30:39] Project mediawiki-core-doxygen build #6311: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen/6311/ [15:31:21] Yippee, build fixed! [15:31:21] Project beta-update-databases-eqiad build #80800: 09FIXED in 11 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/80800/ [15:33:08] Project beta-code-update-eqiad build #524485: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524485/ [15:40:03] maintenance-disconnect-full-disks build 654660 integration-agent-docker-1046 (/: 27%, /srv: 95%, /var/lib/docker: 34%): OFFLINE due to disk space [15:43:09] Project beta-code-update-eqiad build #524486: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524486/ [15:45:03] maintenance-disconnect-full-disks build 654661 integration-agent-docker-1046 (/: 27%, /srv: 43%, /var/lib/docker: 32%): RECOVERY disk space OK [15:47:40] stop job > Jenkins agent emit a SIGTERM to scap > scap exits immediately > lock file is left behind [15:48:36] !log deployment-prep: manually removed scap lock on deployment-deploy04.deployment-prep.eqiad1.wikimedia.cloud [15:48:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:50:03] maintenance-disconnect-full-disks build 654662 integration-agent-docker-1049 (/: 26%, /srv: 97%, /var/lib/docker: 32%): OFFLINE due to disk space [15:53:10] Project beta-code-update-eqiad build #524487: 04STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524487/ [15:54:38] it is running [15:55:03] maintenance-disconnect-full-disks build 654663 integration-agent-docker-1049 (/: 26%, /srv: 21%, /var/lib/docker: 30%): RECOVERY disk space OK [15:55:11] Yippee, build fixed! [15:55:11] Project beta-code-update-eqiad build #524488: 09FIXED in 2 min 0 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/524488/ [15:59:37] (03merge) 10dancy: gitlab-runner: bump image version to alpine-v17.4.2 [repos/releng/gitlab-cloud-runner] - 10https://gitlab.wikimedia.org/repos/releng/gitlab-cloud-runner/-/merge_requests/415 (https://phabricator.wikimedia.org/T380906) (owner: 10jelto) [16:07:45] Yippee, build fixed! [16:07:45] Project beta-scap-sync-world build #183202: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/183202/ [16:22:45] (03merge) 10dancy: Remove check for agreement between all.dblist and wikiversions.json [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/600 (https://phabricator.wikimedia.org/T352113) (owner: 10tstarling) [16:27:00] (03open) 10dancy: Release 4.130.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/602 [16:28:03] (03merge) 10dancy: Release 4.130.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/602 [16:31:07] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10372189 [16:58:05] andre: thcipriani: there is nobody running the train this week ? [ https://phabricator.wikimedia.org/T375665 is unassigned ] [16:58:46] I can pick it up if needed, I don't have a long travel ahead [17:03:21] hashar: the train is self driving this week [17:03:29] It's all the rage in Europe [17:03:32] ahhh that is why [17:03:59] ssh screen journalctl -u mediawiki-train --follow [17:31:55] hashar: I can also do it, or be backup I guess? [17:32:05] shrug [17:46:03] (03open) 10dancy: Revert "Clear stdin before reading user input" [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/603 [17:46:06] (03update) 10dancy: Revert "Clear stdin before reading user input" [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/603 [17:46:45] (03update) 10dancy: Revert "Clear stdin before reading user input" [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/603 [17:46:50] (03update) 10dancy: Revert "Clear stdin before reading user input" [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/603 [17:48:05] (03merge) 10dancy: Revert "Clear stdin before reading user input" [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/603 [17:48:31] (03open) 10dancy: Release 4.131.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/604 [17:49:55] (03merge) 10dancy: Release 4.131.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/604 [18:29:35] (03PS1) 10Pppery: Zuul: Add RV1971 to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1099765 [18:29:42] 10Phabricator, 06collaboration-services, 06Trust-and-Safety, 10Znuny: Retire legalpad phabricator/phorge application? - https://phabricator.wikimedia.org/T363009#10372815 (10Dzahn) a:05Dzahn→03None [18:30:52] (03PS2) 10Pppery: Zuul: Add RV1971 to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1099765 [18:52:44] 10Phabricator: Phabricator 2FA reset for Atieno Njira - https://phabricator.wikimedia.org/T381306 (10MBywater-WMF) 03NEW [19:24:58] hi folks, does anyone know why I can't merge this patch? It's passing on C+2 and V+2 but it's not auto-merging and I can't manually submit it :( https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/1099178 [19:28:15] jenkins has merged it though.. [20:45:44] (03approved) 10dancy: install-world: rework installer [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/601 (https://phabricator.wikimedia.org/T380772) (owner: 10jnuche) [22:01:02] 10Beta-Cluster-Infrastructure, 13Patch-For-Review, 07Spike: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752#10373801 (10bd808) I did some crawling around in various running WMCS instances and configuration repositories to try and figure out how th... [22:22:59] I am about to try a config change for the beta cluster that will either be very boring or cause mass catastrophe until I revert it. Specifically I am going to route the CDN edge MediaWiki traffic to the deployment-mediawiki81.deployment-prep.eqiad1.wikimedia.cloud testing instance. This instance is running PHP 8.1 as the POC for T378752. [22:23:00] T378752: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752 [22:27:53] !log Set ATS default MediaWiki backend to deployment-mediawiki81.deployment-prep.eqiad1.wikimedia.cloud via "deployment-cache-text" prefix hiera rules in Horizon (T378752) [22:27:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:29:00] !log https://meta.wikimedia.beta.wmflabs.org/wiki/Special:Version is showing PHP 8.1.31 (T378752) [22:29:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:29:03] T378752: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752 [22:42:47] !log Running https://integration.wikimedia.org/ci/view/selenium-daily/job/selenium-daily-beta-MediaWiki/2382/console against thePHP 8.1 MediaWiki (T378752) [22:42:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:42:50] T378752: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752 [22:57:17] !log Reverted ATS default MediaWiki backend to deployment-mediawiki14.deployment-prep.eqiad1.wikimedia.cloud via "deployment-cache-text" prefix hiera rules in Horizon (T378752) [22:57:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:57:21] T378752: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752 [23:05:56] 10Beta-Cluster-Infrastructure, 13Patch-For-Review, 07Spike: Figure out how to install PHP 8.1 on bullseye MediaWiki instances - https://phabricator.wikimedia.org/T378752#10374042 (10bd808) With some help from @thcipriani the PHP 8.1 test was a success. The deployment-mediawiki81.deployment-prep.eqiad1.wikime... [23:26:26] 10Release-Engineering-Team (Doing 😎), 05Release, 05Train Deployments: 1.44.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T375665#10374127 (10thcipriani) p:05Triage→03Medium a:03jeena