[01:05:12] (03PS3) 10Jack Phoenix: More phan dependencies for social tools [integration/config] - 10https://gerrit.wikimedia.org/r/691323 [01:36:47] RECOVERY - SSH on contint2001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [07:01:59] 10Release-Engineering-Team, 10GitLab (Initialization), 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10Peter) In the performance team we have a couple simple repos that I think could be a good candidate to move first: performance/mobile-synthetic-monitoring... [07:42:27] 10Continuous-Integration-Infrastructure, 10Zuul, 10Upstream: "This change depends on a change that failed to merge." Please say which change? - https://phabricator.wikimedia.org/T201505 (10hashar) 05Open→03Declined https://review.opendev.org/c/zuul/zuul/+/707620 that one is slightly different. Is when... [07:46:28] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Zuul, 10Epic: Upgrade to Zuul3 - https://phabricator.wikimedia.org/T186426 (10hashar) 05Open→03Declined [08:52:49] (03PS1) 10Volans: Zuul: [operations/software/netbox-extras] add gate and sumbit [integration/config] - 10https://gerrit.wikimedia.org/r/692256 (https://phabricator.wikimedia.org/T264106) [09:11:26] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Aklapper) So this is about sending an `MediaWiki 1.31 soon to be End of Life` heads-up email out to a bunch of lists, like "Please upgrade to the 1.35 LTS version... [09:18:47] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Doing), 10Zuul, and 2 others: Zuul can't stop jobs or set the build description - https://phabricator.wikimedia.org/T281737 (10hashar) 05Open→03Resolved The issue hasn't occurred since I did the upgrade on May 11th, I am th... [09:43:23] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Radar): Restore IRC alerts for beta-scap-eqiad job - https://phabricator.wikimedia.org/T271122 (10hashar) I have looked a bit about it during the week-end, but from my notes above I have not been able to retrieve the password :-\... [09:58:06] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Doing): Restore IRC alerts for beta-scap-eqiad job - https://phabricator.wikimedia.org/T271122 (10hashar) 05Stalled→03Resolved a:03hashar I could not retrieve the password, but after upgrading the plugins it works just fine... [10:06:34] Project mediawiki-core-doxygen-docker build #24955: 04STILL FAILING in 1 min 48 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24955/ [10:10:09] 10Project-Admins, 10Wikimedia-Extension-setup: "Wikimedia-Extension-setup" and "Wikimedia-Site-Requests" projects overlap - Archive Wikimedia-Extension-setup - https://phabricator.wikimedia.org/T230194 (10Zabe) >>! In T230194#6771740, @DannyS712 wrote: > Personally, I agree that the project can probably be arc... [10:21:45] Project beta-update-databases-eqiad build #50434: 04STILL FAILING in 1 min 45 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50434/ [10:22:53] * Majavah looking at beta-update-databases-eqiad [10:28:25] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10Majavah) [11:05:53] Project mediawiki-core-doxygen-docker build #24956: 04STILL FAILING in 1 min 25 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24956/ [11:21:54] Project beta-update-databases-eqiad build #50435: 04STILL FAILING in 1 min 53 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50435/ [11:46:12] 10Project-Admins: Replace tracking bug T5646 by new project tag "Feeds" - https://phabricator.wikimedia.org/T102495 (10Aklapper) Proposing to decline; looks like nobody is interested enough in that task to consistently triage every incoming ticket and add that parent task dependency. Might be another example of... [11:55:26] 10Project-Admins, 10Epic: Replace all "tracking" tasks with tags/projects (if unbounded), relabel them to outcome tasks (if bounded), or kill them entirely (if pointless) - https://phabricator.wikimedia.org/T192655 (10Aklapper) [11:55:34] 10Project-Admins: Replace tracking bug T21719 by new project tag "HTML5" - https://phabricator.wikimedia.org/T102502 (10Aklapper) 05Open→03Resolved a:03Aklapper #HTML5 created in https://phabricator.wikimedia.org/project/profile/5351/ ; T21719 closed as invalid with a comment. [12:00:32] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Radar), 10Upstream: ircbot-plugin emits SEVERE java.lang.IndexOutOfBoundsException when parsing AWAY IRCv3 notifications - https://phabricator.wikimedia.org/T283009 (10hashar) [12:01:50] 10Project-Admins: Rename Wikimedia project tags in Phab (#Wikisource; #Wiktionary) to clarify that they are not about language specific instances? - https://phabricator.wikimedia.org/T196740 (10Aklapper) a:03Aklapper I boldly renamed; see last comment. [12:05:53] Project mediawiki-core-doxygen-docker build #24957: 04STILL FAILING in 1 min 25 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24957/ [12:07:39] 10Project-Admins: Rename Wikimedia project tags in Phab (#Wikisource; #Wiktionary) to clarify that they are not about language specific instances? - https://phabricator.wikimedia.org/T196740 (10Billinghurst) >>! In T196740#7092564, @Aklapper wrote: > I boldly renamed; see last comment. and a naive question, wil... [12:16:53] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Radar), 10Upstream: ircbot-plugin emits SEVERE java.lang.IndexOutOfBoundsException when parsing AWAY IRCv3 notifications - https://phabricator.wikimedia.org/T283009 (10hashar) p:05Triage→03Low [12:17:06] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (Doing): Restore IRC alerts for beta-scap-eqiad job - https://phabricator.wikimedia.org/T271122 (10hashar) And to confirm, the bot did notify about a build failure: ` lang=irc [11:20:11] Project beta-update-databases... [12:19:08] 10Project-Admins: There should be wikibooks, wikiquote and wikivoyage community projects - https://phabricator.wikimedia.org/T93486 (10Aklapper) >>! In T93486#2253360, @Danny_B wrote: > Some use cases: > - find out config requests/changes related to such project Config changes for Wikimedia sites can already be... [12:19:33] 10Project-Admins: There should be wikibooks, wikiquote and wikivoyage community projects - https://phabricator.wikimedia.org/T93486 (10Aklapper) In general, as I tried to express in T93486#1140230: If interest by a specific group is expressed and that group will maintain the Phabricator project and look at its t... [12:21:37] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Formally EOL REL1_31 - https://phabricator.wikimedia.org/T279858 (10Aklapper) [12:21:39] 10Project-Admins, 10MSSQL: Archive MSSQL project - https://phabricator.wikimedia.org/T230583 (10Aklapper) [12:21:41] 10Project-Admins, 10Oracle Database: Archive Oracle Database project - https://phabricator.wikimedia.org/T230582 (10Aklapper) [12:21:43] Project beta-update-databases-eqiad build #50436: 04STILL FAILING in 1 min 42 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50436/ [12:36:43] 10Phabricator, 10Developer-Advocacy (Jul-Sep 2021), 10PM: Decrease issues created many years ago with no recent activity (aka stale tickets) - https://phabricator.wikimedia.org/T248034 (10Aklapper) Quoting from GitLab's concept at https://about.gitlab.com/handbook/engineering/quality/triage-operations/#auto-... [12:52:42] (03CR) 10Jgleeson: "> Patch Set 1: Code-Review-1" [integration/config] - 10https://gerrit.wikimedia.org/r/689947 (https://phabricator.wikimedia.org/T277500) (owner: 10Jforrester) [12:57:14] (03CR) 10Lars Wirzenius: "Pushing new version soon." (036 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/688351 (owner: 10Lars Wirzenius) [13:05:25] 10Beta-Cluster-Infrastructure, 10User-Majavah: Fix scap canary checks on beta cluster (Check failed: returned 302, expecting: 200) - https://phabricator.wikimedia.org/T278686 (10Majavah) a:03Majavah [13:06:11] Project mediawiki-core-doxygen-docker build #24958: 04STILL FAILING in 1 min 42 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24958/ [13:13:40] 10Release-Engineering-Team, 10GitLab (Initialization), 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10Mglaser) BlueSpice would also love to take a part in the early adoption. We can test with Extension:Privacy. [13:21:58] Project beta-update-databases-eqiad build #50437: 04STILL FAILING in 1 min 58 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50437/ [13:26:37] (03CR) 10Hashar: [C: 03+2] Zuul: [operations/software/netbox-extras] add gate and sumbit [integration/config] - 10https://gerrit.wikimedia.org/r/692256 (https://phabricator.wikimedia.org/T264106) (owner: 10Volans) [13:28:15] (03Merged) 10jenkins-bot: Zuul: [operations/software/netbox-extras] add gate and sumbit [integration/config] - 10https://gerrit.wikimedia.org/r/692256 (https://phabricator.wikimedia.org/T264106) (owner: 10Volans) [13:31:07] (03CR) 10Hashar: "Deployed!" [integration/config] - 10https://gerrit.wikimedia.org/r/692256 (https://phabricator.wikimedia.org/T264106) (owner: 10Volans) [13:32:33] hashar: hi! is there anything I can do to get T277078 moving? [13:32:34] T277078: Support Cinder for CI docker workers - https://phabricator.wikimedia.org/T277078 [13:37:52] 10Project-Admins: Rename Wikimedia project tags in Phab (#Wikisource; #Wiktionary) to clarify that they are not about language specific instances? - https://phabricator.wikimedia.org/T196740 (10Aklapper) Try and find out? :P There are aliases. [13:44:35] (03CR) 10Volans: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/692256 (https://phabricator.wikimedia.org/T264106) (owner: 10Volans) [13:49:41] (03PS5) 10Lars Wirzenius: WIP: port to Python 3, drop support for Python 2 [tools/scap] - 10https://gerrit.wikimedia.org/r/688351 [14:06:12] Project mediawiki-core-doxygen-docker build #24959: 04STILL FAILING in 1 min 44 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24959/ [14:16:29] (03PS1) 10Hashar: dockerfiles: add php-intl for doxygen [integration/config] - 10https://gerrit.wikimedia.org/r/692334 (https://phabricator.wikimedia.org/T280810) [14:17:59] (03CR) 10Hashar: "docker-pkg fails:" [integration/config] - 10https://gerrit.wikimedia.org/r/692334 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [14:20:21] 10Beta-Cluster-Infrastructure, 10User-Majavah: Fix scap canary checks on beta cluster (Check failed: returned 302, expecting: 200) - https://phabricator.wikimedia.org/T278686 (10Majavah) 05Open→03Resolved [14:20:31] 10Beta-Cluster-Infrastructure, 10User-Majavah: Investigate setting up HTTPS directly on beta appservers - https://phabricator.wikimedia.org/T206158 (10Majavah) [14:21:43] Project beta-update-databases-eqiad build #50438: 04STILL FAILING in 1 min 42 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50438/ [14:22:17] (03CR) 10Hashar: [C: 03+2] "Worked with docker-pkg 2.1.0:" [integration/config] - 10https://gerrit.wikimedia.org/r/692334 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [14:23:16] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) [14:23:26] (03Merged) 10jenkins-bot: dockerfiles: add php-intl for doxygen [integration/config] - 10https://gerrit.wikimedia.org/r/692334 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [14:25:15] (03PS1) 10Hashar: jjb: update Doxygen jobs with php-intl [integration/config] - 10https://gerrit.wikimedia.org/r/692335 (https://phabricator.wikimedia.org/T280810) [14:29:36] (03CR) 10Hashar: "Image has been created Successfully tagged docker-registry.discovery.wmnet/releng/doxygen:0.9.1" [integration/config] - 10https://gerrit.wikimedia.org/r/692335 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [14:32:46] 10Phabricator, 10Security-Team: Enable risk rating field in Phabricator's task form - https://phabricator.wikimedia.org/T279690 (10sbassett) @mmodell - > @sbassett: To change a task Using the "Add Action" dropdown on any task, just select "change subtype" and then select the security task subtype from the drop... [14:48:11] (03PS1) 10Krinkle: Disable noise from broken Sonar jobs (keep in experimental) [integration/config] - 10https://gerrit.wikimedia.org/r/692342 (https://phabricator.wikimedia.org/T280669) [14:48:54] 10Release-Engineering-Team, 10GitLab (Initialization), 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10Milimetric) https://stats.wikimedia.org/ runs https://gerrit.wikimedia.org/r/admin/repos/analytics/wikistats2 and we'd be happy to migrate to GitLab. We... [14:49:12] (03CR) 10jerkins-bot: [V: 04-1] Disable noise from broken Sonar jobs (keep in experimental) [integration/config] - 10https://gerrit.wikimedia.org/r/692342 (https://phabricator.wikimedia.org/T280669) (owner: 10Krinkle) [14:55:13] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Code-Health: mwcore-codehealth-patch always failing due to OOM when running PHPUnit tests - https://phabricator.wikimedia.org/T282142 (10kostajh) [14:58:16] (03PS2) 10Krinkle: Disable noise from broken Sonar jobs (keep in experimental) [integration/config] - 10https://gerrit.wikimedia.org/r/692342 (https://phabricator.wikimedia.org/T280669) [14:59:36] kostajh: mind if I merge that the other way around? [14:59:50] (older task, I can quote Daimona's detail) [15:00:33] Krinkle: sure, fine with me [15:00:51] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Code-Health: mwcore-codehealth-patch always failing due to OOM when running PHPUnit tests - https://phabricator.wikimedia.org/T282142 (10Krinkle) [15:01:46] kostajh: do you think a fix is immenent or disabling fine for now? [15:02:22] imminent [15:04:00] Krinkle: I can make the patches needed to bump memory limit but I think that is probably more on h.ashar to decide [15:06:25] Project mediawiki-core-doxygen-docker build #24960: 04STILL FAILING in 1 min 57 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24960/ [15:09:46] (03PS1) 10Kosta Harlan: mediawiki-coverage: Bump memory limit [integration/config] - 10https://gerrit.wikimedia.org/r/692343 (https://phabricator.wikimedia.org/T280669) [15:13:05] (03PS1) 10Kosta Harlan: quibble-buster-php73-coverage: Bump image version [integration/config] - 10https://gerrit.wikimedia.org/r/692344 (https://phabricator.wikimedia.org/T280669) [15:13:45] (03CR) 10Hashar: [C: 03+2] "Jobs updated" [integration/config] - 10https://gerrit.wikimedia.org/r/692335 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [15:14:41] (03PS1) 10Kosta Harlan: jjb: Switch to newer coverage images [integration/config] - 10https://gerrit.wikimedia.org/r/692345 (https://phabricator.wikimedia.org/T280669) [15:14:49] Krinkle: patches are up [15:14:56] (03Merged) 10jenkins-bot: jjb: update Doxygen jobs with php-intl [integration/config] - 10https://gerrit.wikimedia.org/r/692335 (https://phabricator.wikimedia.org/T280810) (owner: 10Hashar) [15:16:02] (03PS2) 10Kosta Harlan: mediawiki-coverage: Bump memory limit [integration/config] - 10https://gerrit.wikimedia.org/r/692343 (https://phabricator.wikimedia.org/T280669) [15:21:44] Project beta-update-databases-eqiad build #50439: 04STILL FAILING in 1 min 43 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50439/ [15:29:28] Yippee, build fixed! [15:29:29] Project mediawiki-core-doxygen-docker build #24961: 09FIXED in 16 min: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/24961/ [15:33:37] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T281146 (10dancy) wmf.5 is now deployed to all wikis. [15:34:08] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T281146 (10dancy) [15:38:51] 10LibUp, 10Maps (Kartographer): libup patch sets for Kartographer extensions gets unwanted changes to excluded lib folder - https://phabricator.wikimedia.org/T282952 (10Volker_E) Asking again here, if revert is necessary, let's go for it?! I thought that I checked via `git blame` before proposing the patch an... [15:58:31] 10Release-Engineering-Team, 10Scap: Fix or drop disabled unit tests in Scap - https://phabricator.wikimedia.org/T283018 (10LarsWirzenius) [16:04:49] 10Release-Engineering-Team, 10GitLab (Initialization), 10Software-Licensing, 10User-brennen: Add LICENSE/COPYING to operations/gitlab-ansible - https://phabricator.wikimedia.org/T282496 (10Sergey.Trofimovsky.SF) Here's the LICENSE file finally in the root of the repo, please review: https://gerrit.wikimedi... [16:17:41] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) [16:18:21] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) Added this to the tracker task, since we should clearly only deploy this t... [16:21:49] Project beta-update-databases-eqiad build #50440: 04STILL FAILING in 1 min 49 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50440/ [16:22:39] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) I took the opportunity to split logstash and the kafka broker to different hosts, so I created deployment-logstash04 and deployment-kafka-logging01. Now having some issues with Ka... [16:29:45] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) a:03jsn.sherman @Samwalton9 Per https://www.mediawiki.org/wiki/Writing_... [16:30:21] 10MediaWiki-Releasing, 10Release-Engineering-Team (Radar), 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10sbassett) 05Open→03Stalled Stalling on need to answer questions in T256285#6255281. [16:30:31] 10MediaWiki-Releasing, 10Release-Engineering-Team (Radar), 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10sbassett) p:05Triage→03Low [16:38:06] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Ottomata) IIRC: The broker certs don't have hostnames in them; they all use the same cert signed by the same CA. In labs-private repo, I see modules/secret/secrets/certificates/certificat... [16:49:30] 10LibUp, 10Maps (Kartographer), 10Patch-For-Review: libup patch sets for Kartographer extensions gets unwanted changes to excluded lib folder - https://phabricator.wikimedia.org/T282952 (10Legoktm) 05Open→03Resolved a:03Legoktm [16:50:46] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) I'm not sure of the best way to go find which extens... [16:51:23] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) > IIRC: The broker certs don't have hostnames in them; they all use the same cert signed by the same CA. That's what the certs look like, yeah. Logstash doesn't accept them becau... [16:55:42] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Ottomata) Huh, in prod that is not needed. Perhaps this is a CA issue? IIRC (and it has been a while since I thought about this) The CA used to sign the Kafka broker certs has to be the... [16:56:24] (03PS3) 10Jack Phoenix: Enable phan for some extensions [integration/config] - 10https://gerrit.wikimedia.org/r/692069 [16:57:16] Hello! Could I request some help with SSH access to bastions? [16:57:45] I have a configuration that worked a month ago, but now I can't access `bast1002.wikimedia.org` [16:58:28] apine: bast1002 was replaced by bast1003 some time ago, you likely just missed the email about it [16:58:35] Oohh, got it. [16:59:47] Yes, I definitely don't have that email in my trash or anything. More specifically, I am trying to log in to `stat1006.equiad.wmnet`, which automatically routes through `bast1002` at the moment. Do you have migration instructions handy? [16:59:59] ottomata: re T283013 - do you have any suggestions on how to debug that? I have never touched kafka or logstash before so have no idea where to approach that [16:59:59] T283013: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 [17:00:41] apine: 1) are you on the ops@ mailing list, 2) just switch bast1002 -> bast1003 in your ~/.ssh/config [17:01:51] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) oh, duh, the best place to look is probably the file... [17:03:34] legoktm: 1) I am not; I shall join. 2) That improved things! Now I get a `stdio forwarding failed` error, which I can hopefully debug? [17:04:11] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10Legoktm) @jsn.sherman I would suggest doing something like https:... [17:04:28] you have a typo in the hostname (equiad instead of eqiad) which probably explains that one [17:05:26] 10Release-Engineering-Team, 10GitLab (Initialization), 10Software-Licensing, 10User-brennen: Add LICENSE/COPYING to operations/gitlab-ansible - https://phabricator.wikimedia.org/T282496 (10greg) p:05Triage→03Medium [17:11:43] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10jsn.sherman) @Legoktm Helpful, thanks! [17:13:00] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Jdforrester-WMF) >>! In T279859#7092022, @Aklapper wrote: > So this is about sending an `MediaWiki 1.31 soon to be End of Life` heads-up email out to a bunch of li... [17:13:23] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) I think I found the reason, [[ https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html#plugins-inputs-kafka-ssl_endpoint_identification_algorithm | ssl_endpoint... [17:17:59] Majavah: ... Oops :). Thanks! [17:22:45] Project beta-update-databases-eqiad build #50441: 04STILL FAILING in 2 min 45 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50441/ [17:25:51] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) Live hacking the if case out of the template like this adds the needed lines into the config files. ` root@deployment-puppetmaster04:/var/lib/git/operations/puppet# git diff diff... [17:28:18] 10LibUp, 10Wikidata, 10wdwb-tech: LibUp Wikibase error semver.exceptions.ParseVersionError: Unable to parse "dev-master#ba6fcfd823d623ba73c2faba5a208067281c4c38" - https://phabricator.wikimedia.org/T255248 (10Jdforrester-WMF) So it might work if we backport {f1542cbc246038e78dcbd8e379f6188c5d01bb46} into REL... [17:32:52] (03PS2) 10Jforrester: [zuul/layout.yaml]: Archive the RelatedLinks extension [integration/config] - 10https://gerrit.wikimedia.org/r/691888 (https://phabricator.wikimedia.org/T279221) (owner: 10Zoranzoki21) [17:33:05] (03CR) 10Jforrester: [C: 03+2] [zuul/layout.yaml]: Archive the RelatedLinks extension [integration/config] - 10https://gerrit.wikimedia.org/r/691888 (https://phabricator.wikimedia.org/T279221) (owner: 10Zoranzoki21) [17:33:44] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Beta is unable to update databases due to uncaught extensiondependencyerrors - https://phabricator.wikimedia.org/T283024 (10thcipriani) [17:34:08] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Beta is unable to update databases due to uncaught extensiondependencyerrors - https://phabricator.wikimedia.org/T283024 (10Majavah) [17:34:16] (03Merged) 10jenkins-bot: [zuul/layout.yaml]: Archive the RelatedLinks extension [integration/config] - 10https://gerrit.wikimedia.org/r/691888 (https://phabricator.wikimedia.org/T279221) (owner: 10Zoranzoki21) [17:34:30] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10Majavah) [17:34:58] thanks Majavah [17:38:38] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T281147 (10Jdforrester-WMF) [17:43:41] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10herron) >>! In T283013#7093631, @Majavah wrote: > Puppet manifests are trying to do that but that line is not present on deployment-logstash04 for some reason. Is that there on prod? Yes,... [18:00:30] Finally. [18:01:00] !log Zuul: [mediawiki/extensions/RelatedLinks] Archive T279221 [18:01:02] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:01:02] T279221: Archive the RelatedLinks extension - https://phabricator.wikimedia.org/T279221 [18:03:55] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) https://gerrit.wikimedia.org/r/c/operations/puppet/+/683695 was cherry picked and overriding the value. Mystery solved. [18:07:41] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) We have data! https://logstash-beta.wmcloud.org [18:07:58] Wow. [18:08:05] Beta Cluster logstash actually working? [18:08:08] It must be a mirage. [18:08:09] not yet! [18:08:12] Ha. [18:08:32] but at least something is flowing into the new elk7 based host [18:12:17] 10Continuous-Integration-Config: mediawiki-core-doxygen-docker hasn't run successfully for a month - https://phabricator.wikimedia.org/T280810 (10hashar) 05Open→03Resolved a:03hashar Ran successfully on https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/ ran successfully after I have ad... [18:12:36] good evening [18:12:47] hashar: Hey! Nice. [18:18:27] James_F: I noticed that doxygen failure after I have fixed wmf-insecte Jenkins bot earlier today :] [18:18:39] apparently I have missed the bug report [18:20:11] I guess I broke it in one of my image re-builds? [18:20:14] Sorry about that. [18:22:01] James_F: either you missed adding php-intl or mediawiki suddenly started to enforce the requirement [18:22:04] who knows. It is fixed now! [18:22:24] Oh, yes, we did start enforcing that a little while ago. [18:22:28] Project beta-update-databases-eqiad build #50442: 04STILL FAILING in 2 min 27 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50442/ [18:23:01] hashar: BTW, did you notice that we're down to just one MW repo with ruby selenium tests now? :-) Sadly it's Wikibase, but one is better than lots. [18:23:18] yeah been lurking that effort more or less [18:23:44] I guess Wikibase my be willing to migrate some of those tests to wdio. They must have way more experience with the javascript nowadays [18:23:54] And the only remaining stretch images are the old ruby ones and the PHP 70/71 ones. [18:24:10] I'd have thought so, but I guess they have other things to work on. [18:25:17] 7.0 and 7.1 will be dropped as REL1_31 reaches EOL isn't it? [18:25:21] aren't they [18:26:50] Yes, so next month. [18:26:56] Or sooner if you really want. ;-) [18:30:12] lol [18:30:48] Reedy: Want to send the "1.31 EOL approacheth; GLHF" e-mail? I drafted it for you and everything. Also, want to release 1.36.0? I think we're ready. [18:30:56] last time we unsupported something in CI that has lead to a broken "last ever" patch release ;D [18:30:56] Wot me, busy-body? [18:31:08] hashar: Eh. Fair. Let's not. [18:31:23] hashar: We manage to break 1.31 releases :P even with old PHP support in CI [18:31:29] we can keep the images around for the ~ 15 days they are still needed [18:31:35] Reedy: :-\ [18:31:43] Specifically the patch is https://gerrit.wikimedia.org/r/c/integration/config/+/683779 [18:31:47] because we don't do CI before release [18:31:49] And we have no phan [18:31:53] (03CR) 10Jforrester: [C: 04-1] "Antoine says no. :-(" [integration/config] - 10https://gerrit.wikimedia.org/r/683779 (owner: 10Jforrester) [18:32:17] James_F: I can, yeah [18:32:22] Reedy: Oh, right, the patch to CI happens afterwards because Security™® [18:32:28] I probably need to fix my mailman access first [18:33:06] Psh, details. [18:33:42] well, legoktm and Amir1 have probably broken my workflow by making things better [18:33:44] How dare they, etc [18:33:56] I was relying on that spacebar as a heater! etc. [18:35:38] Reedy: don't worry, Amir1 will happily reset the list password for you [18:35:42] Once we drop REL1_31 support we can upgrade composer to 2.x [18:35:49] I know what the password was! :P [18:36:13] oh nice, the menu goes offscreen [18:36:15] To which I'm really looking forward, principally as then people will stop asking why we can't upgrade to composer 2.x. [18:36:32] and its too wide [18:45:13] https://lists.wikimedia.org/postorius/lists/mediawiki-announce.lists.wikimedia.org/members/owner/ [18:45:24] Only Greg is set as owner [18:45:34] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Ottomata) Nice!!! [18:45:42] I can add more people if greg-g approves [18:45:59] lol [18:46:08] That's it, greg has to send all announces now [18:46:25] If no-one else, I imagine thcipriani should. :-) [18:46:30] https://usercontent.irccloud-cdn.com/file/DWK1mBjw/image.png [18:47:01] ugh, nooooo [18:47:02] I imagine he is getting daily mails for the 64 held messages every day :D [18:47:09] heh, filtered [18:47:33] to a folder I don't look at, I run /usr/bin/listadmin when I think about it and discard held messages [18:47:35] It's the bounce-spam that I hated particularly about being a listadmin. [18:47:58] I do clear them out when I go on to send more spam via the list [18:48:00] James_F: hello, admin of 13 mailing lists here, I feel your pain [18:48:41] just to be clear, is this mailman3 account separate from all of our other accounts? [18:48:48] Yes [18:48:50] * greg-g hasn't made his new account yet [18:49:17] There is a project to make it work with SUL OAuth but not there yet [18:49:23] greg-g: expecting SSO? In Wikimedia? You must be new here [18:49:49] Clearly we need immediate 2FA for all list administration. [18:49:55] Because mailman2 was so very secure. [18:50:00] I knew the old magic for -announce, and would like to know the new magic :) [18:50:14] thcipriani: join the club! ;) [18:50:21] Send e-mail, go to mailman, allow the mail to go through? [18:50:22] hence we're going to delegate it all to greg [18:50:29] ^ [18:50:31] yess [18:50:35] Success! [18:50:39] well, now you don't need to put the special password in [18:50:43] or rather, can't [18:51:14] 10Beta-Cluster-Infrastructure, 10User-Majavah: Migrate beta cluster to ELK7 - https://phabricator.wikimedia.org/T283013 (10Majavah) MediaWiki event delivery is still failing, rsyslogd spams this all over the logs: ` omkafka: kafka delivery FAIL on Topic 'udp_localhost-info', msg [full mediawiki log message con... [18:51:33] thcipriani: TLDR: Create a central account: https://lists.wikimedia.org/accounts/signup it'll handle the rest [18:51:43] well, except it won't [18:51:48] as he's not an admin of the list either [18:52:01] Yet. [18:52:13] Yeah, I can add anyone you tell me to add and Greg agrees [18:52:38] if permission is your excuse Reedy [18:52:49] * thcipriani hopes he didn't add more hats to his collection [18:52:49] It's not an excuse [18:53:03] I literally don't have any access now [18:53:32] I haven't received my email address confirmation email yet :9 [18:53:34] :( [18:53:54] we can get greg to share his password, so we can all be nostalgic for mailman 2 [18:54:27] it was pretty quick for me [18:54:33] maybe all the load broke it [18:55:36] actually mailman3 has memory issues these days but it should be fine [18:55:44] new workflow step 1: login to greg-g 's email [18:56:25] What's the epic SRE project now that mailman3 is done? PHP 7.2 -> 7.3? :-)- [18:57:14] bah, spam folder [18:57:21] hahaha [18:57:52] James_F: mediawiki on kubernetes I guess and then php 8.1? ;] [18:57:55] greg-g: I was actually checking the database to see if you have registered [18:58:07] alright, I'm in, who do I give power to? :) [18:58:15] * hashar ducks [18:58:17] 🍿 [18:59:03] hashar: That's not a thing mortals can help push along, though. [18:59:09] James_F: commons' 360GB image table? :D [18:59:31] Amir1: Hey, the RfC for refactoring the image table is sort-of green-lit. [18:59:35] Amir1: Patches welcome. ;-P [18:59:37] I notice a very strong lack of responses there :) [18:59:59] greg-g: thcipriani and Reedy I think. [19:00:03] James_F: Glad you asked https://gerrit.wikimedia.org/r/c/mediawiki/core/+/687522 [19:00:10] whoever owns mediawiki release process? [19:00:17] HAHAHAHA [19:00:19] HAHAHAHAHAHAHA [19:00:21] AHAHAH [19:00:22] I have lost track of who is owning it [19:00:23] ;D [19:00:23] HAHA [19:00:42] * greg-g walks away slowly [19:00:55] Reedy: you're an owner now, along with thcipriani [19:01:05] ty [19:01:08] ;D [19:01:13] ^ thanks greg-g [19:01:39] oh Amir1 do we have a way to see which lists we are an admin for ? [19:01:46] I yup [19:02:04] https://lists.wikimedia.org/postorius/lists/?role=owner [19:02:20] and I really like mailman 3 (though the web design looks dated, the functionality are a few order of magnitudes better than the 1990's era the old one was) [19:02:30] \o/ [19:03:08] hashar: mm3 is very similar to mw logo, the new one is not perfect but the old one is REALLY BAD [19:03:27] Why are people replying to "[MediaWiki-announce] MediaWiki 1.18.0 released" [19:03:44] Oh dear. [19:04:57] Amir1: and is there a way to archive/disable a list? [19:05:45] not yet, we are working on some sort of standardization [19:06:00] you can delete a mailing list but archive no [19:06:09] no worries :] [19:06:12] I want to archive some [19:06:33] (03PS1) 10Accraze: Review access change [machinelearning/liftwing] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/692082 [19:06:39] the packaging list was some attempt to organize building debian packages, but beside a very few messages 7 years ago, that project never started. So might as well archive it [19:06:43] will do it when it is a thing :-] [19:07:27] I will ping you once we are done, basically we are introducing a black-hole address and make it the sole owner, then put the mailing list in emergency moderation discarding every email and change the description. [19:13:33] Amir1: or just poke the listadmins list (I am on it) :] [19:13:50] haha, sure :D [19:14:50] (03PS4) 10Jack Phoenix: Enable phan for some extensions [integration/config] - 10https://gerrit.wikimedia.org/r/692069 [19:15:24] on other news, I am subscribed to 362 Unbreak Now tasks over roughly 10 years or roughly 3600 days. That is an unbreak now every ten days! :] [19:16:24] a hundred of them assigned to me, so I guess I am abusing the UBN feature by setting it foro things I fix immediately [19:16:44] I don't think that's abuse. [19:16:52] THat's just be a liability. ;-) [19:21:41] Project beta-update-databases-eqiad build #50443: 04STILL FAILING in 1 min 40 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50443/ [19:24:38] hashar: you don't abuse it. I get every public UBN task to my email and I don't notice a lot from you. [19:27:03] (03PS1) 10Kosta Harlan: phpunit: Add support for vendor/bin/phpunit entrypoint [integration/quibble] - 10https://gerrit.wikimedia.org/r/692372 (https://phabricator.wikimedia.org/T90875) [19:28:26] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T281147 (10Jdlrobson) [19:28:47] (03PS1) 10Kosta Harlan: Add a few more directories to git/docker ignore [integration/quibble] - 10https://gerrit.wikimedia.org/r/692373 [19:30:26] (03CR) 10jerkins-bot: [V: 04-1] phpunit: Add support for vendor/bin/phpunit entrypoint [integration/quibble] - 10https://gerrit.wikimedia.org/r/692372 (https://phabricator.wikimedia.org/T90875) (owner: 10Kosta Harlan) [19:30:35] 10MediaWiki-Releasing, 10Documentation, 10MW-1.31-release: Announce upcoming formal EOL of REL1_31 - https://phabricator.wikimedia.org/T279859 (10Reedy) I presume `[0]` is supposed to be https://www.mediawiki.org/wiki/Version_lifecycle ? [19:34:43] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T281147 (10Jdlrobson) Just noting this as a "risky" change the train deployers should eb aware of,. **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/682773/... [19:44:14] RhinosF1: cool! thank you for the feedback :] [19:44:33] I am off, tomorrow is train! [19:45:31] Rest well, hopefully you won't need it! [19:46:26] (03PS2) 10Kosta Harlan: Add a few more directories to git/docker/tox ignore [integration/quibble] - 10https://gerrit.wikimedia.org/r/692373 [19:50:15] hopefully, we shall see [19:50:22] have a good rest of the day everyone [19:52:03] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T281147 (10hashar) I haven't followed the finish up of 1.37.0-wmf.5 today but it got pushed to all wikis so that is a thing. I will catch up with the events from last week... [19:52:23] (03PS2) 10Kosta Harlan: phpunit: Add support for vendor/bin/phpunit entrypoint [integration/quibble] - 10https://gerrit.wikimedia.org/r/692372 (https://phabricator.wikimedia.org/T90875) [19:53:26] 10LibUp, 10Patch-For-Review: libup committed and uploaded an upgrade twice for the same package on the same repo - https://phabricator.wikimedia.org/T280438 (10Umherirrender) 05Open→03Resolved a:03Legoktm Looks good for the me and should be a good guard for the current situation. [19:55:54] (03CR) 10jerkins-bot: [V: 04-1] phpunit: Add support for vendor/bin/phpunit entrypoint [integration/quibble] - 10https://gerrit.wikimedia.org/r/692372 (https://phabricator.wikimedia.org/T90875) (owner: 10Kosta Harlan) [19:58:44] (03CR) 10Hashar: "Or rather yes, but after REL1_31 is formally EOL :] I thought about being stricter and drop languages support as soon as they are EOL ups" [integration/config] - 10https://gerrit.wikimedia.org/r/683779 (owner: 10Jforrester) [19:59:32] (03PS3) 10Kosta Harlan: phpunit: Add support for vendor/bin/phpunit entrypoint [integration/quibble] - 10https://gerrit.wikimedia.org/r/692372 (https://phabricator.wikimedia.org/T90875) [20:21:46] Project beta-update-databases-eqiad build #50444: 04STILL FAILING in 1 min 45 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50444/ [20:50:02] (03PS1) 10Kosta Harlan: Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 [20:52:38] (03CR) 10jerkins-bot: [V: 04-1] Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 (owner: 10Kosta Harlan) [20:55:18] (03PS2) 10Kosta Harlan: Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 [20:58:07] (03CR) 10jerkins-bot: [V: 04-1] Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 (owner: 10Kosta Harlan) [21:03:35] (03PS3) 10Kosta Harlan: Add support for connecting to already running MySQL [integration/quibble] - 10https://gerrit.wikimedia.org/r/692412 [21:12:04] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T281146 (10greg) 05Open→03Resolved [21:21:40] Project beta-update-databases-eqiad build #50445: 04STILL FAILING in 1 min 39 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50445/ [21:35:04] 10phan, 10Upstream: Phan should handle always-throw-function like ApiBase::dieWithError better - https://phabricator.wikimedia.org/T240141 (10Legoktm) Awesome! If there's no conflict with `@return never` I think we should use that since that's what matches PHP 8.1. It's similar to us using `callable` in PHPDoc... [21:45:38] 10LibUp: LibUp needs to handle "main" branches nicely - https://phabricator.wikimedia.org/T273237 (10Legoktm) 05Open→03Resolved a:03Legoktm https://gerrit.wikimedia.org/r/q/owner:tools.libraryupgrader%2540tools.wmflabs.org+branch:main [22:21:52] Project beta-update-databases-eqiad build #50446: 04STILL FAILING in 1 min 51 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50446/ [23:21:54] Project beta-update-databases-eqiad build #50447: 04STILL FAILING in 1 min 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/50447/ [23:38:44] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library, 10Patch-For-Review: beta votewiki: Fatal error: Uncaught ExtensionDependencyError: TheWikipediaLibrary requires CentralAuth to be installed. - https://phabricator.wikimedia.org/T283003 (10Urbanecm) 05Open→03Resolved ` urbanecm@deployment-deploy01:~$...