[00:01:38] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:06:38] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [07:15:23] (03PS1) 10Kevin Bazira: inference-services: add article-country CI pipeline jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1078538 (https://phabricator.wikimedia.org/T371897) [07:23:39] 10Gerrit, 06collaboration-services, 10VPS-project-Codesearch, 10VPS-project-Extdist, 13Patch-For-Review: Move clients off of gerrit-replica.wikimedia.org back to gerrit.wikimedia.org - https://phabricator.wikimedia.org/T336710#10209504 (10hashar) >>! In T336710#10189664, @Dzahn wrote: > A thing that chan... [08:19:04] (03CR) 10Ilias Sarantopoulos: [C:03+1] inference-services: add article-country CI pipeline jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1078538 (https://phabricator.wikimedia.org/T371897) (owner: 10Kevin Bazira) [08:30:13] !log gerrit: closed all connections for user suecarmol due to T371749 [08:30:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:30:15] T371749: Gerrit - Too many concurrent connections (8) - max. allowed: 8 - https://phabricator.wikimedia.org/T371749 [08:31:33] (03PS1) 10Arthur taylor: Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) [08:34:08] (03CR) 10CI reject: [V:04-1] Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) (owner: 10Arthur taylor) [09:05:23] (03CR) 10Hashar: "There is a coverage report generated at https://doc.wikimedia.org/cover/design-codex-php/ . If the Doxygen job fails that should be inves" [integration/config] - 10https://gerrit.wikimedia.org/r/1078493 (https://phabricator.wikimedia.org/T373940) (owner: 10Eric Gardner) [09:05:33] (03CR) 10Hashar: [C:04-1] Codex-PHP: Remove the postmerge tasks for now [integration/config] - 10https://gerrit.wikimedia.org/r/1078493 (https://phabricator.wikimedia.org/T373940) (owner: 10Eric Gardner) [09:39:02] (03PS2) 10Arthur taylor: Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) [09:40:32] codders: :) [09:40:58] (03CR) 10CI reject: [V:04-1] Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) (owner: 10Arthur taylor) [09:41:05] hi! :) gotta love those quibble patches [09:42:22] I can never seem to remember both "black" and "flake8" when running locally - always needs a couple of round-trips [09:42:45] (03PS3) 10Arthur taylor: Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) [09:48:22] 10Release-Engineering-Team (Seen), 10Wikimedia-Site-requests, 07Technical-Debt: Undeploy CodeReview - https://phabricator.wikimedia.org/T116948#10209826 (10Ladsgroup) Hello, CodeReview tables still exist in production. Can I drop them? [09:48:26] (03CR) 10Hashar: [C:03+1] "It is fun how:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) (owner: 10Arthur taylor) [09:48:39] codders: poke me again about it this afternoon and I will +2 it [09:48:43] assuming the tests pass :D [09:48:53] will do - thanks! [10:26:44] 10Release-Engineering-Team (Radar), 06Infrastructure-Foundations, 06serviceops-radar: Allow release engineering to delete images - https://phabricator.wikimedia.org/T354786#10210053 (10Clement_Goubert) [10:27:18] (03merge) 10aklapper: Link LDAP account on user profile to ldap.toolforge.org [repos/phabricator/extensions] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/40 (https://phabricator.wikimedia.org/T366766) [10:31:25] (03PS1) 10Arthur taylor: Match `split-group` log files when checking for mediawiki errors [integration/config] - 10https://gerrit.wikimedia.org/r/1078628 (https://phabricator.wikimedia.org/T375852) [10:32:36] (03CR) 10CI reject: [V:04-1] Match `split-group` log files when checking for mediawiki errors [integration/config] - 10https://gerrit.wikimedia.org/r/1078628 (https://phabricator.wikimedia.org/T375852) (owner: 10Arthur taylor) [10:34:52] (03PS2) 10Arthur taylor: Match `split-group` log files when checking for mediawiki errors [integration/config] - 10https://gerrit.wikimedia.org/r/1078628 (https://phabricator.wikimedia.org/T375852) [11:04:16] 10Release-Engineering-Team (Seen), 10Wikimedia-Site-requests, 07Technical-Debt: Undeploy CodeReview - https://phabricator.wikimedia.org/T116948#10210229 (10Reedy) I think we did make a DB dump and post it somewhere (misc dumps)… If we confirm that, and that its good… I think we should be good to drop th... [11:25:54] @hashar, those tests are passing now: https://gerrit.wikimedia.org/r/c/integration/quibble/+/1078610 [12:00:26] codders: great! [12:03:43] (03CR) 10Hashar: [C:03+2] Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) (owner: 10Arthur taylor) [12:21:45] (03Merged) 10jenkins-bot: Add support for specifying patchsets with `--change` CLI argument [integration/quibble] - 10https://gerrit.wikimedia.org/r/1078610 (https://phabricator.wikimedia.org/T376602) (owner: 10Arthur taylor) [12:45:05] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move commons-query.wikimedia.org and query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10210409 (10Jelto) >>! In T350793#10206686, @Lucas_Werkmeister_WMDE wrote: > AF... [12:45:35] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move commons-query.wikimedia.org and query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10210412 (10Jelto) [13:11:48] 10Release-Engineering-Team (Seen), 10Wikimedia-Site-requests, 07Technical-Debt: Undeploy CodeReview - https://phabricator.wikimedia.org/T116948#10210530 (10Ladsgroup) Found it ({T243055}): https://dumps.wikimedia.org/other/codereview/20200428/ It's a bit hard to find it. It should be linked somewhere out... [13:24:47] (03open) 10samtar: Draft: Use {{deploy}} template in examples [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/121 [13:53:44] thcipriani: fyi, I shorted the weekly triage to the second half this week to make it possible for Marc to attend once. This week only. [14:15:57] Krinkle: oh, nice :) [14:26:32] (03CR) 10Hashar: [C:03+2] "Jobs deployed:" [integration/config] - 10https://gerrit.wikimedia.org/r/1078538 (https://phabricator.wikimedia.org/T371897) (owner: 10Kevin Bazira) [14:27:49] (03Merged) 10jenkins-bot: inference-services: add article-country CI pipeline jobs [integration/config] - 10https://gerrit.wikimedia.org/r/1078538 (https://phabricator.wikimedia.org/T371897) (owner: 10Kevin Bazira) [14:35:32] 10Phabricator (phabricator-next), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-10-08 - https://phabricator.wikimedia.org/T376720 (10brennen) 03NEW [14:36:59] brennen: fyi I also merged some Phab translation updates [14:37:44] (03open) 10brennen: update submodules for 2024-10-08 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/56 (https://phabricator.wikimedia.org/T376720) [14:38:16] andre: think i got those on the MR, but double check me [14:38:23] thanks [14:40:12] andre: re: T363337, i still feel a little weird about this, but assuming we merge are you feeling ready to add extra users to Trusted-Contributors, update docs, etc.? [14:40:13] T363337: Explore restricting editing task priority to Trusted-Contributors/staff/etc - https://phabricator.wikimedia.org/T363337 [14:41:09] brennen: I could handle that but my first quesiton would be do you think you'd have time and patience to test that? If not I'd say let's merge [14:42:09] i haven't tested the restriction - could put some time into that this afternoon [14:43:00] brennen: if you're cool with that I would appreciate and can wait a week. But don't know what else is on your list :) [14:43:18] yeah, let's go that route then. :) [14:43:36] (03PS2) 10Hashar: jjb: change mw coverage publish jobs to use teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1078453 [14:43:36] (03CR) 10Hashar: [C:03+2] "Verified on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/1078614" [integration/config] - 10https://gerrit.wikimedia.org/r/1078453 (owner: 10Hashar) [14:44:36] brennen: I never know how much of your time I should try to steal when there are always also a billion other things to do :) [14:44:54] (03Merged) 10jenkins-bot: jjb: change mw coverage publish jobs to use teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1078453 (owner: 10Hashar) [14:46:11] my current to-do list is embarrassingly long, but also very little of it is really on fire and phab/phorge is generally worth the time investment, i'd say. [14:52:44] (03merge) 10brennen: update submodules for 2024-10-08 deploy [repos/phabricator/deployment] (wmf/stable) - 10https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/56 (https://phabricator.wikimedia.org/T376720) [15:01:55] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥): Phabricator login page should not call developer accounts "Wikitech (LDAP) accounts" - https://phabricator.wikimedia.org/T376288#10211087 (10Aklapper) [15:11:55] 10Phabricator (2024-10-08), 06Release-Engineering-Team, 06collaboration-services: Deploy Phabricator/Phorge 2024-10-08 - https://phabricator.wikimedia.org/T376720#10211123 (10Aklapper) 05Open→03Resolved a:03brennen This was deployed today (thanks brennen!). [15:12:49] 10Phabricator (2024-10-08), 10Release-Engineering-Team (Priority Backlog 📥): Phabricator login page should not call developer accounts "Wikitech (LDAP) accounts" - https://phabricator.wikimedia.org/T376288#10211127 (10Aklapper) 05Open→03Resolved This was deployed today (thanks brennen!). [15:38:52] (03PS2) 10Snwachukwu: Edit Repo Config [schemas/event/primary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1078704 [15:41:42] (03Abandoned) 10Snwachukwu: Edit Repo Config [schemas/event/primary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1078704 (owner: 10Snwachukwu) [15:42:00] (03Abandoned) 10Snwachukwu: Edit Repo Config [schemas/event/primary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1071896 (owner: 10Snwachukwu) [15:43:09] (03PS2) 10Snwachukwu: Edit Repo Config [schemas/event/primary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1078705 [15:43:15] (03CR) 10Snwachukwu: [V:03+2 C:03+2] Edit Repo Config [schemas/event/primary] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1078705 (owner: 10Snwachukwu) [15:43:18] 10Phabricator, 06Release-Engineering-Team: Please disable MFA for Phabricator account for WMDE-leszek - https://phabricator.wikimedia.org/T376605#10211196 (10Aklapper) p:05Triage→03High a:03Aklapper [15:49:18] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [15:52:30] (03update) 10dancy: WIP: SpiderPig Web UI [repos/releng/scap] (master-I0a50f91db8c9aa9147afdd0468e37e001af86167) - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/471 [16:00:02] 10Gerrit: git-review falls back to the "master" branch in repos using the "main" branch - https://phabricator.wikimedia.org/T376727#10211309 (10hashar) [16:05:26] 10Gerrit: git-review falls back to the "master" branch in repos using the "main" branch - https://phabricator.wikimedia.org/T376727#10211316 (10hashar) The workarounds are: * Set you repository default branch to be `master` * Edit the repository `.gitreview` file to have `defaultbranch=main` which overrides `gi... [16:13:25] 10Release-Engineering-Team (Seen), 10Wikimedia-Site-requests, 07Technical-Debt: Undeploy CodeReview - https://phabricator.wikimedia.org/T116948#10211365 (10Reedy) https://gerrit.wikimedia.org/r/c/operations/puppet/+/1039246 actually removed the link that would've made it viewable on https://dumps.wikimed... [16:39:39] (03PS2) 10Hashar: jjb: change selenium-daily jobs to use teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1078440 [16:40:46] (03PS3) 10Hashar: jjb: change selenium-daily jobs to use teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1078440 [16:42:40] (03CR) 10Hashar: [C:03+2] "Verified to be working :)" [integration/config] - 10https://gerrit.wikimedia.org/r/1078440 (owner: 10Hashar) [16:44:10] (03Merged) 10jenkins-bot: jjb: change selenium-daily jobs to use teardown [integration/config] - 10https://gerrit.wikimedia.org/r/1078440 (owner: 10Hashar) [16:57:11] (03CR) 10Hashar: [C:03+1] "Feel free to do the switch, my understanding is that the upgrade is driven by the php 8.4 support." [integration/config] - 10https://gerrit.wikimedia.org/r/1078110 (https://phabricator.wikimedia.org/T376409) (owner: 10Jforrester) [17:41:34] (03PS2) 10Jforrester: jjb: Switch PHP jobs to use images with Composer v2.8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1078110 (https://phabricator.wikimedia.org/T376409) [17:42:10] 10Gerrit, 10Catalyst (Camp Muir): Update gerrit plugin for PatchDemo - https://phabricator.wikimedia.org/T374954#10211809 (10thcipriani) a:03EBomani @EBomani will take a look at updating the gerrit plugin this week! [17:45:12] 10Gerrit, 10Catalyst (Camp Muir): Update gerrit plugin for PatchDemo - https://phabricator.wikimedia.org/T374954#10211815 (10thcipriani) [17:45:40] 10Gerrit, 10Catalyst (Camp Muir): Update gerrit plugin for PatchDemo - https://phabricator.wikimedia.org/T374954#10211816 (10thcipriani) p:05Triage→03Medium [17:50:31] (03CR) 10Jforrester: [C:03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/1078110 (https://phabricator.wikimedia.org/T376409) (owner: 10Jforrester) [17:51:58] (03Merged) 10jenkins-bot: jjb: Switch PHP jobs to use images with Composer v2.8.1 [integration/config] - 10https://gerrit.wikimedia.org/r/1078110 (https://phabricator.wikimedia.org/T376409) (owner: 10Jforrester) [17:52:17] 10Continuous-Integration-Config, 10MediaWiki-Vendor, 07Composer, 13Patch-For-Review, and 2 others: Upgrade composer to 2.8.x - https://phabricator.wikimedia.org/T376409#10211842 (10Jdforrester-WMF) [17:52:21] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10211855 (10bd808) I have spent some time day dreaming about deploying `wmf/next` to a new mwdebug pod so that anyone with the WikimediaDebug add-on could test... [18:22:50] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10211984 (10bd808) If we take mwdebug idea off the table then we have the new test wiki just for this group (test3wiki?) vs move one or more existing wikis to... [19:10:31] (03PS1) 10Hashar: README.md: doc loading a plugin from the browser [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1078735 [19:14:48] (03CR) 10Hashar: "Isabelle once tested one of the JavaScript plugin using the Chromium extension, I have since found the browser supports editing a remote J" [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1078735 (owner: 10Hashar) [19:31:29] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10212245 (10thcipriani) A single wiki would side-step many many of the problems you've unearthed investigating WikimediaDebug idea (though I do love the Wikime... [19:36:00] 10Gerrit, 10Catalyst (Camp Muir): Update gerrit plugin for PatchDemo - https://phabricator.wikimedia.org/T374954#10212274 (10hashar) I have wrote some documentation at https://gerrit.wikimedia.org/r/c/operations/software/gerrit/+/1078735 The plugin is in https://gerrit.wikimedia.org/r/operations/software/gerr... [20:16:02] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10212405 (10bd808) >>! In T375145#10212244, @thcipriani wrote: > I'm curious the benefit of building a test3wiki vs. using test2wiki? This might be a good que... [20:39:33] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10212510 (10bd808) >>! In T375145#10212244, @thcipriani wrote: > I still wonder if there's work to do before deploying `wmf/next` to a single wiki. After we're... [21:27:44] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Python3-Porting: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10212631 (10Dzahn) Added to tech news issue 42 - https://meta.wikimedia.org/wiki/Tech/News/2024/42 [21:34:17] PROBLEM - gerrit process on gerrit2003 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/lib/jvm/java-17-openjdk-amd64/bin/java .*-jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site https://wikitech.wikimedia.org/wiki/Gerrit [21:37:19] ^ this is me, trying to setup a new machine that isn't production yet [21:37:50] downtime expired due to problems with the initial scap deploy, duplicate declaration after that [21:46:07] 10Release-Engineering-Team (Priority Backlog 📥): Draft and get approval for next hypothesis to follow WE6.2.1 - https://phabricator.wikimedia.org/T375145#10212703 (10bd808) More "good question" questions: * Will we need to incorporate mediawiki-config changes into the wmf/next image in real-time? * Will we need... [22:00:17] RECOVERY - gerrit process on gerrit2003 is OK: PROCS OK: 1 process with regex args ^/usr/lib/jvm/java-17-openjdk-amd64/bin/java .*-jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site https://wikitech.wikimedia.org/wiki/Gerrit [22:01:01] apparently I got it deployed to a certain extent :) [23:18:56] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 06collaboration-services, 07Python3-Porting, 07User-notice: Inbound mail to phabricator doesn't work - https://phabricator.wikimedia.org/T356077#10212848 (10Pppery) [23:40:20] 10Phabricator: https://ldap.toolforge.org/ integration assumes that `cn` and `uid` are equivalent - https://phabricator.wikimedia.org/T376769 (10bd808) 03NEW [23:50:53] 10Phabricator, 10Tool-ldap: https://ldap.toolforge.org/ integration assumes that `cn` and `uid` are equivalent - https://phabricator.wikimedia.org/T376769#10212894 (10Peachey88)