[00:17:14] 10Release-Engineering-Team, 10Release Pipeline: Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10jeena) [00:34:48] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10Catrope) [00:37:28] 10Continuous-Integration-Config, 10Patch-For-Review: Introduce non-voting jobs with quibble+apache - https://phabricator.wikimedia.org/T276428 (10Addshore) So it looks like every non voting selenium job for Wikibase using apache as the webserver has failed since being introduced. https://integration.wikimedia.... [01:08:47] 10Release-Engineering-Team, 10Release Pipeline: Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10DannyS712) Its not really important, but the time between the bot uploading the commit and the deployers merging it i... [04:52:03] (03CR) 10Zoranzoki21: "Ping. :)" [integration/config] - 10https://gerrit.wikimedia.org/r/670601 (owner: 10Zoranzoki21) [06:26:56] 10VPS-project-Codesearch: Codesearch (Beta) not reachable - https://phabricator.wikimedia.org/T277459 (10Ammarpad) Is this down completely? A day after (for me) it's still not working. [06:28:00] 10VPS-project-Codesearch: Codesearch (Beta) not reachable - https://phabricator.wikimedia.org/T277459 (10Ammarpad) [08:31:45] 10Continuous-Integration-Infrastructure, 10Browser-Tests: Centrally look for flakey browser tests - https://phabricator.wikimedia.org/T277205 (10hashar) [08:43:21] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review: Switch Jenkins servers to Java 11 - https://phabricator.wikimedia.org/T269354 (10MoritzMuehlenhoff) [08:51:44] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review: Switch Jenkins servers to Java 11 - https://phabricator.wikimedia.org/T269354 (10hashar) Thank you @MoritzMuehlenhof... [09:43:00] (03PS1) 10ZPapierski: Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) [09:43:12] (03PS2) 10ZPapierski: Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) [09:43:59] (03CR) 10ZPapierski: "I don't know what one actually does to create a new image version, but adding a changelog entry + the change seemed to be a common denomin" [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [09:44:23] (03CR) 10jerkins-bot: [V: 04-1] Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [09:48:35] (03PS3) 10ZPapierski: Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) [09:51:53] hashar: how can I release a new docker image to use? I've done this - https://gerrit.wikimedia.org/r/672667 - and I wonder what else should I do [09:53:06] 10Beta-Cluster-Infrastructure, 10SRE, 10Puppet, 10cloud-services-team (Kanban): WMCS standalone puppet master does not lookup cherry picked hiera change - https://phabricator.wikimedia.org/T277526 (10Majavah) [09:59:17] zpapierski: hmm not really, don't really have any bandwith to manage a sonarqube upgrade :] [09:59:51] zpapierski: you probably want a standalone task for that upgrade. https://phabricator.wikimedia.org/T264877 is just about adding the sonar scanner to all repositories, so it is not related :) [10:00:07] zpapierski: beside that the change is probably fine [10:00:15] the devil is figuring out a proper strategy to upgrade the scanner [10:04:13] (03CR) 10Hashar: "I guess that will work (kudos on noticing the changelog file needs a new entry). The devil is to figure out what it is going to break ac" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [10:04:26] zpapierski: reviewed! in short, lets get a new Phabricator task [10:04:37] and yeah probably we can just upgrade it, but I would like others to look at it [10:06:36] any specific tags to add to the task? [10:07:14] I guess #code-health [10:07:24] and you can add #continuous-integration-config [10:07:33] which is for changes made to integration/config.git [10:07:56] I have absolutely zero idea about what would be broken by sonar:scanner though [10:08:06] but at lesat the config change itself is fine and I can get the image build [10:08:59] (03CR) 10Kosta Harlan: "Are there any breaking (or other significant) changes we should know about in going to 4.6.0? I believe QTE maintains this tool now, shoul" [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [10:09:08] zpapierski: and maybe we can change the image version to match the one of the sonar:sccanner ? [10:09:13] from what I looked, every single usage of this image uses a specific version [10:09:24] so that is the changelog file could use 4.6.0.2311-1 as a version instead of 1.0.2 [10:09:39] why not, seems intuitive [10:09:51] 10Continuous-Integration-Config, 10Code-Health: Upgrade sonar-scanner in sonar-scanner docker image in JJB - https://phabricator.wikimedia.org/T277527 (10Zbyszko) [10:10:42] I don't know why we never updated the jobs to use 1.0.1 [10:11:17] 10Continuous-Integration-Config, 10Code-Health: Upgrade sonar-scanner in sonar-scanner docker image in JJB - https://phabricator.wikimedia.org/T277527 (10Zbyszko) [10:11:37] (03PS4) 10ZPapierski: Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T277527) [10:13:39] (03CR) 10ZPapierski: "> Patch Set 3:" [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T277527) (owner: 10ZPapierski) [10:14:16] zpapierski: lets build it ;) [10:14:48] (03CR) 10Hashar: [C: 03+2] "Yeah indeed it is not a big deal :] Lets build it and update the jobs next." [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T277527) (owner: 10ZPapierski) [10:15:15] zpapierski: and you can find which jobs definition require an update byrunning: ./utils/docker-updates|grep sonar [10:15:41] jjb/python-jobs.yaml:50: image: 'docker-registry.wikimedia.org/releng/sonar-scanner:1.0.1' [10:15:56] (03Merged) 10jenkins-bot: Bump sonar scanner version [integration/config] - 10https://gerrit.wikimedia.org/r/672667 (https://phabricator.wikimedia.org/T277527) (owner: 10ZPapierski) [10:17:28] !log Building docker-registry.wikimedia.org/releng/sonar-scanner:4.6.0.2311-1 # T277527 [10:17:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:17:32] T277527: Upgrade sonar-scanner in sonar-scanner docker image in JJB - https://phabricator.wikimedia.org/T277527 [10:20:56] (03PS1) 10Hashar: jjb: releng/quibble-apache image has been renamed [integration/config] - 10https://gerrit.wikimedia.org/r/672682 [10:21:32] (03CR) 10Hashar: [C: 03+2] jjb: releng/quibble-apache image has been renamed [integration/config] - 10https://gerrit.wikimedia.org/r/672682 (owner: 10Hashar) [10:23:25] (03Merged) 10jenkins-bot: jjb: releng/quibble-apache image has been renamed [integration/config] - 10https://gerrit.wikimedia.org/r/672682 (owner: 10Hashar) [10:26:36] (03PS4) 10Addshore: Add configuration for new wikidata/query-builder repo [integration/config] - 10https://gerrit.wikimedia.org/r/670898 (https://phabricator.wikimedia.org/T277060) (owner: 10Michael Große) [10:26:54] zpapierski: Successfully published image docker-registry.discovery.wmnet/releng/sonar-scanner:4.6.0.2311-1 [10:26:54] (03CR) 10Addshore: [C: 03+2] Add configuration for new wikidata/query-builder repo [integration/config] - 10https://gerrit.wikimedia.org/r/670898 (https://phabricator.wikimedia.org/T277060) (owner: 10Michael Große) [10:28:08] (03Merged) 10jenkins-bot: Add configuration for new wikidata/query-builder repo [integration/config] - 10https://gerrit.wikimedia.org/r/670898 (https://phabricator.wikimedia.org/T277060) (owner: 10Michael Große) [10:29:24] !log reload zuul for https://gerrit.wikimedia.org/r/670898 Add configuration for new wikidata/query-builder repo [10:29:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:31:09] hashar: great, thx! [10:32:55] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Tgr) >>! In T185664#6915991, @Ladsgroup wrote: > I have started dropping lots of code that is not enabled in production... [10:34:26] (03PS1) 10ZPapierski: Bump sonar-scanner version for tox with sonar jobs [integration/config] - 10https://gerrit.wikimedia.org/r/672686 (https://phabricator.wikimedia.org/T264877) [10:34:44] 10Beta-Cluster-Infrastructure, 10SRE, 10Puppet, 10cloud-services-team (Kanban): WMCS standalone puppet master does not lookup cherry picked hiera change - https://phabricator.wikimedia.org/T277526 (10jbond) @hashar tl;dr the you should place the value in `hieradata/cloud.yaml` to set a sane default for all... [10:36:24] (03PS2) 10Hashar: dockerfiles: [quibble-stretch-php72-apache] Add custom log to time requests [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [10:36:27] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Ladsgroup) Yes. I haven't removed it yet. So far mostly dead code. Unsupported upgrade paths, unused hooks, etc. but I... [10:37:31] (03CR) 10jerkins-bot: [V: 04-1] dockerfiles: [quibble-stretch-php72-apache] Add custom log to time requests [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [10:37:50] .... [10:46:33] (03PS3) 10Hashar: dockerfiles: [quibble-stretch-php72-apache] Add custom log to time requests [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [10:58:12] (03CR) 10Hashar: [C: 03+1] "I have build it locally, ran it with:" [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [11:28:28] zpapierski: deploying the tox sonar scanner update [11:29:20] (03CR) 10Hashar: "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/672686 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [11:29:25] (03CR) 10Hashar: [C: 03+2] Bump sonar-scanner version for tox with sonar jobs [integration/config] - 10https://gerrit.wikimedia.org/r/672686 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [11:30:42] (03CR) 10Hashar: [C: 03+2] "I have send a dummy change to test it: https://gerrit.wikimedia.org/r/c/wikimedia/discovery/analytics/+/672695" [integration/config] - 10https://gerrit.wikimedia.org/r/672686 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [11:30:47] (03Merged) 10jenkins-bot: Bump sonar-scanner version for tox with sonar jobs [integration/config] - 10https://gerrit.wikimedia.org/r/672686 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [11:31:20] running at https://integration.wikimedia.org/ci/job/wikimedia-discovery-analytics-patch-tox-docker-with-sonar/20/console [11:55:14] hashar: thanks! [11:55:55] (03CR) 10D3r1ck01: [C: 03+1] "LGTM!" [integration/config] - 10https://gerrit.wikimedia.org/r/670601 (owner: 10Zoranzoki21) [12:04:44] zpapierski: and it passed :] [12:09:24] (03CR) 10Hashar: [C: 03+2] Add MassMessage as dependency of the MassMessageEmail extension [integration/config] - 10https://gerrit.wikimedia.org/r/670601 (owner: 10Zoranzoki21) [12:10:34] (03Merged) 10jenkins-bot: Add MassMessage as dependency of the MassMessageEmail extension [integration/config] - 10https://gerrit.wikimedia.org/r/670601 (owner: 10Zoranzoki21) [12:21:33] 10Release-Engineering-Team, 10ApiFeatureUsage, 10Commons, 10Pywikibot, and 2 others: Persistant error 500 getting category members - https://phabricator.wikimedia.org/T255981 (10Inductiveload) [13:49:17] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Zache) @Ladsgroup just some notes: - "Reviewer" user group is enabled in finnish wikipedia. I think it is ok to drop... [13:53:50] 10Release-Engineering-Team (Logspam), 10MediaWiki-Revision-backend, 10MediaWiki-User-management, 10Move-Files-To-Commons, and 3 others: CannotCreateActorException: Cannot create an actor for a usable name that is not an existing user: … (via SpecialImportFile) - https://phabricator.wikimedia.org/T275067 (1... [14:02:18] 10VPS-project-Codesearch: Codesearch (Beta) not reachable - https://phabricator.wikimedia.org/T277459 (10Ladsgroup) I tried finding out what's going on but wasn't obvious. The docker image for frontend is running but connections to its port are being refused :/. I try to check again in depth. [15:04:10] (03CR) 10Hashar: [C: 03+2] "Puppet patch merged (I completely forgot about this)." [integration/docroot] - 10https://gerrit.wikimedia.org/r/666308 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [15:05:31] (03Merged) 10jenkins-bot: scap: restart php-fpm on promote [integration/docroot] - 10https://gerrit.wikimedia.org/r/666308 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [15:49:37] (03PS1) 10Hashar: scap: only restart php-fpm on doc* hosts [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) [15:54:52] (03CR) 10Awight: [C: 03+1] "Love it!" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [15:56:11] (03PS2) 10Hashar: scap: only restart php-fpm on doc* hosts [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) [15:58:38] (03PS3) 10Hashar: scap: only restart php-fpm on doc* hosts [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) [16:02:17] (03CR) 10Hashar: "Mukunda, that follows up https://gerrit.wikimedia.org/r/c/integration/docroot/+/666308/2/scap/checks.yaml" [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [16:08:15] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10thcipriani) p:05Triage→03Medium [16:09:51] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10brennen) > At this point I think the easiest optio... [16:13:23] 10Gerrit, 10Release-Engineering-Team: Gerrit Apache out of workers - https://phabricator.wikimedia.org/T277127 (10thcipriani) 05Open→03Resolved a:03Legoktm Tentatively resolving in the hopes that the sleep will make a difference here. If this problem persists let's reopen/open a new task. [16:13:37] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [quibble-stretch-php72-apache] Add custom log to time requests [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [16:15:10] 10Release-Engineering-Team: Ensure submodule updates (for security patches) are committed in the MW directory under /srv/mediawiki-staging - https://phabricator.wikimedia.org/T229285 (10thcipriani) 05Open→03Declined Declining since there's no action to take. Documentation on the wiki is correct and IIRC poin... [16:15:46] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Containers on releases hosts cannot update apt cache from non-WMF sources - https://phabricator.wikimedia.org/T277109 (10thcipriani) [16:16:28] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10MW-on-K8s, 10serviceops: Investigate how we can provide an mwdebug functionality on kubernetes - https://phabricator.wikimedia.org/T276994 (10thcipriani) [16:17:45] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10Platform Engineering: Enable PHP assertions in WMF CI - https://phabricator.wikimedia.org/T276940 (10thcipriani) [16:18:01] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [16:19:18] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10MW-on-K8s, 10serviceops: Progressive rollout of MediaWiki deployment on Kubernetes - https://phabricator.wikimedia.org/T276487 (10thcipriani) [16:20:01] 10Release-Engineering-Team-TODO, 10Front-end-Standards-Group, 10Product-Infrastructure-Team-Backlog, 10TechCom, and 2 others: Should npm packages maintained by Wikimedia be scoped or unscoped? - https://phabricator.wikimedia.org/T239742 (10thcipriani) [16:20:55] (03Merged) 10jenkins-bot: dockerfiles: [quibble-stretch-php72-apache] Add custom log to time requests [integration/config] - 10https://gerrit.wikimedia.org/r/671207 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [16:22:33] !log Docker: Publishing quibble-stretch-php72-apache:0.0.46-s1 [16:22:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:22:59] 10Gerrit, 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Shape Expressions, 10Wikidata, and 2 others: Fix gerrit / phab mirroring to https://github.com/wikimedia/mediawiki-extensions-EntitySchema - https://phabricator.wikimedia.org/T239264 (10thcipriani) p:05Triage→03Medium [16:23:22] 10Gerrit, 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Shape Expressions, 10Wikidata, and 2 others: Fix gerrit / phab mirroring to https://github.com/wikimedia/mediawiki-extensions-EntitySchema - https://phabricator.wikimedia.org/T239264 (10thcipriani) a:03thcipriani [16:25:32] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review: Switch Jenkins servers to Java 11 - https://phabricator.wikimedia.org/T269354 (10hashar) I have some trouble to set... [16:28:20] 10Release-Engineering-Team-TODO, 10MediaWiki-Debug-Logger, 10observability, 10Developer Productivity, 10Platform Engineering (Icebox): Consider enabling all MW log channels by default for WMF - https://phabricator.wikimedia.org/T228838 (10thcipriani) Does the new ECS stuff mean this is no longer valid? O... [16:32:46] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Readers-Web-Backlog (Tracking), 10Vue.js (Vue.js Search Experience (Vector modern)): Work with RelEng to add PipelineBot to Vector - https://phabricator.wikimedia.org/T257582 (10thcipriani) Noticed this in our backlog -- do you still want to pursue pip... [16:33:16] 10Gerrit, 10Release-Engineering-Team-TODO, 10Upstream: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10thcipriani) [16:34:07] 10Gerrit, 10Release-Engineering-Team-TODO, 10Upstream: Gitiles project landing pages should have an anonymous clone URL - https://phabricator.wikimedia.org/T206049 (10thcipriani) p:05High→03Low [16:37:00] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MediaWiki-Docker, 10Developer Productivity, and 3 others: mediawiki-docker exec feature - https://phabricator.wikimedia.org/T262370 (10thcipriani) [16:38:07] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Add link to risky change template to deployment blockers tasks - https://phabricator.wikimedia.org/T263547 (10thcipriani) a:03thcipriani [16:38:29] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Add link to risky change template to deployment blockers tasks - https://phabricator.wikimedia.org/T263547 (10thcipriani) p:05Triage→03Medium [16:43:54] 10Release-Engineering-Team-TODO, 10Release Pipeline: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10thcipriani) [16:44:53] 10Release-Engineering-Team-TODO, 10MediaWiki-Configuration: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10thcipriani) [16:46:47] 10Release-Engineering-Team-TODO, 10MediaWiki-Configuration, 10Epic: For containerisation, make MediaWiki-land production config injectable as much as possible - https://phabricator.wikimedia.org/T239133 (10thcipriani) [16:49:26] 10Release-Engineering-Team, 10Release Pipeline, 10serviceops-radar: Add testing entry point - https://phabricator.wikimedia.org/T238776 (10thcipriani) [16:49:29] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s: Multi-version MediaWiki image is built and published - https://phabricator.wikimedia.org/T274182 (10thcipriani) [16:49:58] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s: Multi-version MediaWiki image is built and published - https://phabricator.wikimedia.org/T274182 (10thcipriani) [16:50:02] 10Release-Engineering-Team-TODO, 10Release Pipeline, 10serviceops-radar: Create initial production MW-land images with blubber - https://phabricator.wikimedia.org/T238773 (10thcipriani) [16:51:28] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Zuul: zuul promote broken: Exception: Unable to find shared change queue for 534195,1 - https://phabricator.wikimedia.org/T231913 (10hashar) 05Open→03Resolved a:03hashar [17:07:23] (03CR) 1020after4: [C: 03+1] scap: only restart php-fpm on doc* hosts [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [17:10:27] (03PS1) 10Jforrester: jjb: Switch apache jobs to 0.0.46-s1 with request timing logging [integration/config] - 10https://gerrit.wikimedia.org/r/672759 (https://phabricator.wikimedia.org/T277202) [17:13:00] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/672759 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [17:14:26] (03Abandoned) 10Jforrester: jjb: update jobs for quibble-buster-php72:0.0.45-s4 [integration/config] - 10https://gerrit.wikimedia.org/r/662664 (https://phabricator.wikimedia.org/T250515) (owner: 10Hashar) [17:15:06] (03Merged) 10jenkins-bot: jjb: Switch apache jobs to 0.0.46-s1 with request timing logging [integration/config] - 10https://gerrit.wikimedia.org/r/672759 (https://phabricator.wikimedia.org/T277202) (owner: 10Jforrester) [17:21:25] (03CR) 1020after4: [C: 03+2] "I guess I can +2 this but I won't deploy it. I'll leave that up to Hashar" [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [17:23:03] (03Merged) 10jenkins-bot: scap: only restart php-fpm on doc* hosts [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [17:26:27] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Phatality, 10Wikimedia-Logstash, 10observability: Phatality doesn't work with Kibana 7 - https://phabricator.wikimedia.org/T272655 (10mmodell) 05Open→03Resolved a:05mmodell→03colewhite @cwhite deserves most of the credit for this on... [17:28:49] 10Release-Engineering-Team-TODO, 10Wikimedia-Site-requests: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10Legoktm) [17:30:28] 10Phabricator (Upstream), 10Upstream, 10User-zeljkofilipin: It's not possible to remove oneself from Phabricator project milestone - https://phabricator.wikimedia.org/T261642 (10mmodell) Milestones don't have members, if you are a member of the parent project you are automatically a member of the milestones. [17:31:13] 10Release-Engineering-Team-TODO, 10Wikimedia-Site-requests: Replace individual wfLoadExtension calls with a loop operating over an array - https://phabricator.wikimedia.org/T239137 (10Legoktm) I don't think there's much benefit to actually doing this, I guess you save 100 function calls. But extension registry... [17:31:27] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:48:35] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10stjn) I know ruWP used them experimentally for a time and should still have those user groups, so you should look into... [17:50:14] 10Continuous-Integration-Infrastructure, 10Quibble, 10Browser-Tests, 10Patch-For-Review: Can we make the quibble webserver used in CI log the time it has taken to execute calls to the API? - https://phabricator.wikimedia.org/T277202 (10Jdforrester-WMF) 05Open→03Resolved a:03hashar Before: https://int... [17:58:19] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10SRE, 10GitLab (Initialization), 10User-brennen: Define auth strategy for GitLab - https://phabricator.wikimedia.org/T274461 (10brennen) So @Sergey.Trofimovsky.SF reports getting prompted for a credential when trying to follow the user creat... [18:23:09] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10dancy) [18:24:01] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10dancy) [18:52:41] 10Beta-Cluster-Infrastructure: [beta cluster] UnderflowException: Ran out of captcha images - https://phabricator.wikimedia.org/T276176 (10Etonkovidova) I checked - created two new accounts on betacluster, the captcha was not present. Resolved? [19:21:50] 10Beta-Cluster-Infrastructure: [beta cluster] UnderflowException: Ran out of captcha images - https://phabricator.wikimedia.org/T276176 (10Tgr) Depends on your definition of resolved, ideally the captcha would be working. We are getting a steady trickle of spambots (although I don't think that's too much of a pr... [19:23:35] 10Beta-Cluster-Infrastructure: [beta cluster] UnderflowException: Ran out of captcha images - https://phabricator.wikimedia.org/T276176 (10Etonkovidova) >>! In T276176#6918950, @Tgr wrote: > Depends on your definition of resolved, ideally the captcha would be working. We are getting a steady trickle of spambots... [19:50:12] (03PS1) 10Dduvall: zuul: Remove helloworldoid branch condition for publish pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/672792 [19:55:02] (03PS1) 10Tpt: Adds Inductiveload to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/672795 [19:56:06] (03CR) 10Tpt: "Inductiveload is a very long term English Wikisource contributor who started to contribute to the ProofreadPage MediaWiki extension" [integration/config] - 10https://gerrit.wikimedia.org/r/672795 (owner: 10Tpt) [19:56:14] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10brennen) [19:58:56] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [20:00:31] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10Ladsgroup) [20:01:03] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [20:02:57] (03CR) 10DannyS712: [C: 03+1] Adds Inductiveload to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/672795 (owner: 10Tpt) [20:06:48] !log restarting zuul due to seemingly stuck dependency chain [20:06:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:08:57] theres a really long chain of parser patches, eg https://gerrit.wikimedia.org/r/c/mediawiki/core/+/665372/ [20:08:57] Probably going to help once the first three merge [20:28:40] 10Phabricator: Spaces request for Technical-Program-Management - https://phabricator.wikimedia.org/T277107 (10Aklapper) @MBinder_WMF: Project tags should never be completely hidden without a good reason (I'm aware of only one project tag where this is the case). Why would you want to do that? They are already pu... [20:58:57] (03PS1) 10MusikAnimal: Revert "Add labs/tools/wikisource-ocr" [integration/config] - 10https://gerrit.wikimedia.org/r/672808 [20:59:09] (03CR) 10jerkins-bot: [V: 04-1] Revert "Add labs/tools/wikisource-ocr" [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [21:05:00] (03PS2) 10MusikAnimal: Remove labs/tools/wikisource-ocr [integration/config] - 10https://gerrit.wikimedia.org/r/672808 [21:05:57] (03CR) 10Dduvall: [C: 03+2] zuul: Remove helloworldoid branch condition for publish pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/672792 (owner: 10Dduvall) [21:12:54] (03Merged) 10jenkins-bot: zuul: Remove helloworldoid branch condition for publish pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/672792 (owner: 10Dduvall) [21:18:39] (03CR) 10Jeena Huneidi: [V: 03+2 C: 03+2] elasticsearch: Update to latest version of search/extra plugin [releng/dev-images] - 10https://gerrit.wikimedia.org/r/668549 (https://phabricator.wikimedia.org/T276499) (owner: 10Kosta Harlan) [21:22:57] !log reloading zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/672792 [21:23:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:37:16] !log Updating dev-images docker-pkg files on primary contint [21:37:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:38:51] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Pywikibot, 10Pywikibot-tests: Lint test to match function signature and documentation - https://phabricator.wikimedia.org/T277396 (10Huji) @xqt does flake8 even have support for this? I know it supports finding unused arguments and it supports some... [21:43:21] 10Phabricator: Spaces request for Technical-Program-Management - https://phabricator.wikimedia.org/T277107 (10MBinder_WMF) @Aklapper The tag hiding isn't as important as the task hiding, if it's something we must live without. I think the use case is simply that nobody but the people listed would or should have... [22:04:15] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MediaWiki-Docker, 10dev-images, and 2 others: Permissions / ownership interfere with publishing dev-images - https://phabricator.wikimedia.org/T277604 (10brennen) [22:06:26] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MediaWiki-Docker, 10dev-images, and 2 others: Permissions / ownership interfere with publishing dev-images - https://phabricator.wikimedia.org/T277604 (10brennen) [22:06:27] 10Release-Engineering-Team-TODO, 10FR-Docker, 10Fundraising-Backlog, 10Gerrit-Privilege-Requests, and 3 others: dev-images +2 rights and Docker registry credentials for FR-Tech - https://phabricator.wikimedia.org/T274303 (10brennen) [22:46:42] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10SRE, 10GitLab (Initialization), 10User-brennen: Define auth strategy for GitLab - https://phabricator.wikimedia.org/T274461 (10jbond) >>! In T274461#6918624, @brennen wrote: > So @Sergey.Trofimovsky.SF reports getting prompted for a credent... [22:54:48] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Wugapodes) I use FlaggedRevs on enWiki and would like to see it retained as a protection option, so I plan to lend a ha... [23:20:55] 10Release-Engineering-Team (Logspam), 10Wikimedia-General-or-Unknown, 10JavaScript, 10MW-1.36-notes (1.36.0-wmf.34; 2021-03-09), 10Wikimedia-production-error: TypeError: format.replace is not a function in randomToken function used by SearchSatisfaction schema - https://phabricator.wikimedia.org/T272904 (... [23:53:00] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews, 10User-notice: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Ladsgroup) I went down a terrible rabbit hole yesterday. Deleted more than 1k lines of code. All dead...