[00:00:55] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews, 10User-notice: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10MBH) It's a good idea. There was a project in ruwiki to use 2-3 tiers to extended quality check, this... [00:06:16] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10Ladsgroup) [00:08:07] 10Release-Engineering-Team (Code Health), 10MediaWiki-extensions-FlaggedRevs, 10Code-Stewardship-Reviews, 10User-notice: Code stewardship review: FlaggedRevs - https://phabricator.wikimedia.org/T185664 (10Wugapodes) This sounds like the best long-term solution. How should we coordinate moving forward with... [00:17:23] 10Release-Engineering-Team-TODO, 10ci-test-error: Flaky selenium test: Error in "Page should be undoable": Can't call getText on element ... - https://phabricator.wikimedia.org/T277614 (10ori) [00:56:27] 10Release-Engineering-Team-TODO, 10ci-test-error: Flaky selenium test: Error in "Page should be undoable": Can't call getText on element ... - https://phabricator.wikimedia.org/T277614 (10DannyS712) [00:58:29] (03CR) 10Samwilson: [C: 03+1] Remove labs/tools/wikisource-ocr [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [01:17:56] (03PS3) 10Jforrester: Zuul: [labs/tools/wikisource-ocr] Remove CI [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [01:18:01] (03PS4) 10Jforrester: Zuul: [labs/tools/wikisource-ocr] Remove CI [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [01:18:11] (03CR) 10Jforrester: [C: 03+2] "Sorry to see you go." [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [01:19:38] (03Merged) 10jenkins-bot: Zuul: [labs/tools/wikisource-ocr] Remove CI [integration/config] - 10https://gerrit.wikimedia.org/r/672808 (owner: 10MusikAnimal) [01:21:16] !log Zuul: [labs/tools/wikisource-ocr] Remove CI [01:21:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [04:11:51] (03PS1) 10Zoranzoki21: layout.yaml: Archive the CopyToClipboard extension [integration/config] - 10https://gerrit.wikimedia.org/r/672869 (https://phabricator.wikimedia.org/T274015) [07:20:24] 10VPS-project-Codesearch, 10User-Ladsgroup: Codesearch (Beta) not reachable - https://phabricator.wikimedia.org/T277459 (10Ladsgroup) 05Open→03Resolved a:03Ladsgroup Fixed now. The port in docker run command was wrong. My bad, I don't know in `-p 80:3003` which is the host port and which is the container... [08:26:47] (03PS1) 10Hashar: Review access change [dumps] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672817 [08:27:33] (03PS2) 10Hashar: Make operations-dumps group owner of the hierarchy [dumps] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672817 (https://phabricator.wikimedia.org/T277630) [08:28:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-User-login-and-signup, 10MediaWiki-extensions-CentralAuth, 10User-Ryasmeen: Login failing - The provided authentication token is either expired or invalid. - https://phabricator.wikimedia.org/T104212 (10Aklapper) @59Efra: Hi, this tic... [08:30:41] 10Gerrit, 10Release-Engineering-Team-TODO, 10Developer-Advocacy, 10Upstream: Gitiles should add username in git clone links, triggers "Permission denied (publickey)" error - https://phabricator.wikimedia.org/T189182 (10hashar) [08:47:48] (03CR) 10Hashar: [V: 03+2 C: 03+2] Make operations-dumps group owner of the hierarchy [dumps] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672817 (https://phabricator.wikimedia.org/T277630) (owner: 10Hashar) [09:31:20] 10Continuous-Integration-Infrastructure, 10Quibble, 10Browser-Tests: Can we make the quibble webserver used in CI log the time it has taken to execute calls to the API? - https://phabricator.wikimedia.org/T277202 (10hashar) Since Jenkins artifacts tend to disappear: 127.0.0.1 - - [16/Mar/2021:16:54:37 +0000... [09:31:36] addshore: hi, you have request timing in the Quibble Apache logs now! :] [09:35:20] (03CR) 10Hashar: [C: 03+2] "I have rebased the deployment repo and deploy the change. That indeed restarted php7.0-fpm on doc1001 \o/" [integration/docroot] - 10https://gerrit.wikimedia.org/r/672741 (https://phabricator.wikimedia.org/T275468) (owner: 10Hashar) [09:36:28] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10serviceops: Apache on doc1001 does not see updated PHP files for hours/days after deployment - https://phabricator.wikimedia.org/T275468 (10hashar) 05Open→03Resolved We are now restarting php-fpm on the doc hosts. That clears out... [09:38:54] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Zuul, and 3 others: Improve scheduling of CI jobs invoked by zuul - https://phabricator.wikimedia.org/T258630 (10hashar) Bulk of the work has been... [09:44:08] hashar awesome! [10:11:21] 10Gerrit: Clean GerritSite.css from GWT styling - https://phabricator.wikimedia.org/T277645 (10hashar) [10:33:58] 10Gerrit, 10Release-Engineering-Team (Development services), 10Patch-For-Review: Clean GerritSite.css from GWT styling - https://phabricator.wikimedia.org/T277645 (10hashar) p:05Triage→03Low [10:51:43] vanishing [10:51:53] will dig into puppet / hiera madness this afternoon ;) [11:43:07] do none of the Wikibase.git jenkins CI jobs do a composer install any more? =o [11:44:11] did you break it? [11:44:16] I suspect you broke it [11:44:47] we didn't touch it, just realized that a composer install was failing, then had to fix the deps in one of the extensions [11:45:04] it partly our bad, as we missed on of the extensions, but CI was also just fine with it [11:50:20] im not sure if there is already a ticket that basically covers this, or if this is another issue [11:52:17] 10Release-Engineering-Team-TODO, 10Wikidata, 10User-Addshore: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663 (10Addshore) [11:52:53] I think this is https://phabricator.wikimedia.org/T178137 [11:59:33] 10Release-Engineering-Team-TODO, 10Wikidata, 10wdwb-tech-focus, 10User-Addshore: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663 (10Addshore) [12:16:08] 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MediaWiki-extensions-InterwikiSorting, 10Patch-For-Review, 10Technical-Debt: Undeploy the InterwikiSorting extension from Wikipedia production - https://phabricator.wikimedia.org/T253764 (10Nikerabbit) Is anyone driving this task? There are... [12:55:49] 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MediaWiki-extensions-InterwikiSorting, 10Patch-For-Review, 10Technical-Debt: Undeploy the InterwikiSorting extension from Wikipedia production - https://phabricator.wikimedia.org/T253764 (10Addshore) > Also users can opt out from using comp... [12:59:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Wikidata, and 5 others: Create docker based CI job to build the wikidata-query-gui - https://phabricator.wikimedia.org/T210286 (10Addshore) The integration-config patches are alre... [14:01:22] (03PS1) 10ZPapierski: Add verbose logging for sonar [integration/config] - 10https://gerrit.wikimedia.org/r/673022 (https://phabricator.wikimedia.org/T264877) [14:25:17] 10Beta-Cluster-Infrastructure: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Majavah) Taking a look, this is suspiciously timed with my work on {T276462}. ` ● fetch_dbconfig.service loaded failed failed Fetch the dbconfig from et... [14:26:53] (03CR) 10WMDE-leszek: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/673028 (owner: 10WMDE-leszek) [14:28:59] 10Phabricator (Search): Phabricator global search: "Cannot use object of type PhutilSafeHTML as array" error for certain strings - https://phabricator.wikimedia.org/T263063 (10mmodell) 05Open→03Resolved a:03mmodell [14:43:26] 10Beta-Cluster-Infrastructure: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Majavah) The presence of said file is controlled by the hiera key `profile::conftool::state::ensure` which has been set to false [[ https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/b7130... [14:44:51] 10Beta-Cluster-Infrastructure: Beta cron jobs seem broken - https://phabricator.wikimedia.org/T277206 (10Majavah) Might be a duplicate of {T125976}, not sure. [14:49:42] (03CR) 10Addshore: [C: 03+2] "LGTM!" [integration/config] - 10https://gerrit.wikimedia.org/r/673028 (owner: 10WMDE-leszek) [14:52:18] (03Merged) 10jenkins-bot: Run more Wikibase tests jobs for REL1_35 branch [integration/config] - 10https://gerrit.wikimedia.org/r/673028 (owner: 10WMDE-leszek) [14:53:06] !log reload zuul for https://gerrit.wikimedia.org/r/673028 Run more Wikibase tests jobs for REL1_35 branch [14:53:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:02:18] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10Urbanecm) >>! In T277507#6915689, @Jdforrester-WMF... [15:32:31] !log taavi@deployment-restbase01:~$ sudo nodetool decomission # T250574 [15:32:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:32:38] T250574: Replace deployment-restbase0[12] - https://phabricator.wikimedia.org/T250574 [15:37:06] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Core subproject commits aren't updated for backports if wmf branch commit has not yet been merged. - https://phabricator.wikimedia.org/T277507 (10brennen) > Which documentation are you talking abo... [15:37:32] !log shutdown deployment-restbase01 for T250574 [15:37:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:39:39] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation), 10Patch-For-Review: Replace deployment-restbase0[12] - https://phabricator.wikimedia.org/T250574 (10Majavah) I removed deployment-restbase01 from the Cassandra cluster and shut it down. I'll do the same for 02 if no problems arise. [15:40:04] (03PS4) 10Jgleeson: WIP: Added image for civiproxy [releng/dev-images] - 10https://gerrit.wikimedia.org/r/664919 [15:41:28] does anyone happen to know what deployment-kafka-jumbo-3 is used for? it is failing puppet due to it being out of memory to make any new processes [15:41:58] analytics stuff, presumably [15:50:48] (03CR) 10Jgleeson: "Updates in the latest PS." (032 comments) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/664919 (owner: 10Jgleeson) [15:54:21] (03PS5) 10Jgleeson: WIP: Added image for civiproxy [releng/dev-images] - 10https://gerrit.wikimedia.org/r/664919 [16:08:03] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10dancy) group0 at 1.36.0-wmf.35 [17:04:36] 10Release-Engineering-Team-TODO, 10Wikidata, 10wdwb-tech-focus, 10User-Addshore: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663 (10Addshore) This happened today as noted in T178137#6921107 [17:06:39] (03PS1) 10WMDE-leszek: Review access change [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 [17:08:13] (03CR) 10WMDE-leszek: [C: 03+2] Review access change [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 (owner: 10WMDE-leszek) [17:10:43] (03CR) 10WMDE-leszek: [V: 03+2 C: 03+2] Review access change [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 (owner: 10WMDE-leszek) [17:15:42] (03CR) 10WMDE-leszek: [V: 03+2 C: 03+2] "WMDE staff apparently does not have +2 rights on analytics/wmde/scripts repo which I believe is a mistake" [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 (owner: 10WMDE-leszek) [17:16:00] (03CR) 10WMDE-leszek: "removed my votes as they do not do anything" [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 (owner: 10WMDE-leszek) [17:23:17] (03CR) 10Hashar: [C: 03+2] Add verbose logging for sonar [integration/config] - 10https://gerrit.wikimedia.org/r/673022 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [17:24:35] (03Merged) 10jenkins-bot: Add verbose logging for sonar [integration/config] - 10https://gerrit.wikimedia.org/r/673022 (https://phabricator.wikimedia.org/T264877) (owner: 10ZPapierski) [17:25:01] (03CR) 10Ahmon Dancy: [C: 03+1] apt: Support configuration of http/https proxies [blubber] - 10https://gerrit.wikimedia.org/r/671199 (https://phabricator.wikimedia.org/T277109) (owner: 10Dduvall) [17:25:51] (03CR) 10Ahmon Dancy: [C: 03+1] Provide `setup.httpProxy` context variable if a proxy is configured [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/671211 (https://phabricator.wikimedia.org/T277109) (owner: 10Dduvall) [17:28:46] 10Beta-Cluster-Infrastructure, 10Puppet: Unduplicate beta cluster hiera keys set both in Horizon and in ops/puppet - https://phabricator.wikimedia.org/T277680 (10Majavah) [17:44:57] Majavah: o/ yes, its mostly used to reproduce some data ingestion stuff that we have in production [17:45:08] will look at it [17:45:22] thanks! [17:54:40] 10Beta-Cluster-Infrastructure: Beta SWIFT seems to be broken - https://phabricator.wikimedia.org/T276179 (10Majavah) >>! In T276179#6902201, @Tgr wrote: > It's all 503s again. deployment-mwlog01:/srv/mw-log/FileOperation.log at least has 404s when trying to access URLs under http://deployment-ms-fe03.deployment-... [18:02:14] (03PS1) 10Dduvall: wip: Tag images with names of newly pushed git tags [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/673070 [18:04:23] (03CR) 10jerkins-bot: [V: 04-1] wip: Tag images with names of newly pushed git tags [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/673070 (owner: 10Dduvall) [18:56:28] (03CR) 10Dduvall: "Tested against helloworldoid-pipeline-publish where you can see the newly bound `setup.tag` context variable." [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/673070 (https://phabricator.wikimedia.org/T277346) (owner: 10Dduvall) [19:16:15] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.35 deployment blockers - https://phabricator.wikimedia.org/T274939 (10dancy) group0 and group1 at 1.36.0-wmf.35 [19:39:04] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T274940 (10Catrope) ### Risky Patch! 🚂🔥 * **Change**: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/672487 * **What it does/What it's r... [19:44:13] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T274940 (10dancy) Thanks for the risk report @Catrope [20:12:27] (03PS1) 10Brennen Bearnes: check-new-error-tasks/check.sh: List of new error filter task statuses [tools/release] - 10https://gerrit.wikimedia.org/r/673095 [20:13:36] (03PS2) 10Brennen Bearnes: check-new-error-tasks/check.sh: List of new error filter task statuses [tools/release] - 10https://gerrit.wikimedia.org/r/673095 [20:14:07] (03CR) 10Hashar: [C: 03+2] build: Updating eslint-config-wikimedia to 0.18.2 [integration/docroot] - 10https://gerrit.wikimedia.org/r/669950 (owner: 10Libraryupgrader) [20:20:40] (03CR) 10Hashar: [C: 03+2] Adds Inductiveload to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/672795 (owner: 10Tpt) [20:21:52] (03Merged) 10jenkins-bot: Adds Inductiveload to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/672795 (owner: 10Tpt) [20:30:34] !log Reloaded Zuul for I2368478e4c4ab8752581f55a7c5ab493fafdeb41 [20:30:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:34:14] (03PS1) 10Addshore: Use quibble-stretch-php73:0.0.46 for wikibase-*-docker jobs [integration/config] - 10https://gerrit.wikimedia.org/r/673096 [20:36:17] (03CR) 10Addshore: [C: 03+2] "deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/673096 (owner: 10Addshore) [20:38:00] (03CR) 10Hashar: "deploy" [integration/config] - 10https://gerrit.wikimedia.org/r/672795 (owner: 10Tpt) [20:38:04] (03Merged) 10jenkins-bot: Use quibble-stretch-php73:0.0.46 for wikibase-*-docker jobs [integration/config] - 10https://gerrit.wikimedia.org/r/673096 (owner: 10Addshore) [21:28:41] 10MediaWiki-Codesniffer: Namespace confusion - https://phabricator.wikimedia.org/T277708 (10Reedy) [21:30:06] Reedy: why are you putting two namespaces in the same file? [21:30:51] Why not? [21:31:06] More it was a simple way to test https://gerrit.wikimedia.org/r/c/mediawiki/core/+/672533 [21:31:13] Rather than faffing around with multiple files etc [21:35:27] (03CR) 10Legoktm: "See T277709, I believe I've fixed it a different way." [analytics/wmde/scripts] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/672824 (owner: 10WMDE-leszek) [21:36:26] because it breaks the sniff :p [21:36:36] I think if you did it as namespace Foo { .... } it would work correctly [21:40:10] 10MediaWiki-Codesniffer: Namespace confusion - https://phabricator.wikimedia.org/T277708 (10Legoktm) IIRC the sniff works fine with `namespace Foo { .... }`. [21:45:39] legoktm: https://youtrack.jetbrains.com/issue/WI-59299 edge cases! [21:46:39] :o [21:50:50] 10MediaWiki-Codesniffer: Namespace confusion - https://phabricator.wikimedia.org/T277708 (10Reedy) >>! In T277708#6923176, @Legoktm wrote: > IIRC the sniff works fine with `namespace Foo { .... }`. If both are braced, it does indeed. [21:51:36] 10MediaWiki-Codesniffer: Namespace confusion - https://phabricator.wikimedia.org/T277708 (10Reedy) (of course, braced namespaces isn't something we seemingly do in MediaWiki codebases) [22:28:28] 10Release-Engineering-Team, 10Abstract Wikipedia (Phase δ): Test to ensure the independent usage of Front-End components - https://phabricator.wikimedia.org/T272439 (10DVrandecic) [22:41:56] 10MediaWiki-Codesniffer: Namespace confusion - https://phabricator.wikimedia.org/T277708 (10DannyS712) >>! In T277708#6923197, @Reedy wrote: > (of course, braced namespaces isn't something we seemingly do in MediaWiki codebases) Well, it isn't common, but files that need to have code in multiple namespaces use... [22:45:52] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10MW-on-K8s: Proof of concept for multi-version MediaWiki images - https://phabricator.wikimedia.org/T271448 (10dancy) 05Open→03Resolved [22:45:54] 10Release-Engineering-Team, 10Release Pipeline, 10serviceops-radar, 10Epic: Get production MW-land images built and published - https://phabricator.wikimedia.org/T238771 (10dancy) [23:00:53] (03CR) 10Thcipriani: [C: 03+2] check-new-error-tasks/check.sh: List of new error filter task statuses (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/673095 (owner: 10Brennen Bearnes) [23:02:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Abstract Wikipedia: Provide a CI mechanism by which we can assert that front end components can run independently of the MW back-end on why they rely - https://phabricator.wikimedia.org/T277720 (10Jdforrester-WMF) [23:02:26] (03Merged) 10jenkins-bot: check-new-error-tasks/check.sh: List of new error filter task statuses [tools/release] - 10https://gerrit.wikimedia.org/r/673095 (owner: 10Brennen Bearnes) [23:19:50] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Release, 10Train Deployments: 1.36.0-wmf.36 deployment blockers - https://phabricator.wikimedia.org/T274940 (10thcipriani)