[00:05:54] 10VPS-project-Codesearch, 10Patch-For-Review: Add ShoutHow to Codesearch - https://phabricator.wikimedia.org/T253597 (10Legoktm) 05Open→03Resolved a:03Legoktm https://codesearch.wmcloud.org/shouthow/?q=Jack%20Phoenix&i=nope&files=&excludeFiles=&repos= [01:22:29] 10Phabricator, 10Project-Admins, 10phabricator maintenance bot: #phabricator_maintenance_bot should be under #tools, not #toolforge - https://phabricator.wikimedia.org/T273273 (10Ladsgroup) I swear someone told me to make it under toolforge.... I'm sorry for the mistake. I try to find a way to fix it. [02:58:06] (03PS2) 10Jeena Huneidi: PipelineRunner: allowedCredentials [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/668245 (https://phabricator.wikimedia.org/T269902) [03:02:20] (03PS3) 10Jeena Huneidi: PipelineRunner: allowedCredentials [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/668245 (https://phabricator.wikimedia.org/T269902) [03:11:52] (03PS4) 10Jeena Huneidi: PipelineRunner: allowedCredentials [integration/pipelinelib] - 10https://gerrit.wikimedia.org/r/668245 (https://phabricator.wikimedia.org/T269902) [07:34:31] 10Beta-Cluster-Infrastructure, 10Kubernetes: Re-consider setting up a Kubernetes cluster on the Beta cluster - https://phabricator.wikimedia.org/T276650 (10Majavah) [07:35:01] 10Beta-Cluster-Infrastructure, 10Kubernetes: Re-consider setting up a Kubernetes cluster on the Beta cluster - https://phabricator.wikimedia.org/T276650 (10Majavah) [07:35:05] 10Beta-Cluster-Infrastructure, 10SRE, 10Puppet, 10Technical-Debt, 10Tracking-Neverending: Minimize differences between beta and production (Tracking) - https://phabricator.wikimedia.org/T87220 (10Majavah) [07:36:39] legoktm, James_F: FYI I made https://phabricator.wikimedia.org/T276650 based on the discussion yesterday [07:40:58] thanks [07:41:21] 10Beta-Cluster-Infrastructure, 10Kubernetes: Re-consider setting up a Kubernetes cluster on the Beta cluster - https://phabricator.wikimedia.org/T276650 (10Legoktm) There is some discussion at T215217#6610236 that is related to this. [08:01:49] 10Beta-Cluster-Infrastructure: deployment-mx02 puppet failures, deployment-acme-chief03 gives a 403 - https://phabricator.wikimedia.org/T276652 (10Majavah) [08:09:20] !log deployment-acme-chief change authorized regex for mx to use .eqiad1.wikimedia.cloud domain to fix T276652 [08:09:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:09:25] T276652: deployment-mx02 puppet failures, deployment-acme-chief03 gives a 403 - https://phabricator.wikimedia.org/T276652 [08:10:14] 10Beta-Cluster-Infrastructure, 10User-Majavah: deployment-mx02 puppet failures, deployment-acme-chief03 gives a 403 - https://phabricator.wikimedia.org/T276652 (10Majavah) 05Open→03Resolved a:03Majavah [08:12:54] 10Beta-Cluster-Infrastructure, 10Acme-chief: deployment-acme-chief has production non-canonical redirects for some reason - https://phabricator.wikimedia.org/T276653 (10Majavah) [08:19:18] 10Beta-Cluster-Infrastructure: deployment-prometheus02 puppet failures due to Apache not recognizing "RewriteRule" - https://phabricator.wikimedia.org/T276654 (10Majavah) [08:56:23] 10Phabricator: Proposed changes to the SRE Access request - https://phabricator.wikimedia.org/T276473 (10Aklapper) I assume this is about editing https://phabricator.wikimedia.org/maniphest/task/edit/form/8/ [08:56:38] 10Phabricator: Proposed changes to the SRE Access request (Phabricator form 8) - https://phabricator.wikimedia.org/T276473 (10Aklapper) [08:59:33] 10Phabricator: Unhandled exception clicking "Next" button on Notifications page: Rows must have unique IDs. An underlying query may be missing a GROUP BY. - https://phabricator.wikimedia.org/T276447 (10Aklapper) p:05Triage→03Low [09:00:46] 10Phabricator (Upstream), 10Upstream: Unhandled exception clicking "Next" button on Notifications page: Rows must have unique IDs. An underlying query may be missing a GROUP BY. - https://phabricator.wikimedia.org/T276447 (10Aklapper) This seems to be about https://phabricator.wikimedia.org/notification/ [09:01:35] 10Phabricator (Upstream), 10Upstream: Unhandled exception clicking "Next" button on Notifications page: Rows must have unique IDs. An underlying query may be missing a GROUP BY. - https://phabricator.wikimedia.org/T276447 (10Aklapper) [09:21:33] 10Phabricator, 10TechCom-RFC, 10tech-decision-forum: Either update or disable Phabricator Form 82 ("Create a TechCom RFC") - https://phabricator.wikimedia.org/T276655 (10Aklapper) [09:23:06] 10Beta-Cluster-Infrastructure, 10ContentTranslation: Beta cluster Special:ContentTranslation Error: Unsupported operand types - https://phabricator.wikimedia.org/T276656 (10Majavah) [09:31:53] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate deployment-prep away from Debian Jessie to Debian Stretch/Buster - https://phabricator.wikimedia.org/T218729 (10Majavah) [09:32:41] 10Phabricator: Request Edit permissions to the "SRE Access Request" form - https://phabricator.wikimedia.org/T276446 (10Aklapper) The Phab UI for Forms is highly confusing and I also continuously struggle to understand it. First step: Could you check if you can access https://phabricator.wikimedia.org/transacti... [09:34:56] 10Phabricator, 10I18n: Turkish translations issue - https://phabricator.wikimedia.org/T276657 (10Majavah) [09:48:49] !log cherry-pick https://gerrit.wikimedia.org/r/668995 on deployment-puppetmaster04 T276654 [09:48:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:48:54] T276654: deployment-prometheus02 puppet failures due to Apache not recognizing "RewriteRule" - https://phabricator.wikimedia.org/T276654 [09:53:38] 10Beta-Cluster-Infrastructure, 10Patch-For-Review, 10User-Majavah: deployment-prometheus02 puppet failures due to Apache not recognizing "RewriteRule" - https://phabricator.wikimedia.org/T276654 (10Majavah) a:03Majavah Fix cherrypicked on beta, leaving open until it's merged on the actual operations/puppet... [09:56:04] 10Project-Admins: Create an #ai-governance tag - https://phabricator.wikimedia.org/T276639 (10Aklapper) > Project description: Projects or tasks related to AI/ML Governance Hi, could you elaborate what "AI/ML Governance" means (if I was a Wikimedia outsider), and/or link to more information? Thanks a lot! :) [10:03:22] 10Phabricator, 10I18n: Turkish translations issue - https://phabricator.wikimedia.org/T276657 (10Aklapper) What are the "Date Format" and "Time Format" on https://phabricator.wikimedia.org/settings/panel/datetime/ ? [10:10:28] 10Phabricator, 10I18n: Date formats broken in Turkish, shows AM/PM instead of month names - https://phabricator.wikimedia.org/T276657 (10Aklapper) [10:17:50] 10Phabricator, 10I18n: Date formats broken in Turkish, shows AM/PM instead of month names - https://phabricator.wikimedia.org/T276657 (10Joseph) >>! T276657#6888816 içinde, @Aklapper yazdı: > What are the "Date Format" and "Time Format" on https://phabricator.wikimedia.org/settings/panel/datetime/ ? Date form... [10:55:49] 10Beta-Cluster-Infrastructure, 10Kubernetes: Re-consider setting up a Kubernetes cluster on the Beta cluster - https://phabricator.wikimedia.org/T276650 (10Krinkle) [16:02:30] 10Release-Engineering-Team (Logspam), 10Wikimedia-General-or-Unknown, 10JavaScript, 10Wikimedia-production-error: TypeError: format.replace is not a function in randomToken function used by SearchSatisfaction schema - https://phabricator.wikimedia.org/T272904 (10Umherirrender) I have found https://stackove... [16:46:31] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10GitLab (Initialization), 10User-brennen: Remove Speed & Function blockers for GitLab work - https://phabricator.wikimedia.org/T274458 (10wkandek) [16:47:54] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10SRE, 10Traffic, 10GitLab (Initialization), and 2 others: open firewall ports on gitlab1001.wikimedia.org (was: Port map of how Gitlab is accessed) - https://phabricator.wikimedia.org/T276144 (10wkandek) I created https://phabricator.wikimed... [17:35:24] 10Beta-Cluster-Infrastructure, 10ContentTranslation: Beta cluster Special:ContentTranslation Error: Unsupported operand types - https://phabricator.wikimedia.org/T276656 (10Daimona) Note: you must be logged in order to reproduce this bug. Also, I'm not getting why it's reported on line 286, but I guess the +... [18:05:12] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MediaWiki-Vendor: Ensure with test that composer.json matches between mediawiki/vendor and mediawiki/core - https://phabricator.wikimedia.org/T113360 (10Jdforrester-WMF) Surely we implicitly test this, because we test MediaWiki core patches with bo... [19:44:59] !log restart deployment-logstash03 to see if it fixes it being empty [19:45:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:51:05] deployment-logstash03 still does not have any mediawiki events, found "low disk watermark [75%] exceeded on [nPHkwHX1SqS6G5Zdrz3ieA][deployment-logstash03-labs-logstash-eqiad][/srv/elasticsearch/labs-logstash-eqiad/nodes/0] free: 29.4gb[21.4%], replicas will not be assigned to this node" in it, not sure if related or what to do about it [20:13:39] 10Scap, 10MediaWiki-Vendor: Scap sync fails because it tries to lint test files - https://phabricator.wikimedia.org/T202984 (10Jdforrester-WMF) 05Open→03Resolved a:03Tgr I think this is sufficiently solved. [22:34:09] (03PS1) 10Zabe: Add Artrix in the CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/669192