[00:01:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:01:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:16:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [02:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [04:06:47] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<20.00%) [04:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [05:01:00] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:11:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:17:15] (03Abandoned) 10Pwirth: layout: [BlueSpiceSocial] Disable HHVM tests, this doesn't work there [integration/config] - 10https://gerrit.wikimedia.org/r/528622 (owner: 10Pwirth) [07:11:46] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [08:29:13] 10Release-Engineering-Team-TODO, 10CirrusSearch, 10Discovery-Search, 10MW-1.34-notes (1.34.0-wmf.16; 2019-07-30), 10Patch-For-Review: wdio-cucumber-framework fails on NodeJS due to fibers@2.x under nodejs10 - https://phabricator.wikimedia.org/T229033 (10hashar) 05Open→03Stalled Stall / pending NodeJS... [08:29:16] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10MediaWiki-Core-Testing, 10Quibble, and 2 others: Quibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests - https://phabricator.wikimedia.org/T199116 (10hashar) [08:46:22] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Jony) Hi @Aklapper would you update any status. [08:54:34] (03CR) 10Hashar: [C: 03+2] MachineVision: Depend on Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/530598 (owner: 10Mholloway) [08:56:07] (03Merged) 10jenkins-bot: MachineVision: Depend on Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/530598 (owner: 10Mholloway) [09:06:07] (03CR) 10Hashar: "The py3 tests fail though either with master or https://gerrit.wikimedia.org/r/#/c/labs/tools/maintain-kubeusers/+/530458/ :]" [integration/config] - 10https://gerrit.wikimedia.org/r/530606 (https://phabricator.wikimedia.org/T228499) (owner: 10Bstorm) [09:06:18] (03CR) 10Hashar: [C: 04-1] integration: add tests for the labs/tools/maintain-kubeusers repo [integration/config] - 10https://gerrit.wikimedia.org/r/530606 (https://phabricator.wikimedia.org/T228499) (owner: 10Bstorm) [10:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [10:24:11] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (201908), 10Release, 10Train Deployments, 10User-zeljkofilipin: 1.34.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T220744 (10alaa_wmde) [10:37:54] 10Continuous-Integration-Config: Jenkins check for vulnerable libraries in all node.js repos - https://phabricator.wikimedia.org/T96078 (10hashar) We now have package-lock.json being committed and should be able to run `npm audit`. [11:43:11] (03PS1) 10Daimona Eaytoy: Add phan and seccheck to MachineVision [integration/config] - 10https://gerrit.wikimedia.org/r/530840 [11:47:15] James_F: hello! are you there for a quick chat? [12:01:36] 10Continuous-Integration-Config, 10Gerrit, 10Release-Engineering-Team-TODO, 10Operations: Fix operations/puppet.git "rebase hell" - https://phabricator.wikimedia.org/T224033 (10hashar) [12:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [12:18:00] 10Release-Engineering-Team-TODO, 10CirrusSearch, 10Discovery-Search, 10Browser-Tests: Jenkins can not send emails to Discovery-alerts mailling list - https://phabricator.wikimedia.org/T230719 (10hashar) [12:19:58] 10Release-Engineering-Team-TODO, 10CirrusSearch, 10Discovery-Search, 10Browser-Tests: Jenkins can not send emails to Discovery-alerts mailling list - https://phabricator.wikimedia.org/T230719 (10hashar) a:03EBernhardson I have subscribed the Jenkins email to the mailing list. It is pending approval: > Y... [12:23:31] 10Release-Engineering-Team-TODO, 10Cleanup, 10Electron-PDFs, 10Reading-Infrastructure-Team-Backlog, and 2 others: Archive mediawiki/services/electron-render and mediawiki/services/electron-render/deploy repositories - https://phabricator.wikimedia.org/T230720 (10hashar) [12:26:55] 10Release-Engineering-Team-TODO, 10Cleanup, 10Electron-PDFs, 10Reading-Infrastructure-Team-Backlog, and 2 others: Archive mediawiki/services/electron-render and mediawiki/services/electron-render/deploy repositories - https://phabricator.wikimedia.org/T230720 (10hashar) 05Open→03Resolved a:03hashar... [12:32:13] 10Release-Engineering-Team-TODO, 10CirrusSearch, 10Discovery-Search, 10Browser-Tests: Jenkins can not send emails to Discovery-alerts mailling list - https://phabricator.wikimedia.org/T230719 (10hashar) 05Open→03Resolved a:05EBernhardson→03hashar Eventually I have subscribed the jenkins-bot email a... [14:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [14:13:25] (03PS1) 10Volans: Fix alphabetic order of projects [integration/config] - 10https://gerrit.wikimedia.org/r/530858 [14:13:27] (03PS1) 10Volans: Setup CI for operations/software/homer [integration/config] - 10https://gerrit.wikimedia.org/r/530859 (https://phabricator.wikimedia.org/T228388) [14:16:54] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Create mirror of Gerrit repositories for consumption by various tools - https://phabricator.wikimedia.org/T226240 (10hashar) >>! In T226240#5394306, @Dzahn wrote: > @hashar Does this resolve the ticket or is it part... [14:18:55] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Create mirror of Gerrit repositories for consumption by various tools - https://phabricator.wikimedia.org/T226240 (10Paladox) @hashar phabricator has been migrated to use the replica :) (Codesearch and extdist also... [14:23:03] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Quibble: Fix quibble to silence INFO log output from the backend server in karma runs unless a test fails - https://phabricator.wikimedia.org/T229891 (10hashar) I have indeed made Quibble to relay the PHP embedded web serv... [14:26:17] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikimedia-Logstash, 10observability: logstash-beta not recording since April, 20th 2018 - https://phabricator.wikimedia.org/T193133 (10fgiunchedi) [14:26:19] 10Beta-Cluster-Infrastructure, 10Deployments, 10Wikimedia-Logstash, 10observability: scap on beta cluster does not run anymore due to logstash being down - https://phabricator.wikimedia.org/T143982 (10fgiunchedi) [14:26:40] 10Phabricator, 10Wikimedia-Logstash, 10observability, 10Security: Include Phabricator apache access logs in Logstash - https://phabricator.wikimedia.org/T219278 (10fgiunchedi) [14:26:44] 10Gerrit, 10Phabricator, 10Wikimedia-Logstash, 10observability, and 2 others: Include Gerrit apache logs in Logstash - https://phabricator.wikimedia.org/T219271 (10fgiunchedi) [14:26:56] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: deployment prep's logstash-beta.wmflabs.org reports no logs since Jan 14th - https://phabricator.wikimedia.org/T215204 (10fgiunchedi) [14:27:02] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: Beta Cluster ("deployment-prep") logstash has gone silent since 2019-01-02Z01:14:14 - https://phabricator.wikimedia.org/T213129 (10fgiunchedi) [14:27:09] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: Logstash in beta fails periodically - https://phabricator.wikimedia.org/T211984 (10fgiunchedi) [14:27:11] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: Logstash in beta doesn't have any logs - https://phabricator.wikimedia.org/T205863 (10fgiunchedi) [14:27:25] 10Beta-Cluster-Infrastructure, 10Elasticsearch, 10Wikimedia-Logstash, 10observability, 10Discovery-Search (Current work): ApiFeatureUsage data is not being populated in the Beta Cluster - https://phabricator.wikimedia.org/T183156 (10fgiunchedi) [14:28:16] 10Release-Engineering-Team, 10AbuseFilter, 10Wikimedia-Logstash, 10observability, and 2 others: Should we keep StashEdit logs in AbuseFilter? - https://phabricator.wikimedia.org/T146697 (10fgiunchedi) [14:29:14] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10observability: Upgrade ElasticSearch on production logstash boxen to 1.3.6 - https://phabricator.wikimedia.org/T76089 (10fgiunchedi) [14:29:46] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: deployment-logstash2 out of disk space - https://phabricator.wikimedia.org/T170521 (10fgiunchedi) [14:29:52] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability, 10Wikimedia-production-error: Fatalmonitor on logstash still includes deprecated channel:wfLogDBError - https://phabricator.wikimedia.org/T165675 (10fgiunchedi) [14:30:23] 10Beta-Cluster-Infrastructure, 10Cloud-Services, 10Wikimedia-Logstash, 10observability, 10Patch-For-Review: Logstash on beta yields 500 due to NFS outage (can't open /data/project/logstash/.htpasswd) - https://phabricator.wikimedia.org/T102962 (10fgiunchedi) [14:30:25] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability, 10Patch-For-Review: Build jessie based elasticsearch/logstash/kibana (ELK) host for beta testing - https://phabricator.wikimedia.org/T101541 (10fgiunchedi) [14:35:13] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability, 10Patch-For-Review: deployment-logstash02 fails puppet: Apache2 can't start, mod_authz_groupfile not enabled on Jessie - https://phabricator.wikimedia.org/T103804 (10fgiunchedi) [14:35:36] 10Beta-Cluster-Infrastructure, 10Sentry, 10Wikimedia-Logstash, 10observability, 10Patch-For-Review: Channel PHP errors from Logstash to Sentry on the beta cluster - https://phabricator.wikimedia.org/T85239 (10fgiunchedi) [14:35:38] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: Make beta logstash server based on a Trusty base image - https://phabricator.wikimedia.org/T78195 (10fgiunchedi) [14:35:47] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikimedia-Logstash, 10observability, 10Patch-For-Review: Make logstash in beta public - https://phabricator.wikimedia.org/T76784 (10fgiunchedi) [14:36:10] 10Gerrit, 10Wikimedia-Logstash, 10observability: Add gerrit event stream events to logstash - https://phabricator.wikimedia.org/T64505 (10fgiunchedi) [14:36:40] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10observability: Upgrade logstash to 1.4.2 - https://phabricator.wikimedia.org/T4 (10fgiunchedi) [14:36:54] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org default dashboard missing - https://phabricator.wikimedia.org/T184602 (10fgiunchedi) [14:37:16] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, and 3 others: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705 (10fgiunchedi) [14:40:10] (03PS1) 10Kosta Harlan: [quibble-coverage] Generate coverage with unit tests in codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/530866 (https://phabricator.wikimedia.org/T230423) [14:40:12] (03PS1) 10Kosta Harlan: jjb: [mwext-codehealth*] Use quibble-coverage 0.0.35 [integration/config] - 10https://gerrit.wikimedia.org/r/530867 (https://phabricator.wikimedia.org/T230423) [14:41:44] (03Abandoned) 10Kosta Harlan: Add tideways_xhprof [releng/dev-images] - 10https://gerrit.wikimedia.org/r/511063 (https://phabricator.wikimedia.org/T223670) (owner: 10Kosta Harlan) [15:16:40] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services): AbuseFilterParserTest::tearDown() must call parent::tearDown() exception within quibble-vendor-mysql-hhvm-docker - https://phabricator.wikimedia.org/T230452 (10Umherirrender) Sounds that it is T189560, because the erro... [15:37:05] 10Release-Engineering-Team: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10Etonkovidova) [15:38:28] 10Release-Engineering-Team: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) p:05Triage→03Normal [15:38:39] 10Release-Engineering-Team (CI & Testing services): Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [15:39:05] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [15:39:44] 10Release-Engineering-Team (CI & Testing services): Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [15:40:18] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908): Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [15:40:35] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [16:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [16:37:39] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10Epic, 10MW-1.34-release: Expand the set of bundled extensions and skins in MediaWiki 1.34 - https://phabricator.wikimedia.org/T209220 (10Umherirrender) [16:54:43] 10MediaWiki-Codesniffer: Improve or disable SingleSpaceBeforeSingleLineComment - https://phabricator.wikimedia.org/T222853 (10Daimona) Yes please. This sniff bites me pretty often. For instance, if I want to write a comment including an indented list for a delicate part of the code, I have to give up and just in... [17:05:30] PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [140.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:12:58] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [17:13:34] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [17:20:01] (03PS1) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T230727) [17:20:52] (03PS2) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [17:21:25] 10MediaWiki-Codesniffer, 10Patch-For-Review: Improve or disable SingleSpaceBeforeSingleLineComment - https://phabricator.wikimedia.org/T222853 (10Daimona) a:03Daimona [17:23:17] RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 30.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [17:24:47] (03PS3) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [17:28:03] (03CR) 10jerkins-bot: [V: 04-1] Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) (owner: 10Daimona Eaytoy) [17:30:24] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [17:35:06] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [17:44:40] (03PS4) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [17:49:49] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10Patch-For-Review, 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10zeljkofilipin) [17:56:41] (03CR) 10jerkins-bot: [V: 04-1] Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) (owner: 10Daimona Eaytoy) [18:03:19] (03PS5) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [18:06:16] (03PS6) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [18:09:41] (03CR) 10jerkins-bot: [V: 04-1] Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) (owner: 10Daimona Eaytoy) [18:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [18:13:08] 10Continuous-Integration-Config, 10Code-Health-Metrics: Report results from SonarCloud to Gerrit - https://phabricator.wikimedia.org/T217008 (10thcipriani) >>! In T217008#5296569, @kostajh wrote: > @thcipriani wondering if the code health group could move ahead with this. We have the polling script approach wo... [18:14:28] Project beta-code-update-eqiad build #260013: 04FAILURE in 1 min 26 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/260013/ [18:24:29] Project beta-code-update-eqiad build #260014: 04STILL FAILING in 1 min 28 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/260014/ [18:34:28] Project beta-code-update-eqiad build #260015: 04STILL FAILING in 1 min 27 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/260015/ [18:40:59] hrm, looks like a conflict in ConfirmEdit ^ [18:41:35] looks like there's a var_dump in confirmedit/maintenance/GenerateFancyCaptchas.php [18:43:04] !log git -C /srv/mediawiki-staging/php-master/extensions/ConfirmEdit checkout maintenance/GenerateFancyCaptchas.php [18:43:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:43:57] (03PS7) 10Daimona Eaytoy: Allow consecutive single-line comments not to start with a single space [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/530910 (https://phabricator.wikimedia.org/T222853) [18:44:26] Yippee, build fixed! [18:44:27] Project beta-code-update-eqiad build #260016: 09FIXED in 1 min 25 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/260016/ [19:04:28] 10Phabricator: PhabricatorChunkedFileStorageEngine - https://phabricator.wikimedia.org/T230723 (10Reedy) What do you even mean? `PhabricatorChunkedFileStorageEngine` is a PHP class [20:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [20:33:40] 10Phabricator, 10I18n, 10User-Nikerabbit: Please add Brazilian Portuguese (pt-br) language to Phabricator - https://phabricator.wikimedia.org/T215697 (10mmodell) This is published but phabricator doesn't show the language because it currently has less than 500 strings translated. [20:51:30] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201908), 10Patch-For-Review, 10User-zeljkofilipin: Cypress testing framework evaluation - https://phabricator.wikimedia.org/T230729 (10Jpita) I went through this tutorial, it uses wiki as an example https://blog.logrocket.... [20:53:20] 10Phabricator, 10Release-Engineering-Team-TODO (201908), 10Wikimedia-Logstash, 10observability: Deploy phatality into kibana - https://phabricator.wikimedia.org/T230752 (10mmodell) [21:49:20] 10Phabricator, 10I18n, 10User-Nikerabbit: Please add Brazilian Portuguese (pt-br) language to Phabricator - https://phabricator.wikimedia.org/T215697 (10Aklapper) Wondering: Should this task be resolved? [22:11:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [22:22:34] 10Phabricator, 10Release-Engineering-Team-TODO (201908), 10Wikimedia-Logstash, 10observability: Deploy phatality into kibana - https://phabricator.wikimedia.org/T230752 (10mmodell)