[00:01:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:01:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:15:47] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<40.00%) [01:16:02] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:57:42] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) [02:08:20] (03CR) 10Krinkle: Phatality plugin for kibana (031 comment) [releng/phatality] - 10https://gerrit.wikimedia.org/r/531047 (https://phabricator.wikimedia.org/T230752) (owner: 1020after4) [02:11:05] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [03:07:40] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [03:28:42] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) [04:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [04:48:44] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) [06:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [06:55:47] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [10:11:04] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [12:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [13:30:35] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: (Service Check Timed Out) [13:35:35] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 37034 bytes in 8.209 second response time [13:38:43] RECOVERY - Free space - all mounts on deployment-deploy01 is OK: OK: All targets OK [14:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [16:11:03] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [18:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [18:27:42] (03CR) 10Legoktm: [C: 03+2] Allow @template annotations [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/534917 (https://phabricator.wikimedia.org/T232256) (owner: 10Daimona Eaytoy) [18:30:22] (03Merged) 10jenkins-bot: Allow @template annotations [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/534917 (https://phabricator.wikimedia.org/T232256) (owner: 10Daimona Eaytoy) [18:30:52] (03CR) 10jenkins-bot: Allow @template annotations [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/534917 (https://phabricator.wikimedia.org/T232256) (owner: 10Daimona Eaytoy) [18:39:11] (03PS3) 10TheDJ: Add new Discourse extension [integration/config] - 10https://gerrit.wikimedia.org/r/534906 (https://phabricator.wikimedia.org/T215053) (owner: 10Samwilson) [18:44:32] (03PS4) 10Umherirrender: Create new sniff for doc comments [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/528280 [18:45:37] (03PS5) 10Umherirrender: Create new sniff for doc comments [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/528280 [18:45:54] (03CR) 10Umherirrender: "Rebased" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/528280 (owner: 10Umherirrender) [19:50:17] (03CR) 10Hashar: [C: 03+2] Add new Discourse extension [integration/config] - 10https://gerrit.wikimedia.org/r/534906 (https://phabricator.wikimedia.org/T215053) (owner: 10Samwilson) [19:54:00] (03Merged) 10jenkins-bot: Add new Discourse extension [integration/config] - 10https://gerrit.wikimedia.org/r/534906 (https://phabricator.wikimedia.org/T215053) (owner: 10Samwilson) [19:54:53] (03CR) 10Hashar: [C: 03+2] "Deployed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/534906 (https://phabricator.wikimedia.org/T215053) (owner: 10Samwilson) [19:57:04] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909): integration-config-zuul-layout-validate-docker is too long - https://phabricator.wikimedia.org/T232287 (10hashar) [19:57:19] (03CR) 10Hashar: [C: 03+2] "integration-config-zuul-layout-validate-docker SUCCESS in 3m 41s" [integration/config] - 10https://gerrit.wikimedia.org/r/533916 (https://phabricator.wikimedia.org/T189560) (owner: 10Hashar) [20:00:01] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909): integration-config-zuul-layout-validate-docker is too long - https://phabricator.wikimedia.org/T232287 (10hashar) [20:11:05] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [22:11:01] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [22:26:47] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<20.00%)