[01:51:47] 10Continuous-Integration-Config, 10MinusX, 10Google-Code-in-2017, 10Patch-For-Review: Add MinusX to MediaWiki extensions and PHP library repos - https://phabricator.wikimedia.org/T175794#3842151 (10rafidaslam) [02:06:38] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<11.11%) [02:09:37] 10Deployments, 10Scap: Considering adding a --no-touch flag to scap that stops automatic touch of InitialiseSettings.php - https://phabricator.wikimedia.org/T149872#3842155 (10demon) 05Open>03declined D435 was reverted in D654. We will not be reintroducing such a flag. [02:15:20] 10Scap, 10Operations, 10Patch-For-Review: scap 3.7.4-2 is broken - https://phabricator.wikimedia.org/T183046#3842160 (10Dzahn) p:05Unbreak!>03High the whole cluster has been downgraded to 3.7.4-1 , checked with cumin the broken version has been removed from APT ``` [install1002:~] $ sudo -E reprepro... [04:50:09] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [10.0] [05:20:10] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [07:01:42] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:03:06] (03PS1) 10Legoktm: Ignore maintenance scripts in ClassMatchesFilenameSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398621 [08:03:17] (03PS2) 10Legoktm: Ignore maintenance scripts in ClassMatchesFilenameSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398621 (https://phabricator.wikimedia.org/T178725) [08:29:15] 10Release-Engineering-Team (Kanban), 10Scap, 10ORES, 10Scoring-platform-team: Source revision is in Phabricator, but can't be found by deployment tools - https://phabricator.wikimedia.org/T182865#3842232 (10mmodell) [08:46:02] 10Continuous-Integration-Config, 10MinusX, 10Google-Code-in-2017, 10Patch-For-Review: Add MinusX to MediaWiki extensions and PHP library repos - https://phabricator.wikimedia.org/T175794#3842240 (10rafidaslam) [08:52:23] (03PS1) 10Legoktm: Be aware of extension MediaWiki compatibility [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398625 (https://phabricator.wikimedia.org/T175465) [08:52:35] 10MediaWiki-Codesniffer, 10Patch-For-Review: Be aware of extension's minimum MediaWiki version supported - https://phabricator.wikimedia.org/T175465#3842245 (10Legoktm) a:03Legoktm [08:52:48] 10MediaWiki-Codesniffer, 10Patch-For-Review: ClassMatchesFilename.WrongCase should ignore maintenance scripts - https://phabricator.wikimedia.org/T178725#3842246 (10Legoktm) a:03Legoktm [09:01:42] 10MediaWiki-Codesniffer: Handle multiple array types from phpdoc in FunctionCommentSniff - https://phabricator.wikimedia.org/T178196#3684164 (10Legoktm) I think we should probably refactor the UnusedUse sniff code that parses `@param` tags into a utility class so we can reuse it in the FunctionComment sniff. [09:10:07] 10Continuous-Integration-Config, 10MinusX, 10Google-Code-in-2017, 10Patch-For-Review: Add MinusX to MediaWiki extensions and PHP library repos - https://phabricator.wikimedia.org/T175794#3842273 (10rafidaslam) [09:10:31] (03CR) 10MaxSem: "No tests?" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398621 (https://phabricator.wikimedia.org/T178725) (owner: 10Legoktm) [09:12:08] (03CR) 10Legoktm: "Because of the way this sniff works and our test suite is run, PHPCS thinks the filename is "STDIN" and this sniff skips that. I'm sure th" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398621 (https://phabricator.wikimedia.org/T178725) (owner: 10Legoktm) [11:01:53] (03CR) 10Umherirrender: [C: 031] Ignore maintenance scripts in ClassMatchesFilenameSniff (032 comments) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/398621 (https://phabricator.wikimedia.org/T178725) (owner: 10Legoktm) [12:25:26] 10Continuous-Integration-Config, 10MinusX, 10Google-Code-in-2017, 10MW-1.31-release-notes (WMF-deploy-2018-01-02 (1.31.0-wmf.15)), 10Patch-For-Review: Add MinusX to MediaWiki extensions and PHP library repos - https://phabricator.wikimedia.org/T175794#3842357 (10D3r1ck01) I'll like to co-mentor this with... [15:15:01] PROBLEM - Puppet errors on deployment-redis06 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:27:44] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:33:42] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:23:40] Project beta-code-update-eqiad build #185498: 04FAILURE in 39 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185498/ [17:25:34] 10Gerrit, 10Release-Engineering-Team: Unable to log in into gerrit: "Cannot assign user name "ashley" to account ; name already in use." - https://phabricator.wikimedia.org/T181945#3842472 (10demon) I'm sooooooo so so so so sorry for letting this slip. I've been hella busy and the task completely... [17:27:59] PROBLEM - Puppet errors on integration-slave-jessie-1003 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [17:33:40] Yippee, build fixed! [17:33:41] Project beta-code-update-eqiad build #185499: 09FIXED in 40 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/185499/ [18:02:59] RECOVERY - Puppet errors on integration-slave-jessie-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [18:39:23] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:44:51] 10Beta-Cluster-Infrastructure: various .beta.wmflabs.org domains use an invalid ssl certificate - https://phabricator.wikimedia.org/T182927#3842510 (10Krenair) >>! In T182927#3840415, @EddieGP wrote: >>>! In T182927#3839620, @Krenair wrote: >> m.$project? I'm not sure those are useful domains? > > Well they do... [18:52:26] 10Beta-Cluster-Infrastructure: various .beta.wmflabs.org domains use an invalid ssl certificate - https://phabricator.wikimedia.org/T182927#3842511 (10EddieGP) > Actually to mirror prod we'd need a wildcart certificate, which isn't available to us right now. That wasn't my point. "To mirror prod" was meant to sa... [18:54:15] 10Beta-Cluster-Infrastructure: various .beta.wmflabs.org domains use an invalid ssl certificate - https://phabricator.wikimedia.org/T182927#3842513 (10Krenair) Right now the plan is to leave everything as-is until it is possible to get a certificate that covers all subdomains. I don't want to play whack-a-mole w... [18:59:00] 10Beta-Cluster-Infrastructure: various .beta.wmflabs.org domains use an invalid ssl certificate - https://phabricator.wikimedia.org/T182927#3842514 (10Krenair) a:03Krenair [19:11:09] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [19:31:01] 10Beta-Cluster-Infrastructure: various .beta.wmflabs.org domains use an invalid ssl certificate - https://phabricator.wikimedia.org/T182927#3842532 (10EddieGP) >>! In T182927#3842513, @Krenair wrote: > Right now the plan is to leave everything as-is until it is possible to get a certificate that covers all subdo... [19:53:11] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 80.00% of data above the critical threshold [10.0] [20:34:29] PROBLEM - Puppet errors on deployment-eventlogging04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [22:20:15] PROBLEM - Puppet errors on deployment-sentry01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [22:35:44] PROBLEM - Puppet errors on deployment-etcd-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:30:59] PROBLEM - Puppet errors on deployment-kafka-jumbo-2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:36:10] PROBLEM - Puppet errors on deployment-netbox is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:37:24] PROBLEM - Puppet errors on deployment-kafka-jumbo-1 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0]