[00:02:19] 10Continuous-Integration-Infrastructure, 10Release Pipeline (Blubber): jenkins-slave@contint1001 not a member of docker group (CI tests for mathoid broken) - https://phabricator.wikimedia.org/T186790#3957079 (10thcipriani) >>! In T186790#3955843, @hashar wrote: > I fail to see how that ever worked. admin::grou... [00:31:16] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [00:39:16] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3957149 (10demon) [01:30:37] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<55.56%) [03:58:10] 10Gerrit, 10Patch-For-Review: Setup gitiles for use with gerrit as a repo viewer - https://phabricator.wikimedia.org/T184116#3957406 (10Dzahn) I agree we should _not_ use a second domain, yea. Also the short version with just "g" or something similar seems nice. What's up with https://gerrit.wikimedia.org/r... [04:33:58] 10Gerrit, 10Patch-For-Review: Setup gitiles for use with gerrit as a repo viewer - https://phabricator.wikimedia.org/T184116#3872823 (10Legoktm) Regarding cookies, the GerritAccount cookie has a path set of `/r`, so visiting `gerrit.wikimedia.org/g/` shows me as logged out. [04:57:41] 10Gerrit, 10Patch-For-Review: Setup gitiles for use with gerrit as a repo viewer - https://phabricator.wikimedia.org/T184116#3957422 (10jayvdb) [07:00:39] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:08:04] https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs-docker/buildTimeTrend [07:08:10] something is wrong with 1003 again [07:13:16] 10Continuous-Integration-Infrastructure (shipyard): mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856#3957565 (10Legoktm) [07:13:41] !log marked integration-slave-docker-1003 as offline - T186856 [07:13:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:13:47] T186856: mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856 [08:37:03] no_justification: when you have a chance could re update the gitiles plugin from [08:37:12] The 2.14 branch please? [08:37:33] It includes a fix for base urls + adds support for polygerrit if you have a base url [08:37:48] (My backport was merged) :) [08:37:49] (03PS1) 10Legoktm: Add coverage jobs for a few more extensions [integration/config] - 10https://gerrit.wikimedia.org/r/409244 [08:37:56] And should fix your issue too [08:39:24] (03PS1) 10Chad: Remove CleanChanges from wmf deploys [tools/release] - 10https://gerrit.wikimedia.org/r/409245 (https://phabricator.wikimedia.org/T186859) [08:39:26] (03CR) 10Chad: [C: 032] Remove CleanChanges from wmf deploys [tools/release] - 10https://gerrit.wikimedia.org/r/409245 (https://phabricator.wikimedia.org/T186859) (owner: 10Chad) [08:40:17] (03Merged) 10jenkins-bot: Remove CleanChanges from wmf deploys [tools/release] - 10https://gerrit.wikimedia.org/r/409245 (https://phabricator.wikimedia.org/T186859) (owner: 10Chad) [08:50:10] Project mwext-phpunit-coverage-publish build #728: 04FAILURE in 26 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/728/ [08:51:00] Yippee, build fixed! [08:51:01] Project mwext-phpunit-coverage-publish build #729: 09FIXED in 50 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/729/ [08:51:12] (03CR) 10Hashar: [C: 032] Add coverage jobs for a few more extensions [integration/config] - 10https://gerrit.wikimedia.org/r/409244 (owner: 10Legoktm) [08:52:18] hashar: oh, thanks :D [08:52:33] legoktm: I applaud all the coverage stuff you are adding :] [08:52:35] (03Merged) 10jenkins-bot: Add coverage jobs for a few more extensions [integration/config] - 10https://gerrit.wikimedia.org/r/409244 (owner: 10Legoktm) [08:52:47] legoktm: I like sleep / spare time to help on that front though :( [08:53:29] !log Reloading zuul for I73a3142271d76f9640005752b1a336123b921b9f [08:53:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:53:45] heh [08:54:35] (03CR) 10Hashar: [C: 032] docker: Move config.yaml to expected docker-pkg name [integration/config] - 10https://gerrit.wikimedia.org/r/408866 (owner: 10Legoktm) [08:54:59] legoktm: and thanks for the docker-pkg config file rename :] [08:55:09] yw :) [08:55:43] (03Merged) 10jenkins-bot: docker: Move config.yaml to expected docker-pkg name [integration/config] - 10https://gerrit.wikimedia.org/r/408866 (owner: 10Legoktm) [09:22:27] (03PS1) 10Legoktm: Add even more MediaWiki extension coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409297 [09:22:48] (03CR) 10Legoktm: [C: 032] Add even more MediaWiki extension coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409297 (owner: 10Legoktm) [09:23:57] (03Merged) 10jenkins-bot: Add even more MediaWiki extension coverage jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409297 (owner: 10Legoktm) [09:24:39] (03PS1) 10Hashar: Migrate mw-tools-scap-tox-doc-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409298 [09:25:38] !log deploying https://gerrit.wikimedia.org/r/409297 [09:25:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:26:45] (03CR) 10Hashar: [C: 032] "Dirty but works!" [integration/config] - 10https://gerrit.wikimedia.org/r/409298 (owner: 10Hashar) [09:27:13] legoktm: I will have to move that mwext-phpunit-coverage-publish to a docker container :] [09:27:19] it is consuming a bit of nodepool images [09:27:35] is it using php7? [09:28:03] (03Merged) 10jenkins-bot: Migrate mw-tools-scap-tox-doc-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409298 (owner: 10Hashar) [09:30:52] hashar: it doesn't run on nodepool, it's on a permanent slave [09:31:16] the only reason I didn't set it up on docker is because the rest of the MW phpunit scripts/env haven't been ported for docker yet [09:31:32] it's php7/sqlite [09:36:05] legoktm: ah great; thank you :] [09:36:55] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Watching / External), 10Performance-Team, 10Availability (Multiple-active-datacenters): Performance Q2 2017/18 goal: Install and use mcrouter in deployment-prep - https://phabricator.wikimedia.org/T151466#2818204 (10fgiunchedi) I didn't check the c... [09:37:48] hashar: also, I filed T186856 earlier - one of the docker slaves is running much slower than the others again [09:37:48] T186856: mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856 [09:44:54] EEK [09:45:48] legoktm: most probably the instance is on an overloaded server. I looked at that back in october https://phabricator.wikimedia.org/T179378 [09:45:56] turns out that is the same slave [09:46:10] >.< [09:46:50] 10Continuous-Integration-Infrastructure (shipyard): mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856#3957811 (10hashar) It must be runniing on an overloaded labvirt machine. That is T17... [09:47:04] 10Continuous-Integration-Infrastructure (shipyard): mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856#3957817 (10hashar) [09:48:00] !log Deleting integration-slave-docker-1003 and recreating it. It is running on an overloaded labvirt machine. | T179378 T186856 [09:48:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:48:09] T179378: some labvirt servers are at full CPU capacity - https://phabricator.wikimedia.org/T179378 [09:48:09] T186856: mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856 [09:49:29] legoktm: and https://phabricator.wikimedia.org/T182070 gives you an idea of the labvirt being overloaded overtime :( [09:54:50] PROBLEM - Host integration-slave-docker-1003 is DOWN: CRITICAL - Host Unreachable (10.68.23.158) [10:00:24] :) [10:00:24] :(* [10:00:27] hashar: thanks for looking into it [10:00:30] * legoktm -> zzzz [10:01:13] legoktm: good night! [10:16:21] !log Launching new integration-slave-docker-1003 T186856 [10:16:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:16:28] T186856: mediawiki-core-phpcs-docker jobs running on integration-slave-docker-1003 are running significantly slower than other slaves (redux) - https://phabricator.wikimedia.org/T186856 [10:25:43] RECOVERY - Host integration-slave-docker-1003 is UP: PING OK - Packet loss = 0%, RTA = 0.80 ms [10:42:05] 10Release-Engineering-Team (Watching / External), 10Epic, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10User-notice: Deploy refactored comment storage - https://phabricator.wikimedia.org/T166733#3957902 (10jcrespo) Small question regarding old comment format, while BOTH, is the old format truncated i... [10:45:06] 10Beta-Cluster-Infrastructure, 10AbuseFilter, 10User-MarcoAurelio: Purge old IP data from AbuseFilter on the Beta Cluster - https://phabricator.wikimedia.org/T186870#3957905 (10MarcoAurelio) p:05Triage>03Normal [10:45:18] 10Gerrit, 10Wikibase-DataModel, 10Wikidata, 10Wikidata-Sprint-2016-03-01, and 2 others: [Task] Move WikibaseDataModel to Gerrit - https://phabricator.wikimedia.org/T126597#3957919 (10Addshore) Thoughts @WMDE-leszek ? / Team #wikidata ? [12:05:45] (03PS1) 10Hashar: dockerfile: container to run JSDuck [integration/config] - 10https://gerrit.wikimedia.org/r/409320 [12:30:53] PROBLEM - Puppet errors on deployment-ores01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [12:53:40] 10Gerrit, 10Phabricator, 10Operations, 10Traffic, 10periodic-update: Phabricator and Gerrit: Improve the way that maintenance downtime is communicated to users. - https://phabricator.wikimedia.org/T180655#3958098 (10demon) https://gerrit.googlesource.com/plugins/motd/+/master could be useful on gerrit's... [12:56:52] 10Gerrit, 10Wikidata: [Task] trigger packagist.org from repo on gerrit.w.o - https://phabricator.wikimedia.org/T87768#998277 (10demon) Um, I may have found a solution: https://gerrit.googlesource.com/plugins/webhooks/+/master/ [13:20:54] (03PS1) 10Hashar: Remove jsduck job [integration/config] - 10https://gerrit.wikimedia.org/r/409324 [13:30:54] (03CR) 10Hashar: [C: 032] Remove jsduck job [integration/config] - 10https://gerrit.wikimedia.org/r/409324 (owner: 10Hashar) [13:32:09] (03Merged) 10jenkins-bot: Remove jsduck job [integration/config] - 10https://gerrit.wikimedia.org/r/409324 (owner: 10Hashar) [13:34:56] (03PS2) 10Hashar: dockerfile: container to run JSDuck [integration/config] - 10https://gerrit.wikimedia.org/r/409320 [13:37:34] (03CR) 10Hashar: [C: 032] dockerfile: container to run JSDuck [integration/config] - 10https://gerrit.wikimedia.org/r/409320 (owner: 10Hashar) [13:42:21] (03Merged) 10jenkins-bot: dockerfile: container to run JSDuck [integration/config] - 10https://gerrit.wikimedia.org/r/409320 (owner: 10Hashar) [13:43:12] !log Deploying docker-registry.wikimedia.org/releng/jsduck:0.1.0 | https://gerrit.wikimedia.org/r/#/c/409320/ [13:43:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:45:40] Project mwext-phpunit-coverage-publish build #737: 04FAILURE in 23 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/737/ [13:52:37] (03PS1) 10Hashar: jsduck docker job have to clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/409328 [13:54:03] (03CR) 10Hashar: [C: 032] jsduck docker job have to clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/409328 (owner: 10Hashar) [13:55:37] (03Merged) 10jenkins-bot: jsduck docker job have to clone the repo [integration/config] - 10https://gerrit.wikimedia.org/r/409328 (owner: 10Hashar) [13:57:48] (03PS1) 10Hashar: Migrate mediawiki/core jsduck job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409330 [13:59:58] (03PS1) 10Hashar: Migrate mwext-VisualEditor-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409331 [14:03:27] (03PS2) 10Hashar: Migrate mwext-VisualEditor-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409331 [14:03:29] (03PS1) 10Hashar: Migrate oojs-core-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409333 [14:03:39] 10Phabricator, 10Developer-Relations: Find best way to allow CoC committee to temporarily disable Phab user accounts - https://phabricator.wikimedia.org/T183116#3958313 (10Aklapper) For the records, Phab admins are listed in https://phabricator.wikimedia.org/people/query/DktdoFyuGYMN/#R Options mentioned so fa... [14:05:18] (03PS1) 10Hashar: Migrate unicodejs-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409334 [14:06:33] (03CR) 10Hashar: [C: 032] Migrate mediawiki/core jsduck job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409330 (owner: 10Hashar) [14:06:50] (03CR) 10Hashar: [C: 032] Migrate mwext-VisualEditor-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409331 (owner: 10Hashar) [14:07:12] (03CR) 10Hashar: [C: 032] Migrate oojs-core-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409333 (owner: 10Hashar) [14:07:45] (03CR) 10Hashar: [C: 032] Migrate unicodejs-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409334 (owner: 10Hashar) [14:07:53] (03Merged) 10jenkins-bot: Migrate mediawiki/core jsduck job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409330 (owner: 10Hashar) [14:08:13] (03Merged) 10jenkins-bot: Migrate mwext-VisualEditor-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409331 (owner: 10Hashar) [14:08:34] (03Merged) 10jenkins-bot: Migrate oojs-core-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409333 (owner: 10Hashar) [14:10:57] (03CR) 10Hashar: [C: 032] Migrate unicodejs-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409334 (owner: 10Hashar) [14:12:22] (03Merged) 10jenkins-bot: Migrate unicodejs-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409334 (owner: 10Hashar) [14:14:11] 10Phabricator, 10Developer-Relations: Find best way to allow CoC committee to temporarily disable Phab user accounts - https://phabricator.wikimedia.org/T183116#3958331 (10demon) >>! In T183116#3958313, @Aklapper wrote: > For the records, Phab admins are listed in https://phabricator.wikimedia.org/people/query... [14:16:50] 10Phabricator, 10Developer-Relations: Find best way to allow CoC committee to temporarily disable Phab user accounts - https://phabricator.wikimedia.org/T183116#3958341 (10demon) (Sidebar: I'd like to see T100400 fixed as well: right now we have to disable users because of a bug sometimes--it'd make it impossi... [14:17:18] (03PS1) 10Hashar: Remove {name}-jsduck-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/409335 [14:20:14] (03PS1) 10Hashar: Experimental mwgate-jsduck job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409336 [14:20:20] (03CR) 10Hashar: [C: 032] Remove {name}-jsduck-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/409335 (owner: 10Hashar) [14:20:23] (03CR) 10Hashar: [C: 032] Experimental mwgate-jsduck job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409336 (owner: 10Hashar) [14:21:51] (03Merged) 10jenkins-bot: Remove {name}-jsduck-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/409335 (owner: 10Hashar) [14:22:23] (03CR) 10jerkins-bot: [V: 04-1] Experimental mwgate-jsduck job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409336 (owner: 10Hashar) [14:23:16] (03CR) 10Hashar: [C: 032] "I have forgot to deploy mwgate-jsduck-docker" [integration/config] - 10https://gerrit.wikimedia.org/r/409336 (owner: 10Hashar) [14:24:47] (03Merged) 10jenkins-bot: Experimental mwgate-jsduck job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409336 (owner: 10Hashar) [14:27:15] 10MediaWiki-Codesniffer: Sniff to $wmf* -> $wmg* - https://phabricator.wikimedia.org/T186642#3950368 (10Umherirrender) There is a sniff to search for globals not starting with $wg. Your example match that. The sniff is MediaWiki.NamingConventions.ValidGlobalName I would say that is enough. Changes must match t... [14:40:30] Yippee, build fixed! [14:40:30] Project mwext-phpunit-coverage-publish build #738: 09FIXED in 29 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/738/ [14:53:15] PROBLEM - Host deployment-puppetdb01 is DOWN: CRITICAL - Host Unreachable (10.68.23.76) [14:55:51] 10Gerrit, 10Wikibase-DataModel, 10Wikidata, 10Wikidata-Sprint-2016-03-01, and 2 others: [Task] Move WikibaseDataModel to Gerrit - https://phabricator.wikimedia.org/T126597#3958437 (10thiemowmde) Moving this component alone is not really worth much. We are barely touching it any more. It's just some git rep... [15:02:03] (03PS1) 10Hashar: Migrate mwgate-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409348 [15:06:35] (03PS1) 10Hashar: Clean out jsduck configurations [integration/config] - 10https://gerrit.wikimedia.org/r/409349 [15:07:44] (03CR) 10Hashar: [C: 032] Migrate mwgate-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409348 (owner: 10Hashar) [15:08:53] (03CR) 10Hashar: [C: 032] Clean out jsduck configurations [integration/config] - 10https://gerrit.wikimedia.org/r/409349 (owner: 10Hashar) [15:10:46] (03Merged) 10jenkins-bot: Migrate mwgate-jsduck to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409348 (owner: 10Hashar) [15:13:47] (03Merged) 10jenkins-bot: Clean out jsduck configurations [integration/config] - 10https://gerrit.wikimedia.org/r/409349 (owner: 10Hashar) [15:24:40] (03PS1) 10Hashar: Migrate mwext-jsduck-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409351 [15:27:53] (03CR) 10Hashar: [C: 032] "Tested on mediawiki/extensions/TemplateData https://gerrit.wikimedia.org/r/#/c/409272/" [integration/config] - 10https://gerrit.wikimedia.org/r/409351 (owner: 10Hashar) [15:29:10] (03Merged) 10jenkins-bot: Migrate mwext-jsduck-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/409351 (owner: 10Hashar) [15:38:03] 10Release-Engineering-Team (Watching / External), 10Epic, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10User-notice: Deploy refactored comment storage - https://phabricator.wikimedia.org/T166733#3958623 (10Anomie) It's truncated in advance. In case you're curious, the specific place that happens is [... [16:24:16] PROBLEM - Puppet errors on deployment-eventlog02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:24:34] PROBLEM - Puppet errors on deployment-mediawiki04 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:26:35] (03CR) 10Chad: [C: 032] make-release: Simplify excludes, just drop all .dotfiles [tools/release] - 10https://gerrit.wikimedia.org/r/405213 (owner: 10Chad) [16:26:56] (03Abandoned) 10Chad: WIP: Create docker image for linting refs/meta/config [integration/config] - 10https://gerrit.wikimedia.org/r/404753 (owner: 10Chad) [16:27:12] (03Merged) 10jenkins-bot: make-release: Simplify excludes, just drop all .dotfiles [tools/release] - 10https://gerrit.wikimedia.org/r/405213 (owner: 10Chad) [16:28:09] 10Release-Engineering-Team (Watching / External), 10Epic, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10User-notice: Deploy refactored comment storage - https://phabricator.wikimedia.org/T166733#3958823 (10jcrespo) Thanks. [16:30:26] (03CR) 10Chad: [V: 032 C: 032] Modify access rules [All-Projects] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/408430 (owner: 10Paladox) [16:30:46] PROBLEM - Puppet errors on deployment-cache-upload04 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [16:30:46] Thanks. [16:33:17] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:53:09] Project mwext-phpunit-coverage-publish build #744: 04FAILURE in 32 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/744/ [16:56:22] Yippee, build fixed! [16:56:23] Project mwext-phpunit-coverage-publish build #745: 09FIXED in 35 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/745/ [16:59:16] RECOVERY - Puppet errors on deployment-eventlog02 is OK: OK: Less than 1.00% above the threshold [0.0] [17:04:33] RECOVERY - Puppet errors on deployment-mediawiki04 is OK: OK: Less than 1.00% above the threshold [0.0] [17:10:42] RECOVERY - Puppet errors on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [0.0] [17:18:44] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [17:43:43] RECOVERY - Puppet errors on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [17:49:44] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [18:11:28] 10Continuous-Integration-Infrastructure, 10VisualEditor, 10Wikidata, 10Jenkins: ResourcesTest::testFileExistence fails for ULS files in VisualEditor - https://phabricator.wikimedia.org/T186903#3959008 (10Umherirrender) [18:14:47] RECOVERY - Puppet errors on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [18:27:42] RECOVERY - Free space - all mounts on integration-slave-jessie-1004 is OK: OK: All targets OK [18:28:46] PROBLEM - Free space - all mounts on deployment-mediawiki04 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki04.diskspace.root.byte_percentfree (<11.11%) [18:34:50] 10Continuous-Integration-Infrastructure, 10VisualEditor, 10Wikidata, 10Jenkins, 10Patch-For-Review: ResourcesTest::testFileExistence fails for ULS files in VisualEditor - https://phabricator.wikimedia.org/T186903#3959054 (10Jdforrester-WMF) p:05Triage>03Unbreak! a:03Jdforrester-WMF My fault, sorry.... [18:38:48] RECOVERY - Free space - all mounts on deployment-mediawiki04 is OK: OK: All targets OK [18:41:43] 10Diffusion: Inconsistent repo naming - https://phabricator.wikimedia.org/T186910#3959091 (10kaldari) [18:42:02] 10Gerrit, 10Upstream: Polygerrit should fold not current line comments by default - https://phabricator.wikimedia.org/T186408#3959101 (10Paladox) @Dvorapa from upstream "Can you give us a reproducible example where there are comments we should consider automatically collapsing?" [18:45:14] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959123 (10demon) [18:54:29] PROBLEM - Puppet errors on deployment-secureredirexperiment is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:55:20] Hey :) Re: train this week - it looks like everything is now stable on 1.31.0-wmf.20 ? I'm just asking for Tech/News purposes, and whether I can remove (or should keep) the line about train-rollback. Ty! [19:01:28] 10Continuous-Integration-Infrastructure, 10VisualEditor, 10Wikidata, 10Jenkins, and 2 others: ResourcesTest::testFileExistence fails for ULS files in VisualEditor - https://phabricator.wikimedia.org/T186903#3959141 (10Jdforrester-WMF) 05Open>03Resolved [19:05:55] greg-g, question 2 lines above ^. :) [19:12:08] or no_justification (TLDR: Is the train all good this week? Or should I mention the rollback in Tech/News?) [19:12:44] It should be ok. [19:12:47] I hope [19:12:49] I'd like to get lunch [19:12:57] And stop staring at logstash [19:13:09] ty :) goodluckhavelunch. [19:16:28] no_justification ok reviewers works https://gerrit.git.wmflabs.org/r/#/x/reviewers/p/All-Projects [19:18:35] once we have the plugin live, we can start migrating users to the plugin and lock editing of that wiki page. [19:20:52] * paladox wants to move reviewers plugin ui to polygerrit [19:20:58] shoulden't be hard i doin't think [19:23:38] 10Gerrit, 10Upstream: Polygerrit should fold not current line comments by default - https://phabricator.wikimedia.org/T186408#3959175 (10Dvorapa) I tested polygerrit on this one: https://gerrit.wikimedia.org/r/c/372064/, but it has got no so many line comments as some older ones I can not find currently better... [19:24:40] 10Gerrit, 10Upstream: Polygerrit should fold not current line comments by default - https://phabricator.wikimedia.org/T186408#3959179 (10Paladox) Thanks, will forward your comment upstream. [19:32:07] 10Gerrit, 10Upstream: Polygerrit should fold not current line comments by default - https://phabricator.wikimedia.org/T186408#3959193 (10Dvorapa) Another issue I can see (should go to separate task) is that in old ui the patchset numbers were lined up one under another, in polygerrit I have to search for the p... [19:51:25] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959224 (10demon) [19:51:48] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3868515 (10demon) [19:54:34] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959240 (10demon) [19:56:23] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3868515 (10demon) [19:57:46] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959272 (10demon) [20:14:37] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959354 (10Huji) [20:29:48] PROBLEM - Free space - all mounts on deployment-mediawiki04 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki04.diskspace.root.byte_percentfree (<11.11%) [20:39:50] RECOVERY - Free space - all mounts on deployment-mediawiki04 is OK: OK: All targets OK [21:03:14] (03PS1) 10Legoktm: Add sniff to find tests without @covers tags [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/409444 (https://phabricator.wikimedia.org/T179094) [21:06:17] (03CR) 10Legoktm: [C: 032] Avoid assignment in return statements [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/408610 (https://phabricator.wikimedia.org/T170332) (owner: 10Umherirrender) [21:07:09] (03Merged) 10jenkins-bot: Avoid assignment in return statements [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/408610 (https://phabricator.wikimedia.org/T170332) (owner: 10Umherirrender) [21:08:22] PROBLEM - Puppet errors on deployment-mx02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:08:31] (03CR) 10jenkins-bot: Avoid assignment in return statements [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/408610 (https://phabricator.wikimedia.org/T170332) (owner: 10Umherirrender) [21:14:41] 10Beta-Cluster-Infrastructure, 10AbuseFilter, 10User-MarcoAurelio: Purge old IP data from AbuseFilter on the Beta Cluster - https://phabricator.wikimedia.org/T186870#3959556 (10MarcoAurelio) I'll test on which is closed. [21:15:45] 10Beta-Cluster-Infrastructure, 10AbuseFilter, 10User-MarcoAurelio: Purge old IP data from AbuseFilter on the Beta Cluster - https://phabricator.wikimedia.org/T186870#3959560 (10MarcoAurelio) [21:16:11] no_justification: hi - I'd like to T186870 on beta eswiki; is that okay? [21:16:11] T186870: Purge old IP data from AbuseFilter on the Beta Cluster - https://phabricator.wikimedia.org/T186870 [21:16:55] !log deleted test-csteipp-sensiolabs-securityadvisorieschecker from jenkins, obsolete [21:17:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:20:50] Hauskatze: I don't have the mental energy to even click the link [21:21:02] :) [21:21:07] don't worry, can wait [21:21:11] brb [21:42:10] Anyone want to help me understand what scap is doing? I think I must be missing an important step, or misunderstanding how this is supposed to work... [21:43:00] andrewbogott maybe scap deploy-log would help you. [21:49:48] (03PS2) 10Legoktm: Begin migrating php-compile-* jobs to docker [integration/config] - 10https://gerrit.wikimedia.org/r/408760 [21:50:43] (03CR) 10Legoktm: [C: 032] Begin migrating php-compile-* jobs to docker [integration/config] - 10https://gerrit.wikimedia.org/r/408760 (owner: 10Legoktm) [21:50:46] no_justification: I couldn't run it either: T186928 [21:50:46] T186928: Rename Abuse Filter to AbuseFilter in extension.json - https://phabricator.wikimedia.org/T186928 [21:52:06] (03Merged) 10jenkins-bot: Begin migrating php-compile-* jobs to docker [integration/config] - 10https://gerrit.wikimedia.org/r/408760 (owner: 10Legoktm) [21:53:38] !log deploying https://gerrit.wikimedia.org/r/408760 [21:53:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:59:28] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959679 (10Jdforrester-WMF) [22:38:10] (03PS1) 10Legoktm: Add php71-compile and php72-compile docker images [integration/config] - 10https://gerrit.wikimedia.org/r/409458 [22:38:43] (03CR) 10Legoktm: [C: 032] Add php71-compile and php72-compile docker images [integration/config] - 10https://gerrit.wikimedia.org/r/409458 (owner: 10Legoktm) [22:39:53] (03Merged) 10jenkins-bot: Add php71-compile and php72-compile docker images [integration/config] - 10https://gerrit.wikimedia.org/r/409458 (owner: 10Legoktm) [22:40:56] !log deploying https://gerrit.wikimedia.org/r/409458 [22:41:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:48:53] So, I just did a scap deploy that failed with a stream of "Timeout, server ores100*.eqiad.wmnet not responding." [22:49:09] I checked and the ores hosts are online. So I just re-tried. [22:49:25] Now the scap just rockets through the deploy claiming to finish very quickly. [22:49:28] But nothing is happening. [22:49:35] Some cached thing I can blow out? [22:49:48] (03PS1) 10Legoktm: Add php7{1,2}-compile jjb jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409459 [22:50:51] I see that there are some tags (???) created at the time of deployment. [22:50:57] Can I just wipe those out? [22:52:28] PROBLEM - Puppet errors on deployment-ms-be03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [22:53:07] (03PS7) 10Zoranzoki21: Add few extensions to zuul/layout.yaml in so Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [22:53:43] PROBLEM - Puppet errors on saucelabs-02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [22:53:59] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [22:55:39] (03PS2) 10Legoktm: Add php7{1,2}-compile jjb jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409459 [22:55:58] (03CR) 10Legoktm: [C: 032] Add php7{1,2}-compile jjb jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409459 (owner: 10Legoktm) [22:58:25] (03PS8) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [22:58:38] (03Merged) 10jenkins-bot: Add php7{1,2}-compile jjb jobs [integration/config] - 10https://gerrit.wikimedia.org/r/409459 (owner: 10Legoktm) [22:59:27] !log deployed https://gerrit.wikimedia.org/r/409459 [22:59:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:04:15] (03PS1) 10Legoktm: Move php-compile-docker jobs out of experimental [integration/config] - 10https://gerrit.wikimedia.org/r/409461 [23:04:37] (03CR) 10Legoktm: [C: 032] Move php-compile-docker jobs out of experimental [integration/config] - 10https://gerrit.wikimedia.org/r/409461 (owner: 10Legoktm) [23:05:48] (03Merged) 10jenkins-bot: Move php-compile-docker jobs out of experimental [integration/config] - 10https://gerrit.wikimedia.org/r/409461 (owner: 10Legoktm) [23:05:54] !log deploying https://gerrit.wikimedia.org/r/409461 [23:06:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:09:52] halfak: much of the scap team is already out for the weekend, please phile a task if something is blocking you, but hopefully you're almost gone for the weekend, too.l [23:09:55] -l [23:14:38] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T183959#3959855 (10demon) [23:27:27] RECOVERY - Puppet errors on deployment-ms-be03 is OK: OK: Less than 1.00% above the threshold [0.0] [23:28:42] RECOVERY - Puppet errors on saucelabs-02 is OK: OK: Less than 1.00% above the threshold [0.0] [23:29:00] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [23:51:17] no_justification Chad (snark level: 10) lol