[05:38:32] PROBLEM - Puppet errors on deployment-webperf11 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:18:29] RECOVERY - Puppet errors on deployment-webperf11 is OK: OK: Less than 1.00% above the threshold [0.0] [07:22:58] (03PS1) 10Legoktm: Add update script [integration/npm] - 10https://gerrit.wikimedia.org/r/460732 [07:23:00] (03PS1) 10Legoktm: Merge v6.4.1 [integration/npm] - 10https://gerrit.wikimedia.org/r/460733 [07:23:02] (03PS1) 10Legoktm: Build v6.4.1 [integration/npm] - 10https://gerrit.wikimedia.org/r/460734 [07:26:07] 10Phabricator, 10Operations: Phabricator is slow - https://phabricator.wikimedia.org/T204421 (10Legoktm) I'm seeing slowdowns in autocomplete menus and previews, instead of being instantaneous like normal, it's taking 3-5 seconds to get a response. [07:28:33] (03CR) 10Legoktm: [V: 032 C: 032] Add update script [integration/npm] - 10https://gerrit.wikimedia.org/r/460732 (owner: 10Legoktm) [07:29:17] (03CR) 10Legoktm: [V: 032 C: 032] Merge v6.4.1 [integration/npm] - 10https://gerrit.wikimedia.org/r/460733 (owner: 10Legoktm) [07:29:29] (03CR) 10Legoktm: [V: 032 C: 032] Build v6.4.1 [integration/npm] - 10https://gerrit.wikimedia.org/r/460734 (owner: 10Legoktm) [07:43:26] PROBLEM - Free space - all mounts on deployment-deploy02 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy02.diskspace._srv.byte_percentfree (<20.00%) [07:50:37] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace._srv.byte_percentfree (<10.00%) [08:26:15] RECOVERY - Puppet errors on deployment-deploy01 is OK: OK: Less than 1.00% above the threshold [0.0] [08:36:25] 10Project-Admins: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Zoranzoki21) [08:37:36] 10Project-Admins, 10LastModified: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Zoranzoki21) [08:42:27] 10Project-Admins, 10LastModified: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Peachey88) We use both standards for project names, Would suggest closing this as invalid. [09:05:44] 10Project-Admins, 10LastModified: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Zoranzoki21) I know, but this extension is not like #VisualEditor or #checkuser so I think to is ok to this project be renamed. [09:11:37] RECOVERY - Puppet errors on deployment-deploy02 is OK: OK: Less than 1.00% above the threshold [0.0] [09:50:48] 10Phabricator, 10Security-Team, 10Patch-For-Review: Add 'risk' field to tasks created via advanced template - https://phabricator.wikimedia.org/T204138 (10Framawiki) Just note that now it looks like all tasks have a Details box with `Risk Rating: N/A` , like in T204430. [10:18:56] 10Project-Admins, 10LastModified: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Zoranzoki21) 05Open>03Invalid I closing that as invalid. [10:21:04] 10Continuous-Integration-Config, 10MediaWiki-extensions-VEForAll: New extension, VEForAll, needs Jenkins monitoring - https://phabricator.wikimedia.org/T193774 (10Peachey88) [10:22:57] 10Continuous-Integration-Config, 10MediaWiki-extensions-MintyDocs: New extension, MintyDocs, needs to be added to zuul/layout.yaml - https://phabricator.wikimedia.org/T198488 (10Peachey88) [11:23:11] 10Continuous-Integration-Infrastructure, 10Goal, 10Technical-Debt: All repositories should pass jshint test - https://phabricator.wikimedia.org/T62619 (10Liuxinyu970226) [12:23:38] 10Phabricator, 10Operations: Phabricator is slow - https://phabricator.wikimedia.org/T204421 (10MarcoAurelio) For me it is working fine, but if it is not for others indeed this is worth investigating. [12:30:48] 10Phabricator, 10Operations: Phabricator is slow - https://phabricator.wikimedia.org/T204421 (10Zoranzoki21) >>! In T204421#4586666, @MarcoAurelio wrote: > For me it is working fine, but if it is not for others indeed this is worth investigating. [12:35:18] 10Phabricator, 10Security-Team, 10Patch-For-Review: Add 'risk' field to tasks created via advanced template - https://phabricator.wikimedia.org/T204138 (10chasemp) @20after4 have any idea why it's still showing up? Oddly I do not see it [12:38:36] https://gerrit.wikimedia.org/r/#/c/integration/config/+/460578/ was merged but not deployed to zuul, given that a patch is being tested for that repo and should've failed straight away. [12:41:13] 10Continuous-Integration-Config, 10MediaWiki-extensions-CommentStreams, 10Mobile, 10Patch-For-Review: CommentStreams: The module 'ext.CommentStreams' must not have target 'mobile' because its dependency 'jquery.ui.dialog' does not have it - https://phabricator.wikimedia.org/T151863 (10Liuxinyu970226) [12:47:34] !log github: deleting | T204372 [12:47:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:47:42] T204372: Archive the Community Twitter extension - https://phabricator.wikimedia.org/T204372 [12:50:40] 10Gerrit, 10MediaWiki-extensions-SecureAuth, 10translatewiki.net, 10User-MarcoAurelio: Authorize L10n-bot to give +2 for SecureAuth extension - https://phabricator.wikimedia.org/T193227 (10Liuxinyu970226) [13:22:37] 10Project-Admins, 10LastModified: Rename #LastModified to #MediaWiki-extensions-LastModified - https://phabricator.wikimedia.org/T204429 (10Aklapper) > because this is the standard designation for projects for extensions. Please provide references for such statements. In general. [15:31:04] 10Phabricator, 10Security-Team, 10Patch-For-Review: Add 'Risk Rating' field to tasks created via advanced template - https://phabricator.wikimedia.org/T204138 (10Krinkle) [15:31:45] 10Phabricator, 10Security-Team, 10Patch-For-Review: Add 'Risk Rating' field to tasks created via advanced template - https://phabricator.wikimedia.org/T204138 (10Krinkle) Case in point :) [16:26:23] !log phabricator: Account https://phabricator.wikimedia.org/p/Ponani/ disabled by me due to vandalism | T200856 [16:26:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:26:30] T200856: Publicly log account bans made using the phab-ban tool - https://phabricator.wikimedia.org/T200856 [16:31:59] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-09 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:36:47] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-09 is OK: HTTP OK: HTTP/1.1 200 OK - 46975 bytes in 1.072 second response time [16:57:07] (03CR) 10MarcoAurelio: "I noted today that zuul ran checks for this repo despite this change, so it might be that this change was not deployed to zuul and reloade" [integration/config] - 10https://gerrit.wikimedia.org/r/460578 (https://phabricator.wikimedia.org/T204372) (owner: 10MarcoAurelio) [17:41:20] Project beta-scap-eqiad build #222150: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/222150/ [17:43:11] woah wtf, -deploy01 just briefly stopped letting me sudo [17:43:16] it's okay now but... what. [17:48:42] meanwhile [17:48:43] root@deployment-deploy02:~# nutcracker --verbose=11 --mbuf-size=65536 [17:48:43] root@deployment-deploy02:~# echo $? [17:48:43] 1 [17:50:03] okay so apparently that's just what it does when /var/run/nutcracker/ is missing [17:51:24] or that dir has the wrong permissions? [17:53:03] I probably shouldn't have to go through strace to find bind(11, {sa_family=AF_UNIX, sun_path="/var/run/nutcracker/nutcracker.sock"}, 110) = -1 ENOENT (No such file or directory) to know this. [17:55:32] Yippee, build fixed! [17:55:32] Project beta-scap-eqiad build #222151: 09FIXED in 13 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/222151/ [18:02:47] opened https://phabricator.wikimedia.org/T204450 [19:05:34] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) WARN: deployment-prep.deployment-deploy01.diskspace._srv.byte_percentfree (<100.00%) [19:50:35] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) WARN: deployment-prep.deployment-deploy01.diskspace._srv.byte_percentfree (<100.00%) [20:05:37] PROBLEM - Free space - all mounts on deployment-deploy01 is CRITICAL: CRITICAL: deployment-prep.deployment-deploy01.diskspace.root.byte_percentfree (<10.00%) WARN: deployment-prep.deployment-deploy01.diskspace._srv.byte_percentfree (<100.00%) [22:09:52] Project beta-scap-eqiad build #222168: 04FAILURE in 15 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/222168/ [22:26:51] Yippee, build fixed! [22:26:51] Project beta-scap-eqiad build #222169: 09FIXED in 16 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/222169/ [22:51:20] (03PS1) 10Legoktm: Run `npm test` for quarry [integration/config] - 10https://gerrit.wikimedia.org/r/460774 [22:58:47] (03CR) 10Legoktm: [C: 032] Run `npm test` for quarry [integration/config] - 10https://gerrit.wikimedia.org/r/460774 (owner: 10Legoktm) [23:00:24] (03Merged) 10jenkins-bot: Run `npm test` for quarry [integration/config] - 10https://gerrit.wikimedia.org/r/460774 (owner: 10Legoktm) [23:01:21] !log deployed https://gerrit.wikimedia.org/r/460774 https://gerrit.wikimedia.org/r/#/c/integration/config/+/460578/ [23:01:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:01:28] (03CR) 10Legoktm: "I just deployed this now." [integration/config] - 10https://gerrit.wikimedia.org/r/460578 (https://phabricator.wikimedia.org/T204372) (owner: 10MarcoAurelio)