[00:01:32] 10Gerrit, 10Release-Engineering-Team (Someday), 10Patch-For-Review: Update gerrit to 2.14.6 - https://phabricator.wikimedia.org/T156120#3898226 (10Paladox) [00:12:48] 10Gerrit, 10Release-Engineering-Team (Someday): Update gerrit to 2.15 - https://phabricator.wikimedia.org/T177201#3898240 (10Paladox) [00:26:49] 10Release-Engineering-Team (Kanban), 10Wikimedia-Incident: Address proximity of service deployments to train deployments problem - https://phabricator.wikimedia.org/T182733#3898249 (10greg) First, thanks for getting me to get the Deployments gcal back in sync with the wiki schedule (canonical source) ;) Secon... [00:35:04] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [00:43:44] 10Release-Engineering-Team (Kanban), 10Wikimedia-Incident: Address proximity of service deployments to train deployments problem - https://phabricator.wikimedia.org/T182733#3898256 (10greg) p:05Triage>03Normal [00:54:58] 10Release-Engineering-Team (Kanban), 10Wikimedia-Incident: Address proximity of service deployments to train deployments problem - https://phabricator.wikimedia.org/T182733#3898260 (10greg) Moved Wed's morning SWAT one hour earlier (and added the explicit hour break): https://wikitech.wikimedia.org/w/index.php... [00:59:59] 10MediaWiki-Releasing, 10Security: Consider using a single MediaWiki releases key instead of individual keys - https://phabricator.wikimedia.org/T181019#3898263 (10greg) [01:00:01] 10MediaWiki-Releasing: Cleanup https://www.mediawiki.org/keys/keys.html and related - https://phabricator.wikimedia.org/T181017#3898262 (10greg) [01:01:46] 10MediaWiki-Releasing, 10Security, 10Security-Extensions: Backport security fixes to stable + LTS extension branches - https://phabricator.wikimedia.org/T108734#3898264 (10greg) [02:00:05] (03CR) 10Phantom42: "Yes, there is no need in this change anymore" [integration/config] - 10https://gerrit.wikimedia.org/r/403325 (https://phabricator.wikimedia.org/T175794) (owner: 10Phantom42) [02:00:11] (03Abandoned) 10Phantom42: Add BlueSpiceFoundation dependency for BlueSpice extensions [integration/config] - 10https://gerrit.wikimedia.org/r/403325 (https://phabricator.wikimedia.org/T175794) (owner: 10Phantom42) [02:25:38] 10MediaWiki-Releasing, 10Security, 10Security-Extensions: Backport security fixes to stable + LTS extension branches - https://phabricator.wikimedia.org/T108734#1529012 (10Bawolff) Ive always tried to ensure fixes are backported for extension fixes im involved with, but we probably need to do better. The st... [02:37:19] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Customize Gerrit's login UI - https://phabricator.wikimedia.org/T184778#3895960 (10Peachey88) Since this isn't the default/upstream themeing, is it a level of tech debt we should be adding to gerrit as it becomes something we need to maintain? [02:49:38] 10Gerrit, 10Release-Engineering-Team, 10Patch-For-Review: Customize Gerrit's login UI - https://phabricator.wikimedia.org/T184778#3898354 (10Paladox) >>! In T184778#3898351, @Peachey88 wrote: > Since this isn't the default/upstream themeing, is it a level of tech debt we should be adding to gerrit as it beco... [04:00:08] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<55.56%) [06:55:06] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [06:58:03] PROBLEM - Puppet errors on deployment-kafka03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [07:35:45] 10commit-message-validator: 'success' != 'commit-message-validator v0.5.2\...' test failiures - https://phabricator.wikimedia.org/T184844#3898405 (10Dalba) [07:54:16] (03PS1) 10Dalba: Update success message of tests [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404068 (https://phabricator.wikimedia.org/T184844) [08:06:33] 10commit-message-validator: Installed post-commit hook does not run correctly on Windows when using git bash - https://phabricator.wikimedia.org/T184845#3898430 (10Dalba) [08:15:01] (03PS1) 10Dalba: Use forward slash as the path separator for git hook on Windows [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) [08:15:46] (03PS2) 10Dalba: Use forward slash as the path separator for git hook on Windows [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) [08:18:32] (03PS3) 10Dalba: Use forward slash as the path separator for git hook command on Windows [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) [08:22:11] 10commit-message-validator, 10Patch-For-Review: Installed post-commit hook does not run correctly on Windows when using git bash - https://phabricator.wikimedia.org/T184845#3898452 (10Dalba) [08:59:16] (03CR) 10Legoktm: "Should we be using os.sep? Or is hardcoding Windows OK?" [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) (owner: 10Dalba) [09:05:35] (03CR) 10Thiemo Kreuz (WMDE): [C: 031] Remove BetaFeatures from ORES extension dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/403904 (https://phabricator.wikimedia.org/T184554) (owner: 10Ladsgroup) [09:10:55] (03CR) 10Dalba: "> Should we be using os.sep? Or is hardcoding Windows OK?" [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) (owner: 10Dalba) [09:12:30] (03CR) 10Legoktm: [C: 032] "Got it." [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) (owner: 10Dalba) [09:13:10] (03Merged) 10jenkins-bot: Use forward slash as the path separator for git hook command on Windows [integration/commit-message-validator] - 10https://gerrit.wikimedia.org/r/404069 (https://phabricator.wikimedia.org/T184845) (owner: 10Dalba) [09:23:59] 10commit-message-validator, 10Patch-For-Review: Installed post-commit hook does not run correctly on Windows when using git bash - https://phabricator.wikimedia.org/T184845#3898520 (10Dalba) 05Open>03Resolved [10:01:17] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [10:31:08] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [11:21:21] zeljkof: around at all? [11:24:47] addshore: on the phone [11:24:52] What's up? [11:25:10] trying to run the core selenium tests locally (on windows) and hitting a snag [11:25:14] think im pretty close though [11:25:29] $ chromedriver --url-base=/wd/hub --port=4444 --verbose [11:25:29] Starting ChromeDriver 2.35.528161 (5b82f2d2aae0ca24b877009200ced9065a772e73) on port 4444 [11:25:29] Only local connections are allowed. [11:25:47] $ ../../node_modules/.bin/wdio --spec specs/user.js --mochaOpts.grep preferences [11:25:47] ERROR: unhandled request [11:25:47] chrome [11:25:47] Error: unhandled request [11:25:59] $ curl localhost:4444/wd/hub [11:26:01] unhandled request [11:26:10] Just wondering if you have come accrosss something like that before :P [11:27:25] oh wait, got it [11:27:26] Uh [11:27:28] WIN [11:27:42] Do you have chromedriver installed [11:27:46] in the README i think chromedriver --url-base=/wd/hub --port=4444 is wrong, it should be chromedriver --url-base=wd/hub --port=4444 [11:27:49] And in path? [11:28:06] (note the / that was at the start of url-base and now is not [11:28:18] Readme is correct [11:28:34] hmm, if I remove that / the test runs [11:28:41] chrome loads & everything [11:29:56] Maybe a Windows thing [11:30:01] could be [11:30:15] Send patch to core [11:30:20] will do! :) [11:30:31] well, thanks for listening to me spam the channel and solve the issue myself :D [11:30:32] I'll try it on Mac and Linux [11:30:39] :) [11:30:49] I'm glad to help ;) [11:31:01] now for some reason it isnt picking up my MW_SERVER :( [11:31:03] meh [11:34:11] okay, passing them in on the same line I run the command seems to work :) but in my actual env and they dont [11:34:12] WIN [11:34:16] <3 [11:48:00] now im just getting an odd timeout [11:50:44] seeeet, all running and passing [14:20:56] 10Gerrit, 10Release-Engineering-Team, 10releng-201516-q3, 10Developer-Relations, and 3 others: [RfC]: Migrate code review / management from Gerrit to Phabricator - https://phabricator.wikimedia.org/T119908#3898613 (10HappyDog) Apologies if you found my comments condescending. That was not the intention.... [14:27:42] 10Gerrit, 10Release-Engineering-Team, 10releng-201516-q3, 10Developer-Relations, and 3 others: [RfC]: Migrate code review / management from Gerrit to Phabricator - https://phabricator.wikimedia.org/T119908#3898614 (10Paladox) @HappyDog this is what it will eventually look like https://docs.google.com/prese... [14:29:26] 10Beta-Cluster-Infrastructure: Setup multiversion on Beta Cluster for nightly build browser testing support - https://phabricator.wikimedia.org/T67127#694082 (10Addshore) @greg & co, any chance of an update on this ticket? > As such, we'd need to do multiversion in Beta Cluster. I guess beta is multiversion no... [15:15:26] (03PS1) 10Addshore: WIP core browser tests on beta after update [integration/config] - 10https://gerrit.wikimedia.org/r/404082 [15:27:45] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:33:42] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:18:31] PROBLEM - Free space - all mounts on deployment-kafka01 is CRITICAL: CRITICAL: deployment-prep.deployment-kafka01.diskspace.root.byte_percentfree (<100.00%) [18:26:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Recommendation-API, 10Scoring-platform-team (Current): What to do with deployment-sca03? - https://phabricator.wikimedia.org/T184501#3898784 (10mobrovac) +1 on removing it. @Ottomata ok with it too? [18:39:22] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:54:30] PROBLEM - Free space - all mounts on deployment-kafka03 is CRITICAL: CRITICAL: deployment-prep.deployment-kafka03.diskspace.root.byte_percentfree (<100.00%) [18:58:55] PROBLEM - Puppet staleness on deployment-kafka03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [19:42:56] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T180749#3898841 (10greg) [20:18:44] 10RelEng-Archive-FY201718-Q2, 10Patch-For-Review, 10Release, 10Train Deployments: MW-1.28.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T143328#3898877 (10Umherirrender) [21:06:16] PROBLEM - Free space - all mounts on deployment-mx is CRITICAL: CRITICAL: deployment-prep.deployment-mx.diskspace._var_log.byte_percentfree (<100.00%) [23:36:09] PROBLEM - Puppet errors on deployment-netbox is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:41:44] 10Continuous-Integration-Config, 10Patch-For-Review, 10User-Addshore: Allow use of phan 0.8.5 in wikimedia CI - https://phabricator.wikimedia.org/T174339#3898958 (10Addshore) [23:41:47] 10Continuous-Integration-Infrastructure, 10User-Addshore: Upgrade php-ast for CI nodepool image to ^0.1.4 (currently 0.1.2) - https://phabricator.wikimedia.org/T174338#3898956 (10Addshore) 05stalled>03declined phan doesnt run on nodepool anymore so this is not needed. [23:42:01] 10Continuous-Integration-Config, 10Patch-For-Review, 10User-Addshore: Allow use of phan 0.8.5 in wikimedia CI - https://phabricator.wikimedia.org/T174339#3558368 (10Addshore) 05stalled>03Open [23:42:06] 10Continuous-Integration-Config, 10Patch-For-Review, 10User-Addshore: Allow use of phan 0.8.5 in wikimedia CI - https://phabricator.wikimedia.org/T174339#3558368 (10Addshore) a:03Addshore [23:42:15] 10Continuous-Integration-Config, 10Patch-For-Review, 10User-Addshore: Allow use of phan 0.8.5+ in wikimedia CI - https://phabricator.wikimedia.org/T174339#3558368 (10Addshore) [23:42:21] (03PS1) 10Addshore: docker: update phan image to phan 0.8.10 [integration/config] - 10https://gerrit.wikimedia.org/r/404142 (https://phabricator.wikimedia.org/T174339) [23:43:54] 10Continuous-Integration-Config: mw-ext-php70-phan-jessie complains about PHP temp directory not writable to composer - https://phabricator.wikimedia.org/T167969#3898965 (10Addshore) @hashar afaik we don't use this job anymore? [23:45:24] (03Draft1) 10Addshore: WIPDNM phan 0.9 [integration/config] - 10https://gerrit.wikimedia.org/r/404141 [23:45:57] (03Abandoned) 10Addshore: WIPDNM phan 0.9 [integration/config] - 10https://gerrit.wikimedia.org/r/404141 (owner: 10Addshore)