[01:30:46] 3Scap3, 6Revision-Scoring-As-A-Service, 10ores: scap3 doesn't properly load config in targets - https://phabricator.wikimedia.org/T130515#2138106 (10Ladsgroup) [03:13:18] Yippee, build fixed! [03:13:18] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #918: 09FIXED in 23 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/918/ [04:24:28] Yippee, build fixed! [04:24:28] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #757: 09FIXED in 32 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/757/ [05:27:42] Yippee, build fixed! [05:27:42] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #733: 09FIXED in 25 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/733/ [06:21:21] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016, 3Outreachy-Round-12: GSoC/Outreachy Proposal for Improving static analysis tools for Mediawiki - https://phabricator.wikimedia.org/T130382#2138193 (10Aashaka) [06:24:46] 10Gitblit-Deprecate, 10Diffusion: 404 https://phabricator.wikimedia.org/r/branch/wikidata/query/rdf;HEAD - https://phabricator.wikimedia.org/T130475#2138195 (10mmodell) The recent updates to the mapping in GerritProjectController.php haven't been deployed yet. [06:26:58] 6Release-Engineering-Team, 10Phabricator, 6Project-Admins, 7WorkType-Maintenance: Create a Policy Admins project and move all of the acl*various_policy_admins pojects under it as subprojects. - https://phabricator.wikimedia.org/T129515#2138197 (10mmodell) [06:48:39] 6Release-Engineering-Team, 10Phabricator, 6Project-Admins, 7WorkType-Maintenance: Create a Policy Admins project and move all of the acl*various_policy_admins pojects under it as subprojects. - https://phabricator.wikimedia.org/T129515#2138203 (10mmodell) created #policy-admins [06:49:52] 6Release-Engineering-Team, 10Phabricator, 6Project-Admins, 7WorkType-Maintenance: Create a Policy Admins project and move all of the acl*various_policy_admins pojects under it as subprojects. - https://phabricator.wikimedia.org/T129515#2138204 (10mmodell) I haven't converted the existing projects but the d... [06:51:14] Yippee, build fixed! [06:51:15] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce build #764: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/764/ [06:54:00] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016, 3Outreachy-Round-12: GSoC/Outreachy Proposal for Improving static analysis tools for Mediawiki - https://phabricator.wikimedia.org/T130382#2138205 (10Aashaka) [06:55:22] 10MediaWiki-Codesniffer, 10Possible-Tech-Projects, 3Google-Summer-of-Code-2016, 3Outreachy-Round-12: Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T89682#2138207 (10Aashaka) @Legoktm @EBernhardson : Could you provide some microtasks to work on? Thanks! [06:58:38] 6Release-Engineering-Team, 10Phabricator, 6Project-Admins, 7WorkType-Maintenance: Create a Policy Admins project and move all of the acl*various_policy_admins pojects under it as subprojects. - https://phabricator.wikimedia.org/T129515#2138212 (10mmodell) [07:33:31] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce build #355: 04FAILURE in 24 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce/355/ [07:47:24] 10Gitblit-Deprecate, 10Diffusion: Update mediawiki.org templates to link to Diffusion, not gitblit - https://phabricator.wikimedia.org/T108864#1532501 (10Nemo_bis) [07:47:26] 10Gitblit-Deprecate, 6Release-Engineering-Team, 10Diffusion, 7WorkType-NewFunctionality: Use Diffusion as canonical location for browsing code repos (not gitblit) - https://phabricator.wikimedia.org/T752#2138222 (10Nemo_bis) [07:47:29] 5Gerrit-Migration, 10Gitblit-Deprecate: Update {{git file}} to link to diffusion - https://phabricator.wikimedia.org/T101358#2138215 (10Nemo_bis) 5Open>3Resolved a:3Nemo_bis The complete solution is tracked at T129447 / T108864. For {{git file}}, Module:Callsigns is a sufficient solution as the template... [08:13:44] 10Gitblit-Deprecate, 10Diffusion: Diffusion redirect from name to callsign doesn't always work - https://phabricator.wikimedia.org/T129447#2138246 (10Nemo_bis) >>! In T129447#2118845, @demon wrote: > Fixed as part of {D139}, just needs deploying. Is such a manual dump required every time new repositories are... [08:47:15] (03Abandoned) 10Hashar: Add mwext-testextension-{phpflavour}-non-voting to gate and submit [integration/config] - 10https://gerrit.wikimedia.org/r/278394 (owner: 10Paladox) [09:07:19] 5Gerrit-Migration, 10Differential, 7Documentation, 7Upstream: Document best practices to amend a change written by another contributor - https://phabricator.wikimedia.org/T121751#2138260 (10mmodell) http://www.loper-os.org/?p=1545 has an interesting idea about signing a revision with 1 byte of information... [09:13:13] hashar: hi it seems jsduck needs to be installed on the nosepool please reason because MobileFrontend is failing on npm 4.3 because it runs jsduck at the same time as running grunt [09:13:33] 3Scap3: Scap3 should support virtualenv for deployment of python packages - https://phabricator.wikimedia.org/T130205#2138261 (10mmodell) +1 for wheels, it seems like the right tool. I'm not sure what the difficulty is? Scap3 can indeed run remote commands on the target hosts and I can't think of any reason it... [09:13:48] paladox: same case as oo invoking composer from npm [09:13:55] paladox: for jsduck, that should use rake-jessie [09:14:05] with jsduck provided from bundler [09:14:12] Oh ok, for a work around we can add npm-run-doc [09:14:48] hashar ^^ [09:18:07] hashar php is already installed on nodepool it just needs composer to be installed but if php was installed on nodepool woulden it just fail on composer command not php but that's probably because we [09:18:08] Need [09:18:25] To add the php parameter to npm [09:18:46] Jessie provides PHP 5.6 when we need 5.5 and HHVM [09:19:19] Oh [09:20:49] But woulden php 5.6 work for just using it as it is for npm 4.3 since they should use composer if they want to use the php command but if they need to use it in npm they can [09:20:58] hashar ^^ [09:21:41] there is no composer either! [09:21:58] but in short, I am not going to work on PHP support anytime soon [09:22:05] wanna migrate as many npm jobs as possible first [09:22:42] Oh ok, [09:23:17] hashar will you do that that today migrating most of the npm tests or sometime this week [09:30:46] 10MediaWiki-Codesniffer, 7Easy: Add sniff to detect usage of deprecated $wgUser - https://phabricator.wikimedia.org/T50963#2138279 (10hashar) @Aashaka my patch on https://gerrit.wikimedia.org/r/#/c/69090/ is the wrong approach. Reviewer hinted at `DeprecatedGlobals.php`. [09:35:20] 10Continuous-Integration-Config, 10Mathoid, 13Patch-For-Review, 15User-mobrovac: Enable librsvg-dev on jenkins - https://phabricator.wikimedia.org/T128280#2069339 (10hashar) [10:17:01] (03CR) 10Hashar: [C: 04-1] "that would cause a huge amount of checks to be sent to the upstream API and I dont think there is any point to run it on each patchsets. A" [integration/config] - 10https://gerrit.wikimedia.org/r/278571 (https://phabricator.wikimedia.org/T96078) (owner: 10Paladox) [10:19:37] 10Continuous-Integration-Config, 10Dumps-Generation, 6Operations, 13Patch-For-Review, 7WorkType-Maintenance: operations/dumps repo should pass flake8 - https://phabricator.wikimedia.org/T114249#2138340 (10hashar) >>! In T114249#2106764, @ArielGlenn wrote: > Don't despair. I have still on my roadmap to l... [10:48:38] hashar hi, question will you be migrating most of the npm tests to 4.3 today or sometimes this week. [10:54:44] hashar I've updated npm here https://gerrit.wikimedia.org/r/#/c/278572/ I just need to update it so it install the updated npm before running the npm version command. I'm not sure if this is how you would like it to be installed. [10:56:51] (03PS1) 10Hashar: (WIP) HACK tools/trigger-job.py (WIP) [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/278717 [10:58:53] 7Browser-Tests, 10Browser-Tests-Infrastructure, 10Continuous-Integration-Config, 10Wikidata, 3Wikidata-Sprint-2016-03-01: [Task] Remove Wikidata performance-tests jobs - https://phabricator.wikimedia.org/T130017#2138361 (10Tobi_WMDE_SW) I think we can remove the related code from WikidataBrowserTests.git... [11:02:53] !log beta: added Elukey (wikimedia ops) to the project as member and admin [11:02:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:13:29] !log beta: rebased puppet master which had a conflict on https://gerrit.wikimedia.org/r/#/c/274711/ which got merged meanwhile (saves Elukey ) [11:13:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:20:56] 10Browser-Tests-Infrastructure, 6Release-Engineering-Team, 7Epic, 13Patch-For-Review, and 2 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2138382 (10zeljkofilipin) [11:21:40] 7Browser-Tests, 6Collaboration-Team-Backlog, 10Flow, 13Patch-For-Review, 5WMF-deploy-2016-03-22_(1.27.0-wmf.18): Fix Flow scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94153#2138384 (10zeljkofilipin) a:5zeljkofilipin>3None [11:25:44] !log beta: cherry picked https://gerrit.wikimedia.org/r/#/c/278713/ to test an updated to the cdh module (analytics) [11:25:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:27:03] 10Browser-Tests-Infrastructure, 6Release-Engineering-Team, 7Epic, 13Patch-For-Review, and 2 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2138398 (10zeljkofilipin) [11:33:44] Yippee, build fixed! [11:33:44] Project browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #456: 09FIXED in 9 min 49 sec: https://integration.wikimedia.org/ci/job/browsertests-Gather-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/456/ [11:36:01] 10Browser-Tests-Infrastructure, 6Release-Engineering-Team, 7Epic, 13Patch-For-Review, and 2 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2138404 (10zeljkofilipin) [11:36:57] PROBLEM - SSH on deployment-kafka02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:38:08] 10Browser-Tests-Infrastructure, 6Release-Engineering-Team, 7Epic, 13Patch-For-Review, and 2 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2138419 (10zeljkofilipin) [11:44:28] (03CR) 10John Vandenberg: [C: 04-1] (WIP) HACK tools/trigger-job.py (WIP) (032 comments) [integration/zuul] (debian/precise-wikimedia) - 10https://gerrit.wikimedia.org/r/278717 (owner: 10Hashar) [11:53:14] 10Continuous-Integration-Config, 13Patch-For-Review, 7Upstream, 7Zuul: free some repositories from their unintended chain to mediawiki/core - https://phabricator.wikimedia.org/T107529#2138456 (10hashar) I have proposed upstream a patch to have the project templates to be based on the full name of the proje... [11:55:36] Yippee, build fixed! [11:55:36] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce build #356: 09FIXED in 22 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce/356/ [11:55:43] 10Continuous-Integration-Config, 13Patch-For-Review, 7Upstream, 7Zuul: free some repositories from their unintended chain to mediawiki/core - https://phabricator.wikimedia.org/T107529#2138458 (10hashar) We have a bunch of Gerrit projects sharing the same base name: ``` $ rev projectnames.txt |cut -d\/ -f1... [12:31:30] 3Scap3, 6Revision-Scoring-As-A-Service, 10ores: scap3 doesn't properly load config in targets - https://phabricator.wikimedia.org/T130515#2138476 (10Ladsgroup) btw I don't use scap 3.03-1 I use the source code in master and installed it from there. [12:42:47] 3Scap3: Scap3 should support virtualenv for deployment of python packages - https://phabricator.wikimedia.org/T130205#2138484 (10Ladsgroup) >>! In T130205#2138261, @mmodell wrote: > +1 for wheels, it seems like the right tool. > > I'm not sure what the difficulty is? Scap3 can indeed run remote commands on the... [12:51:10] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » sr,contintLabsSlave && UbuntuTrusty build #94: 04FAILURE in 1 hr 7 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=sr,label=contintLabsSlave%20&&%20UbuntuTrusty/94/ [12:51:20] PROBLEM - App Server bits response on deployment-mediawiki02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:33:03] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » hsb,contintLabsSlave && UbuntuTrusty build #94: 04FAILURE in 1 hr 49 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=hsb,label=contintLabsSlave%20&&%20UbuntuTrusty/94/ [13:41:50] RECOVERY - SSH on deployment-kafka02 is OK: SSH OK - OpenSSH_6.6.1p1 Ubuntu-2ubuntu2~wmfprecise2 (protocol 2.0) [14:24:39] !log executed git submodule update --init on deployment-puppetmaster.deployment-prep.eqiad.wmflabs [14:24:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [14:30:25] hashar: so cucumber rake task works fine if we move environments.yml to the root of the repo (from tests/browser) [14:30:34] re https://phabricator.wikimedia.org/T128190 [14:31:06] and you can invoke the rake task from anywhere in the repo, (root, tests/browser...) [14:31:16] PROBLEM - Host cache-rsync is DOWN: CRITICAL - Host Unreachable (10.68.23.165) [14:37:31] (03PS1) 10Hashar: Allow decoupling of npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/278733 (https://phabricator.wikimedia.org/T107529) [14:40:57] (03CR) 10Hashar: [C: 032] Allow decoupling of npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/278733 (https://phabricator.wikimedia.org/T107529) (owner: 10Hashar) [14:42:25] (03Merged) 10jenkins-bot: Allow decoupling of npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/278733 (https://phabricator.wikimedia.org/T107529) (owner: 10Hashar) [14:42:38] 10Continuous-Integration-Config, 6Discovery, 13Patch-For-Review: Take wikimedia/portals repo out of "mediawiki" zuul pipeline in CI - https://phabricator.wikimedia.org/T129591#2138660 (10hashar) I got the project to its own queue via some YAML hacking https://gerrit.wikimedia.org/r/#/c/278733/ . It is all i... [14:42:42] hashar: do you have time to review some patches? [14:42:52] I promise they would be very simple ones [14:43:39] 10Continuous-Integration-Config, 13Patch-For-Review, 7Upstream, 7Zuul: free some repositories from their unintended chain to mediawiki/core - https://phabricator.wikimedia.org/T107529#2138665 (10hashar) Using YAML !!merge, we can extend the generic job and forge new ones with a prefix without duplicating c... [14:47:24] zeljkof: I would avoid moving the environemnets.yml files out of tests/browsers/ to / People will complain and i would rather keep everything related to selenium testing in the same dir [14:48:02] zeljkof: from the rake task executing at the root of the repo, we should be able to give a base path argument and run cucumber in there [14:48:45] hashar: will continue working on it [14:53:31] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016, 3Outreachy-Round-12: GSoC/Outreachy Proposal for Improving static analysis tools for Mediawiki - https://phabricator.wikimedia.org/T130382#2138723 (1001tonythomas) IMPORTANT: The deadline for submitting your proposal for Outreachy'12 falls in less than 24... [14:55:33] require 'mediawiki_selenium/rake_task' [14:55:33] MediawikiSelenium::Rake::Task.new(:basepath => 'tests/browser') [14:55:34] # And our rake task would: [14:55:34] Cucumber::Cli::Main.execute(:paths => @basepath) [14:55:36] zeljkof: $^^^ :D [14:55:41] ok my ruby is lame [14:56:09] hashar: all that already works [14:56:26] but environments.yml file is called from somewhere in the gem [14:56:31] scap3 people: please check this phab card: https://phabricator.wikimedia.org/T130515 [14:56:43] maybe I'm missing something [14:56:45] and looks like it is hardcoded, reading the code [14:56:53] zeljkof: in mediawiki_selenium? [14:56:59] hashar: yes [14:57:11] so the rake task breaks because it can not find it [14:57:12] well you are going to bump that gem version to provide the rake-task [14:57:25] might as well make the default_configuration = 'environements.yml' to be settable [14:57:31] hashar: yes, just looking for the place that does it [14:57:32] and the rake task can then handle it [14:57:42] s/handle/set it [14:59:35] the mediawiki selenium rake task will probably want to set it relatively to where the Rakefile is [15:00:00] Amir1: I am going out in a few [15:00:14] Amir1: for scap, they are out today, will be back tomorrow [15:00:24] yeah, I heard it's a day off [15:00:35] I thought maybe someone is interested [15:01:10] hashar: https://gerrit.wikimedia.org/r/278270 [15:01:13] this is very simple [15:01:25] this one maybe not: https://gerrit.wikimedia.org/r/278271 [15:01:29] thanks [15:08:54] Amir1: ah yeah. I can't merge changes on operations/puppet though :-( [15:09:10] I have dropped myself from reviewer list this morning because of that [15:09:10] :( [15:09:16] ops will get to it eventually [15:11:11] I am off, be back later this evening [15:17:37] 3Scap3, 6Revision-Scoring-As-A-Service, 10ores: scap3 doesn't properly load config in targets - https://phabricator.wikimedia.org/T130515#2138818 (10thcipriani) @Ladsgroup this is //somewhat// expected behavior. In production, each scap target gets a `/etc/scap.cfg` file that is created from an erb template... [15:40:29] (03PS3) 10Paladox: Install npm 2.14.20 for nodejs 4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/278572 [15:44:30] 3Scap3, 6Revision-Scoring-As-A-Service, 10ores: scap3 doesn't properly load config in targets - https://phabricator.wikimedia.org/T130515#2138873 (10Ladsgroup) @thcipriani: Thank you for your detailed answer. I think that's the reason. I'm checking and I'll let you know very soon [15:46:00] !log hacked manually the cdh puppet submodule on deployment-puppetmaster.deployment-prep.eqiad.wmflabs - please let me know if interfere with anybody's tests [15:46:02] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [15:59:08] PROBLEM - SSH on deployment-conftool is CRITICAL: Server answer [16:42:40] 3Scap3, 6Revision-Scoring-As-A-Service, 10ores: scap3 doesn't properly load config in targets - https://phabricator.wikimedia.org/T130515#2139067 (10Ladsgroup) @thcipriani: I couldn't find no scap.cfg in place where deploy-local is being ran nor in /etc/ (maybe it gets permission error when it tries to make... [17:13:08] (03PS2) 10Zfilipin: WIP Simplify creating of Jenkins jobs for running browser tests daily [selenium] - 10https://gerrit.wikimedia.org/r/275820 (https://phabricator.wikimedia.org/T128190) [17:14:52] (03CR) 10jenkins-bot: [V: 04-1] WIP Simplify creating of Jenkins jobs for running browser tests daily [selenium] - 10https://gerrit.wikimedia.org/r/275820 (https://phabricator.wikimedia.org/T128190) (owner: 10Zfilipin) [17:40:15] !log executed git rebase --interactive on deployment-puppetmaster.deployment-prep.eqiad.wmflabs to remove https://gerrit.wikimedia.org/r/#/c/278713/ [17:40:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:13:39] hiii thcipriani :) [19:14:06] ottomata: hello :) [19:15:27] i'm still trying to figure out this event schema deploy thing [19:15:27] q [19:15:49] i'm trying to do the thing with an environment that runs a different command on deploy than the main one [19:15:51] but [19:16:01] the main env scap.cfg, has service_name [19:16:15] which, automatically will restart the service name given there on deploy, yes? [19:16:22] in my environment, how do I disable that? [19:16:33] set service_name in the env's scap.cfg to an empty string? [19:16:37] to false? null? :) [19:16:56] just remove the service_name line from the scap.cfg for the environment [19:17:01] should work [19:17:16] oh [19:17:22] each environment needs the full scap.cfg then? [19:19:12] ottomata: yeah, we don't do any merging for the scap.cfg. That got complicated quickly. [19:19:25] so each environment is independent [19:20:51] oh, ok, so if you are going to have scap.cfg in your env dir, you need the whole thing. [19:20:52] ja? [19:22:11] yup [19:22:52] ok, will try [19:24:18] PROBLEM - Host deployment-analytics104 is DOWN: CRITICAL - Host Unreachable (10.68.17.18) [19:26:13] PROBLEM - Host deployment-analytics103 is DOWN: CRITICAL - Host Unreachable (10.68.16.108) [19:26:45] PROBLEM - Host deployment-analytics101 is DOWN: CRITICAL - Host Unreachable (10.68.23.144) [19:27:57] PROBLEM - SSH on deployment-kafka02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:32:06] thcipriani: is fetch location always hardcoded to /srv/deployment/$git_repo [19:32:06] ? [19:34:18] it's not hardcoded (it's git_deploy_dir config var, IIRC) [19:34:51] although, it may be problematic to change for some fuzzy reason I can't quite remember. [19:35:44] so it'd be: git_deploy_dir/git_repo [19:36:09] is the fetch location [19:36:13] ok [19:42:56] thcipriani: if there is no code change or sha change during deployment, does promote stage still happen? [19:43:59] the stage still happens but nothing happens in the stage if that makes any sense :P You'd have to use --force to make something happen. [19:44:32] it's up to each target to determine if anything needs to be run for promote [19:44:54] if sha doesn't change on target, it won't run anything? [19:45:40] yeah. if the sha that is currently pointed to by the current symlink doesn't change, nothing gets run, correct. [19:46:17] ok [19:46:54] ok, and promote stage checks should always be run with --force? [19:49:14] yup [19:49:17] thcipriani: currently i am trying the this: https://gist.github.com/ottomata/9a6e5c26e9e2ca2e4ebe [19:49:46] the scap.cfg in that env is the same as the main one, except no service_name [19:49:51] and i run [19:49:52] deploy -e event-schemas-beta -v --force [19:49:58] but nothing shows up in /tmp/deployed-at.txt [19:50:26] hmm [19:50:40] what does: .git/DEPLOY_HEAD look like on the deploy host? [19:52:50] git_repo: eventlogging/ottotest/deploy/eventbus, git_rev: b91b106b0fb37a774282c5bb03df2efdd6e2f19c, [19:52:50] git_scheme: http, git_server: deployment-tin.deployment-prep.eqiad.wmflabs, git_submodules: true, [19:52:50] git_upstream_submodules: true, log_json: true, nrpe_dir: /etc/nagios/nrpe.d, perform_checks: false, [19:52:51] service_port: '8085', service_timeout: 120.0, tag: scap/sync/2016-03-21/0014, timestamp: '2016-03-21T19:48:01.284122', [19:52:51] user: otto} [19:53:03] perform_checks: false ? [19:53:30] thcipriani: ^? [19:53:57] that's...weird. It defaults to True... [19:54:22] my main env doesn't have any checks [19:54:27] no checks.yaml [19:54:54] hmm...lemme check to make sure that this is using the env-specific checks.yaml [19:55:18] ok [19:55:38] i just set it manually in the env's scap.cfg, no change though [19:57:14] yeah, looks like it's not being set to an env-specific path for checks.yaml https://github.com/wikimedia/scap/blob/master/scap/deploy.py#L605-L607 [19:57:17] * thcipriani grumbles [19:57:34] ahhh rts! [19:57:35] rats! [19:58:15] I'll file a task for this, this is definitely not the intended behavior :( [19:58:18] ok cool [19:58:18] thanks [19:58:27] thcipriani: so also real quick, lemme run by you my current plan [19:58:32] (my plan seems to change every time i look at this...) [19:58:33] but [20:00:27] https://gerrit.wikimedia.org/r/#/c/278752/ [20:00:32] made that today, not yet merged [20:00:53] * thcipriani looks [20:01:34] oops [20:01:36] sorry [20:01:37] thcipriani: w [20:01:40] wrong chat [20:01:42] haha [20:01:44] thcipriani: this is for you: https://gist.github.com/ottomata/27d7e3cb1a93e1cb190f [20:01:46] to look at [20:01:52] heh [20:01:58] * thcipriani was confused [20:02:22] so [20:02:33] i'm thinking of rearranging things [20:02:38] the * in that gist are submodules [20:02:54] then [20:03:03] for eventbus/scap [20:03:11] i'll have an event-schemas environment [20:03:18] that will have a checks.yaml [20:03:24] that does service eventlogging-service-eventbus reload [20:03:28] so [20:03:38] if I know that i'm only deploying a change in event-schemas submodule [20:03:46] i can deploy using that environment [20:04:14] so, each service that needs event-schemas still has to use a submodule [20:04:34] but, this way, for say, eventlogging, where i really shouldn't have event-schemas/ or scap/ in the eventlogging codebase [20:04:45] i can make both eventlogging and scap and event-schemas part of a deploy/ repo instead [20:05:33] refinery on the other hand, is 100% wmf specific, and has no need for a special deploy/ repo [20:05:37] or special deploy envs [20:11:39] is this setup solely to get the service reload when event-schemas submodule is changed? Or are there other benefits? It sounds like it's workable, just curious. [20:12:33] hm [20:12:39] well, yes and no [20:12:50] i mean, it'd be better if we had a single event-schemas source [20:12:54] that could be deployed to different services [20:13:16] but, that doesn't seem possible, since the .git/HEAD stuff will be changed all the time [20:13:23] for different deploys for different reasons [20:13:36] it seems a deploy source is really only meant for a single service [20:13:43] yeah it is. [20:13:48] its ok to have more environments, but usually that's because it really IS a different source [20:13:53] with all different envs in our case [20:13:54] e.g. [20:14:00] deployment-prep is a different host [20:14:11] so, the envs there actually do have different .git dirs [20:14:16] thcipriani: q: [20:14:23] what if someone wanted to deploy multiple envs in production [20:14:25] or even in labs? [20:14:37] maybe there's a prod 'test' cluster [20:14:47] a deployment to that test cluster [20:15:06] would cause a future newly provisioned non-test prod node to pull the wrong version [20:15:08] ja? [20:15:22] indeed. [20:15:57] thcipriani: eeeee, kinda sounds like envs aren't really envs :P [20:16:11] The puppet provider has no environment concept, so it just looks at whatever the deployment host is. [20:17:09] you know>.>.>> [20:17:17] maybe i should just deploy event-schemas with puppet git [20:17:29] hm. [20:17:40] hm [20:17:41] hm hm hm [20:18:00] yea. [20:18:45] could work. have event-schemas gitignored in the repo, let puppet handle that deploy/reload? [20:20:19] naw, i'd just deploy it to somethin glike /etc/event-schemas [20:20:26] with an event scheams puppet module [20:20:26] then [20:20:30] if the service needs reloaded [20:20:35] i'd just subscribe to the module [20:20:36] in puppet [20:20:43] with a reload i guess somehow [20:20:57] then any host that needs event-schemas can just get it included via puppet. [20:21:29] sounds like it'd be less submodule wrangling for sure. [20:23:57] 10Deployment-Systems, 3Scap3, 10scap: Scap3 checks.yaml should be environment specific - https://phabricator.wikimedia.org/T130558#2139397 (10thcipriani) [20:24:42] 10Deployment-Systems, 3Scap3, 10scap: Scap3 checks.yaml should be environment specific - https://phabricator.wikimedia.org/T130558#2139397 (10Ottomata) Offender: https://github.com/wikimedia/scap/blob/master/scap/deploy.py#L605-L607 [20:28:52] !log Mass running npm-node-4.3 jobs against MediaWiki extensions to make sure they all pass ( https://gerrit.wikimedia.org/r/#/c/278004/ | T119143 ) [20:28:53] T119143: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143 [20:28:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:11:49] hashar: Hi [21:12:11] I see your mass testing with npm node 4.3 [21:31:27] 5Continuous-Integration-Scaling, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2139506 (10hashar) I have trigger the npm-node4.3 job on 311 extensions impacted by https://gerrit.wikimedia.org/r/#/c/278004/ the only failu... [21:32:47] (03CR) 10Hashar: "I have triggered the job npm-node-4.3 against the 311 extensions changed. The list of failures:" [integration/config] - 10https://gerrit.wikimedia.org/r/278004 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:36:02] (03PS2) 10Hashar: Mass switch repositories npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/278004 (https://phabricator.wikimedia.org/T119143) [21:37:49] (03PS3) 10Hashar: Mass switch repositories npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/278004 (https://phabricator.wikimedia.org/T119143) [21:39:49] (03CR) 10Hashar: [C: 032] "Some extensions are failing for various reasons, I have marked them with 'npm-legacy'. All the others are passing properly." [integration/config] - 10https://gerrit.wikimedia.org/r/278004 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:40:53] (03Merged) 10jenkins-bot: Mass switch repositories npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/278004 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:42:45] (03CR) 10Krinkle: [C: 04-1] "Per comments on I789dfbc70. This is not useful." [integration/config] - 10https://gerrit.wikimedia.org/r/278443 (owner: 10Paladox) [21:43:43] hashar: I see that https://gerrit.wikimedia.org/r/#/c/278004/ has been merged [21:46:47] (03PS3) 10Hashar: Zuul: rm 'npm-node-4.3' project template [integration/config] - 10https://gerrit.wikimedia.org/r/278005 (https://phabricator.wikimedia.org/T119143) [21:47:09] paladox: yeah mass tested it [21:47:16] almost all jobs pass just fine [21:47:48] out of 311 I ran test for, only a handful failed -} [21:47:59] (03CR) 10Krinkle: [C: 04-1] "This will be slow, wasteful and fragile. When the job starts, the environment needs to provide reasonable and compatible versions of node/" [integration/config] - 10https://gerrit.wikimedia.org/r/278572 (owner: 10Paladox) [21:51:04] (03CR) 10Hashar: [C: 04-2] "Keeping this one on hold for a few" [integration/config] - 10https://gerrit.wikimedia.org/r/278005 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [21:53:33] 5Continuous-Integration-Scaling, 13Patch-For-Review, 7WorkType-NewFunctionality: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143#2139527 (10hashar) Most Mediawiki extensions have been migrated for which the npm-node-4.3 yields a SUCCESS. There are still a few repo left... [21:55:17] !log zuul: almost all MediaWiki extensions migrated to run the npm job on Nodepool (with Node.js 4.3) T119143 . All tested. Will monitor the build results that ran overnight tomorrow [21:55:18] T119143: Migrate javascript npm CI jobs to Nodepool - https://phabricator.wikimedia.org/T119143 [21:55:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [22:31:47] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016: Improving static analysis tools for MediaWiki(GSoC 2016 Proposal) - https://phabricator.wikimedia.org/T130456#2139541 (10Billghost) [22:37:21] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016: Improving static analysis tools for MediaWiki(GSoC 2016 Proposal) - https://phabricator.wikimedia.org/T130456#2139543 (10Billghost) [22:51:34] PROBLEM - Host deployment-cache-mobile04 is DOWN: CRITICAL - Host Unreachable (10.68.18.110) [22:52:38] 10MediaWiki-Codesniffer, 3Google-Summer-of-Code-2016: Improving static analysis tools for MediaWiki(GSoC 2016 Proposal) - https://phabricator.wikimedia.org/T130456#2139547 (10Billghost)