[00:16:03] (03PS2) 10Paladox: [LifeWeb] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281564 [00:17:27] (03PS1) 10Paladox: [LifeWebCore] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281577 [00:20:30] (03PS3) 10Paladox: [LifeWeb] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281564 [00:21:49] (03PS2) 10Paladox: [LifeWebCore] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281577 [00:24:29] 10Beta-Cluster-Infrastructure, 10Staging, 10DBA, 3Collab-Archive-2015-2016, and 4 others: Use External Store on Beta Cluster - https://phabricator.wikimedia.org/T95871#2179553 (10Mattflaschen) [00:36:03] RECOVERY - Free space - all mounts on deployment-fluorine is OK: OK: All targets OK [00:48:37] (03PS1) 10Paladox: [LightweightRDFa] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281582 [01:46:55] 10scap: [scap] multi datacenter aware without (major) performance hit - https://phabricator.wikimedia.org/T71572#2179625 (10bd808) We did do a couple of things for this that weren't tracked by this task: * per-row rsync slaves in both eqiad and codfw * when multi master support was added, the initial slave rsync... [01:46:59] 7Browser-Tests: Browsertests has a lot of unstable builds - https://phabricator.wikimedia.org/T131822#2179626 (10Luke081515) [03:15:14] Yippee, build fixed! [03:15:15] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #1036: 09FIXED in 33 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/1036/ [06:34:20] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #70: 04FAILURE in 7 min 19 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/70/ [07:38:40] 10Continuous-Integration-Infrastructure, 6Operations, 6Services, 13Patch-For-Review: Package npm 2.14 - https://phabricator.wikimedia.org/T124474#2179870 (10Ricordisamoa) >>! In T124474#2164007, @Krinkle wrote: > It would additionally be interesting to explore actual multi-version node (nvm's primary featu... [08:35:11] Yippee, build fixed! [08:35:12] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #931: 09FIXED in 25 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/931/ [08:52:55] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 13Patch-For-Review: Migrate leftover tox jobs to CI Nodepool - https://phabricator.wikimedia.org/T126588#2179928 (10hashar) a:3hashar [08:54:43] (03PS1) 10Hashar: [pywikibot/core] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281608 (https://phabricator.wikimedia.org/T126588) [08:57:18] (03CR) 10Hashar: [C: 032] [pywikibot/core] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281608 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [08:58:46] (03Merged) 10jenkins-bot: [pywikibot/core] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281608 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:02:28] (03CR) 10Hashar: "Worked just fine on https://gerrit.wikimedia.org/r/#/c/281610/ :-)" [integration/config] - 10https://gerrit.wikimedia.org/r/281608 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:09:31] (03PS1) 10Hashar: [pywikibot/wikibase] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281613 (https://phabricator.wikimedia.org/T126588) [09:13:20] (03CR) 10Hashar: [C: 032] [pywikibot/wikibase] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281613 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:14:42] (03Merged) 10jenkins-bot: [pywikibot/wikibase] move some tox jobs to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281613 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:19:05] (03CR) 10Hashar: "Validated on https://gerrit.wikimedia.org/r/#/c/281431/" [integration/config] - 10https://gerrit.wikimedia.org/r/281613 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:20:10] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 13Patch-For-Review: Migrate leftover tox jobs to CI Nodepool - https://phabricator.wikimedia.org/T126588#2179950 (10hashar) pywikibot repositories got migrated :-) [09:21:55] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 13Patch-For-Review: Migrate leftover tox jobs to CI Nodepool - https://phabricator.wikimedia.org/T126588#2179951 (10hashar) [09:26:01] (03PS1) 10Hashar: [analytics/limn-language-data] drop tox jobs [integration/config] - 10https://gerrit.wikimedia.org/r/281616 [09:26:21] hashar: Does wine on a mac allow you to install windows programes [09:26:24] (03PS2) 10Hashar: [analytics/limn-language-data] drop tox jobs [integration/config] - 10https://gerrit.wikimedia.org/r/281616 (https://phabricator.wikimedia.org/T126588) [09:26:32] hashar: Oh wait it does http://winebottler.kronenberg.org/ [09:26:37] (03CR) 10Hashar: [C: 032] [analytics/limn-language-data] drop tox jobs [integration/config] - 10https://gerrit.wikimedia.org/r/281616 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:27:07] Oh wrong package [09:27:50] https://www.winehq.org/ [09:27:56] (03Merged) 10jenkins-bot: [analytics/limn-language-data] drop tox jobs [integration/config] - 10https://gerrit.wikimedia.org/r/281616 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:30:54] hashar: can we have another look / think about https://gerrit.wikimedia.org/r/#/c/281206/ ? :0 [09:31:56] (03CR) 10Paladox: [C: 031] Remove extension ZeroPortal from gate extensions [integration/config] - 10https://gerrit.wikimedia.org/r/281206 (owner: 10Addshore) [09:34:54] PROBLEM - Host integration-trusty-1026 is DOWN: CRITICAL - Host Unreachable (10.68.17.98) [09:36:33] (03PS1) 10Hashar: [analytics/wikimetrics] Migrate tox job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281618 (https://phabricator.wikimedia.org/T126588) [09:37:45] (03CR) 10Hashar: [C: 032] [analytics/wikimetrics] Migrate tox job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281618 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:38:49] (03Merged) 10jenkins-bot: [analytics/wikimetrics] Migrate tox job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/281618 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:39:36] addshore: what is wrong with ZeroPortal anyway ? :-) [09:39:41] it has been around for a while [09:47:50] (03PS1) 10Hashar: Remove tox templates that ran on permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/281619 (https://phabricator.wikimedia.org/T126588) [09:49:48] (03CR) 10Hashar: [C: 032] Remove tox templates that ran on permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/281619 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:51:09] (03Merged) 10jenkins-bot: Remove tox templates that ran on permanent slaves [integration/config] - 10https://gerrit.wikimedia.org/r/281619 (https://phabricator.wikimedia.org/T126588) (owner: 10Hashar) [09:58:16] hashar: it requires Scribunto which is not loaded for those tests [09:58:39] thus https://gerrit.wikimedia.org/r/#/c/279367/ fails as the special page fatals [09:58:57] so if we cant add Scribunto right now we should remove ZeroPortal [09:59:10] ohh [09:59:27] I can imagine Zeroportal test that relies on Scribunto got added when Scribunto was in the gate job [09:59:42] maybe we can "just" skip the ZeroPortal tests whenever Scribunto is not around? [09:59:48] (03PS1) 10Hashar: Fix job pywikibot-tests-beta-cluster [integration/config] - 10https://gerrit.wikimedia.org/r/281620 [10:00:12] (03CR) 10Hashar: [C: 032] Fix job pywikibot-tests-beta-cluster [integration/config] - 10https://gerrit.wikimedia.org/r/281620 (owner: 10Hashar) [10:00:15] hashar: there are not actually any ZeroProtal phpunit tests [10:00:19] at all [10:02:13] (03Merged) 10jenkins-bot: Fix job pywikibot-tests-beta-cluster [integration/config] - 10https://gerrit.wikimedia.org/r/281620 (owner: 10Hashar) [10:02:35] 10Continuous-Integration-Config, 15User-greg: Migrate python tox CI jobs to Nodepool - https://phabricator.wikimedia.org/T119141#2180017 (10hashar) [10:02:37] 10Continuous-Integration-Config, 5Continuous-Integration-Scaling, 13Patch-For-Review: Migrate leftover tox jobs to CI Nodepool - https://phabricator.wikimedia.org/T126588#2180015 (10hashar) 5Open>3Resolved The publish jobs need a proper infra/system to be able to grab artifacts from the job running on No... [10:04:31] addshore: OHHHHHHHH [10:04:43] so that is a structure test [10:04:54] that remembers me of the mw core ApiDocumentationTest [10:05:01] which ends up breaking a wild range of extensions [10:05:43] yup, its a structure test! that simply loads all special pages and make sure they do not fatal! [10:06:13] and the ZeroPortal special page fatals out because of lack of Scribunto yeah [10:06:14] make sense [10:06:39] yeh, it looks like the whole thing requires Scribunto [10:08:52] the commit message is really misleading / uninformative [10:08:54] anyway, triggering tests via https://gerrit.wikimedia.org/r/281623 :D [10:10:12] addshore: I would rather hack ZeroPortal to raise an exception whenever it is missing Scribunto/Lua [10:10:18] and keep ZeroPortal in the gate job [10:12:54] (03CR) 10Hashar: [C: 04-1] "After talking with Addshore, the commit message is misleading. The reason ZeroPortal test fail is due to the introduction of MediaWiki co" [integration/config] - 10https://gerrit.wikimedia.org/r/281206 (owner: 10Addshore) [10:16:34] Hello! I have a few friends coming over this weekend to hack on a few things wikimedia related. We'd like to have a go at puppet unit tests (https://phabricator.wikimedia.org/T131833). [10:16:57] Do you have any idea what is actually needed to make those tests run as part of our CI? [10:20:14] addshore: did the bits. I would make ZeroPortal special page to output an error message or raise an exception [10:20:29] addshore: and for extensions that have fataling special page, we would use the same method [10:20:54] gehel: we have a task about running puppet rspec tests in CI [10:21:11] hashar: I did not find it. You have a link? [10:21:28] gehel: https://phabricator.wikimedia.org/T78342 [10:21:31] Create a basic RSpec unit test for operations/puppet [10:21:57] we tried a bit back a while ago but havent gone far [10:22:27] we even had a non voting job to run rspec on each module [10:22:32] I got rid of it not so long ago [10:23:02] hashar: anything we can do to move things forward? [10:23:39] we do have a few modules that do have rspec tests. What is missing to have them as part of CI? [10:24:00] * hashar digs in the hstory [10:24:08] so the job was https://gerrit.wikimedia.org/r/#/c/179244/1/jjb/operations-puppet.yaml,cm [10:24:14] which really boils down to running: [10:24:22] bundle install [10:24:27] bundle exec rspec [10:24:35] hoping that rspec will magically find the various spec :-D [10:25:05] (bundler being the magic ruby utility to fetch dependencies and run commands in that list of dependencies) [10:25:31] the very basic rough attempt is still in operations/Puppet in the /Rakefile [10:25:35] bundle install && bundle exec rake spec [10:25:38] * gehel has been hit on the head by bundler before [10:25:52] the Rake task ends up globing modules and attempt to run 'rake spec' in each of them [10:26:04] the thing is each modules has different set of dependencies / test runner / different version expectations [10:26:11] hashar: Hi how can we export the bundle path to npm 0.10 tests please https://gerrit.wikimedia.org/r/#/c/280929/ [10:26:23] paladox: I am not going to do that one [10:26:26] So that we can merge https://gerrit.wikimedia.org/r/#/c/281384/ [10:26:40] hashar: Oh, but then the npm 0.10 tests fail. [10:26:55] paladox: I will get to parsoid eventually. But most probably we will drop 0.10 entirely [10:27:13] gehel: should we use https://phabricator.wikimedia.org/T78342 ? and merge the other as dupe ? [10:27:17] hashar: so we could have a look at uniformizing the dependencies. [10:27:22] gehel: or the other way around I dont really care [10:27:29] then I can write down some informations on the task [10:27:29] hashar: yes, I'll close it as duplicate... [10:27:37] hashar: thanks for the help! [10:28:00] hashar: Oh, they doint want npm 0.10 droped. I asked them and said no. [10:29:35] hashar: Is there any way to use npm code to detect node version so node 0.10 will run bundle install then the path and if and node version is higher then it will run bundle install only. [10:29:37] Please [10:29:39] !log gallium removing libav / ffmpeg. No more needed since jobs are no more running on that server [10:29:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:31:02] 10Continuous-Integration-Infrastructure, 6Operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#2180052 (10Gehel) [10:32:37] !log gallium removing texlive [10:32:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:33:31] PROBLEM - DPKG on gallium is CRITICAL: DPKG CRITICAL dpkg reports broken packages [10:33:50] 10Continuous-Integration-Infrastructure, 6Operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#2180065 (10Gehel) The job used to run those rspec was disabled recently in https://gerrit.wikimedia.org/r/#/c/179244/. The issue was that different modules used... [10:34:01] gehel: and yeah having rspec tests to run on puppet would be rather nice [10:34:11] * gehel would love it! [10:35:02] hashar: we should be able to move this forward a bit this Saturday. If I understand correctly, if we manage to have the main Rakefile run correctly under bundler, that's a good step forward. Correct? [10:35:37] gehel: yup [10:35:52] then each modules have different set of requirements / dependencies [10:36:13] so one would probably have to find modules that potentially have tests [10:36:22] then foreach of them run bundle install && bundle exec rake spec [10:36:23] or rspec [10:36:27] or spec_standalone [10:36:36] so each module can have different dependencies / target [10:36:44] I am writing down a summary on the task [10:37:09] probably a stupid question, but why not have each module independent in term of tests and simply iterate over each module and run bundler in each module? [10:38:01] Ok, so that's what your saying. We should make modules independent ... [10:39:00] RECOVERY - DPKG on gallium is OK: All packages OK [10:42:47] hashar: Looks like you can do if and else in npm. [10:44:21] PROBLEM - DPKG on gallium is CRITICAL: DPKG CRITICAL dpkg reports broken packages [10:46:11] RECOVERY - DPKG on gallium is OK: All packages OK [10:48:37] hashar: Hi [10:48:51] hashar: Could you remove the hack from https://integration.wikimedia.org/ci/job/parsoidsvc-source-npm-node-4.3/16/console that shows npm verb [10:49:06] it makes the log really huge and is causing crashes, and is not needed now. Please [10:51:17] 10Continuous-Integration-Infrastructure, 6Operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#2180095 (10hashar) We gave it a try a while ago. @Gehel proposed to have some kind of hacking sprint to revisit running the Puppet module tests easily. At first... [10:51:24] gehel: pasted some thoughts on https://phabricator.wikimedia.org/T78342 [10:52:23] 10Continuous-Integration-Infrastructure, 6Operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#2180096 (10hashar) A change that got abandoned which might be of interest https://gerrit.wikimedia.org/r/#/c/178810/ . That one is to run unit tests for the vari... [10:52:36] gehel: oh and an attempt https://gerrit.wikimedia.org/r/#/c/178810/ :-D [10:52:42] that one is to run spec against .rb files [10:56:24] hashar: How is bundle installed on trusty. Would bundle install work or not. Or do we have to specify a path [11:15:16] 10Continuous-Integration-Infrastructure, 6Operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#2180130 (10hashar) I have reopened/rebased/fixed https://gerrit.wikimedia.org/r/#/c/178810/ which introduce a very basic spec for wmflib.os_version(). Can be in... [11:15:56] gehel: I have rebased my lame unit test for wmflibs.os_version() https://gerrit.wikimedia.org/r/#/c/178810/ :-} [11:16:28] gehel: if you get to somehow sort out a set of requirements and introduce a basic runner to execute tests from each module, I think that will be a good hacking outcome [11:16:37] but you would need some ruby / rspec knowledge I guess [11:33:05] hashar: I think we will have enough Ruby / Bundler / rspec knowledge to get something working. Probably ugly, but working... [11:35:54] hashar: an exception on the special page load wouldn't work [11:36:10] As the class can be instantiated and the lua class doesn't exist [11:36:20] The other options would be exception on all page loafs [11:36:35] Or just seceretly don't load the extension / log when the lua stuff isn't therr [11:38:17] But with both of those there is no way / point in having them in the job again as with the exception on extension load, no jobs would be able to run, and if the extension just doesn't get loaded, then no structure tests would run on it and it may as well be removed [11:42:18] hashar: Scribuntu is added to the class through autoloader in the extension, i wrote a task because yurik ask me to and said he might get to it last week. I say remove it for now until the extension is fixed to not require scribuntu. [12:21:23] addshore: my idea is something like https://gerrit.wikimedia.org/r/#/c/281623/2/includes/PortalSpecialPage.php,cm [12:21:42] still load the extension but have the special page throw when Scribunto is missing [12:21:52] 10Continuous-Integration-Config, 10Wiki-Loves-Monuments-General, 13Patch-For-Review: Add PHP linting to labs/tools/heritage - https://phabricator.wikimedia.org/T131597#2180198 (10JeanFred) 5Open>3Resolved Merci @hashar :) I could not check this as there are no reviews opened right now but it should be f... [12:22:13] RECOVERY - Host integration-dev is UP: PING OK - Packet loss = 0%, RTA = 0.69 ms [12:22:38] 10Continuous-Integration-Config, 10Wiki-Loves-Monuments-General, 13Patch-For-Review: Add PHP linting to labs/tools/heritage - https://phabricator.wikimedia.org/T131597#2180201 (10hashar) You can comment in Gerrit `recheck` on a change that has been merged. That will run the tests and report back :-} [12:25:21] hashar: Hi, would you be able to remove npm verb from https://integration.wikimedia.org/ci/job/parsoidsvc-source-npm-node-4.3/21/console or the hack you did please. [12:26:48] 10Continuous-Integration-Config, 10Wiki-Loves-Monuments-General, 13Patch-For-Review: Add PHP linting to labs/tools/heritage - https://phabricator.wikimedia.org/T131597#2180202 (10JeanFred) Right! Tested on https://gerrit.wikimedia.org/r/#/c/281280/ − looks fine. The jsonlint is not executed though (it was n... [12:29:17] subbu: Hi do you mind if we not run jsduck on npm 0.10 but run it on 4.3. Because it currently will fail with an error on 0.10 if we use our workaround for 4.3 to run. [12:44:15] PROBLEM - Host integration-dev is DOWN: CRITICAL - Host Unreachable (10.68.17.81) [12:47:44] https://phabricator.wikimedia.org/D18 [12:47:51] * https://phabricator.wikimedia.org/D186 [12:48:02] this one fixes unit test failures too :) [12:49:19] 10Continuous-Integration-Config, 10Wiki-Loves-Monuments-General, 13Patch-For-Review: Add PHP linting to labs/tools/heritage - https://phabricator.wikimedia.org/T131597#2180259 (10hashar) The `jsonlint` job is only executed when a patch modify/introduce a .json file :-} [12:56:32] hashar: I'll take a look! [12:57:13] addshore: my point being to have the Special page to gracefully raise an exception instead of fataling out :-} [12:57:21] (I wish we could catch fatal errors really) [12:57:27] (and get a stacktrace on fatals... :D ) [12:59:40] Well, that won't work nicely though, as it extends the class in question [13:00:03] I guess you could have a big if lua loaded include class version 1, else include class version 2 which just shows an error [13:11:43] (03PS1) 10Rillke: make-wmf-branch: Branch UploadsLink extension [tools/release] - 10https://gerrit.wikimedia.org/r/281644 [13:13:25] (03PS2) 10Rillke: make-wmf-branch: Branch UploadsLink extension [tools/release] - 10https://gerrit.wikimedia.org/r/281644 (https://phabricator.wikimedia.org/T130018) [13:30:10] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2180360 (10Rillke) [13:31:34] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2180378 (10Rillke) [13:32:43] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2180360 (10Rillke) p:5Triage>3Normal [13:43:49] (03PS2) 10Paladox: Enable npm node 4.3 for parsoidsvc [integration/config] - 10https://gerrit.wikimedia.org/r/281384 [14:01:17] (03PS3) 10Hashar: [LifeWebCore] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281577 (owner: 10Paladox) [14:01:19] (03PS2) 10Hashar: [LanguageTag] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281562 (owner: 10Paladox) [14:01:21] (03PS4) 10Hashar: [LifeWeb] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281564 (owner: 10Paladox) [14:01:23] (03PS2) 10Hashar: [LightweightRDFa] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281582 (owner: 10Paladox) [14:01:25] (03PS2) 10Hashar: [LanguageSelector] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281558 (owner: 10Paladox) [14:02:18] (03PS1) 10Hashar: Merge in some npm/composer entry points [integration/config] - 10https://gerrit.wikimedia.org/r/281655 [14:03:54] hashar: Thanks :) [14:04:20] (03CR) 10Hashar: [C: 032] Merge in some npm/composer entry points [integration/config] - 10https://gerrit.wikimedia.org/r/281655 (owner: 10Hashar) [14:11:03] hashar: I got jsduck to run on 0.10 at https://gerrit.wikimedia.org/r/#/c/280906/ and bundle will now only install on npm node 4.3 and then bundle exec jsduck. [14:14:18] (03PS3) 10Paladox: Enable npm node 4.3 for parsoidsvc [integration/config] - 10https://gerrit.wikimedia.org/r/281384 [14:18:25] RECOVERY - Puppet run on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [0.0] [14:18:25] PROBLEM - Puppet run on integration-slave-trusty-1024 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [14:19:10] hashar: ^^ [14:33:41] Project browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce build #477: 04FAILURE in 5 min 40 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce/477/ [14:43:24] RECOVERY - Puppet run on integration-slave-trusty-1024 is OK: OK: Less than 1.00% above the threshold [0.0] [14:53:56] hashar: I'll join your weekly [14:54:13] hashar: if you don't mind having a silent participant... [14:59:37] CI weekly chat https://plus.google.com/hangouts/_/wikimedia.org/btest-ci-weekly [14:59:41] gehel: yeah come around [15:00:20] my webcam is dead anyway... [15:03:05] hashar: Hi [15:12:25] gehel: and another random rake/ruby change for puppet.git this time to expand vcl.erb templates and compile them with varnish vcl2c https://gerrit.wikimedia.org/r/#/c/276733/ :D [15:21:51] quick and short check in :-D [15:22:04] zeljkof: we should get this checkin later in the day maybe [15:22:07] so dan join [15:22:14] seems 8am PST is not that good [15:22:42] hashar: agreed [15:22:50] not sure if later would be too late for jzerebecki [15:22:52] anyway I am not attending next week [15:22:55] yeah [15:23:08] I think Jan is out this week [15:23:26] hashar: could you remove hack you did to https://integration.wikimedia.org/ci/job/parsoidsvc-source-npm-node-4.3/31/ to do npm verb please. [15:23:36] it makes a huge log and is unneded now. [15:26:21] paladox: ohhh I hacked it manually [15:26:31] hashar: Yep. [15:26:56] hashar: We can now merge it. I fixed the issue so now jsduck runs on npm 0.10 and runs bude + jsduck on npm 4.3. [15:26:58] :) [15:27:13] --verbose removed [15:28:03] hashar: Thanks [15:30:19] :) [15:35:31] thcipriani: hey, regarding our yesterday talks: https://phabricator.wikimedia.org/D186 and https://phabricator.wikimedia.org/D186 [15:35:34] * https://phabricator.wikimedia.org/D185 [15:35:53] 186 fixes unit test failures too :) [15:36:37] Amir1: awesome! I'll give these some review today :) [15:36:40] thanks for your help! [15:37:14] \o/ [15:37:17] thanks [15:40:54] thanks Amir1 ! :) [15:41:30] greg-g: you are welcome :) tell me if I can help with anything [15:45:15] Amir1: kudos on the ORES service puppet patch :-}}}}}}}}} [15:45:27] Amir1: I have read your reply on some related task. I am impressed [15:45:29] bravo! [15:45:46] thank you :) hashar [15:45:57] the service is live if you want to check [15:46:04] Amir1: on beta? [15:46:08] yup [15:46:13] ores-beta.wmflabs.org [15:46:35] (the web node is deployment-ores-web.deployment-prep.eqiad.wmflabs) [15:46:52] any clue how the source code get updated on that instance ? [15:47:11] thcipriani and I were talking about having a Jenkins job that would run after a patch got merged and run scap from the beta cluster deploy machine [15:47:12] using deploy from tin [15:47:34] yeah, I thought about that [15:47:36] so past merged --> trigger job beta-scap-deploy REPO=/srv/ores --> run scap -> success [15:47:45] exactly [15:47:57] twentyafterfour: yt? [15:49:12] hashar: It would be a dream come true to do that, so tell me if I need to do anything [15:49:22] (or guide me and I do the rest) [15:49:30] I already have a phab for that [15:49:39] and/or thcipriani ? [15:49:42] yeah there is most probably a phab task about it [15:49:58] ottomata: around. What's up? [15:50:13] two things, first uhhh, how do I make scap deploy happy with -e flag again? [15:50:17] i'm trying on beta [15:50:24] its getting my main env dsh file [15:50:27] I think the hardest parts would be: (a) figure out appropriate sudo rule to have 'jenkins-deploy' to run scap as service-deploy (b) list the needed command (c) and rollback maybe [15:50:58] should dsh_targets be relative again? [15:51:04] maybe that's my problem, its absolute in my scap.cfg [15:51:17] yeah, it should just be the name of the dsh_targets file [15:51:21] Amir1: do you remember a task to get scap deploy harnessed by a Jenkins job on beta? I can't find any :( [15:51:22] thcipriani: ^^ [15:51:49] * Amir1 is looking [15:51:50] ahh ya, ok that worked, i think it was absoulte due to some old bug [15:52:25] PROBLEM - Puppet run on deployment-tin is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [15:52:26] hashar: I'm not sure if we filed that task. We've talked about it. It'd be on the #scap3 board if it was filed... [15:52:48] and I don't see one there... [15:53:01] ok thcipriani other q is about the patch i'm working on [15:53:15] see this [15:53:16] https://gerrit.wikimedia.org/r/#/c/280730/19/hieradata/common.yaml [15:53:30] so, this is a list of scap sources to clone on the scap deploy server [15:53:37] thcipriani: filling it [15:53:49] hashar: thanks. [15:54:09] the keyholder:;agents are also now in hiera after twentyafterfour's work. but he's got them in role classes in prod, and a labs project class in labs [15:54:15] which duplicates the settings [15:54:20] but perhaps that's ok, since they are different in each [15:54:26] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup, 13Patch-For-Review: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2180360 (10greg) Added the security review as a blocker as that also blocks deployment to Beta Clu... [15:54:38] do we want to configure prod and beta servers with the same scap deploy sources? or shoudl that config be separate [15:54:39] subbu: Hi could you merge https://gerrit.wikimedia.org/r/#/c/280906/ please and review https://gerrit.wikimedia.org/r/#/c/281665/ please. [15:54:42] like the keyholder agents [15:55:02] it seems like it would be nice to not have to clone all repos on tin.eqiad in deployment-prep, since many of them will never be used in deployment-prep [15:55:04] I think it makes sense to split the hieradata as long as everything is using the same role. [15:55:06] but, for those that are on both [15:55:08] paladox, i'll take a look later today .. preparing for a meeting shortly. [15:55:10] the hiera configs will be the same [15:55:19] that's true. [15:55:21] subbu: Ok, thanks [15:55:55] ottomata: it is currently the case that the deployment yaml is duplicated for beta and prod for trebuchet. [15:56:12] right [15:56:27] hmm, both keyholder::agents and scap::sources use hiera_hash, so the hashes are merged [15:56:27] I don't think I see a problem with the duplication WRT to hieradata as long as the roles/classes/defines/etc are all the same between beta and prod. [15:56:43] we could put ones that should be in both, like thse eventlogging ones in common.yaml [15:56:49] but ones that are in either in their more specific hiera files [15:56:57] like, this dumps repo could go in role/deployment/server.yaml [15:57:17] since that will only be applied in prod [15:57:22] (at least until role hiera works in labs) [15:57:26] ah yeah, might need some comments to explain it, but it makes sense. [15:57:42] or, i mean, if you are ok with all repos in both places [15:57:46] we could just leave it in common,yaml [15:57:51] or some other more global place that works in both [15:58:01] not sure which is better [15:59:10] eh, I think the duplication is actually clearer in this instance. I have no problem with this kind of duplication: it's config and not logic. [15:59:24] ok, so leave it in common,yaml? [16:00:03] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Config, 3Scap3: Deploy beta cluster services automatically via scap3 - https://phabricator.wikimedia.org/T131857#2180756 (10hashar) [16:00:06] Amir1: thcipriani and here is the lame task https://phabricator.wikimedia.org/T131857 :-} [16:00:28] heading out for kids [16:00:33] thanks [16:01:06] ottomata: oh, I meant split it up for labs and prod like in https://gerrit.wikimedia.org/r/#/c/279198/ [16:01:32] duplicate the hieradata in some instances, but keep environments split. [16:02:15] oh ok cool [16:02:16] will do that then [16:02:18] and comment [16:03:42] cool. thanks. [16:03:53] thank you! [16:18:37] PROBLEM - Puppet run on mira is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [16:18:49] 7Browser-Tests: Browsertests has a lot of unstable builds - https://phabricator.wikimedia.org/T131822#2180809 (10greg) [16:18:51] 10Browser-Tests-Infrastructure, 6Release-Engineering-Team, 7Epic, 13Patch-For-Review, and 2 others: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2180810 (10greg) [16:22:53] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Config, 3Scap3, 6Revision-Scoring-As-A-Service: Deploy beta cluster services automatically via scap3 - https://phabricator.wikimedia.org/T131857#2180818 (10Ladsgroup) [16:28:11] legoktm: We could use this https://php-build.github.io/ for mutiple php. [16:32:13] Project selenium-PdfHandler » firefox,test,Linux,,contintLabsSlave && UbuntuTrusty build #15: 04FAILURE in 1 min 11 sec: https://integration.wikimedia.org/ci/job/selenium-PdfHandler/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,VERSION=,label=contintLabsSlave%20&&%20UbuntuTrusty/15/ [16:32:51] hi! I was running a jenkins job where I wanted a git push to happen from jenkins to gerrit - I had a test user that I gave the right permissions on gerrit - and used SSH-Agent option in the jenkins job config on the UI, and added the user. It does this: [16:32:58] https://www.irccloud.com/pastebin/IxN8YI4C/ [16:33:38] can someone help me figure out if it is actually getting the right ssh key - doing the same git push from my local seems to work [16:38:35] yurik: Hi you doin't mind if i re enable the workbord for https://phabricator.wikimedia.org/project/profile/475/ please since it is hard to track down the task im looking for. [16:39:10] yurik: Could you have a look at https://phabricator.wikimedia.org/T130958 please. [16:39:21] paladox, sure [16:39:35] yurik: Thanks. [16:40:17] paladox, as for the testing - i really don't know much about that - hashar is our guru i think :) [16:40:29] yurik: Ok [16:41:39] yurik: Since the extension requires scribuntu full because it has classes that use it, it is causing some patches to fail in mediawiki/core [16:42:09] yurik: I think hashar did it here https://gerrit.wikimedia.org/r/#/c/281623/ [16:42:44] it has a -2 from the build [16:43:05] ottomata: here now... [16:43:20] madhuvishy: hi! [16:43:28] marxarelli: Hi! [16:43:39] automated releases, huh? [16:43:44] yurik: Yep because wikibase fails but it fixes the problem with scribuntu. [16:43:46] what is this wizardry? [16:43:47] :) [16:44:35] madhuvishy: that is a mighty long console log [16:44:37] * marxarelli parses [16:44:37] marxarelli: yep! [16:45:04] * twentyafterfour read backscroll [16:45:04] it _looks_ like host verification that failed, not the auth key [16:45:07] marxarelli: the part i pasted is so tiny. The whole log takes a couple minutes to load [16:45:27] yeah, i thought my browser had frozen up for a second [16:46:15] madhuvishy: you said it's trying to push to gerrit? [16:46:23] marxarelli: yeah [16:46:33] [INFO] Executing: /bin/sh -c cd /mnt/jenkins-workspace/workspace/analytics-release-test && git push ssh://testaccount123@gerrit.wikimedia.org:29418/analytics/refinery/source refs/heads/release:refs/heads/release [16:46:47] this is the bit that seems to fail [16:46:57] right [16:47:06] i have the ssh keys on my local machine, and tried the git push - and it seems fine [16:47:14] so on the gerrit end it looks alright [16:47:40] madhuvishy: ah, the host key isn't trusted [16:47:44] ooh [16:47:49] what does that mean [16:48:08] i.e. if i manually try to ssh from that instance, i get a prompt for trusting the rsa key [16:48:16] ha [16:48:42] madhuvishy: https://phabricator.wikimedia.org/P2857 [16:49:12] for that to work, we'll need to add some configuration to the slave config in ops/puppet [16:49:22] marxarelli: ahh [16:49:32] i can do that if you point me to where [16:49:44] yurik: It passes now. Just that it depend on a patch to test weather it fixed the problem which it did but found that patch failed which is good because that means it works now. [16:49:59] the idea is to get a "release" user of some sort and have it do this - I am trying to test it with a test user now [16:50:08] 10Continuous-Integration-Infrastructure, 5Gerrit-Migration, 3releng-201516-q4, 6Operations, and 4 others: Make sure phab can talk to gearman and nodepool instances can talk to phabricator - https://phabricator.wikimedia.org/T131375#2180904 (10mmodell) New problem: Apparently jenkins can't access phabricato... [16:50:21] yurik: that task is about a failure related to zeroportal [16:50:24] madhuvishy: https://gerrit.wikimedia.org/r/#/admin/projects/operations/puppet [16:51:01] madhuvishy: manifests/role/ci.pp [16:51:18] (as a starting point. not sure if that's the exact right place to add it) [16:51:30] madhuvishy: feel free to put me on review (dduvall) [16:51:41] yurik: also, please don't pawn off things "related to testing" to other people, they're your tests. [16:51:43] paladox, why do we have depends on in the comment? [16:52:19] marxarelli: i think that got moved. https://github.com/wikimedia/operations-puppet/tree/production/modules/role/manifests/ci now [16:52:22] yurik: because hashar was checking if it fixed the patch since using that he could test something [16:52:49] paladox, so i should remove the depends on comment and commit? or are we still waiting for the core patch to merge? [16:53:10] yurik: Yes you can remove Depends-On please. [16:53:16] I tested and it passed. [16:53:31] marxarelli: it looks like the jenkins-slave user has access? if it's a gerrit user may be i can test with it [16:53:38] madhuvishy: ah! that's what i get for having a stale clone sitting around :) [16:54:16] madhuvishy: it's the host key that we're concerned with in this particular case, not the user's auth key [16:54:30] i.e. we want the slaves to trust the host key for gerrit.wikimedia.org [16:55:06] marxarelli: interesting - it doesn't already because it has never tried to push? [16:55:57] rather, we never had a job try to do that - so we didn't have to have config for this [16:55:59] madhuvishy: not directly to gerrit, no. i actually forget exactly how zuul merger does its thing, and to where it pushes [16:56:16] madhuvishy: right [16:56:57] 5Gerrit-Migration, 10Differential, 10Phabricator, 13Patch-For-Review: Allow liberal close in Differential - https://phabricator.wikimedia.org/T131623#2180943 (10Luke081515) p:5Triage>3Low [16:57:00] 5Gerrit-Migration, 10Differential, 10Phabricator, 13Patch-For-Review: Enable differential.allow-self-accept in phabricator - https://phabricator.wikimedia.org/T131622#2180944 (10Luke081515) p:5Triage>3Low [16:57:18] marxarelli: cool. also re: moving puppet modules - they are being moved as part of https://phabricator.wikimedia.org/T119042 [16:57:43] madhuvishy: oh, nice. thanks for the info. i'm glad that's happening [17:11:33] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup, 13Patch-For-Review: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2180991 (10Rillke) @greg Anything left that I could do except waiting? [17:40:08] marxarelli: i'm not sure I know how to have puppet trust a host - is there an example? [17:40:35] madhuvishy: ah, i was just responding to the email from antoine about it [17:41:10] tl;dr: it's very automatic for production hosts using exported `sshkey` resources, but this won't work in labs [17:41:31] aah [17:42:03] we may be able to simply declare an `sshkey` resource in ci::slave, using gerrit.wikimedia.org's host key [17:42:13] not sure though. i wanted to get antoine's opinion [17:42:45] if declaring an `sshkey` resource is the right way to go, you should be able to get the key value using ssh-keyscan [17:44:46] marxarelli: oh - okay looking it up [17:45:36] madhuvishy: e.g. `ssh-keyscan -p 29418 -t rsa gerrit.wikimedia.org` [17:45:55] marxarelli: right - got it [18:11:04] marxarelli: what's your gerrit username? [18:11:13] madhuvishy: dduvall [18:22:42] marxarelli: cool - added you, and i see you added hashar as well - jenkins finally likes the patch [18:29:01] madhuvishy: great. we'll need ops review for your patch as well (we don't have +2 on ops/puppet) [18:29:31] madhuvishy: also, i think you need to drop the type from the beginning of `key =>` [18:29:50] marxarelli: aah [18:30:14] i can cherry pick the patch on integration-puppetmaster to verify that it works though [18:30:38] makes sense [18:30:42] that'd be cool [18:30:44] let me fix [18:30:45] madhuvishy: oh, and you're missing a space between `{'gerrit'` [18:30:53] i'll poke someone on ops too. [18:30:55] right [18:30:55] surprised puppet lint didn't catch that [18:30:57] fixing [18:30:59] yeah [18:33:07] madhuvishy: i'm running the mw deploy train starting in 30 min but thcipriani or twentyafterfour might be able to help you with the cherry-pick/testing [18:33:25] marxarelli: cool, thanks! [18:33:30] np! [18:39:02] 6Release-Engineering-Team, 6Analytics-Kanban: [Spike] Figure out how to automate releases with jenkins {hawk} - https://phabricator.wikimedia.org/T130576#2181304 (10madhuvishy) I made some progress yesterday. Trying to recollect the bumps I hit, and how they were solved here: 1. The build job was failing - wa... [18:47:10] legoktm: Looks like composer is now stable https://github.com/composer/composer/releases [18:52:18] WOAH [18:52:30] paladox: that's awesome [18:52:41] yep [18:53:07] legoktm: Should we update composer in jenkins to the stable one. [18:53:24] Yes [18:55:25] legoktm: Ok, ive got a patch here https://gerrit.wikimedia.org/r/#/c/270548/ that i will amend to 1.0.0. Plus to can run on php 5.3 since we will use symfony/* 2.x instead of 3.x [18:59:54] hashar: Composer is now stable. Its at 1.0.0 now. [19:02:39] paladox: ah good to know! [19:02:47] (03PS9) 10Paladox: Update composer to dev-master [integration/composer] - 10https://gerrit.wikimedia.org/r/270548 (https://phabricator.wikimedia.org/T125343) [19:02:56] hashar legoktm ^^ [19:03:12] maybe the composer package will get updated https://packages.qa.debian.org/c/composer.html :D [19:05:09] (03PS10) 10Paladox: Update composer to 1.0.0 stable [integration/composer] - 10https://gerrit.wikimedia.org/r/270548 (https://phabricator.wikimedia.org/T125343) [19:05:22] hashar: Oh, do we use the package from there. [19:07:21] legoktm: Ive uploaded the patch using composer 1.0.0 [19:09:03] awesome [19:12:08] legoktm: Ok, i also made sure it uses the package are compatible in php 5.3. [19:12:12] :) [19:13:41] 6Release-Engineering-Team, 5Release: MW-1.27.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T130945#2181438 (10cscott) [19:15:56] RECOVERY - Puppet run on deployment-eventlogging04 is OK: OK: Less than 1.00% above the threshold [0.0] [19:30:44] twentyafterfour: another q (i ping you before t y l e r because we are actively working on this thang) [19:30:45] yt? [19:54:17] ottomata: I'm here [19:55:33] twentyafterfour: i was going to ask if hiera_hash was needed [19:55:52] or if we could make things like keyholder::agent into a class param instead [19:56:03] (03PS2) 10Hashar: [Interlanguage] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281546 (owner: 10Paladox) [19:56:05] (03PS2) 10Hashar: [InterwikiMagic] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281549 (owner: 10Paladox) [19:56:07] (03PS2) 10Hashar: [InlineCategorizer] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/280779 (owner: 10Paladox) [19:56:09] (03PS2) 10Hashar: [JSBreadCrumbs] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281554 (owner: 10Paladox) [19:56:49] marxarelli: I'm around - let me know when you have time! [19:58:04] madhuvishy: k. will do! [19:58:21] ottomata: class param? [19:58:30] ja twentyafterfour [19:58:31] like [19:58:43] ottomata: the reason I used hiera is so that it can be easily overridden for labs projects [19:58:44] scap::server($keyholder_agents, $scap_sources) [19:58:51] ja it would still use hiera [19:58:53] but not hiera_hash [19:58:56] hiera_hash is special [19:59:03] in that it merges hashes [19:59:06] from different scopes [19:59:09] (03CR) 10Hashar: [C: 032] [Interlanguage] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281546 (owner: 10Paladox) [19:59:13] (03CR) 10Hashar: [C: 032] [InterwikiMagic] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281549 (owner: 10Paladox) [19:59:15] (03CR) 10Hashar: [C: 032] [InlineCategorizer] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/280779 (owner: 10Paladox) [19:59:17] (03CR) 10Hashar: [C: 032] [JSBreadCrumbs] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281554 (owner: 10Paladox) [19:59:25] wasn't sure if you did that on purpose [19:59:33] like [19:59:36] for keyholder::agents [19:59:46] it would be possible to not have to specify everything again in labs [19:59:48] with hiera_hash [19:59:49] IF [19:59:58] the role/common/deployment/server.yaml was readable in labs [20:00:03] role yamls are not [20:00:11] say if insetad of in role/common/... [20:00:17] the keyholder::agents were in just common.yaml [20:00:19] ottomata: I have very little experience with hiera [20:00:21] (03Merged) 10jenkins-bot: [InlineCategorizer] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/280779 (owner: 10Paladox) [20:00:23] (03Merged) 10jenkins-bot: [Interlanguage] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281546 (owner: 10Paladox) [20:00:26] (03Merged) 10jenkins-bot: [InterwikiMagic] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281549 (owner: 10Paladox) [20:00:29] (03Merged) 10jenkins-bot: [JSBreadCrumbs] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281554 (owner: 10Paladox) [20:00:34] then in labs, you might only need to change the trusted_groups [20:00:35] or the key [20:00:35] or something [20:00:38] ottomata: for what it is worth I added support for labs in /utils/hiera_lookup [20:00:38] but not have to decalre everything [20:00:44] though there might be a patch still pending in puppet.git [20:00:44] since the hashes woudl b merged [20:00:49] hiera_lookup? [20:00:58] role based lookup in labs? [20:01:02] hashar: ? [20:01:09] in puppet.git there is /utils/hiera_lookup [20:01:14] 10scap, 10Analytics-EventLogging, 6Analytics-Kanban, 13Patch-For-Review, 10Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#2181677 (10Nuria) [20:01:17] which let you look up for a hiera key based on a fqdn [20:01:23] I have made it to learn about wmflabs [20:01:51] oh [20:01:53] yes [20:01:53] so you can lookup a key with something like: hiera_lookup --fqdn instance.project.eqiad.wmflabs foo_key [20:02:00] ok cool, ja [20:02:18] ottomata: the pending patch is https://gerrit.wikimedia.org/r/#/c/276346/ [20:02:22] HMMm, ok twentyafterfour if you didn't do hiera_hash on purpose, i'm going to experiment with using class params and classpath lookup intead [20:02:24] $ ./hiera_lookup --fqdn=host.tools.eqiad.wmflabs classes [20:02:25] role::aptly::client [20:02:26] magic [20:03:10] ah, nice hasha [20:03:24] ottomata: sure, I'm not attached to any particular implementation, I just want it to be flexible so that labs projects don't need to merge into operations/puppet in order to configure a deployment project for testing scap [20:04:16] hashar: wanna land https://phabricator.wikimedia.org/D162 ? realized this fixes doc generation :) [20:04:34] ja for sure, it'd still be hiera, it just wouldn't merge the scopes together [20:04:58] thcipriani: oh I had a left behind comment there [20:05:14] thcipriani: I still havent figured out how to get arcanist to push to staging area so Jenkins does not test anything [20:05:27] thcipriani: maybe you want to refresh the patch / resend it and get the Jenkins jobs to run [20:05:45] sure, I can do that. [20:06:08] then ideally verify the py script in the .deb package have the proper shebang [20:06:30] ja twentyafterfour if this works it'll be better, because then the hiera declaration in wikitech would be [20:06:35] scap::server::agents: ... [20:06:39] scap::server::sources: ... [20:06:43] something like that [20:06:53] ok [20:07:50] thcipriani: I havent even looked at why we have a quilt patch to change the shebang. In theory dh_python2 should take care of it [20:08:48] hashar: I don't have time to fixup those broken commits, Please revert or figure out the correct javascript with paladox. [20:09:47] oh, didn't realize dh_python2 would do that—that's probably why that patch is there. [20:10:35] Krinkle, what commit. [20:10:47] dh_* === magic (aka, it's a total mystery to most people) [20:10:58] paladox: https://gerrit.wikimedia.org/r/281550 and - https://gerrit.wikimedia.org/r/280777 [20:11:05] Oh thanks [20:11:06] paladox: Have you written javascript applications in the past? [20:11:25] Krinkle not really. I just follow what jshint fails with. [20:11:48] paladox: The purpose is not to write fake code that happens to pass JSHint. [20:11:59] 10Continuous-Integration-Infrastructure, 5Gerrit-Migration, 3releng-201516-q4, 6Operations, and 4 others: Make sure phab can talk to gearman and nodepool instances can talk to phabricator - https://phabricator.wikimedia.org/T131375#2181736 (10hashar) Jenkins execute the jobs on labs instances, so it is not... [20:12:01] In that case, removing all files would be the easiest solution. [20:12:19] (and also the reason why I fear an AI superintelligence tasked with fixing the earth) [20:12:35] RECOVERY - Puppet run on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:12:45] paladox: jQuery(document).ready() takes a callback that is scheduled for later when the document is ready [20:12:56] It should be given a function. Not the result of invoking a function. [20:12:56] Krinkle: yeah I guess I should stop reviewing Js ... :( [20:13:19] hashar: I aborted one. It seems I can't revert the other because the job is now enforced voting. [20:13:22] * Krinkle goes back to work. [20:13:28] twentyafterfour: looks like to update D162 I have to commandeer the patch...that doesn't seem right? [20:14:30] (03PS1) 10Hashar: Revert "[JSBreadCrumbs] Add npm test and composer-test" [integration/config] - 10https://gerrit.wikimedia.org/r/281736 [20:14:37] (03CR) 10Hashar: [C: 032] Revert "[JSBreadCrumbs] Add npm test and composer-test" [integration/config] - 10https://gerrit.wikimedia.org/r/281736 (owner: 10Hashar) [20:15:30] (03Merged) 10jenkins-bot: Revert "[JSBreadCrumbs] Add npm test and composer-test" [integration/config] - 10https://gerrit.wikimedia.org/r/281736 (owner: 10Hashar) [20:15:42] * thcipriani struggles to spell "commandeer" well enough for google to guess what I'm talking about [20:15:50] steal [20:16:57] hashar: was curious about your opinion on https://gerrit.wikimedia.org/r/#/c/281706/ if you have a chance [20:17:18] oh [20:17:43] i recommended to madhuvishy that we might just be able to add a sshkey resource but i wasn't sure exactly how we manage host keys in labs [20:17:50] marxarelli: no clue how wrong is sshkey {} :D [20:18:12] it's used for prod :) but resource exporting doesn't work across puppet masters [20:18:14] so on prod, the resources are collected and written down to /etc/ssh/known_hosts or something [20:18:19] but on labs, resources are not collected [20:18:25] yep, exactly [20:18:26] cause the puppet master is common to all labs project [20:18:28] then [20:18:38] on CI we have our own puppet master [20:18:40] so in theory we could collect [20:18:54] and in this case, we want to auth gerrit.wikimedia.org's host key [20:19:13] which isn't part of the labs infra [20:19:29] you can filter resource when you realize them in puppet [20:20:22] the nice trick is [20:20:38] GIT_SSH="ssh -o StrictHostKeyChecking=no" ;-D [20:20:41] Ssshkey <<| host == 'hostname' |>> [20:20:55] yeah [20:21:04] hashar: ah, i didn't want to go that route :) [20:21:08] I have no clue how puppet collection and the diamond operator works though :( [20:21:17] it wouldn't work here anyway [20:21:27] since it's across prod and labs [20:21:52] thcipriani: that's right, you have to commandeer apparently. I made a tiny patch to arcanist to remove that requirement but upstream didn't accept it [20:22:18] :( [20:22:32] marxarelli: so the use case is a Jenkins job on a labs instance to git push to ssh://gerrit.wikimedia.org right ? [20:22:41] right [20:22:42] twentyafterfour: what's the deal with the staging area? Why can't some folks push to that? [20:22:55] hashar: do you think that's the wrong approach? [20:23:40] marxarelli: the Ssshkey <<| host == 'ytterbium' |>> would be nice [20:24:00] marxarelli: but then sshkey{} might just be good enough [20:24:10] hashar: would be, but we can't do it afaik [20:24:22] but I have no idea what it will generate on the host. But the puppet compiler might help [20:25:18] hashar: we could `generate('ssh-keyscan ...')` but that's as good/reckless as disabling strict host checking :) [20:25:38] marxarelli: https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/ fill in change "281706" and host: gallium.wikimedia.org [20:25:57] marxarelli: the sshkey {} has been added in the role::ci::slave which his applied on gallium (but it is NOT used for labs instance :D ) [20:26:22] marxarelli: but having the sshkey in the wrong class let us use the puppet compiler \O/ [20:27:54] hashar: but do they share the same puppet master? [20:28:26] exported resources are stored in the puppetmaster db as so exporting/collecting only works across hosts that are provisioned using the same master [20:32:01] marxarelli: pasted thoughts on https://gerrit.wikimedia.org/r/#/c/281706/4 [20:32:16] marxarelli: yeah I am not sure how the puppet compiler works [20:32:38] marxarelli: so maybe the easiest is to move the sshkey to the role::ci::labs::slave role , cherry pick, run puppet and check what happens [20:32:54] if all happy +1 the patch get ops to merge it in and {done} ? :-} [20:33:14] tis a nice idea really [20:33:17] cool, thanks for looking at it, hashar. madhuvishy and i will test it as soon as i'm done with the train [20:33:25] sounds good [20:33:43] I paired a bit with madhuvishy yesterday (my evening) [20:34:08] can't remember what she handed solved but I have barely done anything beside assisting in reading the weird error message [20:34:16] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup, 13Patch-For-Review: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2181854 (10greg) @Rillke Get the security review on the Security Team's radar and ask them to put... [20:34:19] she received the CI barnstar as a result :D [20:34:25] hashar: :D [20:34:28] \O/ [20:34:38] kudos on the sshkey {} puppet thing [20:34:45] looks about right and straightforward [20:35:00] there is some better way to handle this but that is going to be way to complicated :-/ [20:35:08] coool, moving it to right puppet role now [20:35:14] I have left comment on https://gerrit.wikimedia.org/r/#/c/281706/ [20:35:23] madhuvishy: you want to run the puppet compiler first [20:35:28] well at least give it a try ;-) [20:35:29] yup doing that [20:35:31] Thanks [20:35:38] Krinkle: sorry :-( [20:36:00] Krinkle: I really havent been careful and not knowing javascript that looked legit :-/ [20:36:11] https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/2312/console [20:36:41] madhuvishy: that job is a bit of magic. Got written by _joe_ from the ops team. It let you simulate what would happen when applying a puppet patch on a set of host [20:36:43] it is very handy [20:37:16] then in this case it is not very helpful [20:37:28] https://puppet-compiler.wmflabs.org/2312/gallium.wikimedia.org/ shows: [20:37:29] Added and removed resources: +1 / 0 [20:37:52] cause it does not support collection anyway [20:38:01] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-UploadsLink, 10Wikimedia-Extension-setup, 13Patch-For-Review: Set up UploadsLink extension on the beta cluster - https://phabricator.wikimedia.org/T131844#2181882 (10greg) Ah, @Legoktm pointed out it is on their list for this quarter already: https://ww... [20:38:32] so yeah move the sshkey to the proper class role::ci::slave::labs [20:38:42] then Dan can cherry pick / run puppet and see what happens [20:38:45] hashar: ya it's pretty cool - but you're right doesn't show actual diff in this cas [20:38:46] case [20:38:48] yup [20:38:49] that looks on track ;-) [20:43:02] Krinkle, would instead of doing what i did in https://gerrit.wikimedia.org/r/#/c/281550/3/js/BreadCrumbs.js i should just add $ and mw to global so i doint break any js. [20:43:06] Would that be ok [20:43:54] paladox: I don't have a simple answer. You'd have to know a little more about JavaScript. [20:44:15] These are quite complex applications you pass by. Each has their own behaviour and things. I don't know them all either. [20:44:36] In general, I'd file these as tech debt issues on Phab for maintainers or interested contributors of that extension to pick up. [20:44:49] Krinkle oh ok, i am only going to edit simple js files or just add npm that test jsonlint and banana checker. [20:44:54] And yeah, as a first step you could whitelist the status quo as an exception in jshintrc [20:45:42] Krinkle ok, thanks for replying and thanks for finding that js error. [20:49:17] hashar: patch updated [20:52:29] Krinkle: and hashar https://gerrit.wikimedia.org/r/#/c/281744/ [20:59:44] madhuvishy: great, I will let marxarelli follow up ;-} [21:10:37] hashar: Could you c+2 https://gerrit.wikimedia.org/r/#/c/280777/ please again. I addressed krinkles comment. [21:11:39] paladox: I think I will hold on C+2 javascript changes. I am obviously js illiterate :( [21:12:24] hashar: Ok, i will only do the jsonlint and banana checker changes now. [21:13:18] I am rushing to bed [21:13:23] tis past time! ;-) [21:13:30] hashar: Ok [21:15:42] (03PS1) 10Paladox: [JSBreadCrumbs] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281813 [21:17:24] RECOVERY - Puppet run on deployment-tin is OK: OK: Less than 1.00% above the threshold [0.0] [21:17:50] (03CR) 10Hashar: "Thanks to have looked at upstream code. So it looks like the Sniff in their PEAR standard is enforcing a lot of rules and some we dont wan" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/279615 (https://phabricator.wikimedia.org/T116561) (owner: 10Aashaka) [21:23:14] (03PS1) 10Paladox: Update 2 packages in composer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 [21:23:26] PROBLEM - Puppet run on deployment-tin is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [21:24:13] 6Release-Engineering-Team, 15User-greg: schedule mid-Q4 check-ins - https://phabricator.wikimedia.org/T131891#2182032 (10greg) [21:25:16] (03CR) 10Legoktm: [C: 04-1] "Can you split this into two separate changes please? For the upstream codesniffer upgrade, we also need to revert 0dc23d13bcd54007484438a2" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 (owner: 10Paladox) [21:26:07] legoktm: Do you want me to revert https://gerrit.wikimedia.org/r/#/c/271216/ [21:27:07] (03PS1) 10Paladox: Update phpunit to 4.8.24 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281817 [21:27:22] paladox: Yes, but in the same change as the one that upgrades to the new codesniffer version [21:27:37] legoktm: Ok i will do that now, thanks for replying. [21:27:38] paladox: just the change to rulset.xml though, the test case should stay the same [21:27:46] legoktm: Ok [21:28:39] RECOVERY - Puppet run on mira is OK: OK: Less than 1.00% above the threshold [0.0] [21:30:04] (03PS2) 10Paladox: Update squizlabs/php_codesniffer to 2.6.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 [21:30:15] llegoktm: Ok done ^^ [21:30:24] legoktm: ^^ [21:30:51] (03CR) 10Paladox: "Partially reverted in https://gerrit.wikimedia.org/r/#/c/281816/" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/271216 (https://phabricator.wikimedia.org/T127163) (owner: 10Legoktm) [21:31:45] legoktm It fails with https://integration.wikimedia.org/ci/job/composer-package-hhvm/893/console which would fail since it is a test. [21:32:25] (03PS3) 10Paladox: Update 2 packages in composer [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 [21:32:39] (03PS4) 10Paladox: Update squizlabs/php_codesniffer to 2.6.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 [21:33:18] paladox: I think that's due to an upstream change [21:33:26] legoktm: Oh [21:33:45] we'll have to take a closer look [21:34:28] (03PS5) 10Paladox: Update squizlabs/php_codesniffer to 2.6.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 [21:34:32] legoktm ok [21:35:01] PROBLEM - Puppet run on deployment-salt is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:35:59] PROBLEM - Puppet run on deployment-memc04 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [21:36:17] legoktm: WOuld it be because of Config vars set in ruleset.xml files are now processed earlier, allowing them to be used during sniff registration [21:36:49] Or [21:36:49] Generic ScopeIndent sniff now reports errors for every line that needs the indent changed (request #903) ◦Previously, it ignored lines that were indented correctly in the context of their block [21:36:50] ◦This change produces more technically accurate error messages, but is much more verbose [21:37:53] might be that one? [21:38:00] I don't have time right now to look [21:38:21] legoktm: Ok, should i add the test files to ignored phpcs. [21:38:29] no [21:40:13] legoktm ok [21:42:08] legoktm: Yep its defintly that one it says this https://github.com/squizlabs/PHP_CodeSniffer/issues/903#issuecomment-190437449 [21:42:52] PROBLEM - Puppet run on deployment-mediawiki01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [21:43:01] ok [21:43:18] the errors are auto-fixable which looks good [21:44:26] legoktm: Oh im not sure how to that. [21:45:50] Would you be able to do the aut_fixable please when you have time. [21:45:55] legoktm ^^ [21:46:09] Yeah [21:46:28] legoktm: Thanks [21:46:29] :) [21:46:30] I'll look in a bit [21:49:11] madhuvishy: ready to test your ops/puppet patch? [21:49:13] Ok thanks [21:49:30] marxarelli: Yup! [21:52:19] madhuvishy: i've just cherry picked it and forcing a puppet apply on integration-slave-trusty-1014 [21:53:09] !log Cherry picked https://gerrit.wikimedia.org/r/#/c/281706/ on integration-puppetmaster and applying on integration-slave-trusty-1014 [21:53:09] 6Release-Engineering-Team, 5Release: MW-1.27.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T130945#2182144 (10cscott) [21:53:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:53:21] marxarelli: cool - my last job ran in integration-slave-trusty-1015 - can i tell it to run in 1015? [21:53:28] uhh [21:53:29] 1014? [21:54:04] madhuvishy: oh, i'm looking at https://integration.wikimedia.org/ci/job/analytics-release-test/25/consoleFull [21:54:17] it will eventually apply across all slaves [21:54:25] ah yes [21:54:30] "Warning: Error 400 on SERVER: Cannot allocate memory - fork(2)" [21:54:32] so that's fun [21:54:37] i think 24 ran in 1015 [21:54:39] huh [21:55:03] that.. doesn't even look related? [21:56:31] probably not directly related. just puppetmaster being a pig [21:56:54] 1G resident memory ... [21:58:35] (03CR) 10jenkins-bot: [V: 04-1] Update squizlabs/php_codesniffer to 2.6.0 [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/281816 (owner: 10Paladox) [21:58:58] !log Restarting puppetmaster on integration-puppetmaster [21:59:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:59:29] 43M, that's more like it [22:01:43] oh geez, ruby 1.8.7 even [22:01:56] anyway, on with the cherry-pick/apply [22:03:27] RECOVERY - Puppet run on deployment-tin is OK: OK: Less than 1.00% above the threshold [0.0] [22:08:03] marxarelli: did it go alright? [22:10:04] RECOVERY - Puppet run on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0] [22:10:32] madhuvishy: it doesn't seem to have worked :/ [22:11:00] RECOVERY - Puppet run on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0] [22:12:49] marxarelli: as in - didn't get added to known hosts? [22:13:36] madhuvishy: correct. i don't even see that the resource was applied by puppet on the instance [22:13:45] * marxarelli greps some more [22:15:27] madhuvishy: oh wait. false alarm. it's there! [22:16:13] oh [22:16:14] cool! [22:17:02] marxarelli: I launched a release job - which launched on 1015 though [22:17:54] madhuvishy: darn. it's not working though `ssh -p 29418 gerrit.wikimedia.org` => "RSA host key for IP address '208.80.154.81' not in list of known hosts" [22:18:02] hrm [22:18:04] interesting [22:18:07] hmm [22:18:30] "RhostsRSAAuthentication no" in sshd_config probably has something to do with that ... [22:20:35] oh wait, that shouldn't matter. this is not a server-side issue [22:21:44] what is the entry in known_hosts? [22:22:27] marxarelli: the entry in my laptop for gerrit is [gerrit.wikimedia.org]:29418,[208.80.154.81]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQCF8pwFLehzCXhbF1jfHWtd9d1LFq2NirplEBQYs7AOrGwQ/6ZZI0gvZFYiEiaw1o+F1CMfoHdny1VfWOJF3mJ1y9QMKAacc8/Z3tG39jBKRQCuxmYLO1SWymv7/Uvx9WQlkNRoTdTTa9OJFy6UqvLQEXKYaokfMIUHZ+oVFf1CgQ== [22:22:33] with the ip [22:22:34] https://phabricator.wikimedia.org/P2862 [22:22:44] right [22:22:57] RECOVERY - Puppet run on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [22:23:38] madhuvishy: maybe try changing the sshkey declaration to specify the ip under `host_aliases` [22:23:40] * madhuvishy wonders if it should be added to host_aliases [22:23:42] ha [22:23:43] yes [22:23:50] jinx! :) [22:24:04] :D [22:24:04] * marxarelli is four years old [22:25:43] madhuvishy: ah, we might also need the port in there, also under aliases [22:28:07] madhuvishy: ugh. there is also a permissions issue on /etc/ssh/ssh_known_hosts (it's 0400) [22:28:12] https://projects.puppetlabs.com/issues/21811 [22:28:21] might just have to make it a `file` resource [22:28:42] that's annoying [22:28:47] hmmmm yeah [22:29:47] so maybe this is why we don't use sshkey anywhere ... [22:29:54] yup [22:30:01] hmmm [22:30:13] may be should ask on ops [22:30:29] yeah, good call [22:43:13] (03PS1) 10Paladox: [LinkFilter] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281830 [22:48:49] (03PS1) 10Paladox: [LinkSuggest] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281832 [22:58:48] (03PS1) 10Paladox: [ListSignup] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281835 [23:06:18] (03PS1) 10Paladox: [LiveTranslate] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281837 [23:11:54] (03PS1) 10Paladox: [Lockdown] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281841 [23:21:52] (03PS1) 10Paladox: [LockDownEnglishPages] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281845 [23:22:10] (03CR) 10Paladox: "check experimental" [integration/config] - 10https://gerrit.wikimedia.org/r/281845 (owner: 10Paladox) [23:26:28] (03PS1) 10Paladox: [LogEntry] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281850 [23:31:19] (03PS1) 10Paladox: [LogoFunctions] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281853 [23:33:42] madhuvishy: job's running i see [23:33:44] marxarelli: it launched on 1015 again - does that slave have the change? [23:33:57] greg-g: Got something for you at T131568 [23:33:57] T131568: Change of workboard columns in #Phabricator - https://phabricator.wikimedia.org/T131568 [23:34:24] madhuvishy: crap. it does not. the race is on :) [23:34:31] ha ha [23:35:07] (03PS1) 10Paladox: [LookupUser] Add npm test and composer-test [integration/config] - 10https://gerrit.wikimedia.org/r/281855 [23:35:11] i've run it 27 times :D will run again if it fails (it will fail anyway - just one step further) [23:35:35] madhuvishy: k. applied! [23:35:40] ooh [23:35:43] it might work then [23:36:10] ha [23:36:16] marxarelli: different error [23:36:37] madhuvishy: progress! [23:36:43] Yes! [23:36:44] yay [23:37:11] the commit is made by jenkins-deploy user may be [23:37:27] https://www.irccloud.com/pastebin/RY1yuRJQ/ [23:37:30] yeah, using the default git user.mail and user.name settings [23:37:42] i think i can change that [23:37:44] er, user.email [23:37:51] not sure, looking [23:39:52] madhuvishy: was thinking you could use `git config user.email`, etc. but you might also have env variables as an option [23:39:57] https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables [23:40:01] Luke081515: did you mean "needs triage" or "needs discussion" ? [23:40:17] GIT_AUTHOR_NAME and GIT_AUTHOR_EMAIL [23:40:29] marxarelli: i think i can change it in SCM Additional behaviours [23:40:37] madhuvishy: ah, ok [23:40:40] and specify user.name and user.email there [23:42:03] greg-g: needs triage, the default column [23:42:05] madhuvishy: great. at some point we'll want this defined in integration/config/jjb i think, but we can cross that bridge when we come to it [23:42:43] marxarelli: yes! there is also the part where I have to patch jjb to be able to configure the release plugin (not currently supported) hashar gave me some pointers there [23:42:48] Luke081515: then I disagree :) "Backlog" and "To Triage" are very different things [23:43:27] and if we get rid of "needs discussion" some of those would go into "backlog" while others (less clear reasoning) would go into "to triage" [23:43:40] madhuvishy: ah ok [23:43:41] greg-g: Yeah, but I guess 95% of the tasks there are backlog tasks when they arrived, and no one moves them to backlog. I can see this, when I'm loading the labs workboard. [23:44:33] madhuvishy: well, looks like your ops/puppet patch is good to go at least. i'll +1 it [23:44:47] marxarelli: went further this run - tag already exists - that's fixable :) [23:46:26] Luke081515: that's fair, luckily there aren't nearly as many tasks in #phabricator as there are in #labs (<50 vs 709) [23:46:45] Luke081515: so, I think it's manageable for me, and I will do it (as effectively PM for Phabricator) [23:47:22] hm, ok [23:47:39] * Luke081515 just hates big workboards, where 99% of the tasks are in the default column [23:48:03] marxarelli: cool! thanks for all your help today :) I'll poke you when I get stuck next - which is bound to be soon ;) [23:48:05] I don't think it'll be 99%, there's a lot of administration tasks [23:48:27] and then of course the #phabricator-upstream milestone/sub-project [23:48:47] I mean that generally, not specific to #phabricator ;) [23:48:54] madhuvishy: and so it goes. :) no problem! [23:49:16] Luke081515: totally [23:49:18] :) [23:49:23] and the second thing which could be annoying is, if you upload a patch, and must move a task at five workboards ;) [23:49:49] which is why I dislike the "in-progress" column for most workboards :) [23:56:31] !log Removed cherry-pick and rebased /var/lib/git/operations/puppet on integration-puppetmaster after merge of https://gerrit.wikimedia.org/r/#/c/281706/ [23:56:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master