[00:19:57] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T152563#2874960 (10mmodell) [00:20:02] RECOVERY - Puppet run on integration-slave-trusty-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [00:20:04] RECOVERY - Puppet run on integration-slave-precise-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [00:29:09] 03Scap3 (Scap3-MediaWiki-MVP), 10releng-201516-q3, 03releng-201617-q4, 10scap, and 2 others: [EPIC] Migrate the MW weekly train deploy to scap3 - https://phabricator.wikimedia.org/T114313#2874968 (10greg) [02:00:34] (03Abandoned) 10Legoktm: Allow specifying jobs to use to merge queues in DependentPipelineManager [integration/zuul] - 10https://gerrit.wikimedia.org/r/203290 (https://phabricator.wikimedia.org/T107529) (owner: 10Legoktm) [02:04:11] (03CR) 10Legoktm: [C: 032] doc: Move VisualEditor under MediaWiki extensions [integration/docroot] - 10https://gerrit.wikimedia.org/r/316202 (owner: 10Legoktm) [02:10:20] (03CR) 10jenkins-bot: [V: 04-1] doc: Move VisualEditor under MediaWiki extensions [integration/docroot] - 10https://gerrit.wikimedia.org/r/316202 (owner: 10Legoktm) [02:15:40] (03PS2) 10Legoktm: doc: Move VisualEditor under MediaWiki extensions [integration/docroot] - 10https://gerrit.wikimedia.org/r/316202 [02:16:24] (03CR) 10Legoktm: [C: 032] doc: Move VisualEditor under MediaWiki extensions [integration/docroot] - 10https://gerrit.wikimedia.org/r/316202 (owner: 10Legoktm) [02:16:47] (03Merged) 10jenkins-bot: doc: Move VisualEditor under MediaWiki extensions [integration/docroot] - 10https://gerrit.wikimedia.org/r/316202 (owner: 10Legoktm) [02:50:32] 10scap, 06Operations: Trying to scap while l10nupdate is syncing shows unhelpful error - https://phabricator.wikimedia.org/T153278#2875159 (10Krenair) [04:36:46] 10Continuous-Integration-Config, 06Discovery, 06Maps (Kartotherian): maps/kartotherian and maps/tilerator repos do not have Jenkins jobs - https://phabricator.wikimedia.org/T142740#2875345 (10Yurik) [04:38:06] 03Scap3, 06Discovery, 03Interactive-Sprint, 06Maps (Kartotherian): Break Kartotherian scap3 deployment into 2 groups - https://phabricator.wikimedia.org/T147337#2875363 (10Yurik) [05:06:15] (03CR) 10Legoktm: [C: 04-1] .gitreview: swap defaultbranch for track [integration/uprightdiff] (debian) - 10https://gerrit.wikimedia.org/r/326821 (owner: 10Hashar) [05:07:50] (03CR) 10Legoktm: [] "@Merlijn, are you still pursuing this?" [integration/composer] - 10https://gerrit.wikimedia.org/r/240451 (owner: 10Merlijn van Deen) [05:12:05] (03CR) 10Legoktm: [C: 032] Fix test result parsing, and correct new errors that were exposed [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/324376 (https://phabricator.wikimedia.org/T146439) (owner: 10Samwilson) [05:13:19] (03Merged) 10jenkins-bot: Fix test result parsing, and correct new errors that were exposed [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/324376 (https://phabricator.wikimedia.org/T146439) (owner: 10Samwilson) [05:20:33] (03PS1) 10Legoktm: Don't require documentation for test cases [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327440 [05:20:35] (03PS1) 10Legoktm: Don't require @return annotations for plain "return;" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327441 [05:20:36] thanks legoktm ! ^ [05:20:37] (03PS1) 10Legoktm: Allow filtering documentation requirements based on visibility [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327442 [05:20:51] samwilson: ^ if you wanna review some patches :) [05:21:21] legoktm: cool, will have a look! :) [05:21:40] 10MediaWiki-Codesniffer, 13Patch-For-Review: Undefined index: parenthesis_closer in SpaceBeforeControlStructureBraceSniff.php - https://phabricator.wikimedia.org/T146439#2875569 (10Legoktm) 05Open>03Resolved [05:27:28] I found one more bug with parameters being passed by reference, fixing now [06:24:31] (03CR) 10Krinkle: [] "This should probably have a test to confirm that the sniff passes when no doc is present for a private function. Ideally also a test to sh" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327442 (owner: 10Legoktm) [06:26:16] (03CR) 10Krinkle: [C: 032] Don't require @return annotations for plain "return;" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327441 (owner: 10Legoktm) [06:26:38] (03CR) 10Krinkle: [C: 031] "LGTM, but haven't tested." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327440 (owner: 10Legoktm) [06:32:17] (03PS2) 10Legoktm: Allow filtering documentation requirements based on visibility [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327442 [06:34:10] (03CR) 10Legoktm: [] "Thanks, making me add the test led me too discover it didn't actually work, oops. There's already a test that covers not having a doc bloc" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327442 (owner: 10Legoktm) [07:26:29] 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2875612 (10Osnard) Same issue occurs now with https://gerrit.wikimedia.org/r/#/c/327214/ [07:35:24] 03Scap3, 10ContentTranslation-CXserver, 10MediaWiki-extensions-ContentTranslation, 05Language-Engineering October-December 2016, and 4 others: Enable Scap3 config deploys for CXServer - https://phabricator.wikimedia.org/T147634#2875616 (10KartikMistry) Scheduled on 15th December: https://wikitech.wikimedia... [07:41:44] (03CR) 10Dzahn: [] "you added many reviewers, except elukey, but per T97513#2843117 he is the one who most likely knows in operations. added him" [integration/config] - 10https://gerrit.wikimedia.org/r/325064 (https://phabricator.wikimedia.org/T97513) (owner: 10Paladox) [07:49:00] 10Continuous-Integration-Config, 13Patch-For-Review: mwext-testextension-* jobs does not remove skins from previous runs - https://phabricator.wikimedia.org/T113841#2875624 (10hashar) 05Open>03Resolved a:03hashar Indeed ! ;-} [08:09:36] 10Continuous-Integration-Config, 13Patch-For-Review: mwext-testextension-* jobs does not remove skins from previous runs - https://phabricator.wikimedia.org/T113841#2875640 (10Paladox) Thanks [08:12:21] 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2875644 (10Paladox) p:05Triage>03High Changing to high since it has happened twice ^^ @demon hi but I doint see the commit that broke the whol... [08:36:39] (03CR) 10Krinkle: [C: 032] Allow filtering documentation requirements based on visibility [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327442 (owner: 10Legoktm) [08:37:22] (03CR) 10Krinkle: [C: 032] Don't require documentation for test cases [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327440 (owner: 10Legoktm) [08:38:29] (03Merged) 10jenkins-bot: Don't require documentation for test cases [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327440 (owner: 10Legoktm) [08:38:54] (03Merged) 10jenkins-bot: Don't require @return annotations for plain "return;" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/327441 (owner: 10Legoktm) [09:36:13] o/ [10:30:17] (03PS1) 10Hashar: Apply MediaWiki dependencies to mwext-mw-rspec-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/327473 [10:35:24] (03PS1) 10Hashar: Assert $EXT_NAME is set in jjb macro [integration/config] - 10https://gerrit.wikimedia.org/r/327474 [10:40:05] zeljkof: ^^^^ to do: review / update jenkins jobs / deploy zuul :} [10:40:38] better fix is to always apply mw_set_dependencies() when the repository is a an extension or skin [10:43:01] (03CR) 10Hashar: [] "Can you paste in steps to reproduce ? I am wondering what is happening really and maybe it is just that cucumber no more requires shellwor" [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) (owner: 10Zfilipin) [10:52:24] 10Browser-Tests-Infrastructure, 13Patch-For-Review, 07Ruby, 15User-zeljkofilipin: NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords - https://phabricator.wikimedia.org/T153202#2875907 (10hashar) Can reproduce locally on my machine with the Echo patchset https://gerrit.wikimedia.org/... [10:57:37] (03CR) 10Hashar: [C: 04-1] "Can you enhance the commit message stating that shellwork got removed from rake 11.2.0 and point at upstream PR https://github.com/ruby/ra" [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) (owner: 10Zfilipin) [10:58:18] 10Browser-Tests-Infrastructure, 13Patch-For-Review, 07Ruby, 15User-zeljkofilipin: NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords - https://phabricator.wikimedia.org/T153202#2875916 (10hashar) From 1/1 with Zeljko: Commit is https://github.com/ruby/rake/pull/131/commits/81eb4b98d... [10:59:22] 06Release-Engineering-Team, 10Wikimedia-Developer-Summit, 06Developer-Relations (Oct-Dec-2016), 07Documentation: Developer Summit 2017: Work with TPG and RelEng on solution to event documenting - https://phabricator.wikimedia.org/T132400#2875920 (10Qgil) [11:21:36] 03Scap3, 06Discovery, 03Interactive-Sprint, 06Maps (Kartotherian): Break Kartotherian scap3 deployment into 2 groups - https://phabricator.wikimedia.org/T147337#2689471 (10mobrovac) Just use the `group_size` parameter in `scap.cfg` to achieve that, cf [Parsoid's config](https://github.com/wikimedia/mediawi... [11:33:29] (03PS2) 10Zfilipin: Fix NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) [11:34:21] (03PS3) 10Zfilipin: Fix NameError: uninitialized constant MediawikiSelenium::RakeTask::Shellwords [selenium] - 10https://gerrit.wikimedia.org/r/327211 (https://phabricator.wikimedia.org/T153202) [11:37:25] (03CR) 10Zfilipin: [C: 032] Apply MediaWiki dependencies to mwext-mw-rspec-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/327473 (owner: 10Hashar) [11:38:15] (03Merged) 10jenkins-bot: Apply MediaWiki dependencies to mwext-mw-rspec-jessie [integration/config] - 10https://gerrit.wikimedia.org/r/327473 (owner: 10Hashar) [11:41:28] !log Reloading Zuul to deploy 327473 [11:41:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:43:13] (03CR) 10Zfilipin: [C: 032] Assert $EXT_NAME is set in jjb macro [integration/config] - 10https://gerrit.wikimedia.org/r/327474 (owner: 10Hashar) [11:43:26] (03CR) 10Zfilipin: [] Assert $EXT_NAME is set in jjb macro [integration/config] - 10https://gerrit.wikimedia.org/r/327474 (owner: 10Hashar) [11:43:39] (03CR) 10Zfilipin: [C: 032] Assert $EXT_NAME is set in jjb macro [integration/config] - 10https://gerrit.wikimedia.org/r/327474 (owner: 10Hashar) [11:43:43] \O/ [11:45:46] (03Merged) 10jenkins-bot: Assert $EXT_NAME is set in jjb macro [integration/config] - 10https://gerrit.wikimedia.org/r/327474 (owner: 10Hashar) [11:47:04] (03CR) 10Zfilipin: "Deployed jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/327474 (owner: 10Hashar) [11:47:27] (03CR) 10Zfilipin: "Deployed zuul change." [integration/config] - 10https://gerrit.wikimedia.org/r/327473 (owner: 10Hashar) [11:48:01] (03PS2) 10Zfilipin: WIP mwext-mw-rspec-jessie does not need Vector skin [integration/config] - 10https://gerrit.wikimedia.org/r/327219 (https://phabricator.wikimedia.org/T146916) [11:48:32] (03PS3) 10Zfilipin: mwext-mw-rspec-jessie does not need Vector skin [integration/config] - 10https://gerrit.wikimedia.org/r/327219 (https://phabricator.wikimedia.org/T146916) [11:51:17] hashar: yeah! https://integration.wikimedia.org/ci/job/mwext-mw-rspec-jessie/13/console [11:51:44] the job fails, but because the credentials in the test are wrong 💪 [11:52:09] (03CR) 10Zfilipin: [C: 032] mwext-mw-rspec-jessie does not need Vector skin [integration/config] - 10https://gerrit.wikimedia.org/r/327219 (https://phabricator.wikimedia.org/T146916) (owner: 10Zfilipin) [11:52:43] (03CR) 10Zfilipin: [C: 032] "Deployed mwext-mw-rspec-jessie job." [integration/config] - 10https://gerrit.wikimedia.org/r/327219 (https://phabricator.wikimedia.org/T146916) (owner: 10Zfilipin) [11:53:29] (03Merged) 10jenkins-bot: mwext-mw-rspec-jessie does not need Vector skin [integration/config] - 10https://gerrit.wikimedia.org/r/327219 (https://phabricator.wikimedia.org/T146916) (owner: 10Zfilipin) [12:28:00] 10Continuous-Integration-Config, 06Operations, 06Operations-Software-Development: Add shell scripts CI validations - https://phabricator.wikimedia.org/T148494#2876127 (10hashar) Slightly optimized based on what is in the git tree instead of the local checkout: ``` git ls-tree -z --name-only -r HEAD|xargs -0... [13:47:47] Yippee, build fixed! [13:47:47] Project selenium-VisualEditor » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #244: 09FIXED in 3 min 45 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/244/ [13:55:36] 03Scap3, 10ContentTranslation-CXserver, 10MediaWiki-extensions-ContentTranslation, 05Language-Engineering October-December 2016, and 3 others: Enable Scap3 config deploys for CXServer - https://phabricator.wikimedia.org/T147634#2876309 (10mobrovac) 05Open>03Resolved [14:28:38] 10Deployment-Systems, 05Mediawiki SWAT Deployments, 07Puppet: mwdebug1002 should have PHP extensions - https://phabricator.wikimedia.org/T153316#2876393 (10Mattflaschen-WMF) [14:31:11] Hey RelEng people, Can you merge this patch for beta or should I wait for SWAT? https://gerrit.wikimedia.org/r/#/c/327507/ [14:48:22] !log ladsgroup@deployment-tin:~$ mwscript updateCollation.php --wiki=fawiki (T139110) [14:48:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:49:33] I'm guessing it's not deployed yet [14:49:40] let's wait for some minutes [14:55:42] Amir1: there is a jenkins job that updates the code every 10 minutes or so [14:55:56] they are all at https://integration.wikimedia.org/ci/view/Beta/ [14:55:59] there is a scap running [14:57:49] hashar: thanks for the link [15:03:19] 10Beta-Cluster-Infrastructure: Investigate non-puppetmaster instances configured to be puppetmasters - https://phabricator.wikimedia.org/T153319#2876508 (10Krenair) [15:04:13] 10Beta-Cluster-Infrastructure, 13Patch-For-Review, 07Puppet: puppet failure on deployment-phab01 ... is not a Hash. It looks to be a Array at /etc/puppet/modules/phabricator/manifests/init.pp:68 - https://phabricator.wikimedia.org/T147818#2704482 (10Krenair) Perhaps you would be able to help with T153319 [15:21:22] (03PS5) 10Matthias Mullie: Add AbuseFilter & SpamBlacklist as UploadWizard dependency [integration/config] - 10https://gerrit.wikimedia.org/r/327202 [15:23:52] 10Deployment-Systems, 06Release-Engineering-Team, 06Operations, 05Mediawiki SWAT Deployments, 07Puppet: mwdebug1002 should have PHP extensions - https://phabricator.wikimedia.org/T153316#2876601 (10hashar) From {T146286} , we had the puppet class `::mediawiki::packages::php5` added to `role::deployment::... [15:28:43] 06Release-Engineering-Team, 06Operations, 07Beta-Cluster-reproducible, 07HHVM, 15User-Joe: Switch mwscript from Zend PHP5 to default php alternative (egHHVM) - https://phabricator.wikimedia.org/T146285#2876627 (10hashar) T146286 was to get the PHP5 packages installed on the maintenance servers. This task... [15:37:20] 10Continuous-Integration-Config: Remove 3 extensions from CI - https://phabricator.wikimedia.org/T153035#2876651 (10hashar) [15:39:06] 10Continuous-Integration-Config: Remove 3 extensions from CI - https://phabricator.wikimedia.org/T153035#2867144 (10hashar) CI has a couple few dependencies that would end up being broken: * Ratings depends on Validator * SubPageList depends on ParserHooks. Though the repo has been emptied. [15:39:48] PROBLEM - Puppet run on deployment-phab02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:41:04] (03PS1) 10Hashar: Remove SubPageList from the CI dependency list [integration/config] - 10https://gerrit.wikimedia.org/r/327521 (https://phabricator.wikimedia.org/T147393) [15:43:25] (03PS1) 10Hashar: Unconfigure ParserHooks [integration/config] - 10https://gerrit.wikimedia.org/r/327524 (https://phabricator.wikimedia.org/T153035) [15:44:57] (03PS1) 10Hashar: Unconfigure Maps [integration/config] - 10https://gerrit.wikimedia.org/r/327525 (https://phabricator.wikimedia.org/T153035) [15:46:24] 10Continuous-Integration-Config, 13Patch-For-Review: Remove 3 extensions from CI - https://phabricator.wikimedia.org/T153035#2876670 (10hashar) [15:46:43] 10Continuous-Integration-Config, 13Patch-For-Review: Remove 3 extensions from CI - https://phabricator.wikimedia.org/T153035#2867144 (10hashar) [15:50:55] (03PS1) 10Hashar: Unconfigure Validator [integration/config] - 10https://gerrit.wikimedia.org/r/327527 (https://phabricator.wikimedia.org/T153035) [15:52:55] (03CR) 10Hashar: [C: 032] Unconfigure ParserHooks [integration/config] - 10https://gerrit.wikimedia.org/r/327524 (https://phabricator.wikimedia.org/T153035) (owner: 10Hashar) [15:54:35] (03Merged) 10jenkins-bot: Unconfigure ParserHooks [integration/config] - 10https://gerrit.wikimedia.org/r/327524 (https://phabricator.wikimedia.org/T153035) (owner: 10Hashar) [15:56:40] (03CR) 10Hashar: [C: 032] Unconfigure Maps [integration/config] - 10https://gerrit.wikimedia.org/r/327525 (https://phabricator.wikimedia.org/T153035) (owner: 10Hashar) [15:56:54] (03CR) 10Hashar: [C: 032] Unconfigure Validator [integration/config] - 10https://gerrit.wikimedia.org/r/327527 (https://phabricator.wikimedia.org/T153035) (owner: 10Hashar) [15:59:01] (03Merged) 10jenkins-bot: Unconfigure Validator [integration/config] - 10https://gerrit.wikimedia.org/r/327527 (https://phabricator.wikimedia.org/T153035) (owner: 10Hashar) [16:00:31] 10Continuous-Integration-Config, 13Patch-For-Review: Remove 3 extensions from CI - https://phabricator.wikimedia.org/T153035#2876705 (10hashar) 05Open>03Resolved a:03hashar Removal of Validator breaks ParserFun / Ratings in CI. But they are jobs are not passing anyway. [16:01:02] Reedy: bunch of extensions cleaned :D [16:01:08] whee, thanks :D [16:01:18] Reedy: Validator is in packagist [16:01:27] so maybe we can add it to mediawiki/vendor [16:01:37] and stop using mediawiki/extensions/Validator @0.6 [16:01:40] or whatever version [16:01:49] then [16:01:55] I dont know [16:06:39] lol [16:06:47] Or just remove the lot from wikitech ;) [16:08:20] !log deployment-phab02 : apt-get upgrade T147818 [16:08:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:14:58] 10Beta-Cluster-Infrastructure, 13Patch-For-Review, 07Puppet: puppet failure on deployment-phab01 ... is not a Hash. It looks to be a Array at /etc/puppet/modules/phabricator/manifests/init.pp:68 - https://phabricator.wikimedia.org/T147818#2876837 (10hashar) The mariadb package is broken: ``` # apt-get insta... [16:22:32] Sanity check about i18n-bot — will it update non-en language messages in production if the en message is updated in master but not in production? I don't want fr/de/es/* speakers to expect some functionality that's not advertised in en… [16:33:34] 10Continuous-Integration-Config, 07Easy, 13Patch-For-Review: Fix structure tests for all skins - https://phabricator.wikimedia.org/T113860#2876980 (10hashar) [16:35:40] I am heading out [17:35:10] 10Browser-Tests-Infrastructure, 07Easy: Remove lines from Gemfile that are used by RVM - https://phabricator.wikimedia.org/T1331#2877288 (10Aklapper) Updated list for extensions only: ``` $:foo\> cd /wikimedia/git/mediawiki-extensions/ $:foo\> grep -R -F 'gemset' . ./PoolCounter/Gemfile:#ruby-gemset=PoolCounte... [17:50:19] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 10hardware-requests: codfw: 1 hardware access request for continuous integration - https://phabricator.wikimedia.org/T150865#2877358 (10mark) Approved. [17:54:48] yay [17:55:28] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 10hardware-requests: codfw: 1 hardware access request for continuous integration - https://phabricator.wikimedia.org/T150865#2877378 (10RobH) a:05mark>03RobH [17:56:22] canhasserver? [17:56:28] mmm, servers [17:56:46] Reedy: we have a few server requests, not much in the grand scheme of things, but: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Project/Planning [17:56:59] contint2001, phab2001, and gerrit2001 [17:57:34] phab2001 should soon have a test domain :) [17:58:28] 10Deployment-Systems, 06Release-Engineering-Team, 06Operations, 05Mediawiki SWAT Deployments, 07Puppet: mwdebug1002 should have PHP extensions - https://phabricator.wikimedia.org/T153316#2877386 (10Mattflaschen-WMF) >>! In T153316#2876601, @hashar wrote: > If ones want to test a change via mwscript, I th... [17:59:20] Reedy ^^ wikibus_ has rejoined leaving wikibugs and wikibugs_ now. [18:56:46] (03Abandoned) 10Hoo man: Update Wikidata to the wmf/1.29.0-wmf.6 branch [tools/release] - 10https://gerrit.wikimedia.org/r/326983 (owner: 10Hoo man) [19:02:35] PROBLEM - Host deployment-pdfrendertest02 is DOWN: CRITICAL - Host Unreachable (10.68.19.192) [19:09:45] i deleted that instance ^ [19:13:32] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2877798 (10RobH) [19:14:22] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Secondary production Jenkins for CI - https://phabricator.wikimedia.org/T150771#2877821 (10RobH) [19:14:24] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 10hardware-requests: codfw: 1 hardware access request for continuous integration - https://phabricator.wikimedia.org/T150865#2799514 (10RobH) 05Open>03Resolved With the creation/processing of the setup task T153350, this... [19:18:01] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T149927#2877823 (10greg) 05Open>03Resolved this is olllllld [19:18:34] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2877826 (10RobH) [19:19:54] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T153351#2877832 (10greg) [19:20:09] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.29.0-wmf.6 deployment blockers - https://phabricator.wikimedia.org/T152563#2852689 (10greg) [19:22:23] PROBLEM - Puppet run on deployment-mcs01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [19:23:09] !log Manually rebasing and re-applying cherry picks for operations/puppet on integration-puppetmaster01.eqiad.wmflabs [19:23:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:32:26] RECOVERY - Puppet run on deployment-mcs01 is OK: OK: Less than 1.00% above the threshold [0.0] [19:37:13] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2877910 (10RobH) [20:07:21] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2877798 (10RobH) [20:41:15] 06Release-Engineering-Team, 10MediaWiki-API, 10Monitoring, 06Operations, and 3 others: API action=parsoid-batch not available on Graphite - https://phabricator.wikimedia.org/T152776#2878093 (10Krinkle) [20:47:32] 10Continuous-Integration-Infrastructure: Spike: Evaluate experimental Docker based CI w/ scap builds - https://phabricator.wikimedia.org/T150501#2878111 (10dduvall) From our notes for the 12/8 meeting with Ops regarding the direction of CI: == Proof of concept == A recent proof of concept was implemented to exp... [20:52:15] (03PS3) 10Krinkle: Replace visualeditor-jsduck-jessie with npm-run-doc-jessie template [integration/config] - 10https://gerrit.wikimedia.org/r/324368 [20:55:32] (03PS4) 10Krinkle: Replace visualeditor-jsduck-jessie with npm-run-doc-jessie template [integration/config] - 10https://gerrit.wikimedia.org/r/324368 [20:58:05] (03CR) 10Krinkle: [C: 032] "Compiled and uploaded job 'visualeditor-npm-run-doc-node-4-jessie'." [integration/config] - 10https://gerrit.wikimedia.org/r/324368 (owner: 10Krinkle) [20:59:34] (03Merged) 10jenkins-bot: Replace visualeditor-jsduck-jessie with npm-run-doc-jessie template [integration/config] - 10https://gerrit.wikimedia.org/r/324368 (owner: 10Krinkle) [21:00:12] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/324368 [21:00:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:14:27] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2878168 (10RobH) [21:14:51] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 10hardware-requests: codfw: 1 hardware access request for continuous integration - https://phabricator.wikimedia.org/T150865#2878171 (10RobH) [21:14:54] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2877798 (10RobH) 05Open>03Resolved Handed off to cont-int for use via parent task. [21:15:23] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Secondary production Jenkins for CI - https://phabricator.wikimedia.org/T150771#2795939 (10RobH) contint2001.wikimedia.org is now online in codfw and calling into puppet. It can be deployed into use as needed. [21:15:44] you haz new server ^ [21:18:51] yay! [21:18:54] robh: weee [21:19:21] sorry it took awhile to get approved, there were a lot of orders all in the pipeline =P [21:20:09] robh: no worries! we have a lot of warm spares coming up: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Project/Planning [21:20:21] gerrit2001, phab2001, contint2001 :) [21:24:36] dear jenkins: you suck [21:24:44] LOl. [21:25:12] I'm sure flattery will get me nowhere but I try [21:25:46] https://gerrit.wikimedia.org/r/#/c/327582/1 ... touches nothing related to the failures [21:27:16] * twentyafterfour questions the validity of running a full suite of npm and browser tests on a patch that touches one backend file [21:35:04] /home/jenkins/workspace/npm-node-4/resources/mobile.references.gateway/ReferencesHtmlScraperGateway.js [21:35:06] 21:24:09 32:5 error Expected indentation of 3 tabs but found 4 indent [21:35:13] wtf this patch does not touch that file [21:35:18] lol [21:35:33] twentyafterfour: Jenkins error: twentyafterfour verbally assault me [21:35:42] lol [21:36:53] twentyafterfour [21:36:53] https://gerrit.wikimedia.org/r/#/c/327380/ [21:37:05] npm is tested no matter what [21:37:17] https://gerrit.wikimedia.org/r/#/c/327588/ [21:38:27] how did that make it past the test? [21:38:34] someone added a test that doesn't pass? [21:38:53] twentyafterfour nope eslint is not pined to a specific version [21:38:58] so can break at any time. [21:39:16] wtf that's screwed up [21:39:22] Yep [21:39:39] * paladox submits pull now :) [21:40:16] otherwise we will need to fork the grunt-eslint repo [21:40:22] so we can pin it our selfs [21:43:28] twentyafterfour https://github.com/sindresorhus/grunt-eslint/pull/133 [21:47:43] 10Continuous-Integration-Infrastructure: Spike: Evaluate containerized CI builds using Kubernetes - https://phabricator.wikimedia.org/T153363#2878300 (10dduvall) [21:50:30] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Secondary production Jenkins for CI - https://phabricator.wikimedia.org/T150771#2878325 (10hashar) [21:51:24] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations, 10hardware-requests: codfw: 1 hardware access request for continuous integration - https://phabricator.wikimedia.org/T150865#2878341 (10hashar) Thank you ! [21:51:29] greg-g: ^ feel free to cc interested parties [21:51:52] Is it possible that we can give jenkins some more power temporaly to speed it up [21:52:04] Zppix no. [21:52:50] We have all these datacenters and a new one in TX and we cannot give CI more cpu power... what a side life [21:52:54] Sad* [21:52:57] marxarelli: neat-o [21:53:17] Zppix it dosent work like that [21:53:40] I know but have you seen its speed [21:53:51] 10Continuous-Integration-Infrastructure: Spike: Evaluate containerized CI builds using Kubernetes - https://phabricator.wikimedia.org/T153363#2878349 (10dduvall) [21:53:52] we use labs for tests. ci has a seperate machine for jenkins (hosting), zuul, and others. [21:54:08] yes, it carn't be helped, thats due to nodepool. [21:54:16] a big part if the problem is that we run way too many tests :P [21:54:25] yeh and that too ^^ [21:54:35] more tests the better we catch something. [21:54:36] lol [21:54:52] Maybe since operations dont usually do deployments on friday jenkins can speed up [21:55:22] more tests = more failures = less productivity.... because we spend all our time on crap like https://gerrit.wikimedia.org/r/#/c/327588/ [21:55:36] oh lol [21:55:45] twentyafterfour we should fork grunt-eslint [21:55:49] they doint want to pin it [21:56:28] twentyafterfour could you setup the repo please on phabricator diffusion (differential) and pull https://github.com/sindresorhus/grunt-eslint/ please? [21:56:33] or we should not run grunt-eslint on back-end changes [21:56:45] Oh [21:56:58] what are back-end changes? [21:57:01] 10Continuous-Integration-Infrastructure: Spike: Evaluate containerized CI builds using Kubernetes - https://phabricator.wikimedia.org/T153363#2878360 (10dduvall) [21:57:15] changes that don't touch javascript files [21:58:21] oh [21:58:29] depends as you can use npm for anything [21:58:39] php, js, pp, + more [21:59:03] what really needs supporting is a .wikimedia.ci file [21:59:57] ^^ we could possibly define a file ext that specific tests run so the test is run on only those ext. [22:00:24] 10Continuous-Integration-Infrastructure: Spike: Evaluate containerized CI builds using Kubernetes - https://phabricator.wikimedia.org/T153363#2878368 (10dduvall) [22:00:39] paladox: we do ... [22:00:47] Its called jenkins job builder [22:00:52] lol [22:01:09] Thats how i had my repo set up [22:01:17] LOL [22:01:19] To run a php test on php files only [22:01:20] no we doint [22:01:26] it makes it hard [22:01:35] Zppix: but it still runs a js test when no js files changed [22:01:36] you carn't change a file ext on demand [22:01:40] you need releng envolvement [22:01:46] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: Spike: Evaluate containerized CI builds using Kubernetes - https://phabricator.wikimedia.org/T153363#2878373 (10greg) [22:01:52] Huh? [22:02:11] No i had someone set it up but thats how i had it config [22:02:25] he means jjb files are not part of the repo so you have to submit a patch to integration-config to change something [22:02:31] Yep, but jenkins job builder requires releng to approve + deploy [22:02:43] Obs [22:02:49] All repos are like that [22:02:57] which if we support a small .wikimedia.ci file then a developer can do it instantly when needed. [22:03:07] that will change soon hopefully [22:03:08] That would lead to issues ^ [22:03:11] Yeh [22:03:15] I have a patch for that [22:03:24] Newbie devs could fuck shit up pardon my french [22:03:34] But they wont be able to merge [22:04:16] + ci is safer now with nodepool + We may use Docker instead so tests should be faster as Docker will give ci more resources. [22:04:35] Still its a bitch to have to decline a gerrit change [22:06:04] twentyafterfour lets do what goolgle does and do https://gerrit.wikimedia.org/r/#/c/323198/ [22:06:09] (03PS3) 10Paladox: Create a test that deploy's jjb changes without needing to ssh in and deploy your self [integration/config] - 10https://gerrit.wikimedia.org/r/323198 [22:07:16] twentyafterfour: how can i run a nodejs command using kubectl [22:07:42] yuvpanda will know in -labs [22:08:45] yeah I'm not sure Zppix I haven't used kubectl yet (/me needs to play around with that a bit) [22:08:52] Ok [22:24:23] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 06Operations: setup/install contint2001/WMF6404 - https://phabricator.wikimedia.org/T153350#2878558 (10Papaul) [22:30:05] 10Gerrit, 10BlueSpice: Merge/Submit error on Gerrit: "org.eclipse.jgit.errors.MissingObjectException: Missing unknown" - https://phabricator.wikimedia.org/T153079#2878568 (10demon) >>! In T153079#2875644, @Paladox wrote: > Changing to high since it has happened twice ^^ > > @demon hi but I doint see the comm... [22:33:14] 06Release-Engineering-Team, 10grrrit-wm: Grrrit-wm: fix empty brackets when not adding a cr +/- # or v-/+ # - https://phabricator.wikimedia.org/T153372#2878584 (10Zppix) [23:02:26] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [23:06:22] 10Continuous-Integration-Config, 10CirrusSearch, 06Discovery, 06Discovery-Search: Migrate phan for CirrusSearch to jenkins - https://phabricator.wikimedia.org/T153040#2878784 (10Deskana) p:05Triage>03Normal