Wikimedia Error

[00:33:24] Project browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #218: FAILURE in 23 min: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/218/ [03:54:30] Yippee, build fixed! [03:54:30] Project browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #229: FIXED in 12 min: https://integration.wikimedia.org/ci/job/browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/229/ [03:59:15] 3Wikimedia / 3Continuous integration: Jenkins: Set $wgHTTPProxy in mediawiki config - 10https://bugzilla.wikimedia.org/59253 (10Krinkle) [04:00:18] (03PS1) 10Krinkle: mwconf: Remove references to pmtpa [integration/jenkins] - 10https://gerrit.wikimedia.org/r/166529 [05:36:44] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #105: FAILURE in 8 min 12 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/105/ [05:40:14] !log Setting up integration-slave1004 and integration-slave1009 ({{bug|71873}} fixed) [05:40:20] Logged the message, Master [05:44:06] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #301: ABORTED in 4 min 11 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/301/ [05:44:34] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #265: ABORTED in 24 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/265/ [05:54:29] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #64: ABORTED in 3 min 26 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/64/ [05:54:45] Project browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce build #227: ABORTED in 16 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce/227/ [06:20:16] 3Wikimedia / 3Continuous integration: Jenkins: Figure out long term solution for /tmp management - 10https://bugzilla.wikimedia.org/72011 (10Krinkle) [06:20:18] 3Wikimedia / 3Continuous integration: Jenkins: Figure out long term solution for /tmp management - 10https://bugzilla.wikimedia.org/72011 (10Krinkle) 3NEW p:3Unprio s:3major a:3None While the VM testing will make this problem obsolete, depending on how far away this is, this is a high priority proble... [06:30:18] 3Wikimedia / 3Quality Assurance: QA: Rename /tmp/out.jpg to something else - 10https://bugzilla.wikimedia.org/72012 (10Krinkle) 3NEW p:3Unprio s:3major a:3None I'm not a 100% sure, but I see this on all contint slaves and suspect it comes from browser tests. If that's the case, please: 1) [required... [06:36:32] 3Wikimedia / 3Continuous integration: [Regression] Puppet failing on CI slaves (Trusty) "Unable to locate package python-elasticsearch" - 10https://bugzilla.wikimedia.org/72014 (10Krinkle) 3NEW p:3Unprio s:3critic a:3None https://gerrit.wikimedia.org/r/#/c/163945/ Error: /Stage[main]/Elasticsearch::... [06:43:11] !log Pooled integration-slave1004 [06:43:13] Logged the message, Master [06:43:50] !log Keeping the new integration-slave1009 unpooled because setup could not be completed due to {{bug|72014}}. [06:43:52] Logged the message, Master [06:45:16] 3Wikimedia / 3Continuous integration: [Regression] Puppet failing on CI slaves (Trusty) "Unable to locate package python-elasticsearch" - 10https://bugzilla.wikimedia.org/72014#c1 (10Krinkle) This is affecting integration-slave1006, integration-slave1007, integration-slave1008. But not critical there since t... [06:45:22] And fucking again. Can't do shit because ppl keep breaking stuff without testing. [06:45:52] Another day wasted. [06:47:11] :( [06:49:22] !log Did a slow-rotating graceful depool/reboot/repool of all integration-slave's over the past hour to debug problems whilst waiting for puppet to unblock and set up new slaves. [06:49:24] Logged the message, Master [06:49:29] nn o/ [08:06:15] 3Wikimedia / 3Continuous integration: [Regression] Puppet failing on CI slaves (Trusty) "Unable to locate package python-elasticsearch" - 10https://bugzilla.wikimedia.org/72014#c2 (10Antoine "hashar" Musso (WMF)) p:5Unprio>3Normal s:5critic>3normal Indeed https://gerrit.wikimedia.org/r/#/c/163945/ ad... [08:10:15] 3Wikimedia / 3Continuous integration: Jenkins: Figure out long term solution for /tmp management - 10https://bugzilla.wikimedia.org/72011#c1 (10Antoine "hashar" Musso (WMF)) Seems like a duplicate of Bug 68563 - Jenkins: point TMP/TEMP to workspace and delete it after build completion [08:11:30] 3Wikimedia / 3Continuous integration: ci/jenkins: remove dependency on git.wikimedia.org - 10https://bugzilla.wikimedia.org/72001#c2 (10Antoine "hashar" Musso (WMF)) We have two slave scripts (integration/jenkins.git) which still depends on git.wikimedia.org bin/mw-core-get.sh tools/fetch-mw-ext Most job... [08:19:15] 3Wikimedia / 3Quality Assurance: QA: Rename /tmp/out.jpg to something else - 10https://bugzilla.wikimedia.org/72012#c1 (10Željko Filipin) As far as I know, this is not created by browser tests. (But I could be wrong.) I am adding more people to this bug that could know. [08:30:08] hashar: good morning :) [08:30:13] feeling better? [08:30:34] (03PS4) 10Zfilipin: Create mediawiki-selenium-bundle-rspec [integration/config] - 10https://gerrit.wikimedia.org/r/166029 (owner: 10Hashar) [08:31:15] zeljkof: yeah much better thanks [08:31:38] zeljkof: one of the rspec job cause ruby 2 to spurt out a stacktrace [08:32:13] hashar: hm, saw that somewhere in gerrit and/or mail [08:33:26] I have +2d a few commits in integration/config yesterday, but I did not deploy any jobs, just in case something breaks :) [08:33:34] (03CR) 10Zfilipin: [C: 032] Create mediawiki-selenium-bundle-rspec [integration/config] - 10https://gerrit.wikimedia.org/r/166029 (owner: 10Hashar) [08:33:50] waiting for the last one to get merged, and we can deploy from master ^ [08:34:12] hashar: this one breaks? https://gerrit.wikimedia.org/r/#/c/159644/ [08:35:41] zeljkof: if you +2 a jjb change, you should refresh the job :] [08:35:58] and yeah https://gerrit.wikimedia.org/r/#/c/159644/ cause a stacktrace [08:36:59] hashar: will do, as soon as the last one merges [08:37:08] (03CR) 10Hashar: "Note that the rspec job for mediawiki/selenium cause a stacktrace with proposed patch https://gerrit.wikimedia.org/r/#/c/159644/" [integration/config] - 10https://gerrit.wikimedia.org/r/166029 (owner: 10Hashar) [08:37:26] I haven't looked at the trace [08:37:51] 00:00:55.992 /mnt/jenkins-workspace/workspace/gems/gems/json-1.8.1/lib/json/common.rb:67: [BUG] Segmentation fault [08:37:51] :D [08:37:52] hashar: I did, and looks like ruby crashed [08:38:00] (03Merged) 10jenkins-bot: Create mediawiki-selenium-bundle-rspec [integration/config] - 10https://gerrit.wikimedia.org/r/166029 (owner: 10Hashar) [08:38:32] hashar: ok, everything merged, deploying the jobs [08:38:52] \O/ [08:44:11] hashar: all jobs updated, let's see if anything breaks :) [08:46:30] \O/ [08:46:39] mediawiki-selenium-bundle-rspec is not triggered though [08:46:49] I think it is only in the experimental pipeline in Zuul conf [08:47:05] i.e. needs to be manually triggered by commenting in Gerrit 'check experimental' [08:54:21] hashar: by the way, are you interested in attending selenium workshop? https://www.mediawiki.org/wiki/QA/Selenium_Workshop [08:54:31] (the day after all hands, friday) [08:59:23] yeah probably [08:59:33] have to setup some workshop for Zuul as well [09:20:54] hashar: great, please sign up for the workshop, the space is limited [09:21:18] and please do consider a CI workshop (JJB, zuul and friends) [09:23:31] yeah [09:23:46] probably need to sign up with my wmf account [09:23:52] for which I lost the pass :D [09:24:32] hashar: reset password :) [09:25:00] I would recommend https://lastpass.com/ as a password manager [09:25:06] I use it for years and I love it [09:26:00] !log renamed deployment-cxserver02 node slaves to 03 and updated the ip address [09:26:03] kart_: ^^^ :D [09:26:05] Logged the message, Master [09:26:26] hashar: what ::) [09:36:47] Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #201: FAILURE in 21 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/201/ [09:37:28] hashar: uh oh ^ [09:37:36] not sure what is wrong there, checking [09:40:12] oops, looks like it was my mistake [09:40:24] I have refreshed the jobs, but forgot to pull the latest jjb :( [09:40:26] fixing [09:46:39] hashar: how come this job is running on precise, not trusty? [09:46:43] https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/202/console [09:46:52] 00:00:00.041 Building remotely on integration-slave1001 (hasPhpUnit hasPhpcs contintLabsSlave UbuntuPrecise) [09:47:23] hmm [09:47:31] 3Wikimedia / 3Continuous integration: [Regression] Puppet failing on CI slaves (Trusty) "Unable to locate package python-elasticsearch" - 10https://bugzilla.wikimedia.org/72014#c3 (10Filippo Giunchedi) 5NEW>3RESO/FIX uploaded python-elasticsearch 1.0.0-2chl1~trusty1 to trusty-wikimedia [09:47:35] we haven't migrated them to Trusty yet ? :D [09:47:54] hashar: didn't you say that they are migrated? [09:48:04] or did I misunderstood that? [09:48:04] only the *bundle* ones [09:48:13] what do you mean by that? [09:48:18] if mediawiki/selenium now requires ruby2 I guess we need to migrate all the browertests jobs as well [09:48:41] * 0a8f087 - Migrate bundler based jobs to ruby2.0 (on Trusty) (21 hours ago) [09:48:51] that only changes the bundler job template [09:48:56] not the browser tests [09:49:17] uh [09:49:22] i am slightly lost :) [09:49:28] https://gerrit.wikimedia.org/r/#/c/166049/3/jjb/ruby-jobs.yaml,unified :D [09:49:42] the 'bundle' jjb macro is changed [09:49:49] but that macro is not used by browertests jobs [09:49:54] I see [09:49:58] they use a copy pasted version [09:50:21] so we need to update the macro? [09:50:54] so if mediawiki/selenium requires ruby 2.0 we need to adjust all the copy pasted code [09:51:12] hashar: ok, will work on that [09:51:32] zeljkof: yeah update everything :-D [09:51:34] http://www.yodaquotes.net/try-not-do-or-do-not-there-is-no-try/ [09:51:49] before release v1.0 of the gems, I guess we want to enforce ruby2.0 [09:52:18] hashar: I am not sure if that is doable from the gem [09:52:24] https://www.youtube.com/watch?v=BQ4yd2W50No ! [09:52:33] can't Gemfile enforce a ruby version? [09:52:35] but I think I saw it can be specified in the gemfile [09:52:48] I think so, saw it somewhere, but never used [09:52:55] the time has come to try it out [09:54:29] bundler as http://bundler.io/v1.7/gemfile_ruby.html [09:54:30] has [09:59:36] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #198: FAILURE in 19 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/198/ [10:01:23] :D [10:01:56] zeljkof: you only changed BROWSER_TIMEOUT='' [10:01:56] https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/jobConfigHistory/showDiffFiles?timestamp1=2014-10-10_09-24-42×tamp2=2014-10-14_09-51-02 [10:01:57] :D [10:03:34] hashar: yes, looking :) [10:03:34] but now sure why the jobs fail now [10:04:12] test2 is dead? :D [10:04:20] 00:00:16.226 Scenario: Search suggestions # features/smoke.feature:15 [10:04:20] 00:00:16.644 Timeout::Error (Timeout::Error) [10:04:24] Browser request was cancelled before a Sauce Labs virtual machine was found [10:04:28] https://saucelabs.com/tests/922d43ce06cc4e5883cdc402df16f78a [10:04:34] I wish that message to shows the URL it attempt to reach [10:05:47] could it be that export BROWSER_TIMEOUT= set a timeout of 0 seconds? [10:05:56] hashar: yes, testing [10:06:07] timeout = ENV["BROWSER_TIMEOUT"].to_i [10:06:07] hehe [10:06:38] which would end up setting a timeout of 0 [10:06:50] probably doesn't play well with client = Selenium::WebDriver::Remote::Http::Default.new [10:08:08] Yippee, build fixed! [10:08:09] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #199: FIXED in 1 min 30 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/199/ [10:10:17] hashar: yes, reverting the timeout commit fixed the problem [10:11:18] browser_timeout: 360 [10:11:22] it is only set at one place [10:11:44] hashar: yes, it should be not set elsewhere [10:11:55] will see if the fix is easier to do in jjb or mw_sel [10:12:10] so either mediawiki/selenium should only set it when it is actually a number [10:12:21] (i.e. skip setting the timeout if the env variable is an empty string [10:12:33] or our jjb macro should only use export BROWSER_TIMEOUT={browser_timeout} [10:12:43] when {browser_timeout} is not an empty string [10:12:52] which is the default: jjb/job-templates-browsertests.yaml: browser_timeout: '' [10:12:56] I would wrap the export [10:13:04] and leave mediawiki/selenium untouched [10:13:24] something like: [10:13:26] hashar: yes, releasing the new version of the gem is harder than fixing jjb [10:13:52] if [ -n "{browser_timeout}" ]; then export BROWSER_TIMEOUT={browser_timeout}; fi [10:13:52] Tobi_WMDE_SW: timeout change broke all the builds :) [10:14:12] that export is set in two places [10:20:24] hashar: thanks for the suggestion, it fixed the problem https://integration.wikimedia.org/ci/view/BrowserTests/view/CirrusSearch/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/203/console [10:20:31] pushing the change to gerrit [10:20:45] https://integration.wikimedia.org/ci/view/BrowserTests/view/CirrusSearch/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/jobConfigHistory/showDiffFiles?timestamp1=2014-10-14_09-42-06×tamp2=2014-10-14_10-18-56 :D [10:20:49] Yippee, build fixed! [10:20:49] Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #203: FIXED in 1 min 21 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/203/ [10:20:59] well done [10:21:07] going to lunch with my wife [10:21:12] will be back later on [10:21:44] feel free to start the work on migrating the browsertests to Trusty / ruby2 :] [10:22:04] hashar: gladly, but I have a ton of other stuff to do, probably will not have the time [10:22:33] (03PS1) 10Zfilipin: Fix broken jobs when BROWSER_TIMEOUT is not set [integration/config] - 10https://gerrit.wikimedia.org/r/166547 [10:24:14] zeljkof: well ruby2 is probably going to be a requirement nothlesss isn't it ? [10:24:21] hashar: soon [10:24:24] but not today :) [10:24:43] (03CR) 10Hashar: [C: 031] "Spotted by Zeljkof and confirmed to work." [integration/config] - 10https://gerrit.wikimedia.org/r/166547 (owner: 10Zfilipin) [10:24:45] :D [10:24:46] there is just stuff that I needed to do last week/month [10:24:53] hehe [10:24:58] I am off for lunch [10:25:03] before taking on a new project [10:27:39] (03PS1) 10Zfilipin: Update documentation on how to use jenkins-jobs [integration/config] - 10https://gerrit.wikimedia.org/r/166548 [10:42:38] (03PS1) 10Zfilipin: ZeroPortal has browser tests [selenium] - 10https://gerrit.wikimedia.org/r/166549 [10:43:00] (03CR) 10jenkins-bot: [V: 04-1] ZeroPortal has browser tests [selenium] - 10https://gerrit.wikimedia.org/r/166549 (owner: 10Zfilipin) [11:27:43] hashar: welcome back :) [11:28:07] I am just finishing config commit for zero portal, not sure if I did everything [11:28:12] will push it in a minute [11:29:00] (03PS1) 10Zfilipin: ZeroPortal has browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/166553 [11:30:16] hashar: ^ [11:30:32] also not sure why this one fails https://gerrit.wikimedia.org/r/#/c/166549/ [11:31:13] looks like bundle exec yard [11:31:17] breaks ruby 2.0 [11:31:19] or something [11:32:08] arghh [11:32:24] (03CR) 10Zfilipin: "recheck" [selenium] - 10https://gerrit.wikimedia.org/r/166549 (owner: 10Zfilipin) [11:32:28] 00:00:20.491 /mnt/jenkins-workspace/workspace/gems/gems/json-1.8.1/lib/json/common.rb:67: [BUG] Segmentation fault [11:36:35] this looks related https://bugs.ruby-lang.org/issues/9444 [11:38:16] looking at the json common.rb file there is: [11:38:17] const_set :SAFE_STATE_PROTOTYPE, State.new [11:38:22] might need to be const_set :SAFE_STATE_PROTOTYPE, State.new() [11:38:42] (03PS2) 10Zfilipin: ZeroPortal has browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/166553 [11:39:17] Project browsertests-ZeroPortal-zero.wikimedia.org-linux-firefox-sauce build #1: FAILURE in 1.3 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroPortal-zero.wikimedia.org-linux-firefox-sauce/1/ [11:39:19] AHHH [11:39:42] zeljkof: soo [11:39:51] we maintain a gem cache on a per instance basis [11:39:58] but that gem cache does not vary by ruby version :D [11:40:01] so [11:40:06] hashar: :) [11:40:16] yes, that would be a problem [11:40:23] we probably end up using something compiled against ruby 1.9.3 [11:40:28] which would definitely not work with ruby2 [11:41:08] (03CR) 10Zfilipin: "The jobs fails:" [integration/config] - 10https://gerrit.wikimedia.org/r/166553 (owner: 10Zfilipin) [11:41:15] we need to add the ruby version in GEM_HOME [11:41:45] $ ruby --version [11:41:45] ruby 2.1.2p95 (2014-05-08 revision 45877) [x86_64-darwin13.0] [11:41:46] pff [11:41:55] so stupidly long [11:42:05] (03CR) 10Zfilipin: "Trying to format the message, one more time." [integration/config] - 10https://gerrit.wikimedia.org/r/166553 (owner: 10Zfilipin) [11:43:10] $ ruby -e 'puts RUBY_VERSION' [11:43:10] 2.1.2 [11:43:11] \O/ [11:43:37] exactly what I wanted to propose :) [11:43:42] http://ruby.about.com/od/advancedruby/a/How-To-Display-Your-Ruby-Version-In-Your-Bash-Prompt.htm [11:44:06] I am amending the JJB copy pasted macros [11:46:24] great [11:46:28] * zeljkof is out of lunch [11:48:37] (03CR) 10Hashar: "So the stracktrace is because the json gem has some compiled module. But the cached version in GEM_HOME is for ruby1.9.x which obviously d" [selenium] - 10https://gerrit.wikimedia.org/r/159644 (owner: 10Dduvall) [11:51:05] (03PS1) 10Hashar: Namespace GEM_HOME based on ruby version [integration/config] - 10https://gerrit.wikimedia.org/r/166555 [11:52:53] (03PS2) 10Hashar: Namespace GEM_HOME based on ruby version [integration/config] - 10https://gerrit.wikimedia.org/r/166555 [11:52:59] (03CR) 10Hashar: "recheck" [selenium] - 10https://gerrit.wikimedia.org/r/166549 (owner: 10Zfilipin) [11:57:12] (03PS3) 10Hashar: Namespace GEM_HOME based on ruby version [integration/config] - 10https://gerrit.wikimedia.org/r/166555 [12:00:37] zeljkof: hashar: oh.. [12:00:40] what? [12:00:42] no [12:00:54] damn [12:00:56] :) [12:00:59] sorry for that [12:01:43] is it that export BROWSER_TIMEOUT= sets the timeout to 0? [12:01:53] why? wouldn't that unset the env var? [12:02:54] Tobi_WMDE_SW: yeah [12:03:11] so in JJB BROWSER_TIMEOUT='{browser_timeout}' would define the env variable [12:03:14] set to an empty string [12:03:33] and mediawiki selenium cast that empty string to an integer ( using to_i [12:03:40] which for an empty string yield a 0 [12:04:26] (03CR) 10Hashar: "check experimental" [selenium] - 10https://gerrit.wikimedia.org/r/159644 (owner: 10Dduvall) [12:07:03] (03CR) 10Hashar: "recheck" [selenium] - 10https://gerrit.wikimedia.org/r/159644 (owner: 10Dduvall) [12:13:00] (03CR) 10Hashar: "So the Jenkins job issue should be fixed by varying the gem cache based on the ruby version being used. https://gerrit.wikimedia.org/r/#/" [selenium] - 10https://gerrit.wikimedia.org/r/159644 (owner: 10Dduvall) [12:13:58] (03CR) 10Hashar: "Gave it a try for the job based on the bundle macro (i.e. the ones using ruby2 / gem2.0): https://gerrit.wikimedia.org/r/#/c/159644/ and h" [integration/config] - 10https://gerrit.wikimedia.org/r/166555 (owner: 10Hashar) [12:14:31] (03CR) 10Hashar: "Magically fixed by varying the gem cache with the ruby version being used: https://gerrit.wikimedia.org/r/#/c/166555/" [selenium] - 10https://gerrit.wikimedia.org/r/166549 (owner: 10Zfilipin) [12:14:45] * zeljkof is back [12:17:40] (03CR) 10Hashar: Namespace GEM_HOME based on ruby version (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/166555 (owner: 10Hashar) [12:19:48] (03CR) 10Zfilipin: Namespace GEM_HOME based on ruby version (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/166555 (owner: 10Hashar) [12:24:13] (03CR) 10Hashar: Namespace GEM_HOME based on ruby version (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/166555 (owner: 10Hashar) [12:30:19] hashar: is there a way to move this commit to the correct repo? https://gerrit.wikimedia.org/r/#/c/152926/ [12:30:31] or should I recreate the patch in the correct repo? [12:31:01] yeah [12:31:05] cherry pick it :D [12:31:17] hashar: how? from gerrit? [12:31:23] cd /path/to/your/checkout/of/integration/config [12:31:33] then on the change https://gerrit.wikimedia.org/r/#/c/152926/ [12:31:41] below the Patch set information there is a Download: [12:31:51] choose cherry-pick and anonymous HTTP [12:31:57] that craft a command that you can copy paste [12:32:03] and thus easily fetch and apply the patch [12:32:12] hashar: thanks! [12:32:13] doing it [12:32:27] git might be smart enough to detect the file renaming [12:34:05] it was [12:34:07] fixing conflict [12:34:29] Project beta-scap-eqiad build #25466: FAILURE in 0.43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/25466/ [12:35:20] 12:34:28 12:34:28 scap failed: LockFailedError Failed to lock /var/lock/scap: [Errno 11] Resource temporarily unavailable (duration: 00m 00s) [12:37:46] hashar: no, it was not [12:38:05] the repo now has both layout.yaml and zuul/layout.yaml [12:38:14] and I am not sure how to merge them [12:41:31] 3Wikimedia / 3Continuous integration: [upstream] Jenkins: jobs created via JJB are not properly registered in Zuul Gearman server - 10https://bugzilla.wikimedia.org/63758#c4 (10Antoine "hashar" Musso (WMF)) I have upgraded Jenkins Gearman plugin to fix jobs registrations: * cherry picked https://review.opens... [12:53:15] Yippee, build fixed! [12:53:16] Project beta-scap-eqiad build #25467: FIXED in 8 min 59 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/25467/ [12:56:25] (03PS1) 10Zfilipin: Added job template and builder that runs rubocop, Ruby linter [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [12:57:04] (03Abandoned) 10Zfilipin: Added job template and builder that runs rubocop, Ruby linter [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/152918 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [12:57:20] (03CR) 10Zfilipin: "Moved from https://gerrit.wikimedia.org/r/#/c/166563/" [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:02:59] (03PS2) 10Zfilipin: WIP Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [13:03:23] (03CR) 10Zfilipin: "Moved to https://gerrit.wikimedia.org/r/#/c/166563/" [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/152926 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:07:18] (03CR) 10jenkins-bot: [V: 04-1] WIP Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:17:28] (03PS3) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [13:20:33] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:25:56] (03PS4) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [13:29:10] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:35:21] (03PS5) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [13:38:29] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:39:16] ;:( [13:39:39] zeljkof: you have to run jjb to define the jobs [13:39:55] hashar: ah, that is why it is failing? [13:40:01] will do [13:40:09] yeah look at bottom of https://integration.wikimedia.org/ci/job/integration-zuul-layoutvalidation/1885/console [13:40:22] Job VisualEditor-rubocop not defined [13:40:29] maybe the message should be made nicer :D [13:40:42] I took a look but did not notice that :) [13:40:50] it should be RED! :) [13:41:56] that is build in zuul server though [13:45:08] what does that mean? [13:45:26] (03PS6) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [13:47:39] zeljkof: sorry [13:47:58] zeljkof: I meant the message reported "Job foobar not defined", is a message in Zuul server [13:48:04] we can't really tweak it easily [13:48:51] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [13:49:20] hashar: hm, looks like the above patch does not _create_ any jobs :) [13:49:25] looking... [13:49:52] https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/1437/console [13:50:57] well you are invoking 'mwext-{name}-rubocop' [13:51:07] so that surely creates a bunch of 'mwext-*-rubocop' jobs [13:51:15] you have to run jenkins job builder now [13:51:20] to have the job created in jenkins [13:51:42] hashar: but take a look at this https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/1437/console [13:51:48] no new jobs are created [13:51:55] oh [13:52:02] that is a bug :] [13:52:16] I have tried it locally, not jobs created either [13:52:26] I have messed up something somewhere [13:52:50] ah yeah [13:52:59] so you are creating a template name: '{name}-rubocop' [13:53:01] in JJB [13:53:06] but it is not applied to any JJB project [13:53:19] trying to figure out how to move the commit to use job-template '{name}-bundle-{bundlecommand}' as you have suggested here https://gerrit.wikimedia.org/r/#/c/152918/ [13:53:24] though you applied those new jobs in Zuul layout config https://gerrit.wikimedia.org/r/#/c/166563/6/zuul/layout.yaml [13:54:03] the template '{name}-rubocop' should be invoked in mediawiki-extensions.yaml [13:54:36] ie under a: [13:54:52] - project: [13:54:52] name: mwext-ContentTranslation [13:54:52] jobs: [13:54:52] - '{name}-rubocop' [13:55:59] hashar: I see [13:56:05] doing... [14:02:03] (03PS7) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [14:05:17] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [14:06:28] hashar: ok, extensions are here https://integration.wikimedia.org/ci/job/integration-jjb-config-diff/1438/consoleFull [14:06:31] deploying [14:07:47] what is rubocop and should I be afraid? [14:08:00] Nikerabbit: you should be _very_ afraid :P [14:08:09] it is a ruby linter [14:08:25] helping is have beautiful ruby code by complaining if you commit ugly code [14:08:49] not deployed yet, when deployed will be non voting until we are sure it works fine [14:09:07] (03CR) 10Zfilipin: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [14:11:36] hashar: progress, down to just a few missing jobs https://integration.wikimedia.org/ci/job/integration-zuul-layoutvalidation/1888/console [14:15:47] (03CR) 10Hashar: "The wiki page https://github.com/sebastianbergmann/phpunit/wiki/ChangeLog-for-PHPUnit-3.7" [integration/phpunit] - 10https://gerrit.wikimedia.org/r/164683 (owner: 10BryanDavis) [14:16:14] (03CR) 10Hashar: "recheck" [integration/phpunit] - 10https://gerrit.wikimedia.org/r/164683 (owner: 10BryanDavis) [14:18:20] (03PS8) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [14:18:40] hashar: the only thing I can not figure out is where to define VisualEditor-rubocop job [14:19:48] there is VisualEditor-ruby1.9.3lint, so when I find it, I will put rubocop job there [14:19:52] _when_ I find it... [14:20:52] found it! [14:21:02] in browsertests.yaml, not intuitive at all [14:21:22] but I do remember a problem with it, so it got moved from another file [14:21:26] now it does make sense [14:21:35] (03CR) 10jenkins-bot: [V: 04-1] Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) (owner: 10Zfilipin) [14:22:40] (03PS9) 10Zfilipin: Run rubocop, Ruby linter, for all repositories that have Ruby code [integration/config] - 10https://gerrit.wikimedia.org/r/166563 (https://bugzilla.wikimedia.org/69245) [14:33:09] chrismcmahon: welcome back :) [14:36:13] zeljkof: yeah the browser tests are defining the jjb VisualEditor project [14:36:18] yet another thing to refactor [14:36:19] :D [14:36:47] hashar: the commit is now all green :) https://gerrit.wikimedia.org/r/#/c/166563/ [14:37:06] but I am not sure how to make it use job-template '{name}-bundle-{bundlecommand}' [14:37:11] like you have suggested here [14:37:15] https://gerrit.wikimedia.org/r/#/c/152918/ [14:37:42] so there https://gerrit.wikimedia.org/r/#/c/166563/9/jjb/macro.yaml [14:37:49] the macro is largely a duplicate of the bundle macro [14:38:31] so it could be a '{name}-bundle-{bundlecommand}' template [14:38:40] being passed 'rubocop' as the bundlecommand [14:38:51] that will do the GME_HOME hack and gem install [14:39:24] though my bundle command macro does not let you pass additional parameters to the exec [14:39:34] ei --format progress --format html --out log/rubocop.html [14:39:40] hashar: ok, that is optional [14:39:49] we can improve the macro later [14:40:00] now is a good time to reduce some duplication [14:40:40] with python tox [14:40:59] the target env run a command which is specified in a tox.ini file and can thus take extra args [14:41:19] ok, looking at mediawiki-misc.yaml, I think I have an idea on how to use [14:41:26] the bundlecommand [14:42:32] will amend the commit [14:42:43] if we could define default arguments passed per command that would be nice [14:42:53] i.e. jenkins will invoke bundle exec rubocop [14:43:14] and would read some .bundlerc or .rubocoprc to have some extra args defined [14:45:07] do we have something similar, or should this be implemented? [14:45:16] need to be implemented [14:45:25] can we define our own command with bundler? [14:45:42] if we could define a : bundle doc [14:45:51] hashar: hm [14:45:53] and have that 'doc' to execute rubocop --some --parameters [14:45:53] not sure [14:45:55] that would be nice :D [14:46:04] will check [14:46:11] rubocop probably has a configuration file [14:46:17] ok, for now will move everything to the bundlecommand [14:46:17] so we can just run bundle exec rubocop [14:46:29] hashar: yes, there is a config file there [14:46:38] and expect the rubocop file to publish the report at the same place everything [14:46:44] not sure what options are available, but it should have what we need [14:47:51] yeah [14:47:52] .rubocop.yml [14:48:09] https://github.com/bbatsov/rubocop#defaults [14:49:06] hashar: yes, we use it to ignore violations we do not care about [14:49:17] $ rubocop --format html --out log/rubocop.html [14:49:17] No such file or directory @ rb_sysopen - log/rubocop.html [14:49:18] bah [14:49:18] we could just extend the file with formatters [14:49:28] it doesn't know how to create the directory :-/ [14:49:46] probably expects the directory to exist [14:52:45] https://github.com/bbatsov/rubocop/issues/1389 :D [14:52:46] yeah [14:52:47] filled [14:53:32] hi zeljkof, nice to be back [14:53:44]