[01:19:20] Hello [01:20:58] I am having issue when I tried to run vagrant provision after enabling browsertests. [01:21:33] This is the error that I am getting. Any help on how to resolve this would be appreciated [01:21:35] info: Applying configuration version '1392164773.3520e10' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/bro [09:07:24] (03CR) 10Zfilipin: "Manybubbles: I am pretty sure there is already an URL module somewhere in Ruby, it was a pretty bad name from the start, but it did not br" [selenium] - 10https://gerrit.wikimedia.org/r/112449 (owner: 10Zfilipin) [10:14:34] oh man [10:14:35] zeljkof: :( [10:14:42] the cucumber.yml files have been removed from Flow [10:14:52] hashar_: that is a good thing :) [10:15:00] I used that for the Jenkins jobs [10:15:04] they have been removed from almost everywhere [10:15:10] so we had the job calling cucumber -p ciwmf [10:15:12] what do you use it for? [10:15:21] letting developers finely tune the tags they wanted to use / get rid of [10:15:21] really? let me see. in flow? [10:15:30] you can add it back, it is optional [10:15:35] i think uls still has it [10:15:38] na it is ok [10:15:50] do you still use @needs-custom-setup ? [10:15:55] but didn't you stop using wmfci and started using @clean tag? [10:16:13] ah yeah @clean [10:16:15] .. [10:16:23] hashar: I am not sure what repositories use at the time, but @clean is recommened [10:16:25] I tend to dislike that idea, that force people to whitelist the tests they want to run [10:16:36] hashar: that is the current reality [10:16:44] true :D [10:16:50] when we get to 80%+ of tests having @clean [10:16:54] then we can drop that tag [10:17:05] and blacklist ones that do not run on a clean wiki [10:17:19] now most of the scenarios have @needs-custom-setup [10:17:59] if I change the cucumber run to only run @clean , for most repositories we will end up not running anything I guess [10:18:29] and I did * 77fe5fd - Revert "browsertests: replace -p ciwmf with --tags @clean" (19 hours ago) [10:18:29] :D [10:18:36] guess I should move back to --tags @clean [10:19:05] hashar: we will be tagging more stuff with @clean in the future [10:19:09] this or the next week [10:19:14] okkk [10:19:26] and we have to let people know they should write tests so they run on a clean wiki [10:19:29] not labs or test2 [10:19:37] so I should just use: --tags @clean [10:19:44] yes [10:19:56] and if that runs no tests, well bad luck for the repo [10:20:10] they will have to take the action to tag or write the scenarios [10:20:45] (03PS1) 10Hashar: browsertests: replace -p ciwmf with --tags @clean [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112869 [10:20:52] can you announce it to devs? [10:21:00] I have updated the current jobs [10:21:12] hashar: sure, I can write mail to qa mailing list [10:21:15] (03CR) 10Hashar: [C: 032] browsertests: replace -p ciwmf with --tags @clean [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112869 (owner: 10Hashar) [10:21:22] or even better, you should write the e-mail :) [10:21:32] (03Merged) 10jenkins-bot: browsertests: replace -p ciwmf with --tags @clean [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112869 (owner: 10Hashar) [10:21:37] would prefer skipping that :D [10:21:57] ok, adding to my todo list, will do it today [10:22:01] I am not entirely sure what is you guys strategy nor when people should mark @clean [10:22:10] I have no meetings today, will work on that among other things [10:24:10] (03PS24) 10Hashar: (WIP) (WIP) visualeditor-doitall (WIP) (WIP) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 [10:24:29] (03CR) 10Hashar: "rebased" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 (owner: 10Hashar) [10:30:27] zeljkof: and another question for the HEADLESS env variable [10:30:38] should I set it up to "true" regardless of the browser ? [10:30:41] yes [10:30:43] i.e. for both firefox and phantomjs [10:30:47] well, no :) [10:30:55] phantomjs is headless anyway :) [10:31:00] + export BROWSER_LABEL="{{browser}}" [10:31:00] + if [ "$BROWSER_LABEL" != "phantomjs" ]; then [10:31:00] + export HEADLESS=true [10:31:00] + fi [10:31:09] aka only set it for non phantomjs [10:31:20] you can set it for phantomjs, it will not break [10:31:24] I was wondering if we could set HEADLESS=true for phantomjs without breaking anything [10:31:24] but there is no need [10:31:25] nice [10:31:28] much simpler code this way [10:31:33] well, try :) [10:31:39] I do not think it will break [10:31:52] it will just set up headless environment without really using it [10:32:04] so I guess a little bit of overhead there [10:32:21] so the question is what is more important, the overhead or simpler code [10:32:27] simpler code :] [10:32:40] hashar: then go ahead :) [10:33:14] it will break immediately it is not supported, but I do not think that will happen [10:33:19] another question is for Chrome tests [10:33:59] Ubuntu ships the Chromium version which has for binary "chromium-browser" [10:34:14] but apparently we only support BROWSER_LABEL="chrome" [10:38:14] hashar: yes [10:38:18] so, long story short [10:38:40] https://github.com/watir/watirbook/blob/master/installation/ubuntu.md#chromium [10:38:57] you can drive chromium with chromedriver (the same driver that drives chrome) [10:39:29] but chromedriver by default looks only for chrome at it's usual locations [10:39:56] so if you want to drive chromium, you have to let chromedriver know where the binary is [10:40:21] hashar: but nobody asked for that to be supported in mediawiki_selenium gem [10:40:31] so, if you need that, please create a bugzilla bug :) [10:54:29] zeljkof: ok :-) Against QA ? [10:54:38] yes [10:54:39] I mean, should I fill the bug in our bugzilla in QA? [10:54:42] yes [10:54:45] doing so [10:54:47] not a high priority [10:55:04] I look at those bugs regularly and assign to myself ones that I think should be done soon [10:55:25] and if it becomes urgent, raise the priority and assign to me [10:55:34] but having it in bugzilla means it will not get lost :) [10:57:56] https://bugzilla.wikimedia.org/show_bug.cgi?id=61262 [10:57:58] thank you [10:58:03] hashar: thank you :) [10:58:46] made it low priority [11:06:25] (03PS1) 10Hashar: Firefox based browser tests jobs [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112873 [11:08:26] (03CR) 10Hashar: [C: 032] Firefox based browser tests jobs [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112873 (owner: 10Hashar) [11:08:35] and now we can get both phantomjs and firefox jobs \O/ [11:08:43] (03Merged) 10jenkins-bot: Firefox based browser tests jobs [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112873 (owner: 10Hashar) [11:13:12] experimenting with Flow now [11:13:20] I have marked two random features with @clean https://gerrit.wikimedia.org/r/112874 :D [11:27:57] zeljkof: I am off for lunch [11:28:10] you might want to tag/release mediawiki-selenium gem https://bugzilla.wikimedia.org/show_bug.cgi?id=61217 :D [11:28:16] apparently you released it from a pending change [11:28:24] hashar: it should be merged now [11:28:29] but yes, not tagged yet :( [11:28:32] bump version and tag! :-D [11:29:10] and I could use a new version of the gem that let us specify the screenshots destination (commit 235f247 from yesterday) [11:29:29] will see this afternoon. For now I am hungry hehe [12:38:10] back [13:00:58] re [13:44:16] (03PS25) 10Hashar: (WIP) (WIP) visualeditor-doitall (WIP) (WIP) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 [13:44:23] (03CR) 10Hashar: "rebased" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 (owner: 10Hashar) [13:47:31] (03PS1) 10Hashar: browsertest: make log dir writable by all [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112890 [13:47:34] (03PS1) 10Hashar: browsertest: fix up whitespace in LocalSettings.php [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112891 [13:47:40] (03CR) 10Hashar: [C: 032] browsertest: make log dir writable by all [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112890 (owner: 10Hashar) [13:47:45] (03CR) 10Hashar: [C: 032] browsertest: fix up whitespace in LocalSettings.php [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112891 (owner: 10Hashar) [13:48:21] (03Merged) 10jenkins-bot: browsertest: make log dir writable by all [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112890 (owner: 10Hashar) [13:48:36] (03Merged) 10jenkins-bot: browsertest: fix up whitespace in LocalSettings.php [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112891 (owner: 10Hashar) [13:50:33] (03PS26) 10Hashar: (WIP) (WIP) visualeditor-doitall (WIP) (WIP) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/100800 [14:35:24] (03PS1) 10Zfilipin: Moved tests to UploadWizard repository [qa/browsertests] - 10https://gerrit.wikimedia.org/r/112894 [14:48:26] chrismcmahon: I am here [14:49:36] oh ha, your name is displayed oddly in my IRC client. anyway, seems like connection issues with Sauce labs. Here is one: https://wmf.ci.cloudbees.com/job/browsertests-commons.wikimedia.beta.wmflabs.org-linux-firefox/606/console [14:50:44] zeljkof: here is another one, slightly different connection problem: https://wmf.ci.cloudbees.com/job/browsertests-en.wikipedia.beta.wmflabs.org-linux-chrome/661/console [14:57:32] (03PS2) 10Zfilipin: Moved tests to UploadWizard repository [qa/browsertests] - 10https://gerrit.wikimedia.org/r/112894 [15:02:47] chrismcmahon: I am in the middle of something else: https://bugzilla.wikimedia.org/show_bug.cgi?id=56419 [15:03:08] could you please add links to the relevant bug? https://bugzilla.wikimedia.org/show_bug.cgi?id=60338 [15:04:31] zeljkof: no problem, so am I. I just wanted to mention it since I had seen it. [15:34:13] (03PS1) 10Zfilipin: Moved tests from browsertests repository to UploadWizard repository [selenium] - 10https://gerrit.wikimedia.org/r/112899 [16:17:59] (03PS3) 10Zfilipin: Moved tests to UploadWizard repository [qa/browsertests] - 10https://gerrit.wikimedia.org/r/112894 [16:20:42] (03PS2) 10Zfilipin: Moved tests from browsertests repository to UploadWizard repository [selenium] - 10https://gerrit.wikimedia.org/r/112899 [16:31:10] (03PS3) 10Zfilipin: Moved tests from browsertests repository to UploadWizard repository [selenium] - 10https://gerrit.wikimedia.org/r/112899 [16:58:10] (03CR) 10Jhall: [C: 031] Moved tests from browsertests repository to UploadWizard repository [selenium] - 10https://gerrit.wikimedia.org/r/112899 (owner: 10Zfilipin) [17:00:27] (03CR) 10Jhall: [C: 031] Moved tests to UploadWizard repository [qa/browsertests] - 10https://gerrit.wikimedia.org/r/112894 (owner: 10Zfilipin) [17:09:05] JeffHall: good morning :-] [17:09:15] VE change https://gerrit.wikimedia.org/r/#/c/111890/ [Browser test] Headless browser test(s) [17:09:17] is now passing :-] [17:09:19] hashar: Hi Antoine [17:09:30] using a fresh wiki and a headless firefox : http://integration.wikimedia.org/ci/job/visualeditor-doitall/64/console [17:09:31] hashar: Yeah, I saw - it's looking good! [17:09:39] though there is only two scenario being run [17:10:05] hashar: Yeah, now that is passing, I would like to expand it with additional test cases (Scenarios) [17:10:22] so I have marked the "Basic edit feature" with @clean https://gerrit.wikimedia.org/r/#/c/112659/ [17:11:01] and it was failing as well [17:11:48] hashar: I would have thought that scenario would work in headless mode, but there may be an issue - I can take a look at that one today to see if it needs to be tweaked [17:13:15] would be nice :-] [17:13:27] * hashar I have pick this one because it contains "basic" [17:13:33] hashar: I'll get to work on that this morning [17:13:42] will write doc about how to trigger runs manually [17:13:45] it is a bit tedious [17:13:56] hashar: OK, sounds good [17:21:45] looks like Zeljko upgraded net-http-persistent to (2.9.4). I hope that has an effect on the flaky builds. [17:23:34] JeffHall: laaame doc at https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Trigger_a_job_manually [17:23:35] :D [17:23:47] I should get screenshots [17:40:03] also I am not sure why we got a mediawikiapi_client gem [17:40:09] since there is already a a mediawiki-gateway gem [17:40:30] though it has a bunch of dependencies :D [17:40:34] http://rubydoc.info/gems/mediawiki-gateway/0.5.2/frames [17:41:50] or I am confused [17:45:08] hashar: The mediawiki-gateway Rubygem "is no longer in active development", so we decided not to use it. If I remember, it also did not offer user creation methods, which we do want [17:45:20] ah make sense [17:46:35] hashar: Just made more sense to create our own gem, since we needed very specific functionality to support our "hermetic test" goal(s) [17:53:15] i am off for tonight :-] *wave* [17:53:43] Bonne nuit [17:59:08] Hi [18:06:42] Is there anyone here that can help me with Mediawiki-Vagrant setup issue that I am encountering? [18:09:26] Rochan: I'm in a meeting right now, but post your question and someone should be able to give you some pointers! [18:12:38] Ok. Thanks Jeff. [18:13:23] After I enabled the browsertests. I am getting error when running vagrant provision. [18:13:34] The error that I am getting is the following: info: Applying configuration version '1392228723.3520e10' info: mount[files]: allowing mediawiki-vagrant access notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one [18:18:09] Rochan: Interesting - I'm also getting an error when provisioning the "browsertests" role, although not the same error you are getting - I'll take a quick look and see if there's an obvious issue [18:20:17] Rochan: Ah, I think I know what is going on - we re-named a dependent gem from mediawiki-selenium to mediawiki_selenium (note the underscore), and the vagrant config does not appear to be aware of that change [18:26:28] Rochan: Yeah, the puppet manifest for the "browsertests" role appears to be a little out-of-date [18:32:17] Rochan: I'll have to submit a bug for this, since I'm not really the right person to be updating the mediawiki vagrant package, but it does look like the necessary fix is very likely a small one [18:43:12] greg-g: I am about to step out to lunch. Is there any particular prep you need from me for our meeting in 3 hours? [18:44:24] well, I haven't done the slides yet :) [18:45:16] greg-g: OK, want to pair on that this afternoon then? [18:45:27] greg-g: I think we have the outline correct [18:45:47] can probably copy/paste last times for the beginning, then update where needed [19:01:12] Jeff, Thank for looking into it. [19:03:07] Is there anyway to work around this or do I have to wait for the fix? [19:05:06] rochan: I'm taking a look to see if there is an easy fix - changing the Ruby version specified for the "browsertests" role may be enough to resolve the problem [19:33:02] rochan: I can't find an easy answer, since vagrant is somehow deploying an obsolete Rubygem (and I can't quickly figure out where that call is being made). Will have to log a bug :( [19:34:24] (03PS1) 10Physikerwelt: Add MediaWiki extension MathSearch [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/112935 [19:35:36] Ah ok, thanks for the help. Guess I will just have to wait until it the bug gets resolved. [21:11:48] oh, chrismcmahon one thing you can do ahead of time is putting names next to goals [21:11:57] greg-g: doing that now [21:12:36] coolio [21:13:19] greg-g: edit conflict! [21:13:36] :) I just edited one small sectiont to try to avoid that :) [21:32:36] chrismcmahon: oh, also https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals#Release_and_QA is woefully out of date for the last quarter (and our stupid non-alignment) [21:32:53] or: stupidness of quarters aligned to financial world [21:32:57] take your pick :) [21:34:07] eh, it's not that bad [21:35:35] (I really dislike gdocs slides) [21:45:22] greg-g goes for the minimal effect [21:48:22] Rochan: I filed a bug based on the issue that you reported: https://bugzilla.wikimedia.org/show_bug.cgi?id=61285 [21:48:49] Rochan: But as near as I can tell, the error does not prevent the "browsertests" vagrant role from working as expected [21:49:12] yeah [21:49:25] Rochan: I am able to run browser tests via the vagrant VM successfully despite that provisioning error [21:53:25] pwd [22:29:10] JeffHall, chrismcmahon : for 95% of all browser tests, if a pink .errorbox appears or if a ResourceLoader module has an error, the test should fail. Is there a way to implicitly check these on every step? [22:29:57] Zjelko added a Then(/^page has no ResourceLoader errors$/) , but it's only on one test in browsertests, maybe there's a way to have it everywhere [22:30:16] just an idea, I'll file a bug. Thanks! [22:31:14] spagewmf2: Yes, it ought to be possible to abstract that ResourceLoader check for all browser tests - filing a bug for that task is a great idea :) [22:44:28] (03CR) 10MarkTraceur: [C: 032] "Deployed." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/113033 (owner: 10MarkTraceur) [22:54:23] JeffHall: spagewmf2 we can put that RL check everywhere, it is on our list to talk about doing that. [22:54:29] Jeff: For "Running the tests" section @ http://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Running_tests am I executing the commands while SSH in? [22:56:33] Rochan: You can either run the tests while SSHed into the vagrant VM, or if you cloned the qa/browsertests Git repo locally, you can run the tests locally and point them at your vagrant VM - should work either way [23:18:39] I updated the instruction on where to execute bundle install. The gem file is in browsertests/tests/browser instead of browsertests/ [23:21:26] Rochan: Thanks - that re-location was a recent change, so it's good to have that documented :)