[00:53:27] Project UploadWizard-api-commons.wikimedia.org build #473: SUCCESS in 27 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.org/473/ [00:54:35] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #589: SUCCESS in 8 min 35 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/589/ [03:05:10] Project browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs build #97: STILL FAILING in 9.9 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs/97/ [03:07:41] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #65: STILL FAILING in 1 min 41 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/65/ [03:12:04] Yippee, build fixed! [03:12:05] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #121: FIXED in 2 min 4 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/121/ [03:14:20] Project browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce build #94: STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce/94/ [03:19:20] Project browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce build #122: STILL FAILING in 1 min 20 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce/122/ [03:25:29] Project browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce build #107: STILL FAILING in 24 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce/107/ [03:25:33] 3Wikimedia Labs / 3deployment-prep (beta): bits.beta.wmflabs.org down with 503 - 10https://bugzilla.wikimedia.org/69921#c4 (10Matthew Flaschen) 5NEW>3RESO/FIX a:3Ori Livneh The link works for me now, so it seems fixed. If it is still broken when you test, please re-open. [03:28:28] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce build #132: STILL FAILING in 2 min 58 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/132/ [03:31:21] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #178: STILL FAILING in 19 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/178/ [03:32:27] Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #98: STILL FAILING in 3 min 57 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/98/ [03:46:51] Project browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce build #38: SUCCESS in 51 sec: https://integration.wikimedia.org/ci/job/browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce/38/ [03:52:39] Project browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #127: STILL FAILING in 33 min: https://integration.wikimedia.org/ci/job/browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/127/ [03:53:55] Project browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce build #94: SUCCESS in 1 min 14 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce/94/ [04:07:12] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #14: STILL FAILING in 41 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/14/ [04:11:14] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #157: STILL FAILING in 17 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/157/ [04:23:56] Project browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce build #143: STILL FAILING in 51 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce/143/ [04:27:45] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #54: STILL FAILING in 3 min 47 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/54/ [04:30:49] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #139: STILL FAILING in 3 min 2 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/139/ [05:08:55] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_xp-firefox-sauce build #138: STILL FAILING in 1 hr 37 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_xp-firefox-sauce/138/ [05:10:51] Project browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #97: STILL FAILING in 1 min 53 sec: https://integration.wikimedia.org/ci/job/browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/97/ [05:10:53] Project browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce build #83: STILL FAILING in 1 sec: https://integration.wikimedia.org/ci/job/browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce/83/ [05:11:36] Project browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce build #122: SUCCESS in 41 sec: https://integration.wikimedia.org/ci/job/browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce/122/ [05:12:52] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #97: SUCCESS in 1 min 16 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/97/ [05:14:01] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #65: STILL FAILING in 1 min 8 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/65/ [05:23:57] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #213: STILL FAILING in 1 hr 12 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/213/ [06:12:01] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #162: STILL FAILING in 1 hr 41 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/162/ [06:12:35] Project browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce build #127: STILL FAILING in 48 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce/127/ [06:32:13] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #112: STILL FAILING in 19 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/112/ [06:51:20] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #184: STILL FAILING in 1 hr 37 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/184/ [06:52:39] Project browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce build #109: STILL FAILING in 20 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce/109/ [06:53:57] Yippee, build fixed! [06:53:58] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #122: FIXED in 2 min 35 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/122/ [06:54:41] Yippee, build fixed! [06:54:41] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #123: FIXED in 2 min 0 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/123/ [07:09:52] Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #110: STILL FAILING in 15 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/110/ [07:12:29] Project browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce build #124: SUCCESS in 2 min 35 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/124/ [07:14:15] Yippee, build fixed! [07:14:16] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #123: FIXED in 1 min 46 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/123/ [07:24:21] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #167: STILL FAILING in 1 hr 12 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/167/ [07:33:13] Project browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #126: STILL FAILING in 8 min 50 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/126/ [07:35:28] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #66: STILL FAILING in 2 min 13 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/66/ [07:37:33] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #590: SUCCESS in 2 min 3 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/590/ [07:38:01] Project UploadWizard-api-commons.wikimedia.org build #474: SUCCESS in 27 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.org/474/ [07:39:22] Project browsertests-Translate-sandbox.translatewiki.net-linux-firefox-sauce build #93: STILL FAILING in 25 min: https://integration.wikimedia.org/ci/job/browsertests-Translate-sandbox.translatewiki.net-linux-firefox-sauce/93/ [07:42:05] Project browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #98: STILL FAILING in 1 min 13 sec: https://integration.wikimedia.org/ci/job/browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/98/ [07:47:30] 3Wikimedia Labs / 3deployment-prep (beta): Cannot log in to Beta commons: Infinite redirect - 10https://bugzilla.wikimedia.org/69096#c8 (10Gilles Dubuc) (In reply to Andre Klapper from comment #6) > Gilles: Do you still see this problem? > If not we should close this ticket as RESOLVED WORKSFORME (and put my... [07:50:25] Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #140: STILL FAILING in 55 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/140/ [08:17:35] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #591: SUCCESS in 2 min 33 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/591/ [08:20:28] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #592: SUCCESS in 2 min 26 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/592/ [09:42:33] 3Wikimedia / 3Continuous integration: MediaWiki extensions with failing phpunit tests (tracking) - 10https://bugzilla.wikimedia.org/65874 (10Antoine "hashar" Musso) [10:09:28] (03PS6) 10Hashar: create browser test builds for Echo extension [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/154070 (https://bugzilla.wikimedia.org/69130) (owner: 10Cmcmahon) [10:11:53] (03CR) 10Hashar: [C: 032] "INFO:jenkins_jobs.builder:Creating jenkins job browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce" [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/154070 (https://bugzilla.wikimedia.org/69130) (owner: 10Cmcmahon) [10:12:20] (03Merged) 10jenkins-bot: create browser test builds for Echo extension [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/154070 (https://bugzilla.wikimedia.org/69130) (owner: 10Cmcmahon) [10:15:06] 3Wikimedia / 3Quality Assurance: Create Jenkins job for Echo browser tests - 10https://bugzilla.wikimedia.org/69130#c6 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX a:3Chris McMahon INFO:jenkins_jobs.builder:Creating jenkins job browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce INFO:jenki... [10:21:19] Project browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce build #1: FAILURE in 8 min 35 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce/1/ [10:21:54] Project browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce build #1: FAILURE in 9 min 12 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce/1/ [10:36:19] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #1: FAILURE in 23 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/1/ [11:46:18] hashar: abandon spam :> [11:47:19] addshore: yeah that is the bunch of test changes I have submitted a while back [11:47:22] to test jenkinsjobs [11:47:39] addshore: I wanted to fill bugs for each issue, but gave up the idea [11:47:47] will let extension authors fix their test instead [11:47:53] :> [11:50:35] addshore: while you are around, I proposed a python script to check extensions entry point in mediawiki/extensions.git [11:50:35] https://gerrit.wikimedia.org/r/#/c/154761/ [11:50:46] yields some message whenever foo/foo.php is missing [11:51:02] or the extension only has .gitreview file (i.e. the repo has no code [11:51:05] might help cleaning up [12:15:02] (03PS1) 10Hashar: Comment for some non voting jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156099 [12:15:14] (03CR) 10Hashar: [C: 032] Comment for some non voting jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156099 (owner: 10Hashar) [12:15:27] (03Merged) 10jenkins-bot: Comment for some non voting jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156099 (owner: 10Hashar) [12:23:47] (03PS1) 10Hashar: Make mediawiki-vendor-integration voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156101 [12:25:30] (03PS2) 10Hashar: Make mediawiki-vendor-integration voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156101 [12:26:21] (03CR) 10Hashar: [C: 032] Make mediawiki-vendor-integration voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156101 (owner: 10Hashar) [12:26:29] (03Merged) 10jenkins-bot: Make mediawiki-vendor-integration voting [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156101 (owner: 10Hashar) [12:28:46] hashar: done https://gerrit.wikimedia.org/r/#/c/154761/ :) [12:28:59] looks like it works perfectly [12:29:53] (03PS1) 10Hashar: Properly filter out mediawiki-core-install-sqlite [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156102 [12:30:01] (03CR) 10Hashar: [C: 032] Properly filter out mediawiki-core-install-sqlite [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156102 (owner: 10Hashar) [12:31:02] (03Merged) 10jenkins-bot: Properly filter out mediawiki-core-install-sqlite [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156102 (owner: 10Hashar) [12:37:39] addshore: thx :D [12:41:47] 3Wikimedia / 3Continuous integration: mediawiki-core-qunit should rely on Zuul cloner - 10https://bugzilla.wikimedia.org/69972 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3None The mediawiki-core-qunit should use the Zuul cloner just like mediawiki-vendor-integration. It will let us test me... [12:44:18] (03PS1) 10Hashar: Always trigger mediawiki-core-qunit [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156104 [12:44:34] (03CR) 10Hashar: [C: 032] Always trigger mediawiki-core-qunit [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156104 (owner: 10Hashar) [12:44:43] (03Merged) 10jenkins-bot: Always trigger mediawiki-core-qunit [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156104 (owner: 10Hashar) [12:50:33] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #593: SUCCESS in 4 min 33 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/593/ [12:53:29] Project UploadWizard-api-commons.wikimedia.org build #475: SUCCESS in 28 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.org/475/ [13:06:09] (03PS1) 10Hashar: Typo in branch: regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156107 [13:06:23] (03CR) 10Hashar: [C: 032] Typo in branch: regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156107 (owner: 10Hashar) [13:06:32] (03Merged) 10jenkins-bot: Typo in branch: regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156107 (owner: 10Hashar) [13:20:25] (03PS1) 10Hashar: Fix up mediawiki branch regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156109 [13:20:42] (03CR) 10Hashar: [C: 032] Fix up mediawiki branch regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156109 (owner: 10Hashar) [13:20:51] (03Merged) 10jenkins-bot: Fix up mediawiki branch regex [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156109 (owner: 10Hashar) [13:31:52] hashar: what are jenkins-slave and jenkins-deploy users used for? [13:32:32] they are the users used by the Jenkins master to connect to production slaves (jenkins-slave) and labs slaves (jenkins-deploy) [13:32:38] the jobs are being run under those credentials [13:32:59] okay, and role::ci::slave::labs::common doesnt actually install jenkins does it? [13:34:23] just the dependencies [13:34:28] Jenkins itself is unneeded [13:34:42] the slaves just need some java, a ssh connection and the jenkins-deploy user [13:35:00] ahhh, okay [13:35:05] hmmm [13:35:10] due to jobs workspaces taking a fair share of disk space, I got the labs extended disk space allocated with lvm under /mnt (iirc) [13:35:21] and pointjenkins-deploy home to /mnt/jenkins-deploy with workspace under there [13:35:26] that is a bit of a mess :( [13:35:56] meh, we did a similar thing on our instance [13:36:21] so, instance all created [13:36:51] I guess I have to add a few scripts that we use to the jenkins repo, and also add our stuff to jjbc? [13:37:09] we can create a dedicated orphan branch in jjbc [13:37:14] something like 'wikidata' [13:37:21] and deploy jobs from there [13:37:27] we did that when migrating from cloudbees [13:37:37] then merge the branch whenever the integration work has been completd [13:37:51] the scripts should go in integration/jenkins.git , usually under /bin/ [13:38:07] that repo is maintained up to date by puppet under /srv/deployment/integration/slave-scripts/ [13:38:07] yup, ill give them a prefix of wdjenkins, as some are similar to others in there [13:38:19] so /bin/wikidata.sh will be made available as /srv/deployment/integration/slave-scripts/bin/wikidata.sh [13:39:07] yeah you can prefix the new jobs if you want to add them in the 'master' branch of jjbc [13:43:17] (03PS1) 10Addshore: WIP Add 2 scripts used by wikidatajenkins [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 [13:43:51] hashar: ping [13:44:12] Krinkle: pong [13:44:34] Krinkle: Iguess you are back from vacations and I have missed our 1/1 hangout :D [13:44:42] It's now. [13:44:51] (if you want) [13:46:06] Krinkle: if google let me in :D [13:46:48] no, not that kind of hangout [13:46:53] I can't do a/v right now [13:47:29] lets IRC so :D [13:48:27] So.. what's up lately? Last 3-4 weeks [13:50:08] Krinkle: Trusty instances in labs! [13:50:14] puppet is fixed [13:50:33] the only remaining item is php-parsekit which is used to quickly lint php files [13:50:38] that brings up a new jsduck version [13:50:43] new Doxygen version [13:50:44] etc [13:50:46] nodejs [13:50:47] npm [13:51:14] I have added two new labels in Jenkins: UbuntuPrecise and UbuntuTrusty [13:51:43] it is only used for hhvm jobs for now [13:51:55] the npm job might run just fine there [13:52:36] the jsduck-publish , I think its i hardcoded to run only on gallium. Will need to have them run on Trusty labs instance then use rsync to integration-doc-rsync [13:52:57] another item: Zuul got upgraded to match upstream version [13:53:24] and my Zuul cloner has been merged upstream and OpenStack is going to use it for the tests. They even submitted a few patches last week [13:53:45] hashar: So what has switched to Precise for Jenkins right now, only hhvm? [13:53:47] (Zuul cloner let us clone multiple repositories and checkout the appropriate zuul ref / branch of submitted patch [13:53:57] right only hhvm [13:54:06] OK. I'll look at npm jobs soon. [13:54:20] I think they will be just fine [13:54:34] yeah, but we have unpuppetised things for npm instances [13:54:36] the jsduck / jsduck-publish and doxygen jobs need a bit of adaptation [13:54:42] plain Precise isn't enough. I've documented this on the bug [13:54:50] iirc the unpuppetized stuff is to bump nodejs / npm package on Precise [13:54:58] whereas on trusty they are provided by the debian packages [13:55:03] It's also to install the grunt-cli and jshint package. [13:55:09] the instance is integration-slave1006-trusty.eqiad.wmflabs [13:55:18] via npm ? [13:55:30] If we get slave-scripts in labs, we could add ./bin to PATH maybe. [13:55:33] That'd be interesting. [13:55:53] definitely [13:55:58] not sure where to define the PATH though [13:56:06] will eventually have to do it for thephp jobs [13:56:09] using the full path to /srv/slave-scripts/bin is not an option (like we do right now) because the code needs to work locally with plain npm install as well. [13:56:15] so we can easily switch between Zend and HHVM flavors of php [13:56:25] something like: PHP_FLAVOR=hhvm php foobar.php [13:57:11] dont you have grunt-cli and jshint deployed via integration/jenkins.git ? [13:58:00] hashar: Letting features aside for a minute, there's still two outstanding major issues with the current setup: 1) zuul/jenkins is often idle with 0 or only 1 job executing. There's something somewhere often keeping things blocked. 2) We still have Jenkins-slave connection crashing almost every week requiring a restart. [13:58:07] hashar: jshint, yes, grunt-cli no. [13:58:12] but neither is in labs or in PATH. [13:58:22] Right now we only use that jshint bin for check jobs without npm [13:58:24] jshint is in integration/jenkins.git apparently [13:58:34] ahhh [13:58:45] Yes, but not used by anything other than untrusted/check pipeline. [13:58:57] maybe tweak NODE_MODULES path to look up in /srv/deployment/integration/slave-scripts/node_modules so ? [13:59:13] No, that doesn't help. [13:59:20] IN fact, that will damage things. [13:59:34] on labs we can just use npm I guess [13:59:49] 1) it won't make it available on the shell, where we need it (grunt), 2) will add modules to npm environment thus causing things to work that should fail when done in production or locally. [14:00:41] This is not to provide modules to the source code, but to install the base application we use to run the tests themselves (in this case, grunt, to be installed via `npm install -g grunt-cli` on each node) [14:01:23] pinned at a specific version. Perhaps we can just do a puppet Exec['npm install -g grunt-cli@0.1.13']{ ensure= >once } [14:01:26] or something like that [14:01:35] possibly [14:01:35] exists => /usr/something/node_modules/grunt-cli [14:01:59] maybe we can trick ori in writing a npm provider for puppet [14:02:19] ie: package { 'grunt', provider => npm, ensure => 0.1.13 } [14:02:25] will need a bunch of ruby to be written [14:02:31] That'll introduce more classes and dependencies. I think we can keep this simple. There isn't going to be any more of these. [14:02:39] Everythign is handled by npm, this is only for the grunt-cli itself. [14:02:52] in that case, exec is probably enough [14:02:54] I'm all for abstraction, but I think it's overkill for this one. [14:03:01] not sure how to make puppet update the version though [14:03:08] but that can be figured out [14:03:28] exec {} has a unless => parameter, so we can have unless => verify the version currently deployed [14:03:48] Yeah. [14:03:50] puppet would only update grunt-cli if the result of 'unless' is False [14:03:52] Does it support exec itself? [14:04:14] https://docs.puppetlabs.com/references/latest/type.html#exec-attribute-unless [14:04:55] as for the issues you talked about [14:05:21] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #214: ABORTED in 2 min 16 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/214/ [14:05:34] """1) zuul/jenkins is often idle with 0 or only 1 job executing""" : Zuul does merge commits of the proposed patchset, to do so, it has to 'git remote update' it is local repository [14:05:35] exec { "npm install -g grunt-cli@0.1.13": unless => "which grunt && ( grunt --version | grep grunt-cli ) | matches ' v0.1.13$', path => [ "/usr/local/bin", "/usr/bin", "/bin" ], ", require => Class[ "nodejs" ] [14:05:42] that's what jQuery uses in their puppet manifests [14:05:49] (simplified, sans syntax errors) [14:06:07] on mediawiki/core that cause git to send all the zuul ref to gerrit to compute a diff. With a few hundred of thousands of zuul ref that took ~ 10 - 20 seconds . [14:06:15] the bug https://bugzilla.wikimedia.org/show_bug.cgi?id=68481 [14:06:34] I have worked around it by removing some old references in the most busy repositories ( mobilefrontend / puppet / mediawiki core) [14:06:51] exec { "npm install -g grunt-cli@0.1.13": unless => "which grunt && ( grunt --version | grep 'grunt-cli v0.1.13)', path => [ "/usr/local/bin", "/usr/bin", "/bin" ], require => Class[ "nodejs" ] [14:07:05] the long term fixes: a) have zuul create references that uses the year/month in it so we can easily garbage collect them. b) pool in a second Zuul merger. [14:07:27] I"m still confused why we need the Zuul merger. [14:07:41] It seems like a lot of overhead and complexity for something so simple. [14:07:46] Why can't gearman slaves fetch the commit directly? [14:07:47] Krinkle: that exec{} might just work :] [14:07:56] from gerrit [14:08:04] (or the gerrit replication) [14:08:18] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #185: ABORTED in 3 min 20 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/185/ [14:08:35] the commit does not exist in Gerrit [14:08:47] Zuul test the proposed patchset merged with the tip of the branch [14:08:49] It's done in parallel for each job, so while it's slightly duplicated, it's not slowed down O(N). [14:09:01] that used to be done by the Zuul scheduler but did not scale for OpenStack [14:09:13] so they decoupled the merging process as a worker [14:09:17] The proposed commit exists in Gerrit. The repos would do what all ci systems, jenkins, travis-ci, do: fetch target latest branch, fetch commit, merge, run tests [14:09:19] so they can just add more merger process [14:09:25] Right [14:09:41] But it does centrally now, becoming a bottle neck. [14:09:45] Anyway, that's fine, I trust openstack :) [14:09:51] the reason the merge is done before triggering the jobs is to make sure that all jobs use the exact same merge commit [14:09:55] But why does it cause commits to be missing sometimes? Is there a race condition or something? [14:10:00] (03CR) 10Addshore: [C: 04-1] "wdj-get-client-extensions.sh will not be needed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 (owner: 10Addshore) [14:10:15] It happens like once a day or so that there's a patch set and jenkins fails because "the commit doesn't exist" [14:10:16] if the merge operations was being done in the job, there is a tiny chance that the tip of the branch got updated between two job runs of the same patchset [14:10:27] RIght [14:10:33] Fine by me, but sure. [14:10:41] but yeah, that adds to the complexity [14:10:48] and the merger runs merges in sequence [14:10:56] with git remote update taking a bunch of time [14:11:02] and thus blocking the merge serie :( [14:11:12] latest at time of running is good enough. I don't think it was a problem to have jobs potentially run different versions. [14:11:19] we can probably have it run merges in parallel if they are being done on different projects [14:11:29] it's not bad to have it, but itself is imho not reason to do it that way, but anyway, moving on. [14:11:46] "the commit doesn't exist" I am not aware of the issue [14:12:02] probably need a bug report to be filled for investigation [14:12:04] It sounds like something that is a very low priotity nice to have that can only be justified if it doesn't cause bottle necks and stagnations, e.g. only if done well and lightweight. [14:12:08] unlike what we're seeing.. [14:12:38] I wanted to add a second Zuul merger process on lanthanum [14:12:48] but that server is in production and has a private IP (10.x.x.x) [14:12:54] which is not reachable from Labs instances :D [14:13:21] Can we give labs slaves their own zuul merger? [14:13:23] (03PS2) 10Addshore: WIP Add 2 scripts used by wikidatajenkins [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 [14:13:23] so I might end up creating labs instance that handle merge process [14:13:28] or [14:13:46] e.g. use two mergers in production for production jenkins slaves, and another set of one or two merges in labs for labs slaves. [14:13:47] have Zuul merger running on real hardware inside the labs infrastructure (i.e. not instances) [14:14:00] the production slaves might be able to fetch from labs instance [14:14:01] s [14:14:04] haven't looked [14:14:09] yeah, but that's dependency risk. [14:14:34] unless we don't have any jobs running on production slaves anymore [14:14:46] (03PS1) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [14:14:48] when Zuul scheduler ask for a merge process, a merger worker grab the job and reply back with its git url (something like git://gallium.wikimedia.org/mediawiki/core.git ). That url is passed to jobs as ZUUL_URL [14:15:04] the only one currently with a hard dependency is doc publish right? How does that proxy work you set up? Does the gallium node need a git tree? [14:15:05] so the jobs just git clone $ZUUL_URL/$ZUUL_PROJECT [14:15:21] (03CR) 10jenkins-bot: [V: 04-1] WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [14:15:22] the publish doc is scary [14:15:35] since we can't have labs instance push via rsync the docs directly to gallium [14:15:37] I've seen the rsync logic. [14:15:43] they push it to integration-doc-publisher [14:15:52] then the job trigger a job on gallium which rsync from integration-doc-publisher [14:16:03] Yep, but does it get triggered? E.g. does zuul/german execute the main job on labs or gallium? [14:16:04] that is a bit overkill :/ [14:16:16] how does it get triggered* [14:16:19] zuul executes the doc-publish job which generate the doc somewhere [14:16:26] then the jenkins jobs trigger a publishing job [14:16:31] which is tied on gallium [14:16:36] so there's two jobs? [14:16:38] I should really document it [14:16:39] yeah [14:16:47] let me fill a doc bug [14:17:22] ugh, I can't wait for the VM set up. That will force us to better separate things and not even make it possible for most hacky things to exist. [14:17:33] (03PS2) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [14:18:23] (03CR) 10jenkins-bot: [V: 04-1] WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [14:19:12] one job per pipeline/repo, hot standby provisioned VMs based on one of one or more templates for ci slaves, all execution logic in one place running in sequence (parallelising only individual scripts if needed, not entire jobs, much like Travis CI). I'm not sure whether you want to keep the jobs separate for speed or whever that is a side-effect. I think we did them as separate jobs (e.g. -lint, -phpuni [14:19:13] t etc.) not for speed reasons, but for job config reasons. [14:19:24] doc bug filled as https://bugzilla.wikimedia.org/show_bug.cgi?id=69975 [14:19:32] 3Wikimedia / 3Continuous integration: Document CI doc publishing process - 10https://bugzilla.wikimedia.org/69975 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3Antoine "hashar" Musso Documentation is being generated on labs slave by some -publish jobs. They rsync the generated documentation... [14:19:42] (03PS3) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [14:20:02] Would you principally be opposed to combining more jobs and going for a more uniform approach (like with -npm, but for non-js projects as well, perhaps a .travis.yml-like file) [14:20:26] also Krinkle "2) We still have Jenkins-slave connection crashing almost every week requiring a restart." <-- is there a bug for it? I am unaware of that issue [14:20:40] hashar: You can't be unaware of that. [14:20:49] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #215: ABORTED in 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/215/ [14:20:51] hashar: You wrote the docs to engineering-l on how to restart it [14:20:56] ahh [14:21:01] hashar: e.g. gallium going crazy, requiring slave/node disconnect/restart. [14:21:05] ah yeah [14:21:08] Where all jobs are queued in zuul. [14:21:16] so that is not really the jenkins master / slave ssh connection being wonky [14:21:20] there is two issues: [14:21:39] last I checked we had no idea what it was and require a restart as temp measure until we had time to figure it out [14:21:39] a) the Jenkins Gearman plugin is apparently not very compatible with latest Jenkins LTS version. OpenStack run some old version [14:22:11] b) Zuul on reconfiguration has a cache invalidation issue that can cause the whole scheduler / queue to be borked. A patch got merged for that and I believe it will fix that specific issue. Gotta upgrade zuul [14:22:33] I can definitely reproduce it when deleting/creating a new job [14:22:37] gearman get stalled entirely [14:22:43] the jobs are unregistered from zuul entirely [14:23:02] and since the Gearman server does not know about the jobs, Zuul just enqueue them waiting for workers to grab the jobs [14:23:16] that is fixed by restarted Jenkins or disconnecting the Jenkins gearman plugin and reconnecting it [14:23:18] which is cumbersome [14:23:49] as for combining jobs, yeah I like the approach you have taken with the -npm jobs [14:24:02] though that means developers needs to know what needs to be added (e.g. add the banana checker [14:24:28] we can probably come up reusing the .travis.yaml definition by installing travis command line utility [14:26:36] anyway for this week I want to use Zuul cloner on some more jobs [14:26:43] the mediawiki-core-qunit is the next [14:26:53] will let us clone mediawiki/core and mediawiki/vendor together [14:27:19] I have already phased out the mediawiki-core-phpunit-{x,y,z} jobs for the master and REL* branches [14:27:31] in favor of using the new mediawiki-vendor-integration which run the whole phpunit test suite [14:28:12] hashar: is there anything you can see wrong with https://gerrit.wikimedia.org/r/#/c/156114/3 ? [14:28:17] that was actually surprisingly easy! [14:29:16] oh, actually, I just noticed something wrong :D [14:31:58] Krinkle: also we might want to decouple the jshint and json linter which are both in -jslint jobs [14:32:04] (03PS4) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [14:32:08] Krinkle: some repositories do not pass jshint and could use json lint [14:33:37] (03CR) 10jenkins-bot: [V: 04-1] WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [14:35:37] bd808: good morning :] [14:37:16] !log deployment-prep /var/log/upstart/hhvm.log is filled with hphp notices (Note: unable to serialize ... and others). That cause / to fill up [14:38:48] (03PS5) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [14:41:21] 3Wikimedia Labs / 3deployment-prep (beta): hhvm fill up /var/log/upstart/hhvm.log - 10https://bugzilla.wikimedia.org/69976 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3None deployment-mediawiki02 labs instance only has 2GB of disk. The hhvm upstart process emits log to /var/log/upstart/hhv... [14:43:46] 3Wikimedia Labs / 3deployment-prep (beta): hhvm fill up /var/log/upstart/hhvm.log - 10https://bugzilla.wikimedia.org/69976#c1 (10Chris McMahon) s:5normal>3major changed importance to "major", this makes beta hard to use for real people and causes a lot of false test failures. [14:46:01] 3Wikimedia Labs / 3deployment-prep (beta): hhvm fill up /var/log/upstart/hhvm.log - 10https://bugzilla.wikimedia.org/69976#c2 (10Antoine "hashar" Musso) Bryan told me the log issue is apparently fixed in production. Since mediawiki02 does not run puppet, it is not taking in account the recent changes made i... [14:46:10] (03PS1) 10Addshore: WIP add new zuul triggers for new wikidata jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156120 [14:47:08] hashar: think we could try these three patches and adding the slave today? :) (if your not too busy) [14:55:51] 3Wikimedia Labs / 3deployment-prep (beta): deployment-mediawiki02 enwiki: Memcached error: Error connecting to 127.0.0.1:11211: Connection refused - 10https://bugzilla.wikimedia.org/69978 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3None For some reason the hhvm process on deployment-mediaw... [14:56:38] addshore: hop sorry tomorrow :-( [14:56:47] kk =] [14:56:55] addshore: beta is in some bad shape :-/ [14:56:59] :D [15:04:46] 3Wikimedia Labs / 3deployment-prep (beta): deployment-mediawiki02 enwiki: Memcached error: Error connecting to 127.0.0.1:11211: Connection refused - 10https://bugzilla.wikimedia.org/69978#c1 (10Antoine "hashar" Musso) The MediaWiki configuration cache is in /tmp/mw-cache-master which might end up having nul... [15:06:03] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979 (10Antoine "hashar" Musso) 3NEW p:3Unprio s:3normal a:3None On deployment-mediawiki02:/tmp -rw------- 1 apache apache 625M Aug 22 22:37... [15:14:44] (03PS16) 10Addshore: Create phpcs standard for MW core compatibility [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/153399 [15:15:33] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c1 (10Antoine "hashar" Musso) The instance has some local disk space allocated under /srv/ (via puppet class role::labs::lvm::srv ). Would be a... [15:15:54] hashar: hello ! [15:16:47] How can I add another variable like mediawiki_upload_url for language screenshot job; [15:16:47] Problem is other jobs use the same build script and I have to set mediawiki_upload_url in other jobs too [15:16:47] Is there any work around for this ? [15:17:01] oops ! that was weird [15:18:09] hashar: Is there any way, I can set the variable/key only to a particular job and use it in macro, which is used by other jobs ? [15:18:21] not sure :-] [15:18:25] depends on what you want to do hehe [15:18:36] in jenkins job builder, whenever you define a variable like "{foobar}" [15:18:45] it has to be passed a value or jjb will not expand it [15:18:50] which can cause some side effects [15:19:02] it might even complain and bails out stating it needs a value for variable [15:19:54] hashar: Yup ! I saw it complaining [15:20:11] vikasyaligar: what other variable do you need? [15:20:19] if [[ ! -z $LANGUAGE_SCREENSHOT_CODE ]]; then [15:20:19] export MEDIAWIKI_API_UPLOAD_URL=https://commons.wikimedia.org/w/api.php [15:20:19] "$WORKSPACE"/vendor/bin/bundle exec ruby1.9.1 upload.rb [15:20:23] that one is nice :-] [15:20:37] hashar: ah ! you found it out :) [15:21:12] I am planning to create another job, which uploads to commons beta labs [which can be used for testing] [15:21:33] so the MEDIAWIKI_API_UPLOAD_URL changes to beta labs commons [15:21:59] we had that issue to set the browser version to use [15:22:07] we ended up with a case / esac statement [15:22:24] which, based on a the $BROWSER set, export the proper VERSION [15:23:09] anyway, what would be possible is to have the job accept a MEDIAWIKI_API_UPLOAD_URL parameater [15:23:18] set by default to https://commons.wikimedia.org/w/api.php [15:23:33] so whenever you manually run the job, you can override that parameter in the Jenkins build form [15:24:52] jenkins build form => jenkins config? [15:25:27] depends on what you want to do [15:25:42] I would reuse the existing job [15:25:56] and offer the opportunity to manually override the MEDIAWIKI_API_UPLOAD_URL when needed [15:26:56] vikasyaligar: http://ci.openstack.org/jenkins-job-builder/parameters.html :) [15:27:19] you can define a string property MEDIAWIKI_API_UPLOAD_URL with default value of https://commons.wikimedia.org/w/api.php [15:27:34] that will be exported by Jenkins as a shell env [15:27:36] and [15:28:04] when triggering a build manually Jenkins will show a form asking to fill the parameter (with the field prepopulated with the default value [15:28:10] so you can override it as needed [15:28:20] I think the issue might be that all of our URL env vars are built off a single root URL. Zeljko and I talked about this briefly. [15:29:21] hashar: thank you ! Will try it out :) [15:29:22] chrismcmahonbrb: the upload one is hacked :-] [15:29:30] chrismcmahonbrb: yup ! I agree [15:45:24] addshore: got some bandwidth now [15:45:44] hashar: :D [15:45:48] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c2 (10Bryan Davis) The latest production puppet code for setting up hhvm moves the cores to /var/log/hhvm. We need to get deployment-mediawiki02... [15:46:08] addshore: bah composer :( [15:46:17] but its on labs! [15:46:48] we can do something smarter :-] [15:46:59] assuming all the composer dependencies are on wikimedia Gerrit [15:47:04] we can clone them all :D [15:47:05] nope :p [15:49:20] addshore: how do you get all that deployed on wikkimedia prod ? [15:49:22] surly composer isn't an issue here as it is never running on the cluster, only on this labs instance, which is no different to what is currently being done [15:49:54] hashar: we have a build of the Wikidata product / extension with all deps every morning [15:50:02] also done on labs [15:50:08] in mediawiki/extensions/Wikidata ? [15:50:16] yup [15:50:45] hence jenkins already runs regular extension unit tests for that repo [15:50:54] It would be awesome to have that done in Jenkins, update, test and promote if tests pass [15:51:14] bd808: yeah that is evil plan 2. Get rid of wmf branches and submodules [15:51:32] :P [15:52:08] https://github.com/wmde/puppet-builder and https://github.com/wmde/WikidataBuilder [15:52:18] cause in theory we can run all extensions tests together and if the combinaison pass, have those commits registered as submodule of some git repo and deploy from there [15:52:26] mediawiki/integrated.git ! [15:52:35] which would supersede the wmf branches in mediawiki/core [15:53:15] addshore: You wmde folks are just showing off now. :) [15:53:35] hehe, this has been around for at least half a year now ;p [15:53:45] but yes, it is very nice ;p [15:54:09] MWF only likes to do things that are 5-10 years old. ;) [15:54:32] Change is scarry [15:54:43] change is the only way to move forward ;p [15:54:46] some are very conservatives indeed :-( [15:55:22] on my side I blame lack of focus ™ [15:56:16] (03PS1) 10Vikassy: Environment variable MEDIAWIKI_API_UPLOAD_URL can now be set using build parameter in language screenshot job [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/156138 [15:57:09] soo, is composer blocking this hashar >.>? :< [15:57:10] addshore: so the crazy job https://gerrit.wikimedia.org/r/#/c/156114/ is going to run on your instance right ? :D [15:57:24] yes hashar :) [15:57:37] node: wdjenkins [15:57:46] ah [15:57:50] and guessing I could then add 2 instances to that group? [15:58:01] sure thing [15:58:03] 3Wikimedia Labs / 3deployment-prep (beta): deployment-mediawiki02 enwiki: Memcached error: Error connecting to 127.0.0.1:11211: Connection refused - 10https://bugzilla.wikimedia.org/69978#c2 (10Bryan Davis) From mediawiki-config/wmf-config/mc-labs.php: $wgObjectCaches['memcached-pecl']['servers'] => array... [15:58:11] only 1 way to find out if it works ;p [15:58:21] definitely [15:58:23] though [15:58:34] for composer, I would avoid reinstalling it from upstream over and over [15:58:46] in favor of having it pre installed on the instances [15:59:12] well, I guess I could add https://github.com/wmde/puppet-composer :P [15:59:38] ahaha [15:59:45] that is not evil enough! [15:59:57] you could have written some ruby to add a 'composer' provider to puppet package {} stanza :-D [16:00:07] but yeah that would work [16:00:29] :P [16:00:43] will add composer now and poke a thing or two [16:06:53] (03CR) 10Hashar: [C: 04-1] "bunch of nitpicks :-)" (035 comments) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [16:07:01] addshore: did a few nitpicks [16:07:46] 3Wikimedia Labs / 3deployment-prep (beta): bits.beta.wmflabs.org down with 503 - 10https://bugzilla.wikimedia.org/69921#c5 (10physikerwelt) 5RESO/FIX>3REOP It worked fine this morning but now I'm getting 503 again. It's somehow hard to decide when a Temporarily bug has been fixed. [16:08:08] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c3 (10Chris McMahon) multiple HHVM cores per day seems like a real problem [16:08:30] addshore: you will need the puppet class "role::ci::slave::labs " applied on the instance [16:08:42] addshore: that does a bunch of weird thing such as installing all mediawiki packages dependencies [16:08:49] addshore: and create a /srv/ disk using lvm [16:08:53] or /mnt can't remember [16:08:56] as well as role::ci::slave::labs::common ? [16:09:04] the common is included in role::ci::slave::labs [16:09:36] role::ci::slave::labs::common is the very basis [16:09:50] it is used on beta cluster for slaves that takes care of updating parsoid / cxserver etc [16:09:59] running puppet now [16:10:14] those nodes don't need the mediawiki packages or the contint specific packages [16:10:24] that is going to take roughly an hour per node [16:10:30] since it apt-get install a bunch of thing [16:10:42] such as texlive , php-* packages etc [16:10:57] from there, drop me by email the IP address of the instances and I will add them to the pool of slaves tonight [16:10:59] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #186: ABORTED in 1 hr 50 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/186/ [16:11:13] and we can generate the jobs tomorrow + add the zuul trigger and see what happens :] [16:11:40] cool cool :) [16:12:52] Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Package[php5-cli] :/ [16:12:56] silly puppet [16:14:02] (03CR) 10Hashar: [C: 04-1] "Trick: you can use the 'experimental' pipeline while the job are being setup. Will let us manually trigger them instead of on each patch " (031 comment) [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156120 (owner: 10Addshore) [16:14:08] and another trick there [16:14:12] experimental: pipeline [16:14:17] let you trigger jobs manually [16:14:22] by commenting in Gerrit "check experimental" [16:14:24] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #216: ABORTED in 3 min 21 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/216/ [16:14:29] that is ideal when playing with jobs [16:14:30] nice :) [16:15:10] undocumented feature obviously (see bug: 1) [16:16:46] 3Wikimedia Labs / 3deployment-prep (beta): hhvm creates core file in /tmp/ filling mediawiki02 labs instance root partition - 10https://bugzilla.wikimedia.org/69979#c4 (10Bryan Davis) (In reply to Chris McMahon from comment #3) > multiple HHVM cores per day seems like a real problem Likely some new to us hh... [16:16:52] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #187: ABORTED in 5 min 28 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/187/ [16:19:13] hashar: can you review this(https://gerrit.wikimedia.org/r/#/c/156138/) [16:20:03] 3Wikimedia Labs / 3deployment-prep (beta): deployment-mediawiki02 enwiki: Memcached error: Error connecting to 127.0.0.1:11211: Connection refused - 10https://bugzilla.wikimedia.org/69978#c3 (10Antoine "hashar" Musso) Yeah I though it was related to the /tmp/mw-cache-master being dirty (0 bytes files). But... [16:22:18] beta is still bogged down somehow, very slow [16:24:11] :( [16:24:35] (03CR) 10Hashar: [C: 04-1] "almost :)" (032 comments) [integration/jenkins-job-builder-config] (cloudbees) - 10https://gerrit.wikimedia.org/r/156138 (owner: 10Vikassy) [16:24:49] vikasyaligar: reviewed :D [16:24:58] addshore: vikasyaligar: I am off to home. Will follow up tomorrow [16:25:09] awesome :) [16:25:16] hashar: thank you ! :) [16:27:51] addshore: dont forget to mail me the IP and names of the instances :D [16:27:57] will pool them tonight [16:28:31] (03PS6) 10Addshore: WIP add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [16:29:53] bd808: do you know where we stand on beta labs right now with performance? I'm seeing 20+ seconds to load a small page in VE just now, with stuff from meta and bits the worst offenders [16:30:21] (03PS2) 10Addshore: WIP add new zuul triggers for new wikidata jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156120 [16:30:43] chrismcmahon: We are still pinned to a single mw backend server (deployment-medaiwiki02) with a hand built config. _joe_ and ori need to fix that [16:31:03] oi [16:31:58] hashar: sent the email, naturally puppet is still running ;p [16:33:07] addshore: obviously :] [16:33:08] thanks [16:34:01] chrismcmahon: I pinged _joe_ about it this morning and he said "'ll take a look this week, although I'm on RT duty, which may mean a lot of wasted time" [16:34:44] chrismcmahon: You could use your powers of persuasion to convince him to bump it up on his priority list :) [16:36:34] bd808: 20-second page load times make it basically unusable for most purposes. I am really regretting putting HHVM on beta at all, this has been very expensive. [16:39:24] addshore: your labs project needs to allow sshconnection from gallium; I replied to your mail with details :D [16:39:28] :> [16:39:30] ssh connection times out hehe https://integration.wikimedia.org/ci/computer/wdjenkins-node1/log [16:39:34] chrismcmahon: I agree that it has been supported badly. Not much I can do about that then or now. Raise a stink and pound your fist with the people who are supposed to be managing this (ori, _joe_) [16:39:47] yep [16:39:52] addshore: the jenkins master will ssh to the instance using the jenkins-deploy key [16:40:26] addshore: so you might also have to add jenkins-deploy user to the project but I think that needs to be done directly in LDAP by an ops root [16:41:18] addshore: or maybe add the labs account Jenkins-deploy to the project [16:41:20] might be enough [16:42:43] added the port / range to the security group [16:43:49] also added Jenkins-deploy hashar :) [16:45:28] [08/25/14 16:45:14] [SSH] Authentication successful. [16:45:28] [08/25/14 16:45:19] [SSH] The remote users environment is: [16:45:29] bah [16:45:34] that is almost too easy [16:45:38] Slave successfully connected and online [16:45:51] :p [16:46:10] it should have copied a bunch of .jar under /mnt/jenkins-workspace [16:46:18] that is also where the job will be run [16:46:28] hopefully /mnt/ is an extended disk partition [16:46:31] will check after the puppet run :P [16:46:38] yeah [16:46:52] we can check that tomorrow [16:46:54] + deploy jobs [16:47:00] and add them to experimental pipeline [16:47:22] now I am really off [16:47:33] :P [16:47:39] bd808: I found _joe_ in #mw-core, send him a prompt about deployment-mediawiki02 busted since Friday [16:48:23] * hashar waves [16:48:40] chrismcmahon: cool beans. [16:55:37] Project beta-scap-eqiad build #18740: FAILURE in 1 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18740/ [17:04:58] Yippee, build fixed! [17:04:59] Project beta-scap-eqiad build #18741: FIXED in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18741/ [18:05:15] Project browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs build #98: STILL FAILING in 15 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs/98/ [18:14:41] Project browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce build #95: STILL FAILING in 10 min: https://integration.wikimedia.org/ci/job/browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce/95/ [18:16:29] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #66: STILL FAILING in 1 min 47 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/66/ [18:16:47] Yippee, build fixed! [18:16:48] Project browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce build #2: FIXED in 10 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce/2/ [18:18:41] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #122: SUCCESS in 2 min 11 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/122/ [18:20:39] Project browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce build #123: STILL FAILING in 1 min 57 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce/123/ [18:21:19] Project browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce build #108: STILL FAILING in 20 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce/108/ [18:23:47] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce build #133: STILL FAILING in 2 min 27 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/133/ [18:27:31] Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #99: STILL FAILING in 3 min 43 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/99/ [18:29:30] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #179: STILL FAILING in 12 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/179/ [18:32:30] Project browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #128: STILL FAILING in 11 min: https://integration.wikimedia.org/ci/job/browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/128/ [18:33:45] Project browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce build #95: SUCCESS in 1 min 14 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce/95/ [18:33:47] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #15: STILL FAILING in 2.2 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/15/ [18:33:58] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #158: STILL FAILING in 10 sec: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/158/ [18:34:03] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #217: STILL FAILING in 3.7 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/217/ [18:34:06] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #55: STILL FAILING in 1.5 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/55/ [18:34:09] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #140: STILL FAILING in 1.7 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/140/ [18:34:12] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #163: STILL FAILING in 2.6 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/163/ [18:34:15] Project browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #99: STILL FAILING in 1 sec: https://integration.wikimedia.org/ci/job/browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/99/ [18:36:01] Project browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce build #84: STILL FAILING in 1.3 sec: https://integration.wikimedia.org/ci/job/browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce/84/ [18:37:02] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #2: STILL FAILING in 2.5 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/2/ [18:38:28] FYI beta labs is down, ori is working on a fix in #wikimedia-labs [18:38:52] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_xp-firefox-sauce build #139: STILL FAILING in 11 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_xp-firefox-sauce/139/ [18:40:01] Project browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce build #123: SUCCESS in 1 min 1 sec: https://integration.wikimedia.org/ci/job/browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce/123/ [18:41:01] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #66: STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/66/ [18:41:29] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #98: SUCCESS in 1 min 28 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/98/ [18:42:04] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #188: STILL FAILING in 3.9 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/188/ [18:46:46] Project browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce build #39: FAILURE in 46 sec: https://integration.wikimedia.org/ci/job/browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce/39/ [18:55:03] chrismcmahon, spagewmf, greg-g: robla has suggested that the first thing that should be done when any of us notice that beta is broken that we log it SAL (!log deployment-prep Beta broken) [18:55:07] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #594: SUCCESS in 9 min 6 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/594/ [18:55:15] Then we can track outages better [18:55:45] (and note how it appears it's broken) :-) [18:56:48] bd808: sure, I'll try to remember. [18:56:50] in this case it seems to have been mostly busted since Friday, when I was OOO, so it was only this morning that I discovered it being very slow [19:00:18] chrismcmahon: sounds like three or four issues: core dumps and slow, then today 503s, then everything a 404. The last two seem fixed now \o/ [19:02:39] spagewmf: it shouldn't be slow any more either [19:05:36] I think the root cause this time was we found a new hhvm crashing bug. That filled up /tmp which led to /var being filled. Apache processes went runaway and filled things up again after hashar cleaned up things the first time. [19:06:22] Basically beta lacks basic monitoring and full time sysadmins [19:07:16] yeah [19:07:32] bd808: we have diamond enabled on beta but yuvi/andrew are still working on setting up labsmon [19:07:54] we can then get some dashboard to display the diamond issue and use something to alert on bugged metrics [19:11:51] "I thought Diamond was a decent Spandau Ballet album until I found https://wikitech.wikimedia.org/wiki/Monitoring_package_survey#Diamond " [19:16:33] (03PS7) 10Hashar: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [19:17:47] (03CR) 10Hashar: [C: 031] "Updated commit summary message to reflect jobs being created/updated. Ran Jenkins job builder:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [19:18:11] addshore: ^^^ :D [19:19:06] 3Wikimedia Labs / 3deployment-prep (beta): mwdeploy user has shell /bin/bash in labs LDAP and /bin/false in production/Puppet - 10https://bugzilla.wikimedia.org/65591#c7 (10Tim Landscheidt) On tools-login, /etc/ldap.conf has: | [...] | nss_override_attribute_value loginshell /bin/bash | [...] I haven't rea... [19:21:40] oh hey hashar :P [19:22:26] (03CR) 10Hashar: [C: 04-1] Add jobs for wikibase (031 comment) [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [19:22:40] addshore: there is some typo in JJB [19:22:46] mwext-Wikibase-wikibase-client-tests-tests [19:22:53] double wikibase mentionner in job name [19:23:00] I have commented on https://gerrit.wikimedia.org/r/#/c/156114/7/wdjenkins.yaml [19:25:00] (03PS8) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [19:25:03] =] [19:26:59] addshore: bah [19:27:02] Project browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce build #144: STILL FAILING in 57 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce/144/ [19:27:05] should be simply 'client' hehe [19:27:07] addshore: will amend [19:27:25] (03PS9) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [19:27:25] ^^ [19:27:35] ^^ [19:28:02] (03PS10) 10Hashar: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [19:28:08] :D [19:28:20] 3Wikimedia Labs / 3deployment-prep (beta): mwdeploy user has shell /bin/bash in labs LDAP and /bin/false in production/Puppet - 10https://bugzilla.wikimedia.org/65591#c8 (10Tim Landscheidt) The line (in essence) was introduced by Ryan's commit f8724e60664a33a37a327434f5c3cb71837f4c20 (Sep 13 00:26:36 2011) w... [19:28:47] (03CR) 10Hashar: "$ jenkins-jobs --conf jenkins_jobs.ini update config/ mwext-Wikibase-client-tests" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [19:28:59] (03CR) 10Hashar: [C: 032] WIP add new zuul triggers for new wikidata jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156120 (owner: 10Addshore) [19:29:07] addshore: deploooying [19:29:13] :D [19:29:40] of course creating the jobs in Jenkins caused zuul to be stalled [19:30:37] :P [19:31:31] (03Merged) 10jenkins-bot: WIP add new zuul triggers for new wikidata jobs [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156120 (owner: 10Addshore) [19:31:57] deployed [19:32:35] addshore: you should now be able to comment 'check experimental' on some change [19:32:38] and see what happens [19:32:42] makking a test commit for it :) [19:36:44] (03PS3) 10Addshore: Add script used by wikidatajenkins [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 [19:42:06] Project browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce build #128: STILL FAILING in 55 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce/128/ [19:47:06] Network is unreachable - connect(2) (Faraday::ConnectionFailed) [19:47:06] bah [19:49:21] (03PS1) 10Addshore: Split mw-run-update-script from mw-setup-extension [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156170 [19:50:06] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #113: STILL FAILING in 23 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/113/ [19:51:14] (03PS11) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [19:52:14] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #123: SUCCESS in 2 min 6 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/123/ [19:55:26] (03CR) 10Hashar: [C: 032] "Wikidata related thing. Lets merge that in :)" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 (owner: 10Addshore) [19:55:29] (03Merged) 10jenkins-bot: Add script used by wikidatajenkins [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156110 (owner: 10Addshore) [20:00:22] (03CR) 10Hashar: [C: 04-1] "This change is essentially a noop but change two qunit-mobile jobs:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156170 (owner: 10Addshore) [20:03:06] (03PS1) 10Addshore: -xe for wdj-apply-settings [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 [20:03:20] Yippee, build fixed! [20:03:20] Project browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce build #2: FIXED in 11 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce/2/ [20:04:25] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #168: STILL FAILING in 1 hr 9 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/168/ [20:05:10] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #124: SUCCESS in 1 min 50 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/124/ [20:10:07] Project browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce build #110: STILL FAILING in 27 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce/110/ [20:10:08] (03CR) 10Hashar: [C: 04-1] "make it chmod +x while at it :]" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 (owner: 10Addshore) [20:10:36] Project UploadWizard-api-commons.wikimedia.org build #476: SUCCESS in 27 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.org/476/ [20:11:17] (03PS2) 10Hashar: -xe for wdj-apply-settings [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 (owner: 10Addshore) [20:11:49] 3Wikimedia / 3Quality Assurance: Support MEDIAWIKI_PROXY_URL for browser tests - 10https://bugzilla.wikimedia.org/69725#c1 (10Antoine "hashar" Musso) Maybe it supports HTTP_PROXY and HTTPS_PROXY already? [20:13:32] Project browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce build #125: SUCCESS in 2 min 56 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/125/ [20:14:22] Has anyone setup a Ubuntu environment? I cant seem to use the git review -s in my terminal. I got really far with my setup but I am stuck and need some help [20:14:59] (03PS3) 10Awight: Use the verbose re flag to unminify the email_filter [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/151249 [20:15:06] (03PS12) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [20:15:09] (03CR) 10jenkins-bot: [V: 04-1] Use the verbose re flag to unminify the email_filter [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/151249 (owner: 10Awight) [20:15:19] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #124: SUCCESS in 1 min 46 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/124/ [20:15:25] I can clarify my setup if anyone needs more info on my environment [20:17:27] (03PS3) 10Addshore: -xe shebang and +x for wdj-apply-settings [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 [20:17:34] (03CR) 10Addshore: [C: 031] -xe shebang and +x for wdj-apply-settings [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 (owner: 10Addshore) [20:17:37] (03CR) 10Hashar: "deployed:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [20:18:07] mazza: what error are you getting? [20:18:19] (03PS3) 10Awight: Sort the email_filter literal [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/151250 [20:18:22] (03PS4) 10Awight: Use the verbose re flag to unminify the email_filter [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/151249 [20:20:35] chrismcmahon: antec@antec-ubuntu:~$ ~/virtualenv/bin/git-review -s [20:20:36] Traceback (most recent call last): [20:20:38] File "/home/antec/virtualenv/bin/git-review", line 11, in [20:20:39] sys.exit(main()) [20:20:41] File "/home/antec/virtualenv/local/lib/python2.7/site-packages/git_review/cmd.py", line 1157, in main [20:20:42] raise no_git_dir [20:20:44] git_review.cmd.GitDirectoriesException: Cannot determine where .git directory is. [20:20:45] The following command failed with exit code 128 [20:20:47] "git rev-parse --show-toplevel --git-dir" [20:20:48] ----------------------- [20:20:50] fatal: Not a git repository (or any of the parent directories): .git [20:20:56] (03PS1) 10Awight: DonationInterface passes jslint now [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156179 [20:21:11] sorry if its come out in multiple lines on the chat [20:21:24] (03PS2) 10Awight: DonationInterface passes jslint now [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156179 [20:21:49] 3Wikimedia / 3Quality Assurance: Support MEDIAWIKI_PROXY_URL for browser tests - 10https://bugzilla.wikimedia.org/69725#c2 (10Chris Steipp) You can set that, but that uses the proxy for some Ruby (watir?) connection. The requests are restful requests to localhost:7055. But not for the web requests to MediaWi... [20:22:00] mazza that's OK. you can use http://pastebin.com/ or some such for multi-line IRC posts in the future [20:22:36] mazza: that error is telling you that the directory you are in right now is not a git repository [20:24:14] mazza: what are you trying to accomplish? [20:24:52] chrismcmahon: ok I see. I am trying to finish my ubuntu enviroment so I continue with my automation and help out here [20:24:55] (03PS13) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [20:27:04] chrismcmahon: I am currently following the guide and I am on the part where its asking me to download the Examples extension Git. [20:29:58] chrismcmahon: got it working [20:30:05] Thanks Chris :) [20:31:00] Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #111: STILL FAILING in 26 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/111/ [20:36:10] (03PS14) 10Hashar: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 (owner: 10Addshore) [20:41:20] Project browsertests-Translate-sandbox.translatewiki.net-linux-firefox-sauce build #94: STILL FAILING in 26 min: https://integration.wikimedia.org/ci/job/browsertests-Translate-sandbox.translatewiki.net-linux-firefox-sauce/94/ [20:42:42] Yippee, build fixed! [20:42:42] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #67: FIXED in 1 min 20 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/67/ [20:51:18] (03CR) 10Addshore: [C: 04-1] "going to change the way this is done :)" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 (owner: 10Addshore) [20:54:39] Yippee, build fixed! [20:54:40] Project browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #127: FIXED in 23 min: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/127/ [20:56:07] (03PS15) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [21:07:39] (03PS16) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [21:08:11] Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #141: STILL FAILING in 1 hr 3 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/141/ [21:09:58] (03PS17) 10Addshore: Add jobs for wikibase [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156114 [21:31:26] (03PS1) 10Jforrester: Configure npm pipeline for DonationInterface [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 [21:32:52] (03PS1) 10Jforrester: Enable -npm pipeline voting for DonationInterface repo [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156194 [21:35:07] (03CR) 10Jforrester: "The pipeline was defined in Ib0d0f6c8, now merged." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 (owner: 10Jforrester) [21:35:23] (03CR) 10Jforrester: "The pipeline is configured in I8318fc49 which is not yet merged." [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156194 (owner: 10Jforrester) [21:38:15] (03PS1) 10Addshore: Remove wdj-apply-settings.sh [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156197 [21:38:34] (03Abandoned) 10Addshore: -xe shebang and +x for wdj-apply-settings [integration/jenkins] - 10https://gerrit.wikimedia.org/r/156176 (owner: 10Addshore) [21:47:50] (03CR) 10Awight: [C: 031] Enable -npm pipeline voting for DonationInterface repo [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156194 (owner: 10Jforrester) [21:48:29] (03CR) 10Awight: [C: 031] Configure npm pipeline for DonationInterface [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 (owner: 10Jforrester) [21:49:12] (03CR) 10Awight: [C: 032] Enable -npm pipeline voting for DonationInterface repo [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156194 (owner: 10Jforrester) [21:49:18] (03CR) 10Awight: [C: 032] Configure npm pipeline for DonationInterface [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 (owner: 10Jforrester) [21:49:26] (03Merged) 10jenkins-bot: Enable -npm pipeline voting for DonationInterface repo [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/156194 (owner: 10Jforrester) [21:50:09] sleep sleep [21:51:36] (03Merged) 10jenkins-bot: Configure npm pipeline for DonationInterface [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 (owner: 10Jforrester) [22:50:56] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #189: ABORTED in 46 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/189/ [22:51:02] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #218: ABORTED in 47 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/218/ [22:55:40] Project beta-scap-eqiad build #18774: FAILURE in 1 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18774/ [22:57:04] stupid flakey labs dns [22:57:57] Yippee, build fixed! [22:57:57] Project beta-scap-eqiad build #18775: FIXED in 1 min 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18775/ [23:05:30] Project beta-scap-eqiad build #18776: FAILURE in 1 min 35 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18776/ [23:11:49] In case you didn't see it over on -labs, beta labs is hosed by hhvm, ori is investigating. [23:12:46] HHVM has 100% of cpu, and Apache is trying to cope [23:15:21] Yippee, build fixed! [23:15:21] Project beta-scap-eqiad build #18777: FIXED in 1 min 17 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/18777/ [23:17:09] chrismcmahon: I was vaguely aware. I was just paying attention to scap failures because I upgraded it a few hours ago [23:19:45] Krinkle: BTW, you around to deploy a JJBC patch? Awright +2'ed it earlier, but it needs manual deploying, right? https://gerrit.wikimedia.org/r/156193 [23:21:17] James_F: zuul-config patches are merged and then deployed, jjbc patches must not be merged until they're compiled and published on Jenkins. [23:22:17] (03CR) 10Krinkle: "Please don't merge patches in jjb-config unless you are able, willing and have already compiled and published the config change to Jenkins" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156193 (owner: 10Jforrester) [23:22:46] Krinkle: Yes, I told Adam after he'd +2'ed.