[00:41:01] 3Wikimedia Labs / 3deployment-prep (beta): deployment-graphite.eqiad.wmflabs went away? - 10https://bugzilla.wikimedia.org/71031 (10C. Scott Ananian) 3NEW p:3Unprio s:3normal a:3None The OCG service on beta is crashing because: Host deployment-graphite.eqiad.wmflabs not found: 3(NXDOMAIN) on deploy... [00:46:28] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #722: SUCCESS in 27 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/722/ [00:53:30] Project UploadWizard-api-commons.wikimedia.org build #573: SUCCESS in 29 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.org/573/ [02:52:04] PROBLEM - BetaLabs: Low disk space on /var on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_avail.value (10.00%) [02:59:13] RECOVERY - BetaLabs: Low disk space on /var on labmon1001 is OK: OK: All targets OK [03:15:29] Project browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce build #135: SUCCESS in 11 min: https://integration.wikimedia.org/ci/job/browsertests-TwnMainPage-sandbox.translatewiki.net-linux-firefox-sauce/135/ [03:15:49] Project browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs build #151: SUCCESS in 19 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroBanner-en.m.wikipedia.org-linux-phantomjs/151/ [03:27:21] Yippee, build fixed! [03:27:21] Project browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce build #52: FIXED in 11 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce/52/ [03:28:12] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #115: SUCCESS in 51 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/115/ [03:35:02] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #16: SUCCESS in 6 min 49 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/16/ [03:39:11] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #171: SUCCESS in 4 min 8 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/171/ [03:42:34] Project browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce build #157: STILL FAILING in 41 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-windows_8-internet_explorer-sauce/157/ [03:44:19] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #16: STILL FAILING in 1 min 44 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/16/ [03:45:46] Project browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce build #153: STILL FAILING in 1 min 25 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-language-browsertests.wmflabs.org-linux-firefox-sauce/153/ [03:55:10] Project browsertests-VisualEditor-test2.wikipedia.org-os_x_10.9-safari-sauce build #15: STILL FAILING in 50 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-os_x_10.9-safari-sauce/15/ [03:56:36] Project browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #177: SUCCESS in 10 min: https://integration.wikimedia.org/ci/job/browsertests-Core-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/177/ [03:57:44] Project browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #149: SUCCESS in 1 min 7 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/149/ [04:01:51] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce build #182: SUCCESS in 6 min 39 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-chrome-sauce/182/ [04:06:21] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #229: STILL FAILING in 27 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/229/ [04:07:57] Project browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce build #146: SUCCESS in 1 min 34 sec: https://integration.wikimedia.org/ci/job/browsertests-UniversalLanguageSelector-sandbox.translatewiki.net-linux-firefox-sauce/146/ [04:27:08] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #2: STILL FAILING in 25 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/2/ [04:35:58] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #68: SUCCESS in 8 min 48 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/68/ [04:36:51] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce build #14: STILL FAILING in 28 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/14/ [04:56:47] Yippee, build fixed! [04:56:47] Project browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce build #194: FIXED in 59 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-chrome-sauce/194/ [04:57:34] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #104: SUCCESS in 47 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/104/ [05:01:01] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #207: STILL FAILING in 25 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/207/ [05:04:32] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #190: SUCCESS in 6 min 57 sec: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/190/ [05:07:04] Project browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #148: SUCCESS in 2 min 32 sec: https://integration.wikimedia.org/ci/job/browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/148/ [05:07:06] Project browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce build #114: STILL FAILING in 1.6 sec: https://integration.wikimedia.org/ci/job/browsertests-ContentTranslation-language-stage.wmflabs.org-linux-firefox-sauce/114/ [05:15:33] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #55: SUCCESS in 8 min 26 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/55/ [05:16:28] Project browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce build #173: SUCCESS in 54 sec: https://integration.wikimedia.org/ci/job/browsertests-Translate-meta.wikimedia.org-linux-firefox-sauce/173/ [05:17:51] Project browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce build #147: SUCCESS in 1 min 23 sec: https://integration.wikimedia.org/ci/job/browsertests-CirrusSearch-test2.wikipedia.org-linux-firefox-sauce/147/ [05:18:36] Project browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #115: SUCCESS in 44 sec: https://integration.wikimedia.org/ci/job/browsertests-PageTriage-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/115/ [05:32:49] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #282: SUCCESS in 55 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/282/ [05:44:31] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #213: STILL FAILING in 43 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/213/ [05:45:38] Project browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce build #88: SUCCESS in 1 min 5 sec: https://integration.wikimedia.org/ci/job/browsertests-PdfHandler-test2.wikipedia.org-linux-firefox-sauce/88/ [05:48:09] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #14: STILL FAILING in 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/14/ [06:08:40] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #249: SUCCESS in 50 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/249/ [06:28:39] Project browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce build #177: SUCCESS in 43 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce/177/ [06:32:43] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #162: STILL FAILING in 24 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/162/ [06:39:26] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce build #172: SUCCESS in 6 min 42 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-sauce/172/ [06:42:16] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #218: SUCCESS in 54 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/218/ [06:47:56] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #173: SUCCESS in 5 min 39 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/173/ [06:51:13] Project browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce build #51: SUCCESS in 11 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-firefox-sauce/51/ [06:57:59] Project browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce build #159: STILL FAILING in 29 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-firefox-sauce/159/ [07:04:53] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #15: SUCCESS in 6 min 53 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/15/ [07:05:21] Project browsertests-VisualEditor-test2.wikipedia.org-windows_8-internet_explorer-sauce build #14: STILL FAILING in 17 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-windows_8-internet_explorer-sauce/14/ [07:06:14] Project browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #47: STILL FAILING in 51 sec: https://integration.wikimedia.org/ci/job/browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/47/ [07:13:17] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce build #11: STILL FAILING in 7 min 2 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/11/ [07:16:25] Project browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce build #174: SUCCESS in 3 min 5 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/174/ [07:31:28] Yippee, build fixed! [07:31:29] Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #160: FIXED in 40 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/160/ [07:34:12] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #173: SUCCESS in 2 min 43 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/173/ [07:35:24] 3Wikimedia Labs / 3deployment-prep (beta): deployment-graphite.eqiad.wmflabs went away? - 10https://bugzilla.wikimedia.org/71031#c1 (10Antoine "hashar" Musso) It has been deployed on Sep. 11 16:31 UTC by YuviPanda: Delete deployment-graphite instance. Having statsd/graphite on labs instance did not fit the... [07:38:03] (03PS1) 10Hashar: Reduce browsertests IRC spam [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161432 (https://bugzilla.wikimedia.org/70778) [07:44:02] (03CR) 10Hashar: [C: 032] "Refreshed all browser tests jobs :-)" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161432 (https://bugzilla.wikimedia.org/70778) (owner: 10Hashar) [07:47:04] (03Merged) 10jenkins-bot: Reduce browsertests IRC spam [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161432 (https://bugzilla.wikimedia.org/70778) (owner: 10Hashar) [07:51:22] 3Wikimedia / 3Quality Assurance: Make wmf-insecte only notify the -qa channel of changes in build state - 10https://bugzilla.wikimedia.org/70778#c5 (10Antoine "hashar" Musso) 5PATC>3RESO/FIX Deployed, spam should be reduced now unless I screwed something :-D [07:56:37] 3Wikimedia / 3Continuous integration: Zuul: reporting of successful test runs blocked on unrelated, incomplete test runs - 10https://bugzilla.wikimedia.org/71029 (10Antoine "hashar" Musso) [08:09:22] 3Wikimedia / 3Continuous integration: Zuul: reporting of successful test runs blocked on unrelated, incomplete test runs - 10https://bugzilla.wikimedia.org/71029#c1 (10Antoine "hashar" Musso) I have enabled that behavior yesterday and it is totally unexpected although undocumented/unannounced. The screensho... [08:37:30] (03PS5) 10Vikassy: Environment variable MEDIAWIKI_API_UPLOAD_URL can now be set using build parameter in language screenshot job [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 [08:38:42] hashar: Hello ! [08:40:39] hashar: when you are free can you look into https://gerrit.wikimedia.org/r/157049 [08:42:21] vikasyaligar: add me as a reviewer please :-D [08:42:52] vikasyaligar: nitpick: the commit message is too long :D [08:43:00] the first line should be a 72 chars max [08:43:07] will look at the code in a few [08:43:19] hashar: ok :) [08:43:41] at first glance, that should work [08:44:44] (03PS6) 10Vikassy: Environment variable MEDIAWIKI_API_UPLOAD_URL can now be set using build parameter in language screenshot job [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 [08:46:02] hashar: added a new line; that should make the first line < 72 char :) [08:50:55] vikasyaligar: is that to let one manually trigger the job and use a different URL ? [08:52:24] hashar: It is basically for testing(uploading screenshots to beta labs of commons); The default URL if set to production wikimedia commons. So one can set it manually; rather than making new job for it [08:53:53] (03PS7) 10Hashar: MEDIAWIKI_API_UPLOAD_URL is now a build parameter [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 (owner: 10Vikassy) [08:53:59] rephrased / gave more details [08:54:50] hashar: thank you :) [08:56:05] damn [08:56:09] the macro has been copy pasted [08:56:17] (03PS8) 10Hashar: MEDIAWIKI_API_UPLOAD_URL is now a build parameter [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 (owner: 10Vikassy) [08:57:08] (03CR) 10Hashar: [C: 031] "PS8 remove another occurrence of MEDIAWIKI_API_UPLOAD_URL. The macro has been copy pasted (known issue)." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 (owner: 10Vikassy) [08:57:16] vikasyaligar: jobs being refreshed [09:03:57] hashar: great ! [09:21:25] vikasyaligar: jobs refreshed. [09:24:36] hashar: yay ! thank you ! Let me check [09:28:05] hashar: running it [09:30:53] hashar: one more thing => how do I run the job for a specific language(in matrix project) ? Is there any plugin for that ? [09:31:13] vikasyaligar: I dont think you can [09:56:31] Project browsertests-VisualEditor-language-screenshot-linux-firefox-sauce » ast,contintLabsSlave && UbuntuPrecise build #93: FAILURE in 28 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=ast,label=contintLabsSlave%20&&%20UbuntuPrecise/93/ [10:09:41] Project browsertests-VisualEditor-language-screenshot-linux-firefox-sauce » de,contintLabsSlave && UbuntuPrecise build #93: FAILURE in 41 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=de,label=contintLabsSlave%20&&%20UbuntuPrecise/93/ [10:50:43] vikasyaligar: still around? [10:51:09] vikasyaligar: does the MEDIAWIKI_API_UPLOAD_URL build parameter works for you? [10:51:29] seems so https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=ast,label=contintLabsSlave%20&&%20UbuntuPrecise/93/parameters/ [10:52:00] (03CR) 10Hashar: [C: 032] "Seems to work properly. A build as been triggered manually using the labs URL https://integration.wikimedia.org/ci/job/browsertests-Visual" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 (owner: 10Vikassy) [10:52:55] hashar: yes it did; but the uploading happens only when the cucumber passes(which currently is failing). I will fix that now, then I can confirm it to you :) [10:55:11] vikasyaligar: yeah Jenkins abort the build whenever a process exit with non 0 [10:55:21] and the upload process happens AFTER cucumber [10:55:36] we can work around that potentially [10:55:37] hashar: yup ! [10:55:46] but I think it is better to only upload when all tests/screenshots pass [10:56:06] hashar: I agree it you; it is better to keep it that way. [10:56:10] *with [10:56:20] ensure the screenshots are consistent [10:56:22] (03Merged) 10jenkins-bot: MEDIAWIKI_API_UPLOAD_URL is now a build parameter [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/157049 (owner: 10Vikassy) [10:56:31] VE might well be broken for some reason [10:57:19] oh ! let me check; [10:59:54] vikasyaligar: it is lunch time for me [10:59:55] bbl [11:00:09] ok [11:25:15] Will be back in an hour or something .. [12:45:13] * aude attempts to not break beta :) [13:12:03] Project browsertests-VisualEditor-language-screenshot-linux-firefox-sauce » cs,contintLabsSlave && UbuntuPrecise build #93: FAILURE in 3 hr 44 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-linux-firefox-sauce/LANGUAGE_SCREENSHOT_CODE=cs,label=contintLabsSlave%20&&%20UbuntuPrecise/93/ [14:34:20] (03PS1) 10Hashar: Zuul cloner for extensions is now a macro [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161458 [14:34:37] (03PS2) 10Hashar: Zuul cloner for extensions is now a macro [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161458 [14:39:08] * aude is working to fix beta [14:41:19] (03PS1) 10Hashar: Switch extensions qunit jobs to Zuul cloner [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 [14:43:56] fixed the main page [14:44:27] aude: was it broken? :D [14:44:48] yes [14:44:53] other pages are broken [14:45:01] depending on the wikidata item in use [14:45:11] aude: feel free to announce the fix on qa list. We somehow wonder who is maintaining the beta cluster :] [14:45:19] ok [14:45:30] i need to debug it this time instead of revert [14:45:45] i might revert in a bit and try during the weekend again :) [14:46:19] that will show off that almost anyone can fix it :D [14:46:34] heh [14:50:43] (03PS1) 10Hashar: Mark mw-setup-extension as obsolete [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161460 [14:51:23] (03CR) 10Hashar: [C: 032] Mark mw-setup-extension as obsolete [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161460 (owner: 10Hashar) [14:54:57] it looks like most of hte issues are gone, except we are getting the usual hhvm errors on beta wikidata [14:55:07] 3Wikimedia / 3Continuous integration: Jenkins: Generate KSS docs from LESS and publish to doc.wikimedia.org - 10https://bugzilla.wikimedia.org/58620#c11 (10Antoine "hashar" Musso) Can you reach out to Timo about it? I think for Javascript purposes he would like developers to define npm entry points. So we... [14:55:29] and why http://wikidata.beta.wmflabs.org/wiki/Special:Version doesn't show the latest version of "wikidata" ? [14:57:36] !log Jenkins friday deploy: migrate all MediaWiki extension qunit jobs to Zuul cloner. [14:57:39] Logged the message, Master [14:57:54] aude: bryan wrote a cache for the git version [14:58:01] I think it is done during scap [14:58:17] it basically crawls all .git directory, get the sha1 of the HEAD [14:58:27] and write that down to a .json file which is pushed to the app servers [14:58:34] Special:Version then ends up reading from that json file [15:01:05] i see [15:04:55] (03CR) 10Hashar: [C: 04-2] "That still clones the repo using the git plugin which is used to run the job. Need to update a bunch of paths :-/" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 (owner: 10Hashar) [15:08:40] (03PS2) 10Hashar: Switch extensions qunit jobs to Zuul cloner [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 [15:11:03] (03PS3) 10Hashar: Switch extensions qunit jobs to Zuul cloner [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 [15:32:29] PROBLEM - BetaLabs: Puppet freshness check on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki03.puppetagent.time_since_last_run.value (100.00%) [15:36:55] off for the week-end have a good afternoon [15:37:20] beta is mostly fine, per my email [15:37:33] when ori gets a chance, he might be able to help debug the hhvm issues [15:37:42] (03CR) 10Hashar: [C: 032] Zuul cloner for extensions is now a macro [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161458 (owner: 10Hashar) [15:38:28] (03CR) 10Hashar: "Works for all repository beside VisualEditor https://integration.wikimedia.org/ci/job/mwext-VisualEditor-qunit/11564/consoleFull" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 (owner: 10Hashar) [15:40:37] (03Merged) 10jenkins-bot: Zuul cloner for extensions is now a macro [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161458 (owner: 10Hashar) [16:09:52] 3Wikimedia Labs / 3deployment-prep (beta): wikidata beta (item pages, etc.) inaccessible with 503 errors - 10https://bugzilla.wikimedia.org/69708#c10 (10Aude) 1) restart hhvm 2) maybe do a thing or two (e.g. add a label) 3) click history 4) view a diff and hhvm crashes viewing a diff in vagrant: [Tue Sep 1... [16:24:32] :) 14:57 hashar: Jenkins friday deploy: migrate all MediaWiki extension qunit jobs to Zuul cloner. [16:30:28] greg-g: Except VisualEditor. :-( [16:30:43] James_F: "all" [16:31:03] greg-g: https://gerrit.wikimedia.org/r/#/c/161459/ [16:32:29] * greg-g nods [16:43:09] 3Wikimedia / 3Quality Assurance: Make wmf-insecte only notify the -qa channel of changes in build state - 10https://bugzilla.wikimedia.org/70778#c6 (10Greg Grossmeier) I knew I was going to lose to you getting this done first.... Thanks hashar! :) :) [17:34:35] (03CR) 10Dduvall: [C: 031] "This looks good." [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/160983 (owner: 10Hashar) [18:18:21] hashar: I'm not sure if this is a new error, but it also seems like all jobs using sqlite are broken (not failing, but really badly working) because the database is continuously locked [18:18:39] the tests are passing but basically there is 0 caching and nothing is written to db from apache user [18:18:49] mw-debug-www.log is full of 1000s of db errors [18:19:23] for each job, e.g. DatabaseBase::query: Writes done: INSERT OR IGNORE INTO msg_resource (mr_lang,mr_resource,mr_blob,mr_timestamp) VALUES ('X') [18:20:13] (03CR) 10Hashar: "I have reverted mwext-VisualEditor-qunit job to previous state (i.e. without Zuul cloner)" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161459 (owner: 10Hashar) [18:24:03] Krinkle: ^^^ bug fill it please. can't look at it this evening :/ [18:24:27] some user right might be wrong :/ [18:25:20] I am off for real [18:27:39] 3Wikimedia / 3Continuous integration: [Regression] Jenkins: SQLite database is often locked causing queries to fail - 10https://bugzilla.wikimedia.org/71058 (10Krinkle) 3NEW p:3Unprio s:3normal a:3None https://integration.wikimedia.org/ci/job/mwext-VisualEditor-qunit/11566/ Artifact: mw-debug-www.lo... [18:27:54] 3Wikimedia / 3Continuous integration: [Regression] Jenkins: SQLite database is often locked causing queries to fail - 10https://bugzilla.wikimedia.org/71058 (10Krinkle) p:5Unprio>3High s:5normal>3major [18:33:09] 3Wikimedia / 3Continuous integration: Jenkins: Doxygen job should have extra "html/" subdirectory and publish error as artefact - 10https://bugzilla.wikimedia.org/71060 (10Krinkle) 3NEW p:3Unprio s:3normal a:3None We have: https://doc.wikimedia.org/mediawiki-core/master/js/ and https://doc.wikimedia... [18:38:53] 3Wikimedia / 3Continuous integration: doc.wikimedia.org: Generate documentation for release tags - 10https://bugzilla.wikimedia.org/71062 (10Krinkle) p:5Unprio>3Normal s:5normal>3minor [18:38:55] 3Wikimedia / 3Continuous integration: doc.wikimedia.org: Generate documentation for release tags - 10https://bugzilla.wikimedia.org/71062 (10Krinkle) 3NEW p:3Unprio s:3normal a:3None E.g. "1.23.3" instead of "REL1_23". For some reason those got lost https://doc.wikimedia.org/mediawiki-core/ We shou... [18:39:06] 3Wikimedia / 3Continuous integration: Jenkins: Doxygen job should have extra "html/" subdirectory and publish error as artefact - 10https://bugzilla.wikimedia.org/71060 (10Krinkle) p:5Unprio>3Normal s:5normal>3major [18:54:23] 3Wikimedia / 3Continuous integration: Jenkins: Generate KSS docs from LESS and publish to doc.wikimedia.org - 10https://bugzilla.wikimedia.org/58620#c12 (10Jon) Timo is in the office this week.. Maybe you could grab him S? [19:19:52] 3Wikimedia Labs / 3deployment-prep (beta): Beta Cluster isn't redirecting en.wikipedia.beta.wmflabs.org correctly - 10https://bugzilla.wikimedia.org/70948#c3 (10Greg Grossmeier) http://en.wikipedia.beta.wmflabs.org redirects to the main page now, correctly. Did something change? [19:27:43] (03Abandoned) 10Addshore: Add a READ oauth token for github [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/156768 (owner: 10Addshore) [19:53:58] that was odd, beta labs just gave me a 503 on the main page, but I can't repro. [20:03:56] and gain, just got the "get a cup of coffee" message just now [20:05:30] chrismcmahon: dig in the log maybe ? :D [20:05:58] hashar: yeah, I will if this keeps up. [20:09:08] 3Wikimedia Labs / 3deployment-prep (beta): Beta Cluster isn't redirecting en.wikipedia.beta.wmflabs.org correctly - 10https://bugzilla.wikimedia.org/70948#c4 (10Antoine "hashar" Musso) Aude told me today she was looking at the main page for beta wikidata being broken and apparently fixed it. She was probabl... [20:09:20] aude magically fixing some beta cluster issue :] [20:09:23] we are not alone! [20:12:37] 3Wikimedia Labs / 3deployment-prep (beta): deployment-graphite.eqiad.wmflabs went away? - 10https://bugzilla.wikimedia.org/71031#c2 (10C. Scott Ananian) It is, presumably, I just need to hunt down the puppet configuration for it. Mwalker set it up. [20:21:30] (03PS1) 10XZise: Add mandatory docstring tests [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161572 [20:22:47] (03PS1) 10XZise: Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 [20:23:13] (03CR) 10jenkins-bot: [V: 04-1] Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:24:31] hello beta people! [20:25:15] could anyone suggest someplace I might find the puppet configuration for deployment-pdf01.eqiad.wmflabs? [20:25:22] presumably mwalker left me one somewhere [20:25:28] hashar: ^ maybe you know? [20:26:21] cscott: I noticed a but about pdf01 sending statsd metric to the deleted instance deployment-graphite [20:26:22] and [20:26:30] I have no clue how pdf01 has been set up [20:26:50] nor what is sending the metrics [20:27:00] someone was playing with pdf02 a few days ago, when they were trying to debug deployment-salt. [20:27:14] i know how pdf01 is set up and what is sending the metric, i just don't know *how* pdf01 is set up. ;) [20:28:01] cscott: no clue either :-( [20:28:07] at worth: [20:28:15] fgrep -R deployment-graphite / [20:28:16] :D [20:28:21] also, you can't download files from OCG on beta right now. haven't figured out what's going wrong there. [20:28:38] i'm doing 'fgrep pdf02' in my pidgin logs ;) [20:31:32] cscott: while you around, I have looked at the new parsoid jobs yet :-: [20:31:53] been busy breaking ci all this week + some prod issues [20:31:54] hey, let's break stuff on a friday! :) [20:41:23] cscott: always :D [20:44:22] (03CR) 10Hashar: "Excellent work. I have deployed the job:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161572 (owner: 10XZise) [20:44:28] (03CR) 10Hashar: [C: 032] Add mandatory docstring tests [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161572 (owner: 10XZise) [20:44:47] (03PS2) 10XZise: Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 [20:44:56] (03CR) 10jenkins-bot: [V: 04-1] Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:45:37] (03CR) 10Hashar: [C: 032] "Excellent!" [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:45:50] (03CR) 10jenkins-bot: [V: 04-1] Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:46:50] ah, it was jeremyb who was playing around with pdf02 [20:47:03] jeremyb: do you know where the puppet config for pdf01 is? [20:47:43] https://bugzilla.wikimedia.org/show_bug.cgi?id=71031 [20:47:52] (03Merged) 10jenkins-bot: Add mandatory docstring tests [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161572 (owner: 10XZise) [20:47:54] (03CR) 10Hashar: "That was really:" [integration/jenkins-job-builder-config] - 10https://gerrit.wikimedia.org/r/161572 (owner: 10XZise) [20:49:25] (03CR) 10Hashar: [C: 032] Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:49:38] (03Merged) 10jenkins-bot: Add mandatory docstring tests to pywikibot/core. [integration/zuul-config] - 10https://gerrit.wikimedia.org/r/161575 (owner: 10XZise) [20:53:01] cscott: I am off. I definitely have your Parsoid jobs in mind :] [20:53:18] hashar: let's tackle that monday, if nothing's on fire then. [21:01:12] I should start counting how many times hashar says he's off for the night in a given day. :) [21:07:17] greg-g: once per channel [21:07:29] greg-g: and I make a point to reply to private message / public queries [21:07:33] so that takes a while [21:07:48] greg-g: luckily my wife end up threatening me to shut off internet [21:08:11] trick which is not going to work for long since Netflix finally opened service in France [21:08:13] \O/ [21:08:50] greg-g: meanwhile, stuff got fixed this week :) [21:10:04] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #250: FAILURE in 52 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/250/ [21:11:07] PROBLEM - CI: Puppet failure events on labmon1001 is CRITICAL: CRITICAL: integration.integration-slave1008.puppetagent.failed_events.value (30.00%) [21:16:40] !log puppet is broken on Trusty integration slaves because they try to install the non existing package php-parsekit. WIP will get it sorted on eventually. [21:16:43] Logged the message, Master [21:23:27] greg-g: and I havent found time to read your proposal about beta / nightly etc clusters :-/ [21:23:36] hashar: it's ok, next week :) [21:23:54] greg-g: at first glance, four clusters seems overkill to me :] [21:24:15] hashar: not four clusters, 2 clusters but with multiversion in use :) [21:24:54] greg-g: yeah maybe multi version can work [21:25:11] though different version might well be incompatible (parser cache / database etc) [21:25:34] but yeah next week [21:25:38] it is almost midnight there :D [21:27:11] * greg-g nods [21:27:12] g'night [21:29:09] thx [21:29:15] enjoy the week end [21:29:44] Project browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce build #178: FAILURE in 46 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-test2.m.wikipedia.org-linux-firefox-sauce/178/ [21:36:17] RECOVERY - CI: Puppet failure events on labmon1001 is OK: OK: All targets OK [21:59:52] 3Wikimedia Labs / 3deployment-prep (beta): Import failed Expected tag, got - 10https://bugzilla.wikimedia.org/70658#c7 (10Andre Klapper) 5PATC>3RESO/DUP *** This bug has been marked as a duplicate of bug 66023 *** [22:16:13] Project browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce build #161: ABORTED in 23 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-test2.wikipedia.org-linux-chrome-sauce/161/ [22:16:14] Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #192: ABORTED in 9 min 37 sec: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/192/ [22:16:16] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce build #12: ABORTED in 6 min 6 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/12/ [22:28:36] hey chrismcmahon, where do i find saucelabs sessions that correspond to our jenkins jobs? [22:29:35] marxarelli: depends. mostly you can follow a link from failed Jenkins builds [22:30:18] chrismcmahon: only failures? i'm looking at https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/174/console [22:30:20] marxarelli: if you want to see running jobs and passed jobs, I need to link a sauce account of yours to the main account. it's a bit weird. [22:30:31] which doesn't have any failures, but i was hoping to get a look anyway [22:30:59] chrismcmahon: i'm pretty sure you did that, iirc [22:32:16] marxarelli: yep, "Dan Duvall (dduvall)" shows as a sub-account of the account that has the WMF privileges, so if you log in as that account you should have access [22:32:37] i'm probably just missing it somewhere... [22:32:48] i see my own sessions but nothing from jenkins [22:33:24] chrismcmahon: do you see them? [22:34:29] marxarelli: odd. when I'm at https://saucelabs.com/account I see the jobs controlled by Jenkins [22:34:54] blerg. [22:35:53] chrismcmahon: any chance you could just send me the url for the latest mmv session? [22:36:00] they should be public urls [22:36:12] chrismcmahon: actually, the one for the url i posted [22:36:29] marxarelli: try again. I just gave you admin privs (I think). Sauce has the weirdest account handling I think I've ever seen [22:37:14] chrismcmahon: KABLAM. there they are! [22:37:23] thanks! [22:37:26] marxarelli: schweet [23:18:55] 3Wikimedia Labs / 3deployment-prep (beta): Mobile redirect goes to wrong domain name on beta labs - 10https://bugzilla.wikimedia.org/71079 (10Ryan Kaldari) 3NEW p:3Unprio s:3normal a:3None Tried logging into beta labs from my iPhone and I get redirected to: www.en.m.wikipedia.beta.wmflabs.org which t... [23:23:02] (03PS1) 10Dduvall: Resolution of Sauce session ID with custom browser [selenium] - 10https://gerrit.wikimedia.org/r/161627 [23:26:29] greg-g: so, i haven't been able to identify exactly why the mmv stats dipped suddenly, but i can now say with some certainly that running the test through saucelabs gives completely inaccurate results :) [23:27:20] most likely due to the cucumber-webdriver latency that's introduced over running it locally [23:30:22] greg-g: anyway, i'll see if i can rather some more hard numbers and update the thread, unless you want to go over it first (w/ chrismcmahon maybe?) [23:31:10] btw, it doesn't mean selenium won't work for this case, we just have to run the test local to the webdriver for accurate measurements [23:32:31] s/rather/gather/ [23:32:42] marxarelli: I believe. If you want to drive a browser for perf issues, no need for the whole cuke/rspec/pageobject stack, just some simple commands with watir-webdriver would be best. [23:33:29] chrismcmahon: that would help, but you're still going to have latency between the test client and browser if you're using sauce [23:33:59] it's the proxied connection that b0rks the numbers [23:34:10] marxarelli: yeah, agreed. [23:34:37] you could move the timing measurements to JS but then you might as well drive the whole test through something like phantomjs [23:34:47] and use JS for the entire test [23:35:04] anyway, that's not totally necessary for this case i don't think [23:35:27] just moving the the test runner to a labs instance will help tremendously [23:35:49] i'm already seeing more accurate numbers [23:36:04] congruent w/ what i'm seeing by watching the test anyway [23:49:07] PROBLEM - BetaLabs: Low disk space on /var on labmon1001 is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_avail.value (10.00%)