[00:09:47] PROBLEM - Puppet staleness on integration-slave-jessie-1001 is CRITICAL 10.00% of data above the critical threshold [43200.0] [00:53:54] 10Beta-Cluster, 10MediaWiki-extensions-GWToolset, 6Multimedia, 7HHVM, 5Patch-For-Review: GWToolset XML upload fails with “The file that was uploaded exceeds the upload_max_filesize and/or the post_max_size directive in php.ini” on hhvm 3.6 - https://phabricator.wikimedia.org/T97415#1281332 (10Dzahn) 5Re... [00:58:56] 10Beta-Cluster, 10MediaWiki-extensions-GWToolset, 6Multimedia, 7HHVM, 5Patch-For-Review: GWToolset XML upload fails with “The file that was uploaded exceeds the upload_max_filesize and/or the post_max_size directive in php.ini” on hhvm 3.6 - https://phabricator.wikimedia.org/T97415#1281349 (10Dzahn) 17:5... [00:59:39] hello. just reopened this bug about UploadWizard [00:59:44] https://phabricator.wikimedia.org/T97415 [01:00:02] it seems an existing fix needs to be backported [01:00:16] and the issue got reintroduced by a recent deployment [01:01:13] currently breaks uploads [01:01:22] when using the wizard [01:06:20] 10Beta-Cluster, 10MediaWiki-extensions-GWToolset, 6Multimedia, 7HHVM, 5Patch-For-Review: GWToolset XML upload fails with “The file that was uploaded exceeds the upload_max_filesize and/or the post_max_size directive in php.ini” on hhvm 3.6 - https://phabricator.wikimedia.org/T97415#1281355 (10Dzahn) 18:0... [01:09:30] (see -operations) [01:17:36] 10Beta-Cluster, 10MediaWiki-extensions-GWToolset, 6Multimedia, 7HHVM, 5Patch-For-Review: GWToolset XML upload fails with “The file that was uploaded exceeds the upload_max_filesize and/or the post_max_size directive in php.ini” on hhvm 3.6 - https://phabricator.wikimedia.org/T97415#1281381 (10Dzahn) p:5... [01:24:30] 10Beta-Cluster, 10MediaWiki-extensions-GWToolset, 6Multimedia, 7HHVM, 5Patch-For-Review: GWToolset XML upload fails with “The file that was uploaded exceeds the upload_max_filesize and/or the post_max_size directive in php.ini” on hhvm 3.6 - https://phabricator.wikimedia.org/T97415#1281389 (10Legoktm) 5... [02:10:42] 10Beta-Cluster, 6Labs, 5Patch-For-Review, 15User-Bd808-Test: Move logs off NFS on beta - https://phabricator.wikimedia.org/T98289#1281434 (10bd808) [02:25:17] Project browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-linux-android-sauce build #98: FAILURE in 3 min 16 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-linux-android-sauce/98/ [02:35:41] Yippee, build fixed! [02:35:42] Project browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #570: FIXED in 2 min 41 sec: https://integration.wikimedia.org/ci/job/browsertests-WikiLove-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/570/ [03:29:19] PROBLEM - Puppet staleness on deployment-redis01 is CRITICAL 100.00% of data above the critical threshold [43200.0] [08:30:57] 10Browser-Tests, 6Release-Engineering, 10Continuous-Integration-Config, 6Mobile-Web: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1281670 (10hashar) p:5Triage>3Normal The mails are coming from the email `jenkins-bot@wi... [08:36:51] 10Browser-Tests, 6Release-Engineering, 10Continuous-Integration-Config, 6Mobile-Web: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1281688 (10Jdlrobson) it's Hopefully @bd808 can subscribe... [08:43:15] 10Browser-Tests, 6Release-Engineering, 10Continuous-Integration-Config, 6Mobile-Web: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1281695 (10hashar) a:3hashar [08:50:56] (03PS1) 10Hashar: Browser tests mail notifications for reading-wmf [integration/config] - 10https://gerrit.wikimedia.org/r/210657 (https://phabricator.wikimedia.org/T98477) [08:51:35] (03CR) 10Hashar: [C: 04-1] "Pending subscription of jenkins-bot@wikimedia.org to the reading-wmf list." [integration/config] - 10https://gerrit.wikimedia.org/r/210657 (https://phabricator.wikimedia.org/T98477) (owner: 10Hashar) [09:27:04] (03CR) 10Zfilipin: [C: 032] Browser tests mail notifications for reading-wmf [integration/config] - 10https://gerrit.wikimedia.org/r/210657 (https://phabricator.wikimedia.org/T98477) (owner: 10Hashar) [09:29:14] (03Merged) 10jenkins-bot: Browser tests mail notifications for reading-wmf [integration/config] - 10https://gerrit.wikimedia.org/r/210657 (https://phabricator.wikimedia.org/T98477) (owner: 10Hashar) [09:29:30] zeljkof: I havent deployed the jobs though :) [09:29:48] hashar: great, I was just about to do that myself :) [09:31:02] 10Browser-Tests, 6Release-Engineering, 10Continuous-Integration-Config, 6Mobile-Web, 5Patch-For-Review: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1281751 (10zeljkofilipin) 5Open>3Resolved The commit is merged and... [09:34:44] zeljkof: but the bot is not going to send emails [09:34:50] cause the list is private / require subscription [09:34:55] hence why I -1ed it :] [09:35:04] not a big deal [09:35:16] I guess the mails will just get bounced [09:42:31] hashar: oops, did not notice you have -1d the commit :| [09:42:47] going to restart Jenkins [09:42:48] oh, and you said you have_not_ deployed the jobs [09:42:56] argh, I can not read today [09:42:59] it is ok [09:43:02] the mail will be sent [09:43:03] should I deploy the jobs? [09:43:11] and rejected by mailman [09:43:14] yeah go ahead [09:43:20] ok, deploying [09:43:27] whenever bd808 wake up he will subscribe jenkins-bot to the reading-wmf list :] [09:44:37] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #483: FAILURE in 7 min 36 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/483/ [09:46:16] I canceled it [09:50:36] 10Continuous-Integration-Infrastructure, 7Jenkins, 7Upstream: /etc/init.d/jenkins script provided by Debian doesn't work properly - https://phabricator.wikimedia.org/T53817#1281797 (10hashar) Jenkins is still blocking on stop. I took a jstack for it: ``` Attaching to process ID 4217, please wait... Debugger... [09:58:28] hashar: what have you canceled? [09:58:39] a browser test job [09:59:35] !log switching https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/ from a matrix job to a python based script written by thcipriani [10:05:53] (03CR) 10Hashar: "I have cherry picked the python script from https://gerrit.wikimedia.org/r/#/c/210618/2 and updated the job." (032 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/210619 (https://phabricator.wikimedia.org/T96199) (owner: 10Thcipriani) [10:06:06] (03PS2) 10Hashar: Change beta-update-databases to python script [integration/config] - 10https://gerrit.wikimedia.org/r/210619 (https://phabricator.wikimedia.org/T96199) (owner: 10Thcipriani) [10:06:19] (03PS3) 10Hashar: Change beta-update-databases to python script [integration/config] - 10https://gerrit.wikimedia.org/r/210619 (https://phabricator.wikimedia.org/T96199) (owner: 10Thcipriani) [10:08:44] (03CR) 10Hashar: [C: 032] "All good and much simpler. I have removed the relevant section on https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/Add_a_" [integration/config] - 10https://gerrit.wikimedia.org/r/210619 (https://phabricator.wikimedia.org/T96199) (owner: 10Thcipriani) [10:10:24] 10Beta-Cluster, 10Continuous-Integration-Config, 5Patch-For-Review: beta-update-databases-eqiad should get list of DB to update from mediawiki-config all-labs.dblist - https://phabricator.wikimedia.org/T98342#1281824 (10hashar) a:3thcipriani Aced by @thcipriani ! I have deployed the python script by cherr... [10:10:34] 10Beta-Cluster, 10Continuous-Integration-Config, 5Patch-For-Review: beta-update-databases-eqiad should get list of DB to update from mediawiki-config all-labs.dblist - https://phabricator.wikimedia.org/T98342#1281827 (10hashar) 5Open>3Resolved [10:11:02] (03Merged) 10jenkins-bot: Change beta-update-databases to python script [integration/config] - 10https://gerrit.wikimedia.org/r/210619 (https://phabricator.wikimedia.org/T96199) (owner: 10Thcipriani) [10:14:31] 10Beta-Cluster, 6Release-Engineering, 10Continuous-Integration-Config, 5Patch-For-Review: Rewrite beta-update-databases to not use unstable Configuration Matrix - https://phabricator.wikimedia.org/T96199#1281841 (10hashar) a:3thcipriani The Jenkins job now uses a freestyle project with https://gerrit.wik... [10:14:38] 10Beta-Cluster, 6Release-Engineering, 10Continuous-Integration-Config, 5Patch-For-Review: Rewrite beta-update-databases to not use unstable Configuration Matrix - https://phabricator.wikimedia.org/T96199#1281845 (10hashar) 5Open>3Resolved [10:15:18] 10Beta-Cluster, 10Continuous-Integration-Config, 5Patch-For-Review: beta-update-databases-eqiad should get list of DB to update from mediawiki-config all-labs.dblist - https://phabricator.wikimedia.org/T98342#1281846 (10hashar) 5Resolved>3Open Reopening, this is pending the review/merge of operations/pup... [10:19:08] 10Browser-Tests, 6Release-Engineering, 10Continuous-Integration-Config, 6Mobile-Web, 5Patch-For-Review: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1268931 (10hashar) I have spawned a sub tasks for the mail subscription... [10:19:24] 10Browser-Tests, 10Continuous-Integration-Config, 6Mobile-Web, 5Patch-For-Review: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1281871 (10hashar) [10:28:44] lunch time [10:28:56] then commute and hair cut [10:29:03] will be back slightly before 3pm [10:41:44] (03CR) 10Zfilipin: "Sorry, did not notice that you voted -1. :(" [integration/config] - 10https://gerrit.wikimedia.org/r/210657 (https://phabricator.wikimedia.org/T98477) (owner: 10Hashar) [10:48:30] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » es,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 17 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=es,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [11:05:02] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » fi,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 34 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=fi,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [11:21:12] (03PS6) 10Florianschmidtwelzow: Add OOJsUIAjaxLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/207758 [11:22:38] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » tr,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 51 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=tr,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [11:40:06] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » he,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 1 hr 9 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=he,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [11:56:54] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » nl,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 1 hr 26 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=nl,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [11:59:20] (03PS1) 10Aude: Update Wikidata branch to wmf/1.26wmf6 [tools/release] - 10https://gerrit.wikimedia.org/r/210671 [12:00:09] (03CR) 10Aude: [C: 032] Update Wikidata branch to wmf/1.26wmf6 [tools/release] - 10https://gerrit.wikimedia.org/r/210671 (owner: 10Aude) [12:01:23] (03Merged) 10jenkins-bot: Update Wikidata branch to wmf/1.26wmf6 [tools/release] - 10https://gerrit.wikimedia.org/r/210671 (owner: 10Aude) [12:13:28] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » en,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 1 hr 42 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=en,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [12:30:05] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » fr,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 1 hr 59 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=fr,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [12:43:22] PROBLEM - Free space - all mounts on deployment-eventlogging02 is CRITICAL deployment-prep.deployment-eventlogging02.diskspace._var.byte_percentfree (<30.00%) [12:48:02] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ru,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 2 hr 17 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ru,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [13:06:52] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » id,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 2 hr 36 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=id,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [13:23:32] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » om,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 2 hr 52 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=om,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [13:43:29] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » krc,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 3 hr 12 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=krc,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [14:00:19] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » nb,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 3 hr 29 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=nb,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [14:02:50] (03PS2) 10Hashar: Switch Graph extension to use npm testing for lint checks [integration/config] - 10https://gerrit.wikimedia.org/r/209991 (owner: 10TheDJ) [14:04:57] (03CR) 10Hashar: [C: 04-1] "The extension package.json lacks the definition of the 'test' command. The Jenkins job invokes `npm test`." [integration/config] - 10https://gerrit.wikimedia.org/r/209991 (owner: 10TheDJ) [14:06:18] Project browsertests-Wikidata-PerformanceTests-linux-firefox-sauce build #246: FAILURE in 17 sec: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-PerformanceTests-linux-firefox-sauce/246/ [14:07:02] (03CR) 10Hashar: [C: 04-1] "Forgot to -1 per my previous comment (need to use npm/composer entry points See https://www.mediawiki.org/wiki/Continuous_integration/Entr" [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [14:16:07] (03CR) 10Hashar: [C: 04-1] Run mediawiki/core phpunit tests on sqlite again (in addition to mysql) (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/207135 (owner: 10Legoktm) [14:19:15] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » kn,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 3 hr 48 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=kn,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [14:34:21] 10Continuous-Integration-Infrastructure: Grant Zuul deploy rights to Jan Zerebecki - https://phabricator.wikimedia.org/T98865#1282193 (10hashar) I have requested shell access to ops via T98961 Added Jan to the [[ https://gerrit.wikimedia.org/r/#/admin/groups/10,members | Gerrit integration group ]] so he can +2... [14:34:54] !log added jzerebecki to the Gerrit 'integration' group for +2 and as a project admin for the labs integration project. {{bug|T98865}} [14:35:11] 10Continuous-Integration-Infrastructure: Grant Zuul deploy rights to Jan Zerebecki - https://phabricator.wikimedia.org/T98865#1282195 (10hashar) a:3hashar [14:35:44] 10Continuous-Integration-Infrastructure: Grant Zuul deploy rights to Jan Zerebecki - https://phabricator.wikimedia.org/T98865#1279480 (10hashar) p:5Triage>3Normal [14:35:58] thx. [14:36:01] off poor bot [14:36:51] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » sr,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 4 hr 6 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=sr,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [14:37:02] too lazy to reboot qamorebot [14:37:35] jzerebecki: the shell access is in another sub task https://phabricator.wikimedia.org/T98961 depends on ops approval [14:38:20] Project browsertests-Wikidata-SmokeTests-linux-firefox-sauce build #250: FAILURE in 21 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce/250/ [14:39:00] (03CR) 10Florianschmidtwelzow: [C: 04-1] "Ahh, hashar, sorry, i haven't seen your comment :] Let's use composer and npm (i haven't thoutgh about that :P)." [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [14:46:13] (03CR) 10Hashar: "No worries. The composer/npm as entry point is fairly new but should be easy to setup following the doc or looking at other repositories a" [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [14:54:16] 10Deployment-Systems, 6operations: git fat/git deploy doesn't always unstub files [Trebuchet] - https://phabricator.wikimedia.org/T98962#1282225 (10ArielGlenn) [14:54:21] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » el,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 4 hr 23 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=el,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [14:57:03] 10Deployment-Systems, 6operations: git fat/git deploy doesn't always unstub files [Trebuchet] - https://phabricator.wikimedia.org/T98962#1282232 (10Manybubbles) [14:58:04] (03PS7) 10Florianschmidtwelzow: Add OOJsUIAjaxLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/207758 [14:59:04] (03CR) 10Florianschmidtwelzow: "@hashar: I added test entry points in Ifbb0d7a61f67c57 (could you take a short look if that looks good?). If i read the templates right, t" [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [15:03:32] 10Beta-Cluster, 10Continuous-Integration-Config, 5Patch-For-Review: beta-update-databases-eqiad should get list of DB to update from mediawiki-config all-labs.dblist - https://phabricator.wikimedia.org/T98342#1282238 (10greg) p:5Triage>3Normal [15:04:44] hashar: joined :) [15:04:49] great [15:04:52] so [15:04:52] https://gerrit.wikimedia.org/r/#/c/210693/ [15:04:56] that looks almost fine [15:05:06] packages.json need to be renamed to use singular package.json [15:05:16] fetch that change the npm test and composer test commands fail [15:05:26] because there is no source code. [15:05:38] I would add them directly to the initial commit at https://gerrit.wikimedia.org/r/#/c/207756/ [15:05:38] hashar: i tested package.json and realized that i build a new patch on top of my initial commit and i copied the files manually ("yeah Florian, there is nothing you can do false copying files" ;)) to a new branch for a new patch and made a mistake :( [15:05:49] hehe [15:06:25] you can use git cherry-pick -n [15:06:31] aka [15:06:34] git-review -d 207756 [15:06:45] git cherry-pick -n 843a974e2a57233af0266f4057ae7dfb304f7b88 [15:06:49] (that brings in the json files) [15:06:52] then git commit --amend -a [15:07:44] hashar: doing it, but now i test it first ;) [15:07:46] :D [15:09:28] hashar: https://gerrit.wikimedia.org/r/#/c/207756/ :) [15:10:35] testing :) [15:11:01] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ro,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 4 hr 40 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ro,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [15:13:13] am I back ? [15:13:33] FlorianSW: much better [15:13:47] FlorianSW: jshint fails though :D [15:13:59] hashar: seeing it and fixing it atm :D [15:14:10] i thought i'm faster as you, but.. i'm not :P [15:14:59] hashar: updated :] [15:16:43] (03CR) 10Hashar: [C: 032] "Let see what kind of madness that unleashes :}" [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [15:17:01] :D [15:17:18] the config has been highly refactored recently [15:17:24] and I haven't kept track of everything :D [15:17:27] 10Browser-Tests, 10Continuous-Integration-Config, 6Mobile-Web, 5Patch-For-Review: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1282272 (10Jdlrobson) [15:17:35] great beginning! :D [15:17:53] but i'm sure, that doesn't breaks anything :P [15:17:55] trust me :D [15:18:32] (03Merged) 10jenkins-bot: Add OOJsUIAjaxLogin extension [integration/config] - 10https://gerrit.wikimedia.org/r/207758 (owner: 10Florianschmidtwelzow) [15:19:30] bah failing :/ [15:19:35] I commented 'recheck' on the patch [15:19:41] mwext-qunit [15:19:41] phplintsuccess [15:19:41] mwext-testextension-zendsuccess [15:19:41] php-composer-validatesuccess [15:19:41] php-composer-testsuccess [15:19:41] npmfailure [15:19:42] jsduck [15:19:45] uhh, jsduck [15:19:46] that is a lot of jobs triggered :] [15:20:00] jsduck failure is https://integration.wikimedia.org/ci/job/jsduck/1819/console [15:20:07] npm one is https://integration.wikimedia.org/ci/job/npm/5807/console [15:20:11] (I think) [15:20:28] hashar: testextension-zend seems to doesn't make sense, there aren't phpunit tests atm [15:20:37] ah [15:20:50] that one runs the mediawiki/core extension test suite [15:21:01] they are defined in core tests/phpunit/suites/structure or something like that [15:21:23] the tests are in core, and they do some generic tests such as verifying the autoloader or resource loader declarations [15:21:38] yep, is this needed? I think it can be re-added when i consider to add phpunit tests? [15:21:50] ah, ok, no that makes sense [15:21:52] yeah it is required [15:22:00] so seems you have some more jshint to fix [15:22:22] jsduck I have absolute no idea how it works [15:22:46] apparently you need a jsduck.json file [15:23:05] hashar: i think that has to be fixed in package.json and the jshint errors: my problem :P I think all you can do you did, big thanks! [15:23:22] FlorianSW: https://www.mediawiki.org/wiki/Manual:Coding_conventions/JavaScript#Documentation [15:23:32] there are some links to jsduck.json example file [15:23:43] and directly link to the mw/core and OOjs ones [15:23:47] ah, great :) [15:24:06] yeah I think you are closed to be golden ! [15:24:16] (03CR) 10TheDJ: "@Hashar. Ah, so THAT is the url of that page. I assumed there was something like that somewhere, but was unable to find it so far :)" [integration/config] - 10https://gerrit.wikimedia.org/r/209991 (owner: 10TheDJ) [15:24:50] hashar: again: big thanks! :) [15:26:34] (03CR) 10Hashar: "FlorianSW has just completed the setup for the OOJsUIAjaxLogin extension." [integration/config] - 10https://gerrit.wikimedia.org/r/209991 (owner: 10TheDJ) [15:28:22] hashar: thanks :-) I'm not entirely sure why the compiler isn't working for labs, but aiui it's also not working for prod at the moment [15:28:44] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ar,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 4 hr 57 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ar,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [15:28:58] Yippee, build fixed! [15:28:58] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #532: FIXED in 57 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/532/ [15:28:59] valhallasw: the list of nodes are expected to be found/matched by manifests/site.pp [15:29:08] valhallasw: so it is unlikely your labs node will match :] [15:29:42] valhallasw: you can reach out to _joe_ about the utility. It might be in python [15:30:28] hashar: *nod* [15:30:48] maybe we should just define puppet classes in that way for tool labs as well ;-) [15:31:32] ideally [15:31:43] valhallasw: though currently operations/puppet changes can only be approved by ops [15:31:53] although 'node' in that context might be the i-etc identifier [15:32:08] we would some system with the puppetmaster serving different branches per labs project [15:32:10] hashar: all our puppet manifests are in there anyway, so it doesn't really matter [15:32:39] 6Release-Engineering, 10Hackathon-Lyon-2015, 10Wikipedia-Android-App, 10Wikipedia-iOS-App: Create end-to-end test for Wikipedia Android app - https://phabricator.wikimedia.org/T90177#1282319 (10BGerstle-WMF) [15:34:34] 6Release-Engineering: Net::ReadTimeout shouldn't mark a test as failed - https://phabricator.wikimedia.org/T98968#1282330 (10Jdlrobson) 3NEW [15:36:14] 6Release-Engineering, 10Hackathon-Lyon-2015, 10Wikipedia-Android-App, 10Wikipedia-iOS-App: Create end-to-end test for Wikipedia Android app - https://phabricator.wikimedia.org/T90177#1053692 (10BGerstle-WMF) @zeljkofilipin mainly whether or not anyone is actually hacking on this. i'd be happy to hack on it... [15:39:46] 10Beta-Cluster, 10Staging: Rework beta apache config - https://phabricator.wikimedia.org/T1256#1282356 (10demon) a:5demon>3None [15:42:06] * ^d unlicked a cookie [15:46:14] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ja,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 5 hr 15 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ja,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [15:49:51] 6Release-Engineering, 10Hackathon-Lyon-2015, 10Wikipedia-Android-App, 10Wikipedia-iOS-App: Create end-to-end test for Wikipedia Android app - https://phabricator.wikimedia.org/T90177#1282391 (10greg) >>! In T90177#1282338, @BGerstle-WMF wrote: > @zeljkofilipin mainly whether or not anyone is actually hacki... [15:50:58] 10Browser-Tests, 10Continuous-Integration-Config, 6Mobile-Web, 5Patch-For-Review: reading-wmf@ list should be notified of browser test failures in Gather and mobile web - https://phabricator.wikimedia.org/T98477#1282400 (10Jdlrobson) Thanks guys! Looks good to me! [15:53:58] greg-g: I forgot my headphones at home :( [15:54:12] hashar: do we have much very important? [15:54:42] yeah one point, can mention it via private message on IRC and that should be good enough [15:54:48] awesome [15:56:35] thcipriani: good morning. I have deployed your beta-update-databases python script earlier today. Seems to work appropriately [15:57:32] hashar: I saw that, thanks! [15:58:14] 10Browser-Tests, 6Release-Engineering: Net::ReadTimeout shouldn't mark a test as failed - https://phabricator.wikimedia.org/T98968#1282424 (10greg) [16:01:17] hashar: I just pushed up an update to that script that uses your suggestion of adding in a batch size argument [16:01:36] also got rid of the unused import [16:04:35] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » et,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 5 hr 33 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=et,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [16:09:36] thcipriani: looking :) [16:15:22] 10Continuous-Integration-Infrastructure, 6Release-Engineering, 10Hackathon-Lyon-2015, 10Wikipedia-Android-App, 10Wikipedia-iOS-App: Create end-to-end test for Wikipedia Android app - https://phabricator.wikimedia.org/T90177#1282484 (10greg) [16:19:08] hashar: bingo! :D [16:21:36] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » pt,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 5 hr 50 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=pt,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [16:21:55] thcipriani: followed up at https://gerrit.wikimedia.org/r/#/c/210618/5/modules/beta/files/wmf-beta-update-databases.py,unified [16:22:16] thcipriani: in short argparse can set the defaults for us :] Then they can be made to be shown when one uses --help [16:22:40] FlorianSW: congratulations!!! [16:22:47] FlorianSW: now you can add moaaar tests [16:22:56] hashar: gotcha, wasn't sure what was idiomatic with python. Thanks for feedback. [16:23:28] hashar: yeah :D [16:23:30] thcipriani: I am far from being a rock start pythonist. I have relearned it eventually by following OpenStack best practices [16:23:49] thcipriani: and they have a bunch of experimented pythonist. Makes it fun to learn about the various python modules [16:24:56] nice. their underlying codebase is pretty expansive, plenty to see there :) [16:25:35] thcipriani: an example https://github.com/openstack-infra/zuul/blob/master/zuul/cmd/cloner.py#L37 [16:25:44] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: jenkins job to generate wikibase.git coverage - https://phabricator.wikimedia.org/T88434#1282503 (10Addshore) @Legoktm didn't you recently do something to do with coverage of extensions? [16:25:58] thcipriani: so you get all the argument handling / default / validation done in the same method [16:26:09] though in this case, all functions are wrapped in a class [16:26:26] so the validated args ends up in a class attribute self.args [16:26:36] this way you don't have to pass args between functions [16:26:44] since all methods have the same context and can grab them directly [16:26:58] but that might add to much cruft :] [16:28:53] thcipriani: I am escaping back home. Not sure whether I can follow up tonight [16:29:19] one sure thing, the python script behave properly and seems to handle the update faster than the Jenkins multi config job [16:30:08] hashar: kk, I'll push something up here shortly. Thanks for the review. Glad to hear it's working! [16:30:35] thcipriani: feel free to cherry pick on beta cluster and manually trigger the jenkins job to ensure it works :} [16:34:12] 10Continuous-Integration-Infrastructure, 7Jenkins: Jenkins does not shutdown due to a deadlock in the IRC plugin - https://phabricator.wikimedia.org/T98976#1282505 (10hashar) 3NEW [16:35:08] Project beta-scap-eqiad build #52702: FAILURE in 59 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/52702/ [16:39:33] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » cs,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 6 hr 8 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=cs,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [16:41:46] hashar, i think jenkins doesn't love me anymore :( https://gerrit.wikimedia.org/r/#/c/207756/ (do you already know this error?) [16:42:45] Yeah, git repo cloning is broken [16:48:34] nice [16:48:55] stderr: fatal: unable to connect to zuul.eqiad.wmnet: [16:48:55] 16:33:54 zuul.eqiad.wmnet[0: 208.80.154.135]: errno=Connection timed out [16:51:24] 10Continuous-Integration-Infrastructure: git clone fails: zuul.eqiad.wmnet[0: 208.80.154.135]: errno=Connection timed out - https://phabricator.wikimedia.org/T98980#1282566 (10hashar) 3NEW [16:51:32] marktraceur: FlorianSW filled as https://phabricator.wikimedia.org/T98980 [16:52:24] hashar: thx :] [16:53:32] Yippee, build fixed! [16:53:33] Project beta-scap-eqiad build #52704: FIXED in 1 min 8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/52704/ [16:53:41] so [16:54:51] PROBLEM - Puppet failure on integration-zuul-server is CRITICAL 100.00% of data above the critical threshold [0.0] [16:54:59] PROBLEM - Puppet failure on deployment-db2 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:55:07] PROBLEM - Puppet failure on deployment-upload is CRITICAL 100.00% of data above the critical threshold [0.0] [16:55:30] PROBLEM - Puppet failure on deployment-memc03 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:55:54] PROBLEM - Puppet failure on integration-slave-precise-1013 is CRITICAL 80.00% of data above the critical threshold [0.0] [16:56:00] PROBLEM - Puppet failure on deployment-memc04 is CRITICAL 60.00% of data above the critical threshold [0.0] [16:56:17] PROBLEM - Puppet failure on integration-slave-precise-1014 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:56:21] PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:56:35] PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL 80.00% of data above the critical threshold [0.0] [16:56:39] PROBLEM - Puppet failure on deployment-test is CRITICAL 100.00% of data above the critical threshold [0.0] [16:57:01] PROBLEM - Puppet failure on integration-publisher is CRITICAL 100.00% of data above the critical threshold [0.0] [16:57:07] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » uk,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 6 hr 26 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=uk,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [16:57:42] PROBLEM - Puppet failure on deployment-fluoride is CRITICAL 100.00% of data above the critical threshold [0.0] [16:57:43] PROBLEM - Puppet failure on deployment-kafka02 is CRITICAL 70.00% of data above the critical threshold [0.0] [16:57:46] PROBLEM - Puppet failure on deployment-salt is CRITICAL 50.00% of data above the critical threshold [0.0] [16:57:59] PROBLEM - Puppet failure on integration-saltmaster is CRITICAL 70.00% of data above the critical threshold [0.0] [16:58:07] PROBLEM - Puppet failure on deployment-fluorine is CRITICAL 66.67% of data above the critical threshold [0.0] [16:58:23] 10Continuous-Integration-Infrastructure: git clone fails: zuul.eqiad.wmnet[0: 208.80.154.135]: errno=Connection timed out - https://phabricator.wikimedia.org/T98980#1282611 (10hashar) The Zuul git repositories are served by a git-daemon process on gallium.wikimedia.org. Looking at the syslog messages there is a... [16:58:26] PROBLEM - Puppet failure on deployment-memc02 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:59:17] PROBLEM - Puppet failure on deployment-db1 is CRITICAL 44.44% of data above the critical threshold [0.0] [17:01:14] 10Continuous-Integration-Infrastructure: git clone fails: zuul.eqiad.wmnet[0: 208.80.154.135]: errno=Connection timed out - https://phabricator.wikimedia.org/T98980#1282614 (10hashar) The traffic between labs and prod has been shortly interrupted due to an ongoing operation maintenance. [17:01:32] PROBLEM - Puppet failure on integration-puppetmaster is CRITICAL 60.00% of data above the critical threshold [0.0] [17:01:32] PROBLEM - Puppet failure on deployment-bastion is CRITICAL 100.00% of data above the critical threshold [0.0] [17:02:23] 10Continuous-Integration-Infrastructure: git clone fails: zuul.eqiad.wmnet[0: 208.80.154.135]: errno=Connection timed out - https://phabricator.wikimedia.org/T98980#1282619 (10hashar) 5Open>3Resolved a:3hashar It is back up and solved. Might be interrupted again though. To retrigger jobs just comment `rec... [17:02:48] RECOVERY - Puppet failure on deployment-salt is OK Less than 1.00% above the threshold [0.0] [17:03:10] now I am off [17:04:14] RECOVERY - Puppet failure on deployment-db1 is OK Less than 1.00% above the threshold [0.0] [17:04:57] RECOVERY - Puppet failure on deployment-db2 is OK Less than 1.00% above the threshold [0.0] [17:05:53] RECOVERY - Puppet failure on integration-slave-precise-1013 is OK Less than 1.00% above the threshold [0.0] [17:06:36] RECOVERY - Puppet failure on deployment-logstash1 is OK Less than 1.00% above the threshold [0.0] [17:07:42] RECOVERY - Puppet failure on deployment-kafka02 is OK Less than 1.00% above the threshold [0.0] [17:08:06] RECOVERY - Puppet failure on deployment-fluorine is OK Less than 1.00% above the threshold [0.0] [17:09:49] RECOVERY - Puppet failure on integration-zuul-server is OK Less than 1.00% above the threshold [0.0] [17:11:17] RECOVERY - Puppet failure on integration-slave-precise-1014 is OK Less than 1.00% above the threshold [0.0] [17:11:26] RECOVERY - Puppet failure on integration-puppetmaster is OK Less than 1.00% above the threshold [0.0] [17:12:03] RECOVERY - Puppet failure on integration-publisher is OK Less than 1.00% above the threshold [0.0] [17:15:07] RECOVERY - Puppet failure on deployment-upload is OK Less than 1.00% above the threshold [0.0] [17:15:33] RECOVERY - Puppet failure on deployment-memc03 is OK Less than 1.00% above the threshold [0.0] [17:16:37] RECOVERY - Puppet failure on deployment-test is OK Less than 1.00% above the threshold [0.0] [17:17:08] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » it,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 6 hr 46 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=it,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [17:17:39] RECOVERY - Puppet failure on deployment-fluoride is OK Less than 1.00% above the threshold [0.0] [17:18:26] RECOVERY - Puppet failure on deployment-memc02 is OK Less than 1.00% above the threshold [0.0] [17:21:07] RECOVERY - Puppet failure on deployment-memc04 is OK Less than 1.00% above the threshold [0.0] [17:21:23] RECOVERY - Puppet failure on deployment-videoscaler01 is OK Less than 1.00% above the threshold [0.0] [17:22:59] RECOVERY - Puppet failure on integration-saltmaster is OK Less than 1.00% above the threshold [0.0] [17:26:17] twentyafterfour: around? [17:26:56] aude: yep, what's up? [17:27:08] we have a new branch for group0 [17:27:22] ok .. I was just about to start the branching [17:27:24] but the changes are rather small vs. what is deployed now [17:27:52] so i think chances of problems are small and i don't need to be around all evening [17:28:32] Yeah, if there are problems I can revert, and anyway it's just group 0 ;) [17:28:34] if our stuff massively breaks test wikis, then ok to put our stuff back on our older deployment branch [17:28:37] yeah [17:28:53] and i might be around later, like in 3 hours [17:34:08] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ms,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 7 hr 3 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ms,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [17:40:06] Project browsertests-Wikidata-WikidataTests-linux-firefox-sauce build #224: STILL FAILING in 3 hr 18 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-WikidataTests-linux-firefox-sauce/224/ [17:50:49] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » lb,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 7 hr 20 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=lb,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [17:52:55] branching 1.26wmf6 [18:07:58] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » sv,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 7 hr 37 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=sv,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [18:25:53] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » gl,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 7 hr 55 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=gl,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [18:41:28] RECOVERY - Puppet failure on deployment-bastion is OK Less than 1.00% above the threshold [0.0] [18:43:50] (03CR) 10Krinkle: "You may wanna try to consolidate these into a single job that runs both. We have an outstanding proposal to parallelise phpunit, there's n" [integration/config] - 10https://gerrit.wikimedia.org/r/207135 (owner: 10Legoktm) [18:43:57] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ko,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 8 hr 13 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ko,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [18:53:41] Yippee, build fixed! [18:53:41] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #317: FIXED in 40 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/317/ [19:02:29] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » pl,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 8 hr 31 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=pl,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [19:07:56] 6Release-Engineering, 6operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#1283121 (10hashar) There was some rspec incompatibility issue for which I proposed a fix at https://github.com/rodjek/rspec-puppet/pull/243 which is superseded by https://github.co... [19:08:41] 6Release-Engineering, 6operations: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#1283127 (10hashar) 5Open>3stalled Stalled till @dduvall starts looking at building some rspec for operations/puppet.git [19:19:35] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » hi,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 8 hr 48 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=hi,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [19:36:39] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » vi,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 9 hr 5 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=vi,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [19:54:29] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » hr,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 9 hr 23 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=hr,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [19:55:20] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM: CI for Civi: provision and run tests under Jenkins/Zuul - https://phabricator.wikimedia.org/T86103#1283233 (10hashar) @awight should you add a blocking task to have the job to run concurrently? IIRC there is some cache in $HOME which nee... [20:12:53] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ast,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 9 hr 42 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ast,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [20:30:23] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » sl,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 9 hr 59 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=sl,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [20:31:11] 10Continuous-Integration-Infrastructure, 10Fundraising Tech Backlog, 6Scrum-of-Scrums, 10Wikimedia-Fundraising-CiviCRM, and 2 others: Continuous integration - CiviCRM - https://phabricator.wikimedia.org/T78100#1283357 (10awight) [20:31:13] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM: CI for Civi: provision and run tests under Jenkins/Zuul - https://phabricator.wikimedia.org/T86103#1283355 (10awight) 5stalled>3Open [20:31:43] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM: CI for Civi: provision and run tests under Jenkins/Zuul - https://phabricator.wikimedia.org/T86103#961478 (10awight) [20:50:49] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » zh-hant,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 10 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=zh-hant,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [21:01:21] (03CR) 10Polybuildr: "Could someone please review this?" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/196880 (owner: 10Polybuildr) [21:07:57] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » mk,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 10 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=mk,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [21:25:28] Yippee, build fixed! [21:25:28] Project browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-linux-android-sauce build #99: FIXED in 3 min 12 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-linux-android-sauce/99/ [21:25:52] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » fa,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 10 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=fa,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [21:34:15] 10Deployment-Systems, 7HHVM: HHVM lock-ups - https://phabricator.wikimedia.org/T89912#1283608 (10BBlack) I spent some time digging on this just from a quickstack trace and looking at HHVM source on github, but I haven't had a chance to try to correlate with looking at a live locked-up HHVM. I haven't really f... [21:40:58] (03PS1) 10Florianschmidtwelzow: Remove Mantle from wmf-branch creation script [tools/release] - 10https://gerrit.wikimedia.org/r/210817 (https://phabricator.wikimedia.org/T85890) [21:42:23] (03PS2) 10Florianschmidtwelzow: Remove Mantle from wmf-branch creation script [tools/release] - 10https://gerrit.wikimedia.org/r/210817 (https://phabricator.wikimedia.org/T85890) [21:44:15] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » ilo,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 11 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=ilo,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [21:59:24] 10Deployment-Systems, 7HHVM: HHVM lock-ups - https://phabricator.wikimedia.org/T89912#1283695 (10BBlack) Another parting thought: if we have an easy way to make small source changes and run them live, it might be worth switching the initialization of `s_genLock` (which is at the core of `GenCountGuard`), to `P... [22:00:54] 6Release-Engineering, 6Phabricator: Next Phabricator upgrade on 2015-05-20 (tentative) - https://phabricator.wikimedia.org/T98451#1283698 (10Qgil) [22:01:16] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » zh-hans,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 11 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=zh-hans,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [22:18:25] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » yue,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 11 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=yue,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [22:36:14] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » de,contintLabsSlave && UbuntuTrusty build #42: FAILURE in 12 hr: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=de,label=contintLabsSlave%20&&%20UbuntuTrusty/42/ [22:41:35] I've been playing with writing a go cli tool to read logs from logstash. [22:41:45] Its getting to be fairly usable -- https://github.com/bd808/ggml [22:47:49] we can run go on the servers? [22:52:46] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<50.00%) [23:27:46] RECOVERY - Free space - all mounts on deployment-bastion is OK All targets OK [23:31:20] PROBLEM - Puppet failure on deployment-pdf01 is CRITICAL 100.00% of data above the critical threshold [0.0] [23:31:24] PROBLEM - Puppet staleness on deployment-urldownloader is CRITICAL 100.00% of data above the critical threshold [43200.0] [23:31:44] PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL 100.00% of data above the critical threshold [0.0]