[00:20:26] 10Continuous-Integration-Config, 10Browser-Tests-Infrastructure: rake-jessie parser/current assumes 2.1.6-compliant but runs Ruby 2.1.5 - https://phabricator.wikimedia.org/T122098#1896650 (10Krinkle) 3NEW [00:46:54] Yippee, build fixed! [00:46:55] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #3186: 09FIXED in 53 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/3186/ [01:19:34] 10Deployment-Systems, 10Architecture, 10Wikimedia-Developer-Summit-2016-Organization, 7Availability: WikiDev 16 working area: Software engineering - https://phabricator.wikimedia.org/T119032#1896800 (10bd808) **My must attend sessions related to this area** * {T114803} * {T384} * {T113210} * {T114419} * {T... [03:35:12] Project beta-scap-eqiad build #83209: 04FAILURE in 0.64 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83209/ [03:52:20] Yippee, build fixed! [03:52:20] Project beta-scap-eqiad build #83210: 09FIXED in 7 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83210/ [08:36:38] Yippee, build fixed! [08:36:39] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #821: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/821/ [09:08:07] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Config, 6Discovery, 10Wikimedia-Portals, and 2 others: 'portals' git submodule does not update on beta ( under /srv/mediawiki-staging ) - https://phabricator.wikimedia.org/T122018#1897185 (10hashar) 5Open>3Resolved @DZahn merged the puppet change... [09:43:22] 10Continuous-Integration-Config, 10Browser-Tests-Infrastructure: rake-jessie parser/current assumes 2.1.6-compliant but runs Ruby 2.1.5 - https://phabricator.wikimedia.org/T122098#1897234 (10hashar) Welcome to the ruby world in which patch versions are allowed to change the syntax. The `parser` gem recognizes... [09:43:32] 10Continuous-Integration-Config, 10Browser-Tests-Infrastructure: rake-jessie parser/current assumes 2.1.6-compliant but runs Ruby 2.1.5 - https://phabricator.wikimedia.org/T122098#1897236 (10hashar) p:5Triage>3Low [09:47:09] !sal [09:47:09] https://tools.wmflabs.org/sal/releng [09:57:45] !log deleted cdb-* Jenkins jobs. Repo uses generic jobs [09:57:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:57:49] qa-morebots: ping [09:57:49] I am a logbot running on tools-exec-1211. [09:57:49] Messages are logged to https://tools.wmflabs.org/sal/releng. [09:57:49] To log a message, type !log . [09:58:42] !log Delete integration-zuul-debian-glue-* files. Leftover from an experiment [09:58:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:06:29] !log Restarting Jenkins [10:06:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:26:00] (03PS1) 10Hashar: Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) [10:26:16] (03CR) 10jenkins-bot: [V: 04-1] Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) (owner: 10Hashar) [10:26:22] (03CR) 10Hashar: [C: 032] Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) (owner: 10Hashar) [10:27:10] grmblblb [10:28:45] (03PS2) 10Hashar: Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) [10:29:21] (03CR) 10Hashar: [C: 032] Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) (owner: 10Hashar) [10:34:22] (03Merged) 10jenkins-bot: Remove debugging statements related to TMPDIR [integration/jenkins] - 10https://gerrit.wikimedia.org/r/260545 (https://phabricator.wikimedia.org/T120824) (owner: 10Hashar) [10:39:13] 10Continuous-Integration-Infrastructure, 5Patch-For-Review: Dozens of jobs failing on integration-slave-trusty-1012 because chmod fails for /tmp/jenkins-2 - https://phabricator.wikimedia.org/T120824#1897320 (10hashar) 5Open>3Resolved a:3hashar I am claiming this issue is now fixed. The root cause were PH... [11:37:00] Project beta-scap-eqiad build #83253: 04FAILURE in 2 min 51 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83253/ [11:44:47] 10Continuous-Integration-Config, 10Browser-Tests-Infrastructure: rake-jessie parser/current assumes 2.1.6-compliant but runs Ruby 2.1.5 - https://phabricator.wikimedia.org/T122098#1897417 (10zeljkofilipin) Parser is RuboCop's dependency: https://github.com/bbatsov/rubocop/blob/master/rubocop.gemspec#L42 [11:54:30] 10Deployment-Systems, 6operations, 5Patch-For-Review: Make l10nupdate user a system user - https://phabricator.wikimedia.org/T120585#1897428 (10faidon) >>! In T120585#1885495, @bd808 wrote: > Choosing a low number in the system reserved range (100-999) does seem likely to cause a conflict with another packag... [12:33:35] Yippee, build fixed! [12:33:35] Project beta-scap-eqiad build #83255: 09FIXED in 7 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83255/ [14:14:05] beta folks, where is puppet master in beta? [14:14:34] /var/lib/git/operations/puppet/ - is not there. [14:16:30] https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/How_code_is_updated#Cherry-picking_a_patch_from_gerrit [14:16:33] ^^ [14:33:21] zeljkof: Are you currently here? [14:33:36] Luke081515: I am! [14:34:29] :) One question: are the pages of selenium users at test2 still needed? There are all inactive since february there [14:37:27] Luke081515: delete at will! :) [14:37:58] I do not think they are used, but if they are, somebody will complain ;) [14:39:57] zeljkof: Ok, thanks [15:07:05] Project beta-scap-eqiad build #83267: 04FAILURE in 3 min 8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83267/ [15:12:05] pfff [15:12:21] rsync: rename failed for "/srv/mediawiki/php-master/cache/l10n/upstream/l10n_cache-el.cdb.MD5" (from php-master/cache/l10n/upstream/.~tmp~/l10n_cache-el.cdb.MD5): No such file or directory (2) [15:12:22] hehe [15:12:38] randomly happens, not a big deal [15:17:05] Yippee, build fixed! [15:17:06] Project beta-scap-eqiad build #83268: 09FIXED in 7 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/83268/ [15:32:30] hashar: is ci meeting today still on? [15:32:53] looks like it will be just the two of us [15:33:06] zeljkof: yeah dan cancelled apparently [15:33:13] and Jan is in holidays iirc [15:33:27] I don't think there is much to say anyway [15:33:33] I am currently in another meeting [15:33:59] hashar: ok, i am around, ping me after the meeting [15:35:30] zeljkof: the only thing that comes to mind is me having to fill a bug about CentralAuth / Saucelabs url [16:01:52] 5Testing-Initiative-2015, 10Browser-Tests-Infrastructure, 7JavaScript, 5Patch-For-Review: Experiment with browser testing in other software languages - https://phabricator.wikimedia.org/T108874#1898417 (10zeljkofilipin) I have run the tests three times in a row, got 5-8 failed scenarios, but none of the fa... [16:02:29] 10Continuous-Integration-Infrastructure, 7Nodepool, 7WorkType-Maintenance: Nodepool does not delete nodes in a timely manner (ZeroMQ dead) - https://phabricator.wikimedia.org/T120668#1898422 (10Dzahn) [16:02:30] 10Continuous-Integration-Infrastructure, 7Monitoring, 5Patch-For-Review, 7WorkType-Maintenance: Monitor Jenkins master listens on TCP port 8888 (ZeroMQ) - https://phabricator.wikimedia.org/T120669#1898420 (10Dzahn) 5Open>3Resolved https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=gall... [16:02:39] 7Browser-Tests, 10Browser-Tests-Infrastructure, 10CirrusSearch, 6Discovery, and 2 others: Upgrade CirrusSearch browser tests to use mediawiki_selenium 1.x - https://phabricator.wikimedia.org/T99653#1898423 (10zeljkofilipin) [16:04:09] hashar: cancelling the ci meeting officially? [16:15:33] zeljkof: yeah lets skip [16:15:44] we had meetings on monday not much to talk [16:39:52] 10Continuous-Integration-Infrastructure, 10MediaWiki-API, 10MediaWiki-Unit-tests, 10MediaWiki-extensions-CentralAuth: Fix ApiDocumentationTest on CentralAuth - https://phabricator.wikimedia.org/T122007#1898487 (10hoo) CentralAuth doesn't create its tables during a update.php run, thus the tables in questio... [16:55:23] 10Continuous-Integration-Infrastructure, 10MediaWiki-API, 10MediaWiki-Unit-tests, 10MediaWiki-extensions-CentralAuth, 5Patch-For-Review: Fix ApiDocumentationTest on CentralAuth - https://phabricator.wikimedia.org/T122007#1898508 (10hashar) Since CI only creates a single database, one could look whether $... [17:00:50] !log If in doubt, restart Jenkins. [17:00:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [18:10:21] 10Beta-Cluster-Infrastructure, 6operations, 7HHVM, 5Patch-For-Review: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1898727 (10RobH) I've removed the patch for review, as it seems that all pending patches on this task have been applied. [18:10:28] 10Beta-Cluster-Infrastructure, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1898728 (10RobH) [20:39:18] 10Deployment-Systems, 10Architecture, 10Wikimedia-Developer-Summit-2016-Organization, 7Availability: WikiDev 16 working area: Software engineering - https://phabricator.wikimedia.org/T119032#1899196 (10daniel) Thanks a lot for this analysis @bd808! I think I like it! The problem now is how to fit it into... [21:07:23] 10Deployment-Systems, 10Architecture, 10Wikimedia-Developer-Summit-2016-Organization, 7Availability: WikiDev 16 working area: Software engineering - https://phabricator.wikimedia.org/T119032#1899235 (10bd808) >>! In T119032#1899196, @daniel wrote: > Thanks a lot for this analysis @bd808! I think I like it!... [21:13:46] !log jenkins: kicking gearman connection, nothing is being processed from zuul queue [21:13:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:15:08] Well that did nothing. [21:17:22] Er, maybe it did. [21:17:26] I see jobs 'gain [21:17:32] Krenair: Fixed? [21:19:44] ostriches, suddenly a load of stuff queued in zuul, so probably [21:22:30] and the thing I just had rechecked has appeared [21:22:30] so yep [21:31:28] Krenair: I'm doing the airplane thing today, so if it acts up again and nobody's around to fix shoot me an e-mail, I'm less likely to miss that than an irc ping. [21:33:08] ok [21:45:45] ooh, we're doing the deploy from an airplane thing today? :D [21:48:44] I didn't deploy, I just kicked jenkins gearman [22:10:58] Somebody here, who can tell me, how I can setup a unit test engine for PHP in Arcanist? [22:11:43] https://github.com/wikimedia/mediawiki-tools-scap/blob/master/.arcconfig [22:15:28] Reedy: Thanks, I will try it [22:20:03] 5Release-Engineering-Epics, 10Gather, 10MobileFrontend, 10Reading Web Planning, and 2 others: [EPIC] Create a formal release process for MobileFrontend/Gather - https://phabricator.wikimedia.org/T100296#1899455 (10bd808) [23:31:49] !log (when npm jobs run) - sudo rm -rf /mnt/home/jenkins-deploy/.npm at integration-slave-trusty-1015 (due to cache corruption) [23:31:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [23:51:36] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 3045 bytes in 1.212 second response time [23:51:36] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string 'Wikipedia' not found on 'http://en.m.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 3148 bytes in 0.860 second response time [23:52:55] PROBLEM - App Server Main HTTP Response on deployment-mediawiki01 is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 2655 bytes in 0.897 second response time [23:54:06] PROBLEM - App Server Main HTTP Response on deployment-mediawiki02 is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 2655 bytes in 0.812 second response time [23:54:58] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 2655 bytes in 0.615 second response time [23:56:49] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-chrome-sauce build #274: 04FAILURE in 48 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-windows_7-chrome-sauce/274/ [23:56:54] [e0cc4258] /wiki/Main_Page?debug=true InvalidArgumentException from line 62 of /srv/mediawiki/php-master/extensions/QuickSurveys/includes/SurveyFactory.php: The "drink-survey" survey doesn't have any platforms. [23:58:47] jdlrobson, ^ [23:59:32] jhobs: around? ^ [23:59:47] not sure how that got through jenkins [23:59:54] which patch set is it jdlrobson?