[00:00:20] paladox: I think so? It might be 30, I forget [00:00:50] legoktm: Oh ok. [00:00:56] thanks for replying. [00:05:23] (03PS1) 10Paladox: Remove template extension-checks, Replace with jshint, jsonlint and mw-checks-test [integration/config] - 10https://gerrit.wikimedia.org/r/273398 (https://phabricator.wikimedia.org/T127362) [00:05:43] legoktm: Could you review https://gerrit.wikimedia.org/r/273398 [00:05:43] please. [00:07:21] legoktm: and https://gerrit.wikimedia.org/r/#/c/273394/ too please. [00:07:59] paladox: in about 10 minutes, sure. [00:08:10] legoktm: Thanks. [00:08:11] :) [00:20:13] (03PS1) 10Jforrester: mediawiki.conf: Set $wgScribuntoDefaultEngine for speed [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273405 (https://phabricator.wikimedia.org/T126670) [00:22:33] 10Continuous-Integration-Config, 13Patch-For-Review: Have CI set `$wgScribuntoDefaultEngine = 'luasandbox` to speed up parser tests - https://phabricator.wikimedia.org/T126670#2020381 (10Jdforrester-WMF) >>! In T126670#2021795, @JanZerebecki wrote: > Sounds like we want to change it to do autodetection instead... [00:22:43] 10Continuous-Integration-Config, 13Patch-For-Review: Have CI set `$wgScribuntoDefaultEngine = 'luasandbox` to speed up parser tests - https://phabricator.wikimedia.org/T126670#2065412 (10Jdforrester-WMF) a:3Jdforrester-WMF [00:29:35] (03CR) 10Legoktm: [C: 04-1] "Scribunto isn't using extension.json yet, so this will need to run after the extension is loaded (50)...so rename file to 51_set_wgScribun" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273405 (https://phabricator.wikimedia.org/T126670) (owner: 10Jforrester) [00:30:04] paladox: looking now [00:30:27] (03PS2) 10Jforrester: mediawiki.conf: Set $wgScribuntoDefaultEngine for speed [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273405 (https://phabricator.wikimedia.org/T126670) [00:31:38] (03CR) 10Legoktm: [C: 032] mediawiki.conf: Set $wgScribuntoDefaultEngine for speed [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273405 (https://phabricator.wikimedia.org/T126670) (owner: 10Jforrester) [00:32:40] (03Merged) 10jenkins-bot: mediawiki.conf: Set $wgScribuntoDefaultEngine for speed [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273405 (https://phabricator.wikimedia.org/T126670) (owner: 10Jforrester) [00:41:33] legoktm: I presume ^^^ needs to be deployed. [01:05:20] legoktm: Thanks. [03:27:21] James_F|Away: it auto-deploys [07:15:54] RECOVERY - Free space - all mounts on deployment-fluorine is OK: OK: All targets OK [08:33:07] PROBLEM - Host cache-rsync is DOWN: CRITICAL - Host Unreachable (10.68.23.165) [08:36:28] PROBLEM - SSH on integration-make-wmf-branch is CRITICAL: Connection refused [09:29:37] hashar: good morning! :) [09:29:45] if you have a minute, take a look at this https://gerrit.wikimedia.org/r/#/c/273258/ [09:29:50] hello [09:29:54] took me a while to warm up this morning [09:29:59] it will fix related articles browser tests [09:30:04] I have been fixing stuff up until too late yesterday [09:40:01] (03CR) 10Hashar: [C: 04-1] "That will cause extensions having a .m. URL to point to environment 'beta-mobile' which is not defined for most repositories." (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/273258 (https://phabricator.wikimedia.org/T128084) (owner: 10Zfilipin) [09:40:20] zeljkof: reviewed, it is not going to work for other extensions [09:40:43] since we have bunch of extensions with a mediawiki_url: with .m. that will switch them to use 'beta-mobile' which is not defined [09:41:04] I would drop {mediawiki_url} from JJB entirely and switch to {environment} [09:44:57] hashar: thanks, will take a look [09:45:32] argh, forgot about other extensions :| [09:45:40] oh well, refactoring it is [09:47:01] 10Continuous-Integration-Infrastructure, 10IPSet, 13Patch-For-Review: IPSet::__construct() in gets into infinite loop when called from curl on a CI host - https://phabricator.wikimedia.org/T126495#2065933 (10hashar) a:3PleaseStand My understand is the patch @PleaseStand provided on https://gerrit.wikimedia... [09:47:43] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #729: 04FAILURE in 26 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/729/ [09:53:24] 10Continuous-Integration-Infrastructure, 10IPSet, 13Patch-For-Review: IPSet::__construct() in gets into infinite loop when called from curl on a CI host - https://phabricator.wikimedia.org/T126495#2015906 (10hashar) Filled: * {T128167} * {T128169} [09:54:09] 10Continuous-Integration-Infrastructure, 10IPSet: IPSet::__construct() in gets into infinite loop when called from curl on a CI host - https://phabricator.wikimedia.org/T126495#2015906 (10hashar) p:5Triage>3High [09:58:16] zeljkof: so yeah refactor. But that will change all jobs names ... [09:58:36] hashar: will think about how to do it right [09:58:37] we might want to rename the jobs before deploying so as to keep the ~15 days history of build logs [09:58:48] yes, that is one idea [10:25:32] 5Gerrit-Migration, 10Analytics-Tech-community-metrics, 6Developer-Relations, 3DevRel-March-2016: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#2066001 (10Aklapper) a:3Aklapper [10:26:38] (03CR) 10Hashar: [C: 032] Replace jslint test with jshint and jsonlint tests [integration/config] - 10https://gerrit.wikimedia.org/r/273394 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [10:27:56] (03Merged) 10jenkins-bot: Replace jslint test with jshint and jsonlint tests [integration/config] - 10https://gerrit.wikimedia.org/r/273394 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [10:28:50] hashar: could you please review these simple 3 commits? :) https://gerrit.wikimedia.org/r/#/q/status:open+topic:T127423,n,z [10:28:51] T127423: Cleanup Cucumber tags in Selenium tests - https://phabricator.wikimedia.org/T127423 [10:28:53] (03CR) 10Hashar: "Deleted jobs" [integration/config] - 10https://gerrit.wikimedia.org/r/273394 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [10:29:11] they are the only commits left to resolve the task [10:34:09] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2066022 (10zeljkofilipin) [10:34:12] 7Browser-Tests, 10MediaWiki-extensions-MultimediaViewer, 13Patch-For-Review: Fix MultimediaViewer scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94157#2066016 (10zeljkofilipin) 5Open>3Resolved Resolving this task. I have created separate... [10:35:10] zeljkof: -1 ed https://gerrit.wikimedia.org/r/#/c/271781/1 :D [10:35:13] lets keep IE tags [10:35:20] and get daily job running against IE [10:35:23] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#1672803 (10zeljkofilipin) [10:36:04] hashar: the thing is, who will fix the job when it fails? [10:36:50] I am fine with keeping the tags, but I would not like to create more jobs that nobody cares about [10:36:56] (03CR) 10Hashar: [C: 031] "We only target beta cluster which runs out of master. So Ib5e0a1600c3df45889e97d5ff52cd0a77a15001f does not need any backport to wmf branc" [integration/config] - 10https://gerrit.wikimedia.org/r/271782 (owner: 10Zfilipin) [10:37:24] zeljkof: well you have been fixing browser tests for a couple weeks now and soonish they will be greenish [10:37:44] so I guess from there we would have killed the tech debt of failing tests and start enforcing folks to keep them green [10:37:50] with mwext-mw-selenium triggered to help [10:38:02] the IE might tests might well pass now [10:38:22] yes, I am for moving to mwext* jobs, but not for creating more daily jobs [10:38:23] they got mass removed because mediawiki_selenium lost the ability to support IE on sauce labs at some point. But that is fixed now [10:46:04] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2066035 (10zeljkofilipin) [10:47:11] 7Browser-Tests, 10MediaWiki-extensions-GettingStarted: Delete or fix failed GettingStarted browsertests Jenkins job - https://phabricator.wikimedia.org/T94154#1156375 (10zeljkofilipin) @Mattflaschen what is the hack you need to run to fix the problem? The job is failing again with `expected ​visible? ​to ​retu... [10:49:55] (03PS2) 10Hashar: Migrate operations-puppet tests to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272743 (owner: 10Paladox) [10:50:25] (03CR) 10Hashar: [C: 032] "pep8 / puppet are at the same version on Precise and Trusty slaves." [integration/config] - 10https://gerrit.wikimedia.org/r/272743 (owner: 10Paladox) [10:51:49] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#2066052 (10zeljkofilipin) [10:51:51] 7Browser-Tests, 10VisualEditor, 13Patch-For-Review, 5WMF-deploy-2016-03-01_(1.27.0-wmf.15): Fix VisualEditor scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94162#2066047 (10zeljkofilipin) 5Open>3Resolved Resolving this task. I have cre... [10:52:05] (03Merged) 10jenkins-bot: Migrate operations-puppet tests to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272743 (owner: 10Paladox) [10:52:09] 10Browser-Tests-Infrastructure, 5Release-Engineering-Epics, 7Epic, 7Tracking: Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily - https://phabricator.wikimedia.org/T94150#1674380 (10zeljkofilipin) [10:55:44] (03CR) 10Hashar: [C: 04-1] "phpize is a command shipped by the php Debian package. On Precise that is Zend 5.3 and on Trusty it is currently Zend 5.5." [integration/config] - 10https://gerrit.wikimedia.org/r/272746 (owner: 10Paladox) [11:08:04] hashar / jzerebecki for deployment / beta is there somewhere I cans ee the full debug logs? [11:08:06] *see [11:19:53] addshore: good morning [11:20:06] addshore: the same way one does in production: loooooogggggstashhhhhh jenkiiiiiins [11:20:15] addshore: https://logstash-beta.wmflabs.org [11:20:36] or alternatively some logs are wrote to disk on deployment-fluorine.deployment-prep.eqiad.wmflabs under /srv/mw-logs/ [11:20:53] poor beta has lost memcached apparently [11:20:55] !log Memcached error for key "enwiki:flow_workflow%3Av2%3Apk:63dc3cf6a7184c32477496d63c173f9c:4.8" on server "127.0.0.1:11212": SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY [11:20:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:21:54] * hashar blames nutcracker proxy [11:24:14] 10Beta-Cluster-Infrastructure: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177#2066223 (10hashar) [11:24:32] !log puppet keep restarting nutcracker apparently T128177 [11:24:33] T128177: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177 [11:24:35] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:26:27] !log beta: salt -v '*' cmd.run 'apt-get -y install ruby-msgpack' . I am tired of seeing puppet debug messages: "Debug: Failed to load library 'msgpack' for feature 'msgpack'" [11:26:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:32:30] 10Beta-Cluster-Infrastructure: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177#2066243 (10hashar) On deployment-mediawiki02 ``` # /sbin/status nutcracker nutcracker stop/waiting # ps -A f|grep nutcr 19327 pts/2 S+ 0:00 \_ grep --color... [11:39:06] 10Beta-Cluster-Infrastructure: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177#2066249 (10hashar) The other instances do not have the same issue though :-( [11:40:42] !log deployment-memc04 find /etc/apt -name '*proxy' -delete (prevented apt-get update) [11:40:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:47:38] hashar: cool! [11:47:47] is every call to wfDebugLog in there? [11:48:24] 10Beta-Cluster-Infrastructure: Investigate whether deployment-memc04 instance is still needed - https://phabricator.wikimedia.org/T128178#2066261 (10hashar) [11:48:41] addshore: I dont think [11:48:59] addshore: well there is a wfdebuglog file which might be the output of wfDebugLog calls [11:49:29] addshore: we also have the bucket logs (wfDebugLogGroup) but the groups have to be explicitly enabled via operations/mediawiki-config [11:49:45] maybe the logs to logstash are slightly different / more complete than the one written to disk [11:49:56] feel free to enable your wfDebugLogGroup bucket [11:52:14] okay! [11:53:08] !log memcached process on deployment-memc02 seems to have a nice leak of socket usages (from lost) and plainly refuse connections (bunch of CLOSE_WAIT) [11:53:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:53:12] !log memcached process on deployment-memc02 seems to have a nice leak of socket usages (from lost) and plainly refuse connections (bunch of CLOSE_WAIT) T128177 [11:53:13] T128177: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177 [11:53:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:53:54] !log Restarted memcached on deployment-memc02 T128177 [11:53:55] T128177: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177 [11:53:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [11:57:51] 10Beta-Cluster-Infrastructure: memcached/nutcracker mis behaving on beta cluster - https://phabricator.wikimedia.org/T128177#2066294 (10hashar) 5Open>3Resolved Root cause: memcached service on deployment-memc02 went wild. Restarting it has fixed the issue. [12:33:00] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/273394 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [12:33:14] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/272743 (owner: 10Paladox) [13:52:30] PROBLEM - Puppet failure on integration-slave-trusty-1005 is CRITICAL: CRITICAL: 12.50% of data above the critical threshold [0.0] [13:53:25] boo puppet [13:56:44] PROBLEM - Puppet failure on integration-slave-trusty-1002 is CRITICAL: CRITICAL: 77.78% of data above the critical threshold [0.0] [13:58:52] 10Deployment-Systems, 6Operations, 6Performance-Team, 10Traffic, 5MW-1.27-release-notes: Make Varnish cache for /static/$wmfbranch/ expire when resources change within branch lifetime - https://phabricator.wikimedia.org/T99096#2066474 (10Krinkle) [13:59:06] 10Deployment-Systems, 5Release-Engineering-Epics, 7Epic: Merge to deployed branches instead of cutting a new deployment branch every week. - https://phabricator.wikimedia.org/T89945#2066480 (10Krinkle) [13:59:09] 10Deployment-Systems, 6Operations, 6Performance-Team, 10Traffic, 5MW-1.27-release-notes: Make Varnish cache for /static/$wmfbranch/ expire when resources change within branch lifetime - https://phabricator.wikimedia.org/T99096#1457784 (10Krinkle) 5Open>3Resolved [14:12:32] RECOVERY - Puppet failure on integration-slave-trusty-1005 is OK: OK: Less than 1.00% above the threshold [0.0] [14:26:43] RECOVERY - Puppet failure on integration-slave-trusty-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [14:32:51] 10Continuous-Integration-Infrastructure, 6Operations, 10Traffic: Make CI run Varnish 4 VCL tests - https://phabricator.wikimedia.org/T128188#2066542 (10hashar) [14:44:03] !log Nodepool has triggered 40 000 instances [14:44:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [14:44:16] !log (since it started, dont be that scared!) [14:44:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [14:47:14] * hashar whistles [14:53:21] 10Browser-Tests-Infrastructure: Simplify creating of Jenkins jobs for running browser tests daily - https://phabricator.wikimedia.org/T128190#2066598 (10zeljkofilipin) [14:53:47] 10Browser-Tests-Infrastructure: Simplify creating of Jenkins jobs for running browser tests daily - https://phabricator.wikimedia.org/T128190#2066614 (10zeljkofilipin) [14:53:50] 10Browser-Tests-Infrastructure, 10MediaWiki-extensions-RelatedArticles, 13Patch-For-Review: browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce job fails with: unknown environment `beta` (MediawikiSelenium::ConfigurationError) - https://phabricator.wikimedia.org/T128084#2063500 (10... [14:57:02] (03PS1) 10Hashar: Migrate mediawiki npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/273457 (https://phabricator.wikimedia.org/T119143) [14:57:40] (03PS2) 10Hashar: Migrate mediawiki npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/273457 (https://phabricator.wikimedia.org/T119143) [15:00:03] (03CR) 10Hashar: [C: 032] Migrate mediawiki npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/273457 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [15:02:37] (03Merged) 10jenkins-bot: Migrate mediawiki npm job to Nodepool [integration/config] - 10https://gerrit.wikimedia.org/r/273457 (https://phabricator.wikimedia.org/T119143) (owner: 10Hashar) [15:06:04] (03PS1) 10JanZerebecki: Revert "mediawiki.conf: Set $wgScribuntoDefaultEngine for speed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273460 [15:08:33] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066627 (10aude) [15:08:39] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066639 (10aude) p:5Triage>3Unbreak! [15:09:38] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066627 (10aude) this is causing anything depending on the wikidata build to fail, such as Content... [15:10:21] (03PS2) 10Aude: Revert "mediawiki.conf: Set $wgScribuntoDefaultEngine for speed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273460 (https://phabricator.wikimedia.org/T128191) (owner: 10JanZerebecki) [15:10:55] (03CR) 10Aude: [C: 031] "i like the change to use luasandbox, but we need to make the tests not depend on configuration like this." [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273460 (https://phabricator.wikimedia.org/T128191) (owner: 10JanZerebecki) [15:10:57] (03CR) 10JanZerebecki: [C: 032] Revert "mediawiki.conf: Set $wgScribuntoDefaultEngine for speed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273460 (https://phabricator.wikimedia.org/T128191) (owner: 10JanZerebecki) [15:11:45] (03Merged) 10jenkins-bot: Revert "mediawiki.conf: Set $wgScribuntoDefaultEngine for speed" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/273460 (https://phabricator.wikimedia.org/T128191) (owner: 10JanZerebecki) [15:14:00] !log salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" [15:14:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [15:14:19] !log salt -v --show-timeout '*slave*' cmd.run "bash -c 'cd /srv/deployment/integration/slave-scripts; git pull'" T128191 [15:14:20] T128191: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191 [15:14:22] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [15:17:11] (03Abandoned) 10Hashar: Migrate python-jobs to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272747 (owner: 10Paladox) [15:17:46] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #826: 04FAILURE in 11 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/826/ [15:21:39] (03CR) 10Hashar: [C: 04-1] "This is missing a definition of the job hhvmlint." [integration/config] - 10https://gerrit.wikimedia.org/r/270938 (https://phabricator.wikimedia.org/T97889) (owner: 10Paladox) [15:32:28] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata, 13Patch-For-Review: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066670 (10JanZerebecki) [15:32:31] 10Continuous-Integration-Config, 13Patch-For-Review: Have CI set `$wgScribuntoDefaultEngine = 'luasandbox` to speed up parser tests - https://phabricator.wikimedia.org/T126670#2066669 (10JanZerebecki) [15:32:35] (03CR) 10Hashar: [C: 04-1] "Some jobs can not be moved right now. They should be switched in different changes. See my comments inline." (035 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/272749 (owner: 10Paladox) [15:35:13] hashar: I hate JJB :| [15:35:18] :D [15:41:53] hi zeljkof [15:42:03] Nikerabbit: hi! [15:44:21] (03PS2) 10Paladox: Migrate job-templates to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272749 [15:44:57] (03CR) 10Paladox: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/273398 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [15:45:08] hashar: Hi could you review https://gerrit.wikimedia.org/r/#/c/273398/ please. [15:45:17] (03CR) 10jenkins-bot: [V: 04-1] Remove template extension-checks, Replace with jshint, jsonlint and mw-checks-test [integration/config] - 10https://gerrit.wikimedia.org/r/273398 (https://phabricator.wikimedia.org/T127362) (owner: 10Paladox) [15:46:00] why wouldn't hate TLA [15:47:27] (03PS1) 10Hashar: (WIP) playing with Fabric (WIP) [integration/config] - 10https://gerrit.wikimedia.org/r/273466 [15:47:32] (03PS2) 10Paladox: Remove template extension-checks, Replace with jshint, jsonlint and mw-checks-test [integration/config] - 10https://gerrit.wikimedia.org/r/273398 (https://phabricator.wikimedia.org/T127362) [15:48:47] (03CR) 10Paladox: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) (owner: 10Paladox) [15:55:07] (03PS4) 10Paladox: Add builder npm to npm-node-4.3 [integration/config] - 10https://gerrit.wikimedia.org/r/273269 (https://phabricator.wikimedia.org/T128090) [15:59:00] (03PS3) 10Paladox: Migrate some job templates to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272749 [16:04:20] legoktm: Aha. [16:15:16] (03PS6) 10Paladox: Add new hhvm lint and use it [integration/config] - 10https://gerrit.wikimedia.org/r/270938 (https://phabricator.wikimedia.org/T97889) [16:19:06] (03PS7) 10Paladox: Add new hhvm lint and use it [integration/config] - 10https://gerrit.wikimedia.org/r/270938 (https://phabricator.wikimedia.org/T97889) [16:26:08] Yippee, build fixed! [16:26:08] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #830: 09FIXED in 39 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/830/ [16:46:34] 10Continuous-Integration-Infrastructure, 10Analytics, 13Patch-For-Review: Add json linting test for schemas in mediawiki/event-schemas - https://phabricator.wikimedia.org/T124319#1952779 (10Ottomata) Also related: T124438 [16:53:17] 10Continuous-Integration-Infrastructure, 10Analytics, 13Patch-For-Review: Add json linting test for schemas in mediawiki/event-schemas - https://phabricator.wikimedia.org/T124319#2066845 (10bd808) [16:54:34] (03PS1) 10Paladox: Look for apps-android-wikipedia/www explicity in npm [integration/config] - 10https://gerrit.wikimedia.org/r/273482 [17:02:26] thedj: I belive i found where the problem with the config in timedmediahandler is comming from. It is in js. [17:02:57] thedj: I belive that the config has no impact in js. It search for any files regardless off the setting. [17:10:33] (03PS2) 10Paladox: Migrate php-extensions to UbuntuTrusty [integration/config] - 10https://gerrit.wikimedia.org/r/272746 [17:11:02] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #833: 04FAILURE in 10 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/833/ [17:13:41] Yippee, build fixed! [17:13:41] Project browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #834: 09FIXED in 30 sec: https://integration.wikimedia.org/ci/job/browsertests-Math-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/834/ [17:15:54] legoktm: there seems to be duplicate composer-test template [17:16:40] legoktm: ones named composer-test and the other composer-test55 but composer-test has php55 tests and composer 55 tests. [17:17:38] paladox: do you mean composer-package-test55? [17:17:57] legoktm: Nope. [17:18:13] legoktm: But looking that seems to also be duplicated. [17:18:30] (03PS1) 10Zfilipin: WIP Simplify creating of Jenkins jobs for running browser tests daily [integration/config] - 10https://gerrit.wikimedia.org/r/273489 (https://phabricator.wikimedia.org/T128190) [17:18:31] legoktm: https://dpaste.de/bqGh [17:19:36] (03CR) 10jenkins-bot: [V: 04-1] WIP Simplify creating of Jenkins jobs for running browser tests daily [integration/config] - 10https://gerrit.wikimedia.org/r/273489 (https://phabricator.wikimedia.org/T128190) (owner: 10Zfilipin) [17:20:00] paladox: ah, it's for repositories that don't have the branch filter which prevents php53 jobs from running [17:20:08] paladox: only repositories under mediawiki/* have that [17:20:18] legoktm: Oh ok, thanks for explaning. [18:18:27] (03PS2) 10Paladox: [Metrolook] Add mw-checks-test test [integration/config] - 10https://gerrit.wikimedia.org/r/265099 [18:40:41] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata, 13Patch-For-Review: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066627 (10Anomie) > That change is good, but we need to improve the tests s... [19:11:55] (03CR) 10JanZerebecki: [C: 04-1] "Convention over configuration, please. Fix repo to work with convention." [integration/config] - 10https://gerrit.wikimedia.org/r/273482 (owner: 10Paladox) [19:13:24] (03CR) 10Paladox: "@JanZerebecki actually hashar said we could possibly leave package.json under www/ and update it some how." [integration/config] - 10https://gerrit.wikimedia.org/r/273482 (owner: 10Paladox) [19:15:06] (03CR) 10Hashar: "The idea I proposed is to have /package.json that would define the test entry point, then have the actual command to cd www/ && grunt tes" [integration/config] - 10https://gerrit.wikimedia.org/r/273482 (owner: 10Paladox) [19:15:09] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Scribunto, 10Wikidata, 13Patch-For-Review: Wikibase Scribunto tests are failing, with the tests depending on configuration - https://phabricator.wikimedia.org/T128191#2066627 (10JanZerebecki) Fix for the instanceof problem: https://gerrit.wiki... [19:16:06] (03CR) 10Paladox: "Oh thanks for explaning." [integration/config] - 10https://gerrit.wikimedia.org/r/273482 (owner: 10Paladox) [19:40:01] PROBLEM - Free space - all mounts on deployment-fluorine is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine.diskspace._srv.byte_percentfree (<25.00%) [20:20:25] PROBLEM - Free space - all mounts on deployment-analytics03 is CRITICAL: CRITICAL: deployment-prep.deployment-analytics03.diskspace.root.byte_percentfree (<100.00%) [20:21:02] 6Release-Engineering-Team: [betalabs] "Cross-Origin Request Blocked" and "the content must be served over HTTPS" console errors - https://phabricator.wikimedia.org/T128207#2067356 (10Etonkovidova) [20:22:18] 10Beta-Cluster-Infrastructure, 10MediaWiki-Authentication-and-authorization: [betacluster] "Cross-Origin Request Blocked" and "the content must be served over HTTPS" console errors - https://phabricator.wikimedia.org/T128207#2067374 (10greg) [20:28:03] 10Beta-Cluster-Infrastructure, 10MediaWiki-Authentication-and-authorization: [betacluster] "Cross-Origin Request Blocked" and "the content must be served over HTTPS" console errors - https://phabricator.wikimedia.org/T128207#2067356 (10Catrope) This seems related, but may be a separate bug: when logging in on... [20:29:35] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #937: 15ABORTED in 3 min 34 sec: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/937/ [20:42:53] hashar: Hi i am getting this error https://integration.wikimedia.org/ci/job/npm-node-4.3/67/console [21:01:21] 10Continuous-Integration-Infrastructure, 10Wikimedia-Logstash, 7Jenkins: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#2067488 (10hashar) 5Resolved>3Open p:5Low>3Normal [21:11:55] 10Continuous-Integration-Infrastructure, 10Wikimedia-Logstash, 7Jenkins: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#2067535 (10hashar) From [[ https://wiki.jenkins-ci.org/display/JENKINS/Logstash+Plugin | the plugin page ]]: The plugin can write directly t... [21:34:01] PROBLEM - Puppet failure on deployment-analytics03 is CRITICAL: CRITICAL: 42.86% of data above the critical threshold [0.0] [21:35:18] 10Continuous-Integration-Infrastructure, 10Wikimedia-Logstash, 7Jenkins: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705#2067603 (10hashar) For disk space. Gallium has 273GB used all included (system, websites docroots, build results, console logs etc). I looke... [22:12:24] 6Release-Engineering-Team: MW-1.27-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T128218#2067682 (10greg) [22:14:18] greg-g: have you looked at Phabricator sub project feature ? [22:14:38] that .16 blocker sounds like: Releng => MW Train => .16 (milestone) [22:14:40] a bit [22:14:47] I don't want to go overboard with it yet [22:14:53] yeah [22:14:57] and recreate the mess we had in Bugzilla [22:15:00] lets TPG experiment with it first maybe [22:15:13] ontologies are hard, yo [22:15:17] for realz [22:16:09] 6Release-Engineering-Team: MW-1.27-wmf.16 deployment blockers - https://phabricator.wikimedia.org/T128218#2067697 (10greg) p:5Triage>3Normal [22:22:08] hashar: Hi i doint think it is possible to do cd www/ && grunt test since gruntfile will have to be in the same place as package.json so it can find the packages. [22:27:07] legoktm: Hi could you review https://gerrit.wikimedia.org/r/#/c/243719/ and https://gerrit.wikimedia.org/r/#/c/266068/ [22:27:09] please. [22:29:15] paladox: have you looked at grunt doc ? seems one can: grunt --gruntfile www/Gruntfile.js [22:29:33] hashar: Yes grunt is found fine but packages doint. [22:29:53] I dont really [22:29:54] hashar: Looking for node_modules in grunt is hard coded. [22:29:57] i dont know really [22:30:02] maybe just give up on that repo ? [22:30:35] hashar: Ok. But we need to switch the tests but it is really a simple fix that we move the package.json and gruntfile to parrent directory. [22:35:13] hashar: It looks like jenkins is being bumped to 2.x. [22:35:25] yup [22:35:43] they are building the roadmap / feature set afaik [22:36:03] that https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+2.0 [22:36:38] you can ask around in #jenkins :-} [22:36:58] I am myself heading to bed for real! See you next week and thank you again for all the nice CI patches! [22:37:19] hashar: Ok. [22:37:25] your welcome. [22:39:37] hashar: Im going to do some more jslint to jshint and jsonlint migration in a min. [22:40:42] will probably merge them over the week-end as time allow [22:41:47] hashar: Ok thanks. [22:52:08] hashar: Ive removed the extension-checks template in https://gerrit.wikimedia.org/r/#/c/273398/ [22:52:23] \O/ [22:57:41] hashar: Looks like there are still 163 tests left to be converted before jslint can be removed totatly and then the migration shifts to converting to npm. :). [23:02:59] paladox: Nice. [23:03:42] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [23:04:29] James_F: hey mediawiki-core-npm got moved to nodepool :D [23:04:32] James_F thanks [23:04:46] James_F: and I have noticed all the paladox tasks related to oojs/ve and the crazy dependencies [23:05:01] hashar: I saw. [23:05:08] hashar: Not 'crazy'. ;-) [23:05:10] we will see what happens. But for now internet is going down! [23:05:26] hashar: But yeah, not trivial. [23:05:26] See you, hashar. [23:05:38] yup not trivial [23:05:55] but for sure solvable [23:05:57] hashar: Could chromium be installed on nodepool. I have this patch https://gerrit.wikimedia.org/r/#/c/273269/ [23:06:00] !eject hashar [23:06:07] Bye hashar. [23:06:25] paladox: it is more complicated. Will explain on that task next week ;-} [23:06:37] hashar oh ok.