[00:57:19] (03CR) 10MarkAHershberger: "No longer maintainer" [tools/release] - 10https://gerrit.wikimedia.org/r/182643 (https://phabricator.wikimedia.org/T529) (owner: 10MarkAHershberger) [00:57:21] (03Abandoned) 10MarkAHershberger: update tests to check patches and signatures [tools/release] - 10https://gerrit.wikimedia.org/r/182643 (https://phabricator.wikimedia.org/T529) (owner: 10MarkAHershberger) [00:57:23] (03CR) 10MarkAHershberger: "No longer maintainer" [tools/release] - 10https://gerrit.wikimedia.org/r/166526 (owner: 10MarkAHershberger) [00:57:25] (03Abandoned) 10MarkAHershberger: make-release: Make make-extension-branches more useful for other (e.g. skin) repos [tools/release] - 10https://gerrit.wikimedia.org/r/166526 (owner: 10MarkAHershberger) [00:57:27] (03CR) 10MarkAHershberger: "No longer maintainer" [tools/release] - 10https://gerrit.wikimedia.org/r/166525 (owner: 10MarkAHershberger) [00:57:29] (03Abandoned) 10MarkAHershberger: make-release: Give make-extension-branches the option print out a MW table of branchpoints [tools/release] - 10https://gerrit.wikimedia.org/r/166525 (owner: 10MarkAHershberger) [00:57:31] (03CR) 10MarkAHershberger: "No longer maintainer" [tools/release] - 10https://gerrit.wikimedia.org/r/185657 (owner: 10MarkAHershberger) [00:57:33] (03Abandoned) 10MarkAHershberger: make-release: CamelCase away! [tools/release] - 10https://gerrit.wikimedia.org/r/185657 (owner: 10MarkAHershberger) [01:02:46] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1615332 (10Jdlrobson) @greg any chance of this happening within the next month? If not what can be done to speed that up? I have no knowledge of the deploy process so have no... [03:21:32] 10Beta-Cluster, 10MediaWiki-extensions-OAuth, 10Pywikibot-OAuth, 7Pywikibot-network, 7Pywikibot-tests: "Nonce already used" regularly occurring on beta cluster - https://phabricator.wikimedia.org/T109173#1615396 (10Krinkle) [03:37:46] 6Release-Engineering: Move jquery.i18n, jquery.ime and jquery.uls to gerrit - https://phabricator.wikimedia.org/T111684#1615417 (10santhosh) @Mjbmr, @hashar is correct about the reasons and process. They are mediawiki independed jquery libraries and used by third party projects. We get several PRs from non-medi... [04:26:48] 6Release-Engineering: Move jquery.i18n, jquery.ime and jquery.uls to gerrit - https://phabricator.wikimedia.org/T111684#1615438 (10Qgil) 5Open>3Resolved The Language team has explained their reasons not to have jquery.i18n, jquery.ime, and jquery.uls to Wikimedia's Gerrit. After reading @santhosh's reply, th... [04:30:55] 6Release-Engineering: Move jquery.i18n, jquery.ime and jquery.uls to gerrit - https://phabricator.wikimedia.org/T111684#1615444 (10Legoktm) 5Resolved>3declined Related: T109920. [04:31:08] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #555: FAILURE in 39 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/555/ [06:36:25] RECOVERY - Free space - all mounts on deployment-bastion is OK All targets OK [07:16:30] (03CR) 10Zfilipin: "Superseded by https://gerrit.wikimedia.org/r/#/c/236540/" [integration/config] - 10https://gerrit.wikimedia.org/r/236019 (https://phabricator.wikimedia.org/T111510) (owner: 10Zfilipin) [08:19:48] 10Continuous-Integration-Infrastructure: Move https://github.com/legoktm/tools-ci to gerrit - https://phabricator.wikimedia.org/T111758#1615712 (10Legoktm) 3NEW [08:26:29] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1615736 (10zeljkofilipin) I have managed to reproduce it locally, with this `config.yml` ``` lang=bash # beta inst... [08:34:34] legoktm: thanks for the PR merges :-} [08:35:01] hashar: I didn't merge anything yet, see https://github.com/legoktm/tools-ci/pull/1 [08:37:49] 10Continuous-Integration-Infrastructure, 10Gitblit-Deprecate, 5Patch-For-Review, 7Technical-Debt: Remove CI dependencies on git.wikimedia.org - https://phabricator.wikimedia.org/T74001#1615750 (10hashar) Well done! [08:42:56] 10Continuous-Integration-Infrastructure: Move https://github.com/legoktm/tools-ci to gerrit - https://phabricator.wikimedia.org/T111758#1615772 (10hashar) +@krinkle since he always has good feedback. For the context, the github repo hosts an utility to generate the state of our repos configuration: https://www.... [08:55:25] hashar: merged all of them :) and I gave you access to the repo, feel free to direct push [08:58:10] 10Continuous-Integration-Config, 7I18n: Configure banana checker for i18n files to run on all MediaWiki extensions (tracking) - https://phabricator.wikimedia.org/T94547#1615811 (10Prtksxna) [09:04:10] 5Continuous-Integration-Scaling, 6operations, 7Nodepool: Backport python-shade from debian/testing to jessie-wikimedia - https://phabricator.wikimedia.org/T107267#1615827 (10jcrespo) p:5Triage>3Normal Normal as it doesn't seem #Blocked-on-operations. [09:06:39] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1615831 (10zeljkofilipin) This is the problem: https://github.com/wmde/WikidataApiGem/blob/master/lib/mediawiki_ap... [09:09:20] hashar: I googled for a syntax checker and found a module that checks for several languages (including JS). Looking at its source, what it does is to call uglifyjs and send its output to /dev/null. Presumably it complains if the syntax is invalid, and voila, syntax checker. :P [09:09:28] The package I looked at was https://github.com/xianhuazhou/syntax-checker/ [09:09:39] Probably not the best way to do it, but certainly a "possibly" solution. [09:09:54] Refer to these lines for a look: https://github.com/xianhuazhou/syntax-checker/blob/master/lib/syntax-checker.js#L18-L28 [09:10:09] polybuildr: I guess we could use it instead of jshint so. Would you mind replying to my mail on the wikitech-l list please ? [09:10:10] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1615839 (10zeljkofilipin) ``` lang=ruby [8] pry(#)> resp.data["entities"] =... [09:10:20] hashar: sure! [09:17:47] hashar: replied :) [09:21:03] hashar: Also, what about running jshint against the code with all options disabled? [09:21:35] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1615862 (10zeljkofilipin) [09:25:37] 10Continuous-Integration-Config: Migrate bundle17 jobs to bundle - https://phabricator.wikimedia.org/T111764#1615865 (10hashar) 3NEW [09:26:08] (03CR) 10Hashar: "The Vagrant gem 1.7.3 supports bundle <= 1.10.5. I have filled T111764 to potentially reunify the bundle17 and bundle job templates." [integration/config] - 10https://gerrit.wikimedia.org/r/191375 (owner: 10Dduvall) [09:27:47] 10Beta-Cluster, 6Labs, 10Labs-Infrastructure, 7Graphite, 7Shinken: Delete more specific deployment-prep graphite datapoints - https://phabricator.wikimedia.org/T111540#1615880 (10fgiunchedi) agreed, what would be the easiest way to get a map of project -> list of instances? @yuvipanda @andrew ? [09:30:32] hashar: there's also https://github.com/marijnh/acorn, what do you think of that? [09:31:01] polybuildr: no idea, I am JavaScript illiterate :-( [09:31:12] polybuildr: maybe jshint might works as well [09:31:24] not sure how to make it only validate the syntax though [09:32:08] Yippee, build fixed! [09:32:08] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #715: FIXED in 1 hr 22 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/715/ [09:34:41] hashar: acorn is a JS parser. So it can be used just to check syntax. [09:35:26] polybuildr: looks like. As I said post your suggestions to the list so that people actually knowing about JavaScript can have a look at them :-D [09:35:38] oh you did [09:35:40] \O/ [09:35:49] alright alright :P I'm in the middle of my midterms, so I don't want to spend time crafting an email :P [09:37:24] twentyafterfour, greg-g: the non-deployment of https://gerrit.wikimedia.org/r/#/c/234839/ is a blocker of the train deployment today [09:46:49] 10Beta-Cluster, 6operations, 7Database: Possible to run writes (e.g. UPDATE) on slave - https://phabricator.wikimedia.org/T110115#1615949 (10jcrespo) p:5Triage>3Normal Analytics stores (dbstoreX) are all with read_only=0 (aka s3-analytics-slave). I do not know yet, I suppose that because they have their... [09:54:02] 10Beta-Cluster, 6operations, 7Database: Possible to run writes (e.g. UPDATE) on slave - https://phabricator.wikimedia.org/T110115#1615979 (10hashar) From `wmf-config/db-labs.php` the #beta-cluster has two SQL servers, a master and a slave: ``` 'hostsByName' => array( 'deployment-db1' =>... [10:29:00] 10Beta-Cluster, 10Traffic, 6operations, 5Patch-For-Review: Upgrade beta-cluster caches to jessie - https://phabricator.wikimedia.org/T98758#1616043 (10hashar) 5Resolved>3Open The Parsoid cache `deployment-parsoidcache02` is still on Trusty :( {T103660} [10:29:15] 10Beta-Cluster, 10Traffic, 6operations, 5Patch-For-Review: Upgrade beta-cluster caches to jessie - https://phabricator.wikimedia.org/T98758#1616048 (10hashar) [10:29:17] 10Beta-Cluster, 7Varnish: deployment-parsoidcache02 fails puppet: Role::Cache::Statsd/Varnish::Logging::Statsd[default]/Base::Service_unit[varnishstatsd-default]/Service[varnishstatsd-default]: Could not evaluate: Could not find init script or upstart conf file... - https://phabricator.wikimedia.org/T103660#1395617 [10:30:04] 10Beta-Cluster, 7Varnish: deployment-parsoidcache02 fails puppet: Role::Cache::Statsd/Varnish::Logging::Statsd[default]/Base::Service_unit[varnishstatsd-default]/Service[varnishstatsd-default]: Could not evaluate: Could not find init script or upstart conf file... - https://phabricator.wikimedia.org/T103660#1395617 [10:30:56] 10Browser-Tests: Tutorial to setup a Raita dev environment - https://phabricator.wikimedia.org/T111493#1616057 (10hashar) 5Open>3Resolved a:3hashar I haven't thought about MediaWiki Vagrant :-} Thanks, I guess that cover my needs. [10:31:17] 10Continuous-Integration-Infrastructure, 6Discovery, 6operations, 7Elasticsearch, 5Patch-For-Review: elasticsearch 1.6.0 fails to start after reboot - https://phabricator.wikimedia.org/T109497#1616063 (10jcrespo) I do not know which is the consensus here. Should we apply the patch or should we abandon pr... [10:57:45] 10Beta-Cluster, 6Labs, 10Labs-Infrastructure, 7Graphite, 7Shinken: Delete more specific deployment-prep graphite datapoints - https://phabricator.wikimedia.org/T111540#1616099 (10yuvipanda) eb3e3dbd81d263791d2ba1909f64f8a84531c65e for my revert of my original garbage collector script. It failed because t... [11:06:47] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1616104 (10zeljkofilipin) When sitelink exists (example: `"enwiki", "Asia"`), `resp.data["entities"]` returns ```... [11:11:14] 10Browser-Tests, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: fix no implicit conversion of String into Integer (TypeError) in browsertests - https://phabricator.wikimedia.org/T111069#1616107 (10zeljkofilipin) The documentation for [[ https://github.com/wmde/WikidataApiGem/blob/master/README.md#usa... [11:12:47] (03PS1) 10Aude: Update Wikidata - wmf/1.26wmf22 [tools/release] - 10https://gerrit.wikimedia.org/r/236763 [11:13:09] (03CR) 10Aude: [C: 032] Update Wikidata - wmf/1.26wmf22 [tools/release] - 10https://gerrit.wikimedia.org/r/236763 (owner: 10Aude) [11:31:14] (03Merged) 10jenkins-bot: Update Wikidata - wmf/1.26wmf22 [tools/release] - 10https://gerrit.wikimedia.org/r/236763 (owner: 10Aude) [12:16:23] (03PS1) 10Hashar: nodepool: placeholder setup script [integration/config] - 10https://gerrit.wikimedia.org/r/236770 (https://phabricator.wikimedia.org/T111377) [12:21:52] (03PS2) 10Hashar: nodepool: placeholder setup script [integration/config] - 10https://gerrit.wikimedia.org/r/236770 (https://phabricator.wikimedia.org/T111377) [12:25:39] (03CR) 10Hashar: [C: 032] nodepool: placeholder setup script [integration/config] - 10https://gerrit.wikimedia.org/r/236770 (https://phabricator.wikimedia.org/T111377) (owner: 10Hashar) [12:26:23] (03Merged) 10jenkins-bot: nodepool: placeholder setup script [integration/config] - 10https://gerrit.wikimedia.org/r/236770 (https://phabricator.wikimedia.org/T111377) (owner: 10Hashar) [12:36:05] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616170 (10zeljkofilipin) Unlike T111069, I can not reproduce this one locally. [12:42:56] (03PS15) 10Hashar: Disk image building [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) [12:43:22] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616184 (10zeljkofilipin) I have created a new job, [[ https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce... [12:55:07] Yippee, build fixed! [12:55:07] Project browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #589: FIXED in 1 min 6 sec: https://integration.wikimedia.org/ci/job/browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/589/ [13:05:09] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616201 (10zeljkofilipin) I have created a new job (using JJB) [[ https://integration.wikimedia.org/ci/job/browsertests-Wikidata-T110510-linux-firef... [13:07:52] 10Beta-Cluster, 6Labs, 10Labs-Infrastructure, 7Graphite, 7Shinken: Delete more specific deployment-prep graphite datapoints - https://phabricator.wikimedia.org/T111540#1616217 (10fgiunchedi) a:3fgiunchedi I'll take this, setting to low [13:07:59] 10Beta-Cluster, 6Labs, 10Labs-Infrastructure, 7Graphite, 7Shinken: Delete more specific deployment-prep graphite datapoints - https://phabricator.wikimedia.org/T111540#1616219 (10fgiunchedi) p:5Triage>3Low [13:31:09] (03PS16) 10Hashar: Disk image building [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) [13:31:11] (03PS1) 10Hashar: nodepool: set instance hostname via sudo [integration/config] - 10https://gerrit.wikimedia.org/r/236775 [13:32:07] (03PS2) 10Hashar: nodepool: set instance hostname via sudo [integration/config] - 10https://gerrit.wikimedia.org/r/236775 (https://phabricator.wikimedia.org/T111377) [13:33:16] (03CR) 10Hashar: [C: 032] "Good enough for now." [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) (owner: 10Hashar) [13:33:31] (03CR) 10Hashar: [C: 032] nodepool: set instance hostname via sudo [integration/config] - 10https://gerrit.wikimedia.org/r/236775 (https://phabricator.wikimedia.org/T111377) (owner: 10Hashar) [13:38:54] (03CR) 10Gilles: [C: 031] scm: Enable shallow-clone for git-remoteonly-zuul [integration/config] - 10https://gerrit.wikimedia.org/r/195021 (owner: 10Krinkle) [13:40:37] (03PS11) 10Hashar: (WIP) dib: wikimedia-puppet element (WIP) [integration/config] - 10https://gerrit.wikimedia.org/r/234975 (https://phabricator.wikimedia.org/T110735) [13:41:22] (03CR) 10jenkins-bot: [V: 04-1] Disk image building [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) (owner: 10Hashar) [13:41:24] (03CR) 10jenkins-bot: [V: 04-1] nodepool: set instance hostname via sudo [integration/config] - 10https://gerrit.wikimedia.org/r/236775 (https://phabricator.wikimedia.org/T111377) (owner: 10Hashar) [13:42:01] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616267 (10zeljkofilipin) Strange, the error message says the problem is in Cucumber's default (pretty) formatter. ``` 00:00:04.450 ... bundle exe... [13:43:46] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616272 (10zeljkofilipin) When I switched Cucumber to another formatter (progress), the job does not fail: https://integration.wikimedia.org/ci/job... [13:43:53] (03CR) 10Hashar: [C: 032] "I forgot to deploy the Jenkins job integration-config-tox-dib-trusty" [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) (owner: 10Hashar) [13:45:40] (03Merged) 10jenkins-bot: Disk image building [integration/config] - 10https://gerrit.wikimedia.org/r/234300 (https://phabricator.wikimedia.org/T110733) (owner: 10Hashar) [13:45:42] (03Merged) 10jenkins-bot: nodepool: set instance hostname via sudo [integration/config] - 10https://gerrit.wikimedia.org/r/236775 (https://phabricator.wikimedia.org/T111377) (owner: 10Hashar) [13:47:21] (03PS12) 10Hashar: dib: wikimedia-puppet element [integration/config] - 10https://gerrit.wikimedia.org/r/234975 (https://phabricator.wikimedia.org/T110735) [13:50:05] (03CR) 10Hashar: [C: 04-1] "Still need:" [integration/config] - 10https://gerrit.wikimedia.org/r/234975 (https://phabricator.wikimedia.org/T110735) (owner: 10Hashar) [13:55:35] (03PS13) 10Hashar: dib: wikimedia-puppet element [integration/config] - 10https://gerrit.wikimedia.org/r/234975 (https://phabricator.wikimedia.org/T110735) [14:07:29] Project browsertests-Wikidata-T110510-linux-firefox-sauce build #1: FAILURE in 9 min 9 sec: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-T110510-linux-firefox-sauce/1/ [14:08:29] (03CR) 10Paladox: [C: 04-1] "Requires first the patch https://gerrit.wikimedia.org/r/#/c/225242/ be merged then this one here." [integration/config] - 10https://gerrit.wikimedia.org/r/225239 (owner: 10Paladox) [14:09:31] (03CR) 10Paladox: [C: 04-1] "Requires first that the patch https://gerrit.wikimedia.org/r/#/c/225314/ be merged then this one here." [integration/config] - 10https://gerrit.wikimedia.org/r/225313 (owner: 10Paladox) [14:10:30] (03CR) 10Paladox: [C: 04-1] "Requires that https://gerrit.wikimedia.org/r/#/c/225310/ first be merged then the one here." [integration/config] - 10https://gerrit.wikimedia.org/r/225311 (owner: 10Paladox) [14:11:56] (03CR) 10Paladox: "Requires that https://gerrit.wikimedia.org/r/#/c/225713/ first be merged then this one here." [integration/config] - 10https://gerrit.wikimedia.org/r/225712 (owner: 10Paladox) [14:12:13] (03CR) 10Paladox: [C: 04-1] Update TwitterLogin tests [integration/config] - 10https://gerrit.wikimedia.org/r/225712 (owner: 10Paladox) [14:19:45] (03PS7) 10Paladox: Update BlueSky tests [integration/config] - 10https://gerrit.wikimedia.org/r/226635 [14:21:06] (03CR) 10Paladox: "This is ready to be reviewed because I am deciding to split the patch so composer-test and npm will be added in separate patch. In this pa" [integration/config] - 10https://gerrit.wikimedia.org/r/226635 (owner: 10Paladox) [14:23:00] 10Continuous-Integration-Infrastructure, 6Discovery, 6operations, 7Elasticsearch, 5Patch-For-Review: elasticsearch 1.6.0 fails to start after reboot - https://phabricator.wikimedia.org/T109497#1616339 (10hashar) Can we identify the jobs that actually need ElasticSearch? We could move the jobs to Trusty.... [14:23:14] (03CR) 10Paladox: "Also this extension has js files so adding a test to test them." [integration/config] - 10https://gerrit.wikimedia.org/r/226635 (owner: 10Paladox) [14:24:32] 10Continuous-Integration-Infrastructure, 6Discovery, 6operations, 7Blocked-on-Operations, 7Elasticsearch: Please backport ElasticSearch 1.7.x from wikimedia-trusty to wikimedia-precise for CI needs - https://phabricator.wikimedia.org/T111781#1616342 (10hashar) 3NEW [14:25:21] 10Continuous-Integration-Infrastructure, 6Discovery, 6operations, 7Elasticsearch, 5Patch-For-Review: elasticsearch 1.6.0 fails to start after reboot - https://phabricator.wikimedia.org/T109497#1550327 (10hashar) Filled T111781 to request the backport to Precise. Then we will just upgrade the package on... [14:28:01] PROBLEM - Puppet failure on integration-slave-trusty-1012 is CRITICAL 60.00% of data above the critical threshold [0.0] [14:38:14] 10Continuous-Integration-Infrastructure: use a way to run other PHP versions not provided by the current distribution to stop using precise - https://phabricator.wikimedia.org/T111785#1616402 (10JanZerebecki) 3NEW [14:38:50] 10Continuous-Integration-Infrastructure: use a way to run other PHP versions not provided by the current distribution to stop using precise - https://phabricator.wikimedia.org/T111785#1616409 (10JanZerebecki) [14:39:04] 10Continuous-Integration-Infrastructure: use a way to run other PHP versions not provided by the current distribution to stop using precise - https://phabricator.wikimedia.org/T111785#1616402 (10JanZerebecki) Example: https://github.com/CHH/phpenv [14:41:10] 10Continuous-Integration-Infrastructure, 6Discovery, 6operations, 7Elasticsearch, 5Patch-For-Review: elasticsearch 1.6.0 fails to start after reboot - https://phabricator.wikimedia.org/T109497#1616426 (10JanZerebecki) Yes doing T111781 is the easier short term fix. In the long run we want to abandon pre... [14:45:16] 10Continuous-Integration-Infrastructure: use a way to run other PHP versions not provided by the current distribution to stop using precise - https://phabricator.wikimedia.org/T111785#1616453 (10hashar) Could not find it earlier during the meeting, but that is probably {T103786} [15:00:18] legoktm: the non-deployment is a blocker, aka: we have to have that deployed? [15:01:05] greg-g: it (and the following patch) need to be deployed to both branches simultaneously [15:01:05] (03Abandoned) 10Paladox: MathSearch add dependance on Math extension [integration/config] - 10https://gerrit.wikimedia.org/r/228293 (owner: 10Paladox) [15:01:22] (03CR) 10Hashar: [C: 04-1] "Should put the puppet recipe under /nodepool/scripts so they will be git-deployed on labnodepool and copied in the instance when nodepool " [integration/config] - 10https://gerrit.wikimedia.org/r/234975 (https://phabricator.wikimedia.org/T110735) (owner: 10Hashar) [15:01:47] legoktm: thank you a ton for the follow up on https://github.com/legoktm/tools-ci :-} [15:01:48] (03CR) 10Paladox: "This can be merged." [integration/config] - 10https://gerrit.wikimedia.org/r/228136 (owner: 10Paladox) [15:02:05] :) [15:02:48] legoktm: random irc message not the best for that kind of requirement :) [15:03:04] RECOVERY - Puppet failure on integration-slave-trusty-1012 is OK Less than 1.00% above the threshold [0.0] [15:03:07] legoktm: can you give me the two gerrrit urls? [15:05:12] zeljkof-meeting: is https://plus.google.com/hangouts/_/wikimedia.org/btest-triage?authuser=0 the proper link ? [15:05:36] hashar: yes, sorry, installing hangouts, so I am late :( [15:10:22] 10Beta-Cluster, 10QuickSurveys, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1616583 (10Jdlrobson) This bug is high priority since the quicker we get this deployed the better feedback we'll get. We need to deliver this in 2 weeks so let's set... [15:12:52] legoktm: nvm :) [15:18:29] greg-g: sorry I was doing something IRL. It's not, but I was expecting Krinkle to have deployed it by now... [15:19:57] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1616620 (10zeljkofilipin) Ideas: - update Cucumber and dependencies - make sure to run the same Ruby version on my machine and Jenkins [15:23:14] legoktm: CentralAuth? [15:23:21] Krinkle: yes [15:23:28] I was waiting for you and/or Aaron to be online when I do that [15:23:38] what's the status? [15:23:55] it needs to be deployed or reverted before the train [15:25:03] which is at 11am Pacific, ish [15:52:48] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1616870 (10greg) It all depends on what you mean by "this". :) What'd you have in mind with "generating a url we can share with each deployment"? A jenkins build URL? or somet... [16:06:58] 10Beta-Cluster, 10QuickSurveys, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1616967 (10Jdlrobson) [16:07:15] 6Release-Engineering: Create slides about CI browser tests for Metrics meeting - https://phabricator.wikimedia.org/T110385#1616979 (10dduvall) 5Open>3Resolved [16:07:40] legoktm: OK. Let's do a deploy then. [16:07:47] I'll be back in a few minutes. [16:08:37] 10Beta-Cluster, 10QuickSurveys, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1616983 (10Jdlrobson) a:3rmoen [16:09:28] ok [16:15:35] legoktm: Do you know a simple way of testing them in prod? [16:16:21] 10Continuous-Integration-Config, 3Mobile-App-Sprint-65-Android-Terbium, 3Wikipedia-Android-App: Publish Jenkins built Android APK to well known URL - https://phabricator.wikimedia.org/T99115#1617029 (10Niedzielski) @hashar, we _should_ be using the SDK supplied by our SDK Manager plugin. I forgot that this i... [16:16:57] Krinkle: for API token, using mw.ForeignApi (https://www.mediawiki.org/wiki/Manual:CORS#CentralAuth) should work. And the second one is just trying to login and not getting stuck at loginwiki [16:23:29] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1617076 (10Jdlrobson) I think right now something simple along the lines of this would be a good start: "mediawiki 1.26 was deployed today to the cluster. At the time of depl... [16:30:44] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1617119 (10greg) Ah, I see. We'll put this in our "best effort, but not promise" category for the next couple months; I really don't want to jeopardize our other work right n... [16:30:57] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1617120 (10greg) p:5Triage>3Normal [16:32:37] 10Browser-Tests, 10Deployment-Systems: Display and/or announce build status of wmf branch cut tests (including @integration tests) - https://phabricator.wikimedia.org/T111823#1617125 (10greg) 3NEW [16:32:47] (03PS1) 10Niedzielski: Work around missing Android SDK files w/ emulator [integration/config] - 10https://gerrit.wikimedia.org/r/236827 (https://phabricator.wikimedia.org/T99115) [16:32:52] 10Browser-Tests, 10Deployment-Systems: Display and/or announce build status of wmf branch cut tests (including @integration tests) - https://phabricator.wikimedia.org/T111823#1617135 (10greg) [16:32:53] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1607160 (10greg) [16:58:49] 10Browser-Tests, 10MediaWiki-General-or-Unknown: Run browser tests in core on every commit - https://phabricator.wikimedia.org/T111467#1617222 (10greg) [17:11:09] 10Browser-Tests, 10Continuous-Integration-Config, 10Wikidata: fix `negative argument (ArgumentError) in browsertests - https://phabricator.wikimedia.org/T110510#1617290 (10zeljkofilipin) I have tried running the same Ruby version on my machine and Jenkins. Jenkins runs ruby 1.9.3p484: https://integration.wi... [17:12:30] 10Beta-Cluster, 10QuickSurveys, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1571710 (10Jdlrobson) [17:25:49] 10Continuous-Integration-Config: Migrate bundle17 jobs to bundle - https://phabricator.wikimedia.org/T111764#1617387 (10dduvall) If we get rid of the `bundle17` job, we should pin `bundle` to a specific minor version (i.e. `gem install -v '~> 1.10.5'`). There's really no advantage to using the latest Bundler and... [17:32:36] 10Beta-Cluster, 3Collaboration-Team-Current, 5Patch-For-Review, 5WMF-deploy-2015-09-08_(1.26wmf22), 7WorkType-Maintenance: update.php broken in Beta - https://phabricator.wikimedia.org/T111267#1617420 (10DannyH) 5Open>3Resolved [18:10:31] 10Continuous-Integration-Config: Migrate bundle17 jobs to bundle - https://phabricator.wikimedia.org/T111764#1617513 (10hashar) 5Open>3declined a:3hashar Ah yeah I haven't thought about future breakage. So lets stick with bundle17 until Vagrant start requiring some later version :-) [18:17:35] 10Continuous-Integration-Config, 3Mobile-App-Sprint-65-Android-Terbium, 5Patch-For-Review, 3Wikipedia-Android-App: Publish Jenkins built Android APK to well known URL - https://phabricator.wikimedia.org/T99115#1617541 (10hashar) >>! In T99115#1617029, @Niedzielski wrote: > @hashar, we _should_ be using the... [18:18:13] (03CR) 10Hashar: [C: 032] "Perfect :)" [integration/config] - 10https://gerrit.wikimedia.org/r/236827 (https://phabricator.wikimedia.org/T99115) (owner: 10Niedzielski) [18:18:56] (03CR) 10Hashar: "I have refreshed the apps-android-wikipedia-publish jenkins job." [integration/config] - 10https://gerrit.wikimedia.org/r/236827 (https://phabricator.wikimedia.org/T99115) (owner: 10Niedzielski) [18:28:04] 10Continuous-Integration-Config, 3Mobile-App-Sprint-65-Android-Terbium, 5Patch-For-Review, 3Wikipedia-Android-App: Publish Jenkins built Android APK to well known URL - https://phabricator.wikimedia.org/T99115#1617633 (10Niedzielski) > setup a .jar repository I'm going to keep my fingers crossed for an upd... [18:28:14] bd808: will anything blow up if I restart all the memcached boxes on beta? wrt https://phabricator.wikimedia.org/T109173 [18:31:56] (03Merged) 10jenkins-bot: Work around missing Android SDK files w/ emulator [integration/config] - 10https://gerrit.wikimedia.org/r/236827 (https://phabricator.wikimedia.org/T99115) (owner: 10Niedzielski) [18:50:48] 5Continuous-Integration-Scaling, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: contint-admins can't start/stop nodepool (lack sudo) - https://phabricator.wikimedia.org/T111374#1617728 (10jcrespo) There was no time to go over these access request during the access request meeting. As this requires... [18:54:51] Can I cause some downtime on either deployment-fluorine or deployment-logstash2? [19:00:15] thcipriani|afk, greg-g, hashar ^ ? [19:01:26] andrewbogott: I guess you can :-) [19:01:36] might want to !log and that is all [19:01:41] hashar: prefer which one? [19:01:49] I’m guessing fluorine will cause less breakage? [19:01:56] I have no idea how the log infra works :-/ [19:02:07] I think logs are sent to fluorine and then to logstash2 [19:02:17] oh, hm, maybe I’ll do logstash2 then [19:02:43] hashar: alternative is that I could rebalance some CI instances, but that seems potentially more noticeable [19:05:45] oh it is to move instances between nodes ? [19:06:00] we can depool the ci instances [19:06:14] PROBLEM - Host deployment-cache-mobile03 is DOWN: CRITICAL - Host Unreachable (10.68.16.13) [19:06:41] hashar: ah, ok. I’m migrating a different instance now, when that finishes I will check back about depooling (if you’re still working by then) [19:06:59] tgr: I don't know what a full memcached restart for beta cluster will break. Do you think its really memcached having problems? Seems more likely that it would be nutcracker instances [19:08:36] 10Beta-Cluster, 10QuickSurveys, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1617805 (10phuedx) The QuickSurveys `dev` branch is now deployed to our staging environment and here's the proof: http://reading-web-staging.wmflabs.org/wiki/Special:... [19:11:25] 10Continuous-Integration-Config, 3Mobile-App-Sprint-65-Android-Terbium, 5Patch-For-Review, 3Wikipedia-Android-App: Publish Jenkins built Android APK to well known URL - https://phabricator.wikimedia.org/T99115#1617825 (10Niedzielski) 5Open>3Resolved @hashar, I think we're good to go[0]. Moving this to... [19:17:52] andrewbogott: I am just lurking. But in short one can head to Jenkins https://integration.wikimedia.org/ci/ , login with LDAP account then on the left are all the slaves. Clicking on the slave name shows a page that let one mark a node as offline [19:18:09] ‘k thanks [19:18:17] andrewbogott: it will no more accept new job and as soon as the last job it is running is completed it disconnect from the master [19:18:25] then one can shut it down / maintenance whatever :-} [19:18:35] Can I tell from the web ui when it’s fully drained? [19:20:09] yup [19:20:25] andrewbogott: when on a slave node, the sidebar list the executor slots and jobs running on it [19:28:36] !log removed cherry-pick of https://gerrit.wikimedia.org/r/#/c/197655/ due to rebase conflict [19:28:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [19:34:43] 5Continuous-Integration-Scaling, 10Ops-Access-Requests, 6operations, 5Patch-For-Review: contint-admins can't start/stop nodepool (lack sudo) - https://phabricator.wikimedia.org/T111374#1617950 (10jcrespo) p:5Normal>3High [19:36:49] 10Continuous-Integration-Infrastructure: use a way to run other PHP versions not provided by the current distribution to stop using precise - https://phabricator.wikimedia.org/T111785#1617969 (10JanZerebecki) [19:36:50] 10Continuous-Integration-Infrastructure: Investigate installing php5.3 on a trusty instance - https://phabricator.wikimedia.org/T103786#1617970 (10JanZerebecki) [20:04:56] RECOVERY - Host deployment-cache-mobile03 is UPING OK - Packet loss = 0%, RTA = 1.89 ms [20:20:09] PROBLEM - Host integration-slave-trusty-1011 is DOWN: CRITICAL - Host Unreachable (10.68.17.244) [20:20:32] bd808: nutcracker runs on the appservers, right? [20:20:53] yeah. there would be one on each MW server node [20:20:56] can I restart them via salt somehow? [20:21:15] then again, beta has three appservers... [20:21:25] in theory, yes. Somebody like ori might even know the right command [20:23:11] so if just do service nutcracker restart on all appservers, will *that* break anything? [20:23:51] Why do we bother with the wmf/ branches in Wikimedia* extensions like WikimediaMaintenance, WikimediaMessages, WikimediaEvents, etc.? [20:28:41] marxarelli: ^ fyi re memcache nonce ^ [20:29:28] Krenair: because things get cherry picked into prod for those [20:30:29] yes but why do we bother cherry-picking for them instead of just deploying from master? [20:34:57] Krenair: consistency [20:35:20] If we cared about consistency, every extension would be deployed from a wmf/ branch [20:35:21] do things one way [20:35:32] eventual consistency? [20:35:33] ;) [20:35:37] :) [20:35:51] yet we still have 'wmf_deploy', 'deployment', '1.8.x', '0.5.x' [20:37:00] 10Beta-Cluster, 10MediaWiki-extensions-OAuth, 10Pywikibot-OAuth, 7Pywikibot-network, 7Pywikibot-tests: "Nonce already used" regularly occurring on beta cluster - https://phabricator.wikimedia.org/T109173#1618208 (10Tgr) @jayvdb is there a way to track this error? I looked at the travis history, but build... [20:37:51] Krenair: Other than the FR ones, what other extensions have special handling like that? [20:38:25] CentralNotice (not sure if FR or not?), SemanticMediaWiki, SemanticResultFormats and Validator [20:38:43] oddball, oddball, oddball, not sure [20:38:54] Validator is a dependency for Semantic stuff [20:38:59] ah, then oddball as well [20:39:08] we're consistent, except for the oddball ones [20:39:26] And Wikimedia* stuff aren't odd enough? [20:40:28] I'm mostly joking here, I bet harmonizing things would be welcomed by all [20:44:23] 6Release-Engineering, 6Labs, 10wikitech.wikimedia.org: Missing messages on Wikitech - https://phabricator.wikimedia.org/T101753#1618241 (10greg) [20:46:45] is there a grafana for the beta cluster? [20:47:23] not that I know of [20:48:04] RECOVERY - Host integration-slave-trusty-1011 is UPING OK - Packet loss = 0%, RTA = 1.07 ms [20:54:37] 10Differential, 10Gerrit-Migration: Allow viewing diffs of single commits within a Differential - https://phabricator.wikimedia.org/T125#1618282 (10greg) [20:56:36] 10Differential, 10Gerrit-Migration: Allow viewing diffs of single commits within a Differential - https://phabricator.wikimedia.org/T125#1772 (10greg) Thanks @QChris. Sorry I was dense there. I've sub'd to the upstream task. I'm going to make an executive decision (as much as I can here ;) ) and say this isn'... [20:56:42] 10Differential: Allow viewing diffs of single commits within a Differential - https://phabricator.wikimedia.org/T125#1618290 (10greg) [21:02:07] (03PS1) 10BryanDavis: Add avro-php [integration/config] - 10https://gerrit.wikimedia.org/r/236943 (https://phabricator.wikimedia.org/T111851) [21:03:07] (03CR) 10Legoktm: [C: 032] Add avro-php [integration/config] - 10https://gerrit.wikimedia.org/r/236943 (https://phabricator.wikimedia.org/T111851) (owner: 10BryanDavis) [21:06:31] 6Release-Engineering, 7Epic: [EPIC] Encourage developers to increase code coverage - https://phabricator.wikimedia.org/T100294#1618311 (10greg) [21:08:33] (03Merged) 10jenkins-bot: Add avro-php [integration/config] - 10https://gerrit.wikimedia.org/r/236943 (https://phabricator.wikimedia.org/T111851) (owner: 10BryanDavis) [21:11:07] !log deploying https://gerrit.wikimedia.org/r/236943 [21:11:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:28:19] 6RelEng-Admin, 15User-greg: Setup meeting with $people to discuss code hosting exception policy - https://phabricator.wikimedia.org/T109657#1618383 (10greg) Revising plan.... going to talk with @mark about this in a meeting first to A) get feedback on who from Ops should attend the bigger meeting, if anyone an... [21:49:01] 6RelEng-Admin, 15User-greg: Create #releng-201516-q2 goals - https://phabricator.wikimedia.org/T109115#1618415 (10greg) [22:02:54] PROBLEM - Free space - all mounts on deployment-eventlogging02 is CRITICAL deployment-prep.deployment-eventlogging02.diskspace._srv.byte_percentfree (<10.00%) WARN: deployment-prep.deployment-eventlogging02.diskspace._var.byte_percentfree (<20.00%) [22:17:47] PROBLEM - Free space - all mounts on deployment-kafka02 is CRITICAL deployment-prep.deployment-kafka02.diskspace.root.byte_percentfree (<20.00%) [22:37:47] RECOVERY - Free space - all mounts on deployment-kafka02 is OK All targets OK [23:09:44] 6RelEng-Admin: Propose/enact changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1618632 (10greg) 3NEW a:3Ettorre [23:09:59] 6RelEng-Admin: Propose/enact changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1618643 (10greg) a:5Ettorre>3greg [23:11:22] 6RelEng-Admin: Propose/enact changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1618632 (10greg) For the record: I'm offering to do the heavy lifting of reworking the various projects' workboards to be compliant with this proposal (iow: no one has to do anything to fix the sit... [23:15:26] 6RelEng-Admin, 15User-greg: Propose/enact changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1618662 (10greg) [23:26:54] 6Release-Engineering, 10Gather, 10MobileFrontend, 3Reading-Web: Ensure tests are green before they hit production - https://phabricator.wikimedia.org/T101865#1618687 (10Jdlrobson) [23:27:21] 6Release-Engineering, 10Gather, 10MobileFrontend, 3Reading-Web: Ensure tests are green before they hit production - https://phabricator.wikimedia.org/T101865#1349944 (10Jdlrobson) [23:27:23] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1618689 (10Jdlrobson) [23:28:55] 10Browser-Tests, 10MediaWiki-General-or-Unknown: Run browser tests in core on every commit - https://phabricator.wikimedia.org/T111467#1618696 (10Jdlrobson) [23:28:57] 10Browser-Tests, 10Deployment-Systems: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#1607160 (10Jdlrobson) [23:28:59] 10Browser-Tests, 10Continuous-Integration-Infrastructure, 6Release-Engineering, 7Epic, 7Tracking: [EPIC] trigger browser tests from Gerrit (tracking) - https://phabricator.wikimedia.org/T55697#1618694 (10Jdlrobson) [23:53:25] 6RelEng-Admin, 15User-greg: Propose/enact changes to phab task management for RelEng - https://phabricator.wikimedia.org/T111874#1618793 (10greg) [23:57:28] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<22.22%) [23:59:56] 10Beta-Cluster, 10QuickSurveys, 5Patch-For-Review, 3Reading-Web-Sprint-55-π: Get QuickSurveys enabled on beta cluster - https://phabricator.wikimedia.org/T110199#1618819 (10rmoen) Scheduled for evening swat. https://wikitech.wikimedia.org/wiki/Deployments#Week_of_September_7th