[02:51:51] 10Deployment-Systems: [scap] Log directly to logstash via syslog input - https://phabricator.wikimedia.org/T86969#1284103 (10bd808) [05:34:56] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #418: FAILURE in 32 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/418/ [06:38:22] RECOVERY - Free space - all mounts on deployment-eventlogging02 is OK All targets OK [06:40:14] PROBLEM - Puppet failure on deployment-db1 is CRITICAL 55.56% of data above the critical threshold [0.0] [07:05:15] RECOVERY - Puppet failure on deployment-db1 is OK Less than 1.00% above the threshold [0.0] [08:27:00] 10Deployment-Systems, 6operations: [Trebuchet] Salt times out on parsoid restarts - https://phabricator.wikimedia.org/T63882#1284382 (10ArielGlenn) [08:56:42] PROBLEM - Content Translation Server on deployment-sca02 is CRITICAL: Connection refused [09:03:17] PROBLEM - Puppet failure on deployment-sca02 is CRITICAL 100.00% of data above the critical threshold [0.0] [09:47:46] 10Deployment-Systems, 6operations: [Trebuchet] Salt times out on parsoid restarts - https://phabricator.wikimedia.org/T63882#1284459 (10ArielGlenn) Now that salt is upgraded, we need to fix up /usr/local/bin/service-restart so that it takes a timeout and the format of the args to publish.runner is updated. [10:04:25] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #431: STILL FAILING in 43 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/431/ [10:18:14] 10Deployment-Systems, 6operations: [Trebuchet] Salt times out on parsoid restarts - https://phabricator.wikimedia.org/T63882#1284563 (10ArielGlenn) Something like this: https://gerrit.wikimedia.org/r/#/c/210877/ (UNTESTED) [10:45:48] PROBLEM - Puppet staleness on deployment-restbase02 is CRITICAL 100.00% of data above the critical threshold [43200.0] [13:54:23] PROBLEM - Free space - all mounts on deployment-eventlogging02 is CRITICAL deployment-prep.deployment-eventlogging02.diskspace._var.byte_percentfree (<30.00%) [14:08:32] Yippee, build fixed! [14:08:32] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #618: FIXED in 37 min: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/618/ [15:01:48] marktraceur: "14:52 < marktrace> I gotta deal with Jenkins, sorry" ??? [15:04:48] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<20.00%) [15:48:18] PROBLEM - Content Translation Server on deployment-cxserver03 is CRITICAL: Connection refused [15:49:27] 10Continuous-Integration-Infrastructure, 6Scrum-of-Scrums, 6operations, 7Blocked-on-Operations: Jenkins is using php-luasandbox 1.9-1 for zend unit tests; precise should be upgraded to 2.0-8 or equivalent - https://phabricator.wikimedia.org/T88798#1285023 (10akosiaris) https://gerrit.wikimedia.org/r/#/c/21... [15:58:19] RECOVERY - Content Translation Server on deployment-cxserver03 is OK: HTTP OK: HTTP/1.1 200 OK - 1103 bytes in 0.021 second response time [16:29:07] PROBLEM - Puppet failure on deployment-zotero01 is CRITICAL 100.00% of data above the critical threshold [0.0] [16:35:35] 10Deployment-Systems, 6Release-Engineering: Use subrepos instead of git submodules for deployed MediaWiki extensions - https://phabricator.wikimedia.org/T98834#1285157 (10Legoktm) >>! In T98834#1279062, @mmodell wrote: > One thing I'm a little unsure of: sub-sub-modules - currently 3 of the deployed submodul... [16:37:22] 10Deployment-Systems, 6Release-Engineering: Use subrepos instead of git submodules for deployed MediaWiki extensions - https://phabricator.wikimedia.org/T98834#1285161 (10mmodell) Oh, interesting. [16:48:43] 10Deployment-Systems, 6Release-Engineering: Use subrepos instead of git submodules for deployed MediaWiki extensions - https://phabricator.wikimedia.org/T98834#1285207 (10mobrovac) Just making sure. Are you referring to [git subrepo](https://github.com/ingydotnet/git-subrepo/) in this task? If so, as per its... [17:10:47] 10Deployment-Systems, 10Traffic, 6operations: Fix static asset varnish cache invalidation issues - https://phabricator.wikimedia.org/T99094#1285268 (10BBlack) 3NEW [17:11:05] 10Deployment-Systems, 10Traffic, 6operations: Fix static asset varnish cache invalidation issues - https://phabricator.wikimedia.org/T99094#1285275 (10BBlack) [17:22:16] 10Deployment-Systems, 10Traffic, 6operations: Fix static asset varnish cache invalidation issues - https://phabricator.wikimedia.org/T99094#1285306 (10BBlack) [17:26:21] 10Deployment-Systems, 6operations, 7Varnish: Varnish cache busting desired for /static/$VERSION/ resources which change within the lifetime of a WMF release branch - https://phabricator.wikimedia.org/T99096#1285315 (10bd808) 3NEW [17:29:40] 10Deployment-Systems, 10Traffic, 6operations: Varnish cache busting desired for /static/$VERSION/ resources which change within the lifetime of a WMF release branch - https://phabricator.wikimedia.org/T99096#1285322 (10BBlack) [17:29:47] 10Deployment-Systems, 10Traffic, 6operations: Fix static asset varnish cache invalidation issues - https://phabricator.wikimedia.org/T99094#1285324 (10bd808) [17:29:50] 10Deployment-Systems, 10Traffic, 6operations: Varnish cache busting desired for /static/$VERSION/ resources which change within the lifetime of a WMF release branch - https://phabricator.wikimedia.org/T99096#1285325 (10bd808) [17:30:10] 10Deployment-Systems, 10Traffic, 6operations: Fix static asset varnish cache invalidation issues - https://phabricator.wikimedia.org/T99094#1285327 (10BBlack) 5duplicate>3Invalid a:3BBlack Duplicate of T99096 [17:30:35] 10Deployment-Systems, 10Traffic, 6operations: Varnish cache busting desired for /static/$VERSION/ resources which change within the lifetime of a WMF release branch - https://phabricator.wikimedia.org/T99096#1285337 (10bd808) From @bblack's description at T99094: > Currently, when static assets are deployed... [17:44:34] (03PS2) 10Legoktm: Change procedure for finding global variables in ValidGlobalNameSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/196880 (owner: 10Polybuildr) [17:45:03] (03CR) 10jenkins-bot: [V: 04-1] Change procedure for finding global variables in ValidGlobalNameSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/196880 (owner: 10Polybuildr) [17:45:40] 10Deployment-Systems, 6Release-Engineering: Use subrepos instead of git submodules for deployed MediaWiki extensions - https://phabricator.wikimedia.org/T98834#1285419 (10mmodell) @mobrovac: yes that's the git-subrepo I am testing out currently. I didn't see anything about nested subrepos, where did you find... [17:50:35] 10Deployment-Systems, 6Release-Engineering: Use subrepos instead of git submodules for deployed MediaWiki extensions - https://phabricator.wikimedia.org/T98834#1285442 (10mobrovac) >>! In T98834#1285419, @mmodell wrote: > @mobrovac: yes that's the git-subrepo I am testing out currently. I didn't see anything... [18:04:00] 10Beta-Cluster, 6operations, 5Patch-For-Review, 7Puppet: Trebuchet on deployment-bastion: wrong group owner - https://phabricator.wikimedia.org/T97775#1285452 (10thcipriani) @ArielGlenn I just updated my patch to allow a per-repo override of `deployment_repo_group` in the `repo_config` pillar which overri... [18:21:31] PROBLEM - Puppet failure on integration-zuul-packaged is CRITICAL 100.00% of data above the critical threshold [0.0] [19:28:02] (03PS1) 10Hashar: zuul: point Android app lint result to HTML report [integration/config] - 10https://gerrit.wikimedia.org/r/210944 (https://phabricator.wikimedia.org/T99112) [19:30:12] (03CR) 10Hashar: [C: 032] "instant deploy so we can see what happens :]" [integration/config] - 10https://gerrit.wikimedia.org/r/210944 (https://phabricator.wikimedia.org/T99112) (owner: 10Hashar) [19:31:51] (03Merged) 10jenkins-bot: zuul: point Android app lint result to HTML report [integration/config] - 10https://gerrit.wikimedia.org/r/210944 (https://phabricator.wikimedia.org/T99112) (owner: 10Hashar) [19:45:38] 10Beta-Cluster: Upgrade varnish automatically via puppet in Beta Cluster - https://phabricator.wikimedia.org/T75564#1285780 (10mmodell) a:5mmodell>3None not my cookie? [19:50:29] 6Release-Engineering, 6Phabricator: Next Phabricator upgrade on 2015-05-20 (tentative) - https://phabricator.wikimedia.org/T98451#1285802 (10mmodell) [20:36:12] 6Release-Engineering, 6Phabricator: Next Phabricator upgrade on 2015-05-20 (tentative) - https://phabricator.wikimedia.org/T98451#1285885 (10mmodell) >>! In T98451#1273789, @Qgil wrote: > The next upgrade should bring public Conpherence rooms, now prevented only by a bug that I reported directly upstream: http... [21:17:25] PROBLEM - Puppet staleness on deployment-eventlogging02 is CRITICAL 100.00% of data above the critical threshold [43200.0] [21:19:09] twentyafterfour: how hard/disruptive do you think implementing https://phabricator.wikimedia.org/T67306 would be? using semver for WMF branches so they would look like 1.26.0-wmf6? [21:20:23] legoktm: so essentially just adding a dash in the version? [21:20:35] twentyafterfour: and the .0 yeah [21:20:59] should not be too difficult, I don't know what hidden assumptions might be lurking in the multiversion code though [21:21:50] I know there is some code that currently validates the format of the version string, so that would have to be adjusted [21:22:44] 10Deployment-Systems: Adopt Semantic Versioning for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#1286101 (10Legoktm) 5Invalid>3Open Re-opening, they're not duplicates. [21:27:46] 10Deployment-Systems: Adopt Semantic Versioning for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#1286118 (10Legoktm) ``` [14:19:09] twentyafterfour: how hard/disruptive do you think implementing https://phabricator.wikimedia.org/T67306 would be? using semver for WMF... [21:37:41] 10Deployment-Systems: Adopt Semantic Versioning for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#1286157 (10mmodell) I have no problem with following the format of semantic versioning, however, our weekly branch cuts don't necessarily follow the rules of semantic versioning ch... [21:39:23] 10Deployment-Systems: Adopt Semantic Versioning format for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#1286169 (10Legoktm) [21:39:54] 10Deployment-Systems: Adopt Semantic Versioning format for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#683803 (10Legoktm) >>! In T67306#1286157, @mmodell wrote: > I have no problem with following the format of semantic versioning, however, our weekly branch cuts don't necessa... [21:40:20] 10Deployment-Systems: Adopt Semantic Versioning format for WMF deploy branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T67306#1286172 (10Legoktm) a:5greg>3None [22:33:59] 10Beta-Cluster: Upgrade varnish automatically via puppet in Beta Cluster - https://phabricator.wikimedia.org/T75564#1286324 (10BBlack) Well it sounds like you fixed this in hiera? In any case, the current problem with varnish versions in beta is this: T98758 [23:09:10] 10Beta-Cluster, 6Release-Engineering: Beta cluster "test.wikipedia" thinks it is "test.wikimedia" - https://phabricator.wikimedia.org/T99156#1286436 (10Krinkle) 3NEW [23:30:22] 6Release-Engineering, 6Engineering-Community, 3ECT-May-2015: Lyon -> Annecy Transportation Info to RelEng Team - https://phabricator.wikimedia.org/T93686#1286624 (10Rfarrand) 5Open>3Resolved