[00:00:36] Something is up with qunit builds. They are failing on the last two mediawiki/core patches in gerrit both of which don’t touch JS. See https://integration.wikimedia.org/ci/job/mediawiki-core-qunit/56635/console [00:35:57] 10Continuous-Integration-Infrastructure: Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917229 (10Unicornisaurous) 3NEW [00:42:03] 10Continuous-Integration-Infrastructure: Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917238 (10Paladox) socket.io was updated to 1.4.0 from 1.3.7. https://github.com/karma-runner/karma/issues/1782#issuecomment-169178567 Users have said even downgrading hasent worke... [00:45:04] 10Continuous-Integration-Infrastructure: Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917240 (10Unicornisaurous) [00:54:33] 10Continuous-Integration-Infrastructure, 5Patch-For-Review: Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917258 (10Paladox) Users are reporting now that 0.13.9 is working. Also the author of it has responded and has said a patch is incoming. [00:55:15] 10Continuous-Integration-Infrastructure, 5Patch-For-Review: Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917259 (10Paladox) 0.13.19 has been released now with the socket.io fix. [01:32:09] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:41:56] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39345 bytes in 1.205 second response time [02:59:03] PROBLEM - App Server Main HTTP Response on deployment-mediawiki01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:03:54] RECOVERY - App Server Main HTTP Response on deployment-mediawiki01 is OK: HTTP OK: HTTP/1.1 200 OK - 39350 bytes in 0.811 second response time [03:29:38] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #934: 04FAILURE in 47 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/934/ [03:58:23] 6Release-Engineering-Team, 3releng-201516-q3, 10Wikimedia-Developer-Summit-2016, 15User-greg: Code-review migration to Differential status/discussion - https://phabricator.wikimedia.org/T114320#1917346 (10mmodell) >>! In T114320#1904418, @JanZerebecki wrote: >> In fact, it's actually fairly difficult to lo... [04:33:04] 10Continuous-Integration-Infrastructure, 5MW-1.27-release-notes, 5Patch-For-Review, 5WMF-deploy-2016-01-12_(1.27.0-wmf.10): Qunit jobs consistently failing on all slaves - https://phabricator.wikimedia.org/T122941#1917374 (10Krinkle) 5Open>3Resolved a:3Krinkle Fixed by 522e673e7e6a29232843250279f3a6c... [06:23:08] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:11:37] 10MediaWiki-Releasing, 6Release-Engineering-Team, 5Patch-For-Review: Resolve/remove symlink of README.mediawiki in tarball - https://phabricator.wikimedia.org/T122502#1917541 (10demon) We could just copy the file at release time instead. [08:33:24] Yippee, build fixed! [08:33:24] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #836: 09FIXED in 23 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/836/ [09:21:38] 10Continuous-Integration-Infrastructure, 6Labs, 7Shinken, 7WorkType-Maintenance: Labs Shinken complains about no more existing host integration-t102459 is DOWN - https://phabricator.wikimedia.org/T121767#1917549 (10hashar) [09:24:52] 10Beta-Cluster-Infrastructure, 6Labs, 6operations: deployment-mediawiki03 : apt broken trying to reach out webproxy.eqiad.wmnet - https://phabricator.wikimedia.org/T122953#1917550 (10hashar) 3NEW [09:25:31] 10Beta-Cluster-Infrastructure, 6Labs, 6operations: deployment-mediawiki03 : apt broken trying to reach out webproxy.eqiad.wmnet - https://phabricator.wikimedia.org/T122953#1917557 (10hashar) [09:28:45] 10Beta-Cluster-Infrastructure, 6Labs, 6operations: deployment-mediawiki03 : apt broken trying to reach out webproxy.eqiad.wmnet - https://phabricator.wikimedia.org/T122953#1917559 (10hashar) Command to mass check: `root@deployment-salt:~ # salt -v '*' file.find /etc/apt grep=webproxy` [09:35:33] !log beta/CI: salt -v '*' cmd.run 'rm -v /etc/apt/apt.conf.d/*-proxy' https://phabricator.wikimedia.org/T122953 [09:35:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:35:40] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure, 6Labs, 6operations: deployment-mediawiki03 : apt broken trying to reach out webproxy.eqiad.wmnet - https://phabricator.wikimedia.org/T122953#1917560 (10hashar) The`apt` puppet class has a `$use_proxy` parameter that would create the fi... [09:36:01] 10Beta-Cluster-Infrastructure, 10Continuous-Integration-Infrastructure, 6Labs, 6operations: deployment-mediawiki03 : apt broken trying to reach out webproxy.eqiad.wmnet - https://phabricator.wikimedia.org/T122953#1917562 (10hashar) 5Open>3Resolved a:3hashar [09:39:21] !log restoring puppet hacks on beta cluster puppetmaster. [09:39:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [09:44:01] !log CI/beta: deleting all git tags from /var/lib/git/operations/puppet and doing git repack [09:44:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [10:53:40] 10Continuous-Integration-Infrastructure, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#1917601 (10hashar) [11:07:46] 10Continuous-Integration-Infrastructure, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#1917630 (10hashar) [11:08:19] 10Continuous-Integration-Infrastructure, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#658851 (10hashar) [11:15:20] (03PS1) 10Hashar: (WIP) Make bunch of jslint jobs voting (WIP) [integration/config] - 10https://gerrit.wikimedia.org/r/262700 [12:13:00] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39359 bytes in 0.851 second response time [12:19:06] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:19:18] 10Continuous-Integration-Config, 10pywikibot-core, 5Patch-For-Review: jenkins output is unreadable - https://phabricator.wikimedia.org/T117570#1917657 (10jayvdb) To help alleviate the problem, I've created a flake8 plugin to allow [[https://en.wikipedia.org/wiki/ANSI_escape_code | ANSI escape sequences]] usi... [12:58:52] Yippee, build fixed! [12:58:53] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #901: 09FIXED in 26 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/901/ [13:07:05] 10Continuous-Integration-Infrastructure, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#1917705 (10hashar) [13:10:55] (03PS2) 10Hashar: Make bunch of mwext jslint jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/262700 [13:11:36] (03PS3) 10Hashar: Make bunch of mwext jslint jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/262700 (https://phabricator.wikimedia.org/T62619) [13:13:03] (03CR) 10Hashar: [C: 032] Make bunch of mwext jslint jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/262700 (https://phabricator.wikimedia.org/T62619) (owner: 10Hashar) [13:14:16] (03Merged) 10jenkins-bot: Make bunch of mwext jslint jobs voting [integration/config] - 10https://gerrit.wikimedia.org/r/262700 (https://phabricator.wikimedia.org/T62619) (owner: 10Hashar) [13:16:39] 10Continuous-Integration-Infrastructure, 5Patch-For-Review, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#1917717 (10hashar) [13:18:36] 10Continuous-Integration-Infrastructure, 5Patch-For-Review, 7Technical-Debt, 7Tracking: All repositories should pass jshint test (tracking) - https://phabricator.wikimedia.org/T62619#658851 (10hashar) [14:11:15] (03PS1) 10Hashar: [ContributionTracking] drop jslint job (uses npm) [integration/config] - 10https://gerrit.wikimedia.org/r/262705 (https://phabricator.wikimedia.org/T63599) [14:35:58] Yippee, build fixed! [14:35:59] Project browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce build #376: 09FIXED in 7 min 57 sec: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-SmokeTests-linux-chrome-sauce/376/ [14:40:06] (03PS1) 10Hashar: tests: TestZuulLayout -> TestZuulSchedLayout [integration/config] - 10https://gerrit.wikimedia.org/r/262706 [14:46:21] (03PS2) 10Hashar: tests: TestZuulLayout -> TestZuulScheduler [integration/config] - 10https://gerrit.wikimedia.org/r/262706 [14:46:44] (03CR) 10Hashar: "Now named TestZuulScheduler" [integration/config] - 10https://gerrit.wikimedia.org/r/262706 (owner: 10Hashar) [15:00:19] (03PS1) 10Hashar: tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 [15:01:29] (03CR) 10jenkins-bot: [V: 04-1] tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 (owner: 10Hashar) [15:06:51] (03CR) 10Paladox: [C: 031] "I am +1 to say this patch looks all good. Just waiting for the two source changes to be merged." [integration/config] - 10https://gerrit.wikimedia.org/r/262705 (https://phabricator.wikimedia.org/T63599) (owner: 10Hashar) [15:14:04] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39369 bytes in 6.214 second response time [15:20:11] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:31:48] (03PS2) 10Hashar: tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 [15:32:05] (03CR) 10Hashar: "Related source changes have Change-Id Ibae36050a1d8054b7280804513a258bc7cd8960f" [integration/config] - 10https://gerrit.wikimedia.org/r/262708 (owner: 10Hashar) [15:32:57] (03CR) 10jenkins-bot: [V: 04-1] tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 (owner: 10Hashar) [15:37:45] (03PS5) 10Hashar: Setup RelatedArticles browser tests job [integration/config] - 10https://gerrit.wikimedia.org/r/260764 (https://phabricator.wikimedia.org/T120715) (owner: 10Bmansurov) [15:42:19] (03CR) 10Hashar: [C: 032] "Lets unleash the beast." [integration/config] - 10https://gerrit.wikimedia.org/r/260764 (https://phabricator.wikimedia.org/T120715) (owner: 10Bmansurov) [15:43:04] Project browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #1: 04FAILURE in 3.7 sec: https://integration.wikimedia.org/ci/job/browsertests-RelatedArticles-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/1/ [15:45:37] (03Merged) 10jenkins-bot: Setup RelatedArticles browser tests job [integration/config] - 10https://gerrit.wikimedia.org/r/260764 (https://phabricator.wikimedia.org/T120715) (owner: 10Bmansurov) [16:01:37] 10Continuous-Integration-Config, 10MediaWiki-extensions-RelatedArticles, 5Patch-For-Review, 3Reading-Web-Sprint-63-Ellip…: RelatedArticles browser tests should run on a commit basis - https://phabricator.wikimedia.org/T120715#1917912 (10hashar) [16:17:25] (03PS3) 10Hashar: tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 [16:17:40] (03Abandoned) 10Hashar: [ContributionTracking] drop jslint job (uses npm) [integration/config] - 10https://gerrit.wikimedia.org/r/262705 (https://phabricator.wikimedia.org/T63599) (owner: 10Hashar) [16:21:27] (03CR) 10jenkins-bot: [V: 04-1] tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 (owner: 10Hashar) [16:30:14] (03PS4) 10Hashar: tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 [16:30:43] (03PS3) 10Hashar: tests: TestZuulLayout -> TestZuulScheduler [integration/config] - 10https://gerrit.wikimedia.org/r/262706 [16:32:39] (03PS5) 10Hashar: tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 [16:34:51] (03CR) 10Hashar: [C: 032] tests: TestZuulLayout -> TestZuulScheduler [integration/config] - 10https://gerrit.wikimedia.org/r/262706 (owner: 10Hashar) [16:35:08] (03CR) 10Hashar: [C: 032] tests: mw exts having npm do not need extension-jslint [integration/config] - 10https://gerrit.wikimedia.org/r/262708 (owner: 10Hashar) [16:35:43] (03Merged) 10jenkins-bot: tests: TestZuulLayout -> TestZuulScheduler [integration/config] - 10https://gerrit.wikimedia.org/r/262706 (owner: 10Hashar) [17:09:57] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39352 bytes in 0.469 second response time [17:46:05] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:50:59] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39359 bytes in 0.815 second response time [17:54:01] 10Deployment-Systems, 3Scap3: scap environment-specific host file not working - https://phabricator.wikimedia.org/T121705#1918175 (10thcipriani) 5Open>3Resolved [18:07:09] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:21:36] 10Continuous-Integration-Config, 10MediaWiki-extensions-RelatedArticles, 5Patch-For-Review, 3Reading-Web-Sprint-63-Ellip…: RelatedArticles browser tests should run on a commit basis - https://phabricator.wikimedia.org/T120715#1918228 (10bmansurov) Thanks, @hashar. What can we do to get https://gerrit.wikim... [18:21:58] RECOVERY - App Server Main HTTP Response on deployment-mediawiki03 is OK: HTTP OK: HTTP/1.1 200 OK - 39359 bytes in 0.845 second response time [18:25:04] 10MediaWiki-Releasing, 6Release-Engineering-Team: Unclear #MediaWiki-Tarball-Backports project usage - https://phabricator.wikimedia.org/T122501#1918255 (10greg) I vote for closing, and we just use the respective #MW-1.XX-release projects, eg #MW-1.27-release. I've updated the relevant backporting mw.org page... [18:27:36] 10MediaWiki-Releasing, 6Release-Engineering-Team, 15User-greg: Unclear #MediaWiki-Tarball-Backports project usage - https://phabricator.wikimedia.org/T122501#1918268 (10greg) 5Open>3Resolved a:3greg {{done}} Updated description with what to do instead. Thanks for finding more places to clean up! :) [18:31:13] PROBLEM - App Server Main HTTP Response on deployment-mediawiki02 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:32:12] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #555: 04FAILURE in 2 min 11 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/555/ [18:36:12] 10MediaWiki-Releasing, 6Release-Engineering-Team, 7Security-Extensions: Backport security fixes to stable + LTS extension branches - https://phabricator.wikimedia.org/T108734#1918325 (10greg) Are you sure that there is "no effort" to backport security fixes to release branches of extensions? https://phabric... [18:41:06] RECOVERY - App Server Main HTTP Response on deployment-mediawiki02 is OK: HTTP OK: HTTP/1.1 200 OK - 39325 bytes in 0.964 second response time [18:45:10] 10MediaWiki-Releasing, 6Release-Engineering-Team, 7Security-Extensions: Backport security fixes to stable + LTS extension branches - https://phabricator.wikimedia.org/T108734#1918344 (10greg) [19:14:04] 10Continuous-Integration-Config: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918429 (10Legoktm) 3NEW [19:14:13] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918436 (10greg) 3NEW a:3mmodell [19:18:34] 10Beta-Cluster-Infrastructure, 10Sentry, 10Wikimedia-Logstash: Channel PHP errors from Logstash to Sentry on the beta cluster - https://phabricator.wikimedia.org/T85239#1918487 (10Tgr) [[ https://github.com/antho31/logstash-output-sentry | antho31/logstash-output-sentry ]] is the only logstash-to-sentry plug... [19:18:51] 6Release-Engineering-Team, 3releng-201516-q3, 10Wikimedia-Developer-Summit-2016, 15User-greg: Code-review migration to Differential status/discussion - https://phabricator.wikimedia.org/T114320#1918489 (10greg) Linking relevant tasks for the ACTION items from the meeting: >>! In T114320#1914904, @thcipria... [19:20:25] 6Release-Engineering-Team, 3releng-201516-q3, 10Wikimedia-Developer-Summit-2016, 15User-greg: Code-review migration to Differential status/discussion - https://phabricator.wikimedia.org/T114320#1918496 (10greg) 5Open>3Resolved Session was had, it went really well, follow up actions linked above (and ar... [19:25:02] 10Continuous-Integration-Config: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918518 (10MoritzMuehlenhoff) Simply running "dpkg-buildpackage" will probably be sufficient (to find out whether any of the changes to the debian directory broke the package build) [19:29:43] (03CR) 10Krinkle: [C: 04-1] Fix dirty VisualEditor submodule (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/262432 (https://phabricator.wikimedia.org/T121479) (owner: 10Paladox) [19:34:25] Project beta-scap-eqiad build #85351: 04FAILURE in 1 min 36 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/85351/ [19:43:28] Yippee, build fixed! [19:43:29] Project beta-scap-eqiad build #85352: 09FIXED in 6 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/85352/ [19:51:55] 10Continuous-Integration-Config: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918608 (10hashar) We have some Jenkins jobs using http://jenkins-debian-glue.org/ That is a set of shell scripts that wraps around git-buildpackage/cowbuilder and make it trivial... [19:53:41] 10Continuous-Integration-Config: Make debian-glue to use Gerrit as the upstream repository - https://phabricator.wikimedia.org/T117869#1918613 (10hashar) [19:53:43] 10Continuous-Integration-Config: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918612 (10hashar) [19:54:57] 10Continuous-Integration-Config, 10MediaWiki-Debian: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918615 (10Legoktm) [19:55:02] PROBLEM - App Server Main HTTP Response on deployment-mediawiki01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:55:48] 10Deployment-Systems, 10MediaWiki-Internationalization, 6Performance-Team, 7Performance: Experiment with plain .php files for l10n cache instead of CDB - https://phabricator.wikimedia.org/T99740#1918619 (10Krinkle) [19:56:46] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:58:25] (03PS1) 10Hashar: [mediawiki/debian] experimental debian-glue job [integration/config] - 10https://gerrit.wikimedia.org/r/262744 (https://phabricator.wikimedia.org/T122978) [19:58:37] 10Deployment-Systems, 10MediaWiki-Internationalization, 6Performance-Team, 7Performance: Experiment with plain .php files for l10n cache instead of CDB - https://phabricator.wikimedia.org/T99740#1918640 (10Krinkle) a:5Krinkle>3None [19:58:49] (03CR) 10Hashar: [C: 032] [mediawiki/debian] experimental debian-glue job [integration/config] - 10https://gerrit.wikimedia.org/r/262744 (https://phabricator.wikimedia.org/T122978) (owner: 10Hashar) [19:59:54] (03Merged) 10jenkins-bot: [mediawiki/debian] experimental debian-glue job [integration/config] - 10https://gerrit.wikimedia.org/r/262744 (https://phabricator.wikimedia.org/T122978) (owner: 10Hashar) [20:01:05] thcipriani: twentyafterfour ostriches just a textual reminder, can you look at mediawiki03? chris / darian aren't able to use the security testing header [20:01:38] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 31492 bytes in 0.474 second response time [20:01:52] greg-g: yerp. Beta acting generally funky. [20:02:15] csteipp: ^ [20:04:37] 10Continuous-Integration-Config, 10MediaWiki-Debian, 5Patch-For-Review: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918663 (10hashar) I have send a test change https://gerrit.wikimedia.org/r/#/c/262746/ and ran the `debian-glue` job there. Obviously f... [20:04:53] RECOVERY - App Server Main HTTP Response on deployment-mediawiki01 is OK: HTTP OK: HTTP/1.1 200 OK - 39366 bytes in 0.594 second response time [20:05:33] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918665 (10Paladox) Would this resolve the open patches problem in phabricator where we carnt view open patches but can merged patches. [20:08:24] 10Continuous-Integration-Config, 10MediaWiki-extensions-RelatedArticles, 5Patch-For-Review, 3Reading-Web-Sprint-63-Ellip…: RelatedArticles browser tests should run on a commit basis - https://phabricator.wikimedia.org/T120715#1918679 (10hashar) Oops eggs and chicken really. Since the browser tests are fail... [20:14:16] (03PS3) 10Paladox: Fix dirty VisualEditor submodule [integration/config] - 10https://gerrit.wikimedia.org/r/262432 (https://phabricator.wikimedia.org/T121479) [20:17:37] !log beta: dropped a few more /etc/apt/apt.conf.d/*-proxy files. webproxy is no more reachable from labs [20:17:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [20:28:13] (03CR) 10Polybuildr: "Ping." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:31:16] (03PS4) 10Legoktm: Add sniff to detect consecutive empty lines in a file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:31:25] (03CR) 10Legoktm: [C: 032] Add sniff to detect consecutive empty lines in a file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:33:38] (03CR) 10Polybuildr: "Thanks!" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:34:28] (03Merged) 10jenkins-bot: Add sniff to detect consecutive empty lines in a file [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/261572 (https://phabricator.wikimedia.org/T120570) (owner: 10Polybuildr) [20:39:12] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918714 (10greg) >>! In T122979#1918665, @Paladox wrote: > Would this resolve the open patches problem in phabricator where we carnt view open patches but can merged patches. Since no new changes woul... [20:40:35] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918715 (10Paladox) Ok thanks for replying. But do you know when the migration will start. And would we have to use the arc command or can we use the normal setup with minimal changes such as only a c... [20:48:36] 10MediaWiki-Codesniffer, 5Patch-For-Review: Add sniff to detect double empty lines in php code - https://phabricator.wikimedia.org/T120570#1918737 (10polybuildr) 5Open>3Resolved I've also opened an issue upstream: https://github.com/squizlabs/PHP_CodeSniffer/issues/851 [20:48:53] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918739 (10mmodell) @paladox: You will have to use arc. [20:49:47] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918743 (10Paladox) Oh. Would there be a guide on mediawiki on how to do it as simple as possible like there is one for gerrit and for git. [20:51:01] lol simple and gerrit don't belong in the same sentence :P [20:51:29] PROBLEM - Host wmfbranch is DOWN: CRITICAL - Host Unreachable (10.68.20.214) [20:52:56] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918748 (10mmodell) >>! In T122979#1918743, @Paladox wrote: > Oh. Would there be a guide on mediawiki on how to do it as simple as possible like there is one for gerrit and for git. T117058 [20:53:41] 5Gerrit-Migration: Migrate open changes from Gerrit to Differential - https://phabricator.wikimedia.org/T122979#1918751 (10Paladox) Thanks. [20:53:51] + [20:56:25] 10Continuous-Integration-Infrastructure, 5Patch-For-Review, 7Zuul: Bump python-gear package to 0.5.7 - https://phabricator.wikimedia.org/T98294#1918753 (10hashar) 5Open>3declined a:3hashar No more need a debian package. The gear python package is now embedded in the Zuul package since https://gerrit.wi... [20:57:28] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-Translate: mediawiki-extensions-hhvm: MessageGroupStatesUpdaterJobTest::testHooks is intermittent failing - https://phabricator.wikimedia.org/T88554#1918756 (10hashar) 5Open>3Resolved a:3hashar [20:58:27] 10Continuous-Integration-Infrastructure: Remove PhantomJS from the CI infrastructure - https://phabricator.wikimedia.org/T113279#1918760 (10hashar) p:5Triage>3Normal [21:00:06] any idea what to do when zuul its view of a repo is out of date because of force-merges? i.e. how to bring it up to date again? [21:00:11] hashar: ^^ [21:00:57] jzerebecki: what do you mean by out of date ? [21:02:29] 10Continuous-Integration-Config, 10Math, 10MathSearch: Make Restbase available to Jenkins - https://phabricator.wikimedia.org/T120657#1918764 (10hashar) a:5hashar>3None [21:02:56] 10Continuous-Integration-Config: Set up Jenkins jobs for phabricator/* repos - https://phabricator.wikimedia.org/T70263#1918767 (10hashar) 5Open>3declined a:3hashar Not much interest for it. [21:03:56] 10Continuous-Integration-Config, 10MediaWiki-Debian: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918780 (10hashar) p:5Triage>3Normal [21:04:38] !log manually deleted /srv/ssd/zuul/git/mediawiki/services/mathoid to force zuul to re-clone it [21:04:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:04:43] !log ^ on gallium [21:04:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:04:52] 10Continuous-Integration-Infrastructure: Adjust CI for PHP5.5 support - https://phabricator.wikimedia.org/T119675#1918782 (10hashar) [21:05:11] legoktm: jzerebecki will also have a merger instance on scandium.eqiad.wmnet [21:05:55] 6Release-Engineering-Team, 10Flow, 3Collaboration-Team-Current, 5MW-1.25-release: Flow (or tests) broken in REL1_25 - https://phabricator.wikimedia.org/T122025#1918784 (10hashar) [21:06:11] 2016-01-06 21:05:16,989 INFO zuul.MergeClient: Merge complete, merged: False, updated: False, commit: None [21:06:15] 2016-01-06 21:05:16,990 INFO zuul.IndependentPipelineManager: Unable to merge change [21:06:18] :8 [21:06:21] :-D [21:06:51] GitCommandError: 'git clone -v ssh://jenkins-bot@ytterbium.wikimedia.org:29418/mediawiki/services/mathoid /srv/ssd/zuul/git/mediawiki/services/ma [21:06:53] thoid' returned with exit code 128 [21:06:53] stderr: 'fatal: destination path '/srv/ssd/zuul/git/mediawiki/services/mathoid' already exists and is not an empty directory. [21:06:55] ' [21:06:55] (on scandium [21:07:00] oh wut [21:07:06] need to delete it there too then? [21:07:17] ahh [21:07:27] that is because mediawiki/services/mathoid/deploy got cloned first [21:07:46] zuul merger is rather dumb there [21:08:01] so as zuul user [21:08:03] we can [21:08:14] manually init the repo [21:08:21] something like [21:08:26] cd /srv/ssd/zuul/git/mediawiki/services/mathoid [21:08:28] git init . [21:08:33] git remote add origin ssh://jenkins-bot@ytterbium.wikimedia.org:29418/mediawiki/services/mathoid [21:10:28] or delete deploy [21:10:44] yup [21:10:47] that is the easiest [21:11:00] then recheck till zuul-merger on scandium create it [21:11:11] :-/ [21:11:18] * legoktm isn't touching anything right now [21:11:35] !log on scandium: sudo -u zuul rm -rf /srv/ssd/zuul/git/mediawiki/services/mathoid [21:11:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:13:29] !log kicking integration puppetmaster, weird node unable to find definition. [21:13:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [21:14:27] hashar, legoktm: thx that seems to have fixed it [21:19:44] well done ! ;-} [21:21:18] rushing to bed [21:22:15] o/ [21:25:14] legoktm: Any chance of a CI consult? :-) VE core's npm job is failing every run (e.g https://integration.wikimedia.org/ci/job/npm/45295/console )… [21:25:27] did you update karma? [21:26:33] I didn't. [21:26:41] * James_F looks at history. [21:27:07] James_F: that's the karma error [21:27:31] https://gerrit.wikimedia.org/r/#/c/258510/ did it a month ago. Everything was fine for 20 merges… [21:28:05] Does the karma error come and go? [21:29:06] no the one from yesterday [21:29:25] James_F: you need karma 0.13.19 [21:29:35] Oh, right. [21:29:37] * James_F sighs. [21:30:01] legoktm: Did you not run your bot? [21:30:14] for karma? no [21:30:19] :-P [21:30:39] if you only do things for karma then you need some better motivation ;) [21:36:01] PROBLEM - App Server Main HTTP Response on deployment-mediawiki01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:40:54] RECOVERY - App Server Main HTTP Response on deployment-mediawiki01 is OK: HTTP OK: HTTP/1.1 200 OK - 39366 bytes in 0.576 second response time [21:47:21] (03PS1) 10Paladox: [BlueSky] Change test from mw-checks to mw-checks-test [integration/config] - 10https://gerrit.wikimedia.org/r/262820 [21:56:02] 10Continuous-Integration-Config, 10MediaWiki-Debian: Set up CI auto-building for mediawiki/debian repository - https://phabricator.wikimedia.org/T122978#1918953 (10MoritzMuehlenhoff) The mediawiki/debian repo has it's file in the top level directory of the repo, while apparently debian-glue expects it in a deb... [21:58:26] marxarelli: ryasmeen and I are working on T108874, want to join us some time today? [22:04:05] zeljkof: sure! where are y'all? [22:04:31] marxarelli: 5th [22:04:40] cool, i'll head up [22:14:15] 10Deployment-Systems, 10Traffic, 6operations: Varnish cache busting desired for /static/$VERSION/ resources which change within the lifetime of a branch - https://phabricator.wikimedia.org/T99096#1918985 (10mmodell) [22:14:19] 6Release-Engineering-Team, 6Performance-Team, 10Traffic, 6operations, 7Varnish: Verify traffic to static resources from past branches does indeed drain - https://phabricator.wikimedia.org/T102991#1918983 (10mmodell) 5Open>3Resolved a:3mmodell [22:41:07] PROBLEM - App Server Main HTTP Response on deployment-mediawiki03 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:57:57] dapatrick: Really odd. I can hit deployment-mediawiki03 from the varnish server on port 80, but I can't curl. Locally on mediawiki03, curl works. Still looking... [22:58:42] thcipriani: Weird. Thanks for continuing to bang on it. [23:12:26] also, when I do curl -H 'host: en.wikipedia.beta.wmflabs.org' localhost it sends me to the 00-nonexistent host, which is...not good... [23:21:13] thcipriani: https://phabricator.wikimedia.org/T123007 [23:21:24] dapatrick: awesome. Thank you! [23:21:33] np. Thank you!