[00:05:21] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<44.44%) [00:35:23] RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK [00:54:43] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Test results are not pretty-printed by Jenkins - https://phabricator.wikimedia.org/T113321#1661790 (10awight) 3NEW [01:54:32] (03CR) 1020after4: "I inteded for it to handle loggers passed in explicitly or by the decorator, I must have misunderstood how python keyword args work." [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [03:15:05] PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [03:50:02] RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [03:54:08] (03CR) 10Chad: "Why do we need to pass loggers anymore? I thought that's why we ended up here in the first place..." [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [03:55:55] (03CR) 1020after4: "It's no longer necessary, but I thought it'd be nice to retain the possibility to. If nobody sees a need for this then I suppose we could " [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [04:32:18] Yippee, build fixed! [04:32:18] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce build #569: FIXED in 40 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-9-sauce/569/ [05:17:17] PROBLEM - Puppet failure on deployment-cache-parsoid04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [05:17:17] PROBLEM - Puppet failure on deployment-cache-mobile04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [05:25:35] 10Deployment-Systems: Make make-wmf-branch able to branch extensions with replaced substring of the version of mediawiki being branched - https://phabricator.wikimedia.org/T51392#1662013 (10mmodell) >>! In T51392#1655049, @greg wrote: > Question: will the wiki page names change as well, I presume? > > eg: http... [05:25:57] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #549: FAILURE in 23 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/549/ [05:32:26] 10Deployment-Systems: Make make-wmf-branch able to branch extensions with replaced substring of the version of mediawiki being branched - https://phabricator.wikimedia.org/T51392#1662015 (10greg) That's fine, just need to know how to name the wiki pages :) [06:47:46] PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [07:17:43] 10Deployment-Systems, 5Patch-For-Review: Adopt Semantic Versioning format for WMF deploy branches beginning with 1.27.0-wmf.1 - https://phabricator.wikimedia.org/T67306#1662120 (10Jdforrester-WMF) >>! In T67306#1655031, @greg wrote: > Just to be clear: > > I'm making the deployment calendar for the week of Se... [07:27:46] RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0] [07:46:40] 10Deployment-Systems, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 10Unplanned-Sprint-Work, and 3 others: is shown instead of the actual message on Special:ContentTranslation in ... - https://phabricator.wikimedia.org/T112964#1662233 [07:49:19] PHPUnit tests currently fail for me on Jenkins with "Class undefined: AvroSchema" [07:49:26] (see https://integration.wikimedia.org/ci/job/mwext-Wikibase-repo-tests-sqlite-hhvm/5259/console) [07:50:01] (03CR) 10Filippo Giunchedi: [C: 031] "LGTM, not feeling strongly about the passing/no passing of logger discussion" [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [08:00:50] Adrian_WMDE: https://gerrit.wikimedia.org/r/#/c/229172/ [08:01:27] Thank you [08:01:46] Also, trailing whitespace :P [08:03:04] I'm fixing it right now [08:04:30] https://gerrit.wikimedia.org/r/240047 [08:15:17] * legoktm goes to sleep [08:15:51] Good night [08:34:02] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Test results are not pretty-printed by Jenkins - https://phabricator.wikimedia.org/T113321#1662328 (10hashar) In Jenkins, once the build is completed there are follow up actions known as publishers. That job has a publi... [08:34:31] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 7WorkType-Maintenance: Test results are not pretty-printed by Jenkins - https://phabricator.wikimedia.org/T113321#1661790 (10hashar) [08:40:46] 5Continuous-Integration-Scaling, 6operations, 7Database, 5Patch-For-Review: MySQL database for Nodepool - https://phabricator.wikimedia.org/T110693#1662337 (10hashar) >>! In T110693#1593380, @jcrespo wrote: > > BTW, the `FLUSH PRIVILEGES;` of the Openstack documentation is a bug: http://dbahire.com/stop-u... [08:41:12] 3releng-201516-q2, 5Release: [keyresult] Release MW 1.26 - https://phabricator.wikimedia.org/T110486#1662338 (10Aklapper) [09:07:48] 5Continuous-Integration-Scaling: Speed up the time to get a Nodepool instances to achieve READY state - https://phabricator.wikimedia.org/T113342#1662355 (10hashar) 3NEW [09:32:03] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce build #729: FAILURE in 1 hr 22 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-os_x_10.9-safari-sauce/729/ [11:32:40] (03CR) 1020after4: [C: 031] Use Semantic versioning format for deploy notes [tools/release] - 10https://gerrit.wikimedia.org/r/239562 (https://phabricator.wikimedia.org/T67306) (owner: 10Florianschmidtwelzow) [12:55:44] Project beta-scap-eqiad build #71126: FAILURE in 1 min 30 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/71126/ [13:05:18] Yippee, build fixed! [13:05:19] Project beta-scap-eqiad build #71127: FIXED in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/71127/ [13:29:57] (03CR) 1020after4: Use Semantic versioning format for deploy notes (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/239562 (https://phabricator.wikimedia.org/T67306) (owner: 10Florianschmidtwelzow) [13:36:55] (03CR) 10Florianschmidtwelzow: Use Semantic versioning format for deploy notes (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/239562 (https://phabricator.wikimedia.org/T67306) (owner: 10Florianschmidtwelzow) [13:43:14] (03CR) 10Jforrester: "Dependency is merged." [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [13:43:48] 10Deployment-Systems, 5Patch-For-Review: Adopt Semantic Versioning format for WMF deploy branches beginning with 1.27.0-wmf.1 - https://phabricator.wikimedia.org/T67306#1662724 (10Jdforrester-WMF) [13:43:50] 10Deployment-Systems, 10ReleaseTaggerBot: Update ReleaseTaggerBot to deal with SemVer for WMF deployed branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T107192#1662721 (10Jdforrester-WMF) 5Open>3Resolved a:3Jdforrester-WMF [13:46:46] 10Deployment-Systems, 10ReleaseTaggerBot: Update ReleaseTaggerBot to deal with SemVer for WMF deployed branches (eg 1.23.0-wmf.6) - https://phabricator.wikimedia.org/T107192#1662734 (10Jdforrester-WMF) Deployed as of 13:46 UTC. [13:49:37] (03CR) 10Jforrester: [C: 031] Use Semantic versioning format for deploy notes [tools/release] - 10https://gerrit.wikimedia.org/r/239562 (https://phabricator.wikimedia.org/T67306) (owner: 10Florianschmidtwelzow) [14:31:57] 3releng-201516-q2: [keyresult] Release MW 1.26 - https://phabricator.wikimedia.org/T110486#1662842 (10greg) [14:43:21] 5Continuous-Integration-Scaling, 7Nodepool: Nodepool delay instance deletions by one minute - https://phabricator.wikimedia.org/T113359#1662848 (10hashar) 3NEW [14:45:14] 5Continuous-Integration-Scaling: Switch a job to Nodepool instances - https://phabricator.wikimedia.org/T112750#1662858 (10hashar) >>! In T112750#1645486, @hashar wrote: > The task is essentially complete. Have to fill a few tasks following the log investigation above. Namely: > > * look at speeding up the tim... [14:46:31] zeljkof: sorry could not make it to the ci weekly :/ [14:50:01] 10Deployment-Systems: Make make-wmf-branch able to branch extensions with replaced substring of the version of mediawiki being branched - https://phabricator.wikimedia.org/T51392#1662861 (10mmodell) a:3mmodell [14:59:41] hashar: no problem [14:59:48] is your daughter ok? [15:00:00] yeah [15:00:40] she had a couple surgical suture knots :D [15:00:56] hashar: ouch [15:01:51] yeah, my son did not break any bones yet, but we also had a few stitches here and there :) [15:02:31] coming [15:06:39] 10Browser-Tests, 5Patch-For-Review: Authenticate via the API for login steps - https://phabricator.wikimedia.org/T112908#1662891 (10dduvall) 5Open>3Resolved [15:07:12] greg-g: Could you explain to me why you remove the "Release" tag from T110486 ? Because I don't see any harm with having it [15:08:34] andre___: why have it? [15:09:06] its a planning-only task, in a planning only project, which no one will be assigned to, and it's just for tracking quarterly review goal completion [15:09:55] Hmm. I'd call any task tagged with "Release" a planning-only task. [15:10:06] anyway. as you like :) [15:10:13] :) [15:10:21] g'morning/afternoon/evening andre___ [15:10:53] :) hey hey [15:11:16] andre___: fwiw, I mostly just removed it again because I removed it yesterday, if it comes back later I won't complain. :) [15:11:34] 10Browser-Tests: Empty failing tests - not sure on action - https://phabricator.wikimedia.org/T110898#1662907 (10hashar) 5Open>3Resolved a:3hashar The api python script failed to reach the MediaWiki api.php end point and thus dies after some retries. That indicates an issue with the targeted website, in t... [15:12:36] 10Continuous-Integration-Infrastructure: automatically build and commit mediawiki/vendor (composer) - https://phabricator.wikimedia.org/T101123#1662912 (10JanZerebecki) [15:21:09] 10Browser-Tests, 10MediaWiki-General-or-Unknown: Run browser tests in core on every commit - https://phabricator.wikimedia.org/T111467#1662934 (10zeljkofilipin) p:5Triage>3Normal a:3zeljkofilipin [15:21:25] 10Browser-Tests, 10MediaWiki-General-or-Unknown: Run browser tests in core on every commit - https://phabricator.wikimedia.org/T111467#1604834 (10zeljkofilipin) I will pair with @dduvall on this. [15:21:31] 10Browser-Tests, 10MediaWiki-General-or-Unknown: Run browser tests in core on every commit - https://phabricator.wikimedia.org/T111467#1662938 (10hashar) The MediaWiki core PHPUnit Zend tests are taking 10 minutes. So if the browser test jobs takes less than 10 mins its i not going to cause any slowness in gat... [15:26:59] jzerebecki: I could not make it to the triage meeting earlier today; Kid got injured. [15:27:32] hashar: eek, hope she's recovering well! [15:27:56] all fine :-} [15:28:56] hashar: no problem [15:32:10] 10Browser-Tests, 7I18n, 5Patch-For-Review: Hacking: Load i18n messages from MediaWiki to browser tests - https://phabricator.wikimedia.org/T90577#1662957 (10zeljkofilipin) a:5Vikassy>3zeljkofilipin [15:54:22] 6Release-Engineering-Team: Gerrit clean up day - releng - https://phabricator.wikimedia.org/T113028#1663054 (10greg) Gerrit searches: * [[ https://gerrit.wikimedia.org/r/#/q/status%3Aopen+%28project%3A%5Eintegration%2F.%2A+OR+project%3A%5Emediawiki%2Ftools%2Freleng+OR+project%3A%5Emediawiki%2Ftools%2Frelease+OR+... [15:55:08] 6Release-Engineering-Team: Gerrit clean up day - releng - https://phabricator.wikimedia.org/T113028#1663055 (10greg) [15:57:17] 6Release-Engineering-Team: Gerrit clean up day - releng - https://phabricator.wikimedia.org/T113028#1663059 (10hashar) >>! In T113028#1652580, @hashar wrote: > I have a potential dashboard for Gerrit open changes. Will try to get it completed by Tuesday to demo it. Couldn't finish it up over the week-end nor o... [16:15:33] 10Deployment-Systems, 10MediaWiki-extensions-LocalisationUpdate, 7I18n, 7Wikimedia-log-errors: l10n-update not updating Vector - https://phabricator.wikimedia.org/T103879#1663136 (10Nemo_bis) @supernino reports about 5ccf6668d4e0c17c not being sync'ed on Italian Wikipedia: https://it.wikipedia.org/wiki/Med... [16:18:46] 10Deployment-Systems, 10MediaWiki-extensions-LocalisationUpdate, 7I18n, 7Wikimedia-log-errors: l10n-update not updating Vector and extensions - https://phabricator.wikimedia.org/T103879#1663153 (10Nemo_bis) [16:21:19] 10Deployment-Systems: [l10n] LocalisationUpdate can incorrectly report/log success - https://phabricator.wikimedia.org/T49301#1663158 (10Nemo_bis) [16:21:20] 10Deployment-Systems: Setup alerts for l10nupdate - https://phabricator.wikimedia.org/T107515#1663157 (10Nemo_bis) [16:21:37] (03CR) 10Chad: [C: 032] Stop executing on failure [tools/scap] - 10https://gerrit.wikimedia.org/r/239521 (owner: 10Thcipriani) [16:21:52] (03Merged) 10jenkins-bot: Stop executing on failure [tools/scap] - 10https://gerrit.wikimedia.org/r/239521 (owner: 10Thcipriani) [16:22:04] thcipriani: ^^ :) [16:22:17] ostriches: thanks! [16:22:25] yw [16:23:18] 10Deployment-Systems: Remove (?) l10nupdate from !log'ing - https://phabricator.wikimedia.org/T107480#1663167 (10Nemo_bis) >>! In T107480#1496901, @Legoktm wrote: > I think we should keep it !log'ing. We log all MediaWiki code deploys through scap, and l10nupdate shouldn't be an exception. +1 Given Localisation... [16:24:23] We need to land the context logger today [16:26:18] 10Deployment-Systems: [l10n] [scap] Are Scap and LocalisationUpdate actually fighting over l10nupdates? - https://phabricator.wikimedia.org/T72446#1663188 (10Nemo_bis) >>! In T72446#760571, @bd808 wrote: > Is it really broken? I don't know about one year ago, but it currently is: {T103879}. [16:35:40] (03CR) 10Florianschmidtwelzow: "> I was basing my assumption on T67306" [tools/release] - 10https://gerrit.wikimedia.org/r/239562 (https://phabricator.wikimedia.org/T67306) (owner: 10Florianschmidtwelzow) [16:40:31] twentyafterfour: 239028 is just introducing the logger now? [16:40:39] Then we'll pile on and start using it? [16:40:46] yeah apparently [16:40:57] I don't think it breaks anything this time [16:41:17] Shouldn't, if nothing uses it hehe :p [16:41:45] :) [16:42:12] I've had a lot of problems with rebase kicking my ass [16:42:37] have I mentioned recently how much I hate gerrit? [16:45:11] 5Continuous-Integration-Scaling, 3releng-201516-q1: [keyresult] subset of jobs run in disposable instances - https://phabricator.wikimedia.org/T109914#1663270 (10hashar) [16:45:12] 5Continuous-Integration-Scaling: Switch a job to Nodepool instances - https://phabricator.wikimedia.org/T112750#1663269 (10hashar) 5Open>3Resolved [16:48:46] !log restarting nodepool to take in account logging config change (dropping apscheduler bucket) [16:48:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [16:49:11] 5Continuous-Integration-Scaling: Switch CI config job to Nodepool instances - https://phabricator.wikimedia.org/T112750#1663322 (10greg) [16:49:42] 5Continuous-Integration-Scaling, 3releng-201516-q1, 15User-greg: [keyresult] subset of jobs run in disposable instances - https://phabricator.wikimedia.org/T109914#1663323 (10greg) 5Open>3Resolved a:3greg CI config is running on nodepool! [16:54:32] greg-g: I closed a bunch of ci scaling tasks already. There is a few more to deal with though [16:55:06] (03PS11) 10Chad: A context manager for managing nested loggers [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [16:55:09] (03PS1) 10Chad: Use context logger and stop passing one to sudo_check_call [tools/scap] - 10https://gerrit.wikimedia.org/r/240126 [16:55:23] (03CR) 10jenkins-bot: [V: 04-1] Use context logger and stop passing one to sudo_check_call [tools/scap] - 10https://gerrit.wikimedia.org/r/240126 (owner: 10Chad) [16:55:43] (03CR) 10Chad: "PS10 was just a rebase" [tools/scap] - 10https://gerrit.wikimedia.org/r/239028 (owner: 1020after4) [16:55:52] hashar: cool [16:56:20] not rushing it though, I am very afraid of breaking everything :-D [16:57:20] * hashar escapes [16:57:28] :) [16:58:10] (03PS2) 10Chad: Use context logger and stop passing one to sudo_check_call [tools/scap] - 10https://gerrit.wikimedia.org/r/240126 [16:58:24] twentyafterfour: WFM ^ :) [16:58:40] We should checkout that sha1 to beta and try it ;-) [17:00:06] (03PS1) 10Legoktm: Run flake8 for labs/tools/forrestbot [integration/config] - 10https://gerrit.wikimedia.org/r/240129 [17:00:43] (03CR) 10Jforrester: "Dupe of I4b0b7b1de…" [integration/config] - 10https://gerrit.wikimedia.org/r/240129 (owner: 10Legoktm) [17:02:36] oops [17:03:06] (03Abandoned) 10Legoktm: Run flake8 for labs/tools/forrestbot [integration/config] - 10https://gerrit.wikimedia.org/r/240129 (owner: 10Legoktm) [17:03:14] James_F: I'm not a reviewer on that change :P [17:03:19] (03PS2) 10Legoktm: Add flake8 tox testing for labs/tools/forrestbot [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [17:03:23] legoktm: Now you are. :-) [17:03:42] (03CR) 10Legoktm: [C: 032] Add flake8 tox testing for labs/tools/forrestbot [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [17:04:34] (03Merged) 10jenkins-bot: Add flake8 tox testing for labs/tools/forrestbot [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [17:05:16] !log deploying https://gerrit.wikimedia.org/r/238985 [17:05:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [17:06:05] (03CR) 10Legoktm: "https://integration.wikimedia.org/ci/job/tox-flake8-trusty/325/console woot" [integration/config] - 10https://gerrit.wikimedia.org/r/238985 (owner: 10Jforrester) [17:30:09] Project browsertests-ZeroBanner-en.m.wikipedia.org-linux-chrome build #2: FAILURE in 9.1 sec: https://integration.wikimedia.org/ci/job/browsertests-ZeroBanner-en.m.wikipedia.org-linux-chrome/2/ [17:37:37] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 7WorkType-Maintenance: Test results are not pretty-printed by Jenkins - https://phabricator.wikimedia.org/T113321#1663569 (10awight) 5Open>3Invalid a:3awight @hashar: Thanks for investigating, and sorry for the t... [17:54:01] 10Browser-Tests, 5Patch-For-Review: mediawiki_api gem recursion on log_in - https://phabricator.wikimedia.org/T111133#1663663 (10dduvall) Sent an email to the maintainer and offered to help with merges/releases if they need it. [18:25:25] 10Continuous-Integration-Config, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 7WorkType-Maintenance: Test results are not pretty-printed by Jenkins - https://phabricator.wikimedia.org/T113321#1663749 (10hashar) >>! In T113321#1663569, @awight wrote: > @hashar: Thanks for investigating, and sorry... [18:27:21] 5Continuous-Integration-Scaling, 7Nodepool: Nodepool delay instance deletions by one minute - https://phabricator.wikimedia.org/T113359#1663752 (10hashar) p:5Triage>3Low Will get to it when it starts becoming a problem (i.e.: Nodepool not refilling the pool fast enough). The actual patch is straightforwar... [18:31:08] 5Continuous-Integration-Scaling, 10Deployment-Systems, 6Labs, 10Labs-Infrastructure, and 2 others: Can not use git-deploy from tin.eqiad.wmnet to labnodepool1001.eqiad.wmnet - https://phabricator.wikimedia.org/T111925#1663775 (10hashar) @andrew and @yuvipanda i discovered we can not git-deploy from the dep... [18:47:21] twentyafterfour: Your decorator fails if there's 2 decorators on a method. [18:47:27] Pasting, I have a concrete example... [18:48:27] https://phabricator.wikimedia.org/P2076 [18:50:15] Ah, yours works. The other fails. [18:53:36] pooor git [18:53:53] I found out analytics/aggregator/data.git takes 3.1GBytes on ytterbium :-/ [19:02:52] (03Restored) 10Chad: Simplify logging in ssh module [tools/scap] - 10https://gerrit.wikimedia.org/r/238959 (owner: 10Chad) [19:03:01] (03PS6) 10Chad: Simplify logging in ssh module [tools/scap] - 10https://gerrit.wikimedia.org/r/238959 [19:03:18] (03CR) 10jenkins-bot: [V: 04-1] Simplify logging in ssh module [tools/scap] - 10https://gerrit.wikimedia.org/r/238959 (owner: 10Chad) [19:05:18] 5Continuous-Integration-Scaling, 6Labs, 10Labs-Infrastructure: Evaluate impact of giant VM images on labs infrastructure - https://phabricator.wikimedia.org/T113394#1663916 (10hashar) 3NEW a:3Andrew [19:05:24] hashar: Repacked, down to 2.9 [19:05:55] ostriches: hey thx. I am wondering why it is so large though. [19:06:07] but yeah not a big deal, that is a low traffic repo [19:06:20] It's just a ton of data. [19:06:23] afaict. [19:06:31] yup looks like [19:06:35] good old csv [19:08:09] 127164 ./projectcounts [19:10:38] ostriches: don't obsess over it anyway [19:11:07] legoktm: good morning :D will you ever look at the patch against integration/dashboard or should I just self merge ? -:D [19:11:36] hashar: please self-merge it, I'm unlikely to get to it anytime soon [19:11:57] legoktm: will do [19:12:28] (03CR) 10Hashar: [C: 032] Use lib to determine paths and os.path.join() [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238688 (owner: 10Hashar) [19:12:41] (03Merged) 10jenkins-bot: Use lib to determine paths and os.path.join() [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238688 (owner: 10Hashar) [19:16:07] (03PS7) 10Chad: Simplify logging in ssh module [tools/scap] - 10https://gerrit.wikimedia.org/r/238959 [19:36:57] (03CR) 10Hashar: [C: 032] "Will refactor later :-}" [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238689 (owner: 10Hashar) [19:37:09] (03Merged) 10jenkins-bot: Introduce config file to set the various paths [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238689 (owner: 10Hashar) [19:37:23] (03CR) 10Hashar: [C: 032] Support different zuul server command [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238690 (owner: 10Hashar) [19:37:30] (03Merged) 10jenkins-bot: Support different zuul server command [integration/dashboard] - 10https://gerrit.wikimedia.org/r/238690 (owner: 10Hashar) [19:48:30] PROBLEM - Puppet failure on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:48:34] (03PS1) 10Ottomata: Use jessie to test EventLogging server so etcd can be tested [integration/config] - 10https://gerrit.wikimedia.org/r/240161 [19:49:17] (03CR) 10Hashar: [C: 032] Use jessie to test EventLogging server so etcd can be tested [integration/config] - 10https://gerrit.wikimedia.org/r/240161 (owner: 10Ottomata) [19:50:02] (03Merged) 10jenkins-bot: Use jessie to test EventLogging server so etcd can be tested [integration/config] - 10https://gerrit.wikimedia.org/r/240161 (owner: 10Ottomata) [20:03:44] 6Release-Engineering-Team, 7Tracking: Fix easy problems reported by RuboCop - https://phabricator.wikimedia.org/T91485#1664139 (10zeljkofilipin) [20:03:45] 10Browser-Tests, 5Patch-For-Review, 5WMF-deploy-2015-09-22_(1.26wmf24): Fix problems reported by RuboCop in Math - https://phabricator.wikimedia.org/T111184#1664138 (10zeljkofilipin) 5Open>3Resolved [20:04:14] 6Release-Engineering-Team, 7Tracking: Update repositories that use mediawiki_selenium Ruby gem 1.x - https://phabricator.wikimedia.org/T94083#1664141 (10zeljkofilipin) [20:04:17] 10Browser-Tests, 10Math, 5Patch-For-Review, 5WMF-deploy-2015-09-22_(1.26wmf24): Upgrade Math browser tests to use mediawiki_selenium 1.x - https://phabricator.wikimedia.org/T99656#1664140 (10zeljkofilipin) 5Open>3Resolved [20:05:01] 10Browser-Tests, 5Patch-For-Review, 5WMF-deploy-2015-09-22_(1.26wmf24): Fix problems reported by RuboCop in WikiLove - https://phabricator.wikimedia.org/T110672#1664146 (10zeljkofilipin) 5Open>3Resolved [20:05:02] 6Release-Engineering-Team, 7Tracking: Fix easy problems reported by RuboCop - https://phabricator.wikimedia.org/T91485#1087225 (10zeljkofilipin) [20:06:32] Can I get a review of https://gerrit.wikimedia.org/r/#/c/220489/2 from someone who is interested in ruby linting? [20:08:54] andrewbogott: all those scope.lookupvar() keep confusing me :/ [20:09:50] andrewbogott: maybe you can give it a try on the beta cluster puppetmaster? deployment-puppetmaster.deployment-prep.eqiad.wmflabs [20:10:03] sure, I’ll try that [20:11:07] oh matinee commented on it [20:15:59] it needs a rebase, i’m working on it [20:22:25] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<10.00%) [20:33:16] sleep time *wave [20:33:35] twentyafterfour: https://www.mediawiki.org/w/index.php?title=MediaWiki_1.26/wmf4/Changelog&diff=prev&oldid=1894460 are the notes for wmf*24* on wmf*4*. [20:33:48] twentyafterfour: Assuming it was unintentional? [20:34:47] James_F: maybe a bug in the patch to make-deploy-notes [20:35:04] that was supposed to support the upcoming semver stuff [20:35:07] Thought it might be. :-( [20:38:28] Hey... did our qunit test environment change in the last day? [20:38:39] in any way whatsoever [20:38:51] Seeing lots of Jenkins -2s like this https://integration.wikimedia.org/ci/job/mwext-qunit/5763/console [20:38:57] (test used to pass and can replicate locally) [20:40:33] 10Browser-Tests, 10QuickSurveys, 5Patch-For-Review, 3Reading-Web-Sprint-56-Four Lions: QA: Setup browser tests on beta cluster so we can share test articles - https://phabricator.wikimedia.org/T112204#1664268 (10Jdlrobson) @phuedx think you can +2 it or find someone who can? Really keen to get this deploye... [20:40:35] jdlrobson: Was just talking about it with Krinkle. [20:42:12] jdlrobson: I'm rewriting the test. The test is testing things it shouldn't be testing. [20:42:20] Krinkle: Thanks. [20:43:22] cool. Glad it's being looked into. Krinkle can you add me as reviewer? Would be keen to see how you rewrite it [20:43:34] do we know why it suddenly started failing? [20:43:38] that also had me curious [20:44:08] PROBLEM - Puppet failure on deployment-zookeeper01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [20:45:57] James_F: fixed the wikipages [20:47:00] twentyafterfour: https://www.mediawiki.org/wiki/MediaWiki_1.26/wmf24/Changelog is wmf4..wmf24 [20:50:58] ok so it seems we updated chrome :) [20:53:57] James_F: better [20:54:09] RECOVERY - Puppet failure on deployment-zookeeper01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:54:11] twentyafterfour: Awesome. :-) [20:54:33] one unescaped period. [20:54:55] regex: the write only programming language. [20:59:35] (03PS1) 1020after4: Fix unescaped period in regex. [tools/release] - 10https://gerrit.wikimedia.org/r/240267 [21:07:59] just revert the chrome update? seems like a cheap thing to do here? [21:20:34] jdlrobson: Revert Google's production deployment? [21:20:45] jdlrobson: How exactly do you propose to do that? [21:21:01] James_F: well i assume somewhere we updated the browser just like i have to update my browser.. google can't force changes on me... [21:21:08] jdlrobson: … no. [21:21:15] jdlrobson: We didn't, it's auto ensure "latest" stable [21:21:27] Requires ops to change the Ubuntu package release [21:21:42] ok well if we have that i guess these sort of things are out of our control then. Thanks for the clarification would have helped back at 1:43pm :) [21:26:02] 10Continuous-Integration-Config, 10MediaWiki-Unit-tests, 5Patch-For-Review: localStorage qunit tests causing failures with Chrome 45 - https://phabricator.wikimedia.org/T113413#1664464 (10Jdforrester-WMF) p:5Triage>3Unbreak! a:3Krinkle [21:26:35] jdlrobson: As I said in my commit, it started failing because, per the W3 standard, the localStorage global is read-only. [21:26:47] The previous test was overwriting the global directly (which Sinon does) [21:27:07] yup i understand. The bit missing was how this magically works and gets updated and suddenly starts failing [21:27:24] Yeah, Chrome 45 fixed the standards bug [21:29:31] jdlrobson: Chrome did that mostly to fix regressions in Android, where some of the popular libraries require the global to be read-only for performance reasons. [21:29:52] Scope lookups run much faster that way [21:33:21] (03CR) 10Florianschmidtwelzow: [C: 032] "One sign... ;) Thanks for fixing!" [tools/release] - 10https://gerrit.wikimedia.org/r/240267 (owner: 1020after4) [21:41:12] (03Merged) 10jenkins-bot: Fix unescaped period in regex. [tools/release] - 10https://gerrit.wikimedia.org/r/240267 (owner: 1020after4) [21:42:08] (03PS3) 10Florianschmidtwelzow: Output plain html entities [tools/release] - 10https://gerrit.wikimedia.org/r/231317 [21:44:03] 10Continuous-Integration-Config, 10MediaWiki-Unit-tests, 5Patch-For-Review: localStorage qunit tests causing failures with Chrome 45 - https://phabricator.wikimedia.org/T113413#1664497 (10Legoktm) [22:51:59] zuul stuck? [22:52:53] complaining in here fixed it [22:52:54] looks like stuff is moving to me [22:52:59] hah [23:12:41] zuul stuck? [23:12:45] 3..2..1... [23:23:54] (03PS1) 10Thcipriani: Add config deployment [tools/scap] - 10https://gerrit.wikimedia.org/r/240292 (https://phabricator.wikimedia.org/T109512) [23:27:50] (03CR) 10jenkins-bot: [V: 04-1] Add config deployment [tools/scap] - 10https://gerrit.wikimedia.org/r/240292 (https://phabricator.wikimedia.org/T109512) (owner: 10Thcipriani) [23:42:47] (03CR) 10Bartosz Dziewoński: [C: 04-1] "Needs rebase." [tools/release] - 10https://gerrit.wikimedia.org/r/230705 (owner: 10Awight)