[00:02:16] 10Continuous-Integration, 7Tracking: Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) - https://phabricator.wikimedia.org/T69216#1064429 (10greg) [00:02:17] 6Release-Engineering, 10Wikimedia-Hackathon-2015: Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc) - https://phabricator.wikimedia.org/T90647#1064430 (10greg) [00:02:49] 10Continuous-Integration, 10Wikimedia-Hackathon-2015: Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc) - https://phabricator.wikimedia.org/T90647#1064056 (10greg) [00:03:28] 10Continuous-Integration, 6Mobile-Web, 10VisualEditor, 10Wikimedia-Hackathon-2015: Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc) - https://phabricator.wikimedia.org/T90647#1064441 (10greg) [00:03:47] James_F: ^ add/remove projects as you see fit ^ [00:12:55] 10Continuous-Integration, 10utfnormal, 5Patch-For-Review: utfnormal tests failing on jenkins and travis-ci - https://phabricator.wikimedia.org/T90278#1064449 (10Legoktm) a:3brion Passing on jenkins and travis-ci! https://travis-ci.org/wikimedia/utfnormal/builds/52058228 Just need to make it voting on jenk... [00:19:12] greg-g: Ta. [00:19:29] greg-g: Those projects seem like a good start. [00:20:14] 10Continuous-Integration, 10Flow, 6Mobile-Web, 10VisualEditor, 10Wikimedia-Hackathon-2015: Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc) - https://phabricator.wikimedia.org/T90647#1064461 (10Jdforrester-WMF) [00:23:08] should that be lowercase "editing"? [00:23:27] is there one? [00:23:49] up to james [00:29:01] greg-g: James_F ryasmeen and I brainstormed another possible demo-able Hackathon project, builds to target particular older browser versions/OSs. It would be an interesting spike to see what happens with cursor stuff. I don't think she wrote it up yet. [00:29:32] that'd be good yeah, that should be in the backlog regardless too :) [00:30:24] greg-g: I'm not convinced it would be valuable, thus a Hackathon spike seems appropriate [00:30:49] ah, I thought it was something that the Editing team wanted generally [00:31:27] greg-g: it's valuable to check stuff manually, but Selenium vs contentEditable vs. browsers has been a struggle since contentEditable was a thing. [00:33:01] and then add in VE's approach(es) to cursoring, things get weird historically [00:33:49] legoktm: The Editing Department is "Editing". The concept of editing, if it were a tag (?) in Phabricator, would be "editing", yes. [00:34:33] James_F: er, I meant the title "Create Jenkins builds for Editing across repositories", should that be lowercase "e" because you're talking about "editing tests" rather than the Editing team? [00:35:00] legoktm: But Editing Department responsibilities? Eh. [00:35:38] legoktm: (Theoretically it's our problem even if it's not our fault that e.g. the wikitext editor inside Mobile Web doesn't work in IE4. Not necessarily that we're going to fix it…) [00:35:57] :P [00:36:01] (Also, suppositional example is not a claim of non-working-ness.) [00:45:21] 10Continuous-Integration, 7Upstream: Zuul-cloner failing to acquire .git/config lock sometimes - https://phabricator.wikimedia.org/T86730#1064531 (10Krinkle) a:5Krinkle>3None [00:46:35] 6Release-Engineering, 10Wikimedia-General-or-Unknown: Report more information about php warnings - https://phabricator.wikimedia.org/T45086#1064533 (10Krinkle) a:5Krinkle>3None [00:47:50] Krinkle: are there any guidelines for self-merging in integration/config? [00:48:50] legoktm: Only merge if you're contint-admin and (for jjb) have deployed or (for zuul) will deploy immediately. [00:49:24] As for peer-review, right now its a bit like ops/. Review if you think it needs review. If it's a minor change and fulfilling an approved user request, just do it. [00:49:39] Ok [00:49:59] "if it works i approve, if it breaks it's your problem" /me hides [00:50:19] Generally, anything that's not a bug fix or feature (e.g. adding repos, adding dependencies, insantiating existing jobs) just do it. Anything else (Bug fixes, new features, new job types) need review and a few day waiting otherwise. [00:50:26] Krinkle: I'll be added to contint-admins tomorrow (I think!), is it ok if I deploy jjb-only stuff in the meantime? https://gerrit.wikimedia.org/r/#/c/192171/ for example. [00:51:50] legoktm: Do you intend for that macro to be re-used elsewhere? [00:52:14] Yes, when we start using composer to fetch dependencies for MW [00:52:27] which is https://phabricator.wikimedia.org/T90303 [00:52:49] In that case I would normally leave the patch set in gerrit and write that change on top of it and then merge at the same time when ready. That way we're not adding unused macros to use one day. [00:52:59] shouldn't be blocking anything [00:53:10] and since they expand to the same, existing jobs don't need updating per se. [00:53:17] Although I suppose it's usefuil to verify it still works [00:53:21] So yeah, go ahead :) [00:53:40] Use integration-jjb-config-diff output to know which jobs need pushing [00:53:47] to make sure there's no difference between the repo and Jenkins [00:54:06] (03PS3) 10Legoktm: Create "composer-update" builder macro [integration/config] - 10https://gerrit.wikimedia.org/r/192171 [00:54:14] "File changed:" section [00:54:19] ok [00:56:51] Krinkle: also your input on my question at https://phabricator.wikimedia.org/T90303#1058082 would be appreciated [01:03:27] (03CR) 10Legoktm: [C: 032] Create "composer-update" builder macro [integration/config] - 10https://gerrit.wikimedia.org/r/192171 (owner: 10Legoktm) [01:03:49] 10Continuous-Integration: Fetch dependencies using composer instead of cloning mediawiki/vendor repository for non-WMF deployment branches - https://phabricator.wikimedia.org/T90303#1064563 (10Krinkle) >>! In T90303#1058082, @Legoktm wrote: > Should we have a separate mediawiki-phpunit-{phpflavor} / mediawiki-ph... [01:04:54] legoktm: --ff-only [01:05:11] er, what? [01:05:23] integration/* is fast-forward only [01:05:26] won't merge [01:05:47] I just rebased it? [01:05:49] its in gerrit's change infobox [01:05:50] Yeah [01:05:57] Ok [01:06:02] I hadn't refreshed yet [01:07:34] ok :) [01:09:58] (03Merged) 10jenkins-bot: Create "composer-update" builder macro [integration/config] - 10https://gerrit.wikimedia.org/r/192171 (owner: 10Legoktm) [01:44:21] bd808 & ^d: https://github.com/r0ml/mod_git <- should be possible to modify this (or use mostly unmodified) to do what we want [01:44:44] there is also https://github.com/bnoordhuis/mod_git [01:48:35] (03PS3) 10Legoktm: Add script to create a composer.local.json based on a list of extensions [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192177 [01:50:51] 10Continuous-Integration, 10OOjs-UI, 5OOjs-UI-next-release: Run the OOjs UI PHP test code on patchsets - https://phabricator.wikimedia.org/T87724#1064592 (10Jdforrester-WMF) 5Open>3Resolved a:3matmarex [02:10:34] 10Continuous-Integration: Concurrent builds using local web browsers on Linux host (Chromium/Firefox) fail - https://phabricator.wikimedia.org/T90673#1064651 (10Krinkle) 3NEW [02:10:57] 10Continuous-Integration, 10VisualEditor, 5§ VisualEditor Q3 Blockers: Investigate browser disconnect failures when running MediaWiki tests on labs slaves - https://phabricator.wikimedia.org/T89075#1064658 (10Krinkle) [02:11:05] 10Continuous-Integration: Concurrent builds using local web browsers on Linux host (Chromium/Firefox) fail - https://phabricator.wikimedia.org/T90673#1064660 (10Krinkle) [02:11:07] 10Continuous-Integration, 10VisualEditor, 5§ VisualEditor Q3 Blockers: Investigate browser disconnect failures when running MediaWiki tests on labs slaves - https://phabricator.wikimedia.org/T89075#1027004 (10Krinkle) [02:15:27] !log integration-slave1006 has <700MB free disk space (including /tmp) [02:15:35] Logged the message, Master [02:26:08] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1064677 (10Krinkle) Don't forget the underscores. ``` python >>> import six >>> six.__version__ ``` [02:26:11] do we have a ruby module for setting preferences in browser tests via the API? [02:28:22] 10Continuous-Integration: Jenkins sporadically changes its UI language - https://phabricator.wikimedia.org/T90558#1064679 (10Krinkle) https://wikitech.wikimedia.org/wiki/Release_Engineering/Argh#Jenkins_interface_language [02:28:37] 10Continuous-Integration, 7Jenkins: Jenkins sporadically changes its UI language - https://phabricator.wikimedia.org/T90558#1064680 (10Krinkle) [02:34:25] 10Continuous-Integration, 10Flow: Flow tests are leaving lci_* temporary files behind on integration slaves - https://phabricator.wikimedia.org/T89122#1064701 (10Krinkle) [02:34:26] 10Continuous-Integration, 5Patch-For-Review: Jenkins: Figure out long term solution for /tmp management - https://phabricator.wikimedia.org/T74011#1064700 (10Krinkle) [02:34:36] 10Continuous-Integration, 5Patch-For-Review: Jenkins: Figure out long term solution for /tmp management - https://phabricator.wikimedia.org/T74011#759885 (10Krinkle) [02:34:37] 10Continuous-Integration: Consider running tmpreaper on Jenkins jobs' tmpfs - https://phabricator.wikimedia.org/T89327#1064703 (10Krinkle) [04:21:25] Yippee, build fixed! [04:21:25] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce build #333: FIXED in 39 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/333/ [04:55:26] 6Release-Engineering, 10VisualEditor, 7Browser-Tests: Create VisualEditor tests targeting the older version of browsers (Chrome and Firefox for now) for better backward compatibility - https://phabricator.wikimedia.org/T90678#1064772 (10Ryasmeen) 3NEW a:3Ryasmeen [04:56:53] Yippee, build fixed! [04:56:54] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #535: FIXED in 35 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/535/ [05:14:56] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #537: FAILURE in 18 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/537/ [05:48:45] Project browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #1: SUCCESS in 1 min 7 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/1/ [05:51:31] Yippee, build fixed! [05:51:31] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #489: FIXED in 28 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/489/ [06:01:15] I'm adding few wikis to Beta cluster, any objections or should I file a bug? :) [06:01:21] YuviPanda|zzz: ^^ [06:01:43] When in doubt always file a bug [06:02:02] YuviPanda|zzz: Sure. [06:02:28] yes, tracking please :) [06:02:37] (even if rubber stamped) [06:07:45] 10Beta-Cluster, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 5ContentTranslation-Release4: Setup new wikis in Beta Cluster for Content Translation - https://phabricator.wikimedia.org/T90683#1064892 (10KartikMistry) 3NEW a:3KartikMistry [06:09:07] greg-g: YuviPanda|zzz done. [06:20:06] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1064902 (10mmodell) apparently it's not getting to RSYNC because that didn't make any difference... [06:37:27] 10Deployment-Systems, 6Release-Engineering, 7Epic: Rethinking our deployment process - https://phabricator.wikimedia.org/T89945#1064904 (10mmodell) [07:01:43] 10Beta-Cluster, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 5ContentTranslation-Release4, 3LE-Sprint-83: Setup new wikis in Beta Cluster for Content Translation - https://phabricator.wikimedia.org/T90683#1064916 (10KartikMistry) [07:45:58] 10Beta-Cluster, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 5ContentTranslation-Release4, 3LE-Sprint-83: Setup new wikis in Beta Cluster for Content Translation - https://phabricator.wikimedia.org/T90683#1064934 (10KartikMistry) We need following wikis in first stage (sam... [08:23:09] (03CR) 10Adrian Lang: "Thanks for the fixing, added tests and explanation :)" (031 comment) [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [08:33:25] (03CR) 10Legoktm: "Looks good, see inline comment." (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/192201 (https://phabricator.wikimedia.org/T46875) (owner: 10Sumit) [08:39:33] (03PS4) 10Legoktm: Fixed phpcs sniff verifies wg prefix against all global variables [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/192168 (https://phabricator.wikimedia.org/T90365) (owner: 10Phoenix303) [08:43:41] (03CR) 10Legoktm: [C: 031] "I added a separate test case which verifies that if the first global is fine, but the second is not, the sniff will fail." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/192168 (https://phabricator.wikimedia.org/T90365) (owner: 10Phoenix303) [09:04:43] (03PS1) 10Legoktm: Create oojs-ui-composer job, get rid of legacy oojs-ui-phpcs-HEAD [integration/config] - 10https://gerrit.wikimedia.org/r/192772 [09:07:17] PROBLEM - Puppet staleness on deployment-eventlogging02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [09:08:08] (03CR) 10Legoktm: [C: 031] Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 (owner: 10Gergő Tisza) [09:17:06] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1064996 (10zeljkofilipin) ``` $ python Python 2.7.6 (default, Sep 9 2014, 15:04:36) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin Type "help", "copyright", "credi... [09:19:24] (03CR) 10Legoktm: [C: 04-1] "Code looks fine, but the code style could be improved. No need for extra spaces inside [ ]. See https://www.mediawiki.org/wiki/Manual:Codi" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/192248 (https://phabricator.wikimedia.org/T53279) (owner: 10Phoenix303) [09:20:57] zeljkof: yo [09:21:08] aharoni: hi, sorry, in a meeting [09:21:24] zeljkof: np :) a few patches for your review, not urgent. [09:21:34] aharoni: will take a look after the meeting [09:22:22] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1065017 (10zeljkofilipin) [09:22:38] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1065020 (10Legoktm) Try `sudo pip install --upgrade six` ? [09:39:37] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1065052 (10hashar) Paired with Zeljkof. Pip installs its material in `/Library/Python/2.7/site-packages` which is the last in python path lookup (confirmed via sys.path). The workar... [12:37:34] PROBLEM - SSH on deployment-lucid-salt is CRITICAL: Connection refused [14:23:58] (03CR) 10Jforrester: [C: 031] Create oojs-ui-composer job, get rid of legacy oojs-ui-phpcs-HEAD [integration/config] - 10https://gerrit.wikimedia.org/r/192772 (owner: 10Legoktm) [15:48:06] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1065975 (10bd808) >>! In T76061#1064902, @mmodell wrote: > apparently it's not getting to RSYNC because that didn't make any difference... I'm not eve... [16:15:47] <^d> greg-g: I think I'll be OOO on March 9-11th now. Elastic conference came back and gave me and Nik a joint speaking slot (after originally declining my standalone talk) [16:16:00] <^d> It's here in the city [16:16:59] 10Continuous-Integration, 7Epic, 3releng-201415-Q3: Jenkins: Run jobs in disposable VMs - https://phabricator.wikimedia.org/T47499#1066045 (10greg) [16:17:00] 10Continuous-Integration, 7Browser-Tests, 7Tracking: [project] trigger browser tests from Gerrit (tracking) - https://phabricator.wikimedia.org/T55697#1066044 (10greg) [16:17:48] ^d: cool! do you need to pay for a ticket? or are you covered since you're talking? [16:18:19] <^d> I think the latter. Part of the reason I hadn't planned on going was because the ticket was expensive if I wasn't talking :p [16:18:28] :) [16:20:03] <^d> In any case, since it's here no need for travel/hotel :) [16:20:36] yep, easy peasy [16:21:20] <^d> Dogpatch, to be specific [16:21:21] <^d> lol [16:21:27] .... huh [16:21:29] <^d> Pier 27. I had to look it up [16:27:12] 6Release-Engineering, 7Browser-Tests: Move browser test alerts to responsible teams' channels from -releng - https://phabricator.wikimedia.org/T89375#1066063 (10greg) a:3Cmcmahon [16:34:44] I don't think I've ever paid to go to a software conference. The best ones have no fee, the other ones I present at. [16:43:17] I might have to cancel 1:1s this afternoon, I woke up with a migraine (it happens, unfortunately). I'm going to try to attend this 9-noon meeting of fun, I'll email when I can't take it any more (cc: chrismcmahonbrb twentyafterfour ) [16:51:38] <^d> Ouch. Feel better [16:58:24] Do we have any repos that run actual JUnit on Java code, yet? I'd like to add that to CI for mwdumper... [17:02:04] greg-g: wfm. everything I'm doing you know about, email if you want anything specific [17:05:00] <^d> awight: mwdumper getting some love? :) [17:06:07] ^d: I'm finding catharsis in java programming, of all things. There's only so much PHP a person should be exposed to in a lifetime. [17:07:17] <^d> I enjoy java. Especially well-written Java [17:07:26] <^d> (poorly-written java is a nightmare) [17:08:04] (03CR) 10Brion VIBBER: [C: 031] "lgtm, but I don't have +2 in this repo :)" [integration/config] - 10https://gerrit.wikimedia.org/r/192713 (https://phabricator.wikimedia.org/T90278) (owner: 10Legoktm) [17:08:24] My last professional encounter with Java was around 2003, it was a dirty world back then. They were just working out generics, which were badly needed... [17:13:47] <^d> Hmmm, so this needs a zuul reload? [17:15:02] no idea, the patch is legoktm’s dark magic :) [17:15:22] it removes the blacklist entry though which sounds right ;) [17:15:29] <^d> I think I need do reload zuul [17:15:33] <^d> Let's see if I can do this [17:15:39] <^d> There's docs! [17:16:01] <^d> Oh I can't +2 either [17:16:03] <^d> Thought I could [17:16:12] <^d> Haha [17:16:19] lol [17:16:30] well worst case lego can +2 it himself i think ;) [17:16:37] <^d> I can give myself +2 ;-) [17:16:39] if we don’t get a hashar first [17:16:41] hehe [17:17:20] (03PS2) 10Chad: Make utfnormal-composer voting [integration/config] - 10https://gerrit.wikimedia.org/r/192713 (https://phabricator.wikimedia.org/T90278) (owner: 10Legoktm) [17:17:32] <^d> Rebased [17:18:01] whee [17:18:35] <^d> Ok, I can sudo as zuul too, should be ok to do this now [17:19:11] (03CR) 10Chad: [C: 032] Make utfnormal-composer voting [integration/config] - 10https://gerrit.wikimedia.org/r/192713 (https://phabricator.wikimedia.org/T90278) (owner: 10Legoktm) [17:19:15] (03PS1) 10Krinkle: mw-teardown: Remove entire dbpath, not just db.sqlite [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192821 [17:19:39] 10Continuous-Integration, 10Wikidata, 10§ Wikidata-Sprint-2015-02-03, 10§ Wikidata-Sprint-2015-02-25, and 2 others: mw-debug.log missing in Jenkins jobs (Failed to be created "Permission denied") - https://phabricator.wikimedia.org/T85799#1066166 (10JanZerebecki) 5Open>3Resolved [17:20:17] (03Merged) 10jenkins-bot: Make utfnormal-composer voting [integration/config] - 10https://gerrit.wikimedia.org/r/192713 (https://phabricator.wikimedia.org/T90278) (owner: 10Legoktm) [17:22:47] <^d> !log reloading zuul to pick up utfnormal jobs [17:22:52] Logged the message, Master [17:23:06] it’s kinda fun helping retool my old code like that [17:23:24] unicode normalization is WEIRD STUFF :) [17:23:41] i say we go back to ASCII [17:23:45] <^d> zuul seems ok [17:23:50] lowercase letters are overkill [17:23:54] yay [17:24:10] lemme submit a test job [17:25:41] <^d> https://integration.wikimedia.org/ci/job/utfnormal-composer/15/ is running :) [17:25:47] \o/ [17:28:51] (03CR) 10Krinkle: [C: 032] mw-teardown: Remove entire dbpath, not just db.sqlite [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192821 (owner: 10Krinkle) [17:28:52] <^d> slowwww job [17:29:23] (03Merged) 10jenkins-bot: mw-teardown: Remove entire dbpath, not just db.sqlite [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192821 (owner: 10Krinkle) [17:30:42] yeah [17:30:44] but it works :D [17:31:02] so I setup https://wikitech.wikimedia.org/wiki/Hiera:Staging Running `puppet agent -tv` on staging-palladium and adding `notify {"${::puppet_statsd::statsd_host}":}` inside class puppet_statsd seems to suggest that mwyaml backend may not be getting read. What am I missing? [17:31:29] 10Continuous-Integration, 10utfnormal, 5Patch-For-Review: utfnormal tests failing on jenkins and travis-ci - https://phabricator.wikimedia.org/T90278#1066208 (10brion) 5Open>3Resolved and chad got it merged and it's working :D [17:31:31] Wow, 4 minutes to run phpunit with 97 tests [17:31:38] (not counting setup/teardown of the workspace) [17:31:47] ^d: brion: What's it doing that is so expensive? [17:31:48] yeah i think something’s tripping it up, it shouldn’t be that slow [17:32:15] most of the tests run real fast… [17:32:16] in fact that offset is only for the first 65 tests [17:32:19] then it kinda sticks [17:32:35] <^d> hmmm [17:32:47] approaching parser-tests level of speed. [17:32:51] :( [17:33:16] But great work on getting composer used there. Makes it a lot more portable [17:34:13] i'm seeing similar "sticking" with some rspec tests [17:34:21] https://integration.wikimedia.org/ci/job/mediawiki-selenium-bundle-rspec/79/console [17:34:55] ok it’s def in the tests… i wonder if it’s just hitting worst-case behavior for some of the cases [17:35:04] that test suite usually takes less than 1s to complete locally [17:35:08] takes ~43 seconds on my mac [17:35:24] (without php-intl extension) [17:35:27] <^d> ok it finally finished [17:36:40] <^d> Longest seems to have been NFC? [17:36:54] <^d> hmm nvm [17:38:58] (03CR) 10Aaron Schulz: mw-teardown: Remove entire dbpath, not just db.sqlite (031 comment) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192821 (owner: 10Krinkle) [17:40:26] (03PS1) 10Krinkle: mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192829 [17:40:40] (03CR) 10Krinkle: mw-teardown: Remove entire dbpath, not just db.sqlite (031 comment) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192821 (owner: 10Krinkle) [17:42:10] <^d> Is the beta update job failing? [17:42:36] <^d> Started about an hour ago [17:45:44] 16:05:27 Build timed out (after 30 minutes). Marking the build as failed. [17:45:47] 16:05:27 Build was aborted [17:45:47] stupid timeout [17:45:50] 16:05:27 No emails were triggered. [17:46:12] https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/45844/console [17:46:48] <^d> ya [18:09:09] chrismcmahon: feel like making your eyeballs bleed? https://gerrit.wikimedia.org/r/#/c/192484/ [18:10:14] it's only ~ 440 lines of changes from double to single quotes :) [18:11:04] the last fix to make rubocop 100% happy, then we can cut 1.0.0 stable [18:15:51] marxarelli: net dropped. sure, I'll look... [18:16:47] marxarelli: actually I think we should merge it and see if anything breaks, too much eyebleed [18:17:22] (03CR) 10Cmcmahon: [C: 032] Corrected Style/StringLiterals RuboCop offenses [selenium] - 10https://gerrit.wikimedia.org/r/192484 (owner: 10Dduvall) [18:17:33] chrismcm_: i'm cool with that. the specs pass and i've tested it functionally as well [18:17:45] (03Merged) 10jenkins-bot: Corrected Style/StringLiterals RuboCop offenses [selenium] - 10https://gerrit.wikimedia.org/r/192484 (owner: 10Dduvall) [18:17:52] chrismcm_: it'd be nice to have a handful of acceptance tests for our test framework :) [18:17:59] review doesn't help for certain classes of changes :) [18:21:09] brion, Krinkle|detached: the test suite is just maxing out CPU, it has like 10k+ assertions. And using a proper data provider is waaaaaaaaaay too slow, so it just loops an array. On HHVM it runs in under a minute :P [18:22:43] legoktm: yeah it runs hella iterations on tiny strings as i recall [18:22:48] it just seems to be inherently slow :D [18:22:49] is Phab search results display wacky for anyone else? not scrolling sort of thing... [18:23:06] (03CR) 10Dduvall: [C: 032] "Just in time for our 1.0.0 release!" [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [18:23:18] chrismcm_: are you scrolling while the mouse cursor is over the sidebar? [18:23:24] chrismcm_: it sticks when doing that for me in firefox [18:23:28] maybe so [18:23:29] move the mouse cursor to the right [18:23:30] (03Merged) 10jenkins-bot: Ignore HEADLESS and KEEP_BROWSER_OPEN for phantomjs [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [18:23:31] and it magically works [18:24:28] chrismcm_: yes [18:24:44] it's caused by the fancy scrollbars they implemented [18:24:52] (even if you don't get the fancy scrollbars) [18:25:04] yeah, I'd never noticed it before [18:26:02] bah [18:26:34] 10Continuous-Integration, 6Scrum-of-Scrums, 10Wikimedia-Fundraising-CiviCRM: CI for Civi: provision and run tests under Jenkins/Zuul - https://phabricator.wikimedia.org/T86103#1066522 (10Ejegg) [18:30:01] (03CR) 10Aaron Schulz: [C: 031] mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192829 (owner: 10Krinkle) [18:31:09] (03PS1) 10Dduvall: Releasing stable major version 1.0.0 [selenium] - 10https://gerrit.wikimedia.org/r/192841 [18:31:55] ^ chrismcm_ (woot!) [18:38:34] 7Blocked-on-RelEng, 6Release-Engineering, 10Fundraising Tech Backlog, 6Scrum-of-Scrums, and 2 others: Continuous integration - CiviCRM - https://phabricator.wikimedia.org/T78100#1066583 (10Cmcmahon) p:5Normal>3High [18:45:13] marxarelli: I just realized you didn't join SoS [18:45:21] I don't think [18:46:55] chrismcm_: oh scheisse. i blame it on the today's jackhammering [18:47:00] chrismcm_: worth joining now? [18:47:03] marxarelli: jump on in [18:47:07] thcipriani: maybe there is a labs puppet class that is needed in order to set it up to use the mwyaml backend? [18:47:53] (just noticed nobody responded to your question earlier, not sure if I have any useful knowledge to help though ) [18:48:33] is this the best channel for beta things or is it the labs channel [18:48:38] here [18:48:43] beta cluster that is [18:48:50] (not beta features) [18:49:21] yeah, that was my suspicion initially, a quick git grep in operations/puppet didn't turn up too much for 'mwyaml'. Plus, appropriate files _seem_ to be in place. [18:49:45] ok, great. could you link me to the docs maybe that explained the "merge on beta puppetmaster" [18:50:04] marxarelli: one ticket now blocked on us is https://phabricator.wikimedia.org/T78100 to be shepherded through the process to next week [18:50:20] i know i have been on it before but can't find now. i tried page "Beta" which redirects to Deployment_prep poject page [18:50:30] yeah, names are hard [18:50:53] https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/How_code_is_updated#Puppet_and_Salt [18:51:05] neve mind i found ... [18:51:11] arr. that :) thanks! [18:51:13] :) [18:51:38] chrismcm_: oh no... i thought i'd escaped drupal forever [18:52:13] marxarelli: I haven't even read it much, except to know that Antoine has to do something with it [18:52:29] baahaha [18:52:46] twentyafterfour: for instance `/var/lib/puppet/lib/hiera/backend/mwyaml_backend.rb` is definitely a file and I managed to make puppet agent give me, "Error: Could not retrieve catalog from remote server: Error 400 on SERVER:" so there's something. Who can I bug for info about this setup? [18:53:07] marxarelli: that ticket clearly needs more information [18:53:07] awight: i. hate. you. [18:53:11] :) [18:53:57] greg-g: so that is for testing things in opearations/puppet so you don't an ops to +2 it.. what i'm wondering is what if it's a change to mediawiki-config but it only touches a "-labs" file. Can i merge that in gerrit and then do nothing? [18:54:11] marxarelli: hey, I'm just looking out for the next generation of CiviCRM kids. [18:54:12] or should i put it on swat.. or [18:55:01] specifically changes in wmf-config/InitialiseSettings-labs.php [18:55:48] (also wonder if it makes sense to let devs like jdlrobson +2 stuff there themselves) [18:57:43] mutante: merge it then sync it to prod so there's not a dirty state, beta will pick it up automatically [18:59:18] i don't know what "sync it to prod" means [18:59:30] sync-file [18:59:44] so that the next person deploying won't see a diff they don't know about [18:59:46] oh, you mean deploying [18:59:51] sorry, yeah [19:00:08] it's a nop, just good practice [19:01:17] i'm not sure i want to start doing that (ops deploys stuff between scheduled deploys) [19:02:46] thcipriani: maybe ask in #wikimedia-operations [19:03:02] kk [19:05:21] 10Continuous-Integration, 10VisualEditor, 5Patch-For-Review, 5§ VisualEditor Q3 Blockers: Jenkins: Convert mwext qunit from grunt-contrib-qunit (PhantomJS) to grunt-karma (Chromium) - https://phabricator.wikimedia.org/T74063#1066749 (10Jdforrester-WMF) [19:05:22] mutante: tag it to swat then or get another deployer to do it [19:05:53] ok [19:06:04] <^d> Also if you merge without at least pulling to tin icinga will start complaining [19:06:17] <^d> Don't have to sync-file (although it's best as greg-g said, for completeness) [19:09:17] mutante: everyone does it (seriously: I encourage people to merge/deploy -labs only prod-nop changes whenver, even on Friday's at 5pm!) [19:10:13] greg-g: what about 6pm :p [19:10:23] I'm not online, so I can't encourage them :P [19:10:41] (not really at 5pm either, I'm on a bus then) [19:11:21] 10Beta-Cluster, 10VisualEditor, 10Wikimedia-Search: File search on beta labs returns results from production commons and beta commons. VE search results are in unexpected order also. - https://phabricator.wikimedia.org/T90650#1064118 (10Jdforrester-WMF) [19:15:45] (03PS1) 10Dduvall: Make mediawiki-vagrant rspec job voting [integration/config] - 10https://gerrit.wikimedia.org/r/192857 (https://phabricator.wikimedia.org/T76627) [19:16:48] (03PS1) 10Dduvall: Make mediawiki-vagrant cucumber job voting [integration/config] - 10https://gerrit.wikimedia.org/r/192860 (https://phabricator.wikimedia.org/T89489) [19:17:00] (03CR) 10jenkins-bot: [V: 04-1] Make mediawiki-vagrant rspec job voting [integration/config] - 10https://gerrit.wikimedia.org/r/192857 (https://phabricator.wikimedia.org/T76627) (owner: 10Dduvall) [19:18:17] (03CR) 10jenkins-bot: [V: 04-1] Make mediawiki-vagrant cucumber job voting [integration/config] - 10https://gerrit.wikimedia.org/r/192860 (https://phabricator.wikimedia.org/T89489) (owner: 10Dduvall) [19:33:06] kown?: "Failed deployment on the EQIAD beta cluster :-/ Please contact a member of the beta project to fixup the working directory on the destination server." [19:35:03] chrismcmalunch: ^^ [19:37:56] PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [19:40:31] will look [19:40:41] http://thedailywtf.com/articles/classic-wtf-the-defect-black-market [19:41:02] jzerebecki, where did you read that? [19:41:56] Krenair: https://gerrit.wikimedia.org/r/192502 https://gerrit.wikimedia.org/r/192561 [19:42:47] probably happened on more config changes, those were the only ones i was cc-ed on the last few days [19:43:50] hm [19:44:00] PROBLEM - Puppet failure on deployment-cache-mobile03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [19:44:33] krenair@bastion1:~$ ssh deployment-bastion [19:44:33] ssh_exchange_identification: Connection closed by remote host [19:44:41] what [19:45:23] 10Deployment-Systems, 6Release-Engineering, 10Wikimedia-Hackathon-2015: Code Deploy Dashboard - https://phabricator.wikimedia.org/T280#1067018 (10Legoktm) [19:45:33] PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [19:47:49] [2408883.916166] INFO: task syslog-ng:1026 blocked for more than 120 seconds. [19:47:49] [2408883.917064] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [19:48:19] * Reedy reboots it [19:48:21] is that from "get console output" on wikitech? [19:48:41] indeed [19:52:09] (03PS2) 10Legoktm: Add mediawiki/tools/codesniffer jobs for non-whitelisted users [integration/config] - 10https://gerrit.wikimedia.org/r/192267 [19:53:39] PROBLEM - Host deployment-bastion is DOWN: CRITICAL - Host Unreachable (10.68.16.58) [19:54:19] PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [19:55:46] PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [19:55:51] and now i'm glad i did not touch deployment-prep [19:56:26] PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0] [19:59:01] Yippee, build fixed! [19:59:01] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #506: FIXED in 25 min: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/506/ [20:08:02] PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [20:08:12] (03CR) 10Legoktm: [C: 032] Add mediawiki/tools/codesniffer jobs for non-whitelisted users [integration/config] - 10https://gerrit.wikimedia.org/r/192267 (owner: 10Legoktm) [20:10:30] PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [20:10:42] PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 75.00% of data above the critical threshold [0.0] [20:10:52] PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [20:11:04] PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 28.57% of data above the critical threshold [0.0] [20:12:43] Reedy, doesn't look much better.. [20:13:03] It hasn't rebooted it seems [20:13:11] earlier [20:13:12] ssh_exchange_identification: Connection closed by remote host [20:13:16] now [20:13:16] ssh: connect to host deployment-bastion port 22: No route to host [20:13:41] 10Beta-Cluster, 6MediaWiki-Core-Team, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1067273 (10Dzahn) Or we might switch them over to Debian jessie right away? What do other ops think in this case? still trusty or jessie already? [20:14:07] Do we know how long it can be expected to be restarting? [20:14:35] PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [20:14:41] (03Merged) 10jenkins-bot: Add mediawiki/tools/codesniffer jobs for non-whitelisted users [integration/config] - 10https://gerrit.wikimedia.org/r/192267 (owner: 10Legoktm) [20:15:46] 10Beta-Cluster, 6MediaWiki-Core-Team, 6operations, 7HHVM: Convert work machines (tin, terbium) to Trusty and hhvm usage - https://phabricator.wikimedia.org/T87036#1067281 (10Dzahn) Resolving this should also prevent reverts like https://gerrit.wikimedia.org/r/#/c/192866/ [20:17:10] PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [20:18:03] I think it's hung tbh [20:18:36] !log deployed https://gerrit.wikimedia.org/r/192267 [20:18:40] Logged the message, Master [20:19:23] Reedy, would a labs admin be able to fix his? [20:19:25] this* [20:19:33] You'd hope so [20:19:34] (03PS2) 10Legoktm: Add '{name}-composer-security' template and use it for mediawiki/vendor [integration/config] - 10https://gerrit.wikimedia.org/r/192564 (https://phabricator.wikimedia.org/T74193) [20:19:37] If they can't [20:23:29] if they can't... we'd have to rebuild deployment-bastion? [20:25:58] seems a bit extreme [20:26:11] I suspect the reboot via the web panel is just a soft restart [20:26:23] they may be able to hard restart it? [20:26:24] the machine needs hard shutting down/halting (ie power switch off) and rebooting [20:26:27] right [20:28:02] (03CR) 10Legoktm: [C: 032] Add '{name}-composer-security' template and use it for mediawiki/vendor [integration/config] - 10https://gerrit.wikimedia.org/r/192564 (https://phabricator.wikimedia.org/T74193) (owner: 10Legoktm) [20:34:28] (03Merged) 10jenkins-bot: Add '{name}-composer-security' template and use it for mediawiki/vendor [integration/config] - 10https://gerrit.wikimedia.org/r/192564 (https://phabricator.wikimedia.org/T74193) (owner: 10Legoktm) [20:38:20] !log deployed https://gerrit.wikimedia.org/r/192564 [20:38:24] Logged the message, Master [20:45:20] RECOVERY - Host deployment-bastion is UP: PING OK - Packet loss = 0%, RTA = 3.64 ms [20:45:34] 10Deployment-Systems, 10Librarization, 6MediaWiki-Core-Team, 6Security, 5Patch-For-Review: Have a check for reported security issues in dependencies - https://phabricator.wikimedia.org/T74193#1067388 (10Legoktm) Deployed it as an experimental job for now: * https://integration.wikimedia.org/ci/job/mediaw... [20:48:32] RECOVERY - SSH on deployment-bastion is OK: SSH OK - OpenSSH_5.9p1 Debian-5ubuntu1.4 (protocol 2.0) [20:49:08] jzerebecki, ok, stuff should be less broken now. [20:49:11] hopefully [20:50:10] PROBLEM - Puppet staleness on deployment-bastion is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [43200.0] [20:51:35] PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0] [20:54:44] 10Quality-Assurance, 10MediaWiki-extensions-UploadWizard, 6Multimedia, 10Multimedia-Sprint-2015-02-25, 5Patch-For-Review: UploadWizard browser test for chunked upload - https://phabricator.wikimedia.org/T89289#1067417 (10Gilles) [20:55:46] RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0] [20:58:26] (03PS2) 10Legoktm: Create oojs-ui-composer job, get rid of legacy oojs-ui-phpcs-HEAD [integration/config] - 10https://gerrit.wikimedia.org/r/192772 [20:59:38] RECOVERY - Puppet failure on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0] [21:00:10] RECOVERY - Puppet staleness on deployment-bastion is OK: OK: Less than 1.00% above the threshold [3600.0] [21:00:12] Yippee, build fixed! [21:00:12] Project beta-code-update-eqiad build #45846: FIXED in 11 min: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/45846/ [21:01:26] RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0] [21:02:10] RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0] [21:05:47] RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0] [21:12:59] RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0] [21:15:30] RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0] [21:15:43] RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0] [21:16:05] RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [21:21:33] RECOVERY - Puppet failure on deployment-bastion is OK: OK: Less than 1.00% above the threshold [0.0] [21:27:22] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #490: FAILURE in 31 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/490/ [21:34:56] 10Continuous-Integration, 6Mobile-Web, 5Patch-For-Review, 7Technical-Debt: Publish MobileFrontend JS Documentation - https://phabricator.wikimedia.org/T74794#1067677 (10Jdlrobson) In terms of the css I'm not sure what to do about that... apart from removing this from the `make docs` command. We can't easil... [21:35:54] 10Continuous-Integration, 6Mobile-Web, 5Patch-For-Review, 7Technical-Debt: Publish MobileFrontend JS Documentation - https://phabricator.wikimedia.org/T74794#1067685 (10Jdlrobson) (If we set MEDIAWIKI_LOAD_URL = http://en.m.wikipedia.beta.wmflabs.org/w/load.php this will work - is that an option or does th... [21:42:52] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #489: FAILURE in 26 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/489/ [21:54:20] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1067769 (10mmodell) 3NEW [21:54:59] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1067777 (10mmodell) [21:55:00] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1067776 (10mmodell) [21:55:12] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1067769 (10mmodell) probably related to T76061 [22:09:14] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1067806 (10bd808) Expected behavior with the process I dreamed up for deleting branches: 6Release-Engineering, 6Engineering-Community, 6Team-Practices, 10Wikimedia-Hackathon-2015, 3ECT-February-2015: RelEng team offsite - May 2015 - Pre Wikimedia Hackathon - https://phabricator.wikimedia.org/T89036#1067872 (10Awjrichards) @KLans_WMF will be TPG'ing this - please be sure to include both her a... [22:45:51] legoktm: Ah, can we pin that composer test to hhvm them? [22:46:01] then* [22:47:49] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1067971 (10mmodell) when I do the dsh command I get a permission denied message. [22:49:10] Krinkle: we should probably run it on both 5.3 + hhvm until we drop 5.3 support. I can work on getting it running on hhvm though [22:49:32] legoktm: You can do a -zend variant on the gate [22:49:51] e.g. {}-composer on hhvm, via test pipeline and then on gate {}-composer and {}-composer-zend [22:50:04] so that it's only slow on gate for compat, but main test faster [22:50:10] so today's deploy took 3:45, partially because I was explaining it to thcipriani but mostly because the process is painful [22:50:21] we do the same for mw-core phpunit (or going to soon, not sure what progress is there) [22:50:25] the biggest part is dealing with csteipp's security patches [22:50:40] yeah, knowing what needs to be applied is icky [22:50:42] (03CR) 10Krinkle: [C: 032] mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192829 (owner: 10Krinkle) [22:51:01] Shouldn't csteipp be handling those...? [22:51:03] deploying is a very manual process [22:51:09] Krenair: Why? [22:51:09] TIL [22:51:11] (03Merged) 10jenkins-bot: mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192829 (owner: 10Krinkle) [22:51:15] Not auto-rebased? [22:51:22] (patches) [22:51:27] thanks to thcipriani for tagging along in a hangout, it was good to explain myself along the way I think that helped both of us learn the process a bit better [22:51:55] Reedy, because they're basically his private live unmerged changes? [22:51:58] Krenair: There's no real reason he should have to do the deploys to subsequent branches [22:51:58] Krinkle: the patches have to be ported across because each new branch is not based on the previous [22:52:04] twentyafterfour: definitely more helpful than just going through the docs—thanks for slowing it down for me. [22:52:32] Krenair: In most cases, they should have been CR'd [22:52:38] thcipriani: sadly it wasn't that much slowed down... [22:52:40] They're just not committed as they're not released [22:53:46] I really don't like maintaining live unmerged changes on tin that way. it's disgusting [22:54:26] especially since it's maintained in a shared directory and a lot of people could step on eachothers toes, or silently modify things in some unnoticed way that ends up getting deployed [22:54:28] twentyafterfour: Feel free to suggest a different process. The current amount is rare, due to the recent assessment, but a few is normal. [22:55:18] csteipp: yeah I'm working on a better process in https://phabricator.wikimedia.org/T89945 [22:55:20] _joe_ had some ideas for reapplying and such. But that doesn't fix the "live hack" type of thing [22:55:43] Reedy: I've written scripts, and today we tested them, to port patches from branch to branch [22:56:02] :) [22:56:17] works on main repo and extensions .. but it's still fiddly and error prone because you have to tell it precisely which range of commits to port and that is not foolproof [22:56:30] Step in the right direction [22:56:34] yeah [22:57:59] Krinkle: on the trusty instances, is `php` an alias for hhvm like it is in production? or do we have to use `hhvm` explicitly? [22:58:29] it just feels weird to me that I could change those security patches to make them insecure and they might never get noticed by anyone because the commit message would be the same... since they get ported from branch to branch, the hash changes with each move and that's expected rather than being a red-flag that something might be tampered with [22:58:46] legoktm: I think it's an alias [22:58:58] couldn't we attach diff files to secret phab bugs? [22:59:05] legoktm: https://integration.wikimedia.org/ci/computer/integration-slave1008/script [22:59:12] mutante: we do [22:59:18] > println "php --version".execute().text [22:59:21] HipHop VM 3.3.1 (rel) [22:59:48] oh that's awesome [23:00:04] both the interface and that it's an alias :P [23:01:40] Tarballs are built with patches out of phab. The patches themselves are owned by me and 640 on the server. But yeah, you could write anything, commit it with the same summary. But we trust our deployers. [23:02:30] Heck, you could just make uncommited changes, gitignore them, and deploy anything you want also. Much harder to track :) [23:04:06] you could also ssh to the appservers and use vi with even touching git [23:04:10] without [23:06:02] any ideas what's wrong in https://integration.wikimedia.org/ci/job/mediawiki-extensions-hhvm/4587/console ? [23:07:19] cannot access ‘/mnt/home/jenkins-deploy/tmpfs/mediawiki-extensions-hhvm/wikicache.*’: No such file or directory ? [23:08:02] yep [23:14:27] I agree we should trust our people, it just feels icky that's all [23:18:20] (03PS1) 10Legoktm: Run {name}-composer jobs on HHVM too [integration/config] - 10https://gerrit.wikimedia.org/r/192980 [23:19:00] Krinkle: ^ [23:22:14] mutante: MaxSem: On it. [23:23:10] (03PS1) 10Krinkle: Revert "mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192983 [23:23:16] (03CR) 10Krinkle: [C: 032] Revert "mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192983 (owner: 10Krinkle) [23:23:45] (03Merged) 10jenkins-bot: Revert "mw-install-sqlite: Make MW_DB_PATH/wikicache* writable by Apache" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192983 (owner: 10Krinkle) [23:25:39] (03PS1) 10Krinkle: mw-install-sqlite: Simplify chmod to just match all files [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192984 [23:25:57] (03CR) 10Krinkle: [C: 032] mw-install-sqlite: Simplify chmod to just match all files [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192984 (owner: 10Krinkle) [23:26:35] (03Merged) 10jenkins-bot: mw-install-sqlite: Simplify chmod to just match all files [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192984 (owner: 10Krinkle) [23:36:24] 10Deployment-Systems, 6Release-Engineering: cannot delete non-empty directory: php-1.25wmf14/cache/l10n - https://phabricator.wikimedia.org/T90798#1068208 (10mmodell) 5Open>3Invalid a:3mmodell [23:41:51] 10Continuous-Integration, 7Upstream: Fails npm build failure "File exists: ../esprima/bin/esparse.js" - https://phabricator.wikimedia.org/T90816#1068235 (10Krinkle) 3NEW [23:42:03] 10Continuous-Integration, 7Upstream: Fails npm build failure "File exists: ../esprima/bin/esparse.js" - https://phabricator.wikimedia.org/T90816#1068242 (10Krinkle) [23:42:21] 6Release-Engineering, 10Flow: QUnit jobs for Flow failing with various internal "No such file or directory" - https://phabricator.wikimedia.org/T90817#1068244 (10Mattflaschen) 3NEW [23:42:34] 6Release-Engineering, 10Flow: QUnit jobs for Flow failing with various internal "No such file or directory" - https://phabricator.wikimedia.org/T90817#1068251 (10Mattflaschen) p:5Triage>3High [23:46:32] 10Continuous-Integration, 6Release-Engineering: Jobs using sqlite failing with various internal "No such file or directory" - https://phabricator.wikimedia.org/T90817#1068257 (10Krinkle) a:3Krinkle [23:46:55] 10Continuous-Integration, 6Release-Engineering: Jobs using sqlite failing with various internal "No such file or directory" - https://phabricator.wikimedia.org/T90817#1068244 (10Krinkle) Affects mediawiki-core/extensions, phpunit/qunit, anything using sqlite. Working on it. [23:48:21] (03PS1) 10Krinkle: mw-install-sqlite: Unquote string for wildcard expansion [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192997 (https://phabricator.wikimedia.org/T90817) [23:49:01] (03CR) 10Krinkle: [C: 032] mw-install-sqlite: Unquote string for wildcard expansion [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192997 (https://phabricator.wikimedia.org/T90817) (owner: 10Krinkle) [23:49:30] (03Merged) 10jenkins-bot: mw-install-sqlite: Unquote string for wildcard expansion [integration/jenkins] - 10https://gerrit.wikimedia.org/r/192997 (https://phabricator.wikimedia.org/T90817) (owner: 10Krinkle) [23:54:29] !log Zuul queue is growing. Nothing is added to its dashboard. Jenkins executers all idle. Gearman deadlock? [23:54:34] Logged the message, Master [23:55:21] !log Re-established Jenkins-Gearman connection [23:55:24] Logged the message, Master [23:58:12] 10Continuous-Integration, 6Release-Engineering, 5Patch-For-Review: Jobs using sqlite failing with various internal "No such file or directory" - https://phabricator.wikimedia.org/T90817#1068276 (10Mattflaschen) 5Open>3Resolved