[14:18:28] How can I use a more recent version of PHPUnit when running tests? [14:18:36] I don't see where the copy is that MediaWiki is using. [14:18:49] It's not the one that 'which phpunit' outputs. [14:22:07] I think mediawiki installs it's own version via composer. [14:25:32] Yeah, it does. Version is defined in composer.json and installed into vendor/phpunit. [14:25:54] I don't know whether the tests will work with a newer version of PHPUnit though. [14:30:06] This one is very buggy AFAICT, and I want to at least try with a newer one. [14:30:19] It's really the code coverage that I have a problem with, not PHPUnit itself. [14:31:14] But I think they go together. [14:34:45] There we go, I hacked composer.json and it worked nicely. [14:34:57] So far it seems to work but not fix my bugs. [14:35:18] You can create a composer.local.json (not sure about the name) iirc to avoid hacking composer.json [14:35:28] Oh, nice. [14:48:30] So PHPUnit 6.5 seems to work but doesn't solve my problems, and PHPUnit 7.0 doesn't work because there are conflicts with declarations that use return types. [14:48:32] Oh well. [14:48:55] That was a fairly large waste of time. [14:59:34] anomie: For the record, this is the last window in my contract where we'll both be around at the same time. I'm not around today after 7 PM UTC, nor tomorrow at all, and on Sunday I'm working but I assume you're not normally around. [15:00:04] So plan accordingly, if it makes a difference to anything. [15:03:14] AryehGregor: I don't know about different versions of PHPUnit, but using PHP 7 to run the tests seems to help. [15:03:26] Also I've been using `phpdbg -qrr phpunit.php` instead of `php phpunit.php` with the xdebug extension lately, although I think the difference there is more speed than accuracy. [15:03:44] I'm using PHP 7.0 anyway, because that's what happens to be installed on my machine. [15:04:15] AryehGregor: My plan was already "review Aryeh's patches", so I don't think it changes anything ;) [15:31:37] legoktm: OK, want to land T185753 again? :-) [15:31:37] T185753: MediaWiki should default to using RemexHtml for tidy - https://phabricator.wikimedia.org/T185753 [16:13:24] AryehGregor: PHPUnit 6 is mostly working, my goal is actually to merge https://gerrit.wikimedia.org/r/#/c/394851/ today [16:14:10] anomie: last I heard from the creator of phpunit, phpdbg is less accurate than xdebug in some cases. Unsure if those issues have been fixed though [16:26:39] James_F: yes, I think so [16:47:03] I "fixed" the KafkaHandler issue, but not really [16:56:35] Ha. [16:58:45] anomie, AryehGregor: are you aware of "ApiMainTest::testUselang: Trying to configure method "getRawIP" which cannot be configured because it does not exist, has not been specified, is final, or is static" [16:58:52] it's just a warning though [16:59:43] legoktm: Nope. Any details on how to reproduce? [16:59:53] use PHPUnit 6, see https://integration.wikimedia.org/ci/job/mediawiki-phpunit-php70-jessie/444/console [17:00:00] it's coming from getNonInternalApiMain [17:00:08] I see the problem. [17:01:07] ok, will you submit a patch? I just found the issue too [17:01:13] legoktm: Is there a bug? If not, don't bother filing one [17:01:18] nope [17:01:34] I'll submit a patch as soon as I rebase. [17:01:46] thanks [17:01:55] https://gerrit.wikimedia.org/r/425864 [17:02:40] dunno if you want to look at the AuthManagerTest warnings that are also in that log [17:02:58] Sure [17:29:13] legoktm: https://gerrit.wikimedia.org/r/425868 [18:15:58] anomie: If I want to create a custom message in a test so that it's not isDisabled(), how should I do it? editPage( 'MediaWiki:Foo' ) doesn't seem to work. [18:16:06] (Perhaps I need to clear some kind of cache?) [19:18:32] hmm, I didn't realize that warnings will also cause PHPUnit to fail [19:26:27] James_F: thankfully, I scripted it [19:26:37] legoktm: Even so. :-) [19:26:58] legoktm: Mostly an "ouch" about future work when we scrap the phpunit4 back-compat., if not before. [19:27:31] I think the main issue is that most extensions just extend MediaWikiTestCase by default and Wikibase uses PHPUnit\Framework\TestCase [19:28:23] Yeah, a little bit of our Not Invented Here syndrome being positive for once. [19:30:58] ok down from 1100+ test failures to 150 now [19:31:26] legoktm: The qunit failure is just an async temporary thing, it'll pass next time probably. [19:32:00] legoktm: Focus on the 15 inside core first? https://integration.wikimedia.org/ci/job/mediawiki-phpunit-php70-jessie/455/console [20:27:14] James_F: ok, I put up patches for the rest of the core failures [20:29:51] Neat. Will review. [20:30:05] Was about to look at the ones for Notifications but I'll merge yours first. :-) [20:36:16] legoktm: There's still MediaWikiServicesTest::testDisableStorageBackend :-( [20:37:10] I'm not sure if risky tests will cause it to fail...I guess we'll see [20:37:35] I think we could just add a $this->assertTrue( true ) // No exception thrown [20:38:00] legoktm: It V-1'ed with FAILURE in the Jenkins output, so… ;-) [20:38:11] Yeah, that works for now. [20:42:59] James_F: I'm confused, https://integration.wikimedia.org/ci/job/mediawiki-phpunit-php70-jessie/459/console says "OK, but incomplete, skipped, or risky tests!" but at the bottom [20:43:04] 20:34:16 [xUnit] [ERROR] - The result file '/home/jenkins/workspace/mediawiki-phpunit-php70-jessie/log/junit-mw-phpunit.xml' for the metric 'PHPUnit' is not valid. The result file has been skipped. [20:43:04] 20:34:16 [xUnit] [INFO] - Failing BUILD because 'set build failed if errors' option is activated. [20:44:57] I think jenkins is barfing on the junit file [20:47:29] Hmm. I think we've intentionally failed on warn before? But maybe that's wrong. [20:49:08] https://issues.jenkins-ci.org/browse/JENKINS-42715 [20:50:49] legoktm: So it's been this way for a while? [20:51:21] I'm guessing whatever schema jenkins is using to validate the junit.xml file isn't compatible with something that PHPUnit 6 must have changed [20:51:28] I'll debug further once I'm home in a few hours [20:52:22] Well, "Invalid content was found starting with element 'skipped'. One of '{error, failure, system-err, system-out}' is expected." is pretty clear, yeah. [20:54:36] James_F: where do you see that? [20:54:44] legoktm: James_F the Jenkins JUnit plugin is old and is used for the Java Junit format [20:54:49] phpunit is slightly different [20:54:57] and yeah I guess phpunit 6 came up with its own [20:55:16] also I dont think JUnit has any spec / standard to describe it [20:55:17] legoktm: In the first comment of the Jenkins issue you linked. [20:55:24] * James_F nods. [20:55:34] luckily it is opensource (but in java) [20:55:48] I think there's an .xsd it uses for validation [20:56:04] We could extent the .xsd? [20:56:09] there is also a xunit plugin, which supports several test result files [20:56:24] one of them is "phpunit (junit)" but iirc it internally reuses the Junit plugin logic [22:51:36] James_F: did you already start working on the Echo failures or not yet? [22:52:03] legoktm: Sorry, no. [22:52:19] no worries, just wanted to make sure I wouldn't be duplicating :) [22:52:50] Sure. Also I worry about the 150+ extensions in prod that aren't in mwgate so we won't know are broken until after we merge… [22:53:15] Well that was why I asked people to test their extensions...but I don't think that happened. [22:53:21] Nope. [22:53:33] I mean, I tested VE with it. But then those are in mwgate. :-) [22:53:39] I was going to time the codesniffer + libraryupgrader run after we land PHPUnit 6 and Remex by default and then we'll just see what the fallout is [22:53:58] That'd be great. Speaking of, merge Remex? :-) [22:54:45] Do we have a plan for getting jenkin's xunit to recognise qunit 6's "wrong" values? Should we have a blocking task? [23:31:01] Filed as T192120. [23:31:02] T192120: Work around Jenkins's xunit validator being incompatible with PHPUnit 6's extra output in junit.xml - https://phabricator.wikimedia.org/T192120 [23:32:43] 46 failures left... [23:35:14] I tried diffing the phpunit 4 junit and phpunit 6 junit and the diff is 43k lines [23:35:35] Ouch. [23:46:27] >>> schema=etree.XMLSchema(etree.parse(open('xunit-plugin/src/main/resources/org/jenkinsci/plugins/xunit/types/phpunit-2.0.xsd'))) [23:46:27] >>> schema.validate(etree.parse(open('phpunit4.xml'))) [23:46:27] True [23:46:27] >>> schema.validate(etree.parse(open('phpunit6.xml'))) [23:46:27] False [23:48:18] question: if I use wfMessage to generate messages including arbitrary user data, which output I should use? ->text()? I am not sure what other modes even do... [23:48:56] SMalyshev: https://www.mediawiki.org/wiki/Manual:Messages_API#Which_output_mode_to_use :) [23:49:07] yeah I read it, still not clear :) [23:49:31] I am not putting the result anywhere, really, I am sending it back to the API which decides where to put it [23:49:41] it may end up in html eventually but way upstream [23:49:49] at some level it does need to get escaped [23:50:17] ->text() means it's not escaped, ->escaped() is escaped, ->parse() process all wikitext and does escaping as part of that [23:50:19] I suspect it would be, since this API already displays data from the db anyway [23:50:32] so in this case, text() is the proper way? [23:50:59] Yeah, as long as whatever uses the API knows that it needs to do the escaping [23:51:08] ok, got it, thanks