Node.js-style --require CLI argument

[jokeyrhyme]

I figured I'd have an attempt at being able to pass the --require through to the Node executable used for the child process (#229).

My test project uses import and export, so doesn't execute in Node v5 without babel/register. I can manually confirm using npm link that the changes I have so far function as expected.

Besides general comments (very much appreciated), I'm a little bit at a loss as where to start with the tests. Do I need to confirm (with tests) that the ava --require moduleId works? Or is it sufficient to use mocks and confirm that arguments are being passed through as expected?

[jamestalmage]

Do I need to confirm (with tests) that the ava --require moduleId works?

That would be my approach, does not need to be fancy.

// test/fixture/install-global.js

global.foo = 'bar'

// test/fixture/validate-installed-global.js

import test from '../../'

test(t => t.is(global.foo, 'bar'));

Then use the API to call validate-... using the require hook you've just implemented.

Overall looks pretty good.

[jamestalmage]

Just do

if (this.require) {
 // ...
}

You have already guaranteed it is an array of at least length one above.

[sindresorhus]

Not sure we ever got an agreement on this:

Would exposing the Node.js binary --require flag while disabling our Babel hook solve your problem

• Consider allowing custom test preprocessors #229 (comment)

@jamestalmage ^

[jamestalmage]

I don't know if our Babel hook should be disabled automatically. I still have not decided the best way to make it all work.

My current thinking is using something along the lines of #297, but using a better version of istanbul-lib-hook. https://github.com/jamestalmage/transform-chain would be part of it. Combine transform-chain with a way of detecting when hooks are installed, similar to how @novemberborn did it here in nyc.

[jokeyrhyme]

Unless I somehow accomplished this accidentally, I did not attempt to disable ava's babel hook. My only purpose with this PR was the pass through the --require.

Also, I hope to get time to address the PR feedback over the next few days. I'm sorry for not being very responsive.

[jamestalmage]

Unless I somehow accomplished this accidentally, I did not attempt to disable ava's babel hook

Understood, we are still deciding if / when / how to do that. Once a decision on that gets made we will merge this, and augment it with whatever behavior we finally decide on.

I'm sorry for not being very responsive.

That is not a problem. We still have to answer a few questions, so we are in no rush to merge.

[jokeyrhyme]

I still haven't pushed tests yet. They don't seem quite as trivial as I'd hoped, even with @jamestalmage's suggestions. Soon...

[jokeyrhyme]

Really stuck with the testing part. I'm getting similar errors when I try to use api.js instead of cli.js.

I've tried a few simple approaches, and none of them work. But this functionality doesn't seem complicated enough to warrant a complicated test. Is there an obvious flaw in this approach? Or do I need to take this up a notch?

[jokeyrhyme]

Thanks for the hint, @jamestalmage. At least now, the errors are all the same one:

Test files must be run with the AVA CLI

[jokeyrhyme]

I could use mockery and just confirm that Node.js is called with the expected parameters, without actually calling Node.js? /shrug

[jamestalmage]

Well, I'm having a hard time getting this to work on travis. Test #4 seems to work on OSX. Note that the --require flag does not work on Node <4

[jamestalmage]

@sindresorhus @vdemedes - either of you develop on linux? Any advice on how he should fix this?

[jokeyrhyme]

So I should detect the version of Node in the tests and only run them for >=4.0?
And there's still something funny going on with the paths. The fact that it's so hard to test properly makes me wonder if I've taken a good approach in the first place. :S

[jamestalmage]

I don't know. Maybe it would just be better to manually implement the same semantics as the --require flag ourselves. You could just do require(somePath), for every --require== flag inside babel.js. Might end up being more reliable.

[sindresorhus]

either of you develop on linux? Any advice on how he should fix this?

I have Linux in a VM, but I almost never use it, so pretty noobish with it.

Maybe it would just be better to manually implement the same semantics as the --require flag ourselves.

👍 This is where the --require magic happens: https://github.com/nodejs/node/blob/04b1a2f7564d70dfdab838d2830e04af34b97fe6/lib/module.js#L469-L488 Which is pretty much just require(resolveCwd(cli.flags.require)); (we already depend on resolve-cwd)

[jokeyrhyme]

Okay, well I'll have another crack at it over the next few days. I'll try implementing --require internally so that it works regardless of the Node.js version.

[sindresorhus]

Just use arrify on line 69 and drop this whole thing.

[sindresorhus]

requires.forEach(function (el) {
  require(resolveCwd(el));
));

[sindresorhus]

semicolon

[jokeyrhyme]

Weird. I love my semi-colons, too. I wonder why ESLint didn't bust my chops about this? :)

[sindresorhus]

#296 (comment)

[jokeyrhyme]

Also, I added 4 tests to check different argument orders, but we should probably just pick the one we like and get rid of the rest?

[sindresorhus]

I think opts.require is already guaranteed to be an array, so this check is moot.

https://github.com/sindresorhus/ava/pull/296/files#diff-2cce40143051e25f811b56c79d619bf5R73

[jokeyrhyme]

Nope. In the API tests, new Api() is called without the second parameter, which means it ends up being an empty Object, without a "require" property. Should I change the fallback default value in api.js to include an empty "require" array? e.g. this.options = options || { require: [] }; ?

[sindresorhus]

Oh, never mind then. I read it wrong.

[jokeyrhyme]

Okay, so in total I have 1 api test and 1 cli test. I left the cli test in there to stress the end-to-end process a bit. Of course, if you think this is overkill then I'm happy to drop the cli test completely.

[sindresorhus]

Don't remove

[sindresorhus]

I left the cli test in there to stress the end-to-end process a bit

I think it's overkill to CLI test all options, but I'm ok with this if you rework it into being a generic CLI option test, meaning change the test description. We should have one CLI test making sure options are passed correctly.

[jokeyrhyme]

I gave it a whirl, but it seemed infeasible to mock the boundary from cli.js to the others, and my other approach to CLI argument tests seemed unnecessarily complex (and still didn't work).

If we still feel strongly about having CLI tests, we can have a separate Issue / PR.

[sindresorhus]

LGTM

[vadimdemedes]

LGTM

[sindresorhus]

Thank you @jokeyrhyme! :) 💥👌

Keep the quality contributions coming. We really appreciate your help.

[vadimdemedes]

Thank you for your work, @jokeyrhyme!

[ariporad]

Hey, @sindresorhus, @vdemedes, @jamestalmage: Any idea when this might ship? I'd really prefer to not depend directly on master. Thanks!

[sindresorhus]

@ariporad We're planning a new release when #326 lands.

[ariporad]

Thanks @sindresorhus!

[sindresorhus]

@ariporad Actually, I did a release now. https://github.com/sindresorhus/ava/releases/tag/v0.8.0

[ariporad]

Yay! Thanks!

[jamestalmage]

Any idea why Mocha would be outperforming AVA by such a huge factor, in that case? Are there existing perf issues that you're aware of? (Sorry, not directly related to this PR)

@billyjanitsch

There have been major performance improvements since this was discussed. If possible, could you give AVA a try again and tell us how we're doing?

[billyjanitsch]

@jamestalmage

That's great to hear!

Currently this is the biggest migration blocker for us: it's important that our tests run in the same transpilation environment as our lib code (we now use several custom Babel transforms). Our test suite is large enough (>100 files) that I'd rather not rewrite it in AVA until I know this will be possible.

That said, I'd be happy to convert a couple of our tests to see what the perf looks like relative to Mocha.

[novemberborn]

@billyjanitsch

Currently this is the biggest migration blocker for us: it's important that our tests run in the same transpilation environment as our lib code (we now use several custom Babel transforms).

My approach to this is to transpile using babel-cli and run the tests on the transpiled code as it sits on the disk. That way I know that I'm testing the code that I'll end up publishing.

See identifierfy for an example.

[jamestalmage]

Our test suite is large enough (>100 files) ...

Wow! I would love to see a project that big converted. Is it open source? What is your current test setup? (mocha with chai for assertions, etc)

You may also want to wait for customizable assertions, so we can create one that matches your current assertion api exactly.

[billyjanitsch]

@novemberborn

My approach to this is to transpile using babel-cli and run the tests on the transpiled code as it sits on the disk.

I think you're talking about transpiling lib code (imported by tests). This PR provides an easier way to do that: --require 'babel-core/register'.

I'm talking about having test code transpiled in the same way that your lib code is, rather than letting AVA transpile it. See the comment that I linked to for more info.

@jamestalmage

Is it open source? What is your current test setup?

Unfortunately not. I work on Kensho's front-end and we're currently using Mocha/Chai (assert style). I'd like to move to a flat Tape/AVA-style interface.

We do have an open source React chart lib -- also using Mocha but faking a Tape-ish API -- that we might be able to convert as well.

You may also want to wait for customizable assertions, so we can create one that matches your current assertion api exactly.

I'm actually not a huge fan of our current assertions, so I don't mind rewriting them when the time comes. AVA's defaults seem great.

[jamestalmage]

@billyjanitsch #424 (comment)