Refactor how test files are told to run their tests, and more #1693
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove indirection by inlining code. It's not the prettiest without async-await but it's easier to follow. Remove unnecessary auto-binding of methods. There's no need to "get" blank results either when running a test files fails. Handle timeouts within a test run, without touching the `runStatus` object. Restart the timeout timer whenever a new "fork" is created. This gives new test files the chance to start running tests. Don't start the timer *before* running any tests. Fixes #1377. Clean up "fork" instances when test files exit. Set up a new precompiler on every test run. Don't store it on the `Api` instance. If caching is enabled, fix the caching directory relative to the project directory, without looking it up again. Compile all test and helpers files before running any test. This doesn't improve time-to-first-test but it does mean that test files can depend on each other. AVA will correctly load the precompiled file. No tests are run if any file fails to compile. Pass the real path to the precompiler. Compute `execArgv` when a test file is run, not all at once for all files.
Don't run new test files after: * A timeout has occurred * A test has failed * Running the test file itself caused an error Refs #1158.
This module is necessary since there may be circular dependencies originating in test-worker.js. Make setting and getting the options explicit, prevent options from being replaced and require them to be set. This will help catch any future issues caused by changing the dependency tree.
Replace lib/globals.js by a module which focuses on reexporting Node.js' timers and Date.now(). Remove workarounds from profile.js, presuming they're outdated and don't apply to methods from Node.js' `timers` module.
…ters * Make error type more readable * Always include originating test file * Indent error summary, allowing for multi-line summaries * Correctly handle non-errors in verbose reporter
SyntaxError stacks may begin with the offending code. Include all stack lines up to and including one that begins with SyntaxError.
This focuses lib/main.js on creating and exporting the runner, while the worker focuses on managing IPC, catching errors, and doing generally doing the work. The worker now guards against repeatedly sending stats, or sending test results after it's started to exit. Required and test files are loaded at the end, after handlers for uncaught exceptions and unhandled rejections have been installed.
Test runners will now run tests as soon as (in the next tick) they're declared. The main process no longer instructs the workers to start running tests. Various IPC commands had to be changed to ensure we can still detect whether "ava" is loaded, or whether a test file contains no actual tests. Fixes #1674.
Ensure the IPC channel stays referenced. This is a safe-guard: it's easy to end up in a situation where the channel needs to be referenced but it's unreferenced because other code wasn't aware.
There are no more anonymous tests.
novemberborn
force-pushed
the
refactor-run
branch
from
fbb7205 to
9487140
Compare
|
Additionally: Fixes #1684. Fixes #1416. Refs #1158. Properly support serial hooks. Hooks are divided into the following categories:
For each category all hooks are run in the order they're declared in. This is different from tests, where serial tests are run before concurrent ones. By default hooks run concurrently. However a serial hook is not run before all preceding concurrent hooks have completed. Serial hooks are never run concurrently. Always hooks are now always run, even if --fail-fast is enabled. Internally, TestCollection, Sequence and Concurrent have been removed. This has led to a major refactor of Runner, and some smaller breaking changes and bug fixes:
|
novemberborn
force-pushed
the
refactor-run
branch
from
9487140 to
59d133e
Compare
Fixes #1684. Fixes #1416. Refs #1158. Properly support serial hooks. Hooks are divided into the following categories: * before * beforeEach * afterEach * afterEach.always * after * after.always For each category all hooks are run in the order they're declared in. This is different from tests, where serial tests are run before concurrent ones. By default hooks run concurrently. However a serial hook is not run before all preceding concurrent hooks have completed. Serial hooks are never run concurrently. Always hooks are now always run, even if --fail-fast is enabled. Internally, TestCollection, Sequence and Concurrent have been removed. This has led to a major refactor of Runner, and some smaller breaking changes and bug fixes: * Unnecessary validations have been removed * Macros can be used with hooks * A macro is recognized even if no additional arguments are given, so it can modify the test (or hook) title * --match now also matches todo tests * Skipped and todo tests are shown first in the output * --fail-fast prevents subsequent tests from running as long as the failure occurs in a serial test
novemberborn
force-pushed
the
refactor-run
branch
from
59d133e to
45fae14
Compare
If a failure occurs in one worker, attempt to interrupt other workers. This only works as long as the other worker has not yet started running concurrent tests. Fixes #1158.
novemberborn
force-pushed
the
refactor-run
branch
from
8d3c588 to
788ec7f
Compare
novemberborn
changed the title
|
And finally, if a failure occurs in one worker, attempt to interrupt other workers. This only works as long as the other worker has not yet started running concurrent tests. Fixes #1158. |
|
I've looked through each commit and everything looks good 👌 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apologies, this is a big one! See the commits for details, but the highlights are:
package.jsonfiles at different levels of the project.execArgvis computed when running a test file, not all at once for all files.--fail-fastnow works with limited concurrency mode, preventing new test files from being run. It also fails fast after a timeout or if a test file crashed. This is part of the work needed for --fail-fast keeps running other test files #1158.--fail-fast.