Skip to content

Add a test runner #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nex3 opened this issue Feb 4, 2015 · 3 comments
Closed

Add a test runner #2

nex3 opened this issue Feb 4, 2015 · 3 comments
Assignees
Labels
type-enhancement A request for a change that isn't a bug

Comments

@nex3
Copy link
Member

nex3 commented Feb 4, 2015

We want to make something that can be invoked from the command line to automatically run all the tests in the package. An important component of this is separating out configuration from the actual test code. Right now, the way tests are run and output is formatted is controlled entirely by the test file itself via functions like solo_test and useCompactVmConfig. These should be controlled by the test runner instead.

@nex3 nex3 added the type-enhancement A request for a change that isn't a bug label Feb 4, 2015
@nex3 nex3 self-assigned this Feb 4, 2015
nex3 added a commit that referenced this issue Feb 5, 2015
So far these interfaces don't really do anything, so there aren't any tests.
Future commits will use these as a foundation.

[email protected]
See #2

Review URL: https://codereview.chromium.org//877553009
@a14n
Copy link
Contributor

a14n commented Feb 5, 2015

Do you plan to remove solo_test ?

During development phase it's really straightforward to add solo_ and run a test. It allows to stay in the editor to run a test that needs some attention. Without that it could be boring to find/copy/paste the name to use in an external test runner.

@zoechi
Copy link

zoechi commented Feb 5, 2015

This is not only about enable/disable a test.

Especially the xxxConfig should be easily switchable depending for example whether you run the tests from an IDE (which may have some jUnit output support), or from a browser, or from console, ....

I experimented with a test runner GUI where I planned to add a GUI to run subsets of the tests on some criterias (failed >0 times in the previous x runs, some meta info added to tests/groups, test run duration - only run tests finished < 0,5sec in the last run, ...)
I was able to address this with features from the unittest library it shows that there might be situations where enabling/disabling by changing the source is not the most appropriate way.

There might be configuration necessary depending on the browser used for the test (content_shell, Firefox, IE) like --allow-file-access-from-files when the client code loads ressources not served from pub serve
But this is mostly configuration for https://pub.dartlang.org/packages/test_runner then.

@nex3
Copy link
Member Author

nex3 commented Feb 5, 2015

@a14n I'm hoping that an editor will be able to detect the test's name and invoke only that test without any modification of the test file. That's how it typically works in other languages.

nex3 added a commit that referenced this issue Feb 11, 2015
These provide the core test-declaration API.

[email protected]
See #2

Review URL: https://codereview.chromium.org//917783003
nex3 added a commit that referenced this issue Feb 11, 2015
No tests yet; since this uses stdout, I'm planning to wait until we have a
runner executable and test that.

[email protected]
See #2

Review URL: https://codereview.chromium.org//913123006
nex3 added a commit that referenced this issue Feb 12, 2015
…lates.

This also adds LiveTest.close, which releases resources associated with a
LiveTest.

[email protected]
See #2

Review URL: https://codereview.chromium.org//914963003
nex3 added a commit that referenced this issue Feb 19, 2015
This is still extremely bare-bones and won't work with tests that actually
import "package:unittest/unittest.dart", but it's something.

[email protected]
See #2

Review URL: https://codereview.chromium.org//933083002
nex3 added a commit that referenced this issue Feb 19, 2015
This preserves the shape of the previous API, but stubs out the functionality.
This is a temporary measure designed to make it easier for users to try the
runner out on their existing tests.

[email protected]
Closes #2

Review URL: https://codereview.chromium.org//934413002
@nex3 nex3 closed this as completed Feb 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-enhancement A request for a change that isn't a bug
Projects
None yet
Development

No branches or pull requests

3 participants