Fixes #3771. Replace multitests by static errors tests#3778
Fixes #3771. Replace multitests by static errors tests#3778sgrekhov wants to merge 1 commit intodart-lang:masterfrom
Conversation
|
We have to think about this one: Changing tests from multi-test format to the new outcome expectation format ( |
|
The main motivation for this change is to switch to the new error expectation format ( I checked all these updated tests in the analyzer, we expect errors exactly where they reported. |
|
The main difference that I have in mind is, as you mention, that syntax errors are really tricky to handle in the case where we have several of them in the same library. The parser is perhaps expected to recover from syntax errors and get on track, ready to detect the next syntax error at the location where it occurs. However, we don't specify the recovery at all, and it's really difficult to say whether or not it's OK for a given implementation to report each of the syntax errors in any given test library as specified by our outcome expectations, or there is some other meaningful way to interpret the text as having a different set of syntax errors. Hence, we should probably never have a test with multiple syntax errors using the new test outcome expectations. Next, commands like this one will work for a test library using the multi-test format: If this test library were migrated to use the new format then it would fail because the test runner is unable to distinguish syntax errors from other compile-time errors. For example, this one fails: (I really should create a CL and get this landed, but there's a need to adjust the test runner in So I keep saying that we should not migrate syntax tests to the new test outcome syntax, because this means that running the spec parser on "all tests" (for example, all language tests, which is what I've used most frequently) will get more and more noisy, reporting syntax errors as failed test runs rather than accepting test runs with syntactically wrong tests as successful test runs when a syntax error is reported. |
No description provided.