Skip to content

Refactor cli tests#210

Merged
hildjj merged 6 commits intopeggyjs:1.3from
hildjj:refactor-cli-tests
Nov 11, 2021
Merged

Refactor cli tests#210
hildjj merged 6 commits intopeggyjs:1.3from
hildjj:refactor-cli-tests

Conversation

@hildjj
Copy link
Copy Markdown
Contributor

@hildjj hildjj commented Nov 8, 2021

This turned out to be a lot more work than expected, sorry it took so long.

Fixes #205

"headers": "node ./tools/header.js build/peggy.min.js build/benchmark-bundle.min.js build/test-bundle.min.js",
"deploy": "npm run deploy:peggy && npm run deploy:tests && npm run deploy:bench",
"coverage:bin": "PEGGY_CLI_DEBUG=1 npm run rollup && nyc --reporter lcov jest run.spec.ts",
"coverage:bin": "PEGGY_CLI_DEBUG=1 npm run rollup && jest run.spec.ts",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure not to check in the sourceMap'd version, please.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't understand you :(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jest doesn't follow fork/exec's when recording coverage. the nyc wrapper does. With this refactor, most of the tests don't need to fork/exec, so nyc isn't needed anymore.


if (process.env.PEGGY_CLI_DEBUG) {
cli_config.output.sourcemap = true;
cli_config.output.sourcemap = "inline";
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jest can't handle external sourceMaps. Something to consider for our sourcemap output. We could, for example, make a filename of "inline" be magical, and the default.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. Honestly, never used source maps, don't known how convenient it will be

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's particularly convenient if people are using broken tools like jest. It will make more sense once you run npm run coverage:bin, and look at the end of the bin/peggy.js file.

Copy link
Copy Markdown
Member

@Mingun Mingun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first commit is very big and contains many different changes, so it was hard to review it. It would be better, if you make a separate commits for

  • upgrade libraries
  • move code from peggy.mjs to peggy-cli.mjs
  • made a refactoring in several commits (one for callbacks -> Promise; one for changing options descriptions; ...)
  • rewrite peggy-cli.mjs as a Command

But what is made, made.

Generally, two things need to be fixed:

  • run tests on CI
  • explicitly check exitCode and be sure that it is right in all cases. External utilities should have a clear indication of results of our CLI.

"headers": "node ./tools/header.js build/peggy.min.js build/benchmark-bundle.min.js build/test-bundle.min.js",
"deploy": "npm run deploy:peggy && npm run deploy:tests && npm run deploy:bench",
"coverage:bin": "PEGGY_CLI_DEBUG=1 npm run rollup && nyc --reporter lcov jest run.spec.ts",
"coverage:bin": "PEGGY_CLI_DEBUG=1 npm run rollup && jest run.spec.ts",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't understand you :(

inputStream.on("data", data => { input.push(data); });
inputStream.on("end", () => resolve(Buffer.concat(input).toString()));
inputStream.on("error", er => {
Error.captureStackTrace(er);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that really necessary? If yes it would be worth to add a comment about the reasons

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few places where errors coming into async callbacks don't have their stack traces filled in by default, at least on node 17.0.1. If you specify --verbose, the stack trace is printed. I've put comments on each one.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the V8 docs and this SO answer, the stack captured by Error.captureStackTrace contains a stack of caller location. So it seems it useless to call it here because we in any case cannot add information about where the er is actually was constructed. Besides, I could not find information on whether the existing stack will be rewritten by this call, but if we want to leave it, then this needs to be avoided

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it gave useful information when I tried it, but I'll just remove these calls.

})).resolves.toBe(HELP);
await expect(exec({
error: CommanderError,
errorCode: "commander.helpDisplayed",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that you added a default value for the exitCode property. I'd prefer to leave it explicitly defined to see that we actually check it in all cases. Moreover, it seems that in many cases the exit code changed from 1 to 0 which is incorrect:

  • error when generating a parser should produce 1
  • error when testing an input should produce 2
  • otherwise it should produce 0

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've put exitCode checks on all tests that expect CLI failure. I'm not sure what would have changed from 1 to 0? The only thing that produces 0 that I don't like is --help (which I want to return exit code 64), but I haven't figured out how to make commander do that.


if (process.env.PEGGY_CLI_DEBUG) {
cli_config.output.sourcemap = true;
cli_config.output.sourcemap = "inline";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. Honestly, never used source maps, don't known how convenient it will be

Copy link
Copy Markdown
Member

@Mingun Mingun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hildjj hildjj merged commit f22de58 into peggyjs:1.3 Nov 11, 2021
@hildjj hildjj deleted the refactor-cli-tests branch November 11, 2021 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants