Skip to content

feat: use nyc NPM dependency directly without bin executable #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Mar 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,29 @@ jobs:
executor:
name: node/default
tag: '12'
environment:
# we don't need Cypress to check code style
CYPRESS_INSTALL_BINARY: '0'
steps:
- checkout
- node/with-cache:
steps:
- run: npm ci
- run: CYPRESS_INSTALL_BINARY=0 npm ci
- run: npm run format:check

publish:
description: Publishes the new version of the plugin to NPM
executor:
name: node/default
tag: '12'
environment:
# we don't need Cypress to do the release
CYPRESS_INSTALL_BINARY: '0'
# trick semantic-release into thinking this is NOT a pull request
# (under the hood the module env-ci is used to check if this is a PR)
CIRCLE_PR_NUMBER: ''
CIRCLE_PULL_REQUEST: ''
CI_PULL_REQUEST: ''
steps:
- checkout
- node/with-cache:
Expand Down Expand Up @@ -135,14 +146,38 @@ workflows:
command: npx nyc report --check-coverage true --lines 100
working_directory: examples/before-all-visit

- cypress/run:
attach-workspace: true
name: example-ts-example
requires:
- cypress/install
# there are no jobs to follow this one
# so no need to save the workspace files (saves time)
no-workspace: true
working_directory: examples/ts-example
start: npm start
wait-on: 'http://localhost:1234'
command: '../../node_modules/.bin/cypress run'
post-steps:
# store the created coverage report folder
# you can click on it in the CircleCI UI
# to see live static HTML site
- store_artifacts:
path: examples/ts-example/coverage
- run:
command: npm run coverage:check
working_directory: examples/ts-example

- publish:
filters:
branches:
only:
- master
- beta
requires:
- unit
- frontend coverage
- backend coverage
- example-before-each-visit
- example-before-all-visit
- example-ts-example
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ dist/
cypress-coverage/
examples/*/cypress/videos
examples/*/cypress/screenshots
yarn.lock
86 changes: 40 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,7 @@ This plugin will slow down your tests. There will be more web application JavaSc
npm install -D @cypress/code-coverage
```

and its peer dependencies

```shell
npm install -D nyc istanbul-lib-coverage cypress
```
Note: this plugin assumes `cypress` is a peer dependency already installed in your project.

Add to your `cypress/support/index.js` file

Expand Down Expand Up @@ -57,6 +53,40 @@ If you have instrumented your application's code and see the `window.__coverage_

That should be it!

## Reports

The `coverage` folder has results in several formats, and the coverage raw data is stored in `.nyc_output` folder. You can see the coverage numbers yourself. This plugin has `nyc` as a dependency, so it should be available right away. Here are common examples:

```shell
# see just the coverage summary
$ npx nyc report --reporter=text-summary
# see just the coverage file by file
$ npx nyc report --reporter=text
# save the HTML report again
$ npx nyc report --reporter=lcov
```

It is useful to enforce [minimum coverage](https://github.com/istanbuljs/nyc#common-configuration-options) numbers. For example:

```shell
$ npx nyc report --check-coverage --lines 80
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 100 | 100 | 100 | 100 |
main.js | 100 | 100 | 100 | 100 |
----------|---------|----------|---------|---------|-------------------

$ npx nyc report --check-coverage --lines 101
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 100 | 100 | 100 | 100 |
main.js | 100 | 100 | 100 | 100 |
----------|---------|----------|---------|---------|-------------------
ERROR: Coverage for lines (100%) does not meet global threshold (101%)
```

## Instrument unit tests

If you test your application code directly from `specs` you might want to instrument them and combine unit test code coverage with any end-to-end code coverage (from iframe). You can easily instrument spec files using [babel-plugin-istanbul](https://github.com/istanbuljs/babel-plugin-istanbul) for example.
Expand Down Expand Up @@ -181,17 +211,6 @@ You can specify custom coverage reporter(s) to use. For example to output text s

**Tip:** find list of reporters [here](https://istanbul.js.org/docs/advanced/alternative-reporters/)

## NYC

This module tries to find the `nyc` tool using [bin-up][bin-up], so it would be found in the current `node_modules/.bin` or folders up the parent chain

```text
node_modules/.bin/
../node_modules/.bin/
../../node_modules/.bin/
etc
```

## Custom NYC command

Sometimes NYC tool might be installed in a different folder not in the current or parent folder, or you might want to customize the report command. In that case, put the custom command into `package.json` in the current folder and this plugin will automatically use it.
Expand All @@ -204,42 +223,14 @@ Sometimes NYC tool might be installed in a different folder not in the current o
}
```

See examples below.

### Install NYC on the fly

The simplest solution: let `npx` install `nyc` on the fly

```json
{
"scripts": {
"coverage:report": "npx nyc report --report-dir ./coverage --temp-dir .nyc_output --reporter=lcov --reporter=clover --reporter=json"
}
}
```

### Find NYC in a parent folder

If you have [bin-up][bin-up] installed globally, you can use it to find `nyc` installed somewhere in the higher folder.

```json
{
"scripts": {
"coverage:report": "bin-up nyc report --report-dir ./coverage --temp-dir .nyc_output --reporter=lcov --reporter=clover --reporter=json"
}
}
```

**Tip:** finding and running pre-installed tool is always faster than installing it again and again.

## TypeScript users

TypeScript source files are NOT included in the code coverage report by default, even if they are properly instrumented. In order to tell `nyc` to include TS files in the report, you need to:

1. Add these dev dependencies that let Istanbul work with TypeScript

```shell
npm i -D @istanbuljs/nyc-config-typescript source-map-support ts-node
npm i -D @istanbuljs/nyc-config-typescript source-map-support
```

2. In `package.json` use the following `nyc` configuration object
Expand All @@ -253,6 +244,8 @@ npm i -D @istanbuljs/nyc-config-typescript source-map-support ts-node
}
```

See [examples/ts-example](examples/ts-example)

## Exclude code

You can exclude parts of the code or entire files from the code coverage report. See [Istanbul guide](https://github.com/gotwarlost/istanbul/blob/master/ignoring-code-for-coverage.md). Common cases:
Expand Down Expand Up @@ -337,6 +330,7 @@ npm run dev:no:coverage

- [examples/before-each-visit](examples/before-each-visit) checks if code coverage correctly keeps track of code when doing `cy.visit` before each test
- [examples/before-all-visit](examples/before-all-visit) checks if code coverage works when `cy.visit` is made once in the `before` hook
- [examples/ts-example](examples/ts-example) uses Babel + Parcel to instrument and serve TypeScript file

### External examples

Expand All @@ -353,6 +347,7 @@ npm run dev:no:coverage
- [bahmutov/next-and-cypress-example](https://github.com/bahmutov/next-and-cypress-example) shows how to get backend and frontend coverage for a [Next.js](https://nextjs.org) project. Uses [middleware/nextjs.js](middleware/nextjs.js).
- [akoidan/vue-webpack-typescript](https://github.com/akoidan/vue-webpack-typescript) Pure webpack config with vue + typescript with codecov reports. This setup uses babel-loader with TS checker as a separate thread.
- [bahmutov/code-coverage-subfolder-example](https://github.com/bahmutov/code-coverage-subfolder-example) shows how to instrument `app` folder using `nyc instrument` as a separate step before running E2E tests
- [bahmutov/docker-with-cypress-included-code-coverage-example](https://github.com/bahmutov/docker-with-cypress-included-code-coverage-example) runs tests inside pre-installed Cypress using [cypress/included:x.y.z](https://github.com/cypress-io/cypress-docker-images/tree/master/included) Docker image and reports code coverage.

## Debugging

Expand Down Expand Up @@ -392,4 +387,3 @@ This project is licensed under the terms of the [MIT license](LICENSE.md).

[renovate-badge]: https://img.shields.io/badge/renovate-app-blue.svg
[renovate-app]: https://renovateapp.com/
[bin-up]: https://github.com/bahmutov/bin-up
3 changes: 3 additions & 0 deletions examples/ts-example/.babelrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"plugins": ["istanbul"]
}
11 changes: 11 additions & 0 deletions examples/ts-example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# example: ts-example

Code coverage for TypeScript code. See [nyc TS support](https://github.com/istanbuljs/nyc#typescript-projects) docs too.

Code is instrumented on the fly using [.babelrc](.babelrc) and Parcel to run it.

## Use

- start the server and Cypress with `npm run dev`
- run the Cypress tests
- look at the generated reports in folder `coverage`
4 changes: 4 additions & 0 deletions examples/ts-example/cypress.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"baseUrl": "http://localhost:1234",
"fixturesFolder": false
}
28 changes: 28 additions & 0 deletions examples/ts-example/cypress/integration/spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/// <reference types="cypress" />
describe('ts-example', () => {
beforeEach(() => {
cy.visit('/')
})

it('calls add', () => {
cy.window()
.invoke('add', 2, 3)
.should('equal', 5)
})

it('calls sub', () => {
cy.window()
.invoke('sub', 2, 3)
.should('equal', -1)
})

it('calls abs twice', () => {
cy.window()
.invoke('abs', 2)
.should('equal', 2)

cy.window()
.invoke('abs', -5)
.should('equal', 5)
})
})
3 changes: 3 additions & 0 deletions examples/ts-example/cypress/plugins/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = (on, config) => {
on('task', require('../../../../task'))
}
1 change: 1 addition & 0 deletions examples/ts-example/cypress/support/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
import '../../../../support'
4 changes: 4 additions & 0 deletions examples/ts-example/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<body>
Page body
<script src="main.ts"></script>
</body>
22 changes: 22 additions & 0 deletions examples/ts-example/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
const add = (a: number, b: number) => {
return a + b
}

const sub = (a: number, b: number) => {
return a - b
}

function abs(x: number) {
if (x >= 0) {
return x
} else {
return -x
}
}

// @ts-ignore
window.add = add
// @ts-ignore
window.sub = sub
// @ts-ignore
window.abs = abs
21 changes: 21 additions & 0 deletions examples/ts-example/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"name": "example-before-each-visit",
"description": "Getting code coverage when cy.visit is used in beforeEach hook",
"devDependencies": {
"@babel/core": "7.9.0"
},
"nyc": {
"extends": "@istanbuljs/nyc-config-typescript",
"all": true
},
"scripts": {
"start": "../../node_modules/.bin/parcel serve index.html",
"build": "../../node_modules/.bin/parcel build index.html",
"serve": "../../node_modules/.bin/serve dist",
"cy:open": "../../node_modules/.bin/cypress open",
"coverage": "../../node_modules/.bin/nyc report --report-dir ./coverage --temp-dir .nyc_output --reporter=lcov",
"coverage:check": "../../node_modules/.bin/nyc report --check-coverage true --lines 100",
"dev": "../../node_modules/.bin/start-test 1234 cy:open",
"dev:dist": "CYPRESS_baseUrl=http://localhost:5000 ../../node_modules/.bin/start-test serve 5000 cy:open"
}
}
Loading