Include all files in coverage report, not just the ones with test against them

[yann-combarnous]

We use Cypress for testing with Vite + this plugin, and run our tests in parallel.

As a tester, I would like to be able to get a global code coverage for all files selected by include/exclude options, not just the ones that have already tests against them.

So if include/exclude results in files A, B and C, and my tests just cover file C, I want to see the code coverage for A, B and C, not just C.

NYC supports this option through the --all flag, see https://github.com/istanbuljs/nyc#selecting-files-for-coverage . This also seems supported by Istanbul Instrumenter Loader, see https://www.npmjs.com/package/istanbul-instrumenter-loader .

Not having this option is breaking running tests in parallel with Codecov or Coveralls, for instance, as merged reports are missing for different untested files in each test group.

[iFaxity]

This could be a tricky issue.

As this goes against how Vite works.
Vite only compiles the files that are imported. I would like to guess that if some files are not included in the tests this are actually not compiled by Vite and will not pass by the plugin.

I'm not really sure if there is a way around this but i would imagine that this is not really possible at the moment.
I know that the Vite team is looking into code coverage themselves and might fix this in the future.

[iFaxity]

After thinking about this for a while, i think this is not really do to this plugin and rather what reporter is used and how it is configured.

I don't have any experience with parallel tests, but would it not be the goal of the reporter to include the files that are not covered. As this plugin is only for instrumenting code and not for generating reports. Have you tested the following: https://github.com/cypress-io/code-coverage#include-code, also check the note here: https://github.com/cypress-io/code-coverage#exclude-files-and-folders.

Note: if you have all: true NYC option set, this plugin will check the produced .nyc_output/out.json before generating the final report. If the out.json file does not have information for some files that should be there according to include list, then an empty placeholder will be included, see cypress-io/code-coverage#208.

[yann-combarnous]

After thinking about this for a while, i think this is not really do to this plugin and rather what reporter is used and how it is configured.

I don't have any experience with parallel tests, but would it not be the goal of the reporter to include the files that are not covered. As this plugin is only for instrumenting code and not for generating reports. Have you tested the following: https://github.com/cypress-io/code-coverage#include-code, also check the note here: https://github.com/cypress-io/code-coverage#exclude-files-and-folders.

Note: if you have all: true NYC option set, this plugin will check the produced .nyc_output/out.json before generating the final report. If the out.json file does not have information for some files that should be there according to include list, then an empty placeholder will be included, see cypress-io/code-coverage#208.

Thanks. I already had the setup described in the Cypress links you provided, but it does not work with your plugin. It did work with Webpack and Istanbul plugin.

But I do not think the reporter can count covered / not covered lines / branches. Reporter are just reading .nyc_output/out.json file with stats from Istanbul and converting it into human/machine readable report.

So the trick could be to initialize .nyc_output for all files matching include/exclude. Not sure how to do this yet.

[yann-combarnous]

Actually, this is also discussed on Cypress side: cypress-io/code-coverage#539

[yann-combarnous]

My workaround is to disable chunk-splitting in Vite using:

{
  rollupOptions: {
    output: {
      manualChunks: () => 'bundle',
    }
}

Otherwise with code splitting, Vite only parses the code files loaded by the tests.

[Emrald-TA]

I also am using Cypress with Vite + this plugin (all of this in a docker container) and wanted to achieve the same as @yann-combarnous . However the solution posted above did not work for me. @yann-combarnous if possible, could you explain your approach a bit further? I have a feeling that I am doing something wrong.

Just to be more specific about my question, below is what would like to know:

• How are you testing the preview version of your build with Cypress?
• Is there a way to disable code splitting without building?

[iFaxity]

Closed due to inactivity