Instrument assemblies shared by multiple processes

[kevingosse]

Hi,

I'm trying to add coverlet to our CI but I'm running into some trouble:

System.IO.IOException: The process cannot access the file 'D:\a\1\s\test\Datadog.Trace.ClrProfiler.IntegrationTests\bin\x64\Release\netcoreapp2.1\profiler-lib\netstandard2.0\Datadog.Trace.ClrProfiler.Managed.dll' because it is being used by another process.
    at System.IO.FileSystem.CopyFile(String sourceFullPath, String destFullPath, Boolean overwrite)
    at System.IO.File.Copy(String sourceFileName, String destFileName, Boolean overwrite)
    at Coverlet.Core.Helpers.FileSystem.Copy(String sourceFileName, String destFileName, Boolean overwrite) in /_/src/coverlet.core/Helpers/FileSystem.cs:line 32
    at Coverlet.Core.Helpers.InstrumentationHelper.<>c__DisplayClass13_0.<RestoreOriginalModules>b__0() in /_/src/coverlet.core/Helpers/InstrumentationHelper.cs:line 250
    at Coverlet.Core.Helpers.RetryHelper.<>c__DisplayClass0_0.<Retry>b__0() in /_/src/coverlet.core/Helpers/RetryHelper.cs:line 25
    at Coverlet.Core.Helpers.RetryHelper.Do[T](Func`1 action, Func`1 backoffStrategy, Int32 maxAttemptCount) in /_/src/coverlet.core/Helpers/RetryHelper.cs:line 52<---

It's easy to understand why it fails if you know our build process:

• We publish our assemblies into one folder
• Our integration tests spawn child processes which use the assemblies published above. I'm trying to get the code coverage of those child processes so I spawn them with the coverlet tool
• Since the tests run in parallel, multiple instances of coverlet are trying to patch/restore the same set of assemblies

An obvious fix would be to stop sharing those assemblies, but we started doing that because of disk space issues in the build agent.

I'm wondering what would be my options here. Ideally what I would like to do is:

• Ask coverlet to patch the shared assemblies
• Run all the integration tests
• Have coverlet restore the original assemblies

but I couldn't find how to ask Coverlet to just run the "patch" or "restore" step. What would you suggest?

[MarcoRossignoli]

Hi @kevingosse
an honor have you here using coverlet thanks for giving it a try.

Unfortunately at the moment that scenario is not supported, but it's not the first time someone ask for it(#781).
Today the workflow for .NET tool is:

1. .NET tool starts dlls instrumentation and keeps in memory instrumentation data(where are hits file, where are source files to generate reports)
2. start your target+argument
3. when target exits thanks to "in memory" instrumentation data we can read hits file/source mappings to generate report.

The workflow for this scenario should be:

coverlet instrument <DIRECTORY> --instrumentationDataPath '...' --other parameters filters etc...
run.exe // run you app, load dll etc...on process exits coverlet will fill hits files syncing access to common hits file(we're under mutex).
coverlet report <DIRECTORY> --instrumentationDataPath '...' --other parameters, report format etc...

We should revisit .NET tool logic, add some verbs, breaking a bit, but would be a great feature I think.
I need only to find some time to do it 😃

cc: @jakubch1

[kevingosse]

That would be great 😃

For now I'm trying the following workaround:

• Launch the test project with the IncludeDirectory parameter to also instrument the assemblies from the child processes
• The test project launches the child processes with the COVERLET_ENABLETRACKERLOG environment variable so they emit coverage information
• When the test project finishes, it will collect those reports as if they were its own

I did a few tests locally and it looks promising. I'm struggling to get it to work on the real project, mainly because of the deduplication logic when using IncludeDirectory (basically, the main project references C:\folder1\assembly.dll and the child project references C:\folder2\assembly.dll, so IncludeDirectory C:\folder2 will ignore that additional assembly.dll). But I think I'll find a way, I'll keep you posted.

[kevingosse]

The workaround worked. Of course it's very hacky, so I'd be very happy to switch to #781 if it ever gets implemented 😃

Closing this issue since the feature request is tracked in the other issue. Thanks!