Skip to content

[One .NET] fix parallel .sln builds #5435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 18, 2020

Conversation

jonathanpeppers
Copy link
Member

Fixes: #5280

A test has been randomly failing with:

(CoreCompile target) ->
CSC : error CS2012: Cannot open 'C:\a\1\s\bin\TestRelease\temp\SolutionBuildSeveralProjects\Lib6\obj\Debug\Lib6.dll' for writing -- 'The process cannot access the file 'C:\a\1\s\bin\TestRelease\temp\SolutionBuildSeveralProjects\Lib6\obj\Debug\Lib6.dll' because it is being used by another process.'

I could reproduce this by putting [Repeat(10)] on the test. We
shouldn't commit this change, since the test took 5+ minutes.

The problem happens when:

  • Build a .sln with two Android head projects that reference library
    projects.
  • Use -m:4 to build with multiple MSBuild nodes.
  • One of the head projects is running <Csc/> of a referenced project
    at the same time another.

The two <Csc/> calls can only occur because of the way we have inner
builds setup for $(RuntimeIdentifier) in 51fb93e. One project is
building the referenced library normally, while another is doing an
inner build for a $(RuntimeIdentifier).

We don't really want <Csc/> to compile things at all in an inner
build. It should just be "finding" paths to assemblies for each
$(RuntimeIdentifier) and not doing actual work.

Let's set $(SkipCompilerExecution) to true for the inner build:

This property is set during design-time builds in Visual Studio, so it
should be something we can rely on here.

I think this solved the problem, as I was unable to reproduce it
anymore after a few tries. It's possible it didn't, though!

Fixes: dotnet#5280

A test has been randomly failing with:

    (CoreCompile target) ->
    CSC : error CS2012: Cannot open 'C:\a\1\s\bin\TestRelease\temp\SolutionBuildSeveralProjects\Lib6\obj\Debug\Lib6.dll' for writing -- 'The process cannot access the file 'C:\a\1\s\bin\TestRelease\temp\SolutionBuildSeveralProjects\Lib6\obj\Debug\Lib6.dll' because it is being used by another process.'

I could reproduce this by putting `[Repeat(10)]` on the test. We
shouldn't commit this change, since the test took 5+ minutes.

The problem happens when:

* Build a `.sln` with two Android head projects that reference library
  projects.
* Use `-m:4` to build with multiple MSBuild nodes.
* One of the head projects is running `<Csc/>` of a referenced project
  at the same time another.

The two `<Csc/>` calls can only occur because of the way we have inner
builds setup for `$(RuntimeIdentifier)` in 51fb93e. One project is
building the referenced library normally, while another is doing an
inner build for a `$(RuntimeIdentifier)`.

We don't really want `<Csc/>` to compile things *at all* in an inner
build. It should just be "finding" paths to assemblies for each
`$(RuntimeIdentifier)` and not doing actual work.

Let's set `$(SkipCompilerExecution)` to `true` for the inner build:

* https://github.com/dotnet/roslyn/blob/4929b2e60abbdbb6cbe72c856d6f370c8cf13b35/src/Compilers/Core/MSBuildTask/Microsoft.CSharp.Core.targets#L132
* https://github.com/dotnet/project-system/blob/e217c5fc9f5ec06f593f74762a3bf4d750475a67/docs/well-known-project-properties.md#skipcompilerexecution-bool

This property is set during design-time builds in Visual Studio, so it
should be something we can rely on here.

I *think* this solved the problem, as I was unable to reproduce it
anymore after a few tries. It's possible it didn't, though!
@jonathanpeppers jonathanpeppers marked this pull request as ready for review December 17, 2020 20:33
@jonpryor
Copy link
Contributor

Is this failure in CheckMultipleLibraryProjectReferenceAlias(True) expected/ignorable?

https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4311923&view=ms.vss-test-web.build-test-results-tab&runId=17738128&resultId=100129&paneView=debug

Library should have built.
Expected: True
But was:  False


   at Xamarin.Android.Build.Tests.AndroidUpdateResourcesTest.CheckMultipleLibraryProjectReferenceAlias(Boolean withGlobal) in /Users/builder/azdo/_work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/AndroidUpdateResourcesTest.cs:line 44

Copy link
Member

@radekdoulik radekdoulik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CheckMultipleLibraryProjectReferenceAlias test failed few times in the past on master and succeeded in re-run, so it might be unrelated.

I have restarted the remaining failing build. Otherwise the change LGTM.

@jonpryor jonpryor merged commit 136353a into dotnet:master Dec 18, 2020
@jonathanpeppers jonathanpeppers deleted the dotnet-parallel-sln-build branch December 18, 2020 16:01
@github-actions github-actions bot locked and limited conversation to collaborators Jan 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[One .NET] build solution in parallel can cause <Csc/> to trip on itself
4 participants