.NET 8: MSBuild improvements for containers

[richlander]

.NET 8: MSBuild improvements for containers

I get a lot of my inspiration for SDK improvements from container workflows. Containers constrain the MSBuild environment considerably, mostly due to the docker build context. That makes idiomatic experiences like global config at root less than convenient or performant. Separately, MSBuild is not optimized for an immutable by default build environment, which docker offers as a strength.

Samples

Let's take a look at one our Docker samples:

RUN dotnet restore -r linux-musl-arm

# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app -r linux-musl-arm --self-contained false --no-restore

Here's another one:

RUN dotnet restore "Store.ProductApi/Store.ProductApi.csproj"
COPY . .
WORKDIR "/src/Store.ProductApi"
RUN dotnet build "Store.ProductApi.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "Store.ProductApi.csproj" -c Release -o /app/publish

There are two important aspects at play:

• Repeated arguments
• Careful use of no-blah commands to ensure no repeated work or bookkeeping in an immutable environment.

Ideally, would be a way to lock a configuration and to tell MSBuild that that we're in an immutable environment.

Locking in a configuraton

Directory.Build.props enables locking in a configuration. It's not always convenient to create a file for that purpose. However, that's the effect that I'm after.

Instead, something like the following would be awesome:

dotnet lock-config -c Release -r linux-x64 --self-contained false
dotnet restore
dotnet publish

That's an improvement since the configuration is just specified once. However, it doesn't feel right. I don't want a new command. I just want to be able to lock-in my configuration with an existing command.

Like:

dotnet restore -c Release -r linux-x64 --self-contained false
dotnet publish

Immutable mode

MSBuild (and possibly NuGet) does significant work to check if the environment has changed since the last run of the command. The time since the last command was run could have been as short as <1s ago! There are reasons why that behavior is a good one, but it's also very conservative.

Today, we have --no-restore and --no-build commands. That's a lot of extra ceremony.

Here's a naive option:

dotnet restore
dotnet build --immutable-mode
dotnet publish --immutable-mode

That's not much of an improvement on --no-restore and no-build.

How about:

dotnet restore --immutable-mode
dotnet build
dotnet build
dotnet build
dotnet publish

That's looking much better. MSBuild can manage states for me. In this mode, I would expect that only the first dotnet build would run and the second two would just early-exit. Same with the implicit build within restore. Actually, same with the implicit restore within build and publish.

Pulling it all together

The initial idea with locking in a configuration with restore is likely a breaking change. Let's avoid that.

We could introduce a new --lock argument on restore. It locks in a configuration until restore is run again. That overloads restore a bit, but its also the most basic command. I think it works.

Our experience now looks like the following

dotnet restore -c Release -r linux-x64 --self-contained false --lock
dotnet publish

It's reasonable for --immutable to also force --lock. That enables the following experience.

dotnet restore -c Release -r linux-x64 --self-contained false --immutable
dotnet build
dotnet build
dotnet build
dotnet publish

build is only run once, which includes build within publish.

Clearly, I don't want to run dotnet build multiple times. That's not the point I'm trying to make.

Really, I want just the following:

dotnet restore -c Release -r linux-x64 --self-contained false --immutable
dotnet publish

I want to ensure that three things:

• I specify my configuration just once across a set of SDK commands.
• Restore has all the information it needs to be run just once.
• All commands run optimally or else they fail.

Closing thoughts

These changes have the potential to make it so much easier to make a high-performance docker build (with MSBuild). I'm certain we can make improvements here.

The examples are centered exclusively on locking with restore. That's likely not a good idea. We'd want to rationalize that.

These types of experiences always break with dotnet test and (to a lesser degree) dotnet run. We'd need to validate that. Both are reasonable to run within containers (particularly dotnet test.

[richlander]

It could be useful to do perf tests for this with big and small projects. I assume --no-restore and --no-build are up to that task. We'd want to do that with both small, big, and multiple projects.

More exotically, we'd want to test with global.json and nuget.config.

Looks related: #4140

@baronfel @rainersigwald

[richlander]

@DamianEdwards was telling me that this "container experience" exactly maps to "CI experience". The patterns and needs are identical. I can see that.

[DamianEdwards]

It seems there are certainly similarities. Our .NET default yaml files for GitHub Actions and AzDO Pipelines break out into separate steps for restore, build, test, publish, etc. along with opted out of the implied dependent stages for the later steps (e.g. --no-build on dotnet publish), which means passing the various options to each step is required in order for the stage to work correctly. Once you add a RID you need to ensure it's passed to every stage, which today is a completely manual thing.

[richlander]

Yes. That scenario exactly matches what I'm hoping to improve.

[rainersigwald]

Our .NET default yaml files for GitHub Actions and AzDO Pipelines break out into separate steps for restore, build, test, publish, etc. along with opted out of the implied dependent stages for the later steps (e.g. --no-build on dotnet publish)

This has confused me since the beginning. Whenever I stand up a new repo I always delete like three redundant things. Do we have background on why this was thought to be a good idea? Is it more than "in the ancient days you had to run nuget.exe restore separately"?

For containers with layer caching I think it makes some sense, though for any nontrivial repo I'd bet the caching is causing incorrect builds (because the default prior step @richlander didn't show above is COPY *.csproj . which is insufficient to get accurate package information).

[rainersigwald]

How about:

dotnet restore --immutable-mode
dotnet build
dotnet build
dotnet build
dotnet publish

That's looking much better. MSBuild can manage states for me. In this mode, I would expect that only the first dotnet build would run and the second two would just early-exit. Same with the implicit build within restore. Actually, same with the implicit restore within build and publish.

This is an interesting idea. How could it work? What consistency guarantees would be provided? What constraints on user operations between operations would be imposed?

[DamianEdwards]

@rainersigwald the thinking behind it is that it makes it much clearer when a failure occurs where it happened, as the failed stage will be immediately apparent, rather than a single stage doing dotnet publish that could have failed due to restore, build, or publish.

[rainersigwald]

That doesn't seem worth the cost of increasing the likelihood of an inconsistency failure in a later step, to me.

[richlander]

insufficient to get accurate package information

This is true but also the state of the art, to some degree. With docker build, you have a few choices (most to least common):

• Put your Dockerfile by your project or solution file and match your build context to that scope. Potentially miss MSBuild files in parent directories of your repo.
• Put your build context at repo root and then -f your way to your Dockerfile.
• git clone within your Dockerfile.

These all have side-effects that need to be considered.

What constraints on user operations between operations would be imposed?

User cannot change any files. With a Dockerfile, you certainly can change files, but it is pretty uncommon. Once you've copied the build context to the daemon, only the Dockerfile recipe can alter files and that's somewhat awkward to do with either a COPY or RUN command.

This much like an official build where you use sourcelink where the strong intent/implication is that the resultant binary is a faithful reproduction of a given commit.

[richlander]

That doesn't seem worth the cost of increasing the likelihood of an inconsistency failure in a later step, to me.

Avoiding a failure due to inconsistency is a motivator of this proposal.

[rainersigwald]

Sure, so I'd like to ensure "merge steps so you don't have inconsistencies" is on the table as an option.

[rainersigwald]

• Put your Dockerfile by your project or solution file and match your build context to that scope. Potentially miss MSBuild files in parent directories of your repo.

I want to be super clear that you don't need parent directories here. If you use Central Package Version Management, for instance, copying *.csproj is flat wrong. Likewise if you have a Directory.Build.props next to your solution or in any subfolder that influences restore in any way. Basically I think the suggestions in the docs are useful only for trivial codebases.

[richlander]

All true. Perhaps we need to find some better samples to point to that add additional MSBuild assets. @glennc and I have been talking about this exact issue for 5+ years. It's challenging. docker build and MSBuild are not a match made in heaven.

Also, the additional Publish* properties I'm proposing will result in greater use of Directory.Build.props. I was thinking of creating an "official good one" that people can easily curl down whenever they need it.

[DamianEdwards]

Sure, so I'd like to ensure "merge steps so you don't have inconsistencies" is on the table as an option.

Indeed. And I'd like to ensure that the trade-offs associated with any option are explored and acknowledged.

Adding @timheuer as he's involved in the these defaults for Actions/Pipelines today too.

[richlander]

With docker build, the need for a separate restore step is pretty inherent.

[baronfel]

Some of these concerns might be mitigated by the SDK container work that we're targeting for 7. Part of the motivation here is to get the right build assets at the end of the build into the correct places in the container, and that's the scenario we're optimizing for in this effort.

We've talked a bit about what an 'eject button' for the SDK containerization might look like, and that often looks like generated Dockerfile that is 100% correct for your particular project. It might be generated for the user via something like dotnet build /t:GenerateDockerfile, for example. This might be a better path forward than just documentation.

[richlander]

That seems complementary. Even if we generate a Dockerfile, we want it to be pretty, understandable and easily editable. I'm proposing making the SDK work well for immutable environments. That seems like its still a good direction. As others stated, there are other common environments where that is valuable.

[Nirmal4G]

We mostly cache the restore into a separate layer and sometimes, very rarely the artefacts too when we want so many test suites to run on top of a finished build.

The problems I found when doing the above is that I had to separate NuGet or paket restores out from obj and into a separate directory besides bin and obj. This doesn't work today properly across all workloads (e.g., most targets like WPF use BaseIntermediateOutputPath instead of MSBuildProjectExtensionsPath). Most publish like targets also use bin folder now and needs to moved out to a separate folder for caching artefacts layer to enable incremental builds and tests.

I also foresaw that separate directories are needed for isolating things that are used in a build but doesn't need several entries in various ignore files or overriding them anywhere else. We should only have 2 top most folders holding the build artefacts and the final publish artefacts. Every other tool-specific and context-specific folders comes under them.

Yes, it's a breaking change but it's possible to do it in a non-breaking way that does not break existing builds through opt-in mechanism. The first step is to remove the dependency of BaseIntermediateOutputPath from MSBuildProjectExtensionsPath. This allows caching restores into a separate layer fairly easy.

The 10-foot View

I already had a prototype under MSBuild repo but it needed separate props/targets for easy maintainance. I could've put all logic in existing props/targets files but that meant heavy maintainance cost. I also didn't want to introduce new props/targets in the root when we already have a proposal to pack the common props and targets into an SDK. I tried packing them into an SDK but since I don't know much about MSBuild's build infra, I gave up half way. Tasks needed to be refactored to work as an SDK package. I'm also aware that MSBuild is being deployed in more than one way. I'm interested in doing both but need some pointers from the person who knows the build infra best.

Finally, the end goal, atleast I envision, is to have MSBuild xcopyable with as much as few files as possible or even have a single file deployment for both dotnet and msbuild. Like imagine having an entire dotnet sdk initial download with a single file or archive with only dotnet, msbuild executables present and everything else can be downloaded as SDKs.

[Nirmal4G]

Directory.Build.props enables locking in a configuration. It's not always convenient to create a file for that purpose. However, that's the effect that I'm after.

You could use Directory.Build.rsp instead. Since the goal is flow the options through, a response file might be a better option than a props/targets file. Depending on the build setup, a property could be overriden later in props or targets after the initial import.

[richlander]

I thought about this more. I no longer really like the idea I proposed. The --immutable idea may or nay not be a good idea, but I'm not pushing that. It's really a perf thing and perf data should drive that (which I haven't presented).

This is the key idea ... specifying characteristics for my build once and then not needing to do so again:

dotnet restore -c Release -r linux-x64 --self-contained false
dotnet publish

However, tying that to restore doesn't seem right. Instead, we need some form of config system. I'd be happy with:

export DOTNET_CLI_ARGS="-c Release -r linux-x64 --self-contained false"
dotnet restore
dotnet publish