Issue with assembly load context changes in NUnit3TestAdapter v6 & Microsoft.Build.dll dynamic loading

[jairbubbles]

Hi,

Since the update to version 6.0.0 of the NUnit3TestAdapter package. I'm seeing failures in a project which is related to assembly load contexts.

I created a sample project: MsBuildDynamicLoading.Tests.zip (runs fine on 5.2.0, fails on 6.0.0)

This is a library that's used to introspect .csproj content. To do so, we need to load dynamically the Microsoft.Build.dll from the SDK. When simplified it looks like that:

var loadContext = AssemblyLoadContext.GetLoadContext(Assembly.GetExecutingAssembly()) ?? AssemblyLoadContext.Default;
loadContext.Resolving += (assemblyLoadContext, assemblyName) =>
{
    // Load from the MsBuild folder
};

(With the new adapter AssemblyLoadContext.GetLoadContext(Assembly.GetExecutingAssembly()) now longer returns null)

The test by itsel is calling the ProjectInstance constructor. We have access to the symbols because we reference the Microsoft.Build package only at compile:

<PackageReference Include="Microsoft.Build" Version="15.9.20">
  <IncludeAssets>Compile</IncludeAssets>
  <PrivateAssets>All</PrivateAssets>
</PackageReference>

Lots of trick here right? 😜

Anyway, the loading goes well, it will load the Microsoft.Build.dll then Microsoft.Build.Framework.dll. I can see them in the modules window:

But later during the evaluation, I get:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build.Framework, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Le fichier spécifié est introuvable.

Stack trace:

Lazy<SdkResult>.ViaFactory() at C:/Users/jrichard/AppData/Roaming/JetBrains/Rider2025.3/resharper-host/SourcesCache/43c07a384250af318144a0dcb061248bfe4d85ec1de7be8ea536923238d746/Lazy.cs:line 329
Lazy<SdkResult>.ExecutionAndPublication() at C:/Users/jrichard/AppData/Roaming/JetBrains/Rider2025.3/resharper-host/SourcesCache/43c07a384250af318144a0dcb061248bfe4d85ec1de7be8ea536923238d746/Lazy.cs:line 347
Lazy<SdkResult>.CreateValue()
CachingSdkResolverService.ResolveSdk()
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned()
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.ExpandAndLoadImports()
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.EvaluateImportElement() [4]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.PerformDepthFirstPass() [4]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.EvaluateImportElement() [3]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.PerformDepthFirstPass() [3]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.EvaluateImportElement() [2]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.PerformDepthFirstPass() [2]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.EvaluateImportElement() [1]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.PerformDepthFirstPass() [1]
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.Evaluate()
Evaluator<ProjectPropertyInstance, ProjectItemInstance, ProjectMetadataInstance, ProjectItemDefinitionInstance>.Evaluate()
ProjectInstance.Initialize()
new ProjectInstance()
new ProjectInstance()
ProjectLoadingTests.Test_projects_can_be_loaded()

Looks like it's trying to load in the default load context on which I have not registered my callback. If I always register to AssemblyLoadContext.Default, the test will pass fine.

I'm quite puzzled by this behavior and I don't see how it's possible. Looks like the Lazy could be responsible because it's coming from an assembly which is loaded in the default context? 🤔

[OsirisTerje]

@jairbubbles Can you check this one?

NUnit3TestAdapter.6.0.1-alpha.4.zip

[jairbubbles]

Hi @OsirisTerje, I'm not sure I tested right but it's still failing.

I copied the .dlls in the output folder and ran dotnet test --no-build

NB: I closed the issue by mistake (with a keyboard shortcut), I can't reopen it 😩

[OsirisTerje]

We have several similar issues now, so the root cause is probably the same.

Btw, are you testing using Resharper/Rider ?

If so, try to remove the adapter, and then rebuild the app, then the resharper/Rider tests should work.

[jairbubbles]

Hi @OsirisTerje, I got the issue in both Rider and console using dotnet test.

[OsirisTerje]

@jairbubbles That ...... is .... interesting.

Which version of the nunit.engine components (there are 3) do you see in the bin folder ?

[jairbubbles]

(With package 6.0.0)

[OsirisTerje]

Yes, but if you remove the adapter, these should go away too. And if you then try to run it with Rider, what do you see ?

(Rider doesn't use the adapter, but use the engine, handling that itself though)

[jairbubbles]

Yes it works.

When attached I see that it's using the default loading context:

[OsirisTerje]

@jairbubbles Can you check adapter version 6.0.1-alpha.6 ?

[OsirisTerje]

@CharliePoole Just tried the repro above that @jairbubbles enclosed, and it doesn't work with alpha.6.