System.Numerics.Vectors.dll fails to load

[colgreen]

This relates to use of the System.Numerics.Vectors (nuget package 4.5) in a dotnet standard 2.0 library, and consuming that library from a Windows .NET Framework application.

When the runtime attempts to load System.Numerics.Vectors this error is reported...

System.IO.FileNotFoundException
  HResult=0x80070002
  Message=Could not load file or assembly 'System.Numerics.Vectors, Version=4.1.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

Reverting to System.Numerics.Vectors 4.4.0 resolves the problem.

Example code:

dotnet standard 2.0 library class...

namespace FooLib
{
    public class Class1
    {
        public void DoStuff(int[] arr)
        {
            var va = new Vector<int>(arr, 0);
            var vb = new Vector<int>(arr, 4);
            var vc = va + vb;
            vc.CopyTo(arr, 0);
        }
    }
}

.NET Franework 4.7 console app

    class Program
    {
        static void Main(string[] args)
        {
            int[] arr = new int[32];

            Class1 c1 = new Class1();
            c1.DoStuff(arr);

            Console.WriteLine(arr[0]);
        }
    }

[karelz]

@eerhardt @ViktorHofer can you please check it out? Looks like a regression :(

[ViktorHofer]

This repros on my machine on all tfms from net461 to net472 only when the System.Numerics.Vectors package is added inside a netstandard2.0 library. Referencing it directly from netfx works as expected.

[colgreen]

Referencing it directly from netfx works as expected.

Yes, that's the behaviour I'm seeing too.

[ViktorHofer]

I talked with Eric and he will look into it either later today or tomorrow. I will stand by for now.

[eerhardt]

I can repro this issue as well. However, I am unable to get this behavior:

Reverting to System.Numerics.Vectors 4.4.0 resolves the problem.

This scenario fails for me no matter if I use 4.3.0, 4.4.0, or 4.5.0 of System.Numerics.Vectors. @colgreen - can you share a repo or .zip that contains the projects with it working and targeting 4.4.0? I want to see how you made it work.

The issue appears to be that the System.Numerics.Vectors.dll is not being copied to the output directory. I'm unsure why it isn't. Still investigating

Another workaround is to use an SDK-based .csproj to target net47:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net47</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\VectorsReproLib\VectorsReproLib.csproj" />
  </ItemGroup>

</Project>

That doesn't repro the issue for me.

/cc @joperezr

[saucecontrol]

I had this same problem today. Strange that it's looking for 4.1.3.0. The assembly version for S.N.V 4.5 is 4.1.4.0. I tried an assemblyBinding redirection to 4.1.4.0, but that just gives me a two FileNotFoundExceptions instead of one.

[saucecontrol]

Oh, well there's your problem... the netstandard2.0 ref assembly in the Nuget package is 4.1.3.0 while the lib assembly is 4.1.4.0. The net46 assembly versions match (both 4.1.4.0), which is why that target works.

On a related note, was it strictly necessary to bump the version number on the S.N.V assembly? I'm finding that increasingly, users of my libraries are taking all updates Nuget offers at once, and then my library is the one that throws the error looking for an older version of a MS assembly. And the users that actually do know enough to set up a binding redirect are confused by the version numbers. They try to set up a redirect to 4.5 in this case, not realizing the package version and assembly version have no relationship to each other. If we can't hold the version steady, can we at least make them match?

[eerhardt]

Speaking with @joperezr, he pointed me to dotnet/standard#410, which is the exact same scenario. The underlying issue is that "old style" Desktop projects don't get transitive dependencies from "SDK-style" NETStandard projects by default. To workaround this, you can do any of the following:

1. Reference the dependency (System.Numerics.Vectors) directly in the Desktop project.
2. Convert the Desktop project to an "SDK-style" project (i.e. it has <Project Sdk="Microsoft.NET.Sdk"> at the top).
3. Adding the following property to the Desktop project: <RestoreProjectStyle>PackageReference</RestoreProjectStyle> and auto generating binding redirects if necessary <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>. See Dependencies don't flow from new NETStandard project to old Desktop projects through ProjectReferences sdk#901 (comment).

@colgreen - I still don't see how this could be working with 4.4.0 of System.Numerics.Vectors. Was something else different when it worked? Did you have some other <PackageReference> in the project?

I'm going to close this as a duplicate of dotnet/sdk#901. If this scenario does work with v4.4.0 of System.Numerics.Vectors, please re-open with a repro project that shows it working.

[eerhardt]

To answer @saucecontrol's questions:

was it strictly necessary to bump the version number on the S.N.V assembly?

Whenever we add an API to an assembly, we need to bump the version number. We bump the version across all assemblies for each release, in this case from .NET Core 2.0 to .NET Core 2.1.

not realizing the package version and assembly version have no relationship to each other. If we can't hold the version steady, can we at least make them match?

I agree this isn't an ideal experience. @ericstj @weshaggard may be better to answer exactly why these versions get out of sync. But this is the case for all our assemblies in corefx, not just System.Numerics.Vectors.

[saucecontrol]

@eerhardt I'm fairly certain this is a different issue than the one you referenced. As I said, the ref assembly and lib assembly inside the Nuget package have mismatched assembly version numbers. Reverting to 4.4 does fix it because that version of the package isn't broken.

As far as the version bump, I'm still not clear on why that needed to happen. The only new API I see is the Vector<T> constructor that accepts a Span<T>, and that only appears in the netcoreapp2.1 target, which has only a placeholder in this package since netcoreapp2.1 has Vector<T> built in. Am I missing another change that affects the platforms that actually use the package assembly?

[eerhardt]

@saucecontrol - can you send a .zip, or a link to a repo, with a solution that works with 4.4.0 but fails to work with 4.5.0? Following the repro steps outlined on this bug fails for me for the exact reason as dotnet/sdk#901. And it still fails for me using 4.4.0.

As far as the version bump, I'm still not clear on why that needed to happen.

I believe we "bulk edit" them for all assemblies across the board for each release.

[ericstj]

We can never change the reference assembly version of an netstandard assembly once it goes inbox somewhere: a netstandard library compiled against a newer package must continue to work when run on a framework that contains the older inbox assembly. For this reason you'll see a lower versioned reference. That's what happened with Vectors.

For desktop assemblies we must always increment the version whenever we recompile because GAC always wins over app and someone may install any of our packages in the GAC. For this reason you'll see a higher desktop assembly version.

If you enable automatic bindingRedirects in your desktop app it should automatically generate the redirect for System.Numerics.Vectors.dll and enable it to load correctly. (assuming you also reference the package or enable PackageReference mode).

[saucecontrol]

can you send a .zip, or a link to a repo, with a solution that works with 4.4.0 but fails to work with 4.5.0

Sorry @eerhardt, I swear I got that to work yesterday, but I deleted my test project. Retrying now, it fails with either version, but setting <RestoreProjectStyle>PackageReference</RestoreProjectStyle> fixes it. I was playing with manual bindingRedirects yesterday, so that may have confounded things.

@ericstj, thanks for that explanation and reference. That makes infinitely more sense now.

I believe we "bulk edit" them for all assemblies across the board for each release.

This is unfortunate, especially in light of the fact that the reference assembly version can't change... which I assume means the API can't actually change. There has to be a better way of handling those version numbers (and consequently Strong Names) for assemblies distributed via Nuget, but I do see the problem if someone GACs one of those.

[colgreen]

@eerhardt Here you go:

VectorProto-v1.zip

It took a while to distill the problem down to a demo solution with only the relevant aspects. In summary, the attached solution contains a .NET Framework 4.7 Unit Test project that refers to a dotnet standard 2.0 library project.

Both the unit test project and the lib project refer to the System.Numerics.Vectors nuget directly, in the original solution this was because there was a unit test that indicates if the hardware acceleration is enabled or not:

        [TestMethod]
        public void TestHardwareAccelerated()
        {
            if(!Vector.IsHardwareAccelerated) {
                Assert.Inconclusive("Hardware accelerations not available. Hardware acceleration is available on supporting CPUs only, and only for x64 builds with optimization enabled (i.e. release builds).");
            }
            ...
        }

As well as other tests against the code in the lib that uses System.Numerics.Vectors if IsHardwareAccelerated == true.

Hence, when I saw the new SNV nuget I updated both projects, ran my unit tests and they started failing! I reverted back to nuget 4.4.0 and they all passed.

[saucecontrol]

This sounds much more similar to the problem I had that led me to this issue. In my case, a user has a net47 web site (i.e. no csproj) that references my netstandard2.0 library in a Nuget package. That library package references System.Numerics.Vectors 4.4.0. Since the web site is using packages.config, Nuget Package Manager pulls in System.Numerics.Vectors directly as a dependency when my library is referenced. The site owner then sees the update to System.Numerics.Vectors 4.5.0 offered when she views her available Nuget updates and pulls it in even though her only reference to it is through my library. Boom, my library breaks and the only fix is to either roll back the package update or create a bindingRedirect manually. Is there an equivalent to AutoGenerateBindingRedirects that can be used in cases like that?

I suspect because the reference assembly and lib assembly versions are different within the Nuget package, the same scenario would play out even if I update my library to reference 4.5.0 directly, because I'm still building against a 4.1.3.0 reference and the bin version will still be 4.1.4.0.

@colgreen the following app.config fixes your unit test project up, same as with the web site I mentioned

<configuration>
	<runtime>
		<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
			<dependentAssembly>
				<assemblyIdentity name="System.Numerics.Vectors" publicKeyToken="b03f5f7f11d50a3a"/>
				<bindingRedirect oldVersion="4.0.0.0-4.1.4.0" newVersion="4.1.4.0"/>
			</dependentAssembly>
		</assemblyBinding>
	</runtime>
</configuration>

[eerhardt]

Thanks @colgreen and @saucecontrol. I am able to repro the problem, and agree with @saucecontrol that the reason this is failing is that the netstandard2.0 ref assembly (and all previous versions of the package) have an AssemblyVersion less than 4.1.4, and in the latest NuGet package (4.5.0) has an implementation assembly with AssemblyVersion 4.1.4. When assemblies are referencing different versions of another assembly on .NET Framework projects, you have to use an assembly binding redirect in order for the runtime to load it correctly. (Note on .NET Core this redirect happens automatically using the .deps.json file and TPA list).

Is there an equivalent to AutoGenerateBindingRedirects that can be used in cases like that?

I wasn't part of the original AutoGenerateBindingRedirects implementation, so I might not have all the correct info. Hopefully someone corrects me if I'm wrong. But it appears to only kick in when both AutoGenerateBindingRedirects and GenerateBindingRedirectsOutputType are set to true. GenerateBindingRedirectsOutputType gets defaulted to true when you are an exe or winexe OutputType project. But when you are a unit test or web project (like you both describe), the OutputType isn't set to exe. So to get the binding redirects to kick in, I needed to set both properties in the unit test project:

    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>

I suspect because the reference assembly and lib assembly versions are different within the Nuget package, the same scenario would play out even if I update my library to reference 4.5.0 directly, because I'm still building against a 4.1.3.0 reference and the bin version will still be 4.1.4.0.

Assuming your library is targeting netstandard2.0, then you are correct.

---

This issue is still "closed", but I agree it isn't a dupe of dotnet/sdk#901. Instead, it is by design as @ericstj mentions above - the netstandard2.0 ref assembly was pinned to 4.1.3 because it was made inbox in .NET Core 2.0. Using assembly binding redirects is the correct fix here.

[saucecontrol]

Thanks for the explanation @eerhardt. I wasn’t aware of the GenerateBindingRedirectsOutputType setting, but I guess that doesn’t help with a web site anyway since there’s no project file to put that in. And it’s certainly not intuitive even if that solution is an option. I get that it’s by design and is mitigated in certain toolsets, but man is that a mess…

As a library author, it sucks for me, because a certain (not insignificant, I think) segment of my users will download my package and immediately find it doesn’t work. That reflects poorly on me, even if the fix is to create a bindingRedirect on their side.

As a general user of those packages, it sucks for me because it’s extremely counter-intuitive that I should require a bindingRedirect (whether automagical or manual) to use the library version contained in a package version that I have explicitly and directly referenced. And it’s not exactly easy to track down the correct version to create the bindingRedirect to if you have to add it manually.

If I’m understanding @ericstj ’s explanation correctly, the implementation assembly version number is bumped even though the reference assembly version is frozen just so that desktop users who have GAC’ed the assemblies pick up the new version. If that’s the case, you’ve put the needs of two groups of users at odds with one another. I’d argue (and I’m certainly biased) that people using Nuget distribution as intended should have a clean and intuitive experience and the people who do odd things like GAC-ing a Nuget-distributed assembly should get the pain they’ve brought on themselves. It certainly would work better from my perspective if the implementation assembly version number stayed put once the reference assembly version is frozen.

[saucecontrol]

I continue to have problems with these ref/lib version mismatches, even with current tooling and the settings given above.

This time it's the System.Buffers NuGet package version 4.5.0, which has ref assembly version 4.0.2.0 and lib assembly version 4.0.3.0 for netstandard2.0 targets.

Simple repro .csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net46</TargetFramework>
    <!-- None of the below settings make any difference. Should be default values? -->
    <RestoreProjectStyle>PackageReference</RestoreProjectStyle>
    <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
    <GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
  </PropertyGroup>

  <ItemGroup>
    <!-- This package has a dependency on System.Buffers 4.5.0 for net46 targets -->
    <PackageReference Include="PhotoSauce.MagicScaler" Version="0.9.1" />
  </ItemGroup>
</Project>

and .cs

using System;
using System.IO;
using PhotoSauce.MagicScaler;

class Program
{
    static void Main()
    {
        // 1px gif
        var img = Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAAFElEQVR4XgXAgQwAAACAsPy1LtYyAMUAxBPtybYAAAAASUVORK5CYII=");
        MagicImageProcessor.ProcessImage(img, new MemoryStream(), new ProcessImageSettings());
    }
}

throws:

System.IO.FileLoadException: Could not load file or assembly 'System.Buffers, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies.

I can fix this with a manual binding redirect, but I'm having trouble communicating to my users which assemblies need redirects on which targets and what from/to versions to use.

@eerhardt @ericstj Is there some other magic setting I'm missing that makes the tooling take care of this?

[eerhardt]

We had a bug in System.Memory that was fixed in version 4.5.2.

I've repro'd your issue above and was able to fix it by doing the following:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net46</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="PhotoSauce.MagicScaler" Version="0.9.1" />

    <!-- Add this line --> 
    <PackageReference Include="System.Memory" Version="4.5.2" />

  </ItemGroup>
</Project>

If you own (or know who does) the PhotoSauce.MagicScaler package, you can ask them to update their System.Memory dependency from 4.5.1 to 4.5.2 to fix this issue for everyone.

[eerhardt]

Looking deeper (for tracking purposes) this was https://github.com/dotnet/corefx/issues/32457

[saucecontrol]

Thanks, @eerhardt. I couldn't figure out why the problem seemed to be isolated to a single package this time, but that explains it.

[Havie]

Im having this issue with v4.5.0 not sure what to do