0.13.7 BenchmarkDotNet.Autogenerated project fails build

[LeadAssimilator]

In VS2022 with a .NET 4.8.1 project in release cfg, the Autogenerated project is failing to build using 0.13.7 with the following errors, but it works fine with 0.13.6. The generated project files appear to be the same between the package versions as well so something else is going on. Interestingly, adding an explicit PackageReference for BenchmarkDotNet to the BenchmarkDotNet.Autogenerated.csproj makes it work.

8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(136,48): error CS0234: The type or namespace name 'Engines' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(328,48): error CS0234: The type or namespace name 'Engines' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(245,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(437,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(270,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(462,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(280,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(472,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(305,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]
8e7c92b4-3761-43e1-b9bd-ee799e381dbf.notcs(497,53): error CS0234: The type or namespace name 'Portability' does not exist in the namespace 'BenchmarkDotNet' (are you missing an assembly reference?) [BenchmarkDotNet.Autogenerated.csproj]

Note: Pathing was stripped for brevity.

[timcassell]

The default toolchain for Framework was changed to CsProjClassicNetToolchain instead of RoslynToolchain (#2370). You must include a reference to BenchmarkDotNet (not just BenchmarkDotNet.Annotations) in your projects for that toolchain (#2214), or you can explicitly use the old RoslynToolchain in the config.

[LeadAssimilator]

I already have a reference to BenchmarkDotNet and it does NOT work in 0.13.7.

I literally just created a new console project targetting .net framework 4.8.1, added BenchmarkDotNet 0.13.7 reference, changed configuration to release, copy pasted the example/getting started code and hit run. Same error as above. Downgrading to 0.13.6 works. So if it is related to the toolchain change...

How do I change the toolchain?
Where is this documented?
How is anyone who uses VS/.NET 4.x and installs 0.13.7 going to figure this out?

[timcassell]

@LeadAssimilator I'm not able to repro. I used the simplest benchmark and it worked without issue.

code

public class Program
{
    public static void Main(string[] args)
    {
        BenchmarkRunner.Run<Benchmarks>();
    }
}

public class Benchmarks
{
    [Benchmark]
    public void Benchmark()
    {
        Thread.Sleep(100);
    }
}

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>net481</TargetFrameworks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.13.7" />
  </ItemGroup>

</Project>

[Edit] Sorry, I was able to repro after creating a classic net framework project in Visual Studio. I will look at fixing this, but I recommend you use modern csproj with the net481 tfm (like my example here).

[timcassell]

How do I change the toolchain?

var config = DefaultConfig.Instance.AddJob(Job.Default.WithToolchain(RoslynToolchain.Instance));
BenchmarkRunner.Run<Benchmarks>(config, args);

Where is this documented?

https://benchmarkdotnet.org/articles/configs/toolchains.html

BDN has a lot of toolchains, not all of which are mentioned here. We are accepting PRs to update docs.

[LeadAssimilator]

Thanks! Changing the toolchain as you mentioned worked.

This issue seems related to the use of packages.config nuget package format in a legacy project with the newer toolchain. Once I switched to the PackageReference nuget package format (while keeping the overall legacy project as is) it worked with 0.13.7.

Manually adding a BDN PackageReference to the autogenerated csproj also worked. Perhaps that is a viable option for fixing this or working around it? Not sure what the downsides would be, but it at least makes sure the autogenerated project always can restore/find its references regardless of toolchain.

And if this won't get fixed, I think it would be useful to open a readme.txt on nuget package install with some getting started info which can contain some helpful info/links that could also say that packages.config is not supported and/or how to change the toolchain to make it work.

[Matthias-Heinz]

I would be able to provide a sample solution using .net framework 4.8 - just in case:
BenchmarkTest_MissingTypeEngine.zip