Skip to content

[Feature Request] Add custom option to sort benchmark jobs by numeric orders #2768

@filzrev

Description

@filzrev

Currently BenchmarkDotNet's DefaultOrderer don't have parameter to customize Job display orders.
And JobComparer.Instance is always used.

Currently JobComparer.Instance sort jobs based on characteristics.
And compare each characteristics with Ordinal string comparer.

I want to add option to specify JobOrderPolicy that change behavior of JobComparer with custom NumericStringComparer.
(that similar to CompareOptions.NumericOrdering that added to .NET 10)

Background
Currently there is following job display order issues when using DefaultOrderer.

Example. 1: .NET Runtime orders
.NET 10.0 benchmark is shown on top item.

Method Runtime Mean Error Ratio
Benchmark1 .NET 10.0 16.337 ms NA 3.32
Benchmark1 .NET 8.0 4.922 ms NA 1.00
Benchmark1 .NET 9.0 16.233 ms NA 3.30

Example.2: JobId column orders (and NuGetReferences column orders)
Similarly, JobId/NuGetReferences are sorted by ordinal order.

Type Method Job N Mean
ContainsBenchmark Contains v1.4.1 1000000 29.87 us
ContainsBenchmark Contains v1.4.10 1000000 29.90 us
ContainsBenchmark Contains v1.4.2 1000000 30.31 us

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions