Skip to content

IComparable fallback for Tuple/ValueTuple #2368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 13, 2023
Merged

IComparable fallback for Tuple/ValueTuple #2368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
32dedc2
Catch CompareTo argument exceptions and fallback to string comparison
mrahhal Jul 13, 2023
215abd5
Catch when message contains expected IComparable error
mrahhal Jul 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion src/BenchmarkDotNet/Parameters/ParameterComparer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,16 @@ private int CompareValues(object x, object y)
if (x != null && y != null && x.GetType() == y.GetType() &&
x is IComparable xComparable)
{
return xComparable.CompareTo(y);
try
{
return xComparable.CompareTo(y);
}
// Some types, such as Tuple and ValueTuple, have a fallible CompareTo implementation which can throw if the inner items don't implement IComparable.
// See: https://github.com/dotnet/BenchmarkDotNet/issues/2346
// For now, catch and ignore the exception, and fallback to string comparison below.
catch (ArgumentException ex) when (ex.Message.Contains("At least one object must implement IComparable."))
{
}
}

// Anything else.
Expand Down
68 changes: 68 additions & 0 deletions tests/BenchmarkDotNet.Tests/ParameterComparerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,70 @@ public void IComparableComparisionTest()
Assert.Equal(4, ((ComplexParameter)sortedData[3].Items[0].Value).Value);
}

[Fact]
public void ValueTupleWithNonIComparableInnerTypesComparisionTest()
{
var comparer = ParameterComparer.Instance;
var originalData = new[]
{
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, (new ComplexNonIComparableParameter(), 1), null)
Copy link
Collaborator

@timcassell timcassell Jul 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this test fail without the try/catch? The exception message indicates that it would succeed since int is IComparable. I would expect it to only fail if all types are non-IComparable. Unless the exception message is wrong, and it actually means all objects must implement IComparable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this fails without the try/catch.

I would expect it to only fail if all types are non-IComparable.

Not exactly. The exception message doesn't come from Tuple/ValueTuple, it's from System.Collections.Comparer.Compare, and it's talking about one of the two values in the arguments.

In the ValueTuple's implementation for example, it's just calling the comparer on each item. So CompareTo could work if int was the first item, but only if the compare is non 0.

}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, (new ComplexNonIComparableParameter(), 3), null)
}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, (new ComplexNonIComparableParameter(), 2), null)
}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, (new ComplexNonIComparableParameter(), 4), null)
})
};

var sortedData = originalData.OrderBy(d => d, comparer).ToArray();

Assert.Equal(1, (((ComplexNonIComparableParameter, int))sortedData[0].Items[0].Value).Item2);
Assert.Equal(2, (((ComplexNonIComparableParameter, int))sortedData[1].Items[0].Value).Item2);
Assert.Equal(3, (((ComplexNonIComparableParameter, int))sortedData[2].Items[0].Value).Item2);
Assert.Equal(4, (((ComplexNonIComparableParameter, int))sortedData[3].Items[0].Value).Item2);
}

[Fact]
public void TupleWithNonIComparableInnerTypesComparisionTest()
{
var comparer = ParameterComparer.Instance;
var originalData = new[]
{
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, Tuple.Create(new ComplexNonIComparableParameter(), 1), null)
}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, Tuple.Create(new ComplexNonIComparableParameter(), 3), null)
}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, Tuple.Create(new ComplexNonIComparableParameter(), 2), null)
}),
new ParameterInstances(new[]
{
new ParameterInstance(sharedDefinition, Tuple.Create(new ComplexNonIComparableParameter(), 4), null)
})
};

var sortedData = originalData.OrderBy(d => d, comparer).ToArray();

Assert.Equal(1, ((Tuple<ComplexNonIComparableParameter, int>)sortedData[0].Items[0].Value).Item2);
Assert.Equal(2, ((Tuple<ComplexNonIComparableParameter, int>)sortedData[1].Items[0].Value).Item2);
Assert.Equal(3, ((Tuple<ComplexNonIComparableParameter, int>)sortedData[2].Items[0].Value).Item2);
Assert.Equal(4, ((Tuple<ComplexNonIComparableParameter, int>)sortedData[3].Items[0].Value).Item2);
}

private class ComplexParameter : IComparable<ComplexParameter>, IComparable
{
public ComplexParameter(int value, string name)
Expand Down Expand Up @@ -199,5 +263,9 @@ public int CompareTo(object obj)
return CompareTo(other);
}
}

private class ComplexNonIComparableParameter
{
}
}
}