Skip to content

[Perf] Windows/x64: 22 Regressions on 7/23/2023 2:51:47 PM #20151

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
111.41 μs 167.24 μs 1.50 0.06 False
167.12 μs 250.59 μs 1.50 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf2.CastingPerf.IntObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.2355984042553 > 116.97944968971635.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -3726.623320759 (T) = (0 -167109.8890044402) / Math.Sqrt((571.2598793094108 / (22)) + (3555.2792905077768 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.500096295910723 = (111399.44112920175 - 167109.8890044402) / 111399.44112920175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.5883361678005 > 175.48088696808512.
IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -5923.783812834366 (T) = (0 -250620.0404114637) / Math.Sqrt((1973.6592451522486 / (22)) + (1853.6876434421943 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.4997594603316721 = (167106.8241543474 - 250620.0404114637) / 167106.8241543474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.55 ns 8.88 ns 1.18 0.20 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.878705540561668 > 7.527794079352377.
IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -16.743125525419128 (T) = (0 -8.883865434623916) / Math.Sqrt((0.15964468990841463 / (21)) + (0.06627201280274189 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.2503127434929925 = (7.105314634964933 - 8.883865434623916) / 7.105314634964933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.19 ns 17.48 ns 1.15 0.11 False
2.71 ns 4.69 ns 1.73 0.11 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.475795197343693 > 16.080844317469154.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -6.203605919043107 (T) = (0 -17.42774689480443) / Math.Sqrt((1.955548184237554 / (22)) + (0.007607491354146127 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.1190426359229584 = (15.573800617909843 - 17.42774689480443) / 15.573800617909843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.685479138101039 > 2.8511158780289834.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 6/9/2023 2:21:24 AM, 7/23/2023 4:13:29 AM, 7/26/2023 8:42:30 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -3.6468689678705757 (T) = (0 -4.70843618222053) / Math.Sqrt((0.42700477081647403 / (31)) + (0.5614912247893903 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.5318531556781007 = (3.0736863809483497 - 4.70843618222053) / 3.0736863809483497 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorDefaultSizeNonGeneric

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.93 ns 20.99 ns 1.17 0.07 False
9.23 ns 10.56 ns 1.14 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'

Payloads

Baseline
Compare

System.Collections.CtorDefaultSizeNonGeneric.SortedList

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.99079166861367 > 18.81903421401946.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -35.80942234365815 (T) = (0 -20.70938565882215) / Math.Sqrt((0.1324555059490425 / (22)) + (0.03909698931698644 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.1855520899951712 = (17.46813643498912 - 20.70938565882215) / 17.46813643498912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorDefaultSizeNonGeneric.Stack

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/22/2023 2:06:57 AM, 5/26/2023 6:22:05 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -22.82742841267576 (T) = (0 -10.646050586498978) / Math.Sqrt((0.029809514873644378 / (21)) + (0.029758966317799913 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.13489281049938537 = (9.380666163366046 - 10.646050586498978) / 9.380666163366046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.11 ns 15.42 ns 1.18 0.15 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.417747700122419 > 14.050898038708356.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 6/5/2023 10:43:22 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -10.104957363631982 (T) = (0 -15.301904003449884) / Math.Sqrt((0.12201217222391122 / (20)) + (0.282074348685674 / (15))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (15) - 2, .025) and -0.11634138763393699 = (13.707190446357956 - 15.301904003449884) / 13.707190446357956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.62 μs 6.19 μs 1.10 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/26/2023 6:22:05 AM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -21.56288709257577 (T) = (0 -6182.710107941997) / Math.Sqrt((5245.9110085752945 / (21)) + (6116.608809063844 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.09252157184279658 = (5659.11947854118 - 6182.710107941997) / 5659.11947854118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.19 ns 18.08 ns 1.12 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.077934897039135 > 16.962552614773923.
IsChangePoint: Marked as a change because one of 4/22/2023 6:52:03 AM, 5/19/2023 10:30:38 AM, 6/23/2023 6:04:39 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -11.965758854537114 (T) = (0 -17.629069563087775) / Math.Sqrt((0.11553420623909438 / (22)) + (0.1616058025552287 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.08986578389864751 = (16.17545006324118 - 17.629069563087775) / 16.17545006324118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.34 ns 10.46 ns 1.42 0.19 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.45676124745665 > 7.724404179005982.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 10:42:00 AM, 5/3/2023 7:05:50 AM, 5/23/2023 1:41:12 PM, 6/13/2023 11:57:38 AM, 6/16/2023 8:27:26 PM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -43.20854589034163 (T) = (0 -10.599189758177454) / Math.Sqrt((0.06842522827563671 / (20)) + (0.029056745042022693 / (14))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (14) - 2, .025) and -0.43315434099537375 = (7.395707116105835 - 10.599189758177454) / 7.395707116105835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
176.61 μs 193.70 μs 1.10 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.6962406367041 > 188.24312254213484.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -31.27991059974162 (T) = (0 -193166.83113159175) / Math.Sqrt((3095019.201540684 / (20)) + (446175.9634901736 / (14))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (14) - 2, .025) and -0.07521568464387171 = (179654.03024749554 - 193166.83113159175) / 179654.03024749554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions