Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.CastingPerf2.CastingPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
111.41 μs | 167.24 μs | 1.50 | 0.06 | False | ||||
167.12 μs | 250.59 μs | 1.50 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
Payloads
PerfLabTests.CastingPerf2.CastingPerf.IntObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.2355984042553 > 116.97944968971635.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -3726.623320759 (T) = (0 -167109.8890044402) / Math.Sqrt((571.2598793094108 / (22)) + (3555.2792905077768 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.500096295910723 = (111399.44112920175 - 167109.8890044402) / 111399.44112920175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.5883361678005 > 175.48088696808512.
IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -5923.783812834366 (T) = (0 -250620.0404114637) / Math.Sqrt((1973.6592451522486 / (22)) + (1853.6876434421943 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.4997594603316721 = (167106.8241543474 - 250620.0404114637) / 167106.8241543474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
7.55 ns | 8.88 ns | 1.18 | 0.20 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
Payloads
Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.878705540561668 > 7.527794079352377.
IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -16.743125525419128 (T) = (0 -8.883865434623916) / Math.Sqrt((0.15964468990841463 / (21)) + (0.06627201280274189 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.2503127434929925 = (7.105314634964933 - 8.883865434623916) / 7.105314634964933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.Span<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
15.19 ns | 17.48 ns | 1.15 | 0.11 | False | ||||
2.71 ns | 4.69 ns | 1.73 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'
Payloads
System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.475795197343693 > 16.080844317469154.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -6.203605919043107 (T) = (0 -17.42774689480443) / Math.Sqrt((1.955548184237554 / (22)) + (0.007607491354146127 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.1190426359229584 = (15.573800617909843 - 17.42774689480443) / 15.573800617909843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Char>.IndexOfValue(Size: 4)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.685479138101039 > 2.8511158780289834.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 6/9/2023 2:21:24 AM, 7/23/2023 4:13:29 AM, 7/26/2023 8:42:30 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -3.6468689678705757 (T) = (0 -4.70843618222053) / Math.Sqrt((0.42700477081647403 / (31)) + (0.5614912247893903 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.5318531556781007 = (3.0736863809483497 - 4.70843618222053) / 3.0736863809483497 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.CtorDefaultSizeNonGeneric
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
17.93 ns | 20.99 ns | 1.17 | 0.07 | False | ||||
9.23 ns | 10.56 ns | 1.14 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'
Payloads
System.Collections.CtorDefaultSizeNonGeneric.SortedList
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.99079166861367 > 18.81903421401946.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -35.80942234365815 (T) = (0 -20.70938565882215) / Math.Sqrt((0.1324555059490425 / (22)) + (0.03909698931698644 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.1855520899951712 = (17.46813643498912 - 20.70938565882215) / 17.46813643498912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorDefaultSizeNonGeneric.Stack
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/22/2023 2:06:57 AM, 5/26/2023 6:22:05 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -22.82742841267576 (T) = (0 -10.646050586498978) / Math.Sqrt((0.029809514873644378 / (21)) + (0.029758966317799913 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.13489281049938537 = (9.380666163366046 - 10.646050586498978) / 9.380666163366046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Enum
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
13.11 ns | 15.42 ns | 1.18 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Payloads
System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.417747700122419 > 14.050898038708356.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 6/5/2023 10:43:22 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -10.104957363631982 (T) = (0 -15.301904003449884) / Math.Sqrt((0.12201217222391122 / (20)) + (0.282074348685674 / (15))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (15) - 2, .025) and -0.11634138763393699 = (13.707190446357956 - 15.301904003449884) / 13.707190446357956 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.IterateFor<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
5.62 μs | 6.19 μs | 1.10 | 0.10 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<String>*'
Payloads
System.Collections.IterateFor<String>.ImmutableSortedSet(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/16/2023 12:30:38 AM, 5/26/2023 6:22:05 AM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -21.56288709257577 (T) = (0 -6182.710107941997) / Math.Sqrt((5245.9110085752945 / (21)) + (6116.608809063844 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.09252157184279658 = (5659.11947854118 - 6182.710107941997) / 5659.11947854118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Tests.Perf_StringBuilder
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
16.19 ns | 18.08 ns | 1.12 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Payloads
System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.077934897039135 > 16.962552614773923.
IsChangePoint: Marked as a change because one of 4/22/2023 6:52:03 AM, 5/19/2023 10:30:38 AM, 6/23/2023 6:04:39 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -11.965758854537114 (T) = (0 -17.629069563087775) / Math.Sqrt((0.11553420623909438 / (22)) + (0.1616058025552287 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.08986578389864751 = (16.17545006324118 - 17.629069563087775) / 16.17545006324118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.Tests.Perf_BitArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
7.34 ns | 10.46 ns | 1.42 | 0.19 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
Payloads
System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.45676124745665 > 7.724404179005982.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 10:42:00 AM, 5/3/2023 7:05:50 AM, 5/23/2023 1:41:12 PM, 6/13/2023 11:57:38 AM, 6/16/2023 8:27:26 PM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -43.20854589034163 (T) = (0 -10.599189758177454) / Math.Sqrt((0.06842522827563671 / (20)) + (0.029056745042022693 / (14))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (14) - 2, .025) and -0.43315434099537375 = (7.395707116105835 - 10.599189758177454) / 7.395707116105835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | d40ca00ca2fb5411ff159b2e84b800e8dfbd681e |
Compare | 65536eb2f01ebb85a54e792a2b974fe76af7e230 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in PerfLabTests.LowLevelPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
176.61 μs | 193.70 μs | 1.10 | 0.10 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Payloads
PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.6962406367041 > 188.24312254213484.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 7/23/2023 4:13:29 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsRegressionStdDev: Marked as regression because -31.27991059974162 (T) = (0 -193166.83113159175) / Math.Sqrt((3095019.201540684 / (20)) + (446175.9634901736 / (14))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (14) - 2, .025) and -0.07521568464387171 = (179654.03024749554 - 193166.83113159175) / 179654.03024749554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository