Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 50e6f078bcf4aa5bca1e7e110b9a6cb846390b82 |
| Compare | 607dccee09d06ef3f0d4c015da39fcc81117b4eb |
| Diff | Diff |
Improvements in System.Collections.TryGetValueTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IDictionary - Duration of single invocation | 3.67 μs | 3.29 μs | 0.90 | 0.21 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'Payloads
Histogram
System.Collections.TryGetValueTrue<Int32, Int32>.IDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.292353119603407 < 3.3320735775142913.
IsChangePoint: Marked as a change because one of 1/13/2022 12:53:43 PM, 1/19/2022 6:01:43 AM, 2/3/2022 10:14:29 PM, 2/8/2022 5:50:06 AM falls between 1/30/2022 5:09:41 PM and 2/8/2022 5:50:06 AM.
IsImprovementStdDev: Marked as improvement because 13.45978429545647 (T) = (0 -3319.41666617287) / Math.Sqrt((57678.2147255926 / (299)) + (2302.52563061358 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.06447094058354981 = (3548.1705594943296 - 3319.41666617287) / 3548.1705594943296 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 50e6f078bcf4aa5bca1e7e110b9a6cb846390b82 |
| Compare | 607dccee09d06ef3f0d4c015da39fcc81117b4eb |
| Diff | Diff |
Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| K_Max_Elements - Duration of single invocation | 2.97 μs | 1.88 μs | 0.63 | 0.07 | True | |||||
| Dequeue_And_Enqueue - Duration of single invocation | 196.27 μs | 155.22 μs | 0.79 | 0.05 | True | |||||
| Dequeue_And_Enqueue - Duration of single invocation | 6.73 μs | 3.81 μs | 0.57 | 0.16 | True | |||||
| HeapSort - Duration of single invocation | 83.97 μs | 62.89 μs | 0.75 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>*'Payloads
Histogram
System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 1000)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.8843399046008935 < 2.8442556238569625.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 2/8/2022 5:50:06 AM falls between 1/30/2022 5:09:41 PM and 2/8/2022 5:50:06 AM.
IsImprovementStdDev: Marked as improvement because 157.51575102470886 (T) = (0 -1896.1062557922994) / Math.Sqrt((11801.747216652275 / (299)) + (28.03823059066176 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.34622637315319055 = (2900.2489209259434 - 1896.1062557922994) / 2900.2489209259434 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 1000)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.22076732673267 < 186.52367291666664.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 2/8/2022 5:50:06 AM falls between 1/30/2022 5:09:41 PM and 2/8/2022 5:50:06 AM.
IsImprovementStdDev: Marked as improvement because 45.442178082742466 (T) = (0 -143819.05693712097) / Math.Sqrt((25867647.266612887 / (299)) + (26073293.539635845 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.25495166311021766 = (193033.18968202278 - 143819.05693712097) / 193033.18968202278 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 100)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.809198428210767 < 6.390092128540739.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 2/8/2022 5:50:06 AM falls between 1/30/2022 5:09:41 PM and 2/8/2022 5:50:06 AM.
IsImprovementStdDev: Marked as improvement because 127.9547005317281 (T) = (0 -3815.108622414939) / Math.Sqrt((99047.01017875702 / (299)) + (3126.898205070056 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.41878703158132347 = (6564.045934478749 - 3815.108622414939) / 6564.045934478749 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 1000)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 62.89292706540447 < 79.75953823752413.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 2/8/2022 5:50:06 AM falls between 1/30/2022 5:09:41 PM and 2/8/2022 5:50:06 AM.
IsImprovementStdDev: Marked as improvement because 171.68387208956432 (T) = (0 -62754.727639321354) / Math.Sqrt((2482947.965984783 / (299)) + (208449.67287118908 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.2628599135705369 = (85132.70244641125 - 62754.727639321354) / 85132.70244641125 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository