Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_BitOperations
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
PopCount_uint - Duration of single invocation | 453.72 ns | 406.23 ns | 0.90 | 0.03 | False | |||||
PopCount_ulong - Duration of single invocation | 451.84 ns | 420.33 ns | 0.93 | 0.04 | False | |||||
LeadingZeroCount_uint - Duration of single invocation | 454.40 ns | 408.61 ns | 0.90 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'
Payloads
Histogram
System.Numerics.Tests.Perf_BitOperations.PopCount_uint
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 406.2314717033946 < 429.42437934382434.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 12/14/2022 5:20:21 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 17.188609093835254 (T) = (0 -412.11398326207444) / Math.Sqrt((78.49637844564148 / (299)) + (46.444406073930935 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07314363853182534 = (444.63630007272184 - 412.11398326207444) / 444.63630007272184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_BitOperations.PopCount_ulong
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 420.3347607449879 < 426.41631575377136.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 13.217028237974391 (T) = (0 -418.7124392181921) / Math.Sqrt((83.32547691367718 / (299)) + (44.01909204190214 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.055177902832698036 = (443.16537523153386 - 418.7124392181921) / 443.16537523153386 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 408.6149346180316 < 428.4680663980111.
IsChangePoint: Marked as a change because one of 10/18/2022 1:35:00 AM, 10/26/2022 7:11:30 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 12.194326093671272 (T) = (0 -412.9276941057316) / Math.Sqrt((62.14385772376461 / (299)) + (95.65740715758832 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.0726645741813844 = (445.284071555031 - 412.9276941057316) / 445.284071555031 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in PerfLabTests.DelegatePerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
MulticastDelegateCombineInvoke - Duration of single invocation | 207.10 ms | 192.23 ms | 0.93 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
Payloads
Histogram
PerfLabTests.DelegatePerf.MulticastDelegateCombineInvoke
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 192.23432 < 196.71834616666663.
IsChangePoint: Marked as a change because one of 10/7/2022 12:26:39 PM, 10/17/2022 3:19:47 PM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 35.33066635716004 (T) = (0 -192327150) / Math.Sqrt((32197797739542.355 / (299)) + (33076729639.315556 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05743950471429824 = (204047539.6135749 - 192327150) / 204047539.6135749 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Buffers.Text.Tests.Utf8ParserTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParseUInt64 - Duration of single invocation | 24.72 ns | 20.27 ns | 0.82 | 0.11 | False | |||||
TryParseInt64 - Duration of single invocation | 24.37 ns | 20.90 ns | 0.86 | 0.07 | False | |||||
TryParseInt64 - Duration of single invocation | 25.11 ns | 19.65 ns | 0.78 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.272376452998135 < 23.534776772289625.
IsChangePoint: Marked as a change because one of 10/13/2022 10:51:15 PM, 10/17/2022 3:19:47 PM, 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 18.725809800717432 (T) = (0 -19.99058694799564) / Math.Sqrt((6.786929253712243 / (299)) + (0.274112483189671 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.16150123655644258 = (23.840925973340788 - 19.99058694799564) / 23.840925973340788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.902472592297237 < 23.136469298853115.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 26.45528422240043 (T) = (0 -20.605546692530034) / Math.Sqrt((4.666514229617611 / (299)) + (0.18617627583551816 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.17917187338091128 = (25.103363328206452 - 20.605546692530034) / 25.103363328206452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.64669951855639 < 23.87198689966322.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 32.95813673221547 (T) = (0 -19.73096652387668) / Math.Sqrt((4.684236848286164 / (299)) + (0.06569703293320907 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1924657113447948 = (24.43359594889135 - 19.73096652387668) / 24.43359594889135 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Date_IsNotMatch - Duration of single invocation | 122.56 ns | 108.45 ns | 0.88 | 0.12 | False | |||||
Date_IsMatch - Duration of single invocation | 85.37 ns | 66.80 ns | 0.78 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.44892107481635 < 116.61055508145616.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 10/28/2022 6:19:52 PM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 12/1/2022 5:03:18 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 19.977431199231553 (T) = (0 -108.22826520196124) / Math.Sqrt((159.39263443978754 / (299)) + (0.3792675985016788 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.12138374600337348 = (123.1803585577382 - 108.22826520196124) / 123.1803585577382 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.7961837831784 < 81.09633490277385.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 33.95421911199529 (T) = (0 -66.78840231202358) / Math.Sqrt((21.417937293134155 / (299)) + (0.28852345461768675 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1337509286040287 = (77.10069138012838 - 66.78840231202358) / 77.10069138012838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in PerfLabTests.CastingPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FooObjIsFoo2 - Duration of single invocation | 306.26 μs | 278.55 μs | 0.91 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
Payloads
Histogram
PerfLabTests.CastingPerf.FooObjIsFoo2
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 278.5450818452381 < 290.95703387605045.
IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/14/2022 12:26:48 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 28.6241926155577 (T) = (0 -278513.52740770066) / Math.Sqrt((184331028.25202608 / (299)) + (8558.945363423494 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07470442009320886 = (300999.5221589154 - 278513.52740770066) / 300999.5221589154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
CreateBillboardBenchmark - Duration of single invocation | 23.26 ns | 21.58 ns | 0.93 | 0.05 | False | |||||
CreateConstrainedBillboardBenchmark - Duration of single invocation | 35.05 ns | 31.31 ns | 0.89 | 0.00 | False | |||||
CreateWorldBenchmark - Duration of single invocation | 23.47 ns | 20.40 ns | 0.87 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Payloads
Histogram
System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.58113502755527 < 21.874241751398174.
IsChangePoint: Marked as a change because one of 11/9/2022 12:33:05 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 17.283140737756526 (T) = (0 -21.216400238474176) / Math.Sqrt((0.358671854219886 / (299)) + (0.19453039066704603 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09097795625953006 = (23.33980829680689 - 21.216400238474176) / 23.33980829680689 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.307330386991293 < 33.27858714032986.
IsChangePoint: Marked as a change because one of 10/31/2022 1:09:40 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 62.84940744353342 (T) = (0 -31.19844204854782) / Math.Sqrt((0.18381009892537134 / (299)) + (0.0495204847331618 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.11489190867464856 = (35.248171781857295 - 31.19844204854782) / 35.248171781857295 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.396044883086383 < 22.299526178748415.
IsChangePoint: Marked as a change because one of 12/25/2022 7:48:20 PM, 12/26/2022 6:27:34 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 30.592148062180396 (T) = (0 -20.72803788404705) / Math.Sqrt((1.6798228837375293 / (299)) + (0.08637635760199927 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.1381089818062967 = (24.049488214284402 - 20.72803788404705) / 24.049488214284402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Linq.Tests.Perf_Enumerable
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SingleWithPredicate_LastElementMatches - Duration of single invocation | 778.98 ns | 613.33 ns | 0.79 | 0.37 | False | |||||
Select - Duration of single invocation | 1.11 μs | 996.85 ns | 0.90 | 0.06 | False | |||||
WhereSingle_LastElementMatches - Duration of single invocation | 718.43 ns | 648.16 ns | 0.90 | 0.10 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 613.3278432865375 < 700.5313071424073.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 11:06:27 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 5.558865455326203 (T) = (0 -641.1552497098767) / Math.Sqrt((2477.006009782894 / (299)) + (681.0509762632469 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.06140116504958973 = (683.0982799416657 - 641.1552497098767) / 683.0982799416657 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 996.8460530005264 < 1.049281910340164.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 12/3/2022 2:55:34 AM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 16.390915143421058 (T) = (0 -997.885012681977) / Math.Sqrt((3013.604889585047 / (299)) + (3.5522545327761375 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05015145352027107 = (1050.5727638161661 - 997.885012681977) / 1050.5727638161661 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 648.1639476931698 < 724.9551691860355.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/26/2022 7:11:30 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 16.788477853349 (T) = (0 -647.0110865633864) / Math.Sqrt((1754.8445179438336 / (299)) + (0.6449704458264988 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05936137944912577 = (687.8423577637836 - 647.0110865633864) / 687.8423577637836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Memory.Slice<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
MemoryStartLength - Duration of single invocation | 7.56 ns | 5.79 ns | 0.77 | 0.09 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice<String>*'
Payloads
Histogram
System.Memory.Slice<String>.MemoryStartLength
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.788812364514723 < 6.882472765270317.
IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 6.2314209716666245 (T) = (0 -5.939283168431852) / Math.Sqrt((0.2564978763687566 / (299)) + (0.029051582272479226 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.05376569010839908 = (6.276757359508816 - 5.939283168431852) / 6.276757359508816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 8c58fc2347820ce48e09605d8adddb993df9ebb5 |
Compare | 1d15f2140f7eb30a976c66290491ec89cd628da0 |
Diff | Diff |
Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
EncodeUtf16 - Duration of single invocation | 94.91 ns | 86.15 ns | 0.91 | 0.05 | False | |||||
EncodeUtf16 - Duration of single invocation | 77.40 ns | 69.55 ns | 0.90 | 0.04 | False | |||||
EncodeUtf16 - Duration of single invocation | 99.35 ns | 90.77 ns | 0.91 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.14786438522546 < 89.96255480080958.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 24.174024824127674 (T) = (0 -86.63824615918712) / Math.Sqrt((12.154515880394667 / (299)) + (0.1770387772652945 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.060516855333987186 = (92.21905326464064 - 86.63824615918712) / 92.21905326464064 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 69.55388295510282 < 74.01346580310938.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/31/2022 10:23:25 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 11.2923632898015 (T) = (0 -70.05968722997714) / Math.Sqrt((9.83454572336132 / (299)) + (3.4724102076581227 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.07870582712356904 = (76.04486090608752 - 70.05968722997714) / 76.04486090608752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 90.76837763674179 < 94.37651261825141.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsImprovementStdDev: Marked as improvement because 27.16623868918939 (T) = (0 -90.93143355015957) / Math.Sqrt((12.553652612517256 / (299)) + (0.05710533201690648 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.06025697866810995 = (96.76202055885788 - 90.93143355015957) / 96.76202055885788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository