Skip to content

[Perf] Changes at 7/2/2022 10:25:38 PM #6537

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SelectToArray - Duration of single invocation 314.12 ns 282.27 ns 0.90 0.04 False
WhereSingle_LastElementMatches - Duration of single invocation 385.55 ns 325.62 ns 0.84 0.03 False
WhereAny_LastElementMatches - Duration of single invocation 726.75 ns 681.87 ns 0.94 0.03 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 282.2696316306068 < 297.3743775420964.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/11/2022 9:09:01 PM, 5/28/2022 5:31:50 AM, 6/2/2022 5:58:32 PM, 6/6/2022 10:51:35 AM, 6/10/2022 11:27:32 AM, 6/16/2022 2:51:40 PM, 6/22/2022 6:53:40 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 22.699620401971764 (T) = (0 -283.0813964687496) / Math.Sqrt((185.64799047844892 / (299)) + (1.5277299548377603 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06979412550060705 = (304.32123063197696 - 283.0813964687496) / 304.32123063197696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 325.6155575886316 < 365.9692533323985.
IsChangePoint: Marked as a change because one of 6/1/2022 6:40:18 PM, 6/6/2022 10:51:35 AM, 6/10/2022 11:27:32 AM, 6/17/2022 12:17:51 PM, 6/29/2022 5:07:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 12.822501161235632 (T) = (0 -325.1461132661208) / Math.Sqrt((814.9489856757059 / (299)) + (0.16132588495411868 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06140897360912086 = (346.4193712957071 - 325.1461132661208) / 346.4193712957071 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 681.8708380793831 < 689.8124900350856.
IsChangePoint: Marked as a change because one of 5/24/2022 7:55:26 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 22.22746302778219 (T) = (0 -683.1552807696581) / Math.Sqrt((1243.2874656457639 / (299)) + (6.653545758582071 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06948315249310025 = (734.1675570947602 - 683.1552807696581) / 734.1675570947602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x86
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 26.33 ns 24.13 ns 0.92 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,16)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.134292238877215 < 24.81161114556024.
IsChangePoint: Marked as a change because one of 5/6/2022 8:15:32 AM, 5/11/2022 9:09:01 PM, 5/25/2022 8:13:38 AM, 6/8/2022 9:45:10 AM, 6/21/2022 2:34:11 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 10.533092016245229 (T) = (0 -24.454172123196006) / Math.Sqrt((1.5307696253384955 / (299)) + (0.08507545065014964 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.053576655134245706 = (25.838513236024113 - 24.454172123196006) / 25.838513236024113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterInt32 - Duration of single invocation 15.05 ns 12.31 ns 0.82 0.04 False
FormatterUInt64 - Duration of single invocation 10.22 ns 7.68 ns 0.75 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.312232019767073 < 14.294955198929268.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 64.83043897532639 (T) = (0 -12.338758697454718) / Math.Sqrt((0.37467949977881404 / (299)) + (0.00485246808321711 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.18472623359588636 = (15.134497399415476 - 12.338758697454718) / 15.134497399415476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.67707494062857 < 9.698278718124989.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 70.64656678801862 (T) = (0 -7.693144642681609) / Math.Sqrt((0.24593487472131462 / (299)) + (0.0010464765490264705 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2247753515228235 = (9.923761657725597 - 7.693144642681609) / 9.923761657725597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x86
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 32.00 ms 30.11 ms 0.94 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\s", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.112115 < 30.401563145833332.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 23.391401584913655 (T) = (0 -30135494.432234436) / Math.Sqrt((97620637522.8041 / (299)) + (44050333857.33712 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.05940072179399202 = (32038611.054126523 - 30135494.432234436) / 32038611.054126523 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions