Skip to content

[Perf] Changes at 7/2/2022 10:25:38 PM #6620

Closed
@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 27.67 ms 22.53 ms 0.81 0.02 False
Count - Duration of single invocation 27.59 ms 22.53 ms 0.82 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.529141457142863 < 26.28492639734615.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 126.26099495125112 (T) = (0 -22505278.36706502) / Math.Sqrt((383667160331.1298 / (299)) + (3162534320.713608 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1868923550541973 = (27678104.500622544 - 22505278.36706502) / 27678104.500622544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 22.53093540666667 < 26.25181852891667.
IsChangePoint: Marked as a change because one of 5/15/2022 3:27:46 PM, 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 126.42159187040684 (T) = (0 -22516210.365146525) / Math.Sqrt((493790373266.54706 / (299)) + (196339817.6988187 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1868998620692369 = (27691804.877130423 - 22516210.365146525) / 27691804.877130423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 55.90 ns 46.46 ns 0.83 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.45694036402642 < 53.16648161893434.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/30/2022 12:57:05 PM, 6/6/2022 7:24:37 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 16.799080260304432 (T) = (0 -47.112165899347346) / Math.Sqrt((16.253158721678304 / (299)) + (0.766227383736375 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1213922745281718 = (53.621388173029395 - 47.112165899347346) / 53.621388173029395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SameIndex1 - Duration of single invocation 565.84 ns 513.29 ns 0.91 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.SameIndex1(length: 1024)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 513.2858383956478 < 537.6228157320563.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 251.40156037166304 (T) = (0 -512.3483959713368) / Math.Sqrt((0.23232664867625916 / (299)) + (0.36616597933597994 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.09573040732027703 = (566.5881061565253 - 512.3483959713368) / 566.5881061565253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare b3c3921a888fcf162d6e270178a0e3ea9a40a022
Diff Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 846.32 ns 772.57 ns 0.91 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 772.5705594847882 < 803.2590006420628.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsImprovementStdDev: Marked as improvement because 321.62830742137703 (T) = (0 -773.6292348482507) / Math.Sqrt((5.4687722941748795 / (299)) + (0.24283074623147988 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.08399157597275729 = (844.5656334108587 - 773.6292348482507) / 844.5656334108587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions