Closed
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 130ef3cae8b88593cbd1627d31c2c4bbf6336c40 |
| Compare | 35e47797a36e29475d28af61f0fa0fb3bf067838 |
| Diff | Diff |
Regressions in System.Tests.Perf_Random
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| NextBytes_span_unseeded - Duration of single invocation | 290.03 ns | 322.66 ns | 1.11 | 0.00 | True | |||||
| NextBytes_unseeded - Duration of single invocation | 294.25 ns | 326.54 ns | 1.11 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'Payloads
Histogram
System.Tests.Perf_Random.NextBytes_span_unseeded
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 322.6556835689924 > 304.54658476800495.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 5/18/2022 4:10:31 AM falls between 5/9/2022 3:38:36 PM and 5/18/2022 4:10:31 AM.
IsRegressionStdDev: Marked as regression because -56.41038282341297 (T) = (0 -324.586755783676) / Math.Sqrt((3.4785271349660327 / (20)) + (4.713958925321555 / (26))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (26) - 2, .025) and -0.11555108925921106 = (290.96538823624826 - 324.586755783676) / 290.96538823624826 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextBytes_unseeded
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 326.53914957646003 > 307.44651144312525.
IsChangePoint: Marked as a change because one of 5/11/2022 7:18:06 PM, 5/18/2022 4:10:31 AM falls between 5/9/2022 3:38:36 PM and 5/18/2022 4:10:31 AM.
IsRegressionStdDev: Marked as regression because -29.950541233293794 (T) = (0 -330.0184077911277) / Math.Sqrt((14.833858155215703 / (20)) + (16.4387114696354 / (26))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (26) - 2, .025) and -0.11904148194466645 = (294.9116838972071 - 330.0184077911277) / 294.9116838972071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository