Skip to content

Commit bf31b5d

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3864)
1 parent 2461771 commit bf31b5d

18 files changed

+340
-336
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 572.1 ms | 571.8 ms | 2.01 ms |
20-
| NUnit | 4.4.0 | 696.5 ms | 695.9 ms | 6.14 ms |
21-
| MSTest | 4.0.2 | 666.7 ms | 666.2 ms | 5.78 ms |
22-
| xUnit3 | 3.2.0 | 742.0 ms | 742.6 ms | 4.27 ms |
23-
| **TUnit (AOT)** | 1.2.3 | 124.2 ms | 124.1 ms | 0.46 ms |
19+
| **TUnit** | 1.2.3 | 556.5 ms | 556.3 ms | 2.03 ms |
20+
| NUnit | 4.4.0 | 663.3 ms | 662.2 ms | 9.02 ms |
21+
| MSTest | 4.0.2 | 635.0 ms | 632.0 ms | 11.80 ms |
22+
| xUnit3 | 3.2.0 | 715.8 ms | 716.7 ms | 8.28 ms |
23+
| **TUnit (AOT)** | 1.2.3 | 124.5 ms | 124.5 ms | 0.26 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 891
62-
bar [572.1, 696.5, 666.7, 742, 124.2]
61+
y-axis "Time (ms)" 0 --> 859
62+
bar [556.5, 663.3, 635, 715.8, 124.5]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.102Z*
75+
*Last generated: 2025-11-18T00:28:14.218Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.2.3 | 2.046 s | 2.041 s | 0.0255 s |
22-
| Build_NUnit | 4.4.0 | 1.657 s | 1.655 s | 0.0286 s |
23-
| Build_MSTest | 4.0.2 | 1.743 s | 1.744 s | 0.0212 s |
24-
| Build_xUnit3 | 3.2.0 | 1.640 s | 1.638 s | 0.0151 s |
21+
| **TUnit** | 1.2.3 | 1.947 s | 1.953 s | 0.0381 s |
22+
| Build_NUnit | 4.4.0 | 1.556 s | 1.557 s | 0.0171 s |
23+
| Build_MSTest | 4.0.2 | 1.621 s | 1.624 s | 0.0156 s |
24+
| Build_xUnit3 | 3.2.0 | 1.523 s | 1.527 s | 0.0162 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.046, 1.657, 1.743, 1.64]
63+
bar [1.947, 1.556, 1.621, 1.523]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-11-17T00:29:04.104Z*
72+
*Last generated: 2025-11-18T00:28:14.220Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 507.19 ms | 507.75 ms | 5.647 ms |
20-
| NUnit | 4.4.0 | 537.32 ms | 536.59 ms | 8.966 ms |
21-
| MSTest | 4.0.2 | 504.77 ms | 506.26 ms | 9.497 ms |
22-
| xUnit3 | 3.2.0 | 586.00 ms | 585.07 ms | 9.163 ms |
23-
| **TUnit (AOT)** | 1.2.3 | 24.73 ms | 24.76 ms | 0.142 ms |
19+
| **TUnit** | 1.2.3 | 493.26 ms | 492.16 ms | 3.175 ms |
20+
| NUnit | 4.4.0 | 601.85 ms | 601.22 ms | 13.698 ms |
21+
| MSTest | 4.0.2 | 617.89 ms | 622.69 ms | 12.013 ms |
22+
| xUnit3 | 3.2.0 | 610.52 ms | 609.06 ms | 11.016 ms |
23+
| **TUnit (AOT)** | 1.2.3 | 24.33 ms | 24.31 ms | 0.233 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 704
62-
bar [507.19, 537.32, 504.77, 586, 24.73]
61+
y-axis "Time (ms)" 0 --> 742
62+
bar [493.26, 601.85, 617.89, 610.52, 24.33]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.102Z*
75+
*Last generated: 2025-11-18T00:28:14.218Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 580.3 ms | 579.7 ms | 3.05 ms |
20-
| NUnit | 4.4.0 | 1,199.9 ms | 1,199.0 ms | 7.89 ms |
21-
| MSTest | 4.0.2 | 3,002.4 ms | 3,000.7 ms | 8.30 ms |
22-
| xUnit3 | 3.2.0 | 3,080.0 ms | 3,078.2 ms | 7.99 ms |
23-
| **TUnit (AOT)** | 1.2.3 | 130.5 ms | 130.4 ms | 0.45 ms |
19+
| **TUnit** | 1.2.3 | 595.5 ms | 595.7 ms | 2.63 ms |
20+
| NUnit | 4.4.0 | 1,164.1 ms | 1,163.4 ms | 6.01 ms |
21+
| MSTest | 4.0.2 | 2,947.1 ms | 2,948.2 ms | 6.02 ms |
22+
| xUnit3 | 3.2.0 | 3,048.5 ms | 3,046.6 ms | 14.96 ms |
23+
| **TUnit (AOT)** | 1.2.3 | 130.9 ms | 130.9 ms | 0.41 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3696
62-
bar [580.3, 1199.9, 3002.4, 3080, 130.5]
61+
y-axis "Time (ms)" 0 --> 3659
62+
bar [595.5, 1164.1, 2947.1, 3048.5, 130.9]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.102Z*
75+
*Last generated: 2025-11-18T00:28:14.219Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 567.17 ms | 565.30 ms | 5.174 ms |
20-
| NUnit | 4.4.0 | 1,552.19 ms | 1,551.63 ms | 4.500 ms |
21-
| MSTest | 4.0.2 | 1,523.45 ms | 1,522.39 ms | 14.764 ms |
22-
| xUnit3 | 3.2.0 | 1,607.93 ms | 1,606.94 ms | 15.902 ms |
23-
| **TUnit (AOT)** | 1.2.3 | 79.51 ms | 79.48 ms | 0.251 ms |
19+
| **TUnit** | 1.2.3 | 561.43 ms | 562.61 ms | 4.718 ms |
20+
| NUnit | 4.4.0 | 1,556.36 ms | 1,557.95 ms | 7.815 ms |
21+
| MSTest | 4.0.2 | 1,517.35 ms | 1,517.64 ms | 12.824 ms |
22+
| xUnit3 | 3.2.0 | 1,602.60 ms | 1,603.72 ms | 11.920 ms |
23+
| **TUnit (AOT)** | 1.2.3 | 79.00 ms | 79.03 ms | 0.223 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1930
62-
bar [567.17, 1552.19, 1523.45, 1607.93, 79.51]
61+
y-axis "Time (ms)" 0 --> 1924
62+
bar [561.43, 1556.36, 1517.35, 1602.6, 79]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.103Z*
75+
*Last generated: 2025-11-18T00:28:14.219Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 524.89 ms | 524.09 ms | 5.578 ms |
20-
| NUnit | 4.4.0 | 577.30 ms | 578.87 ms | 6.118 ms |
21-
| MSTest | 4.0.2 | 501.88 ms | 501.83 ms | 13.036 ms |
22-
| xUnit3 | 3.2.0 | 576.33 ms | 573.62 ms | 9.545 ms |
23-
| **TUnit (AOT)** | 1.2.3 | 45.60 ms | 45.71 ms | 3.635 ms |
19+
| **TUnit** | 1.2.3 | 541.57 ms | 540.90 ms | 3.848 ms |
20+
| NUnit | 4.4.0 | 591.49 ms | 591.89 ms | 9.280 ms |
21+
| MSTest | 4.0.2 | 512.41 ms | 507.96 ms | 11.331 ms |
22+
| xUnit3 | 3.2.0 | 596.95 ms | 593.96 ms | 9.270 ms |
23+
| **TUnit (AOT)** | 1.2.3 | 43.84 ms | 43.96 ms | 3.376 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 693
62-
bar [524.89, 577.3, 501.88, 576.33, 45.6]
61+
y-axis "Time (ms)" 0 --> 717
62+
bar [541.57, 591.49, 512.41, 596.95, 43.84]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.103Z*
75+
*Last generated: 2025-11-18T00:28:14.219Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-11-17** from the latest CI run.
10+
This benchmark was automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,10 +16,10 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.2.3 | 585.9 ms | 585.4 ms | 3.06 ms |
20-
| NUnit | 4.4.0 | 1,159.2 ms | 1,161.2 ms | 4.69 ms |
21-
| MSTest | 4.0.2 | 1,139.4 ms | 1,140.1 ms | 9.12 ms |
22-
| xUnit3 | 3.2.0 | 1,216.8 ms | 1,216.2 ms | 5.74 ms |
19+
| **TUnit** | 1.2.3 | 578.7 ms | 578.1 ms | 6.18 ms |
20+
| NUnit | 4.4.0 | 1,182.7 ms | 1,182.8 ms | 8.39 ms |
21+
| MSTest | 4.0.2 | 1,152.5 ms | 1,151.9 ms | 6.97 ms |
22+
| xUnit3 | 3.2.0 | 1,229.3 ms | 1,225.5 ms | 9.31 ms |
2323
| **TUnit (AOT)** | 1.2.3 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-11-17** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1461
62-
bar [585.9, 1159.2, 1139.4, 1216.8, 0]
61+
y-axis "Time (ms)" 0 --> 1476
62+
bar [578.7, 1182.7, 1152.5, 1229.3, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-11-17T00:29:04.103Z*
75+
*Last generated: 2025-11-18T00:28:14.220Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-11-17** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-11-18** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-11-17T00:29:04.104Z*
83+
*Last generated: 2025-11-18T00:28:14.220Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2025-11-17T00:29:04.102Z",
2+
"timestamp": "2025-11-18T00:28:14.218Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.7, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -10,42 +10,42 @@
1010
{
1111
"Method": "TUnit",
1212
"Version": "1.2.3",
13-
"Mean": "572.1 ms",
14-
"Error": "2.41 ms",
15-
"StdDev": "2.01 ms",
16-
"Median": "571.8 ms"
13+
"Mean": "556.5 ms",
14+
"Error": "2.29 ms",
15+
"StdDev": "2.03 ms",
16+
"Median": "556.3 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "696.5 ms",
22-
"Error": "7.35 ms",
23-
"StdDev": "6.14 ms",
24-
"Median": "695.9 ms"
21+
"Mean": "663.3 ms",
22+
"Error": "10.81 ms",
23+
"StdDev": "9.02 ms",
24+
"Median": "662.2 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "666.7 ms",
30-
"Error": "6.92 ms",
31-
"StdDev": "5.78 ms",
32-
"Median": "666.2 ms"
29+
"Mean": "635.0 ms",
30+
"Error": "12.02 ms",
31+
"StdDev": "11.80 ms",
32+
"Median": "632.0 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.0",
37-
"Mean": "742.0 ms",
38-
"Error": "4.82 ms",
39-
"StdDev": "4.27 ms",
40-
"Median": "742.6 ms"
37+
"Mean": "715.8 ms",
38+
"Error": "9.34 ms",
39+
"StdDev": "8.28 ms",
40+
"Median": "716.7 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
4444
"Version": "1.2.3",
45-
"Mean": "124.2 ms",
46-
"Error": "0.49 ms",
47-
"StdDev": "0.46 ms",
48-
"Median": "124.1 ms"
45+
"Mean": "124.5 ms",
46+
"Error": "0.31 ms",
47+
"StdDev": "0.26 ms",
48+
"Median": "124.5 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)