Skip to content

Commit 17e4643

Browse files
thomhurstthomhurst
authored
chore: update benchmark results (#3989)
1 parent dd65654 commit 17e4643

18 files changed

+340
-336
lines changed

‎docs/docs/benchmarks/AsyncTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 2
77
# AsyncTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 589.8 ms | 589.3 ms | 4.89 ms |
20-
| NUnit | 4.4.0 | 688.2 ms | 688.9 ms | 10.89 ms |
21-
| MSTest | 4.0.2 | 645.1 ms | 643.0 ms | 10.60 ms |
22-
| xUnit3 | 3.2.1 | 724.5 ms | 723.6 ms | 9.69 ms |
23-
| **TUnit (AOT)** | 1.3.25 | 124.7 ms | 124.7 ms | 0.25 ms |
19+
| **TUnit** | 1.3.25 | 579.9 ms | 579.1 ms | 2.78 ms |
20+
| NUnit | 4.4.0 | 666.7 ms | 666.3 ms | 6.70 ms |
21+
| MSTest | 4.0.2 | 639.7 ms | 634.5 ms | 8.95 ms |
22+
| xUnit3 | 3.2.1 | 730.4 ms | 732.8 ms | 13.59 ms |
23+
| **TUnit (AOT)** | 1.3.25 | 124.3 ms | 124.3 ms | 0.60 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "AsyncTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 870
62-
bar [589.8, 688.2, 645.1, 724.5, 124.7]
61+
y-axis "Time (ms)" 0 --> 877
62+
bar [579.9, 666.7, 639.7, 730.4, 124.3]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.385Z*
75+
*Last generated: 2025-12-06T00:27:43.842Z*

‎docs/docs/benchmarks/BuildTime.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 8
77
# Build Performance Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -18,10 +18,10 @@ Compilation time comparison across frameworks:
1818

1919
| Framework | Version | Mean | Median | StdDev |
2020
|-----------|---------|------|--------|--------|
21-
| **TUnit** | 1.3.25 | 2.066 s | 2.060 s | 0.0353 s |
22-
| Build_NUnit | 4.4.0 | 1.649 s | 1.646 s | 0.0154 s |
23-
| Build_MSTest | 4.0.2 | 1.720 s | 1.713 s | 0.0206 s |
24-
| Build_xUnit3 | 3.2.1 | 1.630 s | 1.630 s | 0.0259 s |
21+
| **TUnit** | 1.3.25 | 2.098 s | 2.091 s | 0.0329 s |
22+
| Build_NUnit | 4.4.0 | 1.681 s | 1.679 s | 0.0149 s |
23+
| Build_MSTest | 4.0.2 | 1.799 s | 1.792 s | 0.0238 s |
24+
| Build_xUnit3 | 3.2.1 | 1.721 s | 1.723 s | 0.0235 s |
2525

2626
## 📈 Visual Comparison
2727

@@ -60,7 +60,7 @@ xychart-beta
6060
title "Build Time Comparison"
6161
x-axis ["Build_TUnit", "Build_NUnit", "Build_MSTest", "Build_xUnit3"]
6262
y-axis "Time (s)" 0 --> 3
63-
bar [2.066, 1.649, 1.72, 1.63]
63+
bar [2.098, 1.681, 1.799, 1.721]
6464
```
6565

6666
---
@@ -69,4 +69,4 @@ xychart-beta
6969
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7070
:::
7171

72-
*Last generated: 2025-12-05T00:30:14.387Z*
72+
*Last generated: 2025-12-06T00:27:43.843Z*

‎docs/docs/benchmarks/DataDrivenTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 3
77
# DataDrivenTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 490.86 ms | 491.42 ms | 5.726 ms |
20-
| NUnit | 4.4.0 | 517.92 ms | 517.35 ms | 7.205 ms |
21-
| MSTest | 4.0.2 | 489.16 ms | 487.76 ms | 10.467 ms |
22-
| xUnit3 | 3.2.1 | 566.85 ms | 565.79 ms | 6.875 ms |
23-
| **TUnit (AOT)** | 1.3.25 | 23.98 ms | 23.96 ms | 0.185 ms |
19+
| **TUnit** | 1.3.25 | 526.18 ms | 525.63 ms | 9.733 ms |
20+
| NUnit | 4.4.0 | 686.15 ms | 686.99 ms | 11.238 ms |
21+
| MSTest | 4.0.2 | 681.30 ms | 682.63 ms | 10.257 ms |
22+
| xUnit3 | 3.2.1 | 686.97 ms | 687.69 ms | 8.584 ms |
23+
| **TUnit (AOT)** | 1.3.25 | 25.63 ms | 25.66 ms | 0.203 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "DataDrivenTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 681
62-
bar [490.86, 517.92, 489.16, 566.85, 23.98]
61+
y-axis "Time (ms)" 0 --> 825
62+
bar [526.18, 686.15, 681.3, 686.97, 25.63]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.386Z*
75+
*Last generated: 2025-12-06T00:27:43.842Z*

‎docs/docs/benchmarks/MassiveParallelTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 4
77
# MassiveParallelTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 602.6 ms | 603.7 ms | 4.06 ms |
20-
| NUnit | 4.4.0 | 1,191.7 ms | 1,192.7 ms | 6.96 ms |
21-
| MSTest | 4.0.2 | 2,973.0 ms | 2,971.9 ms | 10.85 ms |
22-
| xUnit3 | 3.2.1 | 3,065.1 ms | 3,064.9 ms | 7.78 ms |
23-
| **TUnit (AOT)** | 1.3.25 | 131.1 ms | 131.1 ms | 0.51 ms |
19+
| **TUnit** | 1.3.25 | 608.9 ms | 607.5 ms | 5.48 ms |
20+
| NUnit | 4.4.0 | 1,220.5 ms | 1,222.7 ms | 9.63 ms |
21+
| MSTest | 4.0.2 | 3,006.2 ms | 3,002.3 ms | 10.16 ms |
22+
| xUnit3 | 3.2.1 | 3,087.9 ms | 3,085.6 ms | 11.40 ms |
23+
| **TUnit (AOT)** | 1.3.25 | 132.7 ms | 132.7 ms | 0.62 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "MassiveParallelTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 3679
62-
bar [602.6, 1191.7, 2973, 3065.1, 131.1]
61+
y-axis "Time (ms)" 0 --> 3706
62+
bar [608.9, 1220.5, 3006.2, 3087.9, 132.7]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.386Z*
75+
*Last generated: 2025-12-06T00:27:43.842Z*

‎docs/docs/benchmarks/MatrixTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 5
77
# MatrixTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 564.33 ms | 564.92 ms | 3.395 ms |
20-
| NUnit | 4.4.0 | 1,572.46 ms | 1,570.59 ms | 9.247 ms |
21-
| MSTest | 4.0.2 | 1,525.48 ms | 1,525.44 ms | 8.892 ms |
22-
| xUnit3 | 3.2.1 | 1,611.66 ms | 1,608.48 ms | 8.282 ms |
23-
| **TUnit (AOT)** | 1.3.25 | 80.17 ms | 80.12 ms | 0.488 ms |
19+
| **TUnit** | 1.3.25 | 562.41 ms | 563.18 ms | 3.971 ms |
20+
| NUnit | 4.4.0 | 1,574.15 ms | 1,574.97 ms | 5.369 ms |
21+
| MSTest | 4.0.2 | 1,534.37 ms | 1,529.84 ms | 10.972 ms |
22+
| xUnit3 | 3.2.1 | 1,618.66 ms | 1,618.07 ms | 8.934 ms |
23+
| **TUnit (AOT)** | 1.3.25 | 80.00 ms | 80.17 ms | 0.512 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "MatrixTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1934
62-
bar [564.33, 1572.46, 1525.48, 1611.66, 80.17]
61+
y-axis "Time (ms)" 0 --> 1943
62+
bar [562.41, 1574.15, 1534.37, 1618.66, 80]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.386Z*
75+
*Last generated: 2025-12-06T00:27:43.843Z*

‎docs/docs/benchmarks/ScaleTests.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 6
77
# ScaleTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,11 +16,11 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 520.51 ms | 521.26 ms | 3.185 ms |
20-
| NUnit | 4.4.0 | 601.98 ms | 601.95 ms | 6.523 ms |
21-
| MSTest | 4.0.2 | 614.97 ms | 617.66 ms | 8.666 ms |
22-
| xUnit3 | 3.2.1 | 606.12 ms | 605.35 ms | 9.916 ms |
23-
| **TUnit (AOT)** | 1.3.25 | 42.80 ms | 41.72 ms | 3.704 ms |
19+
| **TUnit** | 1.3.25 | 512.28 ms | 512.24 ms | 2.035 ms |
20+
| NUnit | 4.4.0 | 587.13 ms | 584.27 ms | 10.483 ms |
21+
| MSTest | 4.0.2 | 603.13 ms | 603.16 ms | 7.120 ms |
22+
| xUnit3 | 3.2.1 | 594.15 ms | 593.03 ms | 8.717 ms |
23+
| **TUnit (AOT)** | 1.3.25 | 42.78 ms | 43.30 ms | 2.821 ms |
2424

2525
## 📈 Visual Comparison
2626

@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "ScaleTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 738
62-
bar [520.51, 601.98, 614.97, 606.12, 42.8]
61+
y-axis "Time (ms)" 0 --> 724
62+
bar [512.28, 587.13, 603.13, 594.15, 42.78]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.387Z*
75+
*Last generated: 2025-12-06T00:27:43.843Z*

‎docs/docs/benchmarks/SetupTeardownTests.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 7
77
# SetupTeardownTests Benchmark
88

99
:::info Last Updated
10-
This benchmark was automatically generated on **2025-12-05** from the latest CI run.
10+
This benchmark was automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -16,10 +16,10 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
1616

1717
| Framework | Version | Mean | Median | StdDev |
1818
|-----------|---------|------|--------|--------|
19-
| **TUnit** | 1.3.25 | 572.8 ms | 573.2 ms | 3.27 ms |
20-
| NUnit | 4.4.0 | 1,155.8 ms | 1,157.2 ms | 9.08 ms |
21-
| MSTest | 4.0.2 | 1,128.3 ms | 1,126.0 ms | 5.78 ms |
22-
| xUnit3 | 3.2.1 | 1,202.4 ms | 1,203.1 ms | 6.16 ms |
19+
| **TUnit** | 1.3.25 | 571.1 ms | 570.1 ms | 5.82 ms |
20+
| NUnit | 4.4.0 | 1,138.6 ms | 1,138.2 ms | 5.49 ms |
21+
| MSTest | 4.0.2 | 1,112.2 ms | 1,111.1 ms | 8.69 ms |
22+
| xUnit3 | 3.2.1 | 1,197.6 ms | 1,196.2 ms | 5.01 ms |
2323
| **TUnit (AOT)** | 1.3.25 | NA | NA | NA |
2424

2525
## 📈 Visual Comparison
@@ -58,8 +58,8 @@ This benchmark was automatically generated on **2025-12-05** from the latest CI
5858
xychart-beta
5959
title "SetupTeardownTests Performance Comparison"
6060
x-axis ["TUnit", "NUnit", "MSTest", "xUnit3", "TUnit_AOT"]
61-
y-axis "Time (ms)" 0 --> 1443
62-
bar [572.8, 1155.8, 1128.3, 1202.4, 0]
61+
y-axis "Time (ms)" 0 --> 1438
62+
bar [571.1, 1138.6, 1112.2, 1197.6, 0]
6363
```
6464

6565
## 🎯 Key Insights
@@ -72,4 +72,4 @@ This benchmark compares TUnit's performance against NUnit, MSTest, xUnit3 using
7272
View the [benchmarks overview](/docs/benchmarks) for methodology details and environment information.
7373
:::
7474

75-
*Last generated: 2025-12-05T00:30:14.387Z*
75+
*Last generated: 2025-12-06T00:27:43.843Z*

‎docs/docs/benchmarks/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ sidebar_position: 1
77
# Performance Benchmarks
88

99
:::info Last Updated
10-
These benchmarks were automatically generated on **2025-12-05** from the latest CI run.
10+
These benchmarks were automatically generated on **2025-12-06** from the latest CI run.
1111

1212
**Environment:** Ubuntu Latest • .NET SDK 10.0.100
1313
:::
@@ -80,4 +80,4 @@ These benchmarks run automatically daily via [GitHub Actions](https://github.com
8080
Each benchmark runs multiple iterations with statistical analysis to ensure accuracy. Results may vary based on hardware and test characteristics.
8181
:::
8282

83-
*Last generated: 2025-12-05T00:30:14.388Z*
83+
*Last generated: 2025-12-06T00:27:43.843Z*

‎docs/static/benchmarks/AsyncTests.json‎

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"timestamp": "2025-12-05T00:30:14.385Z",
2+
"timestamp": "2025-12-06T00:27:43.842Z",
33
"category": "AsyncTests",
44
"environment": {
55
"benchmarkDotNetVersion": "BenchmarkDotNet v0.15.8, Linux Ubuntu 24.04.3 LTS (Noble Numbat)",
@@ -10,42 +10,42 @@
1010
{
1111
"Method": "TUnit",
1212
"Version": "1.3.25",
13-
"Mean": "589.8 ms",
14-
"Error": "5.23 ms",
15-
"StdDev": "4.89 ms",
16-
"Median": "589.3 ms"
13+
"Mean": "579.9 ms",
14+
"Error": "2.98 ms",
15+
"StdDev": "2.78 ms",
16+
"Median": "579.1 ms"
1717
},
1818
{
1919
"Method": "NUnit",
2020
"Version": "4.4.0",
21-
"Mean": "688.2 ms",
22-
"Error": "11.65 ms",
23-
"StdDev": "10.89 ms",
24-
"Median": "688.9 ms"
21+
"Mean": "666.7 ms",
22+
"Error": "8.02 ms",
23+
"StdDev": "6.70 ms",
24+
"Median": "666.3 ms"
2525
},
2626
{
2727
"Method": "MSTest",
2828
"Version": "4.0.2",
29-
"Mean": "645.1 ms",
30-
"Error": "11.95 ms",
31-
"StdDev": "10.60 ms",
32-
"Median": "643.0 ms"
29+
"Mean": "639.7 ms",
30+
"Error": "9.57 ms",
31+
"StdDev": "8.95 ms",
32+
"Median": "634.5 ms"
3333
},
3434
{
3535
"Method": "xUnit3",
3636
"Version": "3.2.1",
37-
"Mean": "724.5 ms",
38-
"Error": "10.93 ms",
39-
"StdDev": "9.69 ms",
40-
"Median": "723.6 ms"
37+
"Mean": "730.4 ms",
38+
"Error": "14.53 ms",
39+
"StdDev": "13.59 ms",
40+
"Median": "732.8 ms"
4141
},
4242
{
4343
"Method": "TUnit_AOT",
4444
"Version": "1.3.25",
45-
"Mean": "124.7 ms",
46-
"Error": "0.26 ms",
47-
"StdDev": "0.25 ms",
48-
"Median": "124.7 ms"
45+
"Mean": "124.3 ms",
46+
"Error": "0.64 ms",
47+
"StdDev": "0.60 ms",
48+
"Median": "124.3 ms"
4949
}
5050
]
5151
}

0 commit comments

Comments
 (0)