perf: reduce allocations via Span<T>, ArrayPool, and avoiding defensive copies

[thomhurst]

Parent Epic

Part of #4159 - Performance Optimization: Hot Path Improvements

Problem

Various hot paths allocate intermediate arrays, create defensive copies, or use heap allocations where stack/pool alternatives exist.

Evidence

File	Lines	Pattern	Suggested Fix
TUnit.Engine/Discovery/ReflectionTestDataCollector.cs	112, 125-126	Task<List<T>>[] + merge allocations	Use ArrayPool<Task>
TUnit.Engine/Discovery/ReflectionTestDataCollector.cs	140	new List<TestMetadata>(_discoveredTests)	Return IReadOnlyList
TUnit.Engine/Helpers/DataUnwrapper.cs	41	.Select(p => p.Type).ToArray()	Pre-sized array + loop
TUnit.Engine/Extensions/TestExtensions.cs	37, 46, 57, 208	Multiple .Select().ToArray() chains	Manual loops
TUnit.Engine/Building/ReflectionMetadataBuilder.cs	30-31, 78-79	.Select().ToArray() for parameters	Pre-sized array + loop
TUnit.Engine/Discovery/ReflectionAttributeExtractor.cs	125, 140, 164, 179, 192, 221	Multiple .ToArray() calls	Reuse lists or use ArrayPool

Suggested Approach

Replace LINQ with pre-sized arrays:

// Before: LINQ allocation
var paramTypes = expectedParameters.Select(p => p.Type).ToArray();

// After: Pre-sized array
var paramTypes = new Type[expectedParameters.Length];
for (int i = 0; i < expectedParameters.Length; i++)
{
    paramTypes[i] = expectedParameters[i].Type;
}

Use ArrayPool for temporary buffers:

// Before
var tasks = new Task<List<TestMetadata>>[assemblies.Count];

// After
var tasks = ArrayPool<Task<List<TestMetadata>>>.Shared.Rent(assemblies.Count);
try
{
    // ... use tasks
}
finally
{
    ArrayPool<Task<List<TestMetadata>>>.Shared.Return(tasks);
}

Avoid defensive copies:

// Before: Creates copy
return new List<TestMetadata>(_discoveredTests);

// After: Return read-only view
return _discoveredTests.AsReadOnly();

Verification

1. Use dotnet-trace with allocation tracking
2. Compare object allocation counts before/after at 10k scale
3. Profile with SpeedScope to verify no new hot spots introduced

Risks

• Low-medium risk
• ArrayPool requires careful return (use try/finally)
• Removing defensive copies requires verifying callers don't mutate

Priority

P2 - Mixed complexity, broad impact across many files

[wallstop]

I use System Array Pool and array-pool like things, one thing that has helped is a value-typed disposer (some people don't like this pattern, I think it's great)

Reference impl: https://github.com/wallstop/unity-helpers/blob/a2b408ff0ab0b932d8520c17dd9f8497ea90ed28/Runtime/Utils/Buffers.cs#L1320C38-L1320C41

This lets you do something like using var arrayPoolResource = SystemArrayPool.Get(arraySize), and the using auto-cleans up/returns the array to the pool. No need for tricky try/finally, and no heap allocations.

[wallstop]

Also, just in case you aren't aware, System.ArrayPool returns closest-power-of-two (to my knowledge) array sizes - if you ask for an array of size "n", you will get back an array that is at least size "n", but could be n+1, n+2, n+100, etc. If your code expects precise array sizes, this will break. Also, if you are renting arrays for types that are managed, you need to be careful about clearing the contents, as if you don't, references will exist and leak memory (unless you ensure all return calls clear the arrays).

[TimothyMakkison]

I'm a huge fan of ValueListBuilder.

You can even stack allocate reference types new ValueListBuilder([null, null, null, null]);