Fix PipeStream leak on Windows when pipe is disposed with a pending operation

[stephentoub]

The same pattern occurs in a few places.

Repro:

using System.IO.Pipes;

_ = Task.Run(() =>
{
    while (true)
    {
        Thread.Sleep(1000);
        Console.WriteLine($"{Environment.WorkingSet:N0} bytes");
    }
});

var buffer = new byte[1];
for (int i = 0; ; i++)
{
    Task readTask;
    string pipeName = Guid.NewGuid().ToString("N");

    using (var server = new NamedPipeServerStream(pipeName, PipeDirection.In, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous))
    using (var client = new NamedPipeClientStream(".", pipeName, PipeDirection.Out, PipeOptions.Asynchronous))
    {
        await Task.WhenAll(client.ConnectAsync(), server.WaitForConnectionAsync());

        readTask = server.ReadAsync(buffer, 0, buffer.Length);
    }

    if (i % 100_000 == 0)
    {
        Console.WriteLine($"Iteration {i} completed.");
        GC.Collect();
    }
}

Before:

Iteration 0 completed.
44,621,824 bytes
57,716,736 bytes
65,609,728 bytes
75,116,544 bytes
80,695,296 bytes
91,246,592 bytes
Iteration 100000 completed.
99,852,288 bytes
109,580,288 bytes
119,209,984 bytes
123,969,536 bytes
129,781,760 bytes
143,015,936 bytes
148,815,872 bytes
Iteration 200000 completed.
154,836,992 bytes
164,868,096 bytes
176,857,088 bytes
181,932,032 bytes
191,127,552 bytes
202,846,208 bytes
Iteration 300000 completed.
210,046,976 bytes
221,147,136 bytes
228,446,208 bytes
235,446,272 bytes
246,456,320 bytes
256,700,416 bytes
Iteration 400000 completed.

After:

Iteration 0 completed.
36,007,936 bytes
37,425,152 bytes
37,478,400 bytes
37,486,592 bytes
37,494,784 bytes
37,494,784 bytes
Iteration 100000 completed.
36,401,152 bytes
36,442,112 bytes
36,442,112 bytes
36,442,112 bytes
36,442,112 bytes
36,425,728 bytes
Iteration 200000 completed.
36,425,728 bytes
36,483,072 bytes
36,614,144 bytes
35,520,512 bytes
35,520,512 bytes
35,520,512 bytes
Iteration 300000 completed.
34,996,224 bytes
34,996,224 bytes
34,996,224 bytes
35,033,088 bytes
35,033,088 bytes
35,033,088 bytes
Iteration 400000 completed.

[Copilot]

Pull Request Overview

This PR fixes a pipe leak on Windows by ensuring that the pipe state is set to closed before disposing shared resources and by improving the reuse logic for pending operations.

• In PipeStream.cs, the pipe’s state is set to closed earlier in the disposal process to avoid races.
• In PipeStream.Windows.cs, the reusability logic now calls source.Dispose() instead of _preallocatedOverlapped.Dispose() to better handle concurrent operations.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
src/libraries/System.IO.Pipes/src/System/IO/Pipes/PipeStream.cs	Updates the disposal order by marking the pipe as closed before further disposal.
src/libraries/System.IO/Pipes/src/System/IO/Pipes/PipeStream.Windows.cs	Adjusts the reuse logic to dispose of the whole source instance when a concurrent operation is detected.

Comments suppressed due to low confidence (2)

src/libraries/System.IO.Pipes/src/System/IO/Pipes/PipeStream.cs:150

• The redundant assignment of _state to PipeState.Closed has been removed by moving the state change earlier in the method. Please confirm that all code paths relying on _state consider the new disposal order.

_state = PipeState.Closed;

src/libraries/System.IO/Pipes/src/System/IO/Pipes/PipeStream.Windows.cs:272

• Switching from calling _preallocatedOverlapped.Dispose() to source.Dispose() may have additional side effects if source.Dispose() encompasses extra disposal logic. Verify that invoking source.Dispose() here safely and correctly releases all related resources.

source.Dispose();

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

[adamsitnik]

LGTM. thanks for adding a very detailed explanation @stephentoub 👍

[Copilot]

Pull Request Overview

This PR fixes resource leaks by ensuring pending operations on disposed pipes are cleaned up correctly on Windows and Unix. Key changes include:

• Updating TryToReuse methods to call Dispose() on overlapped/task sources and clear reusable fields when closed.
• Marking the pipe as closed earlier in PipeStream.Dispose to prevent races during disposal.
• Adjusting Unix implementation of DisposeCore to cancel internal tokens only during managed disposal.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/.../SafeFileHandle.OverlappedValueTaskSource.Windows.cs	TryToReuse now calls Dispose() on the source and clears pool on close
src/.../PipeStream.cs	Moved _state = PipeState.Closed before calling into DisposeCore to avoid races
src/.../PipeStream.Windows.cs	Updated TryToReuse to dispose pooled PipeValueTaskSource instances when closed
src/.../NamedPipeServerStream.Windows.cs	Similar TryToReuse update for connection value task sources
src/.../NamedPipeServerStream.Unix.cs	Removed if (disposing) guard around _internalTokenSource.Cancel()

Comments suppressed due to low confidence (2)

src/libraries/System.IO.Pipes/src/System/IO/Pipes/NamedPipeServerStream.Unix.cs:140

• The removal of the if (disposing) guard causes _internalTokenSource.Cancel() to run during finalization, which may race with managed disposal or throw on already-disposed objects. Restore the if (disposing) check to ensure cancellation only occurs during explicit disposal.

if (State != PipeState.Closed)

src/libraries/System.IO.Pipes/src/System/IO/Pipes/PipeStream.cs:150

• By setting _state outside of the if (disposing) check, the finalizer path (disposing == false) will also mark the stream closed, potentially hiding cleanup errors or causing incorrect behavior. Consider moving this assignment inside the if (disposing) block or explicitly documenting that it applies to both managed and unmanaged disposal paths.

_state = PipeState.Closed;

[stephentoub]

/backport to release/9.0-staging

[github-actions]

Started backporting to release/9.0-staging: https://github.com/dotnet/runtime/actions/runs/15370998142

[stephentoub]

/backport to release/8.0-staging

[github-actions]

Started backporting to release/8.0-staging: https://github.com/dotnet/runtime/actions/runs/15370999069