Binding duplicated handle to ThreadPool fails?

[danmoseley]

From @poizan42 on June 13, 2018 4:6

When I run the following code I get a System.ArgumentException: ''handle' has already been bound to the thread pool, or was not opened for asynchronous I/O.'

This seems weird since it is a copy of the handle bound and it is definitely opened for async I/O. From a glance it looks like the error ultimately comes from CreateIoCompletionPort, but that is weird because the documentation implies that you can use DuplicateHandle to share a handle registered to an IO completion port:

It is best not to share a file handle associated with an I/O completion port by using either handle inheritance or a call to the DuplicateHandle function. Operations performed with such duplicate handles generate completion notifications. Careful consideration is advised.

Is the framework doing something funky here, or is the reality more complicated than the documentation for CreateIoCompletionPort would lead you to think?

using Microsoft.Win32.SafeHandles;
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.IO.Pipes;
using System.Runtime.InteropServices;
using System.Security.Principal;

namespace NamedPipeDuplicateHandleTestCore
{
  class Program
  {
    [DllImport("kernel32.dll", EntryPoint = "DuplicateHandle", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    private static extern bool _DuplicateHandleSU(IntPtr hSourceProcessHandle,
       SafeHandle hSourceHandle, IntPtr hTargetProcessHandle, out IntPtr lpTargetHandle,
       uint dwDesiredAccess, [MarshalAs(UnmanagedType.Bool)] bool bInheritHandle, uint dwOptions);

    public static IntPtr DuplicateHandle(SafeHandle hSourceHandle, IntPtr hTargetProcessHandle)
    {
      const int DUPLICATE_SAME_ACCESS = 0x00000002;
      IntPtr targetHandle;
      if (!_DuplicateHandleSU(Process.GetCurrentProcess().Handle, hSourceHandle, hTargetProcessHandle, out targetHandle,
        0, false, DUPLICATE_SAME_ACCESS))
      {
        throw new Win32Exception(Marshal.GetLastWin32Error());
      }
      return targetHandle;
    }

    static void Main(string[] args)
    {
      string pipename = "foopipe" + Guid.NewGuid().ToString("N");
      var pipeServer = new NamedPipeServerStream(pipename, PipeDirection.InOut, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous,
        4096, 4096);
      pipeServer.BeginWaitForConnection(ar => { pipeServer.EndWaitForConnection(ar); }, null);

      IntPtr handle2;
      using (var clientStream1 = new NamedPipeClientStream(".", pipename, PipeDirection.InOut, PipeOptions.Asynchronous, TokenImpersonationLevel.None,
        System.IO.HandleInheritability.None))
      {
        clientStream1.Connect();
        handle2 = DuplicateHandle(clientStream1.SafePipeHandle, Process.GetCurrentProcess().Handle);
      }
      var clientStream2 = new NamedPipeClientStream(PipeDirection.InOut, true, true, new SafePipeHandle(handle2, true));
      pipeServer.WriteAsync(new byte[] { 42 }, 0, 1);
      Console.WriteLine(clientStream2.ReadByte());
    }
  }
}

This is tested on dotnet core 2.1.30 and Windows 10.0.17686.1003.

Copied from original issue: dotnet/coreclr#18450

[antonfirsov]

@poizan42 @JeremyKuhne although I was unable to find any statement in Win32 API documentation, my understanding is that it's not possible to bind a duplicate handle to a second IOCP port, therefore async operations on clientStream2 will always fail on Windows.

This is a known behavior for sockets (see #1760).

[carlossanlop]

Triage: It would be useful to try doing straight P/Invokes and confirm if handle duplication can be done as described.

[adamsitnik]

While working on #58381 I wanted to use NamedPipeServerStream and NamedPipeClientStream to write some tests for non-seekable files (pipes in this case).

Example:

async Task<(SafeFileHandle readHandle, SafeFileHandle writeHandle)> GetNamedPipeHandlesAsync()
{
    string name = FileSystemTest.GetNamedPipeServerStreamName();

    var server = new NamedPipeServerStream(name, PipeDirection.In, -1, PipeTransmissionMode.Byte, PipeOptions);
    var client = new NamedPipeClientStream(".", name, PipeDirection.Out, PipeOptions);

    await Task.WhenAll(server.WaitForConnectionAsync(), client.ConnectAsync());

    bool isAsync = (PipeOptions & PipeOptions.Asynchronous) != 0;
    return (GetFileHandle(server, isAsync), GetFileHandle(client, isAsync));
}

private static SafeFileHandle GetFileHandle(PipeStream pipeStream, bool isAsync)
{
    var serverHandle = new SafeFileHandle(pipeStream.SafePipeHandle.DangerousGetHandle(), ownsHandle: true);
    pipeStream.SafePipeHandle.SetHandleAsInvalid();
    return serverHandle;
}

The problem is that when I use PipeOptions.Asynchronous and wait for the pipes to be connected, the async pipe handle gets bound to Thread Pool:

runtime/src/libraries/System.IO.Pipes/src/System/IO/Pipes/PipeStream.Windows.cs

Lines 256 to 260 in e4b4666

	private void InitializeAsyncHandle(SafePipeHandle handle)
	{
	// If the handle is of async type, bind the handle to the ThreadPool so that we can use
	// the async operations (it's needed so that our native callbacks get called).
	_threadPoolBinding = ThreadPoolBoundHandle.BindHandle(handle);

and when I create a copy of it and try to re-use for async IO, I hit this condition:

runtime/src/coreclr/System.Private.CoreLib/src/System/Threading/ClrThreadPoolBoundHandle.Windows.cs

Lines 33 to 34 in 57bfe47

	if (ex.HResult == HResults.E_INVALIDARG) // Handle already bound or sync handle
	throw new ArgumentException(SR.Argument_AlreadyBoundOrSyncHandle, nameof(handle));

Minimal repro case using FileStream and .NET 5:

Details

using Microsoft.Win32.SafeHandles;
using System;
using System.IO;
using System.Threading.Tasks;

namespace asyncHandles
{
    class Program
    {
        static async Task Main(string[] args)
        {
            string filePath = Path.Combine(Path.GetTempPath(), Path.GetTempFileName());

            try
            {
                using FileStream fs1 = new FileStream(filePath, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write, 1, FileOptions.Asynchronous);

                await fs1.WriteAsync(new byte[] { 1 }); // initializes SafeFileHandle.ThreadPoolBinding

                SafeFileHandle duplicate = new(fs1.SafeFileHandle.DangerousGetHandle(), ownsHandle: false);
                using FileStream fs2 = new FileStream(duplicate, FileAccess.Write, 1, isAsync: true);

                await fs2.WriteAsync(new byte[] { 1 }); // throws
            }
            finally
            {
                if (File.Exists(filePath))
                {
                    File.Delete(filePath);
                }
            }
        }
    }
}

Unhandled exception. System.ArgumentException: Handle does not support asynchronous operations. The parameters to the FileStream constructor may need to be changed to indicate that the handle was opened synchronously (that is, it was not opened for overlapped I/O). (Parameter 'handle')
 ---> System.ArgumentException: 'handle' has already been bound to the thread pool, or was not opened for asynchronous I/O. (Parameter 'handle')
   at System.Threading.ThreadPoolBoundHandle.BindHandleCore(SafeHandle handle)
   at System.Threading.ThreadPoolBoundHandle.BindHandle(SafeHandle handle)
   at System.IO.FileStream.InitFromHandleImpl(SafeFileHandle handle, Boolean useAsyncIO)
   --- End of inner exception stack trace ---
   at System.IO.FileStream.InitFromHandleImpl(SafeFileHandle handle, Boolean useAsyncIO)
   at System.IO.FileStream.ValidateAndInitFromHandle(SafeFileHandle handle, FileAccess access, Int32 bufferSize, Boolean isAsync)
   at System.IO.FileStream..ctor(SafeFileHandle handle, FileAccess access, Int32 bufferSize, Boolean isAsync)
   at asyncHandles.Program.Main(String[] args) in D:\projects\repros\asyncHandles\Program.cs:line 21
   at asyncHandles.Program.<Main>(String[] args)

I've used reflection as un ugly workaround hack:

ThreadPoolBoundHandle threadPoolBinding = (ThreadPoolBoundHandle)typeof(PipeStream).GetField("_threadPoolBinding", Reflection.BindingFlags.NonPublic | Reflection.BindingFlags.Instance).GetValue(pipeStream);
typeof(SafeFileHandle).GetProperty("ThreadPoolBinding", Reflection.BindingFlags.NonPublic | Reflection.BindingFlags.Instance).GetSetMethod(true).Invoke(serverHandle, new object[] { threadPoolBinding });

We should take a look whether it's possible to just get ThreadPoolBoundHandle that is assigned to given handle.

[raffaeler]

I also just had this issue because I am trying to attach a serial port handle to the FileStream.
I explicitly need the ThreadPoolBoundHandle to be able to receive the serial communication events but the SafeFileHandle already creates and hides it, therefore I can write to the serial port but I never get any event telling me how many bytes are available to read.

At the very end this means abandoning FileStream and being stuck in a lot of boilerplate code to read/write as it was a C app.