Improve SocketOutputTests

[halter73]

• This should increase reliability/determinism by removing timeouts.
  Intermittent CI failure WritesDontCompleteImmediatelyWhenTooManyBytesIncludingNonImmediateAreAlreadyPreCompleted #1002

[mikeharder]

I agree with changing the test methods to async Task and removing the 1-second timeouts. However, why is the SynchronousThreadPool required? Wouldn't the tests work just as well with an async threadpool?

[halter73]

@mikeharder We are verifying the tasks are in the correct state immediately after WriteAsync is called or the uv_write callback is called depending on the test.

The default IThreadPool, LoggingThreadPool is used to complete these tasks which basically runs ThreadPool.QueueUserWorkItem(tcs => tcs.TrySetResult(null), tcs); with some additional casting and logging. Same goes for errors and cancellation.

This means that we can't verify the task gets completed or not without a timeout if we use the default threadpool. At that point we have to pick a consistent timeout which was 1 second before, but even that was not long enough on some CI runs. Most of the time, verifying task completion happens fairly quickly (but sometimes over a second apparently), but verifying task noncompletion will take the full lenght of the time.

[mikeharder]

If you need to verify non-completion, I agree it's better to use a SynchronousThreadPool than to wait an arbitrary period of time.

[mikeharder]