stub: optimize ThreadlessExecutor used for blocking calls

[njhill]

The ThreadlessExecutor currently used for blocking calls uses LinkedBlockingQueue which is relatively heavy both in terms of allocations and synchronization overhead (e.g. when compared to ConcurrentLinkedQueue). It accounts for ~10% of allocations and ~5% of allocated bytes per-call in the TransportBenchmark when using in-process transport with stats and tracing disabled.

Changing to use a ConcurrentLinkedQueue results in a ~5% speedup of that benchmark.

[dapengzhang0]

Maybe mark waitAndDrain() with @NotThreadSafe because there must not be two concurrent callers of it.

[njhill]

Sure, the class is only for internal use in an SPSC context.

[carl-mastrangelo]

One possible thing you could try is to make this SPSC. I have a POC (originally for SerializingExecutor) here: https://github.com/grpc/grpc-java/pull/3778/files

[njhill]

@carl-mastrangelo sure, I remember seeing that before, could be worth a try here too. I thought (possibly mistakenly) this would be a simpler change just to circumvent LinkedBlockingQueue which was the main goal.

[njhill]

Thanks @dapengzhang0 @carl-mastrangelo, have addressed the comments, PTAL

[dapengzhang0]

LGTM

[carl-mastrangelo]

@njhill Can you include your before and after JMH numbers for the commit? We typically include them when making performance optimizations.

[njhill]

@carl-mastrangelo I thought I had observed a bigger difference in the non-direct case in other runs, when the system was noisier. I know it's not a huge delta but there's a couple more similar changes I have in mind which cumulatively add up to maybe ~15% (to be confirmed!)

Before:

Benchmark                         (direct)  (transport)  Mode  Cnt      Score     Error  Units
TransportBenchmark.unaryCall1024      true    INPROCESS  avgt   60   1877.339 ±  46.309  ns/op
TransportBenchmark.unaryCall1024     false    INPROCESS  avgt   60  12680.525 ± 208.684  ns/op

After:

Benchmark                         (direct)  (transport)  Mode  Cnt      Score     Error  Units
TransportBenchmark.unaryCall1024      true    INPROCESS  avgt   60   1779.188 ±  36.769  ns/op
TransportBenchmark.unaryCall1024     false    INPROCESS  avgt   60  12532.470 ± 238.271  ns/op

This is with the following changes to default config:

• Set tracingEnabled and statsEnabled to false in channel and server builders
• Bumped forks 1 -> 2, iterations 10 -> 30 and changed mode to AverageTime

[carl-mastrangelo]

LGTM

[carl-mastrangelo]

@njhill merged, thanks!

[dapengzhang0]

Thanks a lot for your PR @njhill