gRPC retry breaks tracing

[mwuertinger]

I'm not sure if this is a gRPC or an OpenCensus issue. If this is the wrong place to report this, please let me know and I will open an issue for gRPC.

What version of OpenCensus are you using?

0.19.1

What version of gRPC are you using?

1.18.0

What JVM are you using (java -version)?

openjdk version "1.8.0_212"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode)

What did you do?

I'm constructing a gRPC channel like below and then use this channel to make calls to a remote service.

ManagedChannelBuilder
	.forAddress(host, port)
	.defaultLoadBalancingPolicy("round_robin")
	.nameResolverFactory(DnsNameResolverProvider.asFactory())
	.idleTimeout(30, TimeUnit.SECONDS)
	.enableRetry()
	.build();

What did you expect to see?

I expect that outgoing calls create a span and that the grpc-trace-bin header is set so that the remote service can pick up the tracing information.

What did you see instead?

No span is created on the client side and the grpc-trace-bin header is missing.

Additional context

Removing the enableRetry() fixes the problem and tracing works as expected.

I'm using the StackdriverTraceExporter on both client and server.

[songy23]

This is a known issue that enabling retry for gRPC will disable OpenCensus. See grpc/grpc-java#5510 (comment):

However, there is a hack: enabling retry will disable census as a side effect for the current version of grpc library (Note: This behavior is subject to change in future releases).

I'm not sure whether this is intentional. /cc @dapengzhang0 @zhangkun83 is there a way to enable opencensus while retry is used?

[dapengzhang0]

@songy23 We are not fixing this issue in the current quarter. Would you like to prioritize it?

[songy23]

Nothing urgent from our side. @mwuertinger do you have any specific use case where you want to enable both retry and OpenCensus?

[mwuertinger]

Thanks for your quick and helpful response @songy23.
I talked to my team and we don't have a strong reason for retries at the moment, so we're just gonna turn it off in favor of observability.