Skip to content

feat(grpc-storage): Add max_recv_msg_size_mib config for gRPC storage client#8576

Merged
yurishkuro merged 3 commits into
jaegertracing:mainfrom
yurishkuro:recv-msg-size
May 14, 2026
Merged

feat(grpc-storage): Add max_recv_msg_size_mib config for gRPC storage client#8576
yurishkuro merged 3 commits into
jaegertracing:mainfrom
yurishkuro:recv-msg-size

Conversation

@yurishkuro

Copy link
Copy Markdown
Member

The gRPC default client receive limit is 4 MiB. When a storage backend returns large traces this results in:

  grpc: received message larger than max (N vs. 4194304)

OTel's configgrpc.ClientConfig does not expose a client-side receive size limit (only ServerConfig has max_recv_msg_size_mib), so we add it directly to the gRPC storage Config and wire it via grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...)).

Closes #7670

AI Usage in this PR (choose one)

  • Heavy: AI generated most or all of the code changes

… client

The gRPC default client receive limit is 4 MiB. When a storage backend
returns large traces this results in:
  grpc: received message larger than max (N vs. 4194304)

OTel's configgrpc.ClientConfig does not expose a client-side receive
size limit (only ServerConfig has max_recv_msg_size_mib), so we add it
directly to the gRPC storage Config and wire it via
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...)).

Closes jaegertracing#7670

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
Copilot AI review requested due to automatic review settings May 14, 2026 16:43
@yurishkuro yurishkuro requested a review from a team as a code owner May 14, 2026 16:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a gRPC storage client configuration knob to raise the default 4 MiB receive-message limit, preventing ResourceExhausted errors when backends return large trace payloads.

Changes:

  • Introduces max_recv_msg_size_mib in the gRPC storage Config.
  • Wires the setting into the gRPC dial via grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(...)).
  • Adds basic tests covering default behavior and presence of the added dial option.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
internal/storage/v2/grpc/factory.go Appends a default call option to increase max receive message size when configured.
internal/storage/v2/grpc/config.go Adds MaxRecvMsgSizeMiB to the storage gRPC client configuration.
internal/storage/v2/grpc/factory_test.go Adds a unit test intended to verify the dial option is added when configured.
internal/storage/v2/grpc/config_test.go Asserts the default config leaves the new setting unset (uses gRPC default).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/storage/v2/grpc/factory.go
Comment thread internal/storage/v2/grpc/factory_test.go
Comment thread internal/storage/v2/grpc/factory.go
…sage size

On 32-bit platforms int is 32 bits, so MaxRecvMsgSizeMiB*1024*1024
overflows for values >= 2048. Compute in int64 and clamp to math.MaxInt
before passing to grpc.MaxCallRecvMsgSize(int).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
@codecov

codecov Bot commented May 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.56%. Comparing base (8211b31) to head (9f1d20f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8576   +/-   ##
=======================================
  Coverage   96.56%   96.56%           
=======================================
  Files         330      330           
  Lines       17356    17363    +7     
=======================================
+ Hits        16759    16766    +7     
  Misses        450      450           
  Partials      147      147           
Flag Coverage Δ
badger_direct 8.94% <0.00%> (-0.01%) ⬇️
badger_e2e 1.04% <0.00%> (-0.01%) ⬇️
cassandra-4.x-direct-manual 14.55% <0.00%> (-0.02%) ⬇️
cassandra-4.x-e2e-auto 1.03% <0.00%> (-0.01%) ⬇️
cassandra-4.x-e2e-manual 1.03% <0.00%> (-0.01%) ⬇️
cassandra-5.x-direct-manual 14.55% <0.00%> (-0.02%) ⬇️
cassandra-5.x-e2e-auto 1.03% <0.00%> (-0.01%) ⬇️
cassandra-5.x-e2e-manual 1.03% <0.00%> (-0.01%) ⬇️
clickhouse-direct 8.97% <0.00%> (-0.01%) ⬇️
clickhouse-e2e 1.16% <0.00%> (-0.01%) ⬇️
elasticsearch-6.x-direct 16.87% <0.00%> (-0.02%) ⬇️
elasticsearch-7.x-direct 16.91% <0.00%> (-0.02%) ⬇️
elasticsearch-8.x-direct 17.05% <0.00%> (-0.02%) ⬇️
elasticsearch-8.x-e2e 1.04% <0.00%> (-0.01%) ⬇️
elasticsearch-9.x-e2e 1.04% <0.00%> (-0.01%) ⬇️
grpc_direct 7.90% <14.28%> (+<0.01%) ⬆️
grpc_e2e 1.04% <0.00%> (-0.01%) ⬇️
kafka-3.x-v2 1.04% <0.00%> (-0.01%) ⬇️
memory_v2 1.04% <0.00%> (-0.01%) ⬇️
opensearch-1.x-direct 16.95% <0.00%> (-0.02%) ⬇️
opensearch-2.x-direct 16.95% <0.00%> (-0.02%) ⬇️
opensearch-2.x-e2e 1.04% <0.00%> (-0.01%) ⬇️
opensearch-3.x-e2e 1.04% <0.00%> (-0.01%) ⬇️
query 1.04% <0.00%> (-0.01%) ⬇️
tailsampling-processor 0.55% <0.00%> (-0.01%) ⬇️
unittests 94.84% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

A negative value silently passes the > 0 guard and falls through with
the gRPC default, giving the user no indication of misconfiguration.
Reject it explicitly at factory creation time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Yuri Shkuro <github@ysh.us>
@yurishkuro yurishkuro merged commit 41bc9e4 into jaegertracing:main May 14, 2026
73 checks passed
@yurishkuro yurishkuro deleted the recv-msg-size branch May 14, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Support max_recv_msg_size_mib for gRPC storage client

3 participants