Skip to content

fix: use correct slice index for Bedrock tool result coalescing - #2237

Merged
nacx merged 4 commits into
envoyproxy:mainfrom
Aias00:fix/bedrock-merge-index
Jun 23, 2026
Merged

fix: use correct slice index for Bedrock tool result coalescing#2237
nacx merged 4 commits into
envoyproxy:mainfrom
Aias00:fix/bedrock-merge-index

Conversation

@Aias00

@Aias00 Aias00 commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Description

Fix two bugs in the Anthropic-to-Bedrock translator message merging logic at internal/translator/anthropic_awsbedrock.go.

Bug 1: Wrong slice index (original report)

When system messages are present in the request body, promoteAnthropicSystemMessagesToParam filters them out, producing a shorter messages slice. The coalescing loop was using body.Messages[i+1] (the original unfiltered slice) instead of messages[i+1] (the filtered slice), causing index misalignment that could read wrong messages or panic with out-of-bounds access.

Bug 2: Coalescing logic was unreachable (root cause)

Tool result messages have Role: "user", so they match case anthropicschema.MessageRoleUser and go through convertUserMessage() — which handles tool results individually but never coalesces consecutive ones. The coalescing logic in the default branch was dead code for tool results since they never reach that branch.

Changes

  • Moved the hasToolResult check + coalescing loop into the MessageRoleUser case branch
  • Non-tool-result user messages still go through convertUserMessage()
  • Changed body.Messages[i+1]messages[i+1] to use the correct filtered slice
  • Simplified the default branch to just return an error for unexpected roles
  • Updated existing test to expect coalesced output (3 messages instead of 4)
  • Added regression test with system messages to verify index alignment

Testing

  • All existing translator tests pass
  • New TestAnthropicToAWSBedrockTranslator_RequestBody_ToolResultMessagesWithSystemMessages verifies the fix works when system messages are present alongside consecutive tool results

@Aias00
Aias00 requested a review from a team as a code owner June 16, 2026 11:14
Copilot AI review requested due to automatic review settings June 16, 2026 11:14
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jun 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR fixes tool_result message coalescing in the Anthropic→AWS Bedrock translator (especially when Anthropic system messages are promoted/filtered), and updates tests to match the new behavior.

Changes:

  • Coalesce consecutive Anthropic tool_result messages into a single Bedrock message content array.
  • Fix coalescing loop to iterate over the filtered messages slice (avoids index misalignment when system messages are promoted).
  • Add a regression test covering system-message promotion + tool_result coalescing; update .gitignore for worktree directories.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File Description
internal/translator/anthropic_awsbedrock.go Fixes coalescing to use the filtered messages slice and properly handles tool_result within the user role path.
internal/translator/anthropic_awsbedrock_test.go Updates expectations for coalesced tool results and adds a regression test with system-message promotion.
.gitignore Ignores local .worktrees/ directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1177 to +1181
// Verify system prompt was promoted.
require.NotNil(t, bedrockReq.System)
require.Len(t, bedrockReq.System, 1)
assert.Contains(t, *bedrockReq.System[0].Text, "You are a helpful assistant.")
assert.Contains(t, *bedrockReq.System[0].Text, "Always be concise.")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about test brittleness. However, promoteAnthropicSystemMessagesToParam uses strings.Join to concatenate all system message texts into a single string, then creates a single SystemPrompt{Text: ...} entry. So Len == 1 is deterministic and correct — there will always be exactly one system block regardless of how many system messages were in the input. I kept the assertion tight to catch any future regression in the system message promotion logic.

@Aias00
Aias00 force-pushed the fix/bedrock-merge-index branch from 651d7a3 to 7566417 Compare June 16, 2026 11:17
@codecov-commenter

codecov-commenter commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 84.84%. Comparing base (3456009) to head (f22a9c3).

Files with missing lines Patch % Lines
internal/translator/anthropic_awsbedrock.go 92.85% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2237      +/-   ##
==========================================
+ Coverage   84.70%   84.84%   +0.14%     
==========================================
  Files         144      144              
  Lines       21250    21246       -4     
==========================================
+ Hits        17999    18026      +27     
+ Misses       2166     2137      -29     
+ Partials     1085     1083       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Aias00
Aias00 force-pushed the fix/bedrock-merge-index branch from 7566417 to fb02f9b Compare June 16, 2026 11:28
When system messages are present in the request body,
promoteAnthropicSystemMessagesToParam filters them out, producing a
shorter messages slice. The coalescing loop was using body.Messages[i+1]
(the original unfiltered slice) instead of messages[i+1] (the filtered
slice), causing index misalignment that could read wrong messages or
panic with out-of-bounds access.

Additionally, tool_result messages have Role user, so they entered
the MessageRoleUser branch and went through convertUserMessage() which
does NOT coalesce consecutive tool results. The coalescing logic in the
default branch was unreachable for this role. Moved the hasToolResult
check into the MessageRoleUser branch so consecutive tool_result
messages are properly coalesced.

Added tests for single tool_result (no coalescing), unexpected role
error path, and system message promotion with tool result coalescing.

Signed-off-by: liuhy <liuhongyu@apache.org>
@Aias00
Aias00 force-pushed the fix/bedrock-merge-index branch from eef59ce to 0db1e84 Compare June 19, 2026 09:03
Aias00 added a commit to Aias00/ai-gateway that referenced this pull request Jun 19, 2026
Codecov flagged the PR because the new Bedrock RequestBody branch left one real error path uncovered and one assistant error path that the code could never reach. The user-message image error is now covered directly, and assistant conversion no longer advertises an impossible error return.

Constraint: PR envoyproxy#2237 is blocked by Codecov patch coverage on internal/translator/anthropic_awsbedrock.go

Rejected: Add a synthetic assistant error test | convertAssistantMessage cannot currently fail, so such a test would require artificial production behavior

Confidence: high

Scope-risk: narrow

Directive: Do not reintroduce an error return for assistant conversion unless a real assistant block conversion can fail

Tested: go test ./internal/translator -coverprofile=/tmp/translator.after.cover

Tested: go test ./internal/translator -run 'TestAnthropicToAWSBedrockTranslator_RequestBody_(ToolResultMessages|ToolResultMessagesWithSystemMessages|SingleToolResultNotCoalesced|UnexpectedRole|UserArrayContent|UserArrayContentError|AssistantMessage)' -count=1

Tested: go vet ./internal/translator

Tested: make lint

Not-tested: go test ./... failed locally because port 6060 is occupied by amass, Docker Hub CRD fetch timed out, required e2e Docker image is missing, and Ollama model qwen3:1.7b is unavailable
Codecov flagged the PR because the new Bedrock RequestBody branch left one real error path uncovered and one assistant error path that the code could never reach. The user-message image error is now covered directly, and assistant conversion no longer advertises an impossible error return.

Constraint: PR envoyproxy#2237 is blocked by Codecov patch coverage on internal/translator/anthropic_awsbedrock.go

Rejected: Add a synthetic assistant error test | convertAssistantMessage cannot currently fail, so such a test would require artificial production behavior

Confidence: high

Scope-risk: narrow

Directive: Do not reintroduce an error return for assistant conversion unless a real assistant block conversion can fail

Tested: go test ./internal/translator -coverprofile=/tmp/translator.after.cover

Tested: go test ./internal/translator -run 'TestAnthropicToAWSBedrockTranslator_RequestBody_(ToolResultMessages|ToolResultMessagesWithSystemMessages|SingleToolResultNotCoalesced|UnexpectedRole|UserArrayContent|UserArrayContentError|AssistantMessage)' -count=1

Tested: go vet ./internal/translator

Tested: make lint

Not-tested: go test ./... failed locally because port 6060 is occupied by amass, Docker Hub CRD fetch timed out, required e2e Docker image is missing, and Ollama model qwen3:1.7b is unavailable
Signed-off-by: liuhy <liuhongyu@apache.org>
@Aias00
Aias00 force-pushed the fix/bedrock-merge-index branch from f644ad6 to e7923b6 Compare June 19, 2026 13:59
@Aias00

Aias00 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

CI failed on E2E Test for Upgrades (k8s v1.33.4) - this appears to be a flaky test unrelated to the PR changes (translator code). Other k8s versions (v1.32.8/v1.34.3/v1.35.0) all passed. Requesting re-run.

@Aias00

Aias00 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

close: #2278

@Aias00 Aias00 closed this Jun 23, 2026
@Aias00 Aias00 reopened this Jun 23, 2026
@nacx
nacx enabled auto-merge (squash) June 23, 2026 09:26
@nacx
nacx merged commit 3e7b0bf into envoyproxy:main Jun 23, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants