feat(jaeger): implement ag-ui endpoint

[Nabil-Salah]

Which problem is this PR solving?

• [Feature]: AI gateway: AG-UI translation for /api/ai/chat and end-to-end and contextual tools wiring #8504
• Resolves [Feature]: Implement ACP-based AI handler #8295
• Part of [Feature]: GenAI integration with Jaeger #7827
• Continues work for [Feature]: Implement AG-UI to ACP Jaeger AI (PoC) #8252

Description of the changes

• Switch ChatRequest to AG-UI's aguitypes.RunAgentInput shape (messages, tools, context, threadId, runId) and translate it into ACP prompt blocks via a new translation.go (latestUserMessageText, contextTextEntries, encodeToolsAsRaw, extractText).
• Switch the chat endpoint to text/event-stream and have streamingClient emit AG-UI lifecycle + content events (RUN_STARTED, TEXT_MESSAGE_START/CONTENT/END, TOOL_CALL_START/ARGS/RESULT/END, RUN_FINISHED, RUN_ERROR). Replaces the old [tool_call] / [tool_result] text markers.
• Wire contextual tools end-to-end: the chat handler builds two snapshots from req.Tools, attaches the UIToolPrefix-prefixed snapshot to NewSessionRequest.Meta under jaegertracing.io/contextual-tools, and writes the unprefixed snapshot into ContextualToolsStore keyed by the assigned ACP SessionId. defer DeleteForSession at turn end.
• Treat contextual tool dispatches as fire-and-forget side effects: handleJaegerToolCall validates payload, strips the ui_ prefix, looks the post-strip name up in the per-session snapshot, returns {result: {acknowledged: true}, isError: false} on hit and InvalidParams on miss. newDispatcher now takes *ContextualToolsStore; a nil store rejects every dispatch as not-registered. Browser performs the side effect locally based on the parallel TOOL_CALL_* SSE events — no synchronous tool result is round-tripped through the LLM.
• Add the ContextualToolsMetaKey constant alongside ExtMethodJaegerToolCall and UIToolPrefix in dispatcher.go so the protocol contracts live in one place.
• One-line clarifying comment on Cwd: "/" in NewSessionRequest (the gateway advertises no fs capability; ACP just requires the field non-empty).
• Add github.com/ag-ui-protocol/ag-ui/sdks/community/go as a direct dependency.
• Update the jaegerai/ README (architecture diagram, request flow, contextual-tools lifecycle), the scripts/ai-sidecar/gemini/ README (ack semantics), and RFC 0002 (fire-and-forget rationale §6.6, edge cases §6.7, implementation plan §7, open questions §8, protocol shapes §9.2).

How was this change tested?

• New unit tests:
  • translation_test.go — 13 tests covering user-message extraction (latest-user, error-when-none, blank skip, InputContent parts), context-entry formatting (description + value, blank filter), tool encoding (round-trip, skip-unmarshalable), and extractText across all branches (string, []InputContent, []any, top-level text / parts, RawMessage direct string, channel-marshal failure).
  • streaming_client_test.go additions — SSE-frame parsing helper, startRun / finishRun / failRun / ensureTextStart lifecycle, writeSSEEvent silently drops unmarshalable, idempotent ensureTextStart, agent message → TEXT_MESSAGE_CONTENT, ToolCall → TOOL_CALL_START + TOOL_CALL_ARGS, ToolCallUpdate → TOOL_CALL_ARGS + TOOL_CALL_RESULT + TOOL_CALL_END (only on completed / failed), in_progress skipped.
  • handler_test.go rewritten for the new request shape: ACP protocol assertions, context-entry → prompt block append, Meta carries prefixed tools, store set-then-cleared, omitted-Meta when no tools, RUN_FINISHED.stopReason propagation, threadId/runId propagation, no-user-message rejection, SSE content-type, RUN_ERROR framing on Prompt failure.
  • dispatcher_test.go additions — freshDispatcher returns a dispatcherFixture (revive function-result-limit fix); store-lookup hit, miss-on-unknown-tool, miss-on-unknown-session, miss-on-nil-store. Existing prefix-strip and warn-on-missing-prefix tests now seed the store before dispatching.
• Existing sidecar workflow test (test_sidecar_workflow.py) still passes.
• go build and focused go test on every touched package.

Checklist

• I have read https://github.com/jaegertracing/jaeger/blob/main/CONTRIBUTING_GUIDELINES.md
• I have signed all commits
• I have added unit tests for the new functionality
• I have run lint and test steps successfully: make lint test

AI Usage in this PR (choose one)

See AI Usage Policy.

• None: No AI tools were used in creating this PR
• Light: AI provided minor assistance (formatting, simple suggestions)
• Moderate: AI helped with code generation or debugging specific parts
• Heavy: AI generated most or all of the code changes

[Copilot]

Pull request overview

This PR updates Jaeger Query’s AI gateway /api/ai/chat endpoint to speak the AG-UI protocol on the wire (AG-UI RunAgentInput in, AG-UI SSE events out) and completes the end-to-end wiring for per-turn “contextual tools” so the sidecar can dispatch UI tool calls back through the gateway with a fire-and-forget acknowledgement.

Changes:

• Switch the chat request/response shape to AG-UI (RunAgentInput + text/event-stream) and translate ACP session updates into structured AG-UI lifecycle/text/tool-call SSE events.
• Implement translation helpers for extracting the latest user message + context entries and for tool snapshot encoding.
• Wire contextual tools end-to-end: attach prefixed tools in NewSessionRequest.Meta, store an unprefixed per-session snapshot, and validate/ack contextual tool dispatches in the ACP extension method handler.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
scripts/ai-sidecar/gemini/README.md	Documents fire-and-forget contextual tool ack semantics.
go.mod	Adds the AG-UI Go SDK dependency.
go.sum	Records checksums for the new AG-UI Go SDK dependency.
docs/rfc/0002-ai-gateway-contextual-tools.md	Updates the RFC to AG-UI request/response shapes and fire-and-forget tool rationale/flows.
cmd/jaeger/internal/extension/jaegerquery/internal/server_test.go	Updates router registration tests to use the new AG-UI request body shape.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/translation.go	Adds AG-UI → ACP prompt/tool translation helpers.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/translation_test.go	Unit tests for translation helpers and edge cases.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/streaming_client.go	Reworks streaming client to emit AG-UI SSE events from ACP updates and lifecycle calls.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/streaming_client_test.go	Adds/updates tests for SSE framing, lifecycle events, text, and tool-call event mapping.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/README.md	Updates package docs/diagrams to reflect AG-UI + SSE + fire-and-forget tools.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/handler.go	Switches chat handler to AG-UI input, builds ACP prompt blocks, wires tool snapshots/meta, and streams SSE.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/handler_test.go	Rewrites handler tests for the new AG-UI shape, SSE framing, meta/store wiring, and error signaling.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/dispatcher.go	Updates dispatcher/ext-method handling to validate against the store and return fire-and-forget acknowledgements.
cmd/jaeger/internal/extension/jaegerquery/internal/jaegerai/dispatcher_test.go	Expands dispatcher tests for store-backed validation and ack behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.60%. Comparing base (04657df) to head (e09e0b0).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8505      +/-   ##
==========================================
+ Coverage   96.53%   96.60%   +0.06%     
==========================================
  Files         330      331       +1     
  Lines       17363    17578     +215     
==========================================
+ Hits        16762    16981     +219     
+ Misses        453      450       -3     
+ Partials      148      147       -1     

Flag	Coverage Δ
badger_direct	8.94% <ø> (ø)
badger_e2e	1.04% <ø> (ø)
cassandra-4.x-direct-manual	14.55% <ø> (ø)
cassandra-4.x-e2e-auto	1.03% <ø> (ø)
cassandra-4.x-e2e-manual	1.03% <ø> (ø)
cassandra-5.x-direct-manual	14.55% <ø> (ø)
cassandra-5.x-e2e-auto	1.03% <ø> (ø)
cassandra-5.x-e2e-manual	1.03% <ø> (ø)
clickhouse-direct	8.97% <ø> (ø)
clickhouse-e2e	1.16% <ø> (?)
elasticsearch-6.x-direct	16.87% <ø> (ø)
elasticsearch-7.x-direct	16.91% <ø> (ø)
elasticsearch-8.x-direct	17.05% <ø> (ø)
elasticsearch-8.x-e2e	1.04% <ø> (ø)
elasticsearch-9.x-e2e	1.04% <ø> (ø)
grpc_direct	7.90% <ø> (ø)
grpc_e2e	1.04% <ø> (ø)
kafka-3.x-v2	1.04% <ø> (ø)
memory_v2	1.04% <ø> (ø)
opensearch-1.x-direct	16.95% <ø> (ø)
opensearch-2.x-direct	16.95% <ø> (ø)
opensearch-2.x-e2e	1.04% <ø> (ø)
opensearch-3.x-e2e	1.04% <ø> (ø)
query	1.04% <ø> (ø)
tailsampling-processor	0.55% <ø> (ø)
unittests	94.90% <100.00%> (+0.08%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.

[yurishkuro]

needs a rebase

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 5 comments.

[github-actions]

CI Summary Report

Metrics Comparison

✅ No significant metric changes

Code Coverage

✅ Coverage 97.3% (baseline 97.3%)

➡️ View CI run | View publish logs
2026-05-19 03:38:47 UTC

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.