Skip to content

feat(dynamo-llm): classify incoming agent requests and emit input_trigger in request trace - #12787

Merged
dagil-nvidia merged 13 commits into
mainfrom
idhanani/dyn-input-trigger-classification
Aug 11, 2026
Merged

dagil-nvidia merged 13 commits into
mainfrom
idhanani/dyn-input-trigger-classification

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Aug 6, 2026 •

Copy link
Copy Markdown
Contributor

Classifies incoming agent requests as user_message, tool_result, or other and records the trigger in request-end traces. This makes multi-turn agent traces reconstructable while keeping the field optional during rollout and avoiding classification work for requests without agent identity.

How This Was Implemented

  • Added optional AgentContext.input_trigger with provider-aware classifiers for OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and OpenAI Completions.
  • Classifies lazily only after agent headers produce an AgentContext, then inserts the completed context once.
  • Preserves additive wire compatibility by accepting unknown AgentContext fields and omitting input_trigger when absent.
  • Added a Dynamo ACP skill for persistent Claude Code, Codex, and OpenCode sessions plus trace validation guidance.
Walkthrough

Mental model

The frontend derives session identity from agent headers and the causal trigger from the provider-specific request shape. The preprocessor carries that metadata into the request trace; Python SGLang, vLLM, and TensorRT-LLM workers receive the request as a generic dictionary and ignore the added nested key.

flowchart LR
    A["Agent request"] --> B["HTTP frontend"]
    B --> C["Classify final causal input"]
    C --> D["AgentContext.input_trigger"]
    D --> E["PreprocessedRequest"]
    E --> F["Backend worker"]
    E --> G["request_end trace"]
Loading

Request lifecycle

Requests without agent identity skip trigger classification. Requests with agent identity classify the final causal message or item, set the optional field before the first registry insertion, and emit it with the same session ID in the request-end trace.

Boundaries and limitations

The schema is intentionally limited to user_message, tool_result, and other. OpenAI Responses non-function tool-output variants currently map to other; the classifier contains a follow-up TODO rather than expanding this PR.

Validation

  • cargo test -p dynamo-llm --lib input_trigger --no-default-features — 5 passed.
  • cargo test -p dynamo-llm --lib request_trace --no-default-features -- --skip tool_relay — 58 passed.
  • cargo clippy -p dynamo-llm --lib --no-default-features -- -D warnings — passed.
  • Harness empty-response regression test — passed.
  • Persistent Claude Code, Codex, and OpenCode tool sessions against Dynamo — passed.
  • Python request-plane integration test — passed.

Benchmark Results

Registry operation Time
Missing Context::get used before the hot-path fix 42.28 ns/op
Missing Context::get_optional reference 3.17 ns/op

The final implementation skips the registry lookup and classifier entirely when agent identity is absent.

Summary by CodeRabbit

  • New Features

    • Added request-trigger classification for supported OpenAI and Anthropic request types.
    • Request traces now record whether an agent turn originated from a user message, tool result, or other input.
    • Trigger metadata is preserved in enriched request traces.
  • Bug Fixes

    • Improved agentic request trace reconstruction by retaining input context across supported request flows.
  • Tests

    • Added coverage for user messages, tool results, other inputs, empty requests, and trace metadata preservation.

@github-actions github-actions Bot added frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` feat labels Aug 6, 2026
@datadog-official

This comment has been minimized.

@ishandhanani
ishandhanani marked this pull request as ready for review August 8, 2026 00:28
@ishandhanani
ishandhanani requested review from a team as code owners August 8, 2026 00:28
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The change adds optional InputTrigger metadata to agent contexts. OpenAI and Anthropic handlers classify requests and attach the metadata. Request-trace loading, conversion, fixtures, and integration tests preserve and validate the value.

Input trigger contract and classification

Layer / File(s) Summary
Input trigger contract and classification
lib/llm/src/protocols/common.rs, lib/llm/src/protocols/common/extensions.rs, lib/llm/src/protocols/common/input_trigger.rs
Defines InputTrigger, adds optional agent-context metadata, classifies Chat Completions, Responses, Anthropic Messages, and Completions requests, and tests key Chat Completions cases.

Protocol handler integration

Layer / File(s) Summary
Protocol handler integration
lib/llm/src/http/service/openai.rs, lib/llm/src/http/service/anthropic.rs
Classifies incoming requests and attaches the trigger to request context before session-affinity processing.

Trace propagation and fixture validation

Layer / File(s) Summary
Trace propagation and fixture validation
lib/llm/src/request_trace/*, lib/data-gen/src/request_trace/*
Initializes the optional field in conversions and fixtures. Tests verify trace preservation and deserialization of input_trigger.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the implementation and validation, but it omits the required Related Issues section and reviewer starting point. Add the required Related Issues section and identify the specific files or areas where the reviewer should start.
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: classifying incoming agent requests and recording input triggers in request traces.

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@ishandhanani
ishandhanani force-pushed the idhanani/dyn-input-trigger-classification branch from 777be08 to 68ef6f4 Compare August 10, 2026 16:56
devin-ai-integration[bot]

This comment was marked as resolved.

@ishandhanani
ishandhanani force-pushed the idhanani/dyn-input-trigger-classification branch from cc8094c to c351142 Compare August 10, 2026 19:06
…gger in request trace

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
- Remove deny_unknown_fields from AgentContext to preserve N-2 frontend/worker
  compatibility when new input_trigger field is present.
- Map Responses API output messages, function calls, and reasoning items to
  InputTrigger::Continuation to match chat completions behavior.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani force-pushed the idhanani/dyn-input-trigger-classification branch from c351142 to 2ce4635 Compare August 10, 2026 19:06
@ishandhanani
ishandhanani force-pushed the idhanani/dyn-input-trigger-classification branch from 2ce4635 to eee7def Compare August 10, 2026 19:15
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani requested review from a team as code owners August 11, 2026 02:23
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 11, 2026
@github-actions

github-actions Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread lib/llm/src/http/service/openai.rs Resolved
@dagil-nvidia
dagil-nvidia merged commit 3367bc6 into main Aug 11, 2026
121 checks passed
@dagil-nvidia
dagil-nvidia deleted the idhanani/dyn-input-trigger-classification branch August 11, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feat frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants