Skip to content

fix(frontend): track Codex subagent threads - #12331

Merged
ishandhanani merged 19 commits into
mainfrom
idhanani/dyn-codex-subagent-headers
Aug 12, 2026
Merged

ishandhanani merged 19 commits into
mainfrom
idhanani/dyn-codex-subagent-headers

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Codex child threads now execute through Dynamo's Responses frontend while retaining thread and parent-thread lineage. With dynamo-protocols 5.3.1 now supplied by main, Codex agent messages use the normal typed parser instead of Dynamo's raw-body fallback.

How This Was Implemented

  • Map Codex thread-id and x-codex-parent-thread-id into Dynamo's agent context; the run-wide session-id remains unused.
  • Flatten namespace function tools for the chat backend and restore their namespace when the flattened name has one unambiguous origin.
  • Use Axum's standard typed Responses JSON extractor and the released protocol parser for Codex agent messages.
  • Require Codex V2 to call spawn_agent then wait_agent; its exec smoke uses exactly {\"cmd\":\"ls\"} so it cannot request unavailable approvals.
Walkthrough

Codex's thread-id identifies an agent; x-codex-parent-thread-id links a child to its immediate parent. The root-run session-id is deliberately not used for Dynamo session affinity.

sequenceDiagram
    participant Parent as Codex parent
    participant Dynamo as Dynamo Responses frontend
    participant Model as Local model
    participant Child as Codex child
    Parent->>Dynamo: thread-id + collaboration tools
    Dynamo->>Model: flattened function tools
    Model-->>Dynamo: collaboration.spawn_agent
    Dynamo-->>Parent: namespaced tool call
    Parent->>Child: spawn_agent(task_name, message)
    Child->>Dynamo: agent_message + child/parent thread headers
    Dynamo->>Model: typed dynamo-protocols request
    Dynamo-->>Child: response; trace child parent context
Loading

Cross-origin name collisions remain rejected because a flat backend function call cannot be routed safely; duplicate top-level functions and duplicate functions from one namespace remain valid. dynamo-protocols owns Codex agent_message deserialization, so Dynamo no longer carries a divergent raw parser.

Validation

  • tests/frontend/test_frontend_api_surface_compliance.py::test_frontend_api_surface_compliance against local Qwen/Qwen3-VL-2B-Instruct (1 passed in 51.14s).
  • cargo test -p dynamo-llm protocols::openai::responses --lib (96 passed).
  • cargo clippy -p dynamo-llm --lib --tests -- -D warnings.
  • cargo fmt --all -- --check and git diff --check.

Summary by CodeRabbit

  • New Features

    • Improved Codex thread tracking, including support for parent and child session relationships.
    • Child-thread requests now preserve parent session context for more accurate tracing.
  • Bug Fixes

    • Corrected Codex session identification to use thread-specific headers instead of the run-wide session identifier.
  • Documentation

    • Updated agent harness and session ID guidance to reflect the revised Codex header mappings.

@github-actions github-actions Bot added fix documentation Improvements or additions to documentation frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` labels Jul 29, 2026
@ishandhanani
ishandhanani force-pushed the idhanani/dyn-codex-subagent-headers branch from 703f5b8 to d821acb Compare July 29, 2026 00:10
@pull-request-size pull-request-size Bot added size/L and removed size/M labels Jul 29, 2026
@github-actions

github-actions Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

@datadog-official

This comment has been minimized.

@ishandhanani
ishandhanani marked this pull request as ready for review July 29, 2026 15:03
@ishandhanani
ishandhanani requested review from a team as code owners July 29, 2026 15:03
devin-ai-integration[bot]

This comment was marked as resolved.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Codex session derivation now uses thread-id and x-codex-parent-thread-id. Protocol mappings, unit tests, frontend compliance checks, and documentation were updated to validate and describe root and child thread lineage.

Changes

Codex thread session mapping

Layer / File(s) Summary
Protocol header mapping
lib/llm/src/protocols/agents.rs
Codex mappings now derive session_id from thread-id and parent_session_id from x-codex-parent-thread-id.
Session lineage validation
lib/llm/src/protocols/common/extensions.rs
Tests cover thread-based root sessions, parent-child lineage, and updated session-affinity header handling.
Frontend verification and documentation
tests/frontend/test_frontend_api_surface_compliance.py, docs/fern/agents/*.mdx
Frontend checks send Codex child-thread headers and match expected trace identifiers; documentation describes the new mappings and clarifies that Codex session-id is run-wide.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The description explains the implementation and validation, but it omits the required Related Issues section and reviewer-start guidance. Add the required Related Issues section and identify the files or areas where the reviewer should start.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: tracking Codex subagent threads in the frontend.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Fix failing CI checks

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
lib/llm/src/protocols/common/extensions.rs (1)

1062-1078: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add a regression test for ignoring Codex session-id.

This verifies the new thread mapping, but would also pass if the legacy session-id mapping remained. Add a case with session-id alone and/or alongside thread-id to lock in the no-shared-session behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/llm/src/protocols/common/extensions.rs` around lines 1062 - 1078, The
test agent_context_from_codex_thread_headers_preserves_subagent_lineage does not
verify that the legacy session-id header is ignored. Add regression coverage for
session-id alone and/or combined with Codex thread headers, asserting no shared
session mapping is derived from session-id while the existing thread-based
lineage behavior remains unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/llm/src/protocols/common/extensions.rs`:
- Around line 1062-1078: The test
agent_context_from_codex_thread_headers_preserves_subagent_lineage does not
verify that the legacy session-id header is ignored. Add regression coverage for
session-id alone and/or combined with Codex thread headers, asserting no shared
session mapping is derived from session-id while the existing thread-based
lineage behavior remains unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 365a66b2-97cc-44cd-9fbd-42db2c555ec5

📥 Commits

Reviewing files that changed from the base of the PR and between f62f28e and d821acb.

📒 Files selected for processing (5)
  • docs/fern/agents/agent-harnesses.mdx
  • docs/fern/agents/session-ids.mdx
  • lib/llm/src/protocols/agents.rs
  • lib/llm/src/protocols/common/extensions.rs
  • tests/frontend/test_frontend_api_surface_compliance.py

Comment thread docs/fern/agents/agent-harnesses.mdx Resolved
Comment thread docs/fern/agents/session-ids.mdx Resolved
Comment thread docs/fern/agents/session-ids.mdx Resolved
devin-ai-integration[bot]

This comment was marked as resolved.

@rmccorm4 rmccorm4 added the Codex Related to Codex self-hosting compatibility label Jul 31, 2026
@rmccorm4

Copy link
Copy Markdown
Contributor

Some hardening after local testing: #12476

devin-ai-integration[bot]

This comment was marked as resolved.

@ishandhanani
ishandhanani force-pushed the idhanani/dyn-codex-subagent-headers branch from c06e604 to 68a6ad2 Compare August 4, 2026 21:24
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani force-pushed the idhanani/dyn-codex-subagent-headers branch from 44db3cc to 369b46d Compare August 11, 2026 02:49
@ishandhanani

Copy link
Copy Markdown
Contributor Author

/ok-to-test 369b46d

@ishandhanani
ishandhanani enabled auto-merge (squash) August 11, 2026 04:52

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review scoped to CI, testing, security and maintainability, as requested. 13 items: 4 blocking, 9 non-blocking. Posting as a comment rather than Request Changes — the blocking labels below are my assessment, not a formal gate.

I've skipped anything an existing thread already covers. In particular I am not re-raising "namespace collisions turn a previously-served request into a hard failure" (Devin, responses/mod.rs, Aug 8) — that's been answered and partly addressed. My comment on that same code is only about the error type and what gets logged, which is unaddressed. Likewise for the session-id removal: the "current Codex emits both headers" answer holds, so all I've left is a request for a deprecations entry.

Blocking

Where What
test_frontend_api_surface_compliance.py:594 The gating Claude trace assertion can be satisfied by a leftover Codex record — the matcher ignores its source_label. The Claude mapping could break completely and stay green.
test_frontend_api_surface_compliance.py:472 The budget comment's own ceilings now sum to exactly the cap (1095), and a pytest-timeout kill doesn't match only_rerun, so there is no retry.
test_frontend_api_surface_compliance.py:1009 The header to parent_session_id mapping this PR exists to add has no gating coverage, while the docstring and session-ids.mdx both tell readers it does.
responses/mod.rs:664 Client input returns HTTP 500 instead of 400, and an unbounded client-supplied tool name is interpolated into ERROR-level logs.

Correction (after testing): agents.rs:83 was originally listed here as blocking. I ran it against the real crate and withdrew it — the behavior is correct, and my suggested fix there would have been a regression. Details in that thread; it's now a non-blocking coverage note.

Two of the remaining items are verified by execution rather than reading: responses/mod.rs:664 reproduces as 500 / ErrorType::Internal and the suggestion turns it into 400 / Validation; and stream_converter.rs:1386 stays green with both SSE sites reverted to namespace: None. Evidence is in those threads.

Non-blocking

CI budget vs the 30-minute sglang stage cap (:474), probe/gating-step cwd isolation (:586), discarded timeout diagnostics (:987), the dropped _ => {} comment plus silently-dropped Custom tools (mod.rs:701), normalize_tools skipping namespace-nested strict (mod.rs:1049), the per-delta rescan in namespace_for_function (mod.rs:1023), the new stream-converter test covering neither SSE site (stream_converter.rs:1386), and a missing parent_session_id length cap plus a deprecations entry (agents.rs:35).

Four carry suggestion blocks that can be committed directly from the UI.

Checked and found clean, in case it saves anyone time: the namespace flatten/restore pair is symmetric, the generated config.toml is valid TOML in both branches, and _SUBPROCESS_ENV_ALLOWLIST keeps secrets out of the Allure artifact.

Comment thread lib/llm/src/protocols/agents.rs Resolved
Comment thread lib/llm/src/protocols/agents.rs Resolved
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 399e72e9f0. This addresses every item from my review, and in several places with a better fix than I proposed. I re-ran things rather than reading the diff — results below.

Verified by execution (cargo test -p dynamo-llm --lib on Linux at this head):

  • 2003 passed, 0 failed. The OnceLock field on ResponseParams doesn't break any derive, and the only external construction site (openai.rs:3079) is updated.
  • The SSE gap is genuinely closed. I re-applied the exact mutation that passed before — namespace: None at lines 491 and 631 — and test_function_call_preserves_namespace now fails with response.output_item.added did not preserve the namespace. Before this commit that mutation was green.
  • The 975 budget is arithmetically sound. 60+60+180+180+35+180+45+180 = 920, so the 55s of claimed headroom is exactly right. That also resolves the scheduler tension I raised on line 472: the marker goes back to 975, so est_duration returns to 325s while real runtime drops by ~175s. Both consumers are now served by the same number.

The best change here is _run_codex_parent_header_check. Replacing the model-mediated probe with an explicit /v1/responses POST that asserts both expected_session_id and expected_parent_session_id turns the PR's headline behavior from untested into a deterministic gate, and drops 210s of GPU pre_merge time to 35s in the process. It also makes the session-ids.mdx:42 claim true rather than aspirational. That one change closes four of my findings at once.

Also confirmed fixed: the typed InvalidArgument (with the client-supplied name no longer interpolated) plus a test pinning 400/Validation; line-based trace cursor; MAX_SESSION_AFFINITY_ID_BYTES on parent_session_id in both branches; namespace-nested strict; memoized function_namespaces; the restored _ => {} comment; the deprecations entry; and the X-Dynamo-Parent-Session-ID doc text. The affinity-granularity note in session-ids.mdx also answers Devin's open thread.

One new issue, inline: the added regression test carries no backend marker, so no CI job collects it.

One question, on agents.rs. You moved the self-parent filter after .or_else — which is the chain I proposed and then withdrew as a regression. I still think my withdrawal was right on the facts, and you've since made the contract uniform and tested it across both mappings, which is a coherent answer. Just confirming one consequence is intended: a Claude Code request with parent-agent-id == agent-id now yields parent_session_id = None, while the same request omitting the header yields Some(root) via inference. So a malformed parent header is now worse than no parent header. Fine if that's the intended reading of "ignore"; worth a line in the test if so.

Not addressed, and still fine to defer: NamespaceToolParamTool::Custom is still silently dropped at mod.rs:690, so a namespace containing only custom tools still reaches the backend as tools: None.

@dmitry-tokarev-nv

Copy link
Copy Markdown
Contributor

/ok to test 399e72e

@dmitry-tokarev-nv dmitry-tokarev-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing comments. remaining issues are non-blocking

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
devin-ai-integration[bot]

This comment was marked as resolved.

@ishandhanani

Copy link
Copy Markdown
Contributor Author

/ok to test 2d25e06

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Contributor Author

/ok to test a8a5d6b

@ishandhanani
ishandhanani merged commit 3b76754 into main Aug 12, 2026
122 checks passed
@ishandhanani
ishandhanani deleted the idhanani/dyn-codex-subagent-headers branch August 12, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Codex Related to Codex self-hosting compatibility documentation Improvements or additions to documentation fix frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants