Skip to content

feat(runtime): support explicit transport host overrides - #11870

Merged
jthomson04 merged 5 commits into
mainfrom
jthomson04/event-plane-host
Sep 5, 2026
Merged

jthomson04 merged 5 commits into
mainfrom
jthomson04/event-plane-host

Conversation

@jthomson04

@jthomson04 jthomson04 commented Jul 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add DYN_EVENT_PLANE_HOST to override the address advertised by direct ZMQ event publishers.
  • Give DYN_EVENT_PLANE_HOST and DYN_TCP_RESPONSE_STREAM_HOST one contract: each accepts an IPv4 or IPv6 literal, or an exact network interface name.
  • Trim configured host values, reject wildcard advertisement addresses and non-Unicode values, and preserve the existing IPv4-first automatic selection with IPv6 fallback.
  • Build advertised endpoints with SocketAddr, preserve the transport host settings and DYN_TCP_RPC_HOST across snapshot restore, and document the supported configuration.

Direct ZMQ continues to bind 0.0.0.0; this PR changes its advertised discovery address only. Full IPv6 binding and address-selection policy remain in #13126. NATS and ZMQ broker mode are unchanged.

Validation

  • cargo fmt --all -- --check
  • cargo test -p dynamo-runtime utils::ip_resolver -- --test-threads=1 — 10 passed
  • cargo test -p dynamo-runtime event_plane -- --test-threads=1 — 43 passed
  • cargo test -p dynamo-runtime --lib pipeline::network::tcp::server::tests -- --test-threads=1 — 39 passed
  • cargo check -p dynamo-runtime --all-targets
  • cargo clippy -p dynamo-runtime --lib --tests -- -D warnings
  • PYTHONPATH=components/src uv run --no-project --with pytest --with pytest-asyncio python -m pytest -c /dev/null -p no:cacheprovider --asyncio-mode=auto components/src/dynamo/common/tests/test_snapshot_restore_context.py — 5 passed
  • fern check
  • fern docs broken-links

@github-actions github-actions Bot added feat documentation Improvements or additions to documentation labels Jul 18, 2026
@github-actions

github-actions Bot commented Jul 18, 2026 •

Copy link
Copy Markdown
Contributor

@datadog-official

datadog-official Bot commented Jul 18, 2026 •

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR | dynamo-runtime / test / sequential cuda13.0, arm64   View in Datadog   GitHub Actions

PR | dynamo-status-check   View in Datadog   GitHub Actions

ℹ️ Info

🎯 Code Coverage (details)
• Patch Coverage: 0.00%
• Overall Coverage: 40.38% (-5.41%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 49a224b | Docs | Give us feedback!

@jthomson04 jthomson04 changed the title feat(event-plane): allow direct ZMQ host override feat(runtime): support explicit transport host overrides Jul 19, 2026
@jthomson04
jthomson04 force-pushed the jthomson04/event-plane-host branch 2 times, most recently from 9c930e0 to 9a30d4a Compare July 19, 2026 04:33
@jthomson04
jthomson04 marked this pull request as ready for review July 19, 2026 04:43
@jthomson04
jthomson04 requested review from a team as code owners July 19, 2026 04:43

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@coderabbitai

coderabbitai Bot commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The change adds DYN_EVENT_PLANE_HOST support across runtime configuration, snapshot restore, documentation, and direct ZMQ publisher discovery. It also centralizes IPv4 resolution and refactors TCP server address handling to use typed IP and socket addresses.

Changes

Event plane networking

Layer / File(s) Summary
Runtime environment registration and restore handling
lib/runtime/src/config/environment_names.rs, components/src/dynamo/common/snapshot/*, docs/design-docs/*
Registers DYN_EVENT_PLANE_HOST, validates its uniqueness, includes it in snapshot restore handling and tests, and documents event-plane and TCP response-stream host and port settings.
Shared advertised IPv4 resolution
lib/runtime/src/utils/ip_resolver.rs
Defines shared resolver abstractions and validates interface or literal IPv4 hosts, deterministic interface selection, invalid-address errors, and loopback fallback.
TCP interface and socket address resolution
lib/runtime/src/pipeline/network/tcp/server.rs
Uses shared interface resolution and typed IpAddr and SocketAddr values through TCP startup, binding, and registration.
Configured direct ZMQ publisher host
lib/runtime/src/transports/event_plane/mod.rs
Uses and validates DYN_EVENT_PLANE_HOST for direct ZMQ discovery, with automatic fallback and unit and integration coverage.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the main changes and validation, but it omits the required Related Issues section and does not identify specific files for reviewer focus. It also uses Summary and Validation … Add the required Related Issues section and select either a linked issue, such as Closes #XXXX``, or confirm that no related issue exists. Add a Where should the reviewer start? section with specific files. Align the description with the Ov…
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary change: support for explicit transport host overrides.
Full details: Description check

Explanation

The description explains the main changes and validation, but it omits the required Related Issues section and does not identify specific files for reviewer focus. It also uses Summary and Validation headings instead of the template's Overview and Details headings.

Resolution

Add the required Related Issues section and select either a linked issue, such as Closes #XXXX``, or confirm that no related issue exists. Add a Where should the reviewer start? section with specific files. Align the description with the Overview and Details sections where practical.

  • Fix all pre-merge checks with AI

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/runtime/src/transports/event_plane/mod.rs (1)

431-456: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Bind the ZMQ listener in the advertised address family. Direct mode always binds tcp://0.0.0.0:0, which is IPv4-only, but DYN_EVENT_PLANE_HOST can advertise [IPv6]:port. Subscribers will discover an unreachable IPv6 endpoint. The underlying bind implementation preserves that IPv4 endpoint in lib/runtime/src/transports/event_plane/zmq_transport.rs, lines 96–125.

  • lib/runtime/src/transports/event_plane/mod.rs#L431-L456: select an IPv6 wildcard bind endpoint (for example, tcp://[::]:0) when advertising an IPv6 address, or reject IPv6 configuration.
  • docs/design-docs/event-plane.md#L93-L96: retain the IPv6 claim only after matching-family binding works; otherwise document IPv4-only support.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/runtime/src/transports/event_plane/mod.rs` around lines 431 - 456, Update
the direct-mode binding in event-plane initialization around
event_plane_host_from_env and ZmqPubTransport::bind to choose an IPv6 wildcard
endpoint when advertised_host is IPv6, or explicitly reject IPv6 configuration
if matching-family binding is unsupported; preserve IPv4 behavior otherwise. In
docs/design-docs/event-plane.md lines 93-96, retain the IPv6 support claim only
if the implementation binds IPv6 correctly; otherwise document that direct mode
supports IPv4 only.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/runtime/src/transports/event_plane/mod.rs`:
- Around line 59-68: Update the host validation around the parsed `IpAddr` to
reject `ip.is_unspecified()` before formatting the advertised address, covering
both `0.0.0.0` and `::` with an appropriate validation error. Preserve the
existing IPv4 formatting, bracketed IPv6 formatting, and invalid-address
handling.

---

Outside diff comments:
In `@lib/runtime/src/transports/event_plane/mod.rs`:
- Around line 431-456: Update the direct-mode binding in event-plane
initialization around event_plane_host_from_env and ZmqPubTransport::bind to
choose an IPv6 wildcard endpoint when advertised_host is IPv6, or explicitly
reject IPv6 configuration if matching-family binding is unsupported; preserve
IPv4 behavior otherwise. In docs/design-docs/event-plane.md lines 93-96, retain
the IPv6 support claim only if the implementation binds IPv6 correctly;
otherwise document that direct mode supports IPv4 only.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e5816421-9f16-4a76-9534-a6ead1ac1be4

📥 Commits

Reviewing files that changed from the base of the PR and between ea89e8b and 9a30d4a.

📒 Files selected for processing (6)
  • components/src/dynamo/common/snapshot/constants.py
  • components/src/dynamo/common/tests/test_snapshot_restore_context.py
  • docs/design-docs/event-plane.md
  • lib/runtime/src/config/environment_names.rs
  • lib/runtime/src/pipeline/network/tcp/server.rs
  • lib/runtime/src/transports/event_plane/mod.rs

@jthomson04 jthomson04 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the transport host overrides. The change is clean and well-tested, and the String → IpAddr/SocketAddr refactor is a solid improvement (IPv6 bracketing stays consistent with the auto-detect path, and validation fails fast before any bind).

One thing I'd like to resolve before merge, plus a couple of optional notes — left inline:

  1. (please address) DYN_TCP_RESPONSE_STREAM_HOST isn't preserved across snapshot restore, unlike its new sibling DYN_EVENT_PLANE_HOST. If a deployment pins the TCP host to a node-specific IP literal, that value freezes at snapshot time and the restored process will try to bind the old node's IP. See comment on constants.py.
  2. (optional) API asymmetry: the event-plane host accepts IP literals only, while the TCP host also accepts interface names (IPv4-only resolution). See comment on event_plane/mod.rs.
  3. (optional) The DYN_TCP_RESPONSE_STREAM_HOST interface-name behavior change isn't documented. See comment on event-plane.md.

Minor cosmetic (no inline): after the IpAddr change, tracing::debug!("tcp transport service on {local_ip}:{local_port}") in tcp/server.rs prints an unbracketed IPv6 (…::1:8080) — ambiguous in logs; consider logging the SocketAddr.

Comment thread docs/design-docs/event-plane.md Resolved
@jthomson04

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@jthomson04 jthomson04 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review

Clean, well-tested change. Comments below are mostly minor.

Strengths

  • Good refactor — moving IpResolver/DefaultIpResolver into ip_resolver.rs (server re-exports them) removes the prior ip_resolver → server import cycle and centralizes resolution. The default trait method for list_afinet_netifas keeps the existing server.rs test mock compiling untouched.
  • SocketAddr is a real correctness fix, not just cosmetic — the old format!("{}:{}", local_ip, port) produced malformed ::1:port for the IPv6 auto-detect fallback; SocketAddr::to_string() correctly brackets it ([::1]:port).
  • Deterministic interface selection — the old path collapsed interfaces into a HashMap<String, IpAddr>, silently discarding all but one arbitrary address per name. Sorting + lowest-IPv4 is a genuine improvement.
  • Strong test coverage — literal / interface / IPv6-reject / unspecified-reject / no-usable-IPv4 / not-found, plus valid+invalid publisher integration tests and the snapshot apply-and-clear update. Validation runs before the ZMQ bind, so an invalid host fails fast.

Issues & suggestions

1. Default event-plane path changed for IPv6-only hosts (low, likely intended). The direct-ZMQ default moved from local_ip_for_advertise() (returns bracketed IPv6 when no IPv4 exists) to local_ipv4_for_advertise(), which falls back to 127.0.0.1. Since direct ZMQ always binds the IPv4 wildcard (tcp://0.0.0.0:0), the old IPv6 advertisement was already unreachable — so this is arguably a fix, but worth a line in the PR description: an IPv6-only host now advertises loopback instead of a scoped-but-broken IPv6.

2. "Lowest usable IPv4" only excludes 0.0.0.0 (low). resolve_advertised_ipv4 filters unspecified addresses but not loopback (127.0.0.0/8) or link-local (169.254.0.0/16). If an interface carries such an address alongside a routable one and it sorts lower (e.g. 169.254.x < 172.16.x), the non-routable address wins. Won't bite the documented IB/IPoIB case, but "lowest" ≠ "lowest routable" — consider filtering loopback/link-local, or noting the assumption in the docs.

3. IP literals aren't verified against local interfaces (low). Interface names are validated against list_afinet_netifas(); an IP literal is only checked for "IPv4 and not unspecified." A typo'd/foreign IP silently becomes the advertised address while the socket binds elsewhere. Matches the "explicit override = operator's responsibility" model, so defensible — just flagging the asymmetry.

4. Snapshot constant grouping is slightly off (nit). In constants.py, DYN_EVENT_PLANE_HOST sits under "Parsed Python runtime config that must also refresh the in-memory config passed to create_runtime()", but it's read only on the Rust side at publisher creation. It fits better next to DYN_TCP_RESPONSE_STREAM_HOST/PORT under "infrastructure env read after restore." RESTORE_RUNTIME_ENV_NAMES is a set, so this is comment accuracy only, not behavior.

5. ServerOptions.interface is now a misnomer (nit). The field accepts an IP literal or an interface name; the name interface (and the Interface not found error text) under-describes it. Rename is out of scope, but a doc comment noting it accepts either would help future readers.

6. Observation (no action): the two transports intentionally differ — the TCP response-stream server binds to the resolved IP, while direct ZMQ binds the wildcard and only changes the advertised address. Both preserve prior per-transport behavior, so correct; just worth being explicit that DYN_TCP_RESPONSE_STREAM_HOST restricts the listen address whereas DYN_EVENT_PLANE_HOST does not.

Verdict

Approve with minor nits. The only ones I'd act on before merge are #1 (one line in the PR description) and #4 (move the constant under the right comment); the rest are optional polish.

@github-actions

github-actions Bot commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added the Stale label Aug 23, 2026
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
jthomson04 force-pushed the jthomson04/event-plane-host branch from 49a224b to fdb106f Compare September 4, 2026 16:34
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@pull-request-size pull-request-size Bot added size/XL and removed size/L labels Sep 4, 2026
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
@jthomson04
jthomson04 merged commit 055d3e0 into main Sep 5, 2026
196 of 199 checks passed
@jthomson04
jthomson04 deleted the jthomson04/event-plane-host branch September 5, 2026 00:17
smb2love added a commit to smb2love/dynamo that referenced this pull request Sep 6, 2026
Interface-name resolution picked the last enumerated address of the
matching interface without filtering, so an IPv6 link-local address
(fe80::/10) could win and TcpListener::bind then fails with EINVAL on
Linux.

Build on ai-dynamo#11870: resolve_host_or_interface now skips link-local,
multicast, broadcast, and unspecified interface addresses and uses the
last usable unicast address of the interface. Explicit IP literals are
unaffected, and loopback interfaces like `lo` remain usable. This also
fixes DYN_EVENT_PLANE_HOST, which shares the same resolution path.

Signed-off-by: smb2love <smb2love@users.noreply.github.com>
Klaud-Cold pushed a commit to SemiAnalysisAI/InferenceX that referenced this pull request Oct 27, 2026
The Dynamo fork pin (upstream 2026-08-07) cannot parse an interface-name
DYN_TCP_RPC_HOST, which the batch3 GB300 nodes export; upstream resolves
it since ai-dynamo/dynamo#11870 and v1.6.1 also contains #12162.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Klaud-Cold pushed a commit to SemiAnalysisAI/InferenceX that referenced this pull request Oct 28, 2026
…disagg

Dynamo 1.4.0.dev20260730 parses DYN_TCP_RPC_HOST as a raw socket address,
so the gb300-nv node value enP22s22f0np0 (an interface name) fails with
'Invalid TCP bind address'. Dynamo resolves interface names since
ai-dynamo/dynamo#11870 (2026-09-05); 1.6.0.dev20261010 matches the green
gb300-nv dsv41flash Dynamo vLLM recipes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feat size/XL Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants