Skip to content

refactor(llm): move out the mocker adapter and the generate-format contract, and remove what the split left behind - #229

Draft
ishandhanani wants to merge 15 commits into
idhanani/dyno-231-pool-relayfrom
idhanani/dyno-232-cleanup
Draft

ishandhanani wants to merge 15 commits into
idhanani/dyno-231-pool-relayfrom
idhanani/dyno-232-cleanup

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Summary

Layer 16 of the lib/llm split stack (DYNO-232), the cleanup. Base: #228 (dynamo-pool-relay), so this PR shows only this layer. After the fifteen layers below, dynamo-llm was the frontend (http, grpc, entrypoint, frontend_load), two modules that are not the frontend (the mocker adapter and the generate_format contract), and the KV block manager, which a separate change removes and which this PR does not touch. The first nine commits remove what the split left behind, one commit per item so that any of them can be dropped. The last six move the two modules into crates of their own, which takes dynamo-llm out of the dependency tree of every sidecar and mocker server.

Modules moved out

  • The mocker adapter is dynamo-mocker-engine at lib/mocker/engine: dynamo_llm::mocker::X is dynamo_mocker_engine::X. Five files move with git mv (6,447 lines, 50 tests); the only source edit is the crate root's doc header. Its one caller is the Python bindings (make_mocker_engine).
  • It is a new crate because no existing one fits. Inside dynamo-mocker it is a cycle: dynamo-backend-common depends on dynamo-mocker (the forward-pass snapshot types in fpm_publisher.rs) and the adapter depends on dynamo-backend-common (kv_publisher, fpm_publisher); it would also give dynamo-mocker the runtime, the protocols and the preprocessor, none of which it has. The mocker servers (lib/mocker/servers/*) each imitate one engine over gRPC and depend on that engine's sidecar. dynamo-mocker-backend (lib/backend-common/examples/mocker) is a binary-only LLMEngine example without KV events, forward-pass metrics or the handoff.
  • The adapter's one dynamo_preprocessor import is backend::ExecutionContext, the return type of make_mocker_engine: an alias defined there for ServerStreamingEngine<PreprocessedRequest, Annotated<LLMEngineOutput>>, whose parts belong to dynamo-runtime and dynamo-frontend-protocols. The new crate depends on dynamo-preprocessor for that name only.
  • The /generate format contract (GenerateFormat, GenerateExchange, ClientRequest, Opened, Bound, Worker, ClientError, WorkerError, ErrorPhase, Registry, SharedFormat, install, named, selected, json_body_error) is dynamo-generate-format at lib/generate-format: dynamo_llm::generate_format::X is dynamo_generate_format::X. Two files move with git mv (490 lines); 35 paths in 16 files name the new crate.
  • It is a new crate because neither candidate fits. dynamo-http-server would be a cycle: the contract uses dynamo-frontend-protocols (PreprocessedRequest, LLMEngineOutput), which depends on dynamo-http-server. dynamo-frontend-protocols has no cycle, but it holds wire types, not a plugin contract with a process-wide registry, and it would gain two direct dependencies it does not have (dynamo-error, mime).
  • In-place cut before the second move: the vLLM sidecar's mm-routing forwarded to dynamo-llm/mm-routing, which only forwards to dynamo-preprocessor/mm-routing. It names the preprocessor now, as the SGLang sidecar's does.
  • Widened before the second move, the only two crate-private items used outside the module: generate_format::test_formats and its install, the stand-in formats dynamo-llm's HTTP tests install. In the new crate they are behind a testing feature (they were #[cfg(test)]), which dynamo-llm enables in its dev-dependencies. Nothing of the mocker adapter needed widening.
  • No compatibility re-export for either module. dynamo-llm keeps dynamo-mocker only as a dev-dependency (the kv_router_bench bench) and gains dynamo-generate-format; no other dependency of it was used by the two modules alone.
  • Log targets: the adapter's 22 log statements (none with an explicit target, all in the crate root) report dynamo_mocker_engine where they reported dynamo_llm::mocker. tests/router/test_router_e2e_with_mockers.py names the new target in its DYN_LOG filter; no other filter, launch script, recipe or doc named the old one. The contract has no log statements.
  • Each new crate has an AGENTS.md and CLAUDE.md, a workspace member and dependency entry, and codeowners: lib/generate-format/ is frontend's, as the module was; lib/mocker/engine/ is runtime's and performance's, as lib/llm/src/mocker/ was. The CI path filters match lib/** and **/*.rs and no labeler rule named either module, so neither changes.
  • Each move is re-runnable: a script reproduces the move commit's tree from its parent (checked: identical tree hashes).

Re-exports removed

  • No compatibility re-export stays: each path is deleted and every caller names the owning crate. 53 paths change: 23 in dynamo-llm, 21 in the Python bindings, 5 in the vLLM sidecar, 2 in the sidecar testkit, 2 in the Runtime EPP.
  • dynamo_llm::worker_type is dynamo_router_protocol::worker_type (31 paths).
  • dynamo_llm::namespace is dynamo_runtime::namespace (7 paths).
  • dynamo_llm::lora is dynamo_lora (8 paths). The Python bindings and the vLLM sidecar gain a direct dynamo-lora dependency; areas.yaml drops the glob of the deleted file.
  • dynamo_llm::tokenizers is dynamo_tokenizers (7 paths). The Runtime EPP gains a direct dynamo-tokenizers dependency.
  • The crate-root file_json_field and log_json_err had no caller through dynamo-llm. Their nine tests were the function's only tests in this repository, and the published dynamo-tokenizers 4.1.2 has none, so they move, bodies unchanged, to lib/model-card/tests/file_json_field.rs, next to the crate that calls the function.
  • common::versioned::Versioned had no implementor and no caller and is deleted. common::dtype (KV block manager) and utils::lora_name_to_id (the Python binding) stay.

Dependencies

  • dynamo-llm drops sysinfo and url: nothing names them since the model card moved in refactor(llm): move the model vocabulary into dynamo-model-card #213. cargo machete misses both (a target-specific table, and the word url in a test string).
  • Left alone because main has them too: the unused dev-dependency proptest and the testing feature of the dev-dependency dynamo-lora.
  • Not done, for a decision: eight normal dependencies are used only by tests or benches since the split (async-nats, dynamo-parsers, dynamo-router-policies, dynamo-slot-tracker, hf-hub, minijinja, tokenizers, tower). Moving them to dev-dependencies changes which features the frontend build unifies, so it needs its own check.

Crates that named dynamo-llm without importing it

  • lib/bindings/c, the TensorRT-LLM sidecar, the SGLang sidecar, and the dev-dependencies of the vLLM and SGLang mocker servers lose the entry.
  • TensorRT-LLM sidecar: its mm-routing feature only forwarded to dynamo-llm/mm-routing and nothing it depends on is gated by it now, so the feature is removed, from its default and from the Python bindings' mm-routing list as well.
  • SGLang sidecar: dynamo-sglang-generate still brings dynamo-llm and the preprocessor into the binary, so mm-routing stays and forwards to the owner, dynamo-preprocessor/mm-routing.
Crate dynamo-llm in its normal tree before After the manifest commits After the two moves Why
dynamo-vllm-sidecar yes yes no imported dynamo_llm::generate_format in one test and forwarded mm-routing through it
dynamo-sglang-sidecar yes yes no reached it through dynamo-sglang-generate
dynamo-trtllm-sidecar yes no no named it only in the manifest
dynamo-lpx-sidecar yes no no reached it through dynamo-trtllm-sidecar
dynamo-vllm-mocker yes yes no reached it through dynamo-vllm-sidecar
dynamo-sglang-mocker yes yes no reached it through dynamo-sglang-generate
dynamo-trtllm-mocker yes no no reached it through dynamo-trtllm-sidecar
dynamo-sglang-generate, dynamo-vllm-generate yes yes no imported dynamo_llm::generate_format
dynamo-generate-format-catalog yes yes no the same; its HTTP test still serves the frontend's routes, so dynamo-llm is a dev-dependency
dynamo-sidecar-common, dynamo-backend-common, dynamo-mocker-backend no no no
libdynamo_llm (C bindings) yes no no named it only in the manifest
dynamo-py3 (Python bindings) yes yes yes hosts the frontend; now only through its own entry
kvbm-py3 (KVBM bindings) yes yes yes the block manager
  • cargo tree -e normal -i dynamo-llm -p dynamo-vllm-sidecar finds nothing, and the same for the other three sidecars, the three mocker servers, dynamo-sidecar-common, dynamo-backend-common, the two format crates and their catalog. In all edges, only dynamo-generate-format-catalog and dynamo-sidecar-testkit still reach it, through dev-dependencies for tests that serve the frontend's HTTP routes.

  • Normal-tree package counts: dynamo-vllm-sidecar 761 to 746, dynamo-sglang-sidecar 760 to 746, the two format crates 722 to 661. dynamo-generate-format has 660 and depends directly on dynamo-error, dynamo-frontend-protocols, dynamo-http-server and dynamo-runtime; dynamo-mocker-engine has 707 and depends directly on dynamo-backend-common, dynamo-frontend-protocols, dynamo-kv-events, dynamo-mocker, dynamo-preprocessor, dynamo-protocols and dynamo-runtime.

  • What the shipped builds compile is unchanged by the manifest commits. cargo tree -e normal -f '{p}|{f}' over the sidecar image's one cargo build (four -p) lists 778 package and feature-set lines before and after, and over the wheel's feature set 810; the only line that differs in each is dynamo-trtllm-sidecar itself, which no longer shows mm-routing, a feature that gated no code in it. The vLLM sidecar, the Runtime EPP and the default Python build are identical line for line; in the SGLang sidecar's own tree only dynamo-llm's forwarding feature differs and dynamo-preprocessor keeps mm-routing.

  • What does change: a build of only dynamo-trtllm-sidecar, dynamo-lpx-sidecar, dynamo-trtllm-mocker or libdynamo_llm no longer compiles dynamo-llm and the 59 packages only it brought (28 for the C library), and resolves fewer features on up to 19 packages that stay, because dynamo-llm no longer asks for them (dynamo-kv-router/metrics, dynamo-mocker/zmq-events, dynamo-error/http, tonic/zstd, and features of base64, console, image, mime_guess, minijinja, moka, nix, num-complex, syn, tokenizers, uuid, zstd). The TensorRT-LLM and LPX SBOMs, which cargo cyclonedx writes per manifest, shrink the same way. No Dockerfile, container/ script or workflow builds these four alone except lib/sidecar/trtllm/ci/build.sh, a test build.

  • After the two moves the wheel is still unchanged: over its feature set the tree gains dynamo-mocker-engine and dynamo-generate-format and is otherwise identical line for line (810 to 812). No --features list in container/, a Dockerfile, a maturin setting or a workflow changes, and the feature names the wheel build passes still exist.

  • After the two moves the sidecar image does change, which is the point: its one cargo build no longer compiles dynamo-llm and the fourteen packages only it brought (dynamo-discovery, dynamo-router-host, dynamo-decisions, dynamo-protocols 8.5.0, utoipa 6, utoipa-gen 6, utoipa-swagger-ui, dialoguer, shell-words, bincode, bincode_derive, unty, virtue, oneshot) and gains dynamo-generate-format (778 to 764 lines). The four sidecar crates keep their feature sets (default,jemalloc,mm-routing for vLLM and SGLang, default,jemalloc for TensorRT-LLM and LPX) and dynamo-preprocessor keeps mm-routing, so the gated code is compiled as before. Ten packages that stay resolve fewer features, because only dynamo-llm asked for them: dynamo-mocker loses zmq-events (gates services::zmq_events only), dynamo-kv-router loses metrics (the metrics of the standalone KV indexer service), tonic 0.13 loses zstd, and zstd, zstd-safe, zstd-sys, base64 0.23, console, mime_guess and syn 3 lose default or codec features. No sidecar source names any of the gated items; the four sidecars compile alone and together without them. The vLLM and SGLang SBOMs shrink the same way.

Docs

  • lib/llm/AGENTS.md opens with what the crate owns and the crate map with its dependency direction: frontend, discovery, router host, sync, then the protocols and the model card, with the preprocessor, telemetry and dynamo-generate-format beside sync; worker-side code in dynamo-backend-common, the mocker engine in dynamo-mocker-engine, the Pool Relay in dynamo-pool-relay. Its check names the sidecars.
  • lib/mocker/README.md gains a section on the crates next to dynamo-mocker; lib/backend-common/AGENTS.md and lib/preprocessor/AGENTS.md name the new dependents and the sidecars' mm-routing forward; the SGLang sidecar's manifest and the sidecar Dockerfile no longer describe a build that links dynamo-llm.
  • Comments, READMEs and guides in 25 files that still placed moved code in lib/llm name the owner (lib/router/*, lib/runtime, lib/kv-events, lib/kv-hashing, lib/frontend-protocols, lib/model-card/AGENTS.md, the router testing guide).
  • The dynamo_llm split entry on the deprecations page gains five rows and the removed trait, each in the commit that removes the path, and three more with the two moves: mocker to dynamo_mocker_engine and generate_format to dynamo_generate_format in the module table, dynamo_llm::mocker to dynamo_mocker_engine in the log-target table.

After this PR

  • lib/llm/src is 83,216 lines of Rust in 170 files (90,377 in 181 before this PR, 90,155 in 177 after the first nine commits): block_manager 40,523 (behind its feature), http 35,340, grpc 3,077, entrypoint 1,979, tokens.rs 1,700, cuda.rs 207, frontend_load.rs 157, bin 116, utils 46, common 45, lib.rs 26.
  • lib.rs declares common, entrypoint, frontend_load, grpc, http, tokens, utils, and block_manager and cuda behind their features.
  • Without block_manager, tokens.rs and cuda.rs, which belong to the block manager, it is 40,786 lines.

Behaviour

  • No status code, message, serialized field, metric, log line, env var or wire format changes; no mocker timing, token output, handoff or KV event behaviour changes. The generated OpenAPI document is byte-identical.
  • One log target changes: dynamo_llm::mocker is dynamo_mocker_engine. A DYN_LOG filter scoped to the old target no longer selects the adapter's lines.

To confirm

  • The home of the nine file_json_field tests: dynamo-model-card here; the alternative is to add them to dynamo-tokenizers in its own repository and delete them here.
  • The SGLang sidecar names dynamo-preprocessor in its manifest only for the mm-routing forward.
  • The smaller TensorRT-LLM and LPX SBOMs, and the narrower feature set of a build of only those crates or the C library.
  • Whether to move the eight test-only dependencies to dev-dependencies.
  • The two crate names and paths: dynamo-mocker-engine at lib/mocker/engine and dynamo-generate-format at lib/generate-format.
  • test_formats moves with the contract, behind a testing feature that also brings dynamo-model-card. The alternative is to keep the two stand-in formats in dynamo-llm's test code, which needs no feature and no optional dependency but splits the file.
  • Codeowners of the mocker engine's crate root: as lib/llm/src/mocker.rs it matched only the lib/llm/ rule (runtime); under lib/mocker/engine/ it is runtime's and performance's like the rest of the adapter. No file loses an owner.
  • The SGLang sidecar's mm-routing feature and its dynamo-preprocessor entry now gate and link nothing the sidecar uses: the entry is the only thing that puts the preprocessor in its tree. Removing both, as this PR did for the TensorRT-LLM sidecar, takes 45 packages out of a build of only the SGLang sidecar (746 to 701) and changes its default feature list and the Python bindings' mm-routing list. Not done here.
  • The sidecar image's narrower build: no dynamo-llm, fourteen fewer packages, and fewer features on ten packages that stay (list above).
  • dynamo-mocker-engine depends on dynamo-preprocessor for the ExecutionContext alias only. Spelling the type out in make_mocker_engine's signature would drop the edge.

Validation

After the rebase onto upstream main d20e045, at the top of the stack: cargo test --locked --workspace --all-targets --no-fail-fast plus --doc gives 268 binaries, 10,022 passed, 0 failed, 87 ignored, the same names and outcomes as before the rebase plus the one test upstream added; sidecar testkit 113 passed; clippy -D warnings, the three bindings --locked, the integration feature check, four lock files --locked, machete, pre-commit and the strict codeowners build pass; every layer head below passed workspace check with all targets, the Python bindings check, the lock files and fmt. The numbers in the rest of this section were measured before that rebase.

The first nine commits (baseline #228 at f2a4994). All runs used the same remote 32 vCPU builder.

  • cargo test --no-fail-fast for dynamo-llm, dynamo-backend-common, dynamo-router-host, dynamo-sync, dynamo-discovery, dynamo-pool-relay, dynamo-model-card, dynamo-frontend-protocols, dynamo-telemetry, dynamo-preprocessor, dynamo-http-server and dynamo-global-frontend: 75 binaries, 4,842 passed, 0 failed, 36 ignored (baseline 74 binaries, 4,842 passed, 0 failed, 36 ignored). The extra binary is tests/file_json_field.rs.
  • Test names: 4,878 before and after. The nine dynamo_llm::file_json_field_tests::<name> are file_json_field::<name> in dynamo-model-card; no other name changes and none is missing.
  • cargo test -p dynamo-sidecar-testkit with the four sidecars built: 113 passed, as at the baseline.
  • cargo check --workspace --all-targets, cargo clippy --no-deps --workspace --all-targets -- -D warnings, cargo fmt --all -- --check: clean. cargo check --locked in lib/bindings/python, lib/bindings/c and lib/bindings/kvbm, and cargo metadata --locked for the four lock files: pass. cargo check -p dynamo-llm --features integration --all-targets: passes.
  • Each of the first eight commits on its own: cargo check --workspace --all-targets, cargo check --locked in lib/bindings/python, cargo metadata --locked for the four lock files, cargo fmt --all -- --check, the strict codeowners build, and cargo test -p dynamo-llm -p dynamo-model-card (1,297 passed, 0 failed, 17 ignored at every commit). The ninth changes comments and Markdown.
  • Alone: cargo check -p libdynamo_llm, cargo check --all-targets for dynamo-trtllm-sidecar, dynamo-lpx-sidecar, dynamo-trtllm-mocker, dynamo-sglang-sidecar, dynamo-vllm-mocker and dynamo-sglang-mocker, the image's cargo check -p dynamo-vllm-sidecar -p dynamo-sglang-sidecar -p dynamo-trtllm-sidecar -p dynamo-lpx-sidecar, and cargo test --locked -p dynamo-trtllm-sidecar --lib --no-run: pass. cargo check --locked --features kv-indexer,slot-tracker,select-service,mm-routing,ais-forward-pass,request-trace-s3,nvtx in lib/bindings/python: passes; with media-ffmpeg added it stops in the ffmpeg-sys-next build script because the builder has no FFmpeg libraries, so the wheel's exact feature list was not compiled.
  • cargo machete at the root, lib/bindings/python and lib/runtime/examples: no unused dependency. pre-commit run --from-ref the base: clean. The strict codeowners build passes and the generated CODEOWNERS matches areas.yaml. Docs lint: 0 errors.
  • OpenAPI: cargo run -p dynamo-llm --bin generate-frontend-openapi gave SHA-256 706f15d951891703fa911b9c659ac54992fa7144365ce4cd6b2a179a7630ede5 at the baseline and on main b20cd50. After the rebase onto main d20e045 (version 1.8.0) the top of the stack and main both give 9a5586a45edbf299454bb0fd761fb9595e734324e7c2d3476e58e0f978762f1d.

The six commits of the two moves (baseline the ninth commit, 1eeeb82; numbers at the head, 034bfa4).

  • cargo test --no-fail-fast for dynamo-llm, dynamo-backend-common, dynamo-mocker, dynamo-discovery, dynamo-http-server, dynamo-frontend-protocols, dynamo-generate-format-catalog, dynamo-sglang-generate, dynamo-vllm-generate, dynamo-global-frontend, the four sidecars, dynamo-sidecar-common, the three mocker servers and dynamo-mocker-backend, plus dynamo-mocker-engine and dynamo-generate-format at the head: 96 binaries, 3,932 passed, 0 failed, 39 ignored (baseline 92 binaries, 3,932 passed, 0 failed, 39 ignored). The four extra binaries are the unit and doc tests of the two new crates.

  • Test names (-- --list): 3,971 before and after. 3,921 keep their binary and path; the adapter's 50 are dynamo_mocker_engine::{tests, handoff::tests, metrics::tests, sglang::tests}::<name> where they were dynamo_llm::mocker::... (21, 9, 11 and 9); none is missing and none is new. dynamo-llm's unit tests go from 756 to 706.

  • cargo test -p dynamo-sidecar-testkit with the four sidecars built: 113 passed, as at the baseline.

  • At the head, 50 of 50 steps pass: cargo check --workspace --all-targets; cargo clippy --no-deps --workspace --all-targets -- -D warnings; cargo fmt --all -- --check; cargo metadata --locked for the four lock files; cargo check --locked in lib/bindings/python, lib/bindings/c, lib/bindings/kvbm and lib/runtime/examples; clippy in lib/bindings/python with the default features, with custom-policy,select-service and with --no-default-features; cargo check -p dynamo-llm --all-targets with each of integration, mm-routing and testing-etcd; CI's kv_router_bench build and clippy (--no-default-features --features kv-router-stress), which now takes dynamo-mocker from the dev-dependencies; check and clippy of each new crate, and of dynamo-generate-format with testing.

  • Alone: cargo check --all-targets for each of the four sidecars, the three mocker servers, dynamo-mocker-backend, dynamo-sidecar-common, dynamo-backend-common, the two format crates, their catalog and the sidecar testkit; the vLLM and SGLang sidecars with --no-default-features (how the wheel links them); the image's cargo check --locked -p dynamo-vllm-sidecar -p dynamo-sglang-sidecar -p dynamo-trtllm-sidecar -p dynamo-lpx-sidecar; cargo check --locked --features kv-indexer,slot-tracker,select-service,mm-routing,ais-forward-pass,request-trace-s3,nvtx in lib/bindings/python (the wheel's list without media-ffmpeg, for which the builder has no FFmpeg libraries). All pass.

  • Each of the five commits below the head on its own: cargo check --workspace --all-targets, clippy of the crates it touches, cargo check --locked and clippy in lib/bindings/python, cargo check --locked in lib/bindings/kvbm, the image's four-sidecar check, the kv_router_bench build, cargo metadata --locked for the four lock files, cargo fmt --all -- --check, the strict codeowners build with the generated CODEOWNERS matching, cargo machete at the root and in lib/bindings/python, the AGENTS.md and CLAUDE.md pair check, and the unit tests of the crates it touches: 756 passed at the mocker move (dynamo-llm 706, dynamo-mocker-engine 50), 135 at the feature cut (vLLM sidecar), 706 at the widening, 908 in 11 binaries at the contract move (dynamo-llm, the vLLM sidecar, the new crate, the two format crates, the catalog and its HTTP test), 129 at the comments commit (SGLang sidecar); 0 failed at every commit. The sixth commit is Markdown and is the head.

  • Each move script, run on the move commit's parent, stages a tree with the same hash as the commit.

  • cargo machete at the root, lib/bindings/python and lib/runtime/examples: no unused dependency. pre-commit run --from-ref 1eeeb825d0 --to-ref HEAD: clean. The strict codeowners build passes and the generated CODEOWNERS matches areas.yaml. Docs lint: 0 errors.

  • OpenAPI: cargo run -p dynamo-llm --bin generate-frontend-openapi gives SHA-256 706f15d951891703fa911b9c659ac54992fa7144365ce4cd6b2a179a7630ede5, unchanged.

  • Dependency graph: cargo tree -e normal -i dynamo-llm -p <crate> and cargo tree -e normal -f '{p}|{f}' over the image's four sidecars and over the wheel's feature set, before and after; the differences are listed above.

  • Not run: the Python test suite (which includes tests/router/test_router_e2e_with_mockers.py, whose DYN_LOG filter names the new target), any GPU or cluster test, and a real image or wheel build.

Closes DYNO-232

@ishandhanani ishandhanani changed the title refactor(llm): remove the re-exports and manifest entries the split left behind refactor(llm): move out the mocker adapter and the generate-format contract, and remove what the split left behind Nov 16, 2026
@ishandhanani
ishandhanani force-pushed the idhanani/dyno-231-pool-relay branch from f2a4994 to c0e93b8 Compare November 16, 2026 16:25
@ishandhanani
ishandhanani force-pushed the idhanani/dyno-232-cleanup branch from 034bfa4 to 9ecdd77 Compare November 16, 2026 16:26
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. `dynamo_llm::worker_type` only re-exported `WorkerType` and `ParseWorkerTypeError` from `dynamo-router-protocol`. The module is deleted and its 31 callers name the owner, `dynamo_router_protocol::worker_type::WorkerType`.

- dynamo-llm: 6 paths in `entrypoint/input/endpoint.rs`, `grpc/service/kserve.rs` and `http/service/{metrics, service_v2, unified_replay_tests}.rs`, and 3 in `tests/{http-service, http_metrics, namespace_scope_routing}.rs`.
- Python bindings: 19 paths in `rust/lib.rs` and `rust/llm/kv.rs`, and the doc comment that cited `lib/llm/src/worker_type.rs`.
- One path each in the Runtime EPP (`ext-proc/src/epp.rs`), the vLLM sidecar's tests and the sidecar testkit. The last two gain `dynamo-router-protocol` as a dev-dependency; the others already depend on it.
- The `dynamo_llm` split entry on the deprecations page gains the row.

No compatibility re-export is left and no behaviour changes; the longer path re-wraps one line.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. `dynamo_llm::namespace` only re-exported `GLOBAL_NAMESPACE`, `NamespaceFilter`, `NamespacePrefixMode` and `is_global_namespace` from `dynamo_runtime::namespace`. The module is deleted and its 7 callers name the owner.

- dynamo-llm: `entrypoint/input/{common, grpc, http}.rs` and `tests/{http_metrics, http_namespace_integration, namespace_scope_routing}.rs`.
- The sidecar testkit: `tests/support/process.rs`. It already depends on `dynamo-runtime`.
- The `dynamo_llm` split entry on the deprecations page gains the row.

No compatibility re-export is left and no behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. `dynamo_llm::lora` was `pub use dynamo_lora::*` and nothing else since the allocation gauges and the load-estimator subscription moved to `dynamo-discovery`. The module is deleted and its 8 callers name the owner, `dynamo_lora`.

- dynamo-llm: `lora_serving_enabled()` in `entrypoint/input/{grpc, http}.rs`.
- Python bindings: the two paths in `rust/llm/lora.rs`. The crate gains a direct `dynamo-lora` dependency.
- The vLLM sidecar: four paths in `src/{engine, lora, model}.rs`. The crate gains a direct `dynamo-lora` dependency.
- The root and `lib/bindings/python` lock files record the two new edges; no package enters either file.
- `areas.yaml` drops the glob for the deleted file and `CODEOWNERS` is regenerated.
- The `dynamo_llm` split entry on the deprecations page gains the row.

No compatibility re-export is left and no behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. `dynamo_llm::tokenizers` was `pub use dynamo_tokenizers as tokenizers`. The alias is deleted and its 7 callers name the owner, `dynamo_tokenizers`.

- dynamo-llm: `Tokenizer` in `entrypoint/input/common.rs` (3 paths), `Encoding` and `traits` in the System One preflight tests, and the two imports of `tests/tokenizers.rs`, which keeps testing the published crate from here.
- The Runtime EPP: `Encoding` in `ext-proc/src/epp.rs`. The crate gains a direct `dynamo-tokenizers` dependency, recorded in the root lock file; no package enters it.
- The `dynamo_llm` split entry on the deprecations page gains the row.

No compatibility re-export is left and no behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The crate root re-exported `file_json_field` and `log_json_err` from the published `dynamo-tokenizers`. No code in dynamo-llm or outside it calls them through that path any more: the model card and the preprocessor, their only callers in this repository, name `dynamo_tokenizers` directly. The re-export is deleted.

- The nine `file_json_field_tests` in `lib/llm/src/lib.rs` were the function's only tests in this repository, and `dynamo-tokenizers` 4.1.2 ships none for it. They move, bodies unchanged, to `lib/model-card/tests/file_json_field.rs`, next to the crate that calls the function, and import `dynamo_tokenizers::file_json_field`. Leaf names are unchanged; the path is `file_json_field::<name>` in `dynamo-model-card` where it was `file_json_field_tests::<name>` in `dynamo-llm`.
- The `dynamo_llm` split entry on the deprecations page gains the row.

No compatibility re-export is left and no behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. `dynamo_llm::common::versioned::Versioned` has no implementor and no caller: the model deployment card implements `dynamo_runtime::storage::kv::Versioned`, in `dynamo-model-card`. The file and its module declaration are deleted. It was already unused on `main` before the split.

- `common` keeps `dtype`, which the KV block manager and the KVBM bindings use.
- `utils` is unchanged: `lora_name_to_id` backs the Python binding of the same name.
- The `dynamo_llm` split entry on the deprecations page names the removed trait.

No behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. Nothing in dynamo-llm names `sysinfo` or `url` since the model card and the local model moved to `dynamo-model-card`: `sysinfo` read the cgroup memory limit that sizes the tokenizer cache and `url` parsed artifact URIs, both in `model_card.rs` and `local_model.rs`. `cargo machete` does not report them (`url` occurs as a word in a test string, and `sysinfo` sits in a target-specific table), so they stayed behind.

- `lib/llm/Cargo.toml` loses `url` and the Linux-only table that held `sysinfo`.
- The root, `lib/bindings/python` and `lib/bindings/kvbm` lock files lose the two edges from `dynamo-llm`; both packages stay in all three files for their other users.

Left alone because the split did not cause it: the dev-dependency `proptest` and the `testing` feature of the dev-dependency `dynamo-lora` are unused on `main` too.

No behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. After the split five manifests named dynamo-llm although no source in the crate imports it: the worker-side code they used is `dynamo-backend-common`. The entries go.

- `lib/bindings/c`: both target tables that held dynamo-llm. `cargo tree -i dynamo-llm -p libdynamo_llm` finds nothing now.
- The TensorRT-LLM sidecar: the dependency and its `mm-routing` feature, which only forwarded to `dynamo-llm/mm-routing`. Nothing the sidecar depends on is gated by it any more (the preprocessor is only a dev-dependency), so the forward is removed, not re-pointed: `default` is `["jemalloc"]`, and the Python bindings' `mm-routing` no longer lists `dynamo-trtllm-sidecar/mm-routing`. dynamo-llm also leaves the trees of the LPX sidecar and the TensorRT-LLM mocker server, which reached it through this crate.
- The SGLang sidecar: the dependency. `dynamo-sglang-generate` still brings dynamo-llm, and with it the preprocessor, into this binary, so `mm-routing` stays and names the crate that owns the gated code: `dynamo-preprocessor/mm-routing`, with `dynamo-preprocessor` as the manifest entry the forward needs.
- The vLLM and SGLang mocker servers: the dev-dependency. Both still reach dynamo-llm through the sidecar crates they wrap.
- The root lock file loses the five edges and the `lib/bindings/python` one the two sidecar edges; both gain the SGLang sidecar's edge to `dynamo-preprocessor`. No package enters or leaves either file.

What the shipped builds compile does not change. The sidecar image builds the four sidecars with one `cargo build`, and the wheel is one `maturin build`; `cargo tree -e normal -f '{p}|{f}'` over each lists the same packages with the same features before and after (778 and 810 lines), except that `dynamo-trtllm-sidecar` itself no longer shows `mm-routing`, a feature that gated no code in it. `dynamo-preprocessor` keeps `mm-routing` in the SGLang sidecar's own tree.

What does change: a build of only the TensorRT-LLM sidecar, the LPX sidecar, the TensorRT-LLM mocker server or the C library no longer compiles dynamo-llm and the 59 (C library: 28) packages only it brought, and resolves fewer features on the packages that stay, because dynamo-llm no longer asks for them (for example `dynamo-kv-router/metrics`, `dynamo-mocker/zmq-events`, `tonic/zstd`). The TensorRT-LLM and LPX SBOMs, which `cargo cyclonedx` writes per manifest, shrink the same way.

The vLLM sidecar keeps dynamo-llm: it imports `dynamo_llm::generate_format`. No source changes apart from one crate doc line and the Dockerfile comment that describes the SBOM arrangement.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…nces at the crates that own the code

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. With the split done, dynamo-llm is the frontend and the KV block manager that is being removed separately. Its agent guide says so, and the comments and docs that still placed moved code in `lib/llm` name the crate that owns it now.

- `lib/llm/AGENTS.md` opens with what the crate owns (the HTTP and gRPC services, the entrypoint, the serving-load publisher, the `/generate` format registry, the mocker adapter), the crate map with the dependency direction (frontend, discovery, router host, sync, then the protocols and the model card, with the preprocessor and telemetry beside sync), where the worker-side code and the Pool Relay are, and how to check the direction. The existing sections stay; the N-2 section names the crates that hold the card, the discovery metadata and the wire types. `CLAUDE.md` is unchanged.
- Router crates: the five `host.rs` lists that are labelled as what the frontend host names say which crates that is now; the cuckoo module and its README name `lib/pool-relay`; the KV indexer's `api.rs`, `residency.rs` and `lower_tier_indexers.rs`, the slot tracker's `multi_worker.rs` and `dynamo-kv-events`' `residency.rs` name `lib/sync`; the scheduler names `lib/model-card` for `ModelRuntimeConfig`; the `ActiveLoad` notes in `dynamo-router-protocol`, the kv-router compat list and its README name `dynamo-sync` and the message's readers.
- `dynamo-runtime`: four comments in three files name `lib/model-card`, `dynamo-router-host` and `dynamo-http-server` where they named `lib/llm` or dynamo-llm; `dynamo-kv-hashing` names `dynamo-preprocessor` for placeholder handling.
- `lib/frontend-protocols`: the command that rewrites the media schemas is `cargo test -p dynamo-frontend-protocols`, and its agent guide names `dynamo-preprocessor` for the media loader. `lib/model-card/AGENTS.md` lists the crates above it.
- The router testing guide points at `lib/router-host` and `lib/sync` and runs their tests.

Comments and Markdown only; no code changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The adapter that serves a `dynamo-mocker` scheduler as a token engine on the runtime was the last worker-side code in dynamo-llm, the frontend. It becomes the workspace crate `dynamo-mocker-engine` at `lib/mocker/engine`, next to the mocker: `dynamo_llm::mocker::X` is `dynamo_mocker_engine::X`. The adapter imported nothing from the rest of dynamo-llm, and dynamo-llm used nothing of it; its only caller is the Python bindings (`make_mocker_engine`).

- `lib/llm/src/mocker.rs` is the crate root, and `mocker/{handoff, handoff_tests, metrics, sglang}.rs` are its modules under the same names. Five files move with `git mv` (6,447 lines with tests); the only source edit is the crate root's doc header. Item names and visibility do not change: the public API is still `make_mocker_engine` and `MOCKER_COMPONENT`, and the `pub(crate)` items of the submodules reach the same code as before. Its 50 tests move with it.
- Why a new crate and not `dynamo-mocker`: `dynamo-backend-common` depends on `dynamo-mocker` (`fpm_publisher.rs` publishes its forward-pass snapshot types), and the adapter depends on `dynamo-backend-common` (`kv_publisher` and `fpm_publisher`), so the adapter inside `dynamo-mocker` is a cycle. It would also give `dynamo-mocker`, which needs no runtime today, `dynamo-runtime`, `dynamo-frontend-protocols` and `dynamo-preprocessor`.
- Why not an existing crate above `dynamo-backend-common`: the mocker servers (`lib/mocker/servers/{vllm, sglang, trtllm}`) are gRPC servers that imitate one engine each and depend on that engine's sidecar, and `dynamo-mocker-backend` (`lib/backend-common/examples/mocker`) is a binary-only example of the `LLMEngine` trait that models neither KV events, forward-pass metrics nor the handoff. The adapter is engine-neutral library code the Python bindings link, and belongs to none of them.
- No item needed widening and no in-place cut: no code outside the module named a crate-private item of it.
- No compatibility re-export: `dynamo_llm::mocker` is gone, and the Python bindings (`rust/llm/entrypoint.rs`) name the new crate.
- `dynamo-preprocessor` is a dependency of the new crate for one name, `backend::ExecutionContext`, the return type of `make_mocker_engine`. It is an alias defined there for `ServerStreamingEngine<PreprocessedRequest, Annotated<LLMEngineOutput>>`, whose parts are `dynamo-runtime`'s and `dynamo-frontend-protocols`'; spelling the type out would drop the edge, which is a change to the signature and is not made here. Two tests use `dynamo_router_host::migration::is_migratable`, so `dynamo-router-host` is a dev-dependency.
- `dynamo-mocker`'s `zmq-events` feature, which the adapter needs for `ZmqKvEventSink`, is enabled by the new crate. No cargo feature gates the moved code.
- dynamo-llm keeps `dynamo-mocker` for the `kv_router_bench` bench only (`loadgen::RouterSequence`): the entry moves to its dev-dependencies, without `zmq-events`. No other dependency of dynamo-llm was used by the adapter alone.
- Log statements in the moved code (22, all in the crate root, none with an explicit target) report `dynamo_mocker_engine` where they reported `dynamo_llm::mocker`. `tests/router/test_router_e2e_with_mockers.py` names the new target in its `DYN_LOG` filter; no other filter, launch script, recipe or doc names the old one. The `dynamo_llm` split entry on the deprecations page gains the module row and the log-target row.
- `lib/mocker/engine/AGENTS.md` states what the crate owns and must not depend on. Two comments that pointed at the adapter name its new home: the `LLMEngine` example and `dynamo-mocker`'s bootstrap transport.
- Codeowners: `lib/mocker/engine/` is runtime's and performance's, as `lib/llm/src/mocker/` was. The crate root, which as `lib/llm/src/mocker.rs` matched only the `lib/llm/` rule (runtime), gains performance as a co-owner; no file loses an owner. The CI path filters match `lib/**` and `**/*.rs`, and no labeler rule named the adapter, so neither changes.

What the shipped builds compile: the wheel's `cargo tree -e normal -f '{p}|{f}'` gains `dynamo-mocker-engine` and is otherwise identical line for line (same packages, same features). The sidecar image's (one `cargo build` of the four sidecars, 778 lines) differs in one line: `dynamo-mocker` no longer has `zmq-events`, which only dynamo-llm asked for there. The feature gates `dynamo_mocker::services::zmq_events` alone, which no sidecar uses. The KVBM bindings lose it the same way (their lock file drops `tmq` from `dynamo-mocker`).

No mocker timing, token output, handoff or KV event behaviour, metric name or label, log message or env var changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…eprocessor

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The `/generate` format contract is about to become the crate `dynamo-generate-format`, and with it the vLLM sidecar stops needing dynamo-llm. Its `mm-routing` feature still named dynamo-llm: it forwarded to `dynamo-llm/mm-routing`, which itself only forwards to `dynamo-preprocessor/mm-routing` and gates no code in dynamo-llm. The feature now names the crate that owns the gated code, as the SGLang sidecar's does.

- `lib/sidecar/vllm/Cargo.toml`: `mm-routing = ["dynamo-preprocessor/mm-routing"]`. `dynamo-preprocessor` is a dependency of the sidecar already: `engine.rs` calls `dynamo_preprocessor::mm_routing::image::resolve_exact_routing_image_token_id` behind this feature.
- `default` is still `["mm-routing", "jemalloc"]`, and the Python bindings' `mm-routing` still lists `dynamo-vllm-sidecar/mm-routing`.
- No lock file changes: features are not recorded there.

What the shipped builds compile does not change. `cargo tree -e normal -f '{p}|{f}'` over the sidecar image's one `cargo build` (four `-p`) lists the same 778 package and feature-set lines except that `dynamo-llm` no longer shows its forwarding feature `mm-routing`; `dynamo-preprocessor` keeps `mm-routing` and `dynamo-vllm-sidecar` keeps `default,jemalloc,mm-routing`. Over the wheel's feature set the 810 lines are identical, because the bindings enable `dynamo-llm/mm-routing` themselves.

The dynamo-llm entry stays in the manifest for one more commit: a test of the sidecar imports `dynamo_llm::generate_format`.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…public

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The `/generate` format contract is about to become the crate `dynamo-generate-format`. The frontend's own HTTP tests install two stand-in formats from `generate_format::test_formats`, because dynamo-llm cannot link the real format crates. They stay with the contract they implement, so the tests that remain in dynamo-llm will reach them across a crate boundary.

- `generate_format::test_formats` and its `install` function are `pub` (they were `pub(crate)`). These are the only two crate-private items of the module that code outside it uses: `http/service/service_v2.rs` (one test) and `http/service/systemone/tests.rs` (three tests).
- The module is still `#[cfg(test)]` here; the next commit puts it behind a `testing` feature of the new crate.

Visibility only; no item moves and no behaviour changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…namo-generate-format

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The contract an engine-native `/generate` format implements for the frontend (`GenerateFormat`, `GenerateExchange`, `ClientRequest`, `Opened`, `Bound`, `Worker`, `ClientError`, `WorkerError`, `ErrorPhase`, `json_body_error`) and the registry the frontend selects a format from (`Registry`, `SharedFormat`, `install`, `named`, `selected`, `FORMAT_ENV`) become the workspace crate `dynamo-generate-format` at `lib/generate-format`: `dynamo_llm::generate_format::X` is `dynamo_generate_format::X`. This module was the only reason the format crates, and through them the vLLM and SGLang sidecars and their mocker servers, linked the frontend.

- `lib/llm/src/generate_format.rs` is the crate root and `generate_format/error.rs` its `error` module: two files move with `git mv` (490 lines). The source edits are the crate root's doc header and the gate of `test_formats` (below). Item names do not change.
- Why a new crate: the module imports `dynamo-frontend-protocols` (`PreprocessedRequest`, `LLMEngineOutput`), `dynamo-http-server` (`metrics`), `dynamo-error` with `http`, `dynamo-runtime`, `axum` and `mime`. `dynamo-http-server` cannot hold it: `dynamo-frontend-protocols` depends on `dynamo-http-server`, so that is a cycle. `dynamo-frontend-protocols` could, without a cycle, but it holds wire types, not a plugin contract with a process-wide registry, and it would gain two direct dependencies it does not have, `dynamo-error` and `mime`. The new crate sits in the family of `lib/generate-formats/*` and `lib/generate-format-catalog`.
- `test_formats`, the two stand-in formats that dynamo-llm's HTTP tests install, moves with the contract it implements. It was `#[cfg(test)]`; it is now behind the new crate's `testing` feature, which also brings its one extra dependency, `dynamo-model-card` (the two capability names). dynamo-llm enables the feature in its dev-dependencies. The previous commit made the module and its `install` public.
- No compatibility re-export: `dynamo_llm::generate_format` is gone and every caller names the crate: dynamo-llm's `http/service/{format_generate, service_v2, systemone}.rs` and `systemone/tests.rs`, `dynamo-sglang-generate`, `dynamo-vllm-generate`, `dynamo-generate-format-catalog` and its HTTP test, the Python bindings (`rust/lib.rs`), and one test of the vLLM sidecar (`src/convert/request_tests.rs`). 35 paths change in 16 files.
- Manifests: `dynamo-sglang-generate`, `dynamo-vllm-generate` and the vLLM sidecar lose dynamo-llm (the sidecar's last use of it was that test; it gains `dynamo-generate-format` as a dev-dependency). In `dynamo-generate-format-catalog` dynamo-llm moves to the dev-dependencies, for the HTTP test that serves the frontend's routes. dynamo-llm and the Python bindings gain `dynamo-generate-format`. No dependency of dynamo-llm was used by the module alone (`mime`, `axum` and `dynamo-error` have other users there).
- No cargo feature gated the moved code, and it has no log statements, so no log target changes. The `dynamo_llm` split entry on the deprecations page gains the module row.
- `lib/generate-format/AGENTS.md` states what the crate owns and must not depend on.
- Codeowners: `lib/generate-format/` is frontend's, as `lib/llm/src/generate_format.rs` and `lib/llm/src/generate_format/` were. The CI path filters match `lib/**` and `**/*.rs`, and no labeler rule named the module, so neither changes.

dynamo-llm leaves the normal dependency tree of `dynamo-vllm-sidecar`, `dynamo-sglang-sidecar`, `dynamo-vllm-mocker`, `dynamo-sglang-mocker`, `dynamo-sglang-generate`, `dynamo-vllm-generate` and `dynamo-generate-format-catalog`: `cargo tree -e normal -i dynamo-llm -p dynamo-vllm-sidecar` finds nothing. The vLLM sidecar's normal tree goes from 761 to 746 packages and the SGLang sidecar's from 760 to 746.

What the shipped builds compile. The wheel: `cargo tree -e normal -f '{p}|{f}'` over its feature set gains `dynamo-generate-format` and is otherwise identical line for line. The sidecar image (one `cargo build` of the four sidecars): the four sidecar crates keep their features (`default,jemalloc,mm-routing` for vLLM and SGLang) and `dynamo-preprocessor` keeps `mm-routing`, so the gated code is compiled as before. The image no longer compiles dynamo-llm and the fourteen packages only it brought (`dynamo-discovery`, `dynamo-router-host`, `dynamo-decisions`, `dynamo-protocols` 8.5.0, `utoipa` 6, `utoipa-gen` 6, `utoipa-swagger-ui`, `dialoguer`, `shell-words`, `bincode`, `bincode_derive`, `unty`, `virtue`, `oneshot`), and nine packages that stay resolve fewer features because only dynamo-llm asked for them: `dynamo-kv-router` loses `metrics` (the metrics of the standalone KV indexer service, which no sidecar runs), `tonic` 0.13 loses `zstd`, and `zstd`, `zstd-safe`, `zstd-sys`, `base64` 0.23, `console`, `mime_guess` and `syn` 3 lose default or codec features. The four sidecars compile alone and together without them.

No status code, response body, metric category or env var of `/generate` and `/inference/v1/generate` changes. The generated OpenAPI document is byte-identical.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…ks dynamo-llm

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. After the two moves before this commit no sidecar and no mocker server has dynamo-llm in its dependency tree, in normal or in dev edges, and nothing else was left to remove: no manifest of a sidecar, a mocker server, `dynamo-sidecar-common`, `dynamo-backend-common` or a format crate names dynamo-llm, except the dev-dependencies of `dynamo-generate-format-catalog` and `dynamo-sidecar-testkit`, whose tests serve the frontend's HTTP routes. Two comments still described the old arrangement.

- `lib/sidecar/sglang/Cargo.toml`: the note on `dynamo-preprocessor` said the frontend crate behind `dynamo-sglang-generate` links the preprocessor into the binary. It no longer does. The entry is now the only thing that puts the preprocessor into the SGLang sidecar's tree, and it exists for the `mm-routing` forward; the note says so. The feature, `default` and the dependency are unchanged.
- `lib/sidecar/Dockerfile`: the note on the shared `cargo build` no longer says that each separate build recompiled dynamo-llm.

Comments only. Not done, for a decision: removing the SGLang sidecar's `mm-routing` feature and its `dynamo-preprocessor` entry, as an earlier commit of this layer did for the TensorRT-LLM sidecar when the same thing became true there. It would take 45 packages out of a build of only the SGLang sidecar (746 to 701 in its normal tree) and change the sidecar's default feature list and the Python bindings' `mm-routing` list, so it is left to its owner.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…erate-format contract

DYNO-232, the cleanup layer of splitting lib/llm into smaller crates. The two commits that moved the mocker adapter and the `/generate` format contract out of dynamo-llm left the guides that describe the crate map behind.

- `lib/llm/AGENTS.md`: dynamo-llm owns the HTTP and gRPC services, the entrypoint and the serving-load publisher. The crate map gains `dynamo-generate-format`, with a note that the format crates and their catalog depend on the contract and not on the frontend; the worker-side note says dynamo-llm depends on `dynamo-backend-common` for the worker entrypoint only; a new note places `dynamo-mocker-engine`; the check lists the crates that must not have dynamo-llm in their tree, the sidecars included.
- `lib/mocker/README.md` gains a short section on the crates next to `dynamo-mocker`: `dynamo-mocker-engine`, and why it is not part of `dynamo-mocker`, and the mocker servers.
- `lib/backend-common/AGENTS.md` names the two crates that depend on its worker-side code, and `lib/preprocessor/AGENTS.md` says the vLLM and SGLang sidecars forward `mm-routing` themselves.

Markdown only; no code changes.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani force-pushed the idhanani/dyno-231-pool-relay branch from c0e93b8 to c1eedd8 Compare November 16, 2026 20:34
@ishandhanani
ishandhanani force-pushed the idhanani/dyno-232-cleanup branch from 9ecdd77 to 5949a86 Compare November 16, 2026 20:34

This branch had an error being deployed

1 failed deployment
external_collaborator — 5949a867 Deployed Nov 16, 2026 by ishandhanani via ok-to-test #742
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant