Skip to content

refactor(error): move the error-class to HTTP status mapping into dynamo-error - #211

Draft
ishandhanani wants to merge 3 commits into
mainfrom
idhanani/dyn-llm-split-error-status
Draft

ishandhanani wants to merge 3 commits into
mainfrom
idhanani/dyn-llm-split-error-status

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

Step 2 of splitting lib/llm into smaller crates, and layer 1 of the 16-layer split stack: main <- #211 <- #209 <- #212 <- #213 <- #218 <- #219 <- #220 <- #221 <- #222 <- #223 <- #224 <- #225 <- #226 <- #227 <- #228 <- #229. The error-class to HTTP status policy moves out of dynamo_llm::http::service::error into dynamo-error, as the http module behind a new http cargo feature, so backend, preprocessor, kv_router and generate_format no longer import the HTTP frontend module to render or assert an error status.

  • Moved to dynamo_error::http: overload_status_code (with parse_overload_status_code), ClientErrorAction, http_action_for_class, http_action_for_error, find_canonical_error_in_chain, SanitizedError, BackendStatusAction (with keeps_retry_semantics), and their 14 unit tests. Names, bodies, messages and status codes are unchanged; the only edits are pub(crate) to pub on five items and dynamo_runtime::error::X to crate::X.
  • Stayed in dynamo_llm::http::service::error: HttpError (the type an engine returns to pick a raw status; it derives thiserror::Error, a dependency dynamo-error does not take, and it is not class vocabulary) and invalid_argument (a pub(crate) constructor only the OpenAI and Anthropic adapters call). The file goes from 551 to 37 lines.
  • Feature cost: http adds the http crate for StatusCode (the type axum re-exports) and nothing else. dynamo-error without the feature is unchanged.
  • One thing that is not a pure move: overload_status_code read the variable name from dynamo_runtime::config::environment_names::llm. dynamo-runtime depends on dynamo-error, so the module now spells DYN_HTTP_OVERLOAD_STATUS_CODE itself; a new test in dynamo-llm asserts the two constants agree. Same variable, same default (529), same parsing, same first-use caching.
  • No re-exports: every caller imports dynamo_error::http directly. dynamo-llm, dynamo-sglang-generate, dynamo-vllm-generate and the Python bindings gain a dynamo-error dependency with features = ["http"].
  • Import metric: files outside lib/llm/src/http/ that import http::service::error go from 10 to 2 (lib/bindings/python/rust/http.rs and lib/llm/tests/http-service.rs, both for HttpError). No dynamo-llm module outside http imports it any more, grpc included.

lib/llm/src/http/service/disconnect.rs changes on three import lines only, because the items it names moved and there is no re-export to fall back on.

Validation

Baseline is upstream main at 8e7d29e, which this branch is rebased on. Baseline and after-runs used the same remote 32 vCPU builder.

The numbers below were measured on that base. The stack has since been rebased onto upstream main d20e045; after the rebase this layer's head passed cargo check --workspace --all-targets, cargo check --locked in lib/bindings/python, cargo metadata --locked for the four lock files, cargo fmt --all -- --check and the Anthropic admission tests, and the top of the stack passed the whole-workspace cargo test --locked run (10,022 passed, 0 failed, 87 ignored), clippy, and the bindings, machete, pre-commit, codeowners and OpenAPI checks.

  • cargo test -p dynamo-llm -p dynamo-http-server -p dynamo-sglang-generate -p dynamo-vllm-generate -p dynamo-generate-format-catalog -p dynamo-global-frontend --no-fail-fast plus cargo test -p dynamo-error --all-features: before 65 test binaries, 4,825 passed, 0 failed, 33 ignored; after 65 test binaries, 4,826 passed, 0 failed, 33 ignored. The difference is the one new test.
  • Per crate, the only counts that change are dynamo-llm 4,623 to 4,610 (lib 3,999 to 3,986: 14 moved out, 1 added) and dynamo-error 34 to 48 (lib 32 to 46: 14 moved in). dynamo-http-server 56, dynamo-sglang-generate 18, dynamo-vllm-generate 37, dynamo-generate-format-catalog 12 and dynamo-global-frontend 45 are identical.
  • Test names: cargo test -p dynamo-llm --lib -- --list goes from 4,014 to 4,001 names. The 14 names that left dynamo_llm::http::service::error::tests all exist as dynamo_error::http::tests::<same name>; one name is new (overload_status_env_name_matches_runtime_registry); no other test name changed in any binary.
  • cargo check --workspace --all-targets: exit 0, also at each earlier commit of the branch, together with cargo check in lib/bindings/python.
  • cargo clippy --no-deps --workspace --all-targets -- -D warnings: exit 0.
  • cargo check --locked in lib/bindings/python: exit 0. cargo metadata --locked passes for the root, lib/bindings/python, lib/bindings/kvbm and lib/runtime/examples lock files (the last needed no change).
  • cargo machete in ., lib/bindings/python and lib/runtime/examples: no unused dependencies.
  • cargo fmt --all -- --check and pre-commit run --from-ref 8e7d29e0de --to-ref HEAD: clean.

Closes DYNO-216

…amo-error

The class-to-status policy lived in dynamo-llm's http::service::error, so
backend, preprocessor, kv_router, generate_format and the gRPC service all
imported the HTTP frontend module to render or assert an error status.

Move it into dynamo-error as the `http` module behind a new `http` feature
whose only added dependency is the `http` crate (the StatusCode axum
re-exports): overload_status_code, ClientErrorAction, http_action_for_class,
http_action_for_error, find_canonical_error_in_chain, SanitizedError and
BackendStatusAction, with their 14 unit tests. Items are unchanged apart from
visibility (pub(crate) to pub) and import paths.

dynamo-error sits below dynamo-runtime, so the module names the
DYN_HTTP_OVERLOAD_STATUS_CODE environment variable itself instead of importing
the runtime's environment_names registry.

HttpError (a thiserror type engines return to pick a raw status) and
invalid_argument (a constructor only the protocol adapters use) stay in
http::service::error. No re-exports: every caller imports dynamo_error::http.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
dynamo-error cannot import dynamo-runtime's environment_names registry, so
DYN_HTTP_OVERLOAD_STATUS_CODE is now spelled in both crates. Assert the two
constants agree from dynamo-llm, the lowest crate that sees both.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
State what the optional `http` feature adds to dynamo-error and that the
`http` crate is its only extra dependency.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani force-pushed the idhanani/dyn-llm-split-error-status branch from 5ef13fd to fc1f85b Compare November 16, 2026 20:34

This branch had an error being deployed

1 failed deployment
external_collaborator — fc1f85b6 Deployed Nov 16, 2026 by ishandhanani via ok-to-test #725
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant