Repository navigation
refactor(llm): move request trace, sinks and recorders into dynamo-telemetry - #220
Draft
ishandhanani wants to merge 2 commits into
Draft
ishandhanani wants to merge 2 commits into
ishandhanani wants to merge 2 commits into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
November 15, 2026 15:43 — with
GitHub Actions
Failure
ishandhanani
force-pushed
the
idhanani/dyno-224-telemetry
branch
from
November 16, 2026 07:07
d55c634 to
f853ab2
Compare
ishandhanani
force-pushed
the
idhanani/dyno-221-zmq-helper
branch
from
November 16, 2026 07:07
3ffce0e to
963805f
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 16, 2026 07:07 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 16, 2026 07:07 — with
GitHub Actions
Failure
This was referenced Nov 16, 2026
Step 1 of DYNO-224 (layer 7 of splitting lib/llm into smaller crates). request_trace and fpm_trace are about to move into their own crate below dynamo-llm. The preprocessor, the HTTP services, the router, the entrypoint and the FPM relay call 15 of their items that are `pub(crate)` today, which stops working across a crate boundary. Those 15 become `pub`, and nothing else: the list comes from compiling the moved crate and widening only what a caller outside it failed to reach. - request_trace: the `payload_stream` module, `X_REQUEST_ID_CONTEXT_KEY`, `start_tool_event_ingest_from_policy`, `record_llm_metric_tokens`, `record_backend_finish_reason_metadata`, `build_request_end_trace_state` with the `RequestEndTraceState` it returns, `finish_reason_metadata_handle`, `wrap_chat_request_end_stream`, `wrap_completion_request_end_stream`, `emit_router_selection` and `payload::http_header_capture_active`. The re-exports in request_trace/mod.rs are split so the rest stays `pub(crate)`. - fpm_trace: `FpmTrace`, `FpmTrace::publish_payload` and `init_from_env_with_shutdown`. Inside dynamo-llm the change has no effect. The remaining 27 `pub(crate)` declarations in the modules that move stay crate-private. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…lemetry Step 2 of DYNO-224 and layer 7 of splitting lib/llm into smaller crates. Everything in dynamo-llm that writes trace or metric records to sinks moves into a new workspace crate, dynamo-telemetry at lib/telemetry, below dynamo-llm: - lib/llm/src/request_trace (record types, policy, the in-process bus, the JSONL, S3, NATS and OTLP sinks, payload capture, agent-context and tool-event records, the tool relay, replay hashing), fpm_trace (forward-pass metrics persistence), recorder.rs (stream-to-JSONL recorder) and perf (response recording and logprob analysis) keep their module names: `dynamo_llm::request_trace::X` is now `dynamo_telemetry::request_trace::X`, likewise fpm_trace, recorder and perf - lib/llm/src/telemetry (the rotating gzip JSONL writer, the plain JSONL writer, the bus, the stream helpers and parse_sink_names) becomes the crate root: `dynamo_llm::telemetry::jsonl_gz` is now `dynamo_telemetry::jsonl_gz` - the integration tests logprob_analysis_integration and request_trace_nats_integration use nothing from dynamo-llm and move to lib/telemetry/tests. The request_trace_finish_metadata bench drives the preprocessor and stays in lib/llm/benches The 28 files move with `git mv`, unchanged apart from import paths (`crate::telemetry::` becomes `crate::`), the module declarations and crate doc in the new lib.rs, the replay path of one logprob test, which now points at lib/llm/tests/data, and the crate name in request_trace/AGENTS.md. No upward import had to be cut: these modules already import only the protocol, model-card, runtime, worker-selection, kv-hashing and tokens crates. Explicit log targets keep the string `dynamo_llm::request_trace`, and the message that names dynamo-llm when the S3 sink is not compiled in is unchanged; log statements without an explicit target now carry the `dynamo_telemetry::` module path. dynamo-llm keeps no re-exports: the preprocessor, the HTTP services, the entrypoint, the router's selection trace, the FPM relay, the bench and the KVBM bindings import dynamo_telemetry directly. The features request-trace-s3 and request-trace-bench move with the code they gate, and dynamo-llm forwards both, so `dynamo-llm/request-trace-s3` in the Python bindings and the bench's required feature keep working. In the new crate request-trace-s3 also turns on the optional object_store and uuid dependencies that only the S3 sink uses. dynamo-llm drops opentelemetry, opentelemetry_sdk, opentelemetry-otlp, object_store and the dev-dependency approx, which only the moved code used. The new crate depends on dynamo-runtime, dynamo-frontend-protocols, dynamo-model-card, dynamo-worker-selection, dynamo-kv-hashing, dynamo-tokens and the published dynamo-protocols. It does not depend on dynamo-llm. The OpenTelemetry span and logging setup in dynamo-runtime is not touched. Codeowners gives every moved file the owners it has today: the crate root, fpm_trace, recorder, perf and the tests go to runtime, the five files of the old telemetry module to observability, and request_trace stays shared by runtime, agents and observability. Two Python comments that cite the moved paths are updated. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyno-224-telemetry
branch
from
November 16, 2026 20:34
f853ab2 to
b269955
Compare
ishandhanani
force-pushed
the
idhanani/dyno-221-zmq-helper
branch
from
November 16, 2026 20:34
963805f to
702911c
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 16, 2026 20:34 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 16, 2026 20:34 — with
GitHub Actions
Failure
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Layer 7 of the lib/llm split stack (DYNO-224). Base: #219 (ZMQ socket helpers in
dynamo-runtime), so this PR shows only this layer. It moves everything indynamo-llmthat writes trace or metric records to sinks into a new workspace crate,dynamo-telemetry, atlib/telemetry.What moved
request_trace/(record types,DYN_REQUEST_TRACEconfig and policy, the in-process bus, the JSONL, S3, NATS and OTLP sinks, payload capture, agent-context and tool-event records, the tool relay, replay hashing),fpm_trace/(persistence of forward-pass metrics),recorder.rs(stream-to-JSONL recorder) andperf.rswithperf/(response recording and logprob analysis) keep their module names:dynamo_llm::request_trace::Xis nowdynamo_telemetry::request_trace::X, likewisefpm_trace,recorderandperf.telemetrymodule is the new crate root:jsonl_gz,jsonl,stream,busandparse_sink_nameslive atdynamo_telemetry::.logprob_analysis_integrationandrequest_trace_nats_integrationuse nothing fromdynamo-llmand move tolib/telemetry/tests. The benchrequest_trace_finish_metadatadrivesOpenAIPreprocessor, so it stays inlib/llm/benchesbehinddynamo-llm'srequest-trace-benchfeature.request_trace/AGENTS.mdand itsCLAUDE.md), moved withgit mv; 16 are byte-identical. Inside the other 12 only this changes:crate::telemetry::becomescrate::(9 lines), the newlib.rsgains a crate doc and four module declarations, one logprob test reads its replay from../llm/tests/data, the two integration tests importdynamo_telemetry, two doc comments drop thetelemetry::prefix, andrequest_trace/AGENTS.mdnames the new crate and its commands.lib/llm/srcgoes from 271,564 to 257,293 lines of Rust (369 to 345 files).dynamo-llmhas nopub use dynamo_telemetry.Commits
One in-place cut inside
dynamo-llm, which builds and passes on its own, then one move commit.dynamo-llmcrate::path of the moved modules, groupeduse crate::{...}and test-only ones included, finds onlyrequest_trace,telemetry,perfandrecorder, which all move. Everything else they need already comes fromdynamo-frontend-protocols,dynamo-model-card(LocalModel),dynamo-runtime(with the ZMQ helper from #219),dynamo-worker-selection(the router selection trace record),dynamo-kv-hashing,dynamo-tokensand the publisheddynamo-protocolspub(crate)declarations in the moved modules, some used by the preprocessor, the HTTP services, the router, the entrypoint and the FPM relaypub: exactly the ones a caller outside the moved crate failed to reach when compiled.request_trace: thepayload_streammodule,X_REQUEST_ID_CONTEXT_KEY,start_tool_event_ingest_from_policy,record_llm_metric_tokens,record_backend_finish_reason_metadata,build_request_end_trace_stateand theRequestEndTraceStateit returns,finish_reason_metadata_handle,wrap_chat_request_end_stream,wrap_completion_request_end_stream,emit_router_selection,payload::http_header_capture_active.fpm_trace:FpmTrace,FpmTrace::publish_payload,init_from_env_with_shutdown. The other 27 declarations stay crate-privatepubrequest-trace-s3andrequest-trace-benchgate moved codedynamo-telemetry, anddynamo-llmforwards them, sodynamo-llm/request-trace-s3in the Python bindings, the wheel build flags and the bench'srequired-featureskeep working unchanged. In the new craterequest-trace-s3also enables the optionalobject_store(featureaws) anduuiddependencies, which only the S3 sink usesobject_storedependency in the new crate, which code outside the S3 sink does not usedynamo-llmhelper, and nodynamo-llmtest uses a test-only item of theirs, so the new crate needs notestingfeature. The one replay file a logprob test reads stays inlib/llm/tests/dataand the test points at it, asdynamo-model-cardanddynamo-frontend-protocolsdoCallers
lib/llm/src(preprocessor.rs,http/service/{openai,anthropic,metadata,metrics}.rs,entrypoint/input.rs,entrypoint/input/http.rs,kv_router.rs,kv_router/embedded.rs,fpm_publisher.rs) and the bench now namedynamo_telemetry. Outsidelib/llmone file does: the KVBM bindings'leader.rs, which usesrecorder::Recorder.lib/bindings/python/rust/llm/fpm.rsreferences none of the moved modules and is unchanged.dynamo-llmandlib/bindings/kvbmgain the dependency.dynamo-llmdropsopentelemetry,opentelemetry_sdk,opentelemetry-otlp,object_storeand the dev-dependencyapprox, which only the moved code used. It keepsflate2(the RDMA media path),bytemuck(tokens.rs),rmp-serde,async-natsanduuid.areas.yamlgives every moved file the owners it has today:lib/telemetry/goes to runtime (it ownedfpm_trace,recorder,perfand the tests throughlib/llm/), the five files of the oldtelemetrymodule are listed under observability, andrequest_trace/stays shared by runtime, agents and observability.CODEOWNERSis regenerated.filters.yamlmatcheslib/**), and no docs page cited a moved path; thedynamo_llmsplit entry in the v1.7.0 section of the deprecations page lists this layer's module paths and log targets. The request-traces page still says the S3 sink needsdynamo-llmbuilt withrequest-trace-s3, which remains true through the forwarded feature.lib/telemetry/AGENTS.md(with itsCLAUDE.mdtwin) states what the crate owns, what it must not depend on, and how to check it.Dependency graph
cargo tree -p dynamo-telemetry | grep -c dynamo-llmis 0.dynamo-memory,nixl-sysandcudarcdo not appear in its tree.object_storedoes, with or withoutrequest-trace-s3:dynamo-lorabrings it in throughdynamo-model-cardanddynamo-frontend-protocols.dynamo-runtime,dynamo-frontend-protocols,dynamo-model-card,dynamo-worker-selection,dynamo-kv-hashing,dynamo-tokens, and the publisheddynamo-protocols.dynamo-runtimeis not touched.Behaviour
dynamo_llm::request_trace, and the error that namesdynamo-llmwhen the S3 sink is not compiled in is unchanged.dynamo_telemetry::...where they reporteddynamo_llm::..., as the statements moved by the earlier layers do. A log filter written againstdynamo_llm::fpm_traceordynamo_llm::recorderhas to name the new path; no filter in this repository does.Validation
Cargo commands ran on a remote builder with the repository toolchain (1.96.1).
cargo test -p dynamo-llm -p dynamo-frontend-protocols -p dynamo-model-card --no-fail-fast: 51 binaries, 4555 passed, 0 failed, 31 ignored.-p dynamo-telemetry: 53 binaries, 4555 passed, 0 failed, 31 ignored. The two extra binaries are the new crate's unit and doc tests.dynamo-llmunit tests go from 2632 passed and 5 ignored to 2455 and 4; the new crate has 177 and 1.cargo test -- --list: all 4586 baseline names exist at head; 4408 keep binary name and path (the 8 tests of the two moved integration binaries among them), 178 moved fromdynamo-llm's unit tests to the same leaf name indynamo-telemetry's; none is missing and none is new.cargo check -p dynamo-llm --all-targets,cargo clippy --no-deps -p dynamo-llm --all-targets -- -D warningsandcargo test -p dynamo-llm --no-fail-fastpass (45 binaries, 3241 passed, 0 failed, 21 ignored, the same per-binary counts as the baseline).cargo check --workspace --all-targets,cargo clippy --no-deps --workspace --all-targets -- -D warnings,cargo fmt --all -- --check, andcargo check --lockedinlib/bindings/python(also with--features request-trace-s3) pass.cargo check -p dynamo-telemetry --all-targetspasses alone, with--features request-trace-s3, and with both features under clippy-D warnings.cargo test -p dynamo-telemetrypasses alone (177 passed, 1 ignored in the unit tests; 6 passed and 2 ignored in the two integration binaries), and the 12 S3 sink tests, which onlyrequest-trace-s3compiles, pass with the feature.cargo check -p dynamo-llm --features request-trace-s3andcargo check -p dynamo-llm --lib --features request-trace-benchpass.cargo check -p dynamo-llm --bench request_trace_finish_metadata --features request-trace-benchstops on one error,E0063atbenches/request_trace_finish_metadata.rs:80, where the initializer ofNvCreateChatCompletionRequestlackspreserve_thinking,responses_tools,thinkingand one more field that the struct already has on main. It is the only error at head, so the bench's import ofdynamo_telemetryand its calls to the bench hooks resolve. This PR does not fix it.cargo macheteat the root,lib/bindings/python,lib/runtime/examplesandlib/telemetryreports no unused dependency.cargo metadata --lockedpasses for the four lock files (root,lib/bindings/python,lib/bindings/kvbm,lib/runtime/examples).pre-commit run --from-ref idhanani/dyno-221-zmq-helper --to-ref HEADpasses, andbuild_codeowners.py --strictpasses withCODEOWNERSmatchingemit_codeowners.py.cargo check --lockedinlib/bindings/kvbmpasses, which compiles the one caller outsidelib/llm.request_trace_nats_integrationtests are ignored by default and need NATS and etcd, at baseline and at head alike. The bench was not run.e3ea6d6d88for the cut,232bba4f1bfor the move).Closes DYNO-224