Skip to content

docs(blog): explain load-aware LoRA routing - #14526

Closed
biswapanda wants to merge 3 commits into
ai-dynamo:mainfrom
biswapanda:bis/load-aware-lora-blog
Closed

biswapanda wants to merge 3 commits into
ai-dynamo:mainfrom
biswapanda:bis/load-aware-lora-blog

Conversation

@biswapanda

@biswapanda biswapanda commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Publish a technical blog post explaining Dynamo's load-aware LoRA routing and placement control loop.
  • Describe demand-proportional replica sizing, sticky capacity-aware HRW placement, availability-first filtering, hysteresis, and optional MCF placement.
  • Report corrected single-seed simulation results for stateless Random, HRW, and MCF while separating desired-placement churn from modeled adapter-cache churn.
  • Document the current control-plane boundary between frontend-local desired placement, backend-reported availability, synthetic residency, and worker lifecycle management.
  • Add the article to blog navigation, the landing page, and both sidebar date-label selectors.
  • Add all six figures produced by the current LoRA simulation plot generator: two cross-workload comparisons and four workload traces.
  • Clarify in the routing-pressure generator and post that logical target counts include inactive cold-start pins and do not imply physical residency.
  • Incorporate an independent Claude Opus 5 review performed from public PR and repository material only.

Validation

  • python3 docs/fern/scripts/docs_lint.py --scan docs — passed with 0 errors and 8 pre-existing warnings.
  • cargo test -p dynamo-llm --test lora_simulation — 19 passed, 1 ignored.
  • cargo test -p dynamo-llm --test lora_simulation -- test_export_csv --ignored --nocapture — passed; regenerated values match the article.
  • python -m py_compile lib/llm/tests/lora_simulation/plot_lora_churn.py — passed.
  • python lib/llm/tests/lora_simulation/plot_lora_churn.py --save — passed; generated all six documented figures.
  • cmp checks between generated plots and committed blog assets — passed for all six figures.
  • cargo clippy -p dynamo-llm --test lora_simulation -- -D warnings — passed.
  • fern check --local --warnings — passed with 0 errors and 2 pre-existing warnings after generating the Python and Rust API references used in CI.
  • fern docs broken-links — found one pre-existing unrelated link error in pages/recipes/model-recipes/deepseek-v4-pro-0813.mdx; no blog link errors.
  • esbuild syntax validation for docs/fern/components — passed.
  • git diff --check — passed.

The reported performance results are deterministic, single-seed simulation results. They do not claim measured GPU adapter-load latency, TTFT, throughput, or end-to-end training-time improvements.

Summary by CodeRabbit

  • Documentation
    • Added the “Load-Aware LoRA Routing and Placement in Dynamo” engineering article, covering routing design, simulation results, configuration, metrics, operational guidance, limitations, and follow-up measurements.
    • Added the article to the blog landing page, navigation, and sidebar archive with its publication date and metadata.
    • Clarified route-pressure visualization terminology, including logical targets, inactive cold-start pins, and values exceeding 100%.

Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
@biswapanda
biswapanda temporarily deployed to external_collaborator September 9, 2026 05:12 — with GitHub Actions Inactive
@biswapanda
biswapanda temporarily deployed to external_collaborator September 9, 2026 05:12 — with GitHub Actions Inactive
@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 8c34a6a

@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot added docs external-contribution Pull request is from an external contributor documentation Improvements or additions to documentation labels Sep 9, 2026
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
@biswapanda
biswapanda temporarily deployed to external_collaborator September 9, 2026 05:36 — with GitHub Actions Inactive
@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test 86d802f

Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
@biswapanda
biswapanda temporarily deployed to external_collaborator September 9, 2026 06:32 — with GitHub Actions Inactive
@dynamo-ops

Copy link
Copy Markdown
Contributor

/ok to test aa820d8

@biswapanda biswapanda removed the external-contribution Pull request is from an external contributor label Sep 9, 2026
@biswapanda

Copy link
Copy Markdown
Contributor Author

/ok to test aa820d8

@biswapanda
biswapanda marked this pull request as ready for review September 9, 2026 07:16
@biswapanda
biswapanda requested review from a team as code owners September 9, 2026 07:16
@biswapanda
biswapanda enabled auto-merge (squash) September 9, 2026 07:16
@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4fd1c4db-b52e-4816-abf9-0421fb0f72da

📥 Commits

Reviewing files that changed from the base of the PR and between 4a205cb and aa820d8.

⛔ Files ignored due to path filters (6)
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/lora_churn_daily.png is excluded by !**/*.png
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/lora_churn_hot_lora_poisson.png is excluded by !**/*.png
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/lora_churn_mmpp.png is excluded by !**/*.png
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/lora_churn_spike.png is excluded by !**/*.png
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/lora_churn_summary.png is excluded by !**/*.png
  • docs/fern/pages/blog/_assets/load-aware-lora-routing/routing_pressure_vs_churn.png is excluded by !**/*.png
📒 Files selected for processing (5)
  • docs/fern/components/BlogLanding.tsx
  • docs/fern/components/BlogStyles.tsx
  • docs/fern/index.yml
  • docs/fern/pages/blog/2026/load-aware-lora-routing.mdx
  • lib/llm/tests/lora_simulation/plot_lora_churn.py

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

Adds a technical article on load-aware LoRA routing and placement in Dynamo. It includes navigation metadata, simulation methodology and results, operational guidance, and updated route-pressure visualization terminology.

Changes

Load-Aware LoRA Routing Article

Layer / File(s) Summary
Technical article content
docs/fern/pages/blog/2026/load-aware-lora-routing.mdx
Adds the article content, including routing state, placement strategies, simulation methods, workload results, operational instructions, limitations, and planned experiments.
Publication metadata and navigation
docs/fern/components/BlogLanding.tsx, docs/fern/index.yml, docs/fern/components/BlogStyles.tsx
Adds the article to the latest-articles list, Dynamo blog archive, and sidebar date labels.
Simulation visualization terminology
lib/llm/tests/lora_simulation/plot_lora_churn.py
Updates route-pressure documentation and chart text to distinguish logical targets, inactive pins, and resident adapter capacity.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~15 minutes

Merge Risk: ⚪ Minimal · up to aa820

This change publishes the load-aware LoRA routing article and clarifies simulation chart terminology. The article and its navigation entries are consistent, with no remaining merge-readiness risk identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description provides detailed scope and validation information, but it does not use the required template sections and omits the required Related Issues declaration. Add the required Overview, Details, Where should the reviewer start?, and Related Issues sections. In Related Issues, either link the relevant issue with a closing or relating statement, or check the confirmation that this PR has no related…
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: documenting load-aware LoRA routing in a technical blog post.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 3 files. (2 skipped: 2 …
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Description check

Resolution

Add the required Overview, Details, Where should the reviewer start?, and Related Issues sections. In Related Issues, either link the relevant issue with a closing or relating statement, or check the confirmation that this PR has no related issue.

  • Fix all pre-merge checks with AI

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

@github-actions

Copy link
Copy Markdown
Contributor

This PR has been closed due to inactivity. If you believe this PR is still relevant, please feel free to reopen it with additional context or information.

@github-actions github-actions Bot closed this Nov 17, 2026
auto-merge was automatically disabled November 17, 2026 09:36

Pull request was closed

This branch was previously deployed

1 inactive deployment
external_collaborator — aa820d81 Deployed Sep 9, 2026 by biswapanda via ok-to-test #15959
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs documentation Improvements or additions to documentation size/L Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants