Skip to content

fix(tests): include native sidecars in deployment diagnostics - #16288

Merged
julienmancuso merged 3 commits into
ai-dynamo:mainfrom
panpan0000:codex/16287-sidecar-test-diagnostics
Nov 5, 2026
Merged

julienmancuso merged 3 commits into
ai-dynamo:mainfrom
panpan0000:codex/16287-sidecar-test-diagnostics

Conversation

@panpan0000

@panpan0000 panpan0000 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Deployment readiness diagnostics omit native sidecars because they only inspect regular-container statuses. Include init-container statuses in pod status reporting and per-restart previous-log capture, including before regular containers start. Reuse the existing log capture and restart deduplication.

Closes #16287.

Validation

  • python -m pytest -q tests/deploy/test_dgd_utils.py: 21 passed.
  • The four sidecar regression cases fail before the fix; the regular-container restart case passes.
  • Changed-file pre-commit checks and git diff --check passed.

Validation uses mocked Kubernetes API responses; no live cluster test was run.

Summary by CodeRabbit

  • Bug Fixes
    • Pod status reports now include init containers and runtime sidecars, including before the main container starts.
    • Restart-log reporting now captures restarts of the main container or a runtime sidecar, whether or not the main container is present.
    • Repeated reports avoid capturing the same restart twice, while recognizing subsequent restarts. Diagnostics also handle pods with no status information.

@panpan0000
panpan0000 requested review from a team as code owners October 13, 2026 06:27
@copy-pr-bot

copy-pr-bot Bot commented Oct 13, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@panpan0000
panpan0000 temporarily deployed to external_collaborator October 13, 2026 06:27 — with GitHub Actions Inactive
@panpan0000
panpan0000 temporarily deployed to external_collaborator October 13, 2026 06:27 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi panpan0000! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor fix labels Oct 13, 2026
@coderabbitai

coderabbitai Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: ai-dynamo/dynamo/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 89a62243-7fa8-417b-87a5-905d3a410f94
📥 Commits

Reviewing files that changed from the base of the PR and between 2210b07 and 2f01463.

📒 Files selected for processing (2)
  • tests/deploy/dgd_utils.py
  • tests/deploy/test_dgd_utils.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

Pod status reporting and in-flight restart-log capture now include init-container statuses as well as regular-container statuses. Tests cover native sidecar status, restart-log capture, and repeated restart handling.

Changes

Native sidecar diagnostics

Layer / File(s) Summary
Pod status details
tests/deploy/dgd_utils.py, tests/deploy/test_dgd_utils.py
Pod status details include init-container and regular-container statuses. Tests cover a waiting runtime sidecar and a running main container.
In-flight restart-log capture
tests/deploy/dgd_utils.py, tests/deploy/test_dgd_utils.py
Restart-log capture scans init-container and regular-container statuses. Tests cover runtime sidecar restarts with or without main-container status, deduplication, and capture after the restart count increases.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Severity of issue fixed: Low

Merge Risk: ⚪ Minimal · up to 2f014

No actionable merge-blocking issue is established for the init-container diagnostic changes. The PR is mergeable after normal checks.

Pre-merge checks · Passed 4 · Failed 1

Failed Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: including native sidecars in deployment diagnostics.
Description check Passed The description explains the change, validation, test results, issue linkage, and test limitations. It does not use the template headings or identify where the reviewer should start, but it provides t…
Linked Issues check Passed The changes satisfy issue #16287. _get_pod_status_details and _dump_in_flight_restart_logs process init_container_statuses together with regular container statuses. Tests cover native sidecar st…
Out of Scope Changes check Passed The pull request changes only tests/deploy/dgd_utils.py and its focused regression tests. The implementation directly addresses issue #16287, and the test changes verify the required behavior. No un…
  • Fix all pre-merge checks with AI
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@panpan0000
panpan0000 temporarily deployed to external_collaborator October 13, 2026 11:27 — with GitHub Actions Inactive
@julienmancuso

Copy link
Copy Markdown
Contributor

@panpan0000 could you rebase ?

Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
@panpan0000
panpan0000 force-pushed the codex/16287-sidecar-test-diagnostics branch from f7c12c9 to 75bd463 Compare November 4, 2026 05:33
@panpan0000
panpan0000 temporarily deployed to external_collaborator November 4, 2026 05:33 — with GitHub Actions Inactive
@panpan0000

Copy link
Copy Markdown
Contributor Author

@julienmancuso done, thanks for reminder

@julienmancuso

Copy link
Copy Markdown
Contributor

/ok to test 75bd463

@julienmancuso
julienmancuso enabled auto-merge (squash) November 4, 2026 14:42
@panpan0000
panpan0000 temporarily deployed to external_collaborator November 5, 2026 03:49 — with GitHub Actions Inactive
@panpan0000

Copy link
Copy Markdown
Contributor Author

update branch to resolve old code CI issue

@panpan0000
panpan0000 temporarily deployed to external_collaborator November 5, 2026 03:57 — with GitHub Actions Inactive
@julienmancuso

Copy link
Copy Markdown
Contributor

/ok to test ed030ac

@julienmancuso
julienmancuso merged commit 0c2d81e into ai-dynamo:main Nov 5, 2026
123 checks passed
ishandhanani added a commit to ishandhanani/dynamo that referenced this pull request Nov 5, 2026
Fifth upstream merge for the split v2 PR: the four commits past
20c5f11, no conflicts, no router-owned path.

- ai-dynamo#18116 pass plugin selectors to the transcription router:
  lib/llm/src/discovery/watcher.rs (auto-merged).
- ai-dynamo#17691 DynamoDB registration store for the Global Discovery Service:
  lib/global-discovery-service, root Cargo.toml workspace deps, Cargo.lock.
- ai-dynamo#16288 deployment diagnostics tests, ai-dynamo#16723 helm KAI queue wait: outside
  the router.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

This branch was previously deployed

1 inactive deployment
external_collaborator — ed030ac8 Deployed Nov 5, 2026 by panpan0000 via ok-to-test #46338
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contribution Pull request is from an external contributor fix size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: Deployment test diagnostics omit native sidecar failures

2 participants