Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
76da164
feat(triton): Deliver dynamo.triton Functionality
whoisj Aug 25, 2026
53773b4
adopt changes required by @dagil-nvidia
whoisj Aug 28, 2026
3488b4a
apply changes recommended by @coderabbitai
whoisj Aug 28, 2026
bd489bc
annotate CODEOWNERS file
whoisj Aug 28, 2026
9f24770
adopt changes recommended by @nv-rinig
whoisj Sep 2, 2026
f59b878
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 2, 2026
42a2e0b
changes requested by @dagil-nvidia
whoisj Sep 4, 2026
7eeaf15
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 4, 2026
39f8599
Update components/src/dynamo/triton/main.py
whoisj Sep 4, 2026
1ce3572
Update components/src/dynamo/triton/handlers.py
whoisj Sep 4, 2026
481f30e
Merge remote-tracking branch 'dst/main' into jwyman/dynamo-triton
whoisj Sep 9, 2026
9342635
Update components/src/dynamo/triton/handlers.py
whoisj Sep 9, 2026
2c71913
adopt changes suggested by @tanmayv25
whoisj Sep 9, 2026
611864b
fixup unit test issues
whoisj Sep 9, 2026
2595c5b
fix github edit corruption
whoisj Sep 9, 2026
6a78bce
Update components/src/dynamo/triton/tests/conftest.py
whoisj Sep 9, 2026
1ef6553
fix codeowners' areas
whoisj Sep 9, 2026
2a65977
feat(triton): Deliver Build, Test, and CI Functionality (#13943)
whoisj Sep 10, 2026
0409e7a
apply change suggested by @Vinya456
whoisj Sep 10, 2026
1a8f446
exclude triton from mypy check
whoisj Sep 10, 2026
56b2681
improve pytest skip logic
whoisj Sep 10, 2026
7ef340f
fix(triton): return None from pytest_ignore_collect for non-triton paths
nv-rinig Sep 11, 2026
c8ecb9a
fix(tests): return None from directory pytest_ignore_collect hooks
nv-rinig Sep 11, 2026
47c566c
fix(tests): seed the real triton package before pytest collection
nv-rinig Sep 11, 2026
10e4822
attempt skip tests when torch unavailable
whoisj Sep 11, 2026
ecd8f17
fix(tests): use try/except torch guard so E402 passes ruff and flake8
nv-rinig Sep 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 29 additions & 22 deletions .github/FILTERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,35 @@ The `filters.yaml` file controls which CI jobs run based on changed files.

When you open a PR, CI checks which files changed and runs only relevant jobs:

| Filter | Triggers |
|--------|----------|
| `core` | Main test suite (vLLM, SGLang, TRT-LLM containers) |
| `dev_images` | dev / local-dev image builds only (no runtime or GPU jobs) |
| `operator` | Kubernetes operator tests |
| `snapshot` | Checkpoint-placeholder image + all-framework standalone Snapshot deploy tests (github.com/ai-dynamo/snapshot is external; this covers Dynamo's integration surface) |
| `snapshot_vllm` / `snapshot_sglang` / `snapshot_trtllm` | That framework's checkpoint deploy suite |
| `deploy` | Deploy-specific tests |
| `vllm` / `sglang` / `trtllm` | Backend-specific tests |
| `sidecar` | Unified multi-architecture sidecar image build, publish, and compliance checks for changes under `lib/sidecar/**` (docs excluded), its shared workflow, and shared compliance inputs |
| `benchmarks` | Dynamo runtime pipeline (runs `tests/benchmarks/**` pytest suite) |
| `sample` | Sample-backend unified test (piggybacks on vllm image) |
| `efa` | EFA runtime image builds for vLLM, SGLang, TRT-LLM (`container/templates/aws.Dockerfile` change) |
| `docs` | Docs Lint, Fern Configuration, Docs Website Composition, and Fern Broken Links checks; Fern preview or publish workflow |
| `fern_components` | Parse custom MDX components (a step inside Fern Configuration Check) |
| `examples` | Recipe Kustomize generation and docs-artifact unit checks |
| `ignore` | Nothing (classification only) |
| `rust` | Rust pre merge checks |

> **Note:** `ignore` doesn't directly trigger CI jobs. It exists to satisfy coverage requirements - every file must match at least one filter. Sidecar source and proto files also match `rust`, so the existing workspace Rust checks cover sidecar tests before the image is built and published. `docs` gates the Docs Lint, Fern Configuration Check, Docs Website Composition Check, and Fern Broken Links Check jobs in `pre-merge.yml`. `examples` gates Recipe Check.

> **TODO:** The sidecar image also consumes root Cargo files, shared libraries, and composite actions. Expanding the filter to cover every remaining build input is deferred until the additional PR CI fan-out is evaluated and agreed.
| Filter | Triggers |
| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `core` | Main test suite (vLLM, SGLang, TRT-LLM containers) |
| `dev_images` | dev / local-dev image builds only (no runtime or GPU jobs) |
| `operator` | Kubernetes operator tests |
| `snapshot` | Checkpoint-placeholder image + all-framework standalone Snapshot deploy tests (github.com/ai-dynamo/snapshot is external; this covers Dynamo's integration surface) |
| `snapshot_vllm` / `snapshot_sglang` / `snapshot_trtllm` | That framework's checkpoint deploy suite |
| `deploy` | Deploy-specific tests |
| `vllm` / `sglang` / `trtllm` / `triton` | Backend-specific tests |
| `sidecar` | Unified multi-architecture sidecar image build, publish, and compliance checks for changes under `lib/sidecar/**` (docs excluded), its shared workflow, and shared compliance inputs |
| `benchmarks` | Dynamo runtime pipeline (runs `tests/benchmarks/**` pytest suite) |
| `sample` | Sample-backend unified test (piggybacks on vllm image) |
| `efa` | EFA runtime image builds for vLLM, SGLang, TRT-LLM (`container/templates/aws.Dockerfile` change) |
| `docs` | Docs Lint, Fern Configuration, Docs Website Composition, and Fern Broken Links checks; Fern preview or publish workflow |
| `fern_components` | Parse custom MDX components (a step inside Fern Configuration Check) |
| `examples` | Recipe Kustomize generation and docs-artifact unit checks |
| `ignore` | Nothing (classification only) |
| `rust` | Rust pre merge checks |

> [!NOTE]
> `ignore` doesn't directly trigger CI jobs.
> It exists to satisfy coverage requirements - every file must match at least one filter.
> Sidecar source and proto files also match `rust`, so the existing workspace Rust checks cover sidecar tests before the image is built and published.
> `docs` gates the Docs Lint, Fern Configuration Check, Docs Website Composition Check, and Fern Broken Links Check jobs in `pre-merge.yml`.
> `examples` gates Recipe Check.

> [!TODO]
> The sidecar image also consumes root Cargo files, shared libraries, and composite actions.
> Expanding the filter to cover every remaining build input is deferred until the additional PR CI fan-out is evaluated and agreed.

## Fixing "Uncovered Files" Errors

Expand Down
7 changes: 6 additions & 1 deletion .github/actions/changed-files/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ outputs:
trtllm:
description: 'Whether trtllm files changed'
value: ${{ steps.filter.outputs.trtllm_any_modified }}
triton:
description: 'Whether triton files changed'
value: ${{ steps.filter.outputs.triton_any_modified }}
sample:
description: 'Whether sample-backend files changed'
value: ${{ steps.filter.outputs.sample_any_modified }}
Expand Down Expand Up @@ -176,6 +179,7 @@ runs:
echo "vllm: ${{ steps.filter.outputs.vllm_any_modified }}"
echo "sglang: ${{ steps.filter.outputs.sglang_any_modified }}"
echo "trtllm: ${{ steps.filter.outputs.trtllm_any_modified }}"
echo "triton: ${{ steps.filter.outputs.triton_any_modified }}"
echo "sample: ${{ steps.filter.outputs.sample_any_modified }}"
echo "frontend: ${{ steps.filter.outputs.frontend_any_modified }}"
echo "benchmarks: ${{ steps.filter.outputs.benchmarks_any_modified }}"
Expand All @@ -202,6 +206,7 @@ runs:
echo "vllm: ${{ steps.filter.outputs.vllm_all_modified_files }}"
echo "sglang: ${{ steps.filter.outputs.sglang_all_modified_files }}"
echo "trtllm: ${{ steps.filter.outputs.trtllm_all_modified_files }}"
echo "triton: ${{ steps.filter.outputs.triton_all_modified_files }}"
echo "sample: ${{ steps.filter.outputs.sample_all_modified_files }}"
echo "frontend: ${{ steps.filter.outputs.frontend_all_modified_files }}"
echo "benchmarks: ${{ steps.filter.outputs.benchmarks_all_modified_files }}"
Expand All @@ -212,7 +217,7 @@ runs:
shell: bash
run: |
# Combine all filter-specific files into one list
COVERED_FILES=$(echo "${{ steps.filter.outputs.api_docs_all_modified_files }} ${{ steps.filter.outputs.docs_all_modified_files }} ${{ steps.filter.outputs.examples_all_modified_files }} ${{ steps.filter.outputs.ignore_all_modified_files }} ${{ steps.filter.outputs.ci_all_modified_files }} ${{ steps.filter.outputs.core_all_modified_files }} ${{ steps.filter.outputs.dev_images_all_modified_files }} ${{ steps.filter.outputs.operator_all_modified_files }} ${{ steps.filter.outputs.snapshot_all_modified_files }} ${{ steps.filter.outputs.snapshot_vllm_all_modified_files }} ${{ steps.filter.outputs.snapshot_sglang_all_modified_files }} ${{ steps.filter.outputs.snapshot_trtllm_all_modified_files }} ${{ steps.filter.outputs.power_agent_all_modified_files }} ${{ steps.filter.outputs.deploy_all_modified_files }} ${{ steps.filter.outputs.dgdr_all_modified_files }} ${{ steps.filter.outputs.planner_all_modified_files }} ${{ steps.filter.outputs.sidecar_all_modified_files }} ${{ steps.filter.outputs.vllm_all_modified_files }} ${{ steps.filter.outputs.sglang_all_modified_files }} ${{ steps.filter.outputs.trtllm_all_modified_files }} ${{ steps.filter.outputs.sample_all_modified_files }} ${{ steps.filter.outputs.frontend_all_modified_files }} ${{ steps.filter.outputs.benchmarks_all_modified_files }} ${{ steps.filter.outputs.efa_all_modified_files }} ${{ steps.filter.outputs.rust_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u)
COVERED_FILES=$(echo "${{ steps.filter.outputs.api_docs_all_modified_files }} ${{ steps.filter.outputs.docs_all_modified_files }} ${{ steps.filter.outputs.examples_all_modified_files }} ${{ steps.filter.outputs.ignore_all_modified_files }} ${{ steps.filter.outputs.ci_all_modified_files }} ${{ steps.filter.outputs.core_all_modified_files }} ${{ steps.filter.outputs.dev_images_all_modified_files }} ${{ steps.filter.outputs.operator_all_modified_files }} ${{ steps.filter.outputs.snapshot_all_modified_files }} ${{ steps.filter.outputs.snapshot_vllm_all_modified_files }} ${{ steps.filter.outputs.snapshot_sglang_all_modified_files }} ${{ steps.filter.outputs.snapshot_trtllm_all_modified_files }} ${{ steps.filter.outputs.power_agent_all_modified_files }} ${{ steps.filter.outputs.deploy_all_modified_files }} ${{ steps.filter.outputs.dgdr_all_modified_files }} ${{ steps.filter.outputs.planner_all_modified_files }} ${{ steps.filter.outputs.sidecar_all_modified_files }} ${{ steps.filter.outputs.vllm_all_modified_files }} ${{ steps.filter.outputs.sglang_all_modified_files }} ${{ steps.filter.outputs.trtllm_all_modified_files }} ${{ steps.filter.outputs.triton_all_modified_files }} ${{ steps.filter.outputs.sample_all_modified_files }} ${{ steps.filter.outputs.frontend_all_modified_files }} ${{ steps.filter.outputs.benchmarks_all_modified_files }} ${{ steps.filter.outputs.efa_all_modified_files }} ${{ steps.filter.outputs.rust_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u)

# Get all modified files
ALL_FILES=$(echo "${{ steps.filter.outputs.all_all_modified_files }}" | tr ' ' '\n' | grep -v '^$' | sort -u)
Expand Down
15 changes: 14 additions & 1 deletion .github/actions/pytest-local/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ inputs:
default: 'true'
hf_token:
required: false
triton_artifactory_user:
description: 'Artifactory username, for the Triton suites that download QA models from it'
required: false
triton_artifactory_token:
description: 'Artifactory token, for the Triton suites that download QA models from it'
required: false
enable_coverage:
description: 'Enable coverage collection (true/false)'
required: false
Expand Down Expand Up @@ -66,6 +72,10 @@ inputs:
description: 'Is this a GPU test job?'
required: false
default: 'false'
verify_gpu:
description: 'Check the GPU through torch before running GPU tests. Set false for images without torch (Triton), leaving the suite to check the GPU it needs.'
required: false
default: 'true'
dd_flaky_retry_enabled:
description: 'Override Datadog automatic test retries for this pytest invocation; empty inherits the job environment'
required: false
Expand Down Expand Up @@ -103,13 +113,16 @@ runs:
DYNAMO_TEST_FRAMEWORK: ${{ inputs.framework }}
HF_XET_HIGH_PERFORMANCE: 1
PYTEST_DD_FLAKY_RETRY_ENABLED: ${{ inputs.dd_flaky_retry_enabled }}
TRITON_ARTIFACTORY_USER: ${{ inputs.triton_artifactory_user }}
TRITON_ARTIFACTORY_TOKEN: ${{ inputs.triton_artifactory_token }}
VERIFY_GPU: ${{ inputs.verify_gpu }}
run: |
# Sanitize test_type for filenames: remove commas and spaces. Exported
# for the per-stage upload steps' artifact names.
STR_TEST_TYPE=$(echo "${TEST_TYPE}" | tr ', ' '_')
echo "STR_TEST_TYPE=${STR_TEST_TYPE}" >> $GITHUB_ENV

if [[ "${GPU_TESTS}" == "true" ]]; then
if [[ "${GPU_TESTS}" == "true" && "${VERIFY_GPU}" != "false" && "${PYTEST_MARKS}" != *"triton_qa"* ]]; then
# Only GPU tests require MinIO (LoRA tests). Wait for the job service.
echo "⏳ Waiting for MinIO to be ready..."
MAX_ATTEMPTS=30
Expand Down
5 changes: 5 additions & 0 deletions .github/codeowners/areas.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@ areas:
- examples/backends/sglang/
- lib/mocker/servers/sglang/
- lib/sidecar/sglang/
- label: backend-triton
github_team: '@ai-dynamo/dynamo-backend-triton-codeowners'
path_globs:
- components/src/dynamo/triton/**
- docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/triton/
- label: kv-memory
github_team: '@ai-dynamo/dynamo-kv-memory-codeowners'
path_globs:
Expand Down
8 changes: 8 additions & 0 deletions .github/filters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# vllm -> vllm build and test
# sglang -> sglang build and test
# trtllm -> trtllm build and test
# triton -> triton build and test
# frontend -> frontend EPP image build, dynamo build-test
# benchmarks -> dynamo build-test (runs tests/benchmarks/** pytest suite)
# efa -> all framework EFA runtime image builds (changes to container/templates/aws.Dockerfile)
Expand Down Expand Up @@ -324,6 +325,7 @@ dgdr:
- 'tests/deploy/test_dgdr_utils.py'
- 'tests/deploy/dgd_utils.py'


planner:
- 'container/templates/planner.Dockerfile'
- 'components/src/dynamo/planner/**'
Expand Down Expand Up @@ -375,6 +377,12 @@ trtllm:
- '!**/*.md'
- '!**/*.rst'

triton:
- 'components/src/dynamo/triton/**'
- 'container/templates/triton_*'
- '!**/*.md'
- '!**/*.rst'

# sample -> sample-unified-test (framework-agnostic CPU-only backend
# reference implementation). Piggybacks on the vllm image, so a sample-only
# change triggers vllm-build + sample-unified-test. The common/backend ABC
Expand Down
23 changes: 22 additions & 1 deletion .github/workflows/build-on-demand.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ on:
description: 'Build and push trtllm image, tagged with branch name'
type: boolean
default: false
build_triton:
description: 'Build and push triton image, tagged with branch name'
type: boolean
default: false
build_operator:
description: 'Build and push operator image, tagged with branch name'
type: boolean
Expand Down Expand Up @@ -91,6 +95,20 @@ jobs:
build_timeout_minutes: 60
secrets: inherit

triton-build:
name: triton-runtime # This name overlaps with other triton jobs to group them in the UI
needs: [init]
if: inputs.build_triton
uses: ./.github/workflows/shared-build-image.yml
with:
framework: triton
target: runtime
cuda_version: '[""]'
platform: 'linux/amd64'
builder_name: ${{ needs.init.outputs.builder_name }}
build_timeout_minutes: 60
secrets: inherit

vllm-copy-to-acr:
name: vllm-runtime # This name overlaps with other vllm jobs to group them in the UI
needs: [vllm-build]
Expand Down Expand Up @@ -122,6 +140,9 @@ jobs:
copy_timeout_minutes: 10
secrets: inherit

# Triton images are published to AWS ECR only (no Azure ACR mirror), so there
# is no triton-copy-to-acr job.

# ============================================================================
# OPERATOR (build-only)
# ============================================================================
Expand Down Expand Up @@ -197,7 +218,7 @@ jobs:
name: Clean K8s builder if exists
runs-on: prod-default-small-v2
if: always()
needs: [init, vllm-build, sglang-build, trtllm-build, operator]
needs: [init, vllm-build, sglang-build, trtllm-build, triton-build, operator]
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/dynamo-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -189,9 +189,11 @@ jobs:
- name: Run mypy (components)
working-directory: /workspace
shell: bash
# We have to exclude the components/src/dynamo/triton folder from the mypy check
# due its required dependencies not being available at verification time.
run: |
MYPYPATH=components/src:lib/bindings/python/src
TARGETS=$(find components/src/dynamo -maxdepth 1 -mindepth 1 -type d | sort | tr "\n" " ")
TARGETS=$(find components/src/dynamo -maxdepth 1 -mindepth 1 -type d ! -name "triton" | sort | tr "\n" " ")
MYPYPATH=$MYPYPATH mypy --explicit-package-bases $TARGETS
- name: Run mypy (bindings)
working-directory: /workspace
Expand Down
Loading
Loading