Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions container/compliance/native_packages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
packages:
# --- EFA (Elastic Fabric Adapter) components ---
- name: libfabric
version: 2.6.0amzn1.0
version: 2.7.0amzn2.0
license: BSD-2-Clause
source: https://github.com/aws/libfabric
images:
Expand All @@ -44,7 +44,7 @@ packages:
- sglang-runtime-efa

- name: aws-ofi-nccl
version: 1.21.1
version: 1.22.0
license: Apache-2.0
source: https://github.com/aws/aws-ofi-nccl
images:
Expand Down
12 changes: 6 additions & 6 deletions container/context.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ dynamo:
nixl_gdrcopy_ref: v2.5.2
nixl_libfabric_repo: https://github.com/ofiwg/libfabric.git
# >= v2.5 for NIXL 1.5.0 (fi_mr_attr.device.rocr). <= the libfabric of
# efa_version (2.6.0): the TRT-LLM EFA image runs NIXL against that copy,
# which lacks the FABRIC_1.10 symbols that v2.7.0 links to.
# efa_version (2.7.0 in EFA 1.51.0): the TRT-LLM EFA image runs NIXL against
# that copy, so a newer ref can link to symbols that copy lacks.
nixl_libfabric_ref: v2.6.0
# Keep in sync with upstream NIXL's contrib/Dockerfile.manylinux.
hwloc_version: 2.12.2
Expand All @@ -90,7 +90,7 @@ dynamo:
nv_codec_headers_ref: "n13.0.19.0"
libvpx_ref: "v1.14.1"
sccache_version: "v0.14.0"
efa_version: 1.50.0
efa_version: 1.51.0

vllm:
cuda13.0:
Expand Down Expand Up @@ -127,7 +127,7 @@ vllm:
nixl_ref: v1.5.0
nixl_ucx_ref: v1.23.0-rc1
vllm_omni_ref: "v0.32.0rc1"
nixl_ref: v1.4.1
nixl_ref: v1.5.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 vLLM EFA misses the NIXL upgrade

When the upstream image carries older NIXL, changing nixl_ref leaves vLLM EFA on that version. The CUDA runtime extracts NIXL from upstream instead of the newly built wheel.

Learn more

The framework-level NIXL ref sets NIXL_REF for both wheel construction and runtime stages through the argument template. wheel_builder builds NIXL at that ref, but the CUDA vLLM runtime extracts its libraries from the upstream Python wheel. The AWS extension only copies builder NIXL for TensorRT-LLM. Thus the new value does not control the shipped vLLM CUDA/EFA NIXL version; any version difference between upstream and 1.5.0 remains in the image.

Example: If the pinned upstream vLLM image bundles NIXL 1.4.1, setting nixl_ref: v1.5.0 builds a 1.5.0 SDK but the published vLLM EFA image still loads 1.4.1 from its upstream wheel.

Recommended fix: Install the matching nixl and nixl-cu13 wheels in the CUDA runtime or explicitly overlay the builder's libraries and Python bindings there. Verify the effective Python package version and loaded libnixl.so in the final AWS image.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

max_jobs: "10"
enable_media_ffmpeg: "true"
enable_gpu_memory_service: "true"
Expand Down Expand Up @@ -173,7 +173,7 @@ sglang:
# (see templates/dev.Dockerfile). Runtime stays on the upstream lmsysorg/sglang
# NIXL Python stack — its wheel COPY is narrowed to ai_dynamo*.whl so the SDK
# build doesn't leak into the runtime image.
nixl_ref: v1.5.0
nixl_ref: v1.5.2
enable_media_ffmpeg: "true"
enable_gpu_memory_service: "true"
enable_modelexpress: "true"
Expand All @@ -187,7 +187,7 @@ trtllm:
runtime_image_tag: 1.4.0rc2
# Per-arch baseline stem (re-captured against tensorrt-llm/release, both arches).
baseline_sbom: release@93a3eaf1
nixl_ref: v1.3.1
nixl_ref: v1.5.2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 TensorRT-LLM pin also affects non-EFA images

nixl_ref upgrades Python packages in standard TensorRT-LLM images while LD_PRELOAD retains upstream libnixl.so. Check compatibility of the new bindings with that library.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

enable_media_ffmpeg: "false"
enable_gpu_memory_service: "true"

Expand Down
2 changes: 1 addition & 1 deletion container/deps/requirements.sglang.txt
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ joblib>=1.0,<1.6 # Keep the dependency-free support line for --no-deps installs
# `mooncake` package, so flooring the generic name overlays the CUDA 13 build
# with an unloadable one instead of upgrading it. CUDA only: stripped for the
# XPU image in sglang_runtime.Dockerfile.
mooncake-transfer-engine-cuda13>=0.3.13.post1,<0.4
mooncake-transfer-engine-cuda13>=0.3.14,<0.4
nltk>=3.10.3,<4
packaging>=23.1,<27
# The upstream SGLang image bundles pillow at an older release; floor it so the
Expand Down
2 changes: 1 addition & 1 deletion container/deps/requirements.vllm.txt
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ lmcache==0.5.6
# Both ship the same `mooncake` package and the generic one links
# libcudart.so.12, so the floor has to name the CUDA 13 distribution. CUDA
# only: stripped for the XPU and CPU images in vllm_runtime.Dockerfile.
mooncake-transfer-engine-cuda13>=0.3.13.post1,<0.4
mooncake-transfer-engine-cuda13>=0.3.14,<0.4
# The Dynamo wheel is installed with --no-deps, so list artifact dependencies here.
opendal==0.47.10
# The upstream vllm-openai image bundles pillow at an older release;
Expand Down
8 changes: 4 additions & 4 deletions container/templates/aws.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,17 @@ USER root

# Install AWS EFA installer with bundled libfabric and aws-ofi-nccl
# Flags explanation:
# --skip-kmod: Skip kernel module installation (handled by host)
# --skip-limit-conf: Skip ulimit configuration (handled by container runtime)
# --no-verify: Skip GPG verification (optional, can be removed if verification is needed)
# --build-ngc turns on --yes, --skip-kmod, --skip-limit-conf and --no-verify (Ubuntu only)
# --skip-rdma-core skips rdma-core packages
# --skip-mpi skips mpi packages
# Cache apt downloads; sharing=locked avoids apt/dpkg races with concurrent builds.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
curl --retry 3 --retry-delay 2 -fsSL -o aws-efa-installer-${EFA_VERSION}.tar.gz \
https://efa-installer.amazonaws.com/aws-efa-installer-${EFA_VERSION}.tar.gz && \
tar -xf aws-efa-installer-${EFA_VERSION}.tar.gz && \
cd aws-efa-installer && \
apt-get update && \
./efa_installer.sh -y --skip-kmod --skip-limit-conf --no-verify && \
./efa_installer.sh --skip-mpi --skip-rdma-core --build-ngc && \
cd .. && rm -rf aws-efa-installer* && \
ldconfig

Expand Down
Loading