Skip to content

add cookbook for Deepseek v4 pro PD disagg#23777

Draft
yan-lgtm wants to merge 1 commit intosgl-project:mainfrom
yan-lgtm:yan/deepseek-v4/doc-update
Draft

add cookbook for Deepseek v4 pro PD disagg#23777
yan-lgtm wants to merge 1 commit intosgl-project:mainfrom
yan-lgtm:yan/deepseek-v4/doc-update

Conversation

@yan-lgtm
Copy link
Copy Markdown

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds documentation and configuration for DeepSeek-V4 PD-Disagg on B200 clusters, including RDMA setup guides and memory optimizations to prevent OOM on 'big' models. Feedback suggests improving the RDMA troubleshooting script by iterating over all InfiniBand devices instead of hardcoding a single device name.

Comment on lines +194 to +197
for i in $(seq 0 7); do
v=$(cat /sys/class/infiniband/mlx5_0/ports/1/pkeys/$i 2>/dev/null)
[ -n "$v" ] && [ "$v" != "0x0000" ] && echo "pkey[$i]=$v"
done
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The troubleshooting script hardcodes mlx5_0 when checking the pkey table. However, the preceding text mentions that mlx5_7 is the typical default for B200, and the previous script iterates over all mlx5_* devices. Hardcoding mlx5_0 here might lead users to inspect the wrong device. It is better to iterate over all devices to ensure the active one is covered.

for d in /sys/class/infiniband/mlx5_*; do
  echo "--- $(basename $d) ---"
  for i in $(seq 0 7); do
    v=$(cat "$d/ports/1/pkeys/$i" 2>/dev/null)
    [ -n "$v" ] && [ "$v" != "0x0000" ] && echo "pkey[$i]=$v"
  done
done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant