Skip to content

Fix XPU segfault when tensor_parallel_size exceeds available devices - #5

Closed
hongbolv with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-segmentation-fault-issue
Closed

hongbolv with Copilot wants to merge 2 commits into
mainfrom
copilot/fix-segmentation-fault-issue

Conversation

Copilot AI commented Mar 20, 2026 •

Copy link
Copy Markdown

Setting tensor_parallel_size higher than available XPU devices causes a segfault (null deref in Level Zero driver) because the device count validation in ParallelConfig only checked CUDA platforms. XPU workers spawn on non-existent device IDs with no guard.

  • Add device_count() classmethod to base Platform class (returns 0; already overridden on CUDA/ROCm/XPU subclasses)
  • Extend the world_size > device_count validation and multi-node nnodes > 1 check to cover XPU
  • Replace CUDA-specific cuda_device_count_stateless() with generic current_platform.device_count() in the validation path

With this change, XPU users get a clear ValueError instead of a driver-level segfault:

ValueError: World size (4) is larger than the number of available GPUs (2) in this node.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • astral.sh
    • Triggering command: /usr/bin/curl curl -LsSf REDACTED s/all_reduce_utils.py s/__�� s/custom_all_reduce.py s/cuda_communicator.py s/quick_all_reduce.py s/symm_mem.py s/cuda_wrapper.p--norc s/base_device_co--noprofile s/all2all.py s/sh�� s/ray_communicator.py s/cpu_communicator.py (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

…lel_size exceeds available devices

Co-authored-by: GitHub Copilot

Co-authored-by: hongbolv <33214277+hongbolv@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix segmentation fault in device initialization Fix XPU segfault when tensor_parallel_size exceeds available devices Mar 20, 2026
Copilot AI requested a review from hongbolv March 20, 2026 07:29
@hongbolv hongbolv closed this Mar 20, 2026
@hongbolv hongbolv mentioned this pull request May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants