Skip to content

Start Ray with enough CPUs for two-GPU Miles v0.2.0 trainers - #345

Merged
Micah Yong (micahtyong) merged 1 commit into
mainfrom
micah/miles-ray-cpus
Nov 17, 2026
Merged

Micah Yong (micahtyong) merged 1 commit into
mainfrom
micah/miles-ray-cpus

Conversation

@micahtyong

@micahtyong Micah Yong (micahtyong) commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A Miles LoRA trainer on the miles-v0.2.0-15afe21 image with exactly two GPUs never finished starting. The client waited on "starting a trainer" indefinitely, and the GPUs never allocated memory.

Spindle starts Ray with max(4, world_size * 2) CPUs. The v0.2.0 runtime reserves one CPU per GPU for its training workers and runs three more single-CPU actors outside that reservation. With two GPUs that needs five CPUs, but Ray had four, so the last actor never scheduled and start-up waited on it forever. One GPU (4 of 4) and four GPUs (7 of 8) happened to fit.

Ray now starts with world_size * 2 + 2 CPUs. This is never fewer than before, so other layouts are unaffected. Multi-node trainers join an existing Ray cluster and don't use this path.

Testing

Stock qwen35-4b-lora-32k with trainer_image = "miles-v0.2.0-15afe21" and two H200 trainer GPUs, using the stock Tinker SDK: create one rank-32 LoRA training client, then run one forward_backward.

Client created forward_backward done
main 48820a5 still waiting after 10 min –
this PR (95eea49) 192 s 228 s

On main, ray status inside the trainer showed 4 total CPUs with a pending demand for one more CPU, and the backend was waiting in RayWorkerManager.launch.

uv run --no-sync pytest -q tests/backends: 336 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@micahtyong
Micah Yong (micahtyong) merged commit 45befec into main Nov 17, 2026
3 checks passed
@micahtyong
Micah Yong (micahtyong) deleted the micah/miles-ray-cpus branch November 17, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant