Skip to content

[Fix] diffusion: report the port each scheduler actually bound to its clients - #42332

Open
Rockdu wants to merge 1 commit into
sgl-project:mainfrom
Rockdu:fix/sgld-scheduler-port-race
Open

Rockdu wants to merge 1 commit into
sgl-project:mainfrom
Rockdu:fix/sgld-scheduler-port-race

Conversation

@Rockdu

@Rockdu Rockdu commented Oct 3, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Diffusion servers starting together on one host can pick the same scheduler port: it is guessed as 5555 + randint(0, 100) and checked by a probe that releases it. The loser's scheduler then binds port + 42, but its HTTP side still connects to the guessed port, so its requests are served by the other server's scheduler and GPU.

We hit this as an intermittent RL CI failure, caught by the UUID check from #32685:

no payload was exported from this worker's GPU 47b7c7e8-..., got ['05a0fd7c-...']

Modifications

  • Each scheduler reports the port it actually bound in its ready message (dp_replica, actual_scheduler_port); the launcher records it in server_args.scheduler_ports before any client connects. A scheduler that moved off a taken port is now reached on its new port.
  • The scheduler also records the bound port in its own server_args, so the metrics replica label matches the real endpoint.
  • --strict-ports now fills scheduler_ports as well, so the list is always set.

Accuracy Tests

Not applicable.

Speed Tests and Profiling

Not applicable.

Checklist


CI States

Latest PR Test (Base): ❌ Run #37186702005
Latest PR Test (Extra): ❌ Run #37186701834
Latest PR Test (AMD ROCm 10): ❌ Run #37186701981

@github-actions github-actions Bot added the diffusion SGLang Diffusion label Oct 3, 2026
@Rockdu
Rockdu force-pushed the fix/sgld-scheduler-port-race branch from f2144d4 to d0dd3b2 Compare October 4, 2026 00:58
@Rockdu Rockdu changed the title [diffusion] fix: stop a scheduler whose port was taken from serving another server's GPU [diffusion] fix: let the OS assign the scheduler ingress port instead of guessing it Oct 4, 2026
@Rockdu
Rockdu force-pushed the fix/sgld-scheduler-port-race branch from d0dd3b2 to 1b88975 Compare October 4, 2026 02:07
@Rockdu Rockdu changed the title [diffusion] fix: let the OS assign the scheduler ingress port instead of guessing it [diffusion] fix: let the OS assign the scheduler port instead of guessing it Oct 4, 2026
@Rockdu
Rockdu force-pushed the fix/sgld-scheduler-port-race branch 2 times, most recently from acd3c4d to b25885f Compare October 4, 2026 02:59
@Rockdu
Rockdu marked this pull request as ready for review October 4, 2026 03:00
@Rockdu
Rockdu force-pushed the fix/sgld-scheduler-port-race branch 4 times, most recently from bd88b5b to 09b745c Compare October 4, 2026 07:41
@Rockdu Rockdu changed the title [diffusion] fix: let the OS assign the scheduler port instead of guessing it [diffusion] fix: report the port each scheduler actually bound to its clients Oct 4, 2026
… clients

Signed-off-by: rockdu <kangrdu@gmail.com>
@Rockdu
Rockdu force-pushed the fix/sgld-scheduler-port-race branch from 09b745c to 1853880 Compare October 4, 2026 07:44
Rockdu added a commit that referenced this pull request Oct 4, 2026
…ler actually bound to its clients (#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>
nv-hwoo pushed a commit to nv-hwoo/sglang that referenced this pull request Oct 20, 2026
…] fix: report the port each scheduler actually bound to its clients (sgl-project#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>
@AgainstEntropy AgainstEntropy changed the title [diffusion] fix: report the port each scheduler actually bound to its clients [Fix] diffusion: report the port each scheduler actually bound to its clients Oct 22, 2026
@AgainstEntropy

AgainstEntropy commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

seems #43732 fixed the same issue the opposite way: the scheduler binds with same_port=True, so it never moves, and the server that loses the race fails at startup.

However I believe in RL case (and most other cases), the losing server should still start, bind a free port and report it faithfully as this PR does.
(except for explicit --scheduler-port or --strict-ports, where same_port=True so port should never be moved)

Rockdu added a commit that referenced this pull request Oct 30, 2026
…rt the port each scheduler actually bound to its clients (#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>
Rockdu added a commit that referenced this pull request Oct 30, 2026
…rt the port each scheduler actually bound to its clients (#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>
Rockdu added a commit that referenced this pull request Oct 31, 2026
…rt the port each scheduler actually bound to its clients (#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>
Rockdu added a commit that referenced this pull request Oct 31, 2026
…rt the port each scheduler actually bound to its clients (#42470)

Signed-off-by: rockdu <kangrdu@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants