Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ overrides:
args:
- --mem-fraction-static
- "0.75"
- --prefill-round-robin-balance
- --watchdog-timeout
- "3600"
prefill:
Expand Down
2 changes: 0 additions & 2 deletions examples/backends/sglang/launch/disagg.sh
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,6 @@ print_launch_banner "Launching Disaggregated Serving (2 GPUs)" "$MODEL" "$HTTP_P
OTEL_SERVICE_NAME=dynamo-frontend \
python3 -m dynamo.frontend &

#AssertionError: Prefill round robin balance is required when dp size > 1. Please make sure that the prefill instance is launched with `--load-balance-method round_robin` and `--prefill-round-robin-balance` is set for decode server.

# run prefill worker
# NOTE: Each worker picks a random NCCL port (get_free_port) for torch.distributed.
# This has a TOCTOU race — the port can be grabbed before init_process_group binds it,
Expand Down
1 change: 0 additions & 1 deletion recipes/deepseek-r1/sglang/disagg-16gpu/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,6 @@ spec:
- "0.75"
- --host
- 0.0.0.0
- --prefill-round-robin-balance
- --watchdog-timeout
- "3600"
prefill:
Expand Down
1 change: 0 additions & 1 deletion recipes/deepseek-r1/sglang/disagg-8gpu/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@ spec:
- "0.75"
- --host
- 0.0.0.0
- --prefill-round-robin-balance
- --watchdog-timeout
- "3600"
prefill:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,6 @@ spec:
- --dyn-tool-call-parser
- deepseek_v41
- --dyn-enable-structural-tag
- --prefill-round-robin-balance
command:
- python3
- -m
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,6 @@ spec:
- --dyn-tool-call-parser
- deepseek_v41
- --dyn-enable-structural-tag
- --prefill-round-robin-balance
command:
- python3
- -m
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,6 @@ spec:
- --dyn-tool-call-parser
- deepseek_v41
- --dyn-enable-structural-tag
- --prefill-round-robin-balance
env: *workerEnv
ports: *workerPorts
resources: *workerResources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,6 @@ spec:
--disaggregation-mode decode
--disaggregation-transfer-backend nixl
--disaggregation-bootstrap-port 30001
--prefill-round-robin-balance
--watchdog-timeout 3600
env:
- name: POD_UID
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,6 @@ spec:
--disaggregation-mode decode
--disaggregation-transfer-backend nixl
--disaggregation-bootstrap-port 30001
--prefill-round-robin-balance
--watchdog-timeout 3600
env:
- name: POD_UID
Expand Down
1 change: 0 additions & 1 deletion recipes/glm-5-nvfp4/sglang/disagg/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,6 @@ spec:
--disaggregation-mode decode
--disaggregation-transfer-backend nixl
--disaggregation-bootstrap-port 30001
--prefill-round-robin-balance
--watchdog-timeout 3600
env:
- name: POD_UID
Expand Down
1 change: 0 additions & 1 deletion recipes/glm-5-nvfp4/sglang/disagg/efa/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,6 @@ spec:
--disaggregation-mode decode
--disaggregation-transfer-backend nixl
--disaggregation-bootstrap-port 30001
--prefill-round-robin-balance
--watchdog-timeout 3600
env:
- name: POD_UID
Expand Down
Loading