Skip to content

Server: when no slot is available, defer the task instead of returning "slot unavailable"#5018

Merged
ggerganov merged 2 commits intoggml-org:masterfrom
ngxson:xsn/defer-server-task
Jan 18, 2024
Merged

Server: when no slot is available, defer the task instead of returning "slot unavailable"#5018
ggerganov merged 2 commits intoggml-org:masterfrom
ngxson:xsn/defer-server-task

Commits

Commits on Jan 18, 2024