Controlling Thinking Budget for Reasoning Models

[joeyy5588]

Hi,

Does sglang support thinking budget like the one demonstrated in the Qwen3's demo?
https://huggingface.co/spaces/Qwen/Qwen3-Demo

[bunbohue19]

You can try this:

def chat_function_(s, prompt, thinking_budget=16, max_tokens=512, temperature=0.7):
    # First stage: generate thinking content
    s += prompt
    s += sgl.gen("thinking", max_tokens=thinking_budget, stop=["</think>"])
    
    # Check if thinking ended naturally or budget exceeded
    thinking_text = s["thinking"]
    if "</think>" not in thinking_text:
        # Force close thinking
        s += "\n\nConsidering the limited time, I'll provide the solution now.\n</think>\n\n"
    
    # Second stage: generate actual response
    s += sgl.gen("translation", max_tokens=max_tokens, temperature=temperature, stop=["<|im_end|>"])