feat: support pythonic tool call and index in tool call streaming

[CatherineSue]

Motivation

Llama4 uses pythonic format for tool call. Same as small Llama3.2 models (1B and 3B). SGLang lacks pythonic format detector.
Also OpenAI has index field in the tool call chunk during streaming, see here, this PR adds the support.

Modifications

• Adds PythonicDetector to function_call_parser.py
• Adds improved llama4 pythonic chat_template to examples/chat_template
• Updates function_calling.ipynb to include pythonic tool call
• Updates the streaming logic to yield tool call chunks with index field

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[CatherineSue]

fixes #5661

[CatherineSue]

Test with Llama4:

python3 -m sglang.launch_server \
--model /models/meta-llama/Llama-4-Scout-17B-16E-Instruct \
--port=8080 \
--tp-size=8 \
--host=0.0.0.0 \
--mem-fraction-static=0.8 \
--context-length=800000 \
--chat-template=llama-4 \
--attention-backend=fa3 \
--tool-call-parser=pythonic \
--chat-template=examples/chat_template/tool_chat_template_llama4_pythonic.jinja 

Request:

import openai

tools = [
    {
            "type": "function",
            "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given location.",
            "parameters": {
                "type": "object",
                "properties": {
                "location": {
                    "type": "string",
                    "description": "The name of the city or location."
                }
                },
                "required": ["location"]
            }
            }
        },
        {
            "type": "function",
            "function": {
            "name": "get_tourist_attractions",
            "description": "Get a list of top tourist attractions for a given city.",
            "parameters": {
                "type": "object",
                "properties": {
                "city": {
                    "type": "string",
                    "description": "The name of the city to find attractions for."
                }
                },
                "required": ["city"]
            }
            }
        }
]

def get_messages():
    return [
        {
            "content": "You are a travel assistant providing real-time weather updates and top tourist attractions. ",
            "role": "system"
        },
        {
            "content": "I'm planning a trip to Tokyo next week. What's the weather like and what are some top tourist attractions? Propose parallel tool calls at once.",
            "role": "user"
        }
    ]


messages = get_messages()

client = openai.Client(
    base_url="http://localhost:8080/v1", 
    api_key="xxxxxx"
)
model_name = client.models.list().data[0].id


response_non_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.8,
    top_p=0.8,
    stream=False,  # Non-streaming
    tools=tools,
)
print("Non-stream response:")
print(response_non_stream)

name_non_stream = response_non_stream.choices[0].message.tool_calls[0].function.name
arguments_non_stream = (
    response_non_stream.choices[0].message.tool_calls[0].function.arguments
)

print(f"Final streamed function call name: {name_non_stream}")
print(f"Final streamed function call arguments: {arguments_non_stream}")


response_stream = client.chat.completions.create(
    model=model_name,
    messages=messages,
    temperature=0.8,
    top_p=0.8,
    stream=True,
    tools=tools,
)
texts = ""
tool_calls = []
name = ""
arguments = ""

for chunk in response_stream:
    if chunk.choices[0].delta.content:
        texts += chunk.choices[0].delta.content
    if chunk.choices[0].delta.tool_calls:
        tool_calls.append(chunk.choices[0].delta.tool_calls[0])
        
print("Streaming Response:")
print("==== Text ====")
print(texts)

print("==== Tool Call ====")
for tool_call in tool_calls:
    print(tool_call)

Result:

UT:

[merrymercy]

@zhaochenyang20 @shuaills can you take a review or ping related people?

[hebiao064]

could you please also try --enable-torch-compile to see whether it works or not

https://github.com/sgl-project/sglang/actions/runs/14676885111/job/41194397615?pr=5509
We noticed that it will fail the CI for the test PR

[zhaochenyang20]

@CatherineSue fix lint plz， and rebase

[CatherineSue]

Fixed lint error in notebook and updated notebook output.
Rebased.

[shuaills]

Thanks for the PR. The code LGTM. For docs please check here.

[merrymercy]

Can we merge this now?

[shuaills]

Can we merge this now?

The failed tests are not related to this PR. However, the docs need to have outputs cleared and should be streamlined. Once these changes are made, it will be ready.

[CatherineSue]

@shuaills I have cleared the outputs in the notebook. Can you be more specific regarding streamlined?

[zhaochenyang20]

@shuaills I have cleared the outputs in the notebook. Can you be more specific regarding streamlined?

Let me ask him and yitong

[CatherineSue]

Test time

➜  srt git:(function-call) python3 -m unittest test_function_calling                                
command=python3 -m sglang.launch_server --model-path meta-llama/Llama-3.2-1B-Instruct --tool-call-parser pythonic --host 127.0.0.1 --port 8000 --api-key sk-123456
(##### Skipped long server outputs #####)

----------------------------------------------------------------------
Ran 6 tests in 55.203s

OK