[Bug]: gemma 3 4b output

[Alireza3242]

Your current environment

docker vllm v0.9.2

🐛 Describe the bug

In v0.9.2, outputs of gemma 3 4b is wrong. But v0.9.1 work true.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[RocketRider]

I see the same issue with the Gemma3-27b-it.

[DarkLight1337]

Try setting VLLM_USE_V1=0 and see if the situation improves

[marseller]

Same issue here, it works on v0.9.1 but same garbage output on v0.9.2

[DarkLight1337]

Can you try downgrading to transformers 4.52?

[hmellor]

If you are using Transformers v4.53, please upgrade to version v4.53.2, this patch contains a fix for Gemma 3 models (huggingface/transformers#39301)

[gohar94]

I can confirm that vLLM 0.9.2 and transformers 4.53.2 works fine for me.
Below is how I am running it:

Dockerfile

FROM vllm/vllm-openai:v0.9.2
RUN uv pip install --system transformers==4.53.2

Build:

DOCKER_BUILDKIT=1 docker build . \
    --tag vllm/vllm-openai-fix \
    --file Dockerfile

Launch:

  export HF_TOKEN=""

  sudo docker run --gpus '"device=4,5,6,7"' \
                  --cap-add SYS_ADMIN \
                  -v $HF_HOME:/root/.cache/huggingface \
                  --env "HUGGING_FACE_HUB_TOKEN=$HF_TOKEN" \
                  -p 8000:8000 \
                  --ipc=host \
                  vllm/vllm-openai-fix \
                  --model google/gemma-3-27b-it \
                  --tensor-parallel-size 4 \
                  --enable-auto-tool-choice \
                  --tool-call-parser "pythonic" \
                  --trust-remote-code \
                  --enable-prefix-caching \
                  --guided-decoding-backend xgrammar \
                  --max-num-seqs 1024 \
                  --limit-mm-per-prompt "image=50"

[hmellor]

Good stuff, I'll go ahead and close this then!

The next vLLM release will use the latest version of Transformers