[wentian-byte]

Purpose

On transformers >= 5.0.0, Qwen/Qwen2-Audio-7B-Instruct (and any multimodal model whose top-level tie_word_embeddings defaults differ from its text_config's) produces garbage output: greedy decoding emits <|audio_bos|> / <|audio_eos|> repeatedly instead of text. This also makes tests/samplers/test_beam_search.py::test_beam_search_passes_multimodal_data fail on transformers 5.x.

The bug is not audio-specific — a plain text prompt to the same model (e.g. "What is 2+2?") also emits 20 consecutive <|audio_eos|> tokens. I verified audio_tower → multi_modal_projector produces byte-identical embeddings on 4.55.0 and 5.10.2, so the audio path is fine.

Root cause

VllmConfig.with_hf_config has a transformers-v5 branch that propagates the top-level tie_word_embeddings flag onto the language-model's text_config:

if model_config.is_multimodal_model and hasattr(
    model_config.hf_config, "tie_word_embeddings"
):
    tie_word_embeddings = model_config.hf_config.tie_word_embeddings
    hf_config.get_text_config().tie_word_embeddings = tie_word_embeddings

That logic is correct when text_config doesn't already know the right answer. For Qwen2-Audio it backfires: the checkpoint ships an independent lm_head.weight, HF logs "both are present in the checkpoints with different values, so we will NOT tie them", and text_config.tie_word_embeddings = False already reflects that. But the top-level Qwen2AudioConfig.tie_word_embeddings defaults to True, so the unconditional propagation flips text_config from False → True. vLLM's Qwen2 LM then takes that True and goes down the lm_head = self.model.embed_tokens + skip_prefixes=["lm_head."] path, aliasing lm_head.weight in memory to embed_tokens.weight and silently dropping the real lm_head.weight from the safetensors. The result is logits in input-embedding space, which repeatedly re-emits whatever token last appeared in the prompt (here, the audio special tokens).

Verified by reading data_ptr() on the live model with the same checkpoint:

	transformers 4.55.0	transformers 5.10.2 (before fix)	transformers 5.10.2 (after fix)
embed_tokens.data_ptr == lm_head.data_ptr	False (correct)	True (bug)	False (correct)
lm_head weight norm mean	0.547	0.647 (= embed)	0.547

Fix

Only propagate when text_config is silent or already agrees with the top-level value. If they actively disagree, trust text_config — that's the value vLLM's language-model code path reads when deciding how to build lm_head, and for models like Qwen2-Audio it's the one that matches the checkpoint.

         if model_config.is_multimodal_model and hasattr(
             model_config.hf_config, "tie_word_embeddings"
         ):
             tie_word_embeddings = model_config.hf_config.tie_word_embeddings
-            hf_config.get_text_config().tie_word_embeddings = tie_word_embeddings
+            text_cfg = hf_config.get_text_config()
+            existing = getattr(text_cfg, "tie_word_embeddings", None)
+            if existing is None or existing == tie_word_embeddings:
+                text_cfg.tie_word_embeddings = tie_word_embeddings

The original intent — fill text_config when it would otherwise be missing the flag — is preserved.

Test Plan

On transformers==5.10.2, 1 × NVIDIA H20:

pytest -xvs tests/samplers/test_beam_search.py::test_beam_search_passes_multimodal_data[2-64-half]

================== 1 passed, 18 warnings in 99.82s (0:01:39) ===================

vLLM output now matches the HF runner output for both beams; beam-0 transcribes the test audio correctly. Plain text prompt "What is 2+2?" also recovers (was 20×<|audio_eos|>, now "2+2 equals 4.").

Risk

• Models where top-level == text_config: unchanged.
• Models where top-level is set and text_config.tie_word_embeddings is None (the case the existing code comment describes): unchanged — the propagation still fills it in.
• Only changed behavior: when top-level != text_config.tie_word_embeddings and both are set — precisely the Qwen2-Audio case. Here we respect text_config, which is what the language-model code reads when building lm_head.

[wentian-byte]

Closing this in favor of a model-scoped fix.

with_hf_config's current propagation behavior is intentional (per #38035): the presence of tie_word_embeddings on text_config is not a reliable signal, so the unconditional override is the documented design. The right place to fix Qwen2-Audio is in the model itself, not in VllmConfig. Will open a small follow-up that only touches vllm/model_executor/models/qwen2_audio.py.