You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Randomly getting output full of repetitive newlines on LLaVA invocations #191
Not sure if anyone else had hit into this but when using liuhaotian/llava-v1.5-13b with llava-hf/llava-1.5-13b-hf tokenizer, randomly I get outputs full of only newlines. The frequency of this happening increases when the temperature is close to 0 but also happens at 1 as well.
I've also hit this bug. My issue was that I had the generation outside of the assistant prompt. I was trying to be able to prompt the generation process (e.g. I give it what the model should start with).
WRONG:
def image_qa(s, image_path, question, system_prompt="", initial_assistant = ""):
s += sgl.user(system_prompt + sgl.image(image_path) + question)
s += sgl.assistant(initial_assistant)
s += sgl.gen("answer")
Activity
jacob-hansen commentedon Apr 24, 2024
I've also hit this bug. My issue was that I had the generation outside of the assistant prompt. I was trying to be able to prompt the generation process (e.g. I give it what the model should start with).
WRONG:
def image_qa(s, image_path, question, system_prompt="", initial_assistant = ""):
s += sgl.user(system_prompt + sgl.image(image_path) + question)
s += sgl.assistant(initial_assistant)
s += sgl.gen("answer")
CORRECT:
def image_qa(s, image_path, question, system_prompt="", initial_assistant = ""):
s += sgl.user(system_prompt + sgl.image(image_path) + question)
s += sgl.assistant(initial_assistant + sgl.gen("answer"))
This fixed the issues.
github-actions commentedon Jul 25, 2024
This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.
Copy OSS code from 8a8a608 on 20251007 (sgl-project#191)
[qwen-vl] add allreduce_norm(_quant) fusion (sgl-project#191)
fix: qwen3.5 precision & quant model load error (sgl-project#191)
Merge PoC_20260331 to sgl-project/sglang/main (sgl-project#282)
[XPU] Fix cu_seqlens_k_new mis-passing in flash_attn_with_kvcache calls
sgl-project#187 T187 audit closure: F5-PUT-on-migrate-error bug + 2 t…
sgl-project#191 — e2e SESSION_END migrate flavor (integration_stress G)