Environment

• mistral.rs v0.8.23, Gemma-4-26B-A4B, Q6K UQFF, PagedAttention on
• DGX Spark (GB10, CUDA 13.0), Python bindings in-process
• The crash is a shape/logic error, so it should be hardware-independent
• Note: this is with the scheduler livelock fixed locally (see companion issue) — without that fix, batched multimodal never gets far enough to hit this

Part A: deterministic crash with identical requests

Submitting two identical vision requests (same image, same prompt, same length -> same scheduler bucket -> both image prefills land in one forward) fails both requests:

ERROR mistralrs_core::engine: step - Model failed with error: unexpected rank, expected: 1, got: 2 ([2, 1080])
ValueError: unexpected rank, expected: 1, got: 2 ([2, 1080])

Something in the vision-embedding path assumes a rank-1 (single-sequence) tensor and receives the batched [batch=2, n] version.

Minimal repro (Python bindings; any image, we used a rendered A1 drawing page):

import base64, threading
from pathlib import Path
from mistralrs import Runner, Which, MultimodalArchitecture, ChatCompletionRequest

b64 = base64.b64encode(Path("page.png").read_bytes()).decode()
messages = [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    {"type": "text", "text": "Describe this drawing. Transcribe all text and numbers."},
]}]

runner = Runner(which=Which.MultimodalPlain(model_id="<gemma4-26b-dir>", arch=MultimodalArchitecture.Gemma4))
def go():
    print(runner.send_chat_completion_request(ChatCompletionRequest(
        model="x", messages=messages, max_tokens=512, temperature=0.0)))
ts = [threading.Thread(target=go) for _ in range(2)]
[t.start() for t in ts]; [t.join() for t in ts]

Part B: silent corruption with mixed-length batches

Real-world concurrent requests have different prompt lengths, land in different length buckets, and get their image prefills serialized — so they dodge the Part A crash. But the outputs still degrade with batch width (greedy decoding, temp 0, same 5-page document each time):

No error is raised in Part B — requests "succeed" with corrupted content, which is worse than the crash.

Context

RFC #2171 deferred PagedAttention interaction / KV heterogeneity / multimodal batching for Gemma 4, so some of this may be known territory. Filing because the failure modes are now cleanly separable: (companion issues) a scheduler livelock and a hot-path perf bug, plus this correctness bug, which is the remaining blocker for batched multimodal on this model. Happy to run diagnostics/patches on our hardware — we have a fast repro loop.