Eval bug: the model get's stuck in a loop

[AbdulrahmanHashem]

Bad Version

version: 9330 (328874d)
i tried building with CUDA 13.1.1 and 13.2.1

cmake -B build -G Ninja \
-DGGML_CUDA=ON \
-DCMAKE_CUDA_ARCHITECTURES="75;120" \
-DGGML_CUDA_FA_ALL_QUANTS=ON \
-DGGML_CUDA_F16=ON \
-DGGML_NATIVE=ON \
-DGGML_AVX2=ON \
-DGGML_FMA=ON \
-DGGML_F16C=ON \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DCMAKE_BUILD_TYPE=Release \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_EXAMPLES=ON \
-DLLAMA_BUILD_SERVER=ON

cmake --build build --config Release --target llama-server

Operating systems

Linux

GGML backends

CUDA

Environment & Hardware

• OS: Linux (Arch-based / CachyOS)
• CPU: AMD Ryzen 5 3600
• GPU: NVIDIA Heterogeneous Dual-GPU Setup (RTX 5060 Ti 16GB + RTX 2060 Super 8GB)

Models

Qwen3.6-35B-A3B-UD-Q4_K_M_MTP.gguf
Qwen3.6-35B-A3B-UD-Q4_K_S_MTP.gguf

Problem description & steps to reproduce

Description

When running a model using llama.cpp, the model gets stuck in an infinite repetition loop very early in the token generation process. This loop occurs explicitly inside the "thinking" phase of the model (before or just after the thinking tokens open), causing it to generate the same bad/repeated answers indefinitely.

I have tested 2 different quantizations of the model, and the exact same repeating behavior occurs.

my latest good build before i observed this behavior is
version: 9306 (5d246a7)
built with GNU 16.1.1 for Linux x86_64

Edit: i tested up to
version: 9320 (ae251b5)
built with GNU 16.1.1 for Linux x86_64
and it works so far.

Reproduction Steps

1. Load a reasoning model 'Qwen3.6-35B-A3B-UD-Q4_K_M_MTP.gguf' or 'Qwen3.6-35B-A3B-UD-Q4_K_S_MTP.gguf'

--verbosity 4 \
-m /home/abdulrahman/Personal/Programs/llama/models/Qwen/Qwen3.6-35B-A3B-UD-Q4_K_M_MTP.gguf \
--chat_template_kwargs '{"preserve_thinking": "True"}' \
--jinja \
--main-gpu 0 \
--temperature 0.6 --top-p 0.95 --top-k 20 --min-p 0.00 --presence-penalty 0.0 --repeat-penalty 1.0 \
-t 10 -tb 10 \
--host 0.0.0.0 --port 8080 \
--mlock \
--cache-type-k q8_0 --cache-type-v q4_0 \
--cache-type-k-draft q8_0 --cache-type-v-draft q4_0 \
-c 0 -fitc 245760 \
--spec-type draft-mtp,ngram-mod --spec-draft-n-max 2 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 16 --spec-ngram-mod-n-max 64 \
-ngl 999 --n-cpu-moe 14 --tensor-split 85,26

3. prompt the model until you reach an somewhere around 40K tokens
4. Observe that token generation breaks into a loop almost immediately inside the thinking phase.

Failure Sample / Log Output

Here's a thinking process:

1.  **Analyze the User's Problem:** The user keeps saying the player is "flying" instead of walking/standing on the ground. This is a core issue in the collision/level system. But the main problem is that the player character isn't properly constrained to move within the level bounds.

2.  **Identify the Core Issue:** The game's physics/collision system is fundamentally broken. The player isn't properly colliding with the ground, and the movement logic isn't working correctly. The player is floating in mid-air instead of being grounded.

3.  **Root Cause:** The player's position is being set, but the game loop isn't properly handling the physics. The player is supposed to be grounded, but the collision system is broken.

Let me think about this more carefully. I need to fix the player's movement. But wait, I need to fix the player's movement. I need to fix the player's movement. I need to fix the player's movement. I need to fix the player's movement. I need to fix the player's movement.

Wait, I'm getting ahead of myself. Let's think about the game loop. The game loop is supposed to be a game loop. I need to fix the player's movement. I need to fix the player's movement. I need to fix the player's movement. I need to fix the player's movement.

// Game loop:
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]
// [x, y]

Here's a thinking process:

1.  **Analyze User Request:**
   - User wants a "super mario game in one html file"
   - Previous attempts had issues: flying character, jumping problems, holes causing death, etc.
   - Need to create a complete, self-contained HTML file with CSS and JavaScript
   - Should include: Mario character, platforms/ground, enemies (goombas/koopas), coins, question blocks, bricks, pipes, flag pole, castle, clouds, hills, bushes
   - Controls: Arrow keys/WASD to move, Space/Up/W to jump
   - Need to fix previous bugs mentioned: flying, jumping, holes causing death, etc. (but I'll just implement a working version)
   - The user wants a full, polished Super Mario game experience in a single HTML file.

2. **Identify Key Requirements:**
   - Single HTML file with embedded CSS and JavaScript
   - Canvas-based rendering for the game
   - Responsive design to fit different screen sizes
   - Game states: start screen, playing, game over, win screen
   - UI elements: score, coins, lives, world indicator
   - Input handling: keyboard events for movement and jumping
   - Game loop: update and draw functions running at 60 FPS for the game
   - Collision detection: proper AABB collision for all game objects
   - Physics: gravity, acceleration, friction, max speed
   - Animation: frame-based animation for Mario, enemies, coins, particles
   - UI: score display, coin counter, world indicator, lives counter
   - Screens: start screen, game over screen, win screen
   - Blinking effect for start screen prompt
   - HTML structure: proper semantic elements
   - CSS: responsive, clean, no overflow issues
   - JavaScript: all game logic, physics, input, animation, collision, particles, floating texts, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes, flag pole, castle, bricks, question blocks, enemies, coins, pipes,

[humbertooliveira]

I got the same error when I recompiled after pulling from master today, it's happening with the qwen 3.6 27b mtp as well.

I rolled back to the b9305 release and it started working again.

[am17an]

I think should be fixed by #23690, please confirm

[AbdulrahmanHashem]

yes #23690 fixed it, thanks ^_^
now the only major problems are

• The runtime allocation in ngram and MTP.
• KV cache memory leak.