Skip to content

server: use pos_next instead of n_tokens for m-rope#22439

Merged
ggerganov merged 1 commit into
ggml-org:masterfrom
am17an:server-spec-pos-rollback
Apr 28, 2026
Merged

server: use pos_next instead of n_tokens for m-rope#22439
ggerganov merged 1 commit into
ggml-org:masterfrom
am17an:server-spec-pos-rollback

Conversation

@am17an

@am17an am17an commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

Overview

Repro: ./build/bin/llama-cli -hf ggml-org/Qwen2.5-VL-7B-Instruct-GGUF:Q8_0 -hfd Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q8_0 -p "what is in this image" --image photo.jpeg, fails in init_batch

init: the tokens of sequence 0 in the input batch have inconsistent sequence positions:                                                                                               
   - the last position stored in the memory module of the context (i.e. the KV cache) for sequence 0 is X = 85                                                                          
   - the tokens for sequence 0 in the input batch have a starting position of Y = 85                                                                                                    
   for M-RoPE, it is required that the position satisfies: X < Y    

seq_rm expects a position and with M-RoPE n_tokens != n_pos so it this bug triggers.

Additional information

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, AI identified this bug, I repro'ed it and fixed it

@am17an
am17an requested a review from a team as a code owner April 27, 2026 16:26

@ngxson ngxson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't test it but seems to be a valid fix, CC @ggerganov if you can have a quick look

@ggerganov ggerganov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch.

@ggerganov
ggerganov merged commit 516e8d7 into ggml-org:master Apr 28, 2026
45 of 46 checks passed
@am17an
am17an deleted the server-spec-pos-rollback branch April 28, 2026 05:42
IntelNav pushed a commit to IntelNav/llama.cpp that referenced this pull request Apr 29, 2026
IntelNav pushed a commit to IntelNav/llama.cpp that referenced this pull request Apr 29, 2026
cnsiva pushed a commit to saas-home/llama.cpp that referenced this pull request Apr 29, 2026
samuraieng pushed a commit to samuraieng/llama.cpp that referenced this pull request May 6, 2026
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
meh pushed a commit to meh/llama.cpp that referenced this pull request May 10, 2026
baramofme pushed a commit to baramofme/llama-cpp-turboquant that referenced this pull request May 23, 2026
winstonma pushed a commit to winstonma/llama.cpp that referenced this pull request May 27, 2026
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request May 30, 2026
MrLordCat referenced this pull request in MrLordCat/llama.cpp-with-GUI Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants