Skip to content

Conversation

@ngxson
Copy link
Collaborator

@ngxson ngxson commented Apr 28, 2025

Cont #13138

I misunderstood the original code:

        // TODO: add mrope pos ids somewhere else
        pos.resize(batch.n_tokens * 4);
        std::fill(pos.begin(), pos.end(), 0);
        for (int j = 0; j < batch.n_tokens * 3; j ++) {
            pos[j] = *st_pos_id + (j % batch.n_tokens);
        }
        batch.pos = pos.data();

Which means 3 first dims are the same, and 4th dim is 0, something like this:

1234...1234...1234...0000...

Thanks @mattjcly for spotting this!

@mattjcly
Copy link
Contributor

Can confirm this fixes bugs we observed! Thanks @ngxson :)

@ggerganov ggerganov merged commit b6ce743 into ggml-org:master Apr 29, 2025
48 checks passed
timwu pushed a commit to timwu/llama.cpp that referenced this pull request Dec 20, 2025
* llama-graph : fix text position for mrope

* fix typo

* explicitly set 4th dim in the loop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants