Feature Request: Add support for Qwen3 Next

[chulucninh09]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Hi, can we add support for Qwen3-Next? Recently, Qwen team also released Qwen3-Next-Coder, which I tested out with llama.cpp upstream and it is really good.

Motivation

The main motivation is to try out new model and technique implemented by Qwen team, the model is very sparse and good at long context, which fit on code usecases

Possible Implementation

No response

[ikawrakow]

Qwen3-Next is a hybrid model and will require a significant investment to get the missing pieces implemented in ik_llama.cpp.

So, if you are interested in having Qwen3-Next support in ik_llama.cpp, please upvote the issue.

[whatever1983]

@ikawrakow:

As much as I hate the chinese LLM release schedules to specifically target shorting of US markets, they are all the same(Kimi, DS, Qwen, minimax, GLM) This one is actually worthwhile implementing in IK_llama for the swe-bench pro=44. It is still way behind GPT-5.2 Codex @ swe-bench pro 56+. The best way to fight back is actually make the A3B little shitter run cheaply on laptops. They are so greedy, trying to charge $1.50 for A3B on openrouter, it is amazingly stupid.

[kiron111]

@ikawrakow:

As much as I hate the chinese LLM release schedules to specifically target shorting of US markets, they are all the same(Kimi, DS, Qwen, minimax, GLM) This one is actually worthwhile implementing in IK_llama for the swe-bench pro=44. It is still way behind GPT-5.2 Codex @ swe-bench pro 56+. The best way to fight back is actually make the A3B little shitter run cheaply on laptops. They are so greedy, trying to charge $1.50 for A3B on openrouter, it is amazingly stupid.

Anyone will use their money to build a LLM to target at shorting other company. (and now computational Power as FLOPS is short in China, compared to US)
If you said Deekseek, I will agree, this company is a quantum fund before. (at least DS opensourced many useful algorithms for LLM, though I seldom use them or paid to them, I appreciate them)

The government may want to short US AI market, but how they sure the market (Only US are caple of controlling war, and shorting market, it's on news)
for a captial company, as Alibaba, all they aim at, is profit, maket share and their own stock price.
Kimi, and minimax, zhipu is a small company, tow of them are just start as a IPO in Hong Kong, drawing capital in stock market, how the have money to short other?

Thoung I doubt your assumption, I would agree with yout at supporting implement Qwen3-Next for ik_llama.

[chulucninh09]

My major interest in Qwen3-Next is the commitment of the team when releasing Coder model after. I believe future models of Qwen team will follow this architecture or some deviation of the architecture too. That's why making this model work in ik_llama seems to be good investment for future releases.

[YurkoHoshko]

Just for fun, I made latest Codex 5.3 bring Qwen 3 next architecture into ik_llama and it actually worked - its on my fork here https://github.com/YurkoHoshko/ik_llama.cpp/commits/main/. I ran the model with OpenCode / Pi agent and it seems to be quite good. I used mxfp4 from Unsloth.

It did have to pretty much hack its way through, so I would advise caution as other models may not work.
On my machine (8945HS, RTX 3060 TI via thunderbolt 3 + RTX 5060ti via oculink + 96gb ddr5 at 65k context) it runs at up to 200 prompt processing and 20-25 tps on generation.

The fact that model pretty much one-shotted it is wild.

[ikawrakow]

The fact that model pretty much one-shotted it is wild.

Haha, yes, soon there will be no need for me to spend time on this project, and all will be done by LLM agents going around GitHub and tirelessly improving all open source projects there!

More seriously, though, if you like you can submit a PR. I need to see it all at once (instead of going over multiple commits in your repository) to see if it will work without breaking other stuff.

In terms of performance how does it compare to the llama.cpp implementation? Based on what I have seen, the major bottleneck in Qwen3-Next are the recurrent parts, so my guess is that performance will be very similar without furter optimizations. Still curious to see a more detailed performance comparison.

[saood06]

I need to see it all at once (instead of going over multiple commits in your repository) to see if it will work without breaking other stuff.

You can view that without a PR: main...YurkoHoshko:ik_llama.cpp:main

In terms of performance how does it compare to the llama.cpp implementation?

I thought the answer to that would be in this Document titled "Qwen3Next Performance-Differences Report (ik_llama.cpp vs llama.cpp)" but I can't seem to find the answer in it. It does seem like the AI thinks it's implementation is still slower from the "Why Current ik Can Still Be Slower" section, but I can't find an actual comparison.

[YurkoHoshko]

Haha, yes, soon there will be no need for me to spend time on this project, and all will be done by LLM agents going around GitHub and tirelessly improving all open source projects there!

@ikawrakow I really respect what you do and hope that you will continue your great work - if anything - I hope LLMs take the burden of "boring" code from you and let you focus on big gains :)

In terms of performance how does it compare to the llama.cpp implementation? Based on what I have seen, the major bottleneck in Qwen3-Next are the recurrent parts, so my guess is that performance will be very similar without furter optimizations. Still curious to see a more detailed performance comparison.

I attached some benchmarks on PR - seems to be roughly at parity with mainline, maybe slightly above - because the implementation really feels like mainline just hacked into this codebase. Lots of things hardcoded too.
There were some development in mainline in the past few days around Kimi Linear and some similar code consolidation - left references to those in PR as well - will see if I can incorporate them and what kind of improvement they bring.

[YurkoHoshko]

It seems like next series of Qwen models (3.5) will be using the same or similar architecture: huggingface/transformers#43830

[ikawrakow]

Closed via #1266, #1275, #1276, #1277, #1278

Didn't see the same level of excitement in the PRs adding/optimizing Qwen3-Next as I see here.

[dinerburger]

I moved over to ik_lcpp this morning for working with Qwen3-Coder-Next and doubled my throughput. While the excitement may be muted, this is a huge win, and thanks again for everyone's work on this project!!

[chulucninh09]

Just came back from my long vacation, this is very good news for us! Gonna try out the latest commit with Qwen3 next. Thank very much on the implementation