Port Deepseek v4 by SamuelOliveirads · Pull Request #2110 · ikawrakow/ik_llama.cpp

SamuelOliveirads · 2026-07-11T16:28:20Z

This implementation is based on the mainline PR and uses DeepSeek-V4-Flash-MXFP4 for validation.

The main reason I created a draft is because the output diverges from the mainline when FA is enabled. For testing purposes, I disabled the IQK FA using the GGML_FLASH_ATTN_EXT_IQK_DISABLED flag, which aligns the model’s output with the mainline, however, the same issue occurs when using CUDA MMA FA path, which produces finite but incorrect logits for the DSV4 512/512/512, GQA 64:1 case, and here it’s a bit more complicated to disable the current implementation for testing. Since I don’t know exactly what to adjust in the current FA implementation in the engine, @ikawrakow I’d like to know the best course of action. I can disable FA for now so it can be fixed in the future, in that case I would revert the main changes made to ggml, concat.cu and GGML_FLASH_ATTN_EXT_IQK_DISABLED path.

As for performance, it’s a work in progress, partly because I can’t achieve a complete offload to the GPU, so I haven’t run a full benchmark yet. Using sweep-bench with a humble -c 256 -n 32 I’ve observed the following performance:

CPU non-FA: 3.83 tok/s.
CPU generic FA: 4.00 tok/s.
CUDA hybrid non-FA: 2.59 tok/s
CUDA FA: 3.20 tok/s

There are other limitations I haven’t addressed yet in this PR, such as context shift, quantized KV, -sm graph, and low GPU performance, for example the routing of l_out but I believe they may come up later.

…oftplus gating function

ikawrakow · 2026-07-12T04:55:39Z

I'll look into it.

But as a general observation, I wasn't expecting a DS4 PR will add over 3k LOC. We now have the main ingredients (indexer via the GLM-DSA PRs, HC and SWA via the openPangu PR) on the main branch, so please reuse as much of that as possible (you will probably need to extract the components from build_deepseek2.cpp and build_openpangu.cpp and move a level up so they can be reused by DS4).

ikawrakow · 2026-07-12T11:03:36Z

Your branch is quite a bit behind the main branch. Please rebase, fix conflicts, and check what has happened on the main branch. Do not reinvent the wheel with sparse attention, HC, sinkhorn, etc.

I have one of the antirez ds4 GGUFs. Mainline works with that, this PR crashes. Not sure if I want to download yet another 160 GB to test the PR and see what can be done about the issues you are seeing. Can you make it work with antirez GGUFs?

OK, one issue is that the GGUF I have has the MTP layer included, and this is not taken into account when building the graph. This leads to a crash in build_deepseek4. After fixing that, I run into an illegal memory access in k_bin_bcast<op_add>. I ran with -no-fa due to the issues you mentioned.

Better luck running CPU-only. Better in the sense of no crash (after fixing the MTP issue). But running llama-perplexity I get

perplexity: calculating perplexity over 140 chunks, n_ctx=2048, batch_size=2048, n_seq=1
perplexity: 24.78 seconds per pass - ETA 57.80 minutes
[1]25.9507,[2]17.3224,[3]21.6921,[4]47.4972, etc.

PPL values are about the same with or without -no-fa (but the no-fa run is quite a bit slower with 32.8 seconds per pass).

In contrast, here is what I get with mainline

.39.346.600 I system_info: n_threads = 64 (n_threads_batch = 64) / 128 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.39.346.616 I perplexity: tokenizing the input ..
0.39.981.243 I perplexity: tokenization took 634.62 ms
0.39.981.412 I perplexity: calculating perplexity over 140 chunks, n_ctx=2048, batch_size=2048, n_seq=1
1.18.973.364 I perplexity: 38.99 seconds per pass - ETA 1 hours 30.97 minutes
[1]1.7743,[2]1.6264,[3]1.9530,[4]2.3941, etc.

Oh, initially I tried to run perplexity for a context of 512 but using u_batch = 2048 (so 4 sequences). I see in the code that there is all this stuff with n_stream which is supposed to provide support for multiple sequences. But with that, it completely disintegrates:

perplexity: calculating perplexity over 561 chunks, n_ctx=512, batch_size=2048, n_seq=4
perplexity: 21.77 seconds per pass - ETA 50.87 minutes
[1]79.8919,[2]5778.6920,[3]11001.4778,[4]16637.9307, etc.

So, it looks like it simply does not work yet, and is not related to issues with IQK flash attention.

…xt management

# Conflicts: # convert_hf_to_gguf.py # ggml/src/ggml.c # src/CMakeLists.txt # src/llama-build-context.cpp # src/llama-build-context.h # src/llama-hparams.cpp # src/llama-load-tensors.cpp

… post-processing with new helper functions

SamuelOliveirads · 2026-07-12T23:34:52Z

The problem with the Antirez GGUF arose mainly because I make use of some of the GLM_DSA’s logic and as a result relied on the generic management of nextn_predict_layers which is not present in the DPV4 GGUF models. I downloaded the IQ2XXS and obtained the following results:

Antirez IQ2XXS CPU non-FA:
- fused MoE: 3.1934, 2.9367
- unfused MoE: 3.2104, 2.9414
- mainline: 3.1485, 2.9126
MXFP4 CPU non-FA: 2.9514, 2.8147

With n_seq=4, the results are:

branch: 3.1678, 2.9091
mainline: 3.1697, 2.9091

As for CUDA non-FA:

CUDA non-FA: 3.1310

Note that for perplexity I used prompts/mnemonics.txt and compared the results here with those in the mainline.

Regarding the rebase, I was going to wait until the logic was working, but now that you sugested I did it now to reuse what had already been done for HC, SWA, Indexer, and Sinkhorn, hopefully I was able to reuse everything that made sense.

Back to FA for CPU, the perplexity is similar but the token selection differs from the mainline. This is most striking in CUDA where attn_csa-2 produces 5.289 NaNs and the output can become more incoherent quickly.

ikawrakow · 2026-07-13T05:57:00Z

On the latest PR version, CPU-only

./bin/llama-perplexity -m DeepSeek-V4-Flash-Q4KExperts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-imatrix.gguf \
    -f wiki.test.raw -t 64 -ub 2048 -no-fa -c 2048
perplexity: calculating perplexity over 140 chunks, n_ctx=2048, batch_size=2048, n_seq=1
Segmentation fault

This after fixing the build error due to the duplicate instantiation of llama_model_loader::get_arr<uint32_t> in llama-model-loader.cpp, and fixing the still missing handling of hparams.nextn_predict_layers in build_deepseek4()

-    for (int il = 0; il < n_layer; ++il) {
+    int n_active_layer = n_layer - hparams.nextn_predict_layers;
+
+    for (int il = 0; il < n_active_layer; ++il) {

Another try

./bin/llama-perplexity -m $same_model -f wiki.test.raw -t 64 -ub 1024 -no-fa -c 1024
perplexity: calculating perplexity over 280 chunks, n_ctx=1024, batch_size=2048, n_seq=2
perplexity: 28.43 seconds per pass - ETA 1 hours 6.33 minutes
[1]3.3205,[2]1010.3598,[3]113.9258,[4]604.1925,^C

CUDA (with --cpu-moe)

Similar nonsense PPL for -c 1024 -ub 1024
Crash with Illegal memory access for -c 2048 -ub 2048

ikawrakow · 2026-07-13T09:13:02Z

+            0);
+}
+
+// Keep DSV4 normalization explicit. The generic helper may fuse RMS


And this is required because? The generic llm_build_norm is used in absolutely all models with absolutely zero issues, but suddenly DS4 requires that the RMS norm is not fused?

At one point during the port, using the mainline RMS helped keep the output consistent. To be fair I ran a test with norm again and didn't encounter any problems so I imagine it was an unintended side effect during development.

ikawrakow · 2026-07-13T09:26:32Z

+    return cur;
+}
+
+static ggml_tensor * dsv4_build_raw_k_from_idxs(


Not used, discard.

ikawrakow · 2026-07-13T09:34:06Z

+    dsv4_new_i64_input(ctx, &inputs.state_write_idxs, (int64_t) plan.state_write_idxs.size(), (std::string(tag) + "_state_write").c_str());
+    dsv4_new_i32_input(ctx, &inputs.state_write_pos, (int64_t) plan.state_write_pos.size(), (std::string(tag) + "_write_pos").c_str());
+    dsv4_new_mask_input(ctx, &inputs.kq_mask, std::max<int64_t>(1, plan.n_kv), n_tokens, (std::string(tag) + "_kq_mask").c_str());
+    if (std::strcmp(tag, "dsv4_lid") == 0) {


llama.cpp style nonsense is not tolerated in this repository. We do have ggml_hadamard, and this is what you need to use.

Oh, that's right! My bad, the plan was to test their code and then adapt it to what we have here but I ended up forgetting to get to that part.

ikawrakow · 2026-07-13T09:34:28Z

+    const int64_t n_indexer_head = hparams.indexer_n_head;
+    const int64_t n_tokens = cur->ne[1];
+    const int64_t n_lid = llm.lctx.dsv4.lid_plan.n_kv;
+    ggml_tensor * k_rot = llm.lctx.dsv4.inputs.lid.k_rot;


ikawrakow · 2026-07-13T09:34:55Z

+            hparams.dsv4_compress_rope_base, llm.freq_scale,
+            llm.ext_factor, dsv4_rope_attn_factor(llm.freq_scale, llm.ext_factor), llm.beta_fast, llm.beta_slow);
+    indexer_q = ggml_concat(ctx0, indexer_q_nope, indexer_q_pe, 0);
+    indexer_q = ggml_mul_mat(ctx0, k_rot, indexer_q);


replace with ggml_hadamard

ikawrakow · 2026-07-13T09:35:11Z

+                        hparams.indexer_head_size,
+                        il,
+                        "lid_state_compress");
+                if (lctx.dsv4.inputs.lid.k_rot != nullptr) {


replace with ggml_hadamard

ikawrakow · 2026-07-13T09:35:24Z

+            struct ggml_tensor * state_write_idxs = nullptr;
+            struct ggml_tensor * state_write_pos = nullptr;
+            struct ggml_tensor * kq_mask = nullptr;
+            struct ggml_tensor * k_rot = nullptr;


ikawrakow · 2026-07-13T09:35:38Z

+        std::vector<float> csa_mask_data;
+        std::vector<float> hca_mask_data;
+        std::vector<float> lid_mask_data;
+        std::vector<float> lid_k_rot_data;


ikawrakow · 2026-07-13T09:36:47Z

+    return *dst;
+}
+
+static int64_t dsv4_k_rot_size(int64_t n_embd) {


Discard and see how the Hadamard block size is determined in the standard ik_llama.cpp Hadamard transform usage.

ikawrakow · 2026-07-13T10:20:50Z

+    llm.cb(indexer_k, "lid_k_stream", il);
+
+    ggml_tensor * lid_mask = dsv4_build_raw_mask_view(ctx0,
+            llm.lctx.dsv4.inputs.lid.kq_mask, n_lid, n_tokens);


In what way is lid.kq_mask different from the standard attention mask that we have been building forever?

At first glance it's semantically similar, the only difference is in the structure of the raw attention mask since the key axis is the compressed LID cache. I reused inputs.csa.kq_mask and removed duplicate inputs in the latest commits.

ikawrakow · 2026-07-13T11:05:55Z

+    return dsv4_concat_named(ctx, t, row, 1, "dsv4_append_zero_row");
+}
+
+static ggml_tensor * dsv4_with_zero_dep(ggml_context * ctx, ggml_tensor * t, ggml_tensor * dep) {


What is this supposed to do?

We sum the elements of dep, to create tensor zero that has 1 element.
We then scale the result stored in zero with 0, to obtain a tensor containing exactly one zero.
We then use ggml_add1 to add this zero to the elements of t.

So, basically, we are wasting a bunch of CPU or GPU cycles to obtain the exact same tensor as we had before? Is this the new way of making a copy of a tensor?

I added a comment above the function for future reference, but in short this forces compression/cache work to complete before state persistence or dependent cache reads can execute.

The graph would originally have had this dependency:

dep -> compression/cache write t -> state persistence or dependent cache read

Since the branches are independent, we need to create a tensor dependency:

dep -> sum -> scale(0) -> add1(t, zero) -> state persistence/read \________________________________________/

The output is numerically equal to t but it now depends on dep.

So, forever we were able to make sure that writing to the KV cache has completed before using it, but now suddenly we cannot?

Just put

if (hca_dep) { ggml_build_forward_expand(gf, hca_dep); }

before

ggml_tensor * hca_persist_kv = ggml_get_rows(ctx0, hca_state_kv, lctx.dsv4.inputs.hca.state_persist_src_idxs);

discard all occurrences of dsv4_with_zero_dep, and lets see how that goes.

SamuelOliveirads · 2026-07-14T01:55:33Z

I was working on this branch in Windows only to later notice that the compiler wasn’t flagging warnings for unused variables or the llama_model_loader instance, in any case I applied the changes and the PPL remained stable for IQ2XXS and MXFP4. Unfortunately, there’s a bug causing PPL to explode for -ns 2, returning [1]3.7139,[2]835.9983 while independent runs stay at [1]3.7139,[2]3.7342, this is something I didn’t have time to fix. I also noticed that -fidx reduces the reserve from 14.89 GiB to 10.95 GiB which is quite significant.

ikawrakow · 2026-07-14T15:37:25Z

+        // DSV4's residual/hyperconnection boundary must stay on the CPU until
+        // CUDA execution of the mixed HC post path has been parity-validated.
+        if (model.arch == LLM_ARCH_DEEPSEEK4 && strcmp(name, "l_out") == 0) {
+            ggml_backend_sched_set_tensor_backend(lctx.sched, cur, lctx.backend_cpu);


Haha, I lost I don't know how much time trying to figure out why the l_out concat op is being run on the CPU. We need this because? Just remove it.

Yeah, I had put those other issues on hold to prioritize the FA. Fortunately I was able to apply the various fixes and adjustments: dsv4_with_zero_dep is gone, l_out no longer resides on the CPU, -ctk Q8_0 works alongside some measures to prevent context shifts, -sm graph, and other changes on the server.

The main thing was that the high perplexity issue I encountered with -ns 2 was resolved by using dsv4_build_mask_stream_view, however once I go past 1024 tokens shape, the fused/IQK MoE operation diverged so I added a workaround in the commit feb77a9 to validate it independently, I’m open to suggestions for a better solution.

Oh, and before I forget, dsv4_with_zero_dep came from the mainline port, it’s practically a faithful adaptation so I didn’t think too deeply about the approach.

This reverts commit 04f9b42.

ikawrakow · 2026-07-15T05:16:56Z

The latest version crashes with an assert that the mask is not f16 when attempting to run with FA on (yesterday it was working).

The more worrying thing is the Wikitext2 perplexity is systematically higher than what I get with llama.cpp: 3.9695 vs 3.9354 for context of 4096 tokens (so we actually exercise the indexer) after a full run of 70 chunks. Such a difference is way too high to be caused by numerical round-off, so something is still not quite right. Oh, it is slower than llama.cpp as well, so as a user why would I use this instead of just using mainline?

ikawrakow · 2026-07-15T11:32:37Z

+                raw_mask = ggml_cast(ctx0, raw_mask, csa_mask->type);
+            }
+            ggml_tensor * k_all = dsv4_concat_named(ctx0, raw_k, csa_k, 2, "dsv4_raw_plus_csa_k");
+            ggml_tensor * kq_mask = dsv4_concat_named(ctx0, raw_mask, csa_mask, 0, "dsv4_raw_plus_csa_mask");


So, we are concatenating the raw mask, which is n_kv x n_token, with the sparse attention mask, which is n_top_k x n_token? Correspondingly K and V are also concatenated versions of the full cache and the cache entries selected by the indexer.

Can you explain why we want to do this?

The raw side represents the recent SWA window while the CSA side represents indexer-selected compressed historical blocks. Their K tensors and additive masks are concatenated so attention performs one joint softmax over exact recent tokens plus longer-range compressed memory. The latent K representation is used for both K and V here.

As for HCA its follow the same composition.

ikawrakow · 2026-07-15T11:33:43Z

+            if (raw_mask->type != hca_mask->type) {
+                raw_mask = ggml_cast(ctx0, raw_mask, hca_mask->type);
+            }
+            ggml_tensor * k_all = dsv4_concat_named(ctx0, raw_k, hca_k, 2, "dsv4_raw_plus_hca_k");


Similar comment to the above. Why on Earth are we concatenating the regular cache/mask with the HC cache/mask?

ikawrakow · 2026-07-15T13:24:26Z


 #include "concat.cuh"

-// contiguous kernels


Why are changes being made to concat.cu? If the reason is that it was not the same as concat.cu in llama.cpp, that is not a valid reason to copy/paste from there. The concat ops here can serve different purposes not exercised in mainline.

In the first version I wrote I was unable to run it using non-FA CUDA so the change was based on the assumption that DSV4 required a more generic F16/BF16 strided concatenation path. However, after making the most recent changes I ran new tests and the same problem did not occur so I reverted the code.

ikawrakow · 2026-07-15T13:26:31Z

    return result;
 }

+struct ggml_tensor * ggml_rope_ext_back(


We absolutely do not need this function here. We have ggml_rope_back instead.

I probably copied this from the mainline port.

Reverted.

ikawrakow · 2026-07-15T13:39:17Z

            float                 beta_fast,
            float                 beta_slow);

+    GGML_API struct ggml_tensor * ggml_rope_ext_back(


Discard this and use ggml_rope_back instead.

ikawrakow · 2026-07-15T14:34:04Z

-        ggml_tensor * normed = ggml_rms_norm(ctx0, flat, hparams.f_norm_rms_eps);
-        normed = ggml_mul(ctx0, normed, gamma);                       // [S*H, T]
-        ggml_tensor * mixes = ggml_mul_mat(ctx0, phi, normed);        // [(S+2)*S, T]
+        ggml_tensor * mixes = build_mhc_pre_projection(Rin, phi, gamma,


It is obviously useful to try to unify DS4 and OpenPangu as there is basically the same thing being done. But after merging #2132, this change results in a ~10% performance penalty for openPangu compared to the main branch. I didn't investigate if it is the combined effect of build_mhc_pre_projection, build_mhc_weighted_sum and build_mhc_post or if it is one of those causing the performance penalty.

From what I could tell by comparing it with main the regression occurs in build_mhc_post. I added conditions to use ggml_sum_rows_ext via OpenPangu, while DSV4 uses the old approach and this restored performance.

ikawrakow · 2026-07-15T15:06:25Z

Will continue in a ~2-3 days.

ikawrakow · 2026-07-15T16:04:59Z

+        int64_t n_stream) {
+    const int64_t n_tokens = x->ne[2];
+    ggml_tensor * out = nullptr;
+


This function assumes that x->ne[3] == 1 and weights->ne[3] == 1. Also, n_stream is not actually necessary as argument because n_stream must be the same as x->ne[1] and weights->ne[0], else it will fail. Also, n_embd must be the same as x->ne[0], no?

Given this, the implementation can be replaced with

auto w = ggml_reshape_4d(ctx0, weights, 1, weights->ne[0], weights->ne[1], weights->ne[2]); return ggml_mul_multi_add(ctx0, x, w);

If we are worried that the above conditions are not met, we can say

if (x->ne[3] == 1 && weights->ne[3] == 1 && x->ne[0] == n_embd && x->ne[1] == n_stream && x->ne[1] == weights->ne[0] && x->ne[2] == weights->ne[1] && weights->ne[2] == 1) { auto w = ggml_reshape_4d(ctx0, weights, 1, weights->ne[0], weights->ne[1], weights->ne[2]); return ggml_mul_multi_add(ctx0, x, w); }

It is not a big deal, it gives just 1-2% performance boost. But still, why wouldn't we use the ik_llama.cpp abilities to their full potential?

The ggml_mul_multi_add op was specifically added to sum up the contributions of the selected experts multiplied with the experts weights. These are not experts, but the operations being performed are exactly the same.

Good idea, I applied ggml_mul_multi_add directly and no longer materializes weights. The retained dimensions are used only to assert the shared DSV4/OpenPangu caller contract. The reshape produces [1, streams, tokens, 1], equivalent to the proposed 4D representation. I also made the ne[3] and source weight dimensions explicit so an incompatible caller fails at the helper boundary.

ikawrakow · 2026-07-15T16:24:13Z

+
+    *comb = ggml_cont(ctx0, dsv4_view_2d(ctx0, mixes, hc*hc, nt, 2*hc));
+    *comb = dsv4_hc_affine(ctx0, *comb, scale_comb, base_comb);
+    *comb = ggml_reshape_3d(ctx0, *comb, hc, hc, nt);


Replace this and the next line with

*comb = ggml_sinkhorn(ctx0, *comb, hc, hparams.dsv4_hc_sinkhorn_iters, hparams.dsv4_hc_eps, false);

Then remove the build_hc_sinkhorn function.

I tested the idea, on the CPU non-FA IQ2XXS c2048 one-chunk control, the existing DSV4 path produced 3.3172 PPL, while the direct ggml_sinkhorn replacement produced 4.5795. An earlier bounded test also changed the selected token so I reverted the experiment. This is not an FA artifact, but I have not yet determined whether the mismatch is the caller’s flattened matrix orientation or the shared operation’s expected layout so I didn't applied the private path for now.

# Conflicts: # src/graphs/build_openpangu.cpp

SamuelOliveirads · 2026-07-16T16:15:44Z

The latest version crashes with an assert that the mask is not f16 when attempting to run with FA on (yesterday it was working).

The more worrying thing is the Wikitext2 perplexity is systematically higher than what I get with llama.cpp: 3.9695 vs 3.9354 for context of 4096 tokens (so we actually exercise the indexer) after a full run of 70 chunks. Such a difference is way too high to be caused by numerical round-off, so something is still not quite right. Oh, it is slower than llama.cpp as well, so as a user why would I use this instead of just using mainline?

I fixed the CPU FA mask assertion in commit abf5c96.

On the Antirez IQ2XXS model at c4096, branch non-FA was 3.747916869 versus mainline 3.717200623. Disabling fused MoE improved the branch to 3.731602469, removing approximately 53% of that bounded gap. -fidx, graph reuse, and MoE chunk size were neutral, while sampled LID top-k overlap was effectively identical.

Tensor probing found a very small initial branch/mainline difference at layer-0 normalization/latent projection input which is amplified by the quantized Q/K path. The fused-MoE path is a measurable contributor, but the sampled tensors do not yet prove that its kernel is the original owner so a same-input fused-expert replay is still required.
I paused that deeper attribution to address the review so I am not claiming the PPL issue is closed.

SamuelOliveirads added 9 commits July 2, 2026 17:07

initial map to load deepseek 4 arch

8b9ce35

wip

b3f2a8b

wip: match graph build and attn logic for dpv4

40eb9f1

wip: Enhance DeepSeek-V4 architecture with new tensor types and sqrts…

3de2b0a

…oftplus gating function

Update DeepSeek-V4 to support raw key indexing with read/write indices

1cffe9b

fix mismatch in attn_raw

3599588

Enable FA with CSA/HCA

129e2f7

Fix logit mismatch with FA path

6bfe51f

Clean traces and logs for debug

88f5247

joelfarthing mentioned this pull request Jul 12, 2026

ggml: add fused sinkhorn op (eps + output-layout params) #2115

Merged

4 tasks

SamuelOliveirads added 5 commits July 12, 2026 11:44

Refactor DSV4 tensor handling for MTP execution and improve raw conte…

2cf9fd9

…xt management

Merge branch 'main' into feat-deepseek-v4-port

39e3c51

# Conflicts: # convert_hf_to_gguf.py # ggml/src/ggml.c # src/CMakeLists.txt # src/llama-build-context.cpp # src/llama-build-context.h # src/llama-hparams.cpp # src/llama-load-tensors.cpp

Refactor DeepSeek4 tensor operations: replace manual weighted sum and…

7130d39

… post-processing with new helper functions

Share mHC pre-projection and fix packed DSV4 writes

41bc2e7

DSV4: add shared top-k selection and improve mask handling

83b3e14

ikawrakow reviewed Jul 13, 2026

View reviewed changes

SamuelOliveirads added 2 commits July 13, 2026 20:29

Share CSA visibility mask with DSV4 LID

bbfbeaa

dsv4: document dependency ordering and reset state

724ebba

Remove DSV4 zero-dependency graph shim

050021d

ikawrakow reviewed Jul 14, 2026

View reviewed changes

SamuelOliveirads added 6 commits July 14, 2026 17:38

Fix DSV4 packed stream execution

feb77a9

Remove DSV4 l_out backend override

78a5226

Enable DSV4 quantized K-only cache

04f9b42

Revert "Enable DSV4 quantized K-only cache"

74fcc71

This reverts commit 04f9b42.

Fix DSV4 quantized cache accounting

e2b0340

Fail closed on unsupported DSV4 cache lifecycle operations

3d6acd3

ikawrakow reviewed Jul 15, 2026

View reviewed changes

SamuelOliveirads added 8 commits July 15, 2026 13:49

Fix DSV4 CPU FA mask adaptation

abf5c96

Merge remote-tracking branch 'ik/main' into feat-deepseek-v4-port

78d5c0d

# Conflicts: # src/graphs/build_openpangu.cpp

Reuse shared inverse RoPE operation for DSV4

802187a

Reuse maintainer CUDA concat implementation

4a652d4

Optimize OpenPangu mHC post reduction

7794e70

Use mul_multi_add for mHC weighted sums

e8538d1

Document DSV4 mixed attention composition

aae55b3

Align mHC reductions with maintainer implementation

af49f92

Conversation

SamuelOliveirads commented Jul 11, 2026

Uh oh!

ikawrakow commented Jul 12, 2026

Uh oh!

ikawrakow commented Jul 12, 2026

Uh oh!

SamuelOliveirads commented Jul 12, 2026

Uh oh!

ikawrakow commented Jul 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SamuelOliveirads commented Jul 14, 2026

Uh oh!

ikawrakow Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikawrakow commented Jul 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikawrakow Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ikawrakow commented Jul 15, 2026

Uh oh!

ikawrakow Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

ikawrakow Jul 14, 2026 •

edited

Loading

ikawrakow Jul 15, 2026 •

edited

Loading

ikawrakow Jul 15, 2026 •

edited

Loading