Another minor readme update#592

Closed

saood06 wants to merge 3795 commits into

mainfrom

s6/readme-minor2

This pull request is big! We're only showing the most recent 250 commits

Commits on Feb 7, 2025

Add additional checks for iq1_s_r4 quantization (#191)

ikawrakow
and
Iwan Kawrakow
authored
cuda: non-contiguous rms norm (#190)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 8, 2025

Revert #79 (#192)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 9, 2025

Use Q8_K_128 for IQ1_S_R4 and IQ1_M_R4 matrix multiplications (#194)

ikawrakow
and
Iwan Kawrakow
authored
FA: Add option to build all FA kernels (#197)

ikawrakow
and
Iwan Kawrakow
authored
Add optional MLA (#188)

authored

Commits on Feb 10, 2025

Load all MoE experts during warmup and make warmup 1 token (#198)

saood06
and
sszymczy
authored

Commits on Feb 11, 2025

DeepSeek FA support (CPU only) (#200)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 12, 2025

Fix imatrix overprotectiveness (#202)

ikawrakow
and
Iwan Kawrakow
authored
Fix iqk_mul_mat on AVX512 systems that are missing BF16 support (#204)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 13, 2025

Faster MLA prompt processing (#205)

ikawrakow
and
Iwan Kawrakow
authored
MLA: allow Q8_0 K-cache for MLA (#206)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 15, 2025

Moving 4D gemm logic from ggml.c to iqk_mul_mat.cpp (#207)

ikawrakow
and
Iwan Kawrakow
authored
Bug fix in activation quantization
Iwan Kawrakow
committed

Commits on Feb 19, 2025

Repack also experts (#210)

ikawrakow
and
Iwan Kawrakow
authored
Q8_KV: 8-bit quantization type targeting the KV cache (#208)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 20, 2025

Optimized GEMM/GEMV for IQ1_S (#212)

ikawrakow
and
Iwan Kawrakow
authored
Fix NEON gemm/gemv for legacy quants when row size is not divisible by 128 (#213)

ikawrakow
and
Iwan Kawrakow
authored
Honor attn_output specified in the command line also for low-bit quants
Iwan Kawrakow
committed

Commits on Feb 21, 2025

Hopefully this really fixes the confusion between AVX512 and FANCY_SIMD (#216)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 22, 2025

Better strategy for attention matrix multiplications when generating tokens (#218)

ikawrakow
and
Iwan Kawrakow
authored
Fuse MoE up and gate matrix multiplications (#219)

ikawrakow
and
Iwan Kawrakow
authored
Fix #217 (#220)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 23, 2025

Fix compilation error with IQK_FA_ALL_QUANTS enabled (#226)

ikawrakow
and
Iwan Kawrakow
authored
Add new sweep-bench benchmark (#225)

saood06
and
sszymczy
authored
Fused MoE ffn_up and ffn_gate (#229)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 24, 2025

Fix #230 (#231)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 25, 2025

Give the user the option to override where model weights are stored (#232)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Feb 27, 2025

Faster MLA on CUDA (#234)

ikawrakow
and
Iwan Kawrakow
authored
Option to use MLA without a transposed cache (#235)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 1, 2025

Reduce size of compute buffers (#237)

ikawrakow
and
Iwan Kawrakow
authored
A better way to measure the cost of ggml_barrier (#238)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 2, 2025

SER - Smart Expert Reduction (#239)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 3, 2025

Flash MLA (CPU only) (#240)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 5, 2025

DeepSeek CUDA Flash Attention (#241)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 7, 2025

Custom quantization rules with regular expressions (#244)

ikawrakow
and
Iwan Kawrakow
authored
Better FlashMLA (#243)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 8, 2025

Faster FlashMLA prompt processing (#246)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 9, 2025

This works on CUDA, but (#247)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 10, 2025

Faster MoE token generation on CUDA (#248)

ikawrakow
and
Iwan Kawrakow
authored
DeepSeek imatrix stuff (#250)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 12, 2025

MLA-2: Allow usage of q8_0 for KV cache on CUDA (#252)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 13, 2025

FlashMLA-2 (CPU): faster and smaller compute buffer size (#253)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 17, 2025

Prepare wk_b tensors of DeepSeek models on the fly (#259)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 18, 2025

FlashMLA-2: reduce compute buffer size (CUDA and CPU) (#260)

ikawrakow
and
Iwan Kawrakow
authored
Compile time option to use bf16 for qunts without MMQ kernels (#261)

ikawrakow
and
Iwan Kawrakow
authored
Fix #261 (#262)

ikawrakow
and
Iwan Kawrakow
authored
Make Q8_0 KV cache work with mla=2,fa on CUDA (#264)

ikawrakow
and
Iwan Kawrakow
authored
Allow q8_0 cache on the CPU for FlashMLA-2 (#265)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 19, 2025

Prevent FlashMLA-1 from running on CUDA (#268)

ikawrakow
and
Iwan Kawrakow
authored
Fix ggml_compute_forward_dup_q (#269)

ikawrakow
and
Iwan Kawrakow
authored
Honor mmap setting when using tensor overrides (#270)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 21, 2025

Convert models to row-interleaved quants using the quantize tool (#272)

ikawrakow
and
Iwan Kawrakow
authored
FlashMLA-3: the best of both worlds (CPU only) (#273)

ikawrakow
and
Iwan Kawrakow
authored
Specify tensor name regex for tensors to be repacked (#274)

ikawrakow
and
Iwan Kawrakow
authored
Fix bug: missing parentheses in logical expression (#275)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 22, 2025

Add Gemma3 support (text only) (#276)

ikawrakow
and
Iwan Kawrakow
authored
Fighting with cmake (#279)

ikawrakow
and
Iwan Kawrakow
authored
Native build ooption for CUDA when GGML_NATIVE is set (#280)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 23, 2025

Test transparent huge pages on Linux (#278)

ikawrakow
and
Iwan Kawrakow
authored
Attempt to improve FlashMLA on the CPU (#277)

ikawrakow
and
Iwan Kawrakow
authored
Improve DeepSeek batched processing speed (#282)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 25, 2025

CUDA: better MoE implementation (#283)

ikawrakow
and
Iwan Kawrakow
authored
Update sweep bench (depracating .jsonl support) (#289)
saood06
authored
llama-bench: enable having different number of threads for tg and pp (#284)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 27, 2025

Use bf16 instead of fp16 block scales for q8_1 (#292)

ikawrakow
and
Iwan Kawrakow
authored
Make sure tensor row size is multiple of block size also when quantizing with --pure (#294)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Mar 29, 2025

Quantization improvements (#295)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 1, 2025

Fix #300 (#301)

ikawrakow
and
Iwan Kawrakow
authored
Additional guards for interleaved quants (#299)

ikawrakow
and
Iwan Kawrakow
authored
Quantization improvements (2) (#302)

ikawrakow
and
Iwan Kawrakow
authored
Fix ARM_NEON build failure due to q8_2 (#303)

ikawrakow
and
Iwan Kawrakow
authored
docs: update README.md (#304)
eltociear
authored

Commits on Apr 3, 2025

Metal: much faster MoE prompt processing (#307)

ikawrakow
and
Iwan Kawrakow
authored
Fix GCC compilation errors on ARM (#309)

ikawrakow
and
Iwan Kawrakow
authored
Metal: FA and FlashMLA (#310)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 4, 2025

Add -flax-vector-conversions for GCC on ARM (#311)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 5, 2025

We need to synchronize before using device to host async memcpy (#313)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 7, 2025

Update LICENSE
ikawrakow
authored
Add copyright notices (#317)

ikawrakow
and
Iwan Kawrakow
authored
Better iq2_xs quantization (#312)

ikawrakow
and
Iwan Kawrakow
authored
Use links for ggml/llama.cpp authors (#318)

ikawrakow
and
Iwan Kawrakow
authored
Update AUTHORS
ikawrakow
authored
Update AUTHORS
ikawrakow
authored

Commits on Apr 8, 2025

Guard against attempts to use MLA for non-MLA models (#320)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 10, 2025

LlaMA-4 support (text only) (#321)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 11, 2025

Correct L4 rms_norm (#324)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 12, 2025

Fix KLD precision (#325)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 13, 2025

Improved IQ1_M quantization (#327)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 14, 2025

Add ability to hide imatrix details in llama-quantize (#329)

ikawrakow
and
Iwan Kawrakow
authored
imatrix: collect layer influence statistics (#328)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 15, 2025

Allow q8_0 KV cache for head size 256 (#330)

ikawrakow
and
Iwan Kawrakow
authored
Better gemm/gemv on AVX2 fr q4_0_r8 (#331)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 17, 2025

Better TG performance for GQA models (CPU) (#332)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 21, 2025

Fix termux/android build (#336)
saood06
authored

Commits on Apr 22, 2025

Add support for bitnet2b_2501 model (#337)

saood06
and
potassiummmm
authored
BitNet adjustments (#338)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 24, 2025

Update gguf-py constants (#298)
saood06
authored
cuda: use switch in constexpr funcs (#343)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 25, 2025

Fix LLaMA-4 attention (#342)

ikawrakow
and
Iwan Kawrakow
authored
Fix FA on ARM (#346)

ikawrakow
and
Iwan Kawrakow
authored
Add ability to manually set arch flags (#347)

ikawrakow
and
Iwan Kawrakow
authored
Fix q4_1 and q5_1 on Arm (#348)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 26, 2025

Add support for Cohere2 (#341)

ikawrakow
and
Iwan Kawrakow
authored
Fix division by zero bug (#349)

ikawrakow
and
Iwan Kawrakow
authored
Add GLM-4-0414 Model Support (#344)
ubergarm
authored

Commits on Apr 29, 2025

CPU FA improvements (#351)

ikawrakow
and
Iwan Kawrakow
authored
Update AUTHORS
ikawrakow
authored
Apply Qwen3 PR from llama.cpp (#355)
bharrisau
authored
Add missing enum values for qwen3 and qwen3moe (#356)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Apr 30, 2025

Fix IQK_FA_ALL_QUANTS on AVX2 (#360)

ikawrakow
and
Iwan Kawrakow
authored
Update README.md (#352)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 2, 2025

Fix model architecture name (#366)

saood06
and
junhuihe
authored
Fix FA bug on AVX2 (#364)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 3, 2025

Trying to fix iq1_s_r4/iq1_m_r4 quantization failure (#368)

ikawrakow
and
Iwan Kawrakow
authored
cmake: force MSVC compiler charset to utf-8 (#369)
Gaolingx
authored

Commits on May 4, 2025

Another attempt to fix #367 (#371)

ikawrakow
and
Iwan Kawrakow
authored
CUDA: faster FA TG for GQA models (#370)

ikawrakow
and
Iwan Kawrakow
authored
Update README.md
ikawrakow
authored
Update README.md
ikawrakow
authored
CUDA: MMQ for IQ4_KS (#374)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 5, 2025

Fix DeepSeek FA (#382)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 6, 2025

Update README.md
ikawrakow
authored

Commits on May 7, 2025

Fix build for Xeon Gold 6226R (#390)

ikawrakow
and
Iwan Kawrakow
authored
Fix DeepSeek q8_0 cache (#391)

ikawrakow
and
Iwan Kawrakow
authored
fix some MSVC build problem. (#392)
Gaolingx
authored
FlashMLA-3 for DeepSeek models on CUDA (#386)

ikawrakow
and
Iwan Kawrakow
authored
Update README.md
ikawrakow
authored

Commits on May 9, 2025

Commits on May 10, 2025

TG improvements for MoE models (#404)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 11, 2025

Fix race in the CUDA DeepSeek FA kernel (#406)

ikawrakow
and
Iwan Kawrakow
authored
Revert "Fix race in the CUDA DeepSeek FA kernel (#406)"
Iwan Kawrakow
committed

Commits on May 12, 2025

GPU offload policy (#405)

ikawrakow
and
Iwan Kawrakow
authored
Faster DeepSeek FA on CUDA (#408)

ikawrakow
and
Iwan Kawrakow
authored
Enable faster prompt processing with mainline llama.cpp GGUFs (#409)

ikawrakow
and
Iwan Kawrakow
authored
Add batch warmup to sweep-bench (#375)

ikawrakow
and
Iwan Kawrakow
authored
Fix new CUDA FA on Touring (#413)

ikawrakow
and
Iwan Kawrakow
authored
Update README.md
ikawrakow
authored

Commits on May 13, 2025

Better CPU FA performance for DeepSeek-Lite (#410)

ikawrakow
and
Iwan Kawrakow
authored
Fix imatrix calculation for MLA models (#411)

ikawrakow
and
Iwan Kawrakow
authored
Fix SER (CPU) (#415)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 14, 2025

Fix SER (CUDA) (#416)

ikawrakow
and
Iwan Kawrakow
authored
CUDA: quantized GEMM for for IQ4_K, IQ5_K, IQ6_K (#417)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 15, 2025

CUDA: quantized GEMM for for IQ2_KS, IQ2_K, IQ3_K (#418)

ikawrakow
and
Iwan Kawrakow
authored
Fix standard attention on the CPU (#421)

ikawrakow
and
Iwan Kawrakow
authored
Adding IQ5_KS - 5.25 bpw quants (#422)

ikawrakow
and
Iwan Kawrakow
authored
Adding forgotten template instance for iq5_ks (#424)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 16, 2025

Fix AVX2 implementation of IQ4_K, IQ4_KS, IQ5_K, IQ6_K (#427)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 17, 2025

IQ5_KS_R4: row-interleaved IQ5_KS (#426)

ikawrakow
and
Iwan Kawrakow
authored
Zen4: Faster PP for IQ2_KS, IQ4_KS, IQ5_KS (#428)

ikawrakow
and
Iwan Kawrakow
authored
Option to enable disable the IQK CPU FA kernels (#429)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 18, 2025

Forgotten MMQ ref and typo (#431)
Nexesenex
authored

Commits on May 20, 2025

Bug fixes from mainline (#439)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 22, 2025

Commits on May 23, 2025

gguf-split : update (#444)
Nexesenex
authored
Trellis quants with CPU inference (#441)

andrewkchan
and
Iwan Kawrakow
authored
Fix typo in non-AVX2 code branch (#445)

ikawrakow
and
Iwan Kawrakow
authored
Fix MSVC compilation (#448)

ikawrakow
and
Iwan Kawrakow
authored
Fix bug in MMVQ kernel (#446)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 24, 2025

Faster IQ3_KT and IQ4_KT (#453)

ikawrakow
and
Iwan Kawrakow
authored
Legacy quants conversion schemes in convert_hf_to_gguf.py (#449)
Nexesenex
authored

Commits on May 25, 2025

Add missing gguf-py constants (#458)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 26, 2025

CUDA implementation for IQ2_K_R4, IQ3_K_R4, IQ4_K_R4, IQ5_K_R4 (#461)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 27, 2025

CUDA GEMM and GEMV for IQ4_KS_R4 and IQ5_KS_R4 (#462)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 28, 2025

set cache_prompt default to true (#465)
saood06
authored

Commits on May 29, 2025

NEON implementation for trellis quants (#471)

ikawrakow
and
Iwan Kawrakow
authored

Commits on May 30, 2025

Replace MLA-specific KV cache with the standard KV cache (#469)

authored

Commits on May 31, 2025

forgotten refs and typo (#478)
Nexesenex
authored

Commits on Jun 1, 2025

Metal implementatio for the trellis quants. (#475)

ikawrakow
and
Iwan Kawrakow
authored
Trellis quants: faster CPU prompt processing (#482)

ikawrakow
and
Iwan Kawrakow
authored
Minor (~2%) iq2_ks TG performance improvement on CUDA (#468)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 3, 2025

convert_hf_to_gguf.py : conversion from hf weights to Q6_0 (#483)
Nexesenex
authored
Adding the XTC sampler (#486)

ikawrakow
and
Iwan Kawrakow
authored
Adding top-n-sigma sampler (#489)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 5, 2025

CUDA implementation for IQ1_S_R4 (#492)

ikawrakow
and
Iwan Kawrakow
authored
Faster CPU prompt processing for Trellis quants and MoE models (#488)

ikawrakow
and
Iwan Kawrakow
authored
MMQ implementation for IQ4_KS_R4 and IQ5_KS_R4 (#493)

ikawrakow
and
Iwan Kawrakow
authored
IQ1_M_R4 CUDA implementation (#494)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 6, 2025

Make prompt cache saving and restoring MLA aware (#497)

saood06
and
Iwan Kawrakow
authored

Commits on Jun 7, 2025

Fix #499 (#501)

ikawrakow
and
Iwan Kawrakow
authored
Add an endpoint that lists all the saved prompt caches to server (#502)
saood06
authored

Commits on Jun 8, 2025

Webui improvement (#481)
firecoperana
authored
Update AUTHORS
ikawrakow
authored
Rpc improvement (#480)

authored
Revert "Rpc improvement (#480)"
Iwan Kawrakow
committed
Fix non rpc build error (#506)

authored

Commits on Jun 9, 2025

Docs update (#509)

saood06
and
ngxson
authored

Commits on Jun 10, 2025

Fix Compile error (C2668) (#508)
Gaolingx
authored

Commits on Jun 11, 2025

IQ2_XXS: much faster CPU prompt processing (#515)

ikawrakow
and
Iwan Kawrakow
authored
Much faster iq3_xxs GEMM via repacking to q8_0_r8 (AVX2) (#516)

ikawrakow
and
Iwan Kawrakow
authored
Faster iq1_s GEMM via repacking to Q8_0_R8 (#517)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 12, 2025

iq3_s: much faster GEMM via repacking to q8_0_r8 (#518)

ikawrakow
and
Iwan Kawrakow
authored
Add top n sigma sampler and other webui fix (#512)
firecoperana
authored
Better strategy for GPU offload (#520)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 13, 2025

Perhaps a slightly better version for IQ2_XXS, IQ3_XXS, IQ3_S GEMV (#524)

ikawrakow
and
Iwan Kawrakow
authored
Update News section of readme (#510)
saood06
authored
Faster CPU prompt processing for Q4_K and Q5_K (#525)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 14, 2025

Call iqk_convert_repack in MoE GEMM (#528)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 17, 2025

Much faster CPU prompt processing (part 1) (#531)

ikawrakow
and
Iwan Kawrakow
authored
Send [DONE] for OAI compatibility (#470)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 18, 2025

Much faster CPU prompt processing (part 2) (#533)

ikawrakow
and
Iwan Kawrakow
authored
Much faster CPU prompt processing (part 3) (#534)

ikawrakow
and
Iwan Kawrakow
authored
New IQ2_KT, IQ3_KT and IQ4_KT, V2 (#529)

ikawrakow
and
Iwan Kawrakow
authored
Fix MSVC compilation error
Iwan Kawrakow
committed
Fix KT Neon / ARM typo (#536)
louiehelm
authored

Commits on Jun 19, 2025

Commits on Jun 20, 2025

New integer trellis on ARM_NEON (#544)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 21, 2025

Perhaps slightly faster trellis quants (#541)

ikawrakow
and
Iwan Kawrakow
authored
Faster ARM_NEON GEMM implementation for legacy quants (#546)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 22, 2025

Abort if IQK_IMPLEMENT is not defined
Iwan Kawrakow
committed
To use GGML_ABORT we need to include ggml-impl.h.
Iwan Kawrakow
committed

Commits on Jun 23, 2025

Much faster prompt processing for IQK quants (ARM_NEON) (#549)

ikawrakow
and
Iwan Kawrakow
authored
Much faster prompt processing for I-quants (ARM_NEON) (#550)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 24, 2025

Much faster prompt processing for k-quants (ARM_NEON) (#552)

ikawrakow
and
Iwan Kawrakow
authored
Much faster prompt processing for IQ1_S and IQ1_M on ARM_NEON (#553)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 26, 2025

Add Falcon-Edge support (#555)

ikawrakow
and
Iwan Kawrakow
authored
CUDA: MMQ for iqX_r4 quants (#557)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jun 27, 2025

Use cuBLAS for large batches and quants with block size 16 (#559)

ikawrakow
and
Iwan Kawrakow
authored
Remove what appears to be unnecessary asserts in ggml_cuda_cpy (#560)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jul 2, 2025

Merge vulkan code from mainline up to commit of 6/28/2025 (#563)

firecoperana
and
0cc4m
authored
Conditionally disable fused ops when building with Vulkan enabled (#569)

ikawrakow
and
Iwan Kawrakow
authored
Minor CUDA PP speed improvement (#567)

ikawrakow
and
Iwan Kawrakow
authored
Adding IQ3_KS quants (#566)

ikawrakow
and
Iwan Kawrakow
authored
Fix CMakeLists (#571)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jul 3, 2025

Chnage KQ mask padding to 64 (#574)

ikawrakow
and
Iwan Kawrakow
authored
Fix debug build failure with RPC off (#579)

ikawrakow
and
Iwan Kawrakow
authored
Do not crash when there is no DRY sampler (#578)

ikawrakow
and
Iwan Kawrakow
authored
Vulkan: fused rms norm (#577)

ikawrakow
and
Iwan Kawrakow
authored
Vulkan: add GGML_OP_FUSED_MUL_UNARY (#580)

ikawrakow
and
Iwan Kawrakow
authored
Vulkan: Disable multi-add for now (#581)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jul 4, 2025

Vulkan: adding GGML_OP_MULTI_ADD implementation (#582)

ikawrakow
and
Iwan Kawrakow
authored
Adding forgotten file (#583)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jul 5, 2025

Vulkan: flash attention for DeepSeek models (#584)

authored

Commits on Jul 6, 2025

Commits on Jul 7, 2025

CUDA: small PP performance improvement for MoE models (#589)

ikawrakow
and
Iwan Kawrakow
authored

Commits on Jul 8, 2025

Commits on Jul 9, 2025