Merge tag 'b9946' of https://github.com/ggml-org/llama.cpp into feature/turboquant-kv-cache

[sxlmnwb]

Summary Merge 'b9888' of https://github.com/ggml-org/llama.cpp

Resolved files conflicts:

1. ggml/src/ggml-cuda/fattn.cu

• Added TurboQuant types (turbo2, turbo3, turbo4) to the ggml_cuda_fattn_kv_type_supported function
• Used the upstream helper function for KV type validation with an additional 64-byte alignment check for turbo types

2. ggml/src/ggml-metal/ggml-metal.metal

• Combined template instantiation for kernel_cpy_tq3_1s and kernel_cpy_tq4_1s (TurboQuant copy kernels)

3. src/llama-kv-cache.cpp

• Incorporate Hadamard rotation logic for DeepSeek architectures (DEEPSEEK32 and DEEPSEEK4)
• Use more comprehensive upstream conditions

4. src/llama-model-loader.cpp

• Adopts the llama_ftype_name structure from upstream with guessed_prefix_len
• Adds the TQ3_1S and TQ4_1S types to the switch case

5. tests/test-backend-ops.cpp

• Incorporates test cases from upstream (Q4_0, Q8_0, MXFP4)
• Retained the TurboQuant TQ4_1S test cases for kernel validation

6. tests/test-quantize-fns.cpp

• Added TQ3_1S to the list of types that use MAX_DOT_PRODUCT_ERROR_LOWBIT

7. tools/server/server-context.cpp (5 conflicts)

• Fixed slot selection logic by using task.id_slot instead of id_slot
• Merged the cache_key slot selection from the fork
• Used batch.size() instead of batch.n_tokens for consistency with upstream
• Fixed the prompt_checkpoint_restored flag
• Merged checkpoint logic with is_last_user_message
• Replaced continue/break with return in the iterate lambda
• Removed references to nonexistent functions (llama_get_ctx_other, process_chunk)

8. tools/server/server-models.cpp (6 conflicts)

• Use the upstream state-based protocol structure (CMD_CHILD_TO_ROUTER_STATE)
• Retain the CMD_CHILD_TO_ROUTER_ERROR handler for the abort callback
• Use the update_status_args struct for the update_status signature
• Fix the should_wake lambda with an inline lambda
• Use the child_proc->is_alive() and child_proc->terminate() methods instead of direct C functions
• Merged update_last_error and update_download_progress

9. tools/server/server-models.h

• Added the CMD_CHILD_TO_ROUTER_STATE define
• Merged method declarations from upstream and fork

10. tools/server/server.cpp

• Merged the common_models_handler initialization from upstream
• Retained the abort callback for error reporting from child processes
• Used a more comprehensive router condition (path.empty() && hf_repo.empty())

Additional information

Build environment:

export MY_FLAGS_NVIDIA="-march=znver5 -mtune=znver5 -O3 -pipe -flto=thin -fno-fat-lto-objects -fno-strict-aliasing -Wno-unused-command-line-argument -Wno-missing-noreturn -Wno-unreachable-code-break -mllvm -polly -mllvm -polly-tiling=1 -mllvm -polly-run-inliner -mllvm -polly-dependences-computeout=0 -mllvm -polly-ast-use-context"
export MY_LINKER_FLAGS="-fuse-ld=mold -Wl,--thinlto-jobs=$(nproc) -Wl,-rpath,/home/sxlmnwb/kernel-build/xBoreUp_Clang/lib/x86_64-unknown-linux-gnu"

cmake -B build-nvidia \
    -DGGML_CUDA=ON \
    -DGGML_CUDA_CUDNN=ON \
    -DGGML_RPC=ON \
    -DCUDAToolkit_ROOT=/opt/cuda \
    -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc \
    -DCMAKE_CUDA_ARCHITECTURES=89 \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_C_COMPILER=/home/sxlmnwb/kernel-build/xBoreUp_Clang/bin/clang \
    -DCMAKE_CXX_COMPILER=/home/sxlmnwb/kernel-build/xBoreUp_Clang/bin/clang++ \
    -DCMAKE_CUDA_HOST_COMPILER=clang++ \
    -DCMAKE_C_FLAGS="$MY_FLAGS_NVIDIA" \
    -DCMAKE_CXX_FLAGS="$MY_FLAGS_NVIDIA -stdlib=libc++" \
    -DCMAKE_EXE_LINKER_FLAGS="$MY_LINKER_FLAGS -lc++ -lc++abi" \
    -DCMAKE_SHARED_LINKER_FLAGS="$MY_LINKER_FLAGS -lc++ -lc++abi"

Build log:
PANd6hWU

Tested on:
CPU: AMD Ryzen AI 9 HX 370 (8+16) @ 5.16 GHz
GPU 2: NVIDIA GeForce RTX 4060 Max-Q / Mobile [Discrete]

MoE model:
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive - Q4_K_P

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: NO

[sxlmnwb]

Summary Merge 'b9946' of https://github.com/ggml-org/llama.cpp

Resolved files conflicts:

1. tools/server/server.cpp

• Adopted upstream refactoring: wrapper llama_server(params, argc, argv) split from main()
• Retained TurboQuant router GPU-skip optimization (!is_router_server guard for llama_backend_init())
• Retained abort callback with CMD_CHILD_TO_ROUTER_ERROR for child process crash reporting

2. src/llama-kv-cache.cpp

• Retained TurboQuant ggml_mul_mat_aux helper for Hadamard matmul
• Retained InnerQ CUDA cross-TU shared state stubs (g_innerq_finalized, g_innerq_scale_inv_host)
• Added fallback stub implementations when CUDA is not available

3. include/llama.h

• Added upstream LLAMA_FTYPE_MOSTLY_Q2_0 = 41 for new Q2_0 quantization
• Retained TurboQuant LLAMA_FTYPE_MOSTLY_TQ3_1S = 43 and LLAMA_FTYPE_MOSTLY_TQ4_1S = 44

4. ggml/include/ggml.h

• Added upstream GGML_TYPE_Q2_0 = 42
• Renumbered TurboQuant types: TURBO2_0=43, TURBO3_0=44, TURBO4_0=45, TQ3_1S=46, TQ4_1S=47
• Updated GGML_TYPE_COUNT from 47 to 48

5. ggml/src/ggml-vulkan/ggml-vulkan.cpp

• Retained TurboQuant SET_ROWS validation with 128-element block alignment check for turbo2/3/4
• Kept turbo types in the supported SET_ROWS type switch

6. gguf-py/gguf/constants.py (3 conflicts)

• Added upstream Q2_0 = 42 to GGMLQuantizationType enum
• Retained TurboQuant types with new numbering: TURBO2_0=43, TURBO3_0=44, TURBO4_0=45, TQ3_1S=46, TQ4_1S=47
• Added upstream MOSTLY_Q2_0 = 41 to LlamaFileType enum
• Retained MOSTLY_TQ3_1S = 43 and MOSTLY_TQ4_1S = 44
• Added upstream Q2_0 quantization traits: (64, 2 + 16)
• Retained TQ3_1S and TQ4_1S traits

7. ggml/src/ggml-cuda/ggml-cuda.cu (4 conflicts)

• Simplified TurboQuant dispatch in ggml_cuda_mul_mat to match upstream's refactored structure
• Removed deprecated split-buffer multi-GPU logic (ggml_backend_buft_is_cuda_split, get_row_rounding, etc.)
• Removed duplicate hint and cc declarations that conflicted with upstream
• Replaced ggml_cuda_mul_mat_batched_cublas (removed upstream) with ggml_cuda_mul_mat_cublas
• Added TQ weight type detection (is_tq_weight) with early dispatch to fused WHT kernel or TQ4_1S cuBLAS path
• Added back ggml_cuda_host_staging_pool struct and ggml_cuda_get_staging() for cross-device copies
• Added back ggml_cuda_copy_across_devices and ggml_cuda_copy2d_across_devices definitions
• Merged upstream F16 SET_ROWS support alongside TurboQuant SET_ROWS alignment checks

8. common/speculative.cpp (post-merge fixes)

• Renamed has_draft_mtp → has_mtp to match upstream variable naming
• Removed duplicate common_speculative_impl_draft_dflash struct definition (introduced by merge)

9. common/peg-parser.cpp (post-merge fixes)

• Added prefix_and_next struct and collect_prefix_and_next() methods to trie class
• Restored gbnf_char_class() function alongside upstream's gbnf_excluding_pattern()
• Removed extraneous closing brace } left from merge

10. tests/test-backend-ops.cpp (post-merge fix)

• Fixed type → type_dst in TQ4_1S max error check inside max_nmse_err() override

11. tests/peg-parser/test-gbnf-generation.cpp (post-merge fix)

• Replaced old trie-based grammar test expectations with upstream's simplified excluding pattern format
• All until/ac grammar tests updated to new pattern format

12. docs/speculative.md (post-merge fix)

• Retained full TurboQuant documentation (EAGLE-3, DFlash, n-gram variants, ngram-mod)
• Upstream had removed documentation in favor of CLI-only reference