Misc. bug: [CUDA error] In a dual RTX 4090 environment.

[LateLinux]

Name and Version

I/ version
(base) sdrdc@sdrdc-Super-Server:~/buun-llama-cpp$ ./build/bin/llama-cli --version
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 97020 MiB):
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, VRAM: 48510 MiB
Device 1: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, VRAM: 48510 MiB
version: 9312 (16ebdc5)
built with GNU 12.3.0 for Linux x86_64

II/ Envs

• CPU: Intel(R) Xeon(R) w5-3435X * 1
• RAM: 256GB DDR5@4800MHz
• GPU: RTX-4090 * 2 (48GB/card, 96GB total)
• OS: ubuntu 22.04 ( Kernel: 6.8.0-111-generic)
• CUDA: release 13.2, V13.2.78
• Nvidia Driver Version: 595.71.05 CUDA Version: 13.2
• cmake: 4.3.0
• gcc/g++ : 12.3.0

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

(base) sdrdc@sdrdc-Super-Server:~/buun-llama-cpp$ cat start_server.sh
  ./build/bin/llama-server \
    -m "models/unsloth/Qwen3.6-27B/Qwen3.6-27B-Q4_K_M.gguf" \
    -md "models/spiritbuun/Qwen3.6-27B-Dflash/dflash-draft-3.6-q8_0.gguf" \
    --spec-type dflash \
    --ts 1,1
    -ngl 99 -ngld 99 \
    -c 262144 -cd 1024 -ub 512 -b 2048 \
    -fa on \
    --draft-max 16 \
    --draft-min 1 \
    -ctk turbo4 -ctv turbo4 \
    --host 0.0.0.0 --port 8080 \
    -v

Problem description & steps to reproduce

I/ build process
Follow the build steps listed below:
(base) sdrdc@sdrdc-Super-Server:/buun-llama-cpp$ cmake -B build
-DGGML_CUDA=ON
-DGGML_NATIVE=ON
-DGGML_CUDA_FA=ON
-DGGML_CUDA_FA_ALL_QUANTS=ON
-DCMAKE_BUILD_TYPE=Release
(base) sdrdc@sdrdc-Super-Server:/buun-llama-cpp$ cmake --build build --config Release -j $(nproc)

II/ results got in shell (only extract the error, more details in the attachment)
/home/sdrdc/buun-llama-cpp/ggml/src/ggml-cuda/ggml-cuda.cu:98: CUDA error
CUDA error: an illegal memory access was encountered
current device: 0, in function ggml_backend_cuda_buffer_set_tensor at /home/sdrdc/buun-llama-cpp/ggml/src/ggml-cuda/ggml-cuda.cu:683
cudaStreamSynchronize(((cudaStream_t)0x2))
[New LWP 14900]
[New LWP 14909]
[New LWP 14910]
[New LWP 14911]
[New LWP 14912]
[New LWP 14913]
[New LWP 14914]
[New LWP 14915]
[New LWP 14916]
[New LWP 14917]
[New LWP 14918]
[New LWP 14919]
[New LWP 14920]
[New LWP 14921]
[New LWP 14922]
[New LWP 14923]
[New LWP 14924]
[New LWP 14925]
[New LWP 14926]
[New LWP 14927]
[New LWP 14928]
[New LWP 14929]
[New LWP 14930]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007c80894ea42f in __GI___wait4 (pid=16922, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30 ../sysdeps/unix/sysv/linux/wait4.c: 没有那个文件或目录.
#0 0x00007c80894ea42f in __GI___wait4 (pid=16922, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30 in ../sysdeps/unix/sysv/linux/wait4.c
#1 0x00007c8089b3731b in ggml_print_backtrace () from /home/sdrdc/buun-llama-cpp/build/bin/libggml-base.so.0
#2 0x00007c8089b374b3 in ggml_abort () from /home/sdrdc/buun-llama-cpp/build/bin/libggml-base.so.0
#3 0x00007c8085484fa7 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) () from /home/sdrdc/buun-llama-cpp/build/bin/libggml-cuda.so.0
#4 0x00007c80854858c4 in ggml_backend_cuda_buffer_set_tensor(ggml_backend_buffer*, ggml_tensor*, void const*, unsigned long, unsigned long) () from /home/sdrdc/buun-llama-cpp/build/bin/libggml-cuda.so.0
#5 0x00007c8089b54e8e in ggml_backend_sched_graph_compute_async () from /home/sdrdc/buun-llama-cpp/build/bin/libggml-base.so.0
#6 0x00007c8089cafff1 in llama_context::graph_compute(ggml_cgraph*, bool) () from /home/sdrdc/buun-llama-cpp/build/bin/libllama.so.0
#7 0x00007c8089cb300a in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /home/sdrdc/buun-llama-cpp/build/bin/libllama.so.0
#8 0x00007c8089cb9418 in llama_context::encode(llama_batch const&) () from /home/sdrdc/buun-llama-cpp/build/bin/libllama.so.0
#9 0x00007c8089cbf32f in llama_decode () from /home/sdrdc/buun-llama-cpp/build/bin/libllama.so.0
#10 0x00007c808a2c4721 in common_speculative_state_dflash::draft(common_params_speculative const&, std::vector<int, std::allocator > const&, int, std::vector<int, std::allocator >&, std::vector<float, std::allocator >) () from /home/sdrdc/buun-llama-cpp/build/bin/libllama-common.so.0
#11 0x00007c808a2b650d in common_speculative_draft(common_speculative, common_params_speculative const&, std::vector<int, std::allocator > const&, int, std::vector<float, std::allocator >*) () from /home/sdrdc/buun-llama-cpp/build/bin/libllama-common.so.0
#12 0x00005def8ddd3b30 in server_context_impl::update_slots() ()
#13 0x00005def8de5fa11 in server_queue::start_loop(long) ()
#14 0x00005def8dd3d7ed in main ()
[Inferior 1 (process 14894) detached]

III/ Turn off the dflash settings and everything seems OK

server_log.txt

First Bad Commit

No response

Relevant log output

Logs