Misc. bug: pressing stop while model is reasoning in webui breaks send button and doesn't stop generation

[zazarpro]

Name and Version

$ ./build/bin/llama-server --version
version: 9809 (c16c35b)
built with GNU 15.2.1 for Linux x86_64

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

Problem description & steps to reproduce

Enable reasoning in the UI, send a message, press stop button while model is reasoning. Buttons wont return to normal and once the page is refreshed in your browser you will not be able to connect until the model is finished generating the entire message ( 'Connecting to Server' screen)

First Bad Commit

Unsure, however it was not happening before so recently.

Relevant log output

No response

[atmosfar]

Possibly related? #23226

[ggerganov]

@ServeurpersoCom @allozaur PTAL

[ServeurpersoCom]

OK, it's single-model mode, not router mode. I'm going to fix it.

[ServeurpersoCom]

Not reproduced. Even in single-model mode...

[ServeurpersoCom]

@zazarpro which model are you running, and can you paste the full command line? Also which browser?

[ggerganov]

@ServeurpersoCom I can reproduce it with this command:

make -j && ./bin/llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF:Q4_0

[ServeurpersoCom]

0.8B! Even BF16 hits 550 tokens/s, and that literally DoS the UI. I'm going to test Q4 on the CPU.
(and the model often enters an infinite loop on RTX)

[zazarpro]

@ServeurpersoCom it happens with any model, however the one I used when creating this issue was Gemma4-E2B. I am using Firefox 152 and here is the command line, if still needed:

$ ./llama-server -m /mnt/L/Gemma4-U2B.gguf -t 6 --host 127.0.0.1 --port 2283 --no-mmap -c 65536 --min-p 0.0 --fit on
0.00.029.335 I log_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.029.341 I device_info:
0.00.029.548 I   - Vulkan0 : AMD Radeon RX 6500 XT (RADV NAVI24) (4080 MiB, 3467 MiB free)
0.00.029.555 I   - CPU     : 12th Gen Intel(R) Core(TM) i5-12400F (31915 MiB, 31915 MiB free)
0.00.029.606 I system_info: n_threads = 6 (n_threads_batch = 6) / 12 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 
0.00.029.611 I srv  llama_server: n_parallel is set to auto, using n_parallel = 4 and kv_unified = true
0.00.029.652 I srv          init: running without SSL
0.00.029.725 I srv          init: using 11 threads for HTTP server
0.00.030.433 I srv         start: binding port with default address family
0.00.031.602 I srv  llama_server: loading model
0.00.031.666 I srv    load_model: loading model '/mnt/L/Gemma4-U2B.gguf'
0.00.031.680 I common_init_result: fitting params to device memory ...
0.00.031.681 I common_init_result: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.01.232.326 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.232.742 W load: control-looking token:     50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.238.138 W load: control-looking token:      1 '<eos>' was not control-type; this is probably a bug in the model. its type will be overridden
0.01.260.684 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
0.02.125.734 W llama_context: n_ctx_seq (65536) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
0.02.152.323 I common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.03.176.317 I srv    load_model: initializing slots, n_slots = 4
0.03.203.509 W common_speculative_init: no implementations specified for speculative decoding
0.03.203.514 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 65536
0.03.203.521 I slot   load_model: id  1 | task -1 | new slot, n_ctx = 65536
0.03.203.521 I slot   load_model: id  2 | task -1 | new slot, n_ctx = 65536
0.03.203.521 I slot   load_model: id  3 | task -1 | new slot, n_ctx = 65536
0.03.203.614 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.03.203.615 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.03.203.616 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.03.203.616 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.03.203.632 I srv          init: idle slots will be saved to prompt cache and cleared upon starting a new task
0.03.215.160 I init: chat template, example_format: '<|turn>system
<|think|>
You are a helpful assistant<turn|>
<|turn>user
Hello<turn|>
<|turn>model
Hi there<turn|>
<|turn>user
How are you?<turn|>
<|turn>model
'
0.03.216.322 I srv          init: init: chat template, thinking = 1
0.03.216.354 I srv  llama_server: model loaded
0.03.216.359 I srv  llama_server: server is listening on http://127.0.0.1:2283
0.03.216.364 I srv  update_slots: all slots are idle
0.15.062.819 I srv    operator(): Chat format: peg-gemma4
0.15.063.096 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.15.063.097 I srv  get_availabl: updating prompt cache
0.15.063.101 I srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
0.15.063.105 I srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 65536 tokens, 8589934592 est)
0.15.063.106 I srv  get_availabl: prompt cache update took 0.01 ms
0.15.063.159 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
0.15.063.161 I slot process_sing: id  0 | task -1 | saving idle slot to prompt cache
0.15.063.161 I slot prompt_clear: id  0 | task -1 | clearing prompt with 0 tokens
0.15.063.199 I slot process_sing: id  1 | task -1 | saving idle slot to prompt cache
0.15.063.200 I slot prompt_clear: id  1 | task -1 | clearing prompt with 0 tokens
0.15.063.226 I slot process_sing: id  2 | task -1 | saving idle slot to prompt cache
0.15.063.227 I slot prompt_clear: id  2 | task -1 | clearing prompt with 0 tokens
0.15.063.385 I srv  stream_sessi: stream_session_attach_pipe: conv_id=f0cb4a6e-c736-4471-ab68-3285fa5885f5 (empty=0)
0.16.146.359 I slot create_check: id  3 | task 0 | created context checkpoint 1 of 32 (pos_min = 0, pos_max = 7, n_tokens = 8, size = 0.094 MiB)
0.17.660.478 I reasoning-budget: activated, budget=2147483647 tokens
# i pressed pause in the ui around here
0.18.373.598 I srv         close: stream_pipe close: draining conv=f0cb4a6e-c736-4471-ab68-3285fa5885f5
0.20.419.302 I slot print_timing: id  3 | task 0 | n_decoded =    181, tg =  60.25 t/s, tg_3s =  60.25 t/s
0.20.956.320 I reasoning-budget: deactivated (natural end)
0.21.104.525 I slot print_timing: id  3 | task 0 | prompt eval time =    2352.01 ms /    18 tokens (  130.67 ms per token,     7.65 tokens per second)
0.21.104.527 I slot print_timing: id  3 | task 0 |        eval time =    3689.24 ms /   226 tokens (   16.32 ms per token,    61.26 tokens per second)
0.21.104.528 I slot print_timing: id  3 | task 0 |       total time =    6041.25 ms /   244 tokens
0.21.104.531 I slot print_timing: id  3 | task 0 |    graphs reused =        225
0.21.104.572 I slot      release: id  3 | task 0 | stop processing: n_tokens = 243, truncated = 0
0.21.104.578 I srv  update_slots: all slots are idle
0.21.104.650 I srv         close: stream_pipe close: drain ended conv=f0cb4a6e-c736-4471-ab68-3285fa5885f5 bytes=93197

[ServeurpersoCom]

At my level I see 2 issues:

• One of ergonomics, if we don't enable the "continue" option then hitting stop doesn't really make sense anymore, it implies having to resend a message to the model with an inconsistent context.
• Also, in single model, the function to exit the thinking block doesn't work (another real bug).

single-model.mp4

[ServeurpersoCom]

And now that I'm running on CPU, I can see that after the stop, the process keeps doing inference and stops with a delay. So there's indeed a real problem, there is a sequencing bug: A "stop and continue" action within the reasoning prevents the "skip reasoning" function from working correctly afterwards.
I now have what I need to work with.