Misc. bug: Repeated command line options are discarded in router mode (--override-kv is broken)

[personalmountains]

Name and Version

master

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server --models-dir "c:/models" --override-kv "tokenizer.ggml.add_bos_token=bool:false" --override-kv "tokenizer.ggml.add_eos_token=bool:false"

Problem description & steps to reproduce

common_params_to_map() (from f32ca51) uses a std::map, which only keeps the last command line option if multiple options with the same name are specified. Running llama-server with multiple --override-kv only keeps the last one.

First Bad Commit

f32ca51

Relevant log output

llama-server --models-dir "c:/models" --override-kv "tokenizer.ggml.add_bos_token=bool:false" --override-kv "tokenizer.ggml.add_eos_token=bool:false"

[...]

load: spawning server instance with name=(MODEL) on port 59446
load: spawning server instance with args:
load:   llama-server.exe
load:   --host
load:   127.0.0.1
load:   --override-kv
load:   tokenizer.ggml.add_eos_token=bool:false
load:   --port
load:   59446
load:   --alias
load:   (MODEL)
load:   --model
load:   (MODEL PATH)

[...]

validate_override: Using metadata override ( bool) 'tokenizer.ggml.add_eos_token' = false

[ngxson]

we don't plan to support repeated args, the same way duplicated environment variables aren't accepted by most programs

however, in the future, we can add a special "separator", for example using comma character , to separate between values. but IMO it's currently not our priority to support this

[personalmountains]

I'm not sure I understand. This used to work fine up until a few days ago. I'm not talking about environment variables, these are command line options, and plenty of programs accept the same option multiple times. The documentation for llama-server also says "may be specified multiple times" for --override-kv. There's even a few "TODO: support repeated args in the future" in the code that were added in this commit.

To clarify, this is a regression.

As far as I can tell, it's now impossible to override more than one metadata key from the command line anymore. Is there any workaround?

[pwilkin]

@ngxson The problem is, people need to be able to specify multiple keys for --override-kv. For example, -ot allows this with commas, so it's not a problem. So I think the least intrusive thing would be to add support for a similar separator here and maybe add a note in the docs that multiple calls are now obsolete.

[ServeurpersoCom]

I'll refactor --override-kv to work exactly like --override-tensor (using comma-separated values) and update the documentation accordingly. This will resolve the regression while maintaining consistency with existing patterns in the codebase

[ServeurpersoCom]

After analyzing the codebase, here are all arguments that currently document support for multiple invocations:

• --in-file: "repeat to specify multiple files"
• --image / --audio: "can be repeated if you have multiple files"
• --lora: "can be repeated to use multiple adapters"
• --lora-scaled: "can be repeated to use multiple adapters"
• --control-vector: "can be repeated to add multiple control vectors"
• --control-vector-scaled: "can be repeated to add multiple scaled control vectors"
• --context-file: "repeat to specify multiple files"
• --override-kv: "may be specified multiple times"

Note that --override-tensor (-ot) already uses comma-separated values and doesn't have this issue.