Skip to content

server: add "schema" and validation#24150

Merged
ngxson merged 5 commits into
masterfrom
xsn/server_schema
Jun 18, 2026
Merged

server: add "schema" and validation#24150
ngxson merged 5 commits into
masterfrom
xsn/server_schema

Conversation

@ngxson

@ngxson ngxson commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Overview

Add the notion of server_schema that will define the input schema in a more systematic way.

This is actually something I wanted to do since a long time ago. Motivations for this proposal:

  1. To address bugs like GHSA-8947-pfff-2f3c due to lack of input validation
  2. Generalize the validation logic
  3. Inline documentation, can be exported to markdown in the future

TODO in follow-up PRs:

  • Migrate other schema (for example, chat completion) to using this system
  • Export schema to markdown
  • Allow linking arg.cpp <--> server-schema.cpp via an enum? (not sure how it will be useful)

Example of the code:

// old way
params.sampling.top_k              = json_value(data, "top_k",               defaults.sampling.top_k);

// new way
add((new field_num("top_k", params.sampling.top_k))
    ->set_limits(0, INT32_MAX)
    ->set_desc("Limit the next token selection to the K most probable tokens (0 = disabled)"));

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: to spot and fix mistakes during the migration

@ngxson ngxson requested a review from a team as a code owner June 4, 2026 23:34
@ServeurpersoCom

ServeurpersoCom commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Testing this on my pod. It seems to me that the OAI spec can have sampling values less than 0 even if it is rarely useful in practice, such as encouraging repetitions with frequency_penalty. ( https://developers.openai.com/api/reference/python/resources/completions/methods/create search for "-2")

Comment thread tools/server/server-schema.cpp
Comment thread tools/server/server-schema.h Outdated
@ngxson

ngxson commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator Author

@ServeurpersoCom can you have a look? let's try to merge it today ~ tmr to avoid conflicts

@ServeurpersoCom

Copy link
Copy Markdown
Contributor

I'm merging on my server and doing a quick non-reg :)

@ServeurpersoCom ServeurpersoCom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested

@ngxson ngxson merged commit e1efd09 into master Jun 18, 2026
25 of 29 checks passed
@ngxson ngxson deleted the xsn/server_schema branch June 20, 2026 08:23
papamoose pushed a commit to papamoose/llama.cpp that referenced this pull request Jun 27, 2026
* wip

* working

* correct some limits

* add field name to error message
adrianhoehne pushed a commit to adrianhoehne/llama.cpp that referenced this pull request Jul 5, 2026
* wip

* working

* correct some limits

* add field name to error message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants