Add the endpoints /api/tags and /api/chat #13659
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add the endpoints /api/tags and /api/chat, and improved the model metadata response
ggerganov
reviewed
May 21, 2025
ngxson
reviewed
May 21, 2025
tools/server/server.cpp
Outdated
| @@ -4086,6 +4101,19 @@ int main(int argc, char ** argv) { | |||
| { "llama.context_length", ctx_server.slots.back().n_ctx, }, | |||
| } | |||
| }, | |||
| {"modelfile", ""}, // Specific to ollama and does not seem to be needed | |||
| {"parameters", ""}, // TODO: add parameters | |||
There was a problem hiding this comment.
Does the API client expect this to be a string or an object?
There was a problem hiding this comment.
modelfile is a string. Parameters i am not sure, but it seems to be enough as is.
|
I think this is the minimal code that we need for copilot to work with it. |
ngxson
reviewed
May 21, 2025
| @@ -3740,7 +3741,7 @@ int main(int argc, char ** argv) { | |||
| if (req.path == "/" || tmp.back() == "html") { | |||
| res.set_content(reinterpret_cast<const char*>(loading_html), loading_html_len, "text/html; charset=utf-8"); | |||
| res.status = 503; | |||
| } else if (req.path == "/models" || req.path == "/v1/models") { | |||
| } else if (req.path == "/models" || req.path == "/v1/models" || req.path == "/api/tags") { | |||
| // allow the models endpoint to be accessed during loading | |||
There was a problem hiding this comment.
during loading, common_chat_templates_source call will fail
There was a problem hiding this comment.
Do you want me to remove that endpoint from the if-case?
Edit:
The current code does not use common_chat_templates_source in that endpoint after the latest changes.
There was a problem hiding this comment.
hmm I think it's ok to keep this then
ngxson
approved these changes
May 21, 2025
infil00p
pushed a commit
to baseweight/llama.cpp
that referenced
this pull request
May 22, 2025
* Add the endpoints /api/tags and /api/chat Add the endpoints /api/tags and /api/chat, and improved the model metadata response * Remove trailing whitespaces * Removed code that is not needed for copilot to work.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the endpoints /api/tags and /api/chat, and improved the model metadata response.
These changes made llama-server work with Copilot Chat again for me. both /api/tags and /api/chat are added because of a discussion on open-webui's page where they point out that both need to be handled. The old json values are kept as is to keep backwards compatibility, in case anyone uses them.
This issue was also pointed out in the comments of #12896.