Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Update llama.cpp related code to use openai format #107

Merged
merged 1 commit into from
Nov 27, 2024

Conversation

ptelang
Copy link
Contributor

@ptelang ptelang commented Nov 27, 2024

This PR modifies the llama.cpp related implementation to use openai format request/response.

It assumes that Continue plugin is configured as follows.

"models": [
...
  {
      "title": "Llama CPP",
      "provider": "openai",
      "model": "qwen2.5-coder-1.5b-instruct-q5_k_m",
      "apiBase": "http://localhost:8989/llamacpp"
  },
...
],
...
"tabAutocompleteModel": {
    "title": "Llama CPP",
    "provider": "openai",
    "model": "qwen2.5-coder-1.5b-instruct-q5_k_m",
    "apiBase": "http://localhost:8989/llamacpp",
},
...

@ptelang ptelang force-pushed the llama-cpp-openai-format branch from f571a42 to da54d16 Compare November 27, 2024 20:08
@ptelang ptelang force-pushed the llama-cpp-openai-format branch from da54d16 to bbb5e94 Compare November 27, 2024 20:14
Copy link
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good and works fine. Thank you!

@ptelang ptelang merged commit 94308e5 into main Nov 27, 2024
2 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants