Skip to content

[Story]: Extend Muxing to Differentiate Between Chat and FIM Requests #1059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
aponcedeleonch opened this issue Feb 14, 2025 · 1 comment
Closed

[Story]: Extend Muxing to Differentiate Between Chat and FIM Requests #1059

aponcedeleonch opened this issue Feb 14, 2025 · 1 comment
Assignees
@peppescg @aponcedeleonch
Labels
feature-spotlight needs-docs

Comments

@aponcedeleonch
Copy link
Contributor

Description

We currently support routing logic based on filenames, and now we need to extend that functionality to differentiate between chat requests and Fill-In-Middle (FIM) requests.

Example

  • A chat that references app.py → Model 1
  • A FIM request within app.py → Model 2

Implementation Outline

  • Extend the existing muxing rules to include a check for request type (chat or FIM).
  • Allow users to configure which model should handle chat requests vs. FIM for specific filenames.

Additional Context

No response
@aponcedeleonch aponcedeleonch changed the title [Story]: Extend muxing for support FIM and Chat requests [Story]: Extend Muxing to Differentiate Between Chat and FIM Requests Feb 14, 2025
@aponcedeleonch aponcedeleonch mentioned this issue Feb 17, 2025
Closed
@aponcedeleonch
Copy link
Contributor Author

Closed by the tickets mentioned above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@peppescg peppescg

@aponcedeleonch aponcedeleonch

Labels
feature-spotlight needs-docs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dussab @peppescg @danbarr @lukehinds @aponcedeleonch