Skip to content

Support TGI as another alternative backend #72

@kerthcet

Description

@kerthcet

What would you like to be added:

TGI is also a popular inference backend that we should support. More importantly, we should set this as an example once people would like to support another backend in the future.

Why is this needed:

Adopte more backends and make this a reference.

Completion requirements:

This enhancement requires the following artifacts:

  • Design doc
  • API change
  • Docs update

The artifacts should be linked in subsequent comments.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureCategorizes issue or PR as related to a new feature.help wantedExtra attention is neededneeds-priorityIndicates a PR lacks a label and requires one.needs-triageIndicates an issue or PR lacks a label and requires one.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions