Skip to content

Adding vendor RBLN(Rebellions) #3416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rebel-ysseo opened this issue May 8, 2025 · 0 comments
Open

Adding vendor RBLN(Rebellions) #3416

rebel-ysseo opened this issue May 8, 2025 · 0 comments

Comments

@rebel-ysseo
Copy link

TorchServe has a varying structure for different accelerator types through recently added #3371.

Although Rebellions provides a guide on how to utilize TorchServe with the RBLN(Rebellions) NPUs through its official document page(https://docs.rbln.ai/software/model_serving/torchserve/torchserve.html), the current implementation of TorchServe does not recognize the RBLN NPU as a valid accelerator vendor. As a result, even when gpu_id is set in configuration using the RBLN NPU, the specified RBLN NPUs cannot be properly utilized.

We would like to propose adding RBLN NPU as a recognized accelerator vendor in TorchServe, along with an official user guide. This addition will enable seamless integration and usage of TorchServe in environments equipped with RBLN NPUs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant