You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TorchServe has a varying structure for different accelerator types through recently added #3371.
Although Rebellions provides a guide on how to utilize TorchServe with the RBLN(Rebellions) NPUs through its official document page(https://docs.rbln.ai/software/model_serving/torchserve/torchserve.html), the current implementation of TorchServe does not recognize the RBLN NPU as a valid accelerator vendor. As a result, even when gpu_id is set in configuration using the RBLN NPU, the specified RBLN NPUs cannot be properly utilized.
We would like to propose adding RBLN NPU as a recognized accelerator vendor in TorchServe, along with an official user guide. This addition will enable seamless integration and usage of TorchServe in environments equipped with RBLN NPUs.
The text was updated successfully, but these errors were encountered:
TorchServe has a varying structure for different accelerator types through recently added #3371.
Although Rebellions provides a guide on how to utilize
TorchServe with the RBLN(Rebellions) NPUs
through its official document page(https://docs.rbln.ai/software/model_serving/torchserve/torchserve.html), the current implementation of TorchServe does not recognize the RBLN NPU as a valid accelerator vendor. As a result, even whengpu_id
is set in configuration using theRBLN NPU
, the specified RBLN NPUs cannot be properly utilized.We would like to propose adding RBLN NPU as a recognized accelerator vendor in TorchServe, along with an official user guide. This addition will enable seamless integration and usage of TorchServe in environments equipped with RBLN NPUs.
The text was updated successfully, but these errors were encountered: