Skip to content

Commit 35d0963

Browse files
authored
Expose all CPUs to TorchServe
Handle the error described [in sagemaker docs here](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model-troubleshoot.html)
1 parent 6610a41 commit 35d0963

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/sagemaker_pytorch_serving_container/torchserve.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ def _generate_ts_config_properties():
151151
env = environment.Environment()
152152

153153
user_defined_configuration = {
154-
"default_response_timeout": env.model_server_timeout,
154+
"vmargs":"-XX:-UseContainerSupport -XX:InitialRAMPercentage=8.0 -XX:MaxRAMPercentage=10.0 -XX:-UseLargePages -XX:+UseG1GC -XX:+ExitOnOutOfMemoryError", "default_response_timeout": env.model_server_timeout,
155155
"default_workers_per_model": env.model_server_workers,
156156
"inference_address": "http://0.0.0.0:{}".format(env.inference_http_port),
157157
"management_address": "http://0.0.0.0:{}".format(env.management_http_port),

0 commit comments

Comments
 (0)