Summary
SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker. We identified an issue where the HMAC secret key is stored in environment variables and disclosed via the DescribeTrainingJob API.
Impact
- Function and Payload Tampering: Attackers with DescribeTrainingJob permissions may extract HMAC secret keys and forge serialized function payloads stored in S3. These tampered payloads would be processed and executed without triggering integrity validation errors, enabling unintended code substitution.
- Arbitrary Code Execution in the Training Environment: An third party with both DescribeTrainingJob permissions and write access to the job's S3 output location can extract the HMAC key, craft inappropriate Python objects, and achieve remote code execution in the client's Python process when the victim retrieves remote function results.
- Data and Credentials Handling: Arbitrary remote code execution may interact with sensitive data, model artifacts, environment variables, and potentially AWS metadata.
- Cross-Tenant or Shared Environment Risks: In multi-tenant, shared S3 bucket, a disclosed HMAC key could act as a pivot point to perform inappropriate actions against other users' remote function workloads. This could leverage the IAM permissions, shared S3 buckets, or VPC resources to compromise adjacent services or data.
Impacted versions
- SageMaker Python SDK v3 < v3.2.0
- SageMaker Python SDK v2 < v2.256.0
Patches
This issue has been addressed in SageMaker Python SDK version v3.2.0 and v2.256.0. We recommend upgrading to the latest version immediately and ensuring any forked or derivative code is patched to incorporate the new fixes.
Workarounds
Customers using self-signed certificates for internal model downloads should add their private Certificate Authority (CA) certificate to the container image rather than relying on the SDK’s previous insecure configuration. This opt-in approach maintains security while accommodating internal trusted domains.
References
If you have any questions or comments about this advisory, we ask that you contact AWS Security via our vulnerability reporting page or directly via email to aws-security@amazon.com. Please do not create a public GitHub issue.
Summary
SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker. We identified an issue where the HMAC secret key is stored in environment variables and disclosed via the DescribeTrainingJob API.
Impact
Impacted versions
Patches
This issue has been addressed in SageMaker Python SDK version v3.2.0 and v2.256.0. We recommend upgrading to the latest version immediately and ensuring any forked or derivative code is patched to incorporate the new fixes.
Workarounds
Customers using self-signed certificates for internal model downloads should add their private Certificate Authority (CA) certificate to the container image rather than relying on the SDK’s previous insecure configuration. This opt-in approach maintains security while accommodating internal trusted domains.
References
If you have any questions or comments about this advisory, we ask that you contact AWS Security via our vulnerability reporting page or directly via email to aws-security@amazon.com. Please do not create a public GitHub issue.