Merge branch 'master' into inference-id

Author: ahsan-z-khan

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## v2.23.6 (2021-01-20)
+
+### Bug Fixes and Other Changes
+
+ * add artifact, action, context to virsualizer
+
 ## v2.23.5 (2021-01-18)
 
 ### Bug Fixes and Other Changes

VERSION

@@ -1 +1 @@
-2.23.6.dev0
+2.23.7.dev0

doc/api/training/smd_data_parallel_pytorch.rst renamed to doc/api/training/sdp_versions/smd_data_parallel_pytorch.rst

@@ -1,6 +1,6 @@
-###################################
+##########################
 Distributed data parallel
-###################################
+##########################
 
 SageMaker's distributed data parallel library extends SageMaker’s training
 capabilities on deep learning models with near-linear scaling efficiency,
@@ -68,5 +68,5 @@ model.
 .. toctree::
    :maxdepth: 2
 
-   smd_data_parallel_pytorch
-   smd_data_parallel_tensorflow
+   sdp_versions/smd_data_parallel_pytorch
+   sdp_versions/smd_data_parallel_tensorflow

doc/api/training/smd_data_parallel_tensorflow.rst renamed to doc/api/training/sdp_versions/smd_data_parallel_tensorflow.rst

@@ -20,20 +20,31 @@ Use the following sections to learn more about the model parallelism and the lib
    <https://integ-docs-aws.amazon.com/sagemaker/latest/dg/model-parallel-use-api.html#model-parallel-customize-container>`__
    for more information.
 
-How to Use this Guide
-=====================
+Use with the SageMaker Python SDK
+=================================
+
+Use the following page to learn how to configure and enable distributed model parallel
+when you configure an Amazon SageMaker Python SDK `Estimator`.
+
+.. toctree::
+   :maxdepth: 1
+
+   smd_model_parallel_general
+
+API Documentation
+=================
 
 The library contains a Common API that is shared across frameworks, as well as APIs
-that are specific to supported frameworks, TensorFlow and PyTorch. To use the library, reference the
+that are specific to supported frameworks, TensorFlow and PyTorch.
+
+Select a version to see the API documentation for version. To use the library, reference the
 **Common API** documentation alongside the framework specific API documentation.
 
 .. toctree::
    :maxdepth: 1
 
-   smd_model_parallel_general
-   smd_model_parallel_common_api
-   smd_model_parallel_pytorch
-   smd_model_parallel_tensorflow
+   smp_versions/v1_2_0.rst
+   smp_versions/v1_1_0.rst
 
 It is recommended to use this documentation alongside `SageMaker Distributed Model Parallel
 <http://docs.aws.amazon.com/sagemaker/latest/dg/model-parallel.html>`__ in the Amazon SageMaker
@@ -49,11 +60,11 @@ developer guide. This developer guide documentation includes:
    -  `Configuration tips and pitfalls
       <https://docs.aws.amazon.com/sagemaker/latest/dg/model-parallel-customize-tips-pitfalls.html>`__
 
-Latest Updates
-==============
 
-New features, bug fixes, and improvements are regularly made to the SageMaker distributed model parallel library.
+Release Notes
+=============
 
+New features, bug fixes, and improvements are regularly made to the SageMaker distributed model parallel library.
 To see the the latest changes made to the library, refer to the library
 `Release Notes
 <https://github.com/aws/sagemaker-python-sdk/blob/master/doc/api/training/smd_model_parallel_release_notes/>`_.

doc/api/training/smd_data_parallel.rst

@@ -6,7 +6,7 @@
 .. _sm-sdk-modelparallel-params:
 
 SageMaker Python SDK ``modelparallel`` parameters
--------------------------------------------------
+=================================================
 
 The TensorFlow and PyTorch ``Estimator`` objects contains a ``distribution`` parameter,
 which is used to enable and specify parameters for the
@@ -306,7 +306,7 @@ table are optional.
 .. _ranking-basics:
 
 Ranking Basics
---------------
+==============
 
 The library maintains a one-to-one mapping between processes and available GPUs:
 for each GPU, there is a corresponding CPU process. Each CPU process

doc/api/training/smd_model_parallel.rst

@@ -1,3 +1,44 @@
+# Sagemaker Distributed Model Parallel 1.2.0 Release Notes
+
+- New Features
+- Bug Fixes
+- Known Issues
+
+## New Features
+
+### PyTorch
+
+#### Add support for PyTorch 1.7
+
+- Adds support for `gradient_as_bucket_view` (PyTorch 1.7 only), `find_unused_parameters` (PyTorch 1.7 only) and `broadcast_buffers` options to `smp.DistributedModel`. These options behave the same as the corresponding options (with the same names) in
+`torch.DistributedDataParallel` API. Please refer to the [SageMaker distributed model parallel API documentation](https://sagemaker.readthedocs.io/en/stable/api/training/smd_model_parallel_pytorch.html#smp.DistributedModel) for more information.
+
+- Adds support for `join` (PyTorch 1.7 only) context manager, which is to be used in conjunction with an instance of `smp.DistributedModel` to be able to train with uneven inputs across participating processes.
+
+- Adds support for `_register_comm_hook` (PyTorch 1.7 only) which will register the callable as a communication hook for DDP. NOTE: Like in DDP, this is an experimental API and subject to change.
+
+### Tensorflow
+
+- Adds support for Tensorflow 2.4
+
+## Bug Fixes
+
+### PyTorch
+
+- `Serialization`: Fix a bug with serialization/flattening where instances of subclasses of dict/OrderedDicts were serialized/deserialized or internally flattened/unflattened as
+regular dicts.
+
+### Tensorflow
+
+- Fix a bug that may cause a hang during evaluation when there is no model input for one partition.
+
+## Known Issues
+
+### PyTorch
+
+- A performance regression was observed when training on SMP with PyTorch 1.7.1 compared to 1.6. The rootcause was found to be the slowdown in performance of `.grad` method calls in PyTorch 1.7.1 compared to 1.6. Please see the related discussion: https://github.com/pytorch/pytorch/issues/50636.
+
+
 # Sagemaker Distributed Model Parallel 1.1.0 Release Notes
 
 - New Features