diff --git a/README.md b/README.md index c1de4d752..f6052220a 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Model Zoo for Intel® Architecture -This repository contains **links to pre-trained models, sample scripts, best practices, and step-by-step tutorials** for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors. +This repository contains **links to pre-trained models, sample scripts, best practices, and step-by-step tutorials** for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors and Intel® Data Center GPUs. Model packages and containers for running the Model Zoo's workloads can be found at the [Intel® Developer Catalog](https://software.intel.com/containers). diff --git a/docs/general/FLEX_DEVCATALOG.md b/docs/general/FLEX_DEVCATALOG.md new file mode 100644 index 000000000..8ba23ef91 --- /dev/null +++ b/docs/general/FLEX_DEVCATALOG.md @@ -0,0 +1,23 @@ +# Model Zoo for Intel® Architecture Workloads Optimized for the Intel® Data Center GPU Flex Series + +This document provides links to step-by-step instructions on how to leverage Model Zoo docker containers to run optimized open-source Deep Learning inference workloads using Intel® Extension for PyTorch* and Intel® Extension for TensorFlow* on the [Intel® Data Center GPU Flex Series](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/data-center-gpu/flex-series/overview.html). + +## Base Containers + +| AI Framework | Extension | Documentation | +| -----------------------------| ------------- | ----------------- | +| PyTorch | Intel® Extension for PyTorch* | [Intel® Extension for PyTorch Container](https://github.com/IntelAI/models/blob/master/quickstart/ipex-tool-container/gpu/devcatalog.md) | +| TensorFlow | Intel® Extension for TensorFlow* | [Intel® Extension for TensorFlow Container](https://github.com/IntelAI/models/blob/master/quickstart/tf-tool-container/gpu/devcatalog.md)| + +## Optimized Workloads + +The table below provides links to run each workload in a docker container. The containers are optimized for Linux*. + + +| Model | Framework | Mode | Documentation | Dataset | +| ----------------------------| ---------- | ----------| ------------------- | ------------ | +| [ResNet 50 v1.5](https://github.com/tensorflow/models/tree/v2.11.0/official/legacy/image_classification/resnet) | TensorFlow | Inference| [INT8](https://github.com/IntelAI/models/blob/master/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md) | [ImageNet 2012](https://github.com/IntelAI/models/tree/master/datasets/imagenet/README.md) | +| [ResNet 50 v1.5](https://arxiv.org/pdf/1512.03385.pdf) | PyTorch | Inference | [INT8 ](https://github.com/IntelAI/models/blob/master/quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/devcatalog.md) | [ImageNet 2012](https://github.com/IntelAI/models/tree/master/datasets/imagenet/README.md) | +| [SSD-MobileNet v1](https://arxiv.org/pdf/1704.04861.pdf) | PyTorch | Inference | [INT8](https://github.com/IntelAI/models/blob/master/quickstart/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/devcatalog.md) | [COCO 2017](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/README.md#datasets) | +| [YOLO v4](https://arxiv.org/pdf/1704.04861.pdf) | PyTorch | Inference |[INT8](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/yolov4/inference/gpu/devcatalog.md) | [COCO 2017](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/README.md#datasets) | +| [SSD-MobileNet](https://arxiv.org/pdf/1704.04861.pdf) | TensorFlow | Inference | [INT8](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/tensorflow/ssd-mobilenet/inference/gpu/devcatalog.md)| [COCO 2017 validation dataset](https://github.com/IntelAI/models/tree/master/datasets/coco#download-and-preprocess-the-coco-validation-images) | diff --git a/quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/DEVCATALOG_FLEX.md b/quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/DEVCATALOG_FLEX.md new file mode 100644 index 000000000..c54eb5c09 --- /dev/null +++ b/quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/DEVCATALOG_FLEX.md @@ -0,0 +1,102 @@ +# Running ResNet50 v1.5 Inference with Int8 on Intel® Data Center GPU Flex Series using Intel® Extension for PyTorch* + + +## Overview + +This document has instructions for running ResNet50v1.5 inference using Intel(R) Extension for PyTorch with GPU. + +## Requirements +| Item | Detail | +| ------ | ------- | +| Host machine | Intel® Data Center GPU Flex Series | +| Drivers | GPU-compatible drivers need to be installed: [Download Driver 476.14](https://dgpu-docs.intel.com/releases/stable_476_14_20221021.html) +| Software | Docker* Installed | + +## Get Started + +## Download Datasets + +The [ImageNet](http://www.image-net.org/) validation dataset is used. + +Download and extract the ImageNet2012 dataset from http://www.image-net.org/, +then move validation images to labeled subfolders, using +[the valprep.sh shell script](https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh) + +A after running the data prep script, your folder structure should look something like this: + +``` +imagenet +└── val + ├── ILSVRC2012_img_val.tar + ├── n01440764 + │ ├── ILSVRC2012_val_00000293.JPEG + │ ├── ILSVRC2012_val_00002138.JPEG + │ ├── ILSVRC2012_val_00003014.JPEG + │ ├── ILSVRC2012_val_00006697.JPEG + │ └── ... + └── ... +``` +The folder that contains the `val` directory should be set as the +`DATASET_DIR` +(for example: `export DATASET_DIR=/home//imagenet`). + +## Quick Start Scripts + +| Script name | Description | +|-------------|-------------| +| `inference_block_format.sh` | Runs ResNet50 inference (block format) for the specified precision (int8) | + +## Run Using Docker + +### Set up Docker Image + +``` +docker pull intel/image-recognition:pytorch-flex-gpu-resnet50v1-5-inference +``` +### Run Docker Image +The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run the `inference_block_format.sh` quickstart script using this container, you'll need to provide volume mounts for the ImageNet dataset. You will need to provide an output directory where log files will be written. + +``` +export PRECISION=int8 +export OUTPUT_DIR= +export DATASET_DIR= +export SCRIPT=quickstart/inference_block_format.sh + +DOCKER_ARGS=${DOCKER_ARGS:---rm -it} +IMAGE_NAME=intel/image-recognition:pytorch-flex-gpu-resnet50v1-5-inference + + +VIDEO=$(getent group video | sed -E 's,^video:[^:]*:([^:]*):.*$,\1,') +RENDER=$(getent group render | sed -E 's,^render:[^:]*:([^:]*):.*$,\1,') + +test -z "$RENDER" || RENDER_GROUP="--group-add ${RENDER}" + +docker run \ + -v :/workspace \ + --group-add ${VIDEO} \ + ${RENDER_GROUP} \ + --device=/dev/dri \ + --ipc=host \ + --env PRECISION=${PRECISION} \ + --env OUTPUT_DIR=${OUTPUT_DIR} \ + --env DATASET_DIR=${DATASET_DIR} \ + --env http_proxy=${http_proxy} \ + --env https_proxy=${https_proxy} \ + --env no_proxy=${no_proxy} \ + --volume ${OUTPUT_DIR}:${OUTPUT_DIR} \ + --volume ${DATASET_DIR}:${DATASET_DIR} \ + ${DOCKER_ARGS} \ + ${IMAGE_NAME} \ + /bin/bash $SCRIPT + ``` + +## Documentation and Sources + +[GitHub* Repository](https://github.com/IntelAI/models/tree/master/dockerfiles/model_containers) + +## Support +Support for Intel® Extension for PyTorch* is found via the [Intel® AI Analytics Toolkit.](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.qbretz) Additionally, the Intel® Extension for PyTorch* team tracks both bugs and enhancement requests using [GitHub issues](https://github.com/intel/intel-extension-for-pytorch/issues). Before submitting a suggestion or bug report, please search the GitHub issues to see if your issue has already been reported. + +## License Agreement + +LEGAL NOTICE: By accessing, downloading or using this software and any required dependent software (the “Software Package”), you agree to the terms and conditions of the software license agreements for the Software Package, which may also include notices, disclaimers, or license terms for third party software included with the Software Package. Please refer to the [license file](https://github.com/IntelAI/models/tree/master/third_party) for additional details. diff --git a/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md b/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md index b22fef32d..2026ad446 100644 --- a/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md +++ b/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md @@ -1,11 +1,20 @@ -# ResNet50 v1.5 Inference +# Running ResNet50 v1.5 Inference with Int8 on Intel® Data Center GPU Flex Series using Intel® Extension for TensorFlow* -## Description +## Overview -This document has instructions for running ResNet50 v1.5 inference using -Intel(R) Extension for TensorFlow* with Intel(R) Data Center GPU Flex Series. +This document has instructions for running ResNet50 v1.5 inference using Intel(R) Extension for TensorFlow* with Intel(R) Data Center GPU Flex Series. -## Datasets + +## Requirements +| Item | Detail | +| ------ | ------- | +| Host machine | Intel® Data Center GPU Flex Series | +| Drivers | GPU-compatible drivers need to be installed: [Download Driver 476.14](https://dgpu-docs.intel.com/releases/stable_476_14_20221021.html) +| Software | Docker* Installed | + +## Get Started + +### Download Datasets Download and preprocess the ImageNet dataset using the [instructions here](https://github.com/IntelAI/models/blob/master/datasets/imagenet/README.md). After running the conversion script you should have a directory with the @@ -13,44 +22,44 @@ ImageNet dataset in the TF records format. Set the `DATASET_DIR` to point to the TF records directory when running ResNet50 v1.5. -## Quick Start Scripts +### Quick Start Scripts | Script name | Description | |:-------------:|:-------------:| -| `online_inference` | Runs online inference for int8 precision | +| `online_inference` | Runs online inference for int8 precision | | `batch_inference` | Runs batch inference for int8 precision | | `accuracy` | Measures the model accuracy for int8 precision | -## Docker -Requirements: -* Host machine has Intel(R) Data Center GPU Flex Series -* Follow instructions to install GPU-compatible driver [419.40](https://dgpu-docs.intel.com/releases/stable_419_40_20220914.html) -* Docker +## Run Using Docker -### Docker pull command: +### Set up Docker Image ``` -docker pull intel/image-recognition:tf-atsm-gpu-resnet50v1-5-inference +docker pull intel/image-recognition:tf-flex-gpu-resnet50v1-5-inference ``` -The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run one of the inference quickstart scripts using this container, you'll need to provide volume mounts for the ImageNet dataset for running `accuracy.sh` script. For `online_inference.sh` and `batch_inference.sh` dummy dataset will be used. You will need to provide an output directory where log files will be written. +### Run Docker Image +The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run one of the inference quickstart scripts using this container, you'll need to provide volume mounts for the ImageNet dataset for running `accuracy.sh` script. For `online_inference.sh` and `batch_inference.sh` dummy dataset will be used. You will need to provide an output directory where log files will be written. ``` export PRECISION=int8 export OUTPUT_DIR= export DATASET_DIR= -IMAGE_NAME=intel/image-recognition:tf-atsm-gpu-resnet50v1-5-inference +DOCKER_ARGS=${DOCKER_ARGS:---rm -it} +IMAGE_NAME=intel/image-recognition:tf-flex-gpu-resnet50v1-5-inference VIDEO=$(getent group video | sed -E 's,^video:[^:]*:([^:]*):.*$,\1,') RENDER=$(getent group render | sed -E 's,^render:[^:]*:([^:]*):.*$,\1,') +test -z "$RENDER" || RENDER_GROUP="--group-add ${RENDER}" + docker run \ + -v :/workspace \ --group-add ${VIDEO} \ ${RENDER_GROUP} \ --device=/dev/dri \ --ipc=host \ - --privileged \ --env PRECISION=${PRECISION} \ --env OUTPUT_DIR=${OUTPUT_DIR} \ --env DATASET_DIR=${DATASET_DIR} \ @@ -59,16 +68,22 @@ docker run \ --env no_proxy=${no_proxy} \ --volume ${OUTPUT_DIR}:${OUTPUT_DIR} \ --volume ${DATASET_DIR}:${DATASET_DIR} \ - --rm -it \ - $IMAGE_NAME \ + ${DOCKER_ARGS} \ + ${IMAGE_NAME} \ /bin/bash quickstart/