Skip to content

Port macOS tests from CircleCI got GHA #7175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/unittest.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although it seems weird to have a standalone script for a single workflow, this script works fine for Linux as well. See #7149. I'll migrate them to this script next.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #7189.


set -euo pipefail

echo '::group::Prepare conda'
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line as well as the accompanying echo '::endgroup::' is used to group the output of some steps in the logs. This is only needed now, since we no longer have individual steps, but rather a single step.

CONDA_PATH=$(which conda)
eval "$(${CONDA_PATH} shell.bash hook)"
# The `setuptools` package installed through `conda` includes a patch that errors if something is installed
# through `setuptools` while the `CONDA_BUILD` environment variable is set.
# https://github.com/AnacondaRecipes/setuptools-feedstock/blob/f5d8d256810ce28fc0cf34170bc34e06d3754041/recipe/patches/0002-disable-downloads-inside-conda-build.patch
# (Although we are not using the `-c conda-forge` channel, the patch is equivalent but not public for
# `setuptools` from the `-c defaults` channel)
# Since we aren't using `conda build` here, we unset it to avoid installation problems later
# TODO: investigate where `CONDA_BUILD` is set and maybe fix unset it there
unset CONDA_BUILD
echo '::endgroup::'

echo '::group::Set PyTorch conda channel'
# TODO: Can we maybe have this as environment variable in the job template? For example, `IS_RELEASE`.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: I need to open an issue about that.

if [[ (${GITHUB_EVENT_NAME} = 'pull_request' && (${GITHUB_BASE_REF} = 'release'*)) || (${GITHUB_REF} = 'refs/heads/release'*) ]]; then
POSTFIX=test
else
POSTFIX=nightly
fi
PYTORCH_CHANNEL=pytorch-"${POSTFIX}"
echo "${PYTORCH_CHANNEL}"
echo '::endgroup::'

echo '::group::Set PyTorch GPU mutex'
case $GPU_ARCH_TYPE in
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed for macOS since we don't have GPU workflows. This is in preparation for the Linux GPU jobs. Will happen a few more times below.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #7189.

cpu)
PYTORCH_MUTEX=cpuonly
;;
cuda)
PYTORCH_MUTEX="pytorch-cuda=${GPU_ARCH_VERSION}"
;;
*)
echo "Unknown GPU_ARCH_TYPE=${GPU_ARCH_TYPE}"
exit 1
;;
esac
echo "${PYTORCH_MUTEX}"
echo '::endgroup::'

echo '::group::Create build environment'
conda create \
--name ci \
--quiet --yes \
python="${PYTHON_VERSION}" pip \
setuptools ninja \
libpng jpeg \
Pillow numpy requests
conda activate ci
pip install 'av<10'
echo '::endgroup::'

echo '::group::Install PyTorch'
conda install \
--quiet --yes \
-c "${PYTORCH_CHANNEL}" \
-c nvidia \
pytorch \
"${PYTORCH_MUTEX}"

if [[ $GPU_ARCH_TYPE = 'cuda' ]]; then
python3 -c "import torch; exit(not torch.cuda.is_available())"
fi
echo '::endgroup::'

echo '::group::Install TorchVision'
python setup.py develop
echo '::endgroup::'

echo '::group::Collect PyTorch environment information'
python -m torch.utils.collect_env
echo '::endgroup::'

echo '::group::Install testing utilities'
pip install --progress-bar=off pytest pytest-mock pytest-cov
echo '::endgroup::'

echo '::group::Run tests'
pytest --durations=25
echo '::endgroup::'
50 changes: 0 additions & 50 deletions .github/workflows/test-m1.yml

This file was deleted.

34 changes: 34 additions & 0 deletions .github/workflows/test-macos.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Unit-tests on macOS

on:
pull_request:
push:
branches:
- nightly
- main
- release/*
workflow_dispatch:

jobs:
tests:
strategy:
matrix:
python-version:
- "3.8"
- "3.9"
- "3.10"
runner: ["macos-12"]
include:
- python-version: "3.8"
runner: macos-m1-12
fail-fast: false
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
with:
repository: pytorch/vision
timeout: 120
runner: ${{ matrix.runner }}
script: |
export PYTHON_VERSION=${{ matrix.python-version }}
export GPU_ARCH_TYPE=cpu

./.github/unittest.sh