-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Skip gpu tests in regions without gpu #461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## master #461 +/- ##
=======================================
Coverage 93.87% 93.87%
=======================================
Files 56 56
Lines 4199 4199
=======================================
Hits 3942 3942
Misses 257 257 Continue to review full report at Codecov.
|
tests/integ/test_tf_cifar.py
Outdated
@@ -35,6 +35,7 @@ def __call__(self, data): | |||
|
|||
@pytest.mark.continuous_testing | |||
@pytest.mark.skipif(PYTHON_VERSION != 'py2', reason="TensorFlow image supports only python 2.") | |||
@pytest.mark.skipif(REGION in ['us-west-1', 'eu-west-2', 'ca-central-1']) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add a reason. same below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other integ tests also run gpu tests (e.g. pytorch)
@nadiaya Only need to skip the tests that marked as continuous_testing (run as canary). For other p2 tests like pytorch async, it's not part of canary. But I guess I can still make the change though. |
@@ -35,6 +35,8 @@ def test_distributed_cpu_training(sagemaker_session, chainer_full_version): | |||
_run_mnist_training_job(sagemaker_session, "ml.c4.xlarge", 2, chainer_full_version) | |||
|
|||
|
|||
@pytest.mark.skipif(REGION in ['us-west-1', 'eu-west-2', 'ca-central-1'], | |||
reason='No ml.p2.xlarge supported in these regions') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about p3? can't we use p3 then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3 is either not supported or supported only by 1/3 of availability zones. So for these 3 regions we just skip tests for now to make canary good for now.
Issue #, if available:
Description of changes:
In us-west-1, eu-west-2 and ca-central-1, there's no gpu ec2 instance like P2/P3. So the tf tests using gpu instance will fail and we should skip them in these regions.
Merge Checklist
Put an
x
in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.