Skip to content

Add Python-3.10 (build and test) #5420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Feb 20, 2022
Merged

Add Python-3.10 (build and test) #5420

merged 6 commits into from
Feb 20, 2022

Conversation

malfet
Copy link
Contributor

@malfet malfet commented Feb 14, 2022

No description provided.

@facebook-github-bot
Copy link

facebook-github-bot commented Feb 14, 2022

💊 CI failures summary and remediations

As of commit ab5eae0 (more details on the Dr. CI page):


  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build build_docs (1/1)

Step: "Build docs" (full log | diagnosis details | 🔁 rerun)

Warning, treated as error:
loading intersphinx inventory from https://matplotlib.org/stable/objects.inv...
generating gallery...
Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files.
generating gallery for auto_examples... [ 16%] plot_scripted_tensor_transforms.py
generating gallery for auto_examples... [ 33%] plot_optical_flow.py
generating gallery for auto_examples... [ 50%] plot_repurposing_annotations.py
generating gallery for auto_examples... [ 66%] plot_transforms.py
generating gallery for auto_examples... [ 83%] plot_video_api.py
generating gallery for auto_examples... [100%] plot_visualization_utils.py

Warning, treated as error:
/root/project/gallery/plot_visualization_utils.py failed to execute correctly: Traceback (most recent call last):
  File "/root/project/gallery/plot_visualization_utils.py", line 86, in <module>
    outputs = model(batch)
  File "/root/project/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/project/torchvision/models/detection/generalized_rcnn.py", line 98, in forward
    proposals, proposal_losses = self.rpn(images, features, targets)
  File "/root/project/env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/project/torchvision/models/detection/rpn.py", line 353, in forward

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@malfet malfet force-pushed the malfet/add-python-3.10 branch 2 times, most recently from 1a93454 to ba639fc Compare February 14, 2022 23:58
@malfet malfet requested review from atalman and NicolasHug February 15, 2022 06:10
@malfet malfet force-pushed the malfet/add-python-3.10 branch 2 times, most recently from 5c8497b to 77821ec Compare February 15, 2022 06:40
@malfet malfet force-pushed the malfet/add-python-3.10 branch from 77821ec to 3062d2e Compare February 15, 2022 21:00
@jdsgomes
Copy link
Contributor

I think these fails can be solved by replacing this

with

gamma_map = [round((255 + 1 - 1e-3) * gain * pow(ele / 255.0, gamma)) for ele in range(256)] * 3

Looking at the code I believe this is equivalent and solves the float/int issue.

@malfet
Copy link
Contributor Author

malfet commented Feb 15, 2022

gamma_map = [round((255 + 1 - 1e-3) * gain * pow(ele / 255.0, gamma)) for ele in range(256)] * 3

round or int (also, not sure which one is faster)?
Indeed it needs int otherwise it causes another failure: https://app.circleci.com/pipelines/github/pytorch/vision/14807/workflows/baa8f094-988d-4ab1-a643-b7f34788a702/jobs/1191741

@malfet malfet force-pushed the malfet/add-python-3.10 branch from 06f1625 to 3062d2e Compare February 16, 2022 03:29
@malfet
Copy link
Contributor Author

malfet commented Feb 16, 2022

Moved adjust_gamma change to #5427

@datumbox
Copy link
Contributor

datumbox commented Feb 16, 2022

@jdsgomes We will probably need to cherrypick #5427 on the release branch. I assume 3.10 support will be added to this release right?

@pmeier
Copy link
Collaborator

pmeier commented Feb 16, 2022

I'm guessing this is not synchronized with #5320? cc @vfdev-5

@datumbox
Copy link
Contributor

@pmeier that was just a check, we can close the other one.

@pmeier
Copy link
Collaborator

pmeier commented Feb 16, 2022

Note that #5320 was mostly a superset of what is proposed here. It also handled the README compatibility table, as well as docker and OSX wheels.

@malfet
Copy link
Contributor Author

malfet commented Feb 16, 2022

Note that #5320 was mostly a superset of what is proposed here. It also handled the README compatibility table, as well as docker and OSX wheels.

Wow, I wasn't aware of osx_wheel.sh, but at glance at this script it does not seem to be used (nor work) right now, so I propose we delete them in #5433

@datumbox
Copy link
Contributor

Seems like the failing tests are due to dependency issues from core and numpy.

@datumbox
Copy link
Contributor

@malfet @atalman I had a look at the failing tests and it seems the issues are related to dependencies issues from Core.

The binary_win_conda_py3.10_cpu fails due to multiple conflicts:

Package pytorch-mutex conflicts for:
pytorch==1.12.0.dev20220207 -> pytorch-mutex==1.0[build='cuda|cpu']
pytorch-mutex==1.0=cpu

Package python conflicts for:
pytorch==1.12.0.dev20220207 -> typing_extensions -> python[version='>=2.7,<2.8.0a0|>=3.5|>=3.8,<3.9.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
python=3.10
pytorch==1.12.0.dev20220207 -> python[version='>=3.10,<3.11.0a0']

Package setuptools conflicts for:
setuptools
python=3.10 -> pip -> setuptools

The unittest_windows_cpu_py3.10 fails because PyTorch core has been compiled with a different version of numpy than the one installed:

Failed to initialize NumPy: module compiled against API version 0xf but this version of numpy is 0xe 

Nevertheless all other unittest_windows_cpu_py* jobs pass which makes me wonder if core gets compiled on the specific configuration with a different numpy version accidentally.

I understand that these issues can be fixed upstream, but if you think otherwise let me know to further investigate. Thanks!

@malfet
Copy link
Contributor Author

malfet commented Feb 20, 2022

The binary_win_conda_py3.10_cpu fails due to multiple conflicts:

Package pytorch-mutex conflicts for:
pytorch==1.12.0.dev20220207 -> pytorch-mutex==1.0[build='cuda|cpu']
pytorch-mutex==1.0=cpu

Package python conflicts for:
pytorch==1.12.0.dev20220207 -> typing_extensions -> python[version='>=2.7,<2.8.0a0|>=3.5|>=3.8,<3.9.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
python=3.10
pytorch==1.12.0.dev20220207 -> python[version='>=3.10,<3.11.0a0']

Package setuptools conflicts for:
setuptools
python=3.10 -> pip -> setuptools

This was resolved by pytorch/builder@3e6551c

The unittest_windows_cpu_py3.10 fails because PyTorch core has been compiled with a different version of numpy than the one installed:

Failed to initialize NumPy: module compiled against API version 0xf but this version of numpy is 0xe 

Nevertheless all other unittest_windows_cpu_py* jobs pass which makes me wonder if core gets compiled on the specific configuration with a different numpy version accidentally.

Seems to be connected to the previous failure

@malfet malfet merged commit e88a554 into main Feb 20, 2022
@malfet malfet deleted the malfet/add-python-3.10 branch February 20, 2022 22:03
@github-actions
Copy link

Hey @malfet!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

facebook-github-bot pushed a commit that referenced this pull request Feb 25, 2022
Summary:
* Add Python-3.10 (build and test)

* Fix functional.adjust_gamma

Reviewed By: jdsgomes

Differential Revision: D34475318

fbshipit-source-id: cffcd3ec4b333267da2b5993bd3fac4e7cf37a87

Co-authored-by: Vasilis Vryniotis <[email protected]>
Co-authored-by: Joao Gomes <[email protected]>
malfet added a commit that referenced this pull request Mar 9, 2022
* Add Python-3.10 (build and test)

* Fix functional.adjust_gamma

Co-authored-by: Vasilis Vryniotis <[email protected]>
Co-authored-by: Joao Gomes <[email protected]>
(cherry picked from commit e88a554)
malfet added a commit that referenced this pull request Mar 9, 2022
* Add Python-3.10 (build and test)

* Fix functional.adjust_gamma

Co-authored-by: Vasilis Vryniotis <[email protected]>
Co-authored-by: Joao Gomes <[email protected]>
(cherry picked from commit e88a554)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants