`create_venvs.py` yields `ERROR: No matching distribution found for apache-airflow==2.10.1`

[metersk]

Describe the bug
On a clean main running python3 create_venvs.py --target development yields the following error:

~/code/amazon-mwaa-docker-images main
❯ python3 create_venvs.py --target development
>>> Creating a virtual environment under the path .venv...
> Virtualenv already exists in .venv
> Upgrade pip...
Requirement already satisfied: pip in ./.venv/lib/python3.13/site-packages (24.0)
Collecting pip
  Using cached pip-25.0.1-py3-none-any.whl.metadata (3.7 kB)
Using cached pip-25.0.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 24.0
    Uninstalling pip-24.0:
      Successfully uninstalled pip-24.0
Successfully installed pip-25.0.1

> No need to create dev requirements for /Users/amm/code/amazon-mwaa-docker-images.  Using default.
> Install dependencies from requirements.txt...
Requirement already satisfied: Jinja2==3.1.4 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 6)) (3.1.4)
Requirement already satisfied: MarkupSafe==2.1.5 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 7)) (2.1.5)
Requirement already satisfied: nodeenv==1.8.0 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 8)) (1.8.0)
Collecting pip==24.0 (from -r requirements.txt (line 9))
  Using cached pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Requirement already satisfied: pydocstyle==6.3.0 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 10)) (6.3.0)
Collecting pyright==1.1.360 (from -r requirements.txt (line 11))
  Using cached pyright-1.1.360-py3-none-any.whl.metadata (6.2 kB)
Collecting ruff==0.4.2 (from -r requirements.txt (line 12))
  Using cached ruff-0.4.2-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl.metadata (23 kB)
Requirement already satisfied: setuptools==70.0.0 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 13)) (70.0.0)
Requirement already satisfied: snowballstemmer==2.2.0 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 14)) (2.2.0)
Requirement already satisfied: trufflehog==2.2.1 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 15)) (2.2.1)
Requirement already satisfied: wheel==0.43.0 in ./.venv/lib/python3.13/site-packages (from -r requirements.txt (line 16)) (0.43.0)
Requirement already satisfied: GitPython==3.0.6 in ./.venv/lib/python3.13/site-packages (from trufflehog==2.2.1->-r requirements.txt (line 15)) (3.0.6)
Requirement already satisfied: truffleHogRegexes==0.0.7 in ./.venv/lib/python3.13/site-packages (from trufflehog==2.2.1->-r requirements.txt (line 15)) (0.0.7)
Requirement already satisfied: gitdb2>=2.0.0 in ./.venv/lib/python3.13/site-packages (from GitPython==3.0.6->trufflehog==2.2.1->-r requirements.txt (line 15)) (4.0.2)
Requirement already satisfied: gitdb>=4.0.1 in ./.venv/lib/python3.13/site-packages (from gitdb2>=2.0.0->GitPython==3.0.6->trufflehog==2.2.1->-r requirements.txt (line 15)) (4.0.12)
Requirement already satisfied: smmap<6,>=3.0.1 in ./.venv/lib/python3.13/site-packages (from gitdb>=4.0.1->gitdb2>=2.0.0->GitPython==3.0.6->trufflehog==2.2.1->-r requirements.txt (line 15)) (5.0.2)
Using cached pip-24.0-py3-none-any.whl (2.1 MB)
Using cached pyright-1.1.360-py3-none-any.whl (18 kB)
Using cached ruff-0.4.2-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (16.7 MB)
Installing collected packages: ruff, pip, pyright
  Attempting uninstall: ruff
    Found existing installation: ruff 0.9.6
    Uninstalling ruff-0.9.6:
      Successfully uninstalled ruff-0.9.6
  Attempting uninstall: pip
    Found existing installation: pip 25.0.1
    Uninstalling pip-25.0.1:
      Successfully uninstalled pip-25.0.1
  Attempting uninstall: pyright
    Found existing installation: pyright 1.1.394
    Uninstalling pyright-1.1.394:
      Successfully uninstalled pyright-1.1.394
Successfully installed pip-24.0 pyright-1.1.360 ruff-0.4.2

[notice] A new release of pip is available: 24.0 -> 25.0.1
[notice] To update, run: /Users/amm/code/amazon-mwaa-docker-images/.venv/bin/python -m pip install --upgrade pip

> Install/Upgrade development tools: ['pydocstyle', 'pyright', 'ruff']...
Requirement already satisfied: pydocstyle in ./.venv/lib/python3.13/site-packages (6.3.0)
Requirement already satisfied: pyright in ./.venv/lib/python3.13/site-packages (1.1.360)
Collecting pyright
  Using cached pyright-1.1.394-py3-none-any.whl.metadata (6.6 kB)
Requirement already satisfied: ruff in ./.venv/lib/python3.13/site-packages (0.4.2)
Collecting ruff
  Using cached ruff-0.9.6-py3-none-macosx_11_0_arm64.whl.metadata (25 kB)
Requirement already satisfied: snowballstemmer>=2.2.0 in ./.venv/lib/python3.13/site-packages (from pydocstyle) (2.2.0)
Requirement already satisfied: nodeenv>=1.6.0 in ./.venv/lib/python3.13/site-packages (from pyright) (1.8.0)
Requirement already satisfied: typing-extensions>=4.1 in ./.venv/lib/python3.13/site-packages (from pyright) (4.12.2)
Requirement already satisfied: setuptools in ./.venv/lib/python3.13/site-packages (from nodeenv>=1.6.0->pyright) (70.0.0)
Using cached pyright-1.1.394-py3-none-any.whl (5.7 MB)
Using cached ruff-0.9.6-py3-none-macosx_11_0_arm64.whl (11.1 MB)
Installing collected packages: ruff, pyright
  Attempting uninstall: ruff
    Found existing installation: ruff 0.4.2
    Uninstalling ruff-0.4.2:
      Successfully uninstalled ruff-0.4.2
  Attempting uninstall: pyright
    Found existing installation: pyright 1.1.360
    Uninstalling pyright-1.1.360:
      Successfully uninstalled pyright-1.1.360
Successfully installed pyright-1.1.394 ruff-0.9.6

[notice] A new release of pip is available: 24.0 -> 25.0.1
[notice] To update, run: /Users/amm/code/amazon-mwaa-docker-images/.venv/bin/python -m pip install --upgrade pip

>>> Finished creating a virtual environment under the path .venv.


>>> Creating a virtual environment under the path images/airflow/2.10.1/.venv...
> Virtualenv already exists in images/airflow/2.10.1/.venv
> Upgrade pip...
Requirement already satisfied: pip in ./images/airflow/2.10.1/.venv/lib/python3.13/site-packages (25.0.1)

> Creating images/airflow/2.10.1/requirements-dev.txt from images/airflow/2.10.1/requirements.txt
> Install dependencies from images/airflow/2.10.1/requirements-dev.txt...
ERROR: Ignored the following yanked versions: 1.10.11, 1.10.13
ERROR: Could not find a version that satisfies the requirement apache-airflow==2.10.1 (from versions: 1.8.1, 1.8.2rc1, 1.8.2, 1.9.0, 1.10.0, 1.10.1b1, 1.10.1rc2, 1.10.1, 1.10.2b2, 1.10.2rc1, 1.10.2rc2, 1.10.2rc3, 1.10.2, 1.10.3b1, 1.10.3b2, 1.10.3rc1, 1.10.3rc2, 1.10.3, 1.10.4b2, 1.10.4rc1, 1.10.4rc2, 1.10.4rc3, 1.10.4rc4, 1.10.4rc5, 1.10.4, 1.10.5rc1, 1.10.5, 1.10.6rc1, 1.10.6rc2, 1.10.6, 1.10.7rc1, 1.10.7rc2, 1.10.7rc3, 1.10.7, 1.10.8rc1, 1.10.8, 1.10.9rc1, 1.10.9, 1.10.10rc1, 1.10.10rc2, 1.10.10rc3, 1.10.10rc4, 1.10.10rc5, 1.10.10, 1.10.11rc1, 1.10.11rc2, 1.10.12rc1, 1.10.12rc2, 1.10.12rc3, 1.10.12rc4, 1.10.12, 1.10.13rc1, 1.10.14rc1, 1.10.14rc2, 1.10.14rc3, 1.10.14rc4, 1.10.14, 1.10.15rc1, 1.10.15, 2.0.0b1, 2.0.0b2, 2.0.0b3, 2.0.0rc1, 2.0.0rc2, 2.0.0rc3, 2.0.0, 2.0.1rc1, 2.0.1rc2, 2.0.1, 2.0.2rc1, 2.0.2, 2.1.0rc1, 2.1.0rc2, 2.1.0, 2.1.1rc1, 2.1.1, 2.1.2rc1, 2.1.2, 2.1.3rc1, 2.1.3, 2.1.4rc1, 2.1.4rc2, 2.1.4, 2.2.0b1, 2.2.0b2, 2.2.0rc1, 2.2.0, 2.2.1rc1, 2.2.1rc2, 2.2.1, 2.2.2rc1, 2.2.2rc2, 2.2.2, 2.2.3rc1, 2.2.3rc2, 2.2.3, 2.2.4rc1, 2.2.4, 2.2.5rc1, 2.2.5rc2, 2.2.5rc3, 2.2.5, 2.3.0b1, 2.3.0rc1, 2.3.0rc2, 2.3.0, 2.3.1rc1, 2.3.1, 2.3.2rc1, 2.3.2rc2, 2.3.2, 2.3.3rc1, 2.3.3rc2, 2.3.3rc3, 2.3.3, 2.3.4rc1, 2.3.4, 2.4.0b1, 2.4.0rc1, 2.4.0, 2.4.1rc1, 2.4.1, 2.4.2rc1, 2.4.2, 2.4.3rc1, 2.4.3, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1rc1, 2.5.1rc2, 2.5.1, 2.5.2rc1, 2.5.2rc2, 2.5.2, 2.5.3rc1, 2.5.3rc2, 2.5.3, 2.6.0b1, 2.6.0rc1, 2.6.0rc2, 2.6.0rc3, 2.6.0rc4, 2.6.0rc5, 2.6.0, 2.6.1rc1, 2.6.1rc2, 2.6.1rc3, 2.6.1, 2.6.2rc1, 2.6.2rc2, 2.6.2, 2.6.3rc1, 2.6.3, 2.7.0b1, 2.7.0rc1, 2.7.0rc2, 2.7.0, 2.7.1rc1, 2.7.1rc2, 2.7.1, 2.7.2rc1, 2.7.2)
ERROR: No matching distribution found for apache-airflow==2.10.1
Traceback (most recent call last):
  File "/Users/amm/code/amazon-mwaa-docker-images/create_venvs.py", line 151, in <module>
    main()
    ~~~~^^
  File "/Users/amm/code/amazon-mwaa-docker-images/create_venvs.py", line 143, in main
    create_venv(
    ~~~~~~~~~~~^
        dir_path,
        ^^^^^^^^^
        development_build=args.target == development_target_choice,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        recreate=args.recreate
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/amm/code/amazon-mwaa-docker-images/create_venvs.py", line 62, in create_venv
    pip_install(venv_path, "-r", str(requirements_path))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amm/code/amazon-mwaa-docker-images/create_venvs.py", line 112, in pip_install
    subprocess.run(
    ~~~~~~~~~~~~~~^
        [os.path.join(venv_dir, "bin", "python"), "-m", "pip", "install", *args],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        check=True,
        ^^^^^^^^^^^
    )
    ^
  File "/opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['images/airflow/2.10.1/.venv/bin/python', '-m', 'pip', 'install', '-r', 'images/airflow/2.10.1/requirements-dev.txt']' returned non-zero exit status 1.

To Reproduce
Steps to reproduce the behavior:

1. clone repo
2. run python3 create_venvs.py --target development

note: this also happens if i rm -rf .venv and rerun to create a new vm

i also tried removing the images/airflow/2.10.1 dir and error persists with the remaining dirs

---

System Information

• OS: macos 14.4

---

Expected behavior
no error

[metersk]

update -- this seems to be a python version issue. i got it to install for 2.10.1 and 2.10.3 with python versions 3.11 and 3.12.

[dhegberg]

Were you on Python 3.13?

Or did this check fail for a version under 3.11?

amazon-mwaa-docker-images/create_venvs.py

Lines 26 to 32 in f97892e

	def verify_python_version():
	"""Check if the current Python version is at least 3.9."""
	major, minor, *_ = sys.version_info
	if major != 3 or minor < 11:
	print("Python 3.11 or higher is required.")
	sys.exit(1)

[metersk]

Yes I was on Python 3.13 when I got the original error in the OP

[dhegberg]

We should update the check to ensure the required python version matches the version used in MWAA.