RHOAIENG-97474: drop py311 default runtime image — only py312 supported by szaher · Pull Request #1169 · project-codeflare/codeflare-sdk

szaher · 2026-09-30T22:00:07Z

Summary

Remove CUDA_PY311_RUNTIME_IMAGE and "3.11" from SUPPORTED_PYTHON_VERSIONS
No Ray 2.58.0 image exists for Python 3.11 on quay.io/modh/ray
Python 3.11 users now get the existing "provide your own image" warning (same as py313)
Updated all tests that assumed py311 had a default image

Test plan

Verify constants.py no longer references py311
All utils tests pass (20/20)
All validation tests pass (13/13)
Lint and format checks pass

🤖 Generated with Claude Code

openshift-ci-robot · 2026-09-30T22:00:11Z

@szaher: This pull request references RHOAIENG-97474 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.1.0" version, but no target version was set.

Details

In response to this:

Summary

Update Python 3.11 default runtime image from Ray 2.52.1 to Ray 2.58.0

Aligns py311 image with the SDK's pinned RAY_VERSION (2.58.0)

Without this fix, Python 3.11 users get a version mismatch rejection at runtime

Test plan

Verify constants.py has correct image reference

Verify unit tests pass

Verify image tag exists on quay.io/modh/ray

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-09-30T22:00:11Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kryanbeane for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2026-09-30T22:02:41Z

Codecov Report

❌ Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.46%. Comparing base (a3a0598) to head (6fd2794).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/codeflare_sdk/ray/cluster/config.py	92.85%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1169      +/-   ##
==========================================
- Coverage   96.50%   96.46%   -0.05%     
==========================================
  Files          24       24              
  Lines        2319     2346      +27     
==========================================
+ Hits         2238     2263      +25     
- Misses         81       83       +2

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

No Ray 2.58.0 image exists for Python 3.11. Remove CUDA_PY311_RUNTIME_IMAGE and the "3.11" entry from SUPPORTED_PYTHON_VERSIONS. Python 3.11 users must provide their own image (existing warning guides them). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

pawelpaszki

Summary

I agree with removing the Python 3.11 default image from SUPPORTED_PYTHON_VERSIONS (no Ray 2.58.0 py311 image on quay.io/modh/ray, and the old pin was 2.52.1 vs RAY_VERSION). That part should land.

However, this PR mixes several concerns and leaves important follow-up undone. I’m requesting changes before merge.

1. Unrelated work bundled under RHOAIENG-97474

Most of the diff (~75%) is cluster CPU/memory request vs limit validation (_parse_cpu, _parse_memory, _validate_resource_requests in config.py + ~116 lines in test_config.py, tied to RHOAIENG-59338). That is a separate feature (behavior change, release notes, Jira target version).

Please split into a dedicated PR (or at minimum a separate commit + Jira reference in title/description). Reviewers and release tooling shouldn’t have to untangle image defaults from cluster validation.

2. Incomplete cleanup after dropping py311

Constants/tests were updated, but the repo still advertises py311 defaults elsewhere. None of these are touched in this PR:

.github/workflows/update-versions.yaml — still bumps CUDA_PY311_RUNTIME_IMAGE
docs/sphinx/user-docs/cluster-configuration.rst — still documents py311 default image
Guided demo / additional notebooks — still list quay.io/modh/ray:2.52.1-py311-cu121

Users and automation will believe 3.11 still has a supported default. Please include cleanup in this PR or an immediate follow-up tracked on the same ticket.

3. Fallback behavior for 3.11 is inconsistent and under-tested

get_ray_image_for_python_version() in utils.py (unchanged in this PR) still does:

warn_on_unsupported=True (used by update_image()): warn + None for 3.11 — correct after this change.
warn_on_unsupported=False: silent fallback to CUDA_PY312_RUNTIME_IMAGE for any unsupported version, including explicit "3.11".

That means callers using warn_on_unsupported=False get a 3.12 Ray image while requesting 3.11 — a Python/Ray skew risk. The new test test_get_ray_image_for_python_version_explicit_311_unsupported actually asserts the py312 image, not None, but the docstring says “returns None” (see inline comment).

If the product intent is “only py312 supported,” consider whether silent fallback should still apply to 3.11 (or document clearly that 3.11 must always pass an explicit image). At minimum, fix the misleading test name/docstring and align tests with intended behavior.

4. Release / process

openshift-ci noted RHOAIENG-97474 has no target version (expected 5.1.0 for main) — please set before merge.
Breaking change: SDK on Python 3.11 without image no longer gets a default modh image; call out in release notes.

5. Validation change (if kept in this PR)

ClusterConfiguration(..., head_cpu_requests=4) with defaults now raises — good for 59338, but it’s a breaking config API change and should not ride only on the py311 image ticket.

Codecov also reports 2 uncovered lines in config.py on this patch.

Happy to re-review once scope is split and cleanup/fallback behavior is addressed.

pawelpaszki · 2026-10-01T13:34:17Z

        )


+_MEMORY_BINARY_UNITS = {


Unrelated to RHOAIENG-97474. This entire block (_MEMORY_*, _parse_cpu, _parse_memory, _validate_resource_requests) is cluster resource validation for RHOAIENG-59338. It dominates the PR (+62 LOC production, +116 tests) and changes ClusterConfiguration behavior (e.g. head_cpu_requests=4 with default limits now fails at init).

Please move to a separate PR with its own Jira + release note. Keeping it here makes review, rollback, and target-version tracking much harder.

pawelpaszki · 2026-10-01T13:34:17Z

    assert worker_group["maxReplicas"] == 3


+def test_resource_requests_exceed_limits_head_cpu():


These tests belong with the config.py validation change (59338), not the py311 default-image removal. Please split with the production code above.

pawelpaszki · 2026-10-01T13:34:17Z

-    image = get_ray_image_for_python_version("3.11")
-    assert image == CUDA_PY311_RUNTIME_IMAGE
+def test_get_ray_image_for_python_version_explicit_311_unsupported():
+    """Test get_ray_image_for_python_version() returns None for Python 3.11."""


Incorrect / misleading test documentation. The docstring says this returns None for Python 3.11, but the test calls get_ray_image_for_python_version("3.11", warn_on_unsupported=False) and asserts CUDA_PY312_RUNTIME_IMAGE.

That documents the silent fallback to 3.12 path, not the unsupported path users hit via update_image() (warn_on_unsupported=True → warn + None).

Please fix the docstring (and consider renaming the test) to match behavior, or change behavior if product intent is that 3.11 should never silently get a 3.12 image.

pawelpaszki · 2026-10-01T13:34:17Z

 CUDA_PY312_RUNTIME_IMAGE = "quay.io/modh/ray@sha256:8183007c9b46b359bdc75ea74346f10d642e585bb4f22fa3aa02524b3c966fa0"

 # Centralized image selection
 SUPPORTED_PYTHON_VERSIONS = {


Dropping "3.11" here is the right fix, but cleanup is incomplete in this PR:

.github/workflows/update-versions.yaml still maintains CUDA_PY311_RUNTIME_IMAGE

Sphinx docs + multiple notebooks still document the old py311 default (2.52.1-py311-cu121)

Please update or remove those references so docs/CI match runtime behavior.

pawelpaszki · 2026-10-01T13:34:18Z

-    """Test get_ray_image_for_python_version() with explicit Python 3.11."""
-    image = get_ray_image_for_python_version("3.11")
-    assert image == CUDA_PY311_RUNTIME_IMAGE
+def test_get_ray_image_for_python_version_explicit_311_unsupported():


Related to fallback semantics: after removing 3.11 from SUPPORTED_PYTHON_VERSIONS, explicit "3.11" with warn_on_unsupported=False still returns the 3.12 image (utils.py unchanged). If we mean “only py312 supported,” we should either stop falling back for 3.11 or add a test that matches the primary path (update_image / warn_on_unsupported=True → None), which you already cover in test_update_image_with_*_python_311_unsupported — this test should not contradict that story via a wrong docstring.

openshift-ci · 2026-10-05T11:43:59Z

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot added the jira/valid-reference label Sep 30, 2026

openshift-ci Bot requested review from DavidAdaRH and laurafitzgerald September 30, 2026 22:00

szaher force-pushed the RHOAIENG-97474 branch from d7f9854 to 6fd2794 Compare October 1, 2026 09:46

szaher changed the title ~~RHOAIENG-97474: update py311 default runtime image to Ray 2.58.0~~ RHOAIENG-97474: drop py311 default runtime image — only py312 supported Oct 1, 2026

pawelpaszki requested changes Oct 1, 2026

View reviewed changes

openshift-ci Bot assigned pawelpaszki Oct 1, 2026

openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 5, 2026

Navigation Menu

Repository navigation

RHOAIENG-97474: drop py311 default runtime image — only py312 supported - #1169

RHOAIENG-97474: drop py311 default runtime image — only py312 supported#1169
szaher wants to merge 1 commit into
mainfrom
RHOAIENG-97474

szaher commented Sep 30, 2026 •

edited

Loading

Uh oh!

openshift-ci-robot commented Sep 30, 2026 •

edited by openshift-ci Bot

Loading

Summary

Test plan

Uh oh!

openshift-ci Bot commented Sep 30, 2026

Uh oh!

codecov Bot commented Sep 30, 2026 •

edited

Loading

Uh oh!

pawelpaszki left a comment

Uh oh!

pawelpaszki Oct 1, 2026

Uh oh!

pawelpaszki Oct 1, 2026

Uh oh!

pawelpaszki Oct 1, 2026

Uh oh!

pawelpaszki Oct 1, 2026

Uh oh!

pawelpaszki Oct 1, 2026

Uh oh!

openshift-ci Bot commented Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		assert worker_group["maxReplicas"] == 3


		def test_resource_requests_exceed_limits_head_cpu():

		)


		_MEMORY_BINARY_UNITS = {

Conversation

szaher commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

openshift-ci-robot commented Sep 30, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

openshift-ci Bot commented Sep 30, 2026

Uh oh!

codecov Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pawelpaszki left a comment

Choose a reason for hiding this comment

Summary

1. Unrelated work bundled under RHOAIENG-97474

2. Incomplete cleanup after dropping py311

3. Fallback behavior for 3.11 is inconsistent and under-tested

4. Release / process

5. Validation change (if kept in this PR)

Uh oh!

pawelpaszki Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

pawelpaszki Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

pawelpaszki Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

pawelpaszki Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

pawelpaszki Oct 1, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

szaher commented Sep 30, 2026 •

edited

Loading

openshift-ci-robot commented Sep 30, 2026 •

edited by openshift-ci Bot

Loading

codecov Bot commented Sep 30, 2026 •

edited

Loading