[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #671

Allenz5 · 2025-06-17T17:09:38Z

Description of changes

Add Priority to subnets when creating fleet using capacity-optimized-prioritized and prioritized Allocation Strategy
Add SingleAvailabilityZone flag when creating fleet if EnableSingleAvailabilityZone is true

Tests

Extended test_fleet_manager.py::test_fleet_manager.py to test that Priority is appended in overrides when using capacity-optimized-prioritized and prioritized Allocation Strategy
Extended test_fleet_manager.py::test_fleet_manager.py to test that SingleAvailabilityZone flag is correctly set when EnableSingleAvailabilityZone is true

References

Checklist

Make sure you are pointing to the right branch.
If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
Check all commits' messages are clear, describing what and why vs how.
Make sure to have added unit tests or integration tests to cover the new/modified code.
Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…leet call Signed-off-by: Hanxuan Zhang <[email protected]>

Signed-off-by: Hanxuan Zhang <[email protected]>

…rams launching with EnableSingleAvailabilityZone and prioritized|capacity-optimized-prioritized AllocationStrategy Signed-off-by: Hanxuan Zhang <[email protected]>

Signed-off-by: Hanxuan Zhang <[email protected]>

…ity-optimized-prioritized when using EnableSingleAvailabilityZone Signed-off-by: Hanxuan Zhang <[email protected]>

… into develop

himani2411 · 2025-06-18T17:32:42Z

src/slurm_plugin/fleet_manager.py

-                # set SingleAvailabilityZone to False
+                "SingleAvailabilityZone": (
+                    self._compute_resource_config["Networking"]["SingleAvailabilityZone"]
+                    if self._compute_resource_config["Networking"]["SingleAvailabilityZone"] is not None


I would suggest that you add a condition here where we check if the allocation strategy is the one we want and then apply SingleAvailibilityZone. We don't want to use this parameter if we use lowest-price AllocationStrategy

himani2411 · 2025-06-18T17:37:30Z

src/slurm_plugin/fleet_manager.py

        for instance_type in self._compute_resource_config["Instances"]:
            subnet_ids = self._compute_resource_config["Networking"]["SubnetIds"]
            for subnet_id in subnet_ids:
-                overrides.update({"InstanceType": instance_type["InstanceType"], "SubnetId": subnet_id})
+                if (
+                    self._compute_resource_config.get("AllocationStrategy") == "prioritized"


We should also check if the CapacityType is the one that we want along with allocation strategy, the reason we should add that even if that's already taken care in CLI validator is because customer can suppress or ignore and move forward.

hanwen-cluster · 2025-06-19T16:27:42Z

CHANGELOG.md

- There were no changes for this version.
+- Support prioritized|capacity-optimized-prioritized Allocation Strategy and EnableSingleAvailabilityZone


The change should be in 3.14.0

hanwen-cluster · 2025-06-19T17:01:46Z

src/slurm_plugin/fleet_manager.py

+                    self._compute_resource_config["Networking"]["SingleAvailabilityZone"]
+                    if self._compute_resource_config["Networking"]["SingleAvailabilityZone"] is not None


Are you sure self._compute_resource_config always has "Networking" section?

himani2411 · 2025-06-20T13:30:54Z

src/slurm_plugin/fleet_manager.py

+                    self._compute_resource_config["Networking"]["SingleAvailabilityZone"]
+                    if self._compute_resource_config["Networking"]["SingleAvailabilityZone"] is not None
+                    and (
+                        self._compute_resource_config.get("AllocationStrategy") == "prioritized"


Nitpick: Can we check these conditions using a function as we keep checking this above an here

Signed-off-by: Hanxuan Zhang <[email protected]>

himani2411 · 2025-06-24T14:37:22Z

tests/slurm_plugin/test_fleet_manager.py

+            (5, "queue-single-az", "fleet1", False, {}, None),
+            # Use "prioritized" Allocation Strategy AND Launch Override with Priority
+            (5, "queue-prioritized", "fleet1", False, {}, None),
+            # Use "capacity-optimized-prioritized" Allocation Strategy AND Launch Override with Priority


Can we add one of them to. have all_or_nothing True?

himani2411 · 2025-06-24T14:42:12Z

src/slurm_plugin/fleet_manager.py

@@ -312,13 +319,26 @@ def _uses_single_az(self):
        subnet_ids = self._compute_resource_config.get("Networking", {}).get("SubnetIds", [])
        return len(subnet_ids) == 1

+    def _uses_subnet_prioritization(self):
+        return (
+            self._compute_resource_config.get("AllocationStrategy") == "prioritized"


Can we add a valid condition where we check prioritized is used with On-demand Capacity and capacity-optimized-prioritized is checked with Spot Capacity type. Please update the unit test accordingly as they are passing without this change

himani2411 · 2025-06-24T14:42:59Z

tests/common.py

+            "Api": "create-fleet",
+            "Instances": [{"InstanceType": "t2.medium"}, {"InstanceType": "t2.large"}],
+            "AllocationStrategy": "capacity-optimized-prioritized",
+            "CapacityType": "on-demand",


Nitpick: The unit test after the change https://github.com/aws/aws-parallelcluster-node/pull/671/files#r2164221669 should not pass

…zed' is used with On-Demand Capacity and 'capacity-optimized-prioritized' is used with Spot Capacity type. Signed-off-by: Hanxuan Zhang <[email protected]>

himani2411 · 2025-06-30T20:45:34Z

src/slurm_plugin/fleet_manager.py

+                "SingleAvailabilityZone", None
+            )
+            if enable_single_availability_zone is None or (
+                enable_single_availability_zone and self._uses_subnet_prioritization() is False


If we suppress the CLI validator and CX uses EnableSingleVaialibilityZone as False when using all-or-nothing with 1 Az and multiple Instances then Ec2 create Fleet call will Fail as we do not set SingleInstanceType or SingleAvailibilityZone which is a requirement from EC2

Signed-off-by: Hanxuan Zhang <[email protected]>

himani2411 · 2025-07-07T20:21:53Z

Update the PR description to mention what is no longer relevant

Allenz5 and others added 5 commits June 11, 2025 11:03

[Subnet Prioritization] Add SingleAvailabilityZone parameter to ec2 f…

430c9a3

…leet call Signed-off-by: Hanxuan Zhang <[email protected]>

[Subnet Prioritization] Add Priority parameter to ec2 fleet call

2d21a5a

Signed-off-by: Hanxuan Zhang <[email protected]>

[Subnet Prioritization] Extend test_evaluate_launch_params to test pa…

7f70084

…rams launching with EnableSingleAvailabilityZone and prioritized|capacity-optimized-prioritized AllocationStrategy Signed-off-by: Hanxuan Zhang <[email protected]>

[Subnet Prioritization] Update CHANGELOG.md

01d3219

Signed-off-by: Hanxuan Zhang <[email protected]>

Merge branch 'aws:develop' into develop

c905f70

Allenz5 requested review from a team as code owners June 17, 2025 17:09

Allenz5 added 2 commits June 18, 2025 15:59

[Subnet Prioritization] Check AllocationStrategy is prioritized|capac…

4f2a6cb

…ity-optimized-prioritized when using EnableSingleAvailabilityZone Signed-off-by: Hanxuan Zhang <[email protected]>

Merge branch 'develop' of github.com:Allenz5/aws-parallelcluster-node…

ddd8e41

… into develop

himani2411 reviewed Jun 18, 2025

View reviewed changes

hanwen-cluster reviewed Jun 19, 2025

View reviewed changes

himani2411 reviewed Jun 20, 2025

View reviewed changes

Allenz5 added 3 commits June 20, 2025 10:47

[Subnet Prioritization] Update CHANGELOG.md

56309ea

Signed-off-by: Hanxuan Zhang <[email protected]>

[Subnet Prioritization] Reformat the code to make it more clear

cf11486

Signed-off-by: Hanxuan Zhang <[email protected]>

[Subnet Prioritization] Reformat the code to make it more clear

b028b40

Signed-off-by: Hanxuan Zhang <[email protected]>

himani2411 reviewed Jun 24, 2025

View reviewed changes

[Subnet Prioritization] Add a valid condition to check that 'prioriti…

a0a4775

…zed' is used with On-Demand Capacity and 'capacity-optimized-prioritized' is used with Spot Capacity type. Signed-off-by: Hanxuan Zhang <[email protected]>

himani2411 reviewed Jun 30, 2025

View reviewed changes

[Subnet Prioritization] Remove EnableSingleAvailabilityZone parameter

1efee32

Signed-off-by: Hanxuan Zhang <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #671

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #671

Uh oh!

Allenz5 commented Jun 17, 2025

Uh oh!

himani2411 Jun 18, 2025

Uh oh!

himani2411 Jun 18, 2025

Uh oh!

hanwen-cluster Jun 19, 2025

Uh oh!

hanwen-cluster Jun 19, 2025

Uh oh!

himani2411 Jun 20, 2025

Uh oh!

himani2411 Jun 24, 2025

Uh oh!

himani2411 Jun 24, 2025

Uh oh!

himani2411 Jun 24, 2025 •

edited

Loading

Uh oh!

himani2411 Jun 30, 2025

Uh oh!

himani2411 commented Jul 7, 2025

Uh oh!

Uh oh!

		- There were no changes for this version.
		- Support prioritized\|capacity-optimized-prioritized Allocation Strategy and EnableSingleAvailabilityZone

		self._compute_resource_config["Networking"]["SingleAvailabilityZone"]
		if self._compute_resource_config["Networking"]["SingleAvailabilityZone"] is not None

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #671

Are you sure you want to change the base?

[Subnet Prioritization] Support capacity-optimized-prioritized and prioritized Allocation Strategy #671

Uh oh!

Conversation

Allenz5 commented Jun 17, 2025

Description of changes

Tests

References

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

himani2411 Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

himani2411 commented Jul 7, 2025

Uh oh!

Uh oh!

himani2411 Jun 24, 2025 •

edited

Loading