State Machine: Implement check & Unit Tests for "StartAt"

[r-heimann]

Issue #, if available:

Fixes #4074

Description of changes:

This PR implements a check for

https://states-language.net/spec.html#toplevelfields

A State Machine MUST have a string field named "StartAt", whose value MUST match exactly the name of one of the "States" fields. The interpreter starts executing the machine at the named state.

Since i am not a professional Python programmer i used the help of a LLM to implement the check and the unit tests, while trying to keep it as simple as possible.
The cfn-lint "Error" messages are almost identical to the State Machine CloudFormation API, but i made it a little bit more human friendly to read.

I tested the following CloudFormation templates, which all displayed the error message in the correct State:

StartAt JSONata / JSONPath

  StartAtJSONata:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONata
        StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /StartAt
        States:
          Pass: # MISSING_TRANSITION_TARGET: State "Pass" is not reachable. at /States/Pass (not yet implemented)
            Type: Pass
            Next: Success
          Success:
            Type: Succeed

  StartAtJSONPath:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONPath
        StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /StartAt
        States:
          Pass: # MISSING_TRANSITION_TARGET: State "Pass" is not reachable. at /States/Pass (not yet implemented)
            Type: Pass
            Next: Success
          Success:
            Type: Succeed

Parallel StartAt JSONata / JSONPath

  ParallelStartAtJSONata:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONata
        StartAt: ParallelState
        States:
          ParallelState:
            Type: Parallel
            Branches:
              - StartAt: BranchState1
                States:
                  BranchState1:
                    Type: Pass
                    End: true
              - StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /States/ParallelState/Branches[1]/StartAt
                States:
                  BranchState2: # MISSING_TRANSITION_TARGET: State "BranchState2" is not reachable. at /States/ParallelState/Branches[1]/States/BranchState2 (not yet implemented)
                    Type: Pass
                    End: true
            End: true

  ParallelStartAtJSONPath:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONPath
        StartAt: ParallelState
        States:
          ParallelState:
            Type: Parallel
            Branches:
              - StartAt: BranchState1
                States:
                  BranchState1:
                    Type: Pass
                    End: true
              - StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /States/ParallelState/Branches[1]/StartAt
                States:
                  BranchState2: # MISSING_TRANSITION_TARGET: State "BranchState2" is not reachable. at /States/ParallelState/Branches[1]/States/BranchState2 (not yet implemented)
                    Type: Pass
                    End: true
            End: true

MapStartAt JSONata / JSONPath

  MapStartAtJSONata:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONata
        StartAt: MapState
        States:
          MapState:
            Type: Map
            ItemProcessor:
              ProcessorConfig:
                Mode: INLINE
              StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /States/MapStateItemProcessor/ItemProcessor/StartAt
              States:
                ItemProcessStart: # MISSING_TRANSITION_TARGET: State "ItemProcessStart" is not reachable. at /States/MapStateItemProcessor/ItemProcessor/States/ItemProcessStart (not yet implemented)
                  Type: Pass
                  End: true
            End: true


  MapStartAtJSONPath:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: !GetAtt StepFunctionRole.Arn
      Definition:
        QueryLanguage: JSONPath
        StartAt: MapState
        States:
          MapState:
            Type: Map
            ItemProcessor:
              ProcessorConfig:
                Mode: INLINE
              StartAt: FAIL # MISSING_TRANSITION_TARGET: Missing 'Next' target: FAIL at /States/MapStateItemProcessor/ItemProcessor/StartAt
              States:
                ItemProcessStart: # MISSING_TRANSITION_TARGET: State "ItemProcessStart" is not reachable. at /States/MapStateItemProcessor/ItemProcessor/States/ItemProcessStart (not yet implemented)
                  Type: Pass
                  End: true
            End: true

The PR also includes Unit Tests for

• StartAt
• Map StartAt
• Parallel StartAt

which were succesful in local testing.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[r-heimann]

@kddejong The line length limit is mandatory for cfn-lint code, right?

[tool.ruff]
# Enable pycodestyle (`E`) and Pyflakes (`F`) codes by default.
lint.select = ["E", "F"]
lint.ignore = []
line-length = 88

[kddejong]

Just a few checks to validate the types are we hope they are. Happy path says these are the right types but since we are linting we have to assume they could be wrong.

[r-heimann]

I have rebased my branch, implemented your requested changes and also run ruff over the changed file:
All checks passed!

Can you please trigger the pipeline again?

[r-heimann]

@kddejong

[r-heimann]

@kddejong Thank you, i have rebased the pull request and applied ruff format on both files. Can you please trigger the pipeline again?

[r-heimann]

@kddejong I believe the proposed change

-        if start_at is None:
+        if not isinstance(start_at, str) and not isinstance(states, dict):

caused the updated rule to not work properly and fail the following unittest:

unittest

(
    "Invalid string definition",
    {
        "Definition": (
            """
            {
                "States": {
                    "NoType": {}
                }
            }
        """
        )
    },
    [
        ValidationError(
            "'Type' is a required property at 'States/NoType'",
            rule=StateMachineDefinition(),
            validator="required",
            schema_path=deque(
                [
                    "properties",
                    "States",
                    "patternProperties",
                    "^.{1,80}$",
                    "required",
                ]
            ),
            path=deque(["Definition", "States", "NoType"]),
        ),
        ValidationError(
            "'StartAt' is a required property",
            rule=StateMachineDefinition(),
            validator="required",
            schema_path=deque(
                [
                    "required",
                ]
            ),
            path=deque(["Definition"]),
        ),
    ],
),

E       AssertionError: 'Invalid string definition' test failed with [<ValidationError: "'Type' is a required property at 'States/NoType'">, <ValidationError: "'StartAt' is a required property">, <ValidationError: "Missing 'Next' target 'None' at /StartAt">]
E       assert [<ValidationE...at /StartAt">] == [<ValidationE...ed property">]
E
E         Left contains one more item: <ValidationError: "Missing 'Next' target 'None' at /StartAt">
E
E         Full diff:
E           [
E               <ValidationError: "'Type' is a required property at 'States/NoType'">,
E               <ValidationError: "'StartAt' is a required property">,
E         +     <ValidationError: "Missing 'Next' target 'None' at /StartAt">,
E           ]

The rule should have ignored the state machine definition since it didn't contain a StartAt. I modified the code again:

-        if not isinstance(start_at, str) and not isinstance(states, dict):
+        if not isinstance(start_at, str) or not isinstance(states, dict):

and locally the state machine unittest, using

tox -e py313 -- test/unit/rules/resources/stepfunctions/test_state_machine_definition.py -v

was passing now. Could you please trigger the GitHub Actions pipeline again?

[r-heimann]

@kddejong actions/checkout@v5 failed with Error: fatal: unable to access 'https://github.com/aws-cloudformation/cfn-lint/': The requested URL returned error: 500 - i am unsure how i could affect this, maybe this was part of the Cloudflare outage. Could you please re-run the pipeline?

[r-heimann]

@kddejong Could you please restart the GitHub Actions pipeline?

[codecov]

Codecov Report

❌ Patch coverage is 92.59259% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.35%. Comparing base (7cd97ee) to head (89a1b35).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
.../resources/stepfunctions/StateMachineDefinition.py	92.59%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4264      +/-   ##
==========================================
- Coverage   94.36%   94.35%   -0.02%     
==========================================
  Files         414      414              
  Lines       13991    14018      +27     
  Branches     2776     2784       +8     
==========================================
+ Hits        13203    13227      +24     
- Misses        443      444       +1     
- Partials      345      347       +2     

Flag	Coverage Δ
unittests	94.34% <92.59%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.