(aws_stepfunctions): Overriding to_stete_json() fails if result_path=sfn.JsonPath.DISCARD

[wisaac407]

Overriding to_stete_json() on the DynamoUpdateItem construct raises an exception if result_path=sfn.JsonPath.DISCARD

Reproduction Steps

Run cdk synth on the provided sample

from aws_cdk import aws_dynamodb as dynamodb
from aws_cdk import aws_stepfunctions as sfn
from aws_cdk import aws_stepfunctions_tasks as tasks
from aws_cdk import core


class CustomDynamoUpdateItem(tasks.DynamoUpdateItem):
    def to_state_json(self):
        return super().to_state_json()


class TestStack(core.Stack):
    def __init__(self, scope: core.Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        table = dynamodb.Table(
            self,
            "Table",
            partition_key=dynamodb.Attribute(
                name="id", type=dynamodb.AttributeType.STRING
            ),
        )
        sfn.StateMachine(
            self,
            "StateMachine",
            definition=CustomDynamoUpdateItem(
                self,
                "Update Item",
                table=table,
                result_path=sfn.JsonPath.DISCARD,
                key={
                    "id": tasks.DynamoAttributeValue.from_string(
                        sfn.JsonPath.string_at("$.id")
                    )
                },
                update_expression="SET value=:value",
                expression_attribute_values={
                    ":value": tasks.DynamoAttributeValue.from_string(
                        sfn.JsonPath.string_at("$.value")
                    )
                },
            ),
        )

What did you expect to happen?

It should produce a valid cloudformation stack.

What actually happened?

It produces this error:

jsii.errors.JavaScriptError: 
  Error: Got 'undefined' for non-optional instance of {"type":{"primitive":"json"}}
      at nullAndOk (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:9318:23)
      at Object.deserialize (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:8880:25)
      at Kernel._toSandbox (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:8505:69)
      at /private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:9007:81
      at mapValues (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:9341:35)
      at Object.deserialize (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:9007:36)
      at Object.deserialize (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:8884:59)
      at Kernel._toSandbox (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:8505:69)
      at DynamoUpdateItem.value (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/tmp84la7qu9/lib/program.js:8333:41)
      at StateGraph.toGraphJson (/private/var/folders/m1/lyhg812d7wn8tgcxk8_xmhbm0000gp/T/jsii-kernel-6nqdST/node_modules/@aws-cdk/aws-stepfunctions/lib/state-graph.js:106:43)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 15, in <module>
    TestStack(app, "TestStack")
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 83, in __call__
    inst = super().__call__(*args, **kwargs)
  File "/Users/weaveri/test/test/test_stack.py", line 23, in __init__
    sfn.StateMachine(
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_runtime.py", line 83, in __call__
    inst = super().__call__(*args, **kwargs)
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/aws_cdk/aws_stepfunctions/__init__.py", line 5100, in __init__
    jsii.create(StateMachine, self, [scope, id, props])
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_kernel/__init__.py", line 287, in create
    obj.__jsii_ref__ = _callback_till_result(self, response, CreateResponse)
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_kernel/__init__.py", line 220, in _callback_till_result
    response = kernel.sync_complete(response.cbid, None, result, response_type)
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_kernel/__init__.py", line 386, in sync_complete
    return self.provider.sync_complete(
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 382, in sync_complete
    resp = self._process.send(_CompleteRequest(complete=request), response_type)
  File "/Users/weaveri/test/.venv/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 326, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: Got 'undefined' for non-optional instance of {"type":{"primitive":"json"}}
Subprocess exited with error 1

Environment

• CDK CLI Version : 1.103.0
• Framework Version: 1.103.0
• Node.js Version: v14.16.0
• OS : macOS Catalina
• Language (Version): Python 3.8.2

Other

I'm attempting to use States.Format() in the update expression to parameterize the index of a dynamodb list based on the index of a Map state. Because States.Format() isn't supported yet (#11286) my attempted work-around is to override to_stete_json(). However, that's producing the above error.

---

This is 🐛 Bug Report

[robjongbloed]

I am having the same problem overriding to_state_json in the LambdaInvoke task.

Error: Got \'undefined\' for non-optional instance of {"type":{"primitive":"json"}}
    at nullAndOk (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9531:23)
    at Object.deserialize (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9093:25)
    at Kernel._toSandbox (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:8718:69)
    at /private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9220:81
    at mapValues (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9554:35)
    at Object.deserialize (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9220:36)
    at Object.deserialize (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9097:59)
    at Kernel._toSandbox (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:8718:69)
    at mapJsonValue (/private/var/folders/wt/7pp002_x6l950ysl45b1k7ljl8ytr0/T/tmp_c0tz4nx/lib/program.js:9120:37)
    at Array.map (<anonymous>)

If interested, this is to add the unsupported "TimeoutSecondsPath" field.

[robjongbloed]

To assist anyone else with this issue, I have worked out what the cause is.

When you use JsonPath.DISCARD as the value for result_path, then that gets translated in the to_state_json() function as "ResultPath": None which then blows up the deserialiser in the TypeScript side. This bug should be fixed.

As a workaround, leaving the result_path out completely, or explicitly set it to None, seems to fix the issue, as then the field in to_state_json() is left out. Note, I think this means the result ends up in '$' which might blow something else up in your use case. You can also explicitly set it to something like "$.discardedResult".

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.