DynamicListTerraformIterator: Sets of more than one object unhandled

[micchickenburger]

Description

Dynamic complex list iterations introduced in PR #3273 don't seem to handle sets containing more than one object. Consider a certificate request with more than one Subject Alternative Name, viz.:

const cert = new AcmCertificate(this, "cert", {
  domainName: "example.com",
  subjectAlternativeNames: ["www.example.com"], // tf will automatically add 'example.com' to this list
  validationMethod: "DNS",
});
const dataAwsRoute53ZoneExample = new DataAwsRoute53Zone(this, "dns_zone", {
  name: "example.com",
  privateZone: false,
});

const exampleForEachIterator = TerraformIterator.fromComplexList(
  cert.domainValidationOptions,
  "domain_name"
);

const records = new Route53Record(this, "record", {
  forEach: exampleForEachIterator,
  allowOverwrite: true,
  ttl: 60,
  name: exampleForEachIterator.getString("resource_record_name"),
  records: [exampleForEachIterator.getString("resource_record_value")],
  type: exampleForEachIterator.getString("resource_record_type"),
  zoneId: dataAwsRoute53ZoneExample.zoneId,
});

This results in the following new resource:

+ resource "aws_acm_certificate" "cert" {
  + arn                       = (known after apply)
  + domain_name               = "example.com"
  + domain_validation_options = [
      + {
          + domain_name           = "www.example.com"
          + resource_record_name  = (known after apply)
          + resource_record_type  = (known after apply)
          + resource_record_value = (known after apply)
        },
      + {
          + domain_name           = "example.com"
          + resource_record_name  = (known after apply)
          + resource_record_type  = (known after apply)
          + resource_record_value = (known after apply)
        },
    ]
  + id                        = (known after apply)
  + key_algorithm             = (known after apply)
  + not_after                 = (known after apply)
  + not_before                = (known after apply)
  + pending_renewal           = (known after apply)
  + renewal_eligibility       = (known after apply)
  + renewal_summary           = (known after apply)
  + status                    = (known after apply)
  + subject_alternative_names = [
      + "www.example.com",
      + "example.com",
    ]
  + type                      = (known after apply)
  + validation_emails         = (known after apply)
  + validation_method         = "DNS"
}

However, the Route53 records will fail to create:

Error: Invalid for_each argument

 on cdk.tf.json line 207, in resource.aws_route53_record.acm_records:
 207: "for_each": "${{ for key, val in tolist(aws_acm_certificate.cert.domain_validation_options): val.domain_name => val }}",
 ├────────────────
 │ aws_acm_certificate.cert.domain_validation_options is set of object with 2 elements

The "for_each" map includes keys derived from resource attributes that cannot
be determined until apply, and so Terraform cannot determine the full set of
keys that will identify the instances of this resource.

When working with unknown values in for_each, it's better to define the map
keys statically in your configuration and place apply-time results only in
the map values.

Alternatively, you could use the -target planning option to first apply only
the resources that the for_each value depends on, and then apply a second
time to fully converge.

If aws_acm_certificate.cert.domain_validation_options is a set of one object, then the code works correctly.

System information

$ cdktf debug
language: typescript
cdktf-cli: 0.20.8
node: v20.14.0
cdktf: 0.20.8
constructs: null
jsii: null
terraform: 1.9.4
arch: x64
os: linux 5.15.0-119-generic
providers
aws@ = 5.60 (LOCAL)
        terraform provider version: 5.60.0
@cdktf/provider-aws (PREBUILT)
        terraform provider version: 5.60.0 
        prebuilt provider version: 19.28.0
        cdktf version: ^0.20.0

Links

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate_validation?lang=typescript

• terraform-cdk/examples/typescript/documentation/iterators.ts

  Line 95 in 4f8c0c9

  const cert = new AcmCertificate(this, "cert", {

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[so0k]

seems related to actual bug in Terraform/Terraform Provider AWS:

• [Bug]: The "for_each" map includes keys derived from resource attributes that cannot be determined until apply terraform-provider-aws#27299
• for_each not working well when creating aws_route53_record terraform-provider-aws#14447
• ACM certificate issue when domain_name is known before apply but subject_alternative_names are not known before apply terraform-provider-aws#23260

if you know the validation options as part of your CDK code, best to generate each route53 validation record directly without iterator or foreach.

[aclemmensen]

@so0k how do you know the validation options up-front if you're using DNS validation? Aren't they always random values provided by AWS that cannot be pre-assigned at creation? If it's possible to assign them, do you have any information or examples on how to do this?

I'm also using values for domain_name that are known at plan time, yet I get the same error:

        var iter = TerraformIterator.FromComplexList(cert.DomainValidationOptions, "domain_name");
        
        var records = new Route53Record(this, "validation_record", new Route53RecordConfig
        {
            ForEach = iter,
            ZoneId = _opts.ZoneId,
            Name = iter.GetString("resource_record_name"),
            Type = iter.GetString("resource_record_type"),
            Records = [iter.GetString("resource_record_value")],
            Ttl = 60,
            AllowOverwrite = true,
        });

Resulting in output:

              + resource "aws_acm_certificate" "cert" {
                  + arn                       = (known after apply)
                  + domain_name               = "example.com"
                  + domain_validation_options = [
                      + {
                          + domain_name           = "example.com"
                          + resource_record_name  = (known after apply)
                          + resource_record_type  = (known after apply)
                          + resource_record_value = (known after apply)
                        },
                      + {
                          + domain_name           = "*.example.com"
                          + resource_record_name  = (known after apply)
                          + resource_record_type  = (known after apply)
                          + resource_record_value = (known after apply)
                        },
                    ]
                  + id                        = (known after apply)
                  + key_algorithm             = (known after apply)
...
...
            │ Error: Invalid for_each argument
            │
            │   on cdk.tf.json line 794, in resource.aws_route53_record.validation_record (validation_record):
            │  794:         "for_each": "${{ for key, val in tolist(aws_acm_certificate.cert (cert).domain_validation_options): val.domain_name => val }}",
            │     ├────────────────
            │     │ aws_acm_certificate.cert.domain_validation_options is set of object with 2 elements
            │
            │ The "for_each" map includes keys derived from resource attributes that
            │ cannot be determined until apply, and so Terraform cannot determine the
            │ full set of keys that will identify the instances of this resource.
            │
            │ When working with unknown values in for_each, it's better to define the map
            │ keys statically in your configuration and place apply-time results only in
            │ the map values.
            │
            │ Alternatively, you could use the -target planning option to first apply
            │ only the resources that the for_each value depends on, and then apply a
            │ second time to fully converge.

[so0k]

The order and count can be calculated from the Subject Name and SANS

https://github.com/envtio/base/blob/bf99747f44f0fed912084052aa54a4142f8df835/src/aws/edge/certificate.ts#L172-L193

so far my integration tests always pass, so this seems consistent...

[mhaley-tv]

This issue is particularly confusing as the CDK official doc uses this exact example to show how to use complex iterators.