Problems referencing a new resource using for_each

[kinghuang]

I added the aws_iam_policy and aws_iam_role_policy_attachment resources shown below to an existing Terraform configuration. They both use for_each to create a resource per entry in the local.node_group_names_with_rexray_plugins map.

During plan, I'm getting an Invalid index error. The aws_iam_role_policy_attachment is saying that a aws_iam_policy resource doesn't already exist. But, it should be created as part of this plan & apply.

Terraform Version

Terraform v0.12.10
+ provider.archive v1.3.0
+ provider.aws v2.31.0
+ provider.dns v2.2.0
+ provider.local v1.4.0
+ provider.template v2.1.2
+ provider.tls v2.1.1

Terraform Configuration Files

Added the following:

resource "aws_iam_policy" "rexray" {
  for_each = local.node_group_names_with_rexray_plugins

  name        = "${module.cluster.config.cluster_name}-${each.key}-rexray"
  description = "REX-Ray policies for Docker volumes"
  policy      = data.aws_iam_policy_document.rexray[each.key].json
}

resource "aws_iam_role_policy_attachment" "rexray" {
  for_each = local.node_group_names_with_rexray_plugins

  role       = local.node_group_iam_roles[each.key].name
  policy_arn = aws_iam_policy.rexray[each.key].arn
}

Debug Output

Debug Output

2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": entering dynamic subgraph
2019/10/08 15:21:33 [TRACE] dag/walk: updating graph
2019/10/08 15:21:33 [TRACE] dag/walk: added new vertex: "aws_iam_role_policy_attachment.rexray"
2019/10/08 15:21:33 [TRACE] dag/walk: added new vertex: "root"
2019/10/08 15:21:33 [TRACE] dag/walk: added new vertex: "aws_iam_role_policy_attachment.rexray[\"metadata\"]"
2019/10/08 15:21:33 [TRACE] dag/walk: added edge: "root" waiting on "aws_iam_role_policy_attachment.rexray[\"metadata\"]"
2019/10/08 15:21:33 [TRACE] dag/walk: added edge: "root" waiting on "aws_iam_role_policy_attachment.rexray"
2019/10/08 15:21:33 [TRACE] dag/walk: dependencies changed for "root", sending new deps
2019/10/08 15:21:33 [TRACE] dag/walk: visiting "aws_iam_role_policy_attachment.rexray[\"metadata\"]"
2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray[\"metadata\"]": starting visit (*terraform.NodeRefreshableManagedResourceInstance)
2019/10/08 15:21:33 [TRACE] NodeRefreshableManagedResourceInstance: aws_iam_role_policy_attachment.rexray["metadata"] has no existing state to refresh
2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray[\"metadata\"]": evaluating
2019/10/08 15:21:33 [TRACE] [walkRefresh] Entering eval tree: aws_iam_role_policy_attachment.rexray["metadata"]
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalSequence
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalGetProvider
2019/10/08 15:21:33 [TRACE] dag/walk: visiting "aws_iam_role_policy_attachment.rexray"
2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": starting visit (*terraform.NodeRefreshableManagedResourceInstance)
2019/10/08 15:21:33 [TRACE] NodeRefreshableManagedResourceInstance: aws_iam_role_policy_attachment.rexray will be refreshed
2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": evaluating
2019/10/08 15:21:33 [TRACE] [walkRefresh] Entering eval tree: aws_iam_role_policy_attachment.rexray
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalSequence
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalGetProvider
2019/10/08 15:21:33 [TRACE] EvalWriteOutput: Saving value for output.nodes in state
2019-10-08T15:21:33.819Z [DEBUG] plugin: plugin process exited: path=/Users/king.huang/Repositories/gitlab.transzap.com/spendanalytics/kafka/sa-kafka-ext/.terraform/plugins/linux_amd64/terraform-provider-local_v1.4.0_x4 pid=381
2019-10-08T15:21:33.819Z [DEBUG] plugin: plugin exited
2019/10/08 15:21:33 [TRACE] [walkRefresh] Exiting eval tree: provider.local (close)
2019/10/08 15:21:33 [TRACE] vertex "provider.local (close)": visit complete
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalReadState
2019/10/08 15:21:33 [TRACE] EvalReadState: reading state for aws_iam_role_policy_attachment.rexray["metadata"]
2019/10/08 15:21:33 [TRACE] EvalReadState: no state present for aws_iam_role_policy_attachment.rexray["metadata"]
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalDiff
2019/10/08 15:21:33 [TRACE] GetResourceInstance: aws_iam_policy.rexray is a single instance
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalReadState
2019/10/08 15:21:33 [TRACE] EvalReadState: reading state for aws_iam_role_policy_attachment.rexray
2019/10/08 15:21:33 [TRACE] UpgradeResourceState: schema version of aws_iam_role_policy_attachment.rexray is still 0; calling provider "aws" for any other minor fixups
2019/10/08 15:21:33 [TRACE] GRPCProvider: UpgradeResourceState
2019/10/08 15:21:33 [ERROR] <root>: eval: *terraform.EvalDiff, err: Invalid index: The given key does not identify an element in this collection value.
2019/10/08 15:21:33 [ERROR] <root>: eval: *terraform.EvalSequence, err: Invalid index: The given key does not identify an element in this collection value.
2019/10/08 15:21:33 [TRACE] [walkRefresh] Exiting eval tree: aws_iam_role_policy_attachment.rexray["metadata"]
2019/10/08 15:21:33 [TRACE] vertex "aws_iam_role_policy_attachment.rexray[\"metadata\"]": visit complete
2019/10/08 15:21:33 [TRACE] <root>: eval: *terraform.EvalRefresh
2019/10/08 15:21:33 [TRACE] GRPCProvider: ReadResource
aws_iam_role_policy_attachment.rexray: Refreshing state... [id=sa-kafka-ext-metadata-2019081918572918300000000b]
2019/10/08 15:21:33 [TRACE] EvalWriteOutput: Saving Create change for output.nodes in changeset
2019/10/08 15:21:33 [TRACE] [walkRefresh] Exiting eval tree: output.nodes
2019/10/08 15:21:33 [TRACE] vertex "output.nodes": visit complete
2019/10/08 15:21:34 [TRACE] <root>: eval: *terraform.EvalWriteState
2019/10/08 15:21:34 [TRACE] EvalWriteState: writing current state object for aws_iam_role_policy_attachment.rexray
2019/10/08 15:21:34 [TRACE] [walkRefresh] Exiting eval tree: aws_iam_role_policy_attachment.rexray
2019/10/08 15:21:34 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": visit complete
2019/10/08 15:21:34 [TRACE] dag/walk: upstream of "root" errored, so skipping
2019/10/08 15:21:34 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": dynamic subgraph encountered errors
2019/10/08 15:21:34 [TRACE] vertex "aws_iam_role_policy_attachment.rexray": visit complete
2019/10/08 15:21:34 [TRACE] dag/walk: upstream of "provider.aws (close)" errored, so skipping
2019/10/08 15:21:34 [TRACE] dag/walk: upstream of "root" errored, so skipping

Error: Invalid index

  on iam.tf line 110, in resource "aws_iam_role_policy_attachment" "rexray":
 110:   policy_arn = aws_iam_policy.rexray[each.key].arn
    |----------------
    | aws_iam_policy.rexray is object with 7 attributes
    | each.key is "metadata"

The given key does not identify an element in this collection value.

2019-10-08T15:21:34.234Z [DEBUG] plugin: plugin process exited: path=/Users/king.huang/Repositories/gitlab.transzap.com/spendanalytics/kafka/sa-kafka-ext/.terraform/plugins/linux_amd64/terraform-provider-aws_v2.31.0_x4 pid=395
2019-10-08T15:21:34.235Z [DEBUG] plugin: plugin exited

Crash Output

N/A

Expected Behavior

aws_iam_policy and aws_iam_role_policy_attachment resources should be created, based on the local.node_group_names_with_rexray_plugins map.

Actual Behavior

Terraform reports that the aws_iam_policy policy "metadata" that aws_iam_role_policy_attachment is referencing doesn't already exist. But, it should be created as part of this plan & apply.

Steps to Reproduce

I had an existing Terraform configuration that I added the above two resources to.

If I destroy the rest of the resources in the project and apply everything together, then no error occurs.

Additional Context

N/A

References

N/A

[Richard-Payne]

I'm seeing the same issue. If I comment out the dependent resources, run terraform to create the roles and then uncomment the dependents and build them, it all works.

From a clean state, it complains about the role collection key. I tried adding an explicit depends_on but that didn't help.

[pselle]

Thanks for providing your debug output, @kinghuang. I'm having difficulty replicating this with a simplified use case, since I'm missing more of your config -- perhaps some of the issue lies there? I will add that when you get the invalid index like this:

Error: Invalid index

  on iam.tf line 110, in resource "aws_iam_role_policy_attachment" "rexray":
 110:   policy_arn = aws_iam_policy.rexray[each.key].arn
    |----------------
    | aws_iam_policy.rexray is object with 7 attributes
    | each.key is "metadata"

The given key does not identify an element in this collection value.

Note that the output tells you that the resource is an object with 7 attributes -- that signals to me that this is likely pointing to an individual resource somehow, and of course, your single resource does not have a key of the name you're looking for -- you're looking for an instance keyed with that name.

This line in your debug log:

2019/10/08 15:21:33 [TRACE] GetResourceInstance: aws_iam_policy.rexray is a single instance

Also makes me wonder if you still see this issue with an upgrade to the latest (now 0.12.13) version of Terraform? We made a fairly significant change in how we fetch resources, namely #22846, that could possibly impact this.

Additionally, thanks for linking the other similar issue to yours :)

[kinghuang]

I'll give this a try in the latest release and report back, hopefully by next week!

[pselle]

@Richard-Payne if you're interested/able, I'd appreciate sharing a config (ideally simplified, and of course, less any secrets) that replicates this case.

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.