terraform show does not properly associate resource configuration with provider when proxied

[DaKaZ]

I am not sure if this is a bug or a feature request :)

When looking at the JSON output from a terraform show, we cannot associate the resource's provider_config_key with the actual provider when we have a module using proxied providers. In this below example, the two resources in the good module have different AWS providers. If I wanted to determine which REGION each resource was going to be created in, I would need to know which providers are mapped into the module proxies. The current output shows (abbreviated):

"resources": [
  { 
    "address": "aws_dynamodb_table.first_table",
    "provider_config_key": "good:aws" 
  },
  { 
    "address": "aws_dynamodb_table.second_table",
    "provider_config_key": "good:aws.second" 
  }
]

But the provider_configs only shows:

        "provider_config": {
            "aws": {
                "name": "aws",
                "expressions": {
                    "region": {
                        "constant_value": "us-east-1"
                    }
                }
            },
            "aws.west2": {
                "name": "aws",
                "alias": "west2",
                "expressions": {
                    "region": {
                        "constant_value": "us-west-2"
                    }
                }
            },
            "module.good:aws": {
                "name": "aws",
                "version_constraint": ">= 2.7.0",
                "module_address": "module.good"
            }
        },

I believe that provider_config["aws"] and provider_config["aws.west2"] need a new key that maintains a list of which modules and the respective provider aliases they are mapped to. I am thinking something like:

"proxies": [ "good:aws"]

Terraform Version

Terraform v1.0.11
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.68.0
+ provider registry.terraform.io/hashicorp/time v0.7.2

Terraform Configuration Files

main.tf:

provider aws {
  region = "us-east-1"
}

provider aws {
  alias = "west2"
  region = "us-west-2"
}

variable good_count {
  default = 1
}

module good {
  count = var.good_count
  source = "./good"
  providers = {
    aws = aws
    aws.second = aws.west2
  }
}

./good/main.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 2.7.0"
      configuration_aliases = [ aws, aws.second ]
    }
  }
}

resource aws_dynamodb_table first_table {
  name = "first-table"
  hash_key = "pk"
  attribute {
    name = "pk"
    type = "S"
  }
  provider = aws
}

resource aws_dynamodb_table second_table {
  name = "second-table"
  hash_key = "pk"
  attribute {
    name = "pk"
    type = "S"
  }
  provider = aws.second
}

Steps to Reproduce

1. terraform init
2. terraform plan -out plan.out
3. terraform show -json plan.out > plan.json

Additional Information

Here is the complete plan JSON for reference:

{
    "format_version": "0.2",
    "terraform_version": "1.0.11",
    "variables": {
        "good_count": {
            "value": 1
        }
    },
    "planned_values": {
        "root_module": {
            "child_modules": [
                {
                    "resources": [
                        {
                            "address": "module.good[0].aws_dynamodb_table.first_table",
                            "mode": "managed",
                            "type": "aws_dynamodb_table",
                            "name": "first_table",
                            "provider_name": "registry.terraform.io/hashicorp/aws",
                            "schema_version": 1,
                            "values": {
                                "attribute": [
                                    {
                                        "name": "pk",
                                        "type": "S"
                                    }
                                ],
                                "billing_mode": "PROVISIONED",
                                "global_secondary_index": [],
                                "hash_key": "pk",
                                "local_secondary_index": [],
                                "name": "first-table",
                                "range_key": null,
                                "read_capacity": null,
                                "replica": [],
                                "stream_enabled": null,
                                "tags": null,
                                "timeouts": null,
                                "ttl": [],
                                "write_capacity": null
                            },
                            "sensitive_values": {
                                "attribute": [
                                    {}
                                ],
                                "global_secondary_index": [],
                                "local_secondary_index": [],
                                "point_in_time_recovery": [],
                                "replica": [],
                                "server_side_encryption": [],
                                "tags_all": {},
                                "ttl": []
                            }
                        },
                        {
                            "address": "module.good[0].aws_dynamodb_table.second_table",
                            "mode": "managed",
                            "type": "aws_dynamodb_table",
                            "name": "second_table",
                            "provider_name": "registry.terraform.io/hashicorp/aws",
                            "schema_version": 1,
                            "values": {
                                "attribute": [
                                    {
                                        "name": "pk",
                                        "type": "S"
                                    }
                                ],
                                "billing_mode": "PROVISIONED",
                                "global_secondary_index": [],
                                "hash_key": "pk",
                                "local_secondary_index": [],
                                "name": "second-table",
                                "range_key": null,
                                "read_capacity": null,
                                "replica": [],
                                "stream_enabled": null,
                                "tags": null,
                                "timeouts": null,
                                "ttl": [],
                                "write_capacity": null
                            },
                            "sensitive_values": {
                                "attribute": [
                                    {}
                                ],
                                "global_secondary_index": [],
                                "local_secondary_index": [],
                                "point_in_time_recovery": [],
                                "replica": [],
                                "server_side_encryption": [],
                                "tags_all": {},
                                "ttl": []
                            }
                        }
                    ],
                    "address": "module.good[0]"
                }
            ]
        }
    },
    "resource_changes": [
        {
            "address": "module.good[0].aws_dynamodb_table.first_table",
            "module_address": "module.good[0]",
            "mode": "managed",
            "type": "aws_dynamodb_table",
            "name": "first_table",
            "provider_name": "registry.terraform.io/hashicorp/aws",
            "change": {
                "actions": [
                    "create"
                ],
                "before": null,
                "after": {
                    "attribute": [
                        {
                            "name": "pk",
                            "type": "S"
                        }
                    ],
                    "billing_mode": "PROVISIONED",
                    "global_secondary_index": [],
                    "hash_key": "pk",
                    "local_secondary_index": [],
                    "name": "first-table",
                    "range_key": null,
                    "read_capacity": null,
                    "replica": [],
                    "stream_enabled": null,
                    "tags": null,
                    "timeouts": null,
                    "ttl": [],
                    "write_capacity": null
                },
                "after_unknown": {
                    "arn": true,
                    "attribute": [
                        {}
                    ],
                    "global_secondary_index": [],
                    "id": true,
                    "local_secondary_index": [],
                    "point_in_time_recovery": true,
                    "replica": [],
                    "server_side_encryption": true,
                    "stream_arn": true,
                    "stream_label": true,
                    "stream_view_type": true,
                    "tags_all": true,
                    "ttl": []
                },
                "before_sensitive": false,
                "after_sensitive": {
                    "attribute": [
                        {}
                    ],
                    "global_secondary_index": [],
                    "local_secondary_index": [],
                    "point_in_time_recovery": [],
                    "replica": [],
                    "server_side_encryption": [],
                    "tags_all": {},
                    "ttl": []
                }
            }
        },
        {
            "address": "module.good[0].aws_dynamodb_table.second_table",
            "module_address": "module.good[0]",
            "mode": "managed",
            "type": "aws_dynamodb_table",
            "name": "second_table",
            "provider_name": "registry.terraform.io/hashicorp/aws",
            "change": {
                "actions": [
                    "create"
                ],
                "before": null,
                "after": {
                    "attribute": [
                        {
                            "name": "pk",
                            "type": "S"
                        }
                    ],
                    "billing_mode": "PROVISIONED",
                    "global_secondary_index": [],
                    "hash_key": "pk",
                    "local_secondary_index": [],
                    "name": "second-table",
                    "range_key": null,
                    "read_capacity": null,
                    "replica": [],
                    "stream_enabled": null,
                    "tags": null,
                    "timeouts": null,
                    "ttl": [],
                    "write_capacity": null
                },
                "after_unknown": {
                    "arn": true,
                    "attribute": [
                        {}
                    ],
                    "global_secondary_index": [],
                    "id": true,
                    "local_secondary_index": [],
                    "point_in_time_recovery": true,
                    "replica": [],
                    "server_side_encryption": true,
                    "stream_arn": true,
                    "stream_label": true,
                    "stream_view_type": true,
                    "tags_all": true,
                    "ttl": []
                },
                "before_sensitive": false,
                "after_sensitive": {
                    "attribute": [
                        {}
                    ],
                    "global_secondary_index": [],
                    "local_secondary_index": [],
                    "point_in_time_recovery": [],
                    "replica": [],
                    "server_side_encryption": [],
                    "tags_all": {},
                    "ttl": []
                }
            }
        }
    ],
    "configuration": {
        "provider_config": {
            "aws": {
                "name": "aws",
                "expressions": {
                    "region": {
                        "constant_value": "us-east-1"
                    }
                }
            },
            "aws.west2": {
                "name": "aws",
                "alias": "west2",
                "expressions": {
                    "region": {
                        "constant_value": "us-west-2"
                    }
                }
            },
            "module.good:aws": {
                "name": "aws",
                "version_constraint": ">= 2.7.0",
                "module_address": "module.good"
            }
        },
        "root_module": {
            "module_calls": {
                "good": {
                    "source": "./good",
                    "count_expression": {
                        "references": [
                            "var.good_count"
                        ]
                    },
                    "module": {
                        "resources": [
                            {
                                "address": "aws_dynamodb_table.first_table",
                                "mode": "managed",
                                "type": "aws_dynamodb_table",
                                "name": "first_table",
                                "provider_config_key": "good:aws",
                                "expressions": {
                                    "attribute": [
                                        {
                                            "name": {
                                                "constant_value": "pk"
                                            },
                                            "type": {
                                                "constant_value": "S"
                                            }
                                        }
                                    ],
                                    "hash_key": {
                                        "constant_value": "pk"
                                    },
                                    "name": {
                                        "constant_value": "first-table"
                                    }
                                },
                                "schema_version": 1
                            },
                            {
                                "address": "aws_dynamodb_table.second_table",
                                "mode": "managed",
                                "type": "aws_dynamodb_table",
                                "name": "second_table",
                                "provider_config_key": "good:aws.second",
                                "expressions": {
                                    "attribute": [
                                        {
                                            "name": {
                                                "constant_value": "pk"
                                            },
                                            "type": {
                                                "constant_value": "S"
                                            }
                                        }
                                    ],
                                    "hash_key": {
                                        "constant_value": "pk"
                                    },
                                    "name": {
                                        "constant_value": "second-table"
                                    }
                                },
                                "schema_version": 1
                            }
                        ]
                    }
                }
            },
            "variables": {
                "good_count": {
                    "default": 1
                }
            }
        }
    }
}

[DaKaZ]

For discussion: I see three ways to solve this problem:

Solution 1

Update the Provider struct in the internal/configs/provider.go to contain

Proxies []string

where we could stuff the proxy provider address like good:aws.second when that provider is mapped into a proxy.

Solution 2

Update the provider_config_key on the resource to contain the ACTUAL provider used with this resources after all mapping/proxies. So the resource config section for the two resources would look like:

"resources": [
  { 
    "address": "aws_dynamodb_table.first_table",
    "provider_config_key": "aws" 
  },
  { 
    "address": "aws_dynamodb_table.second_table",
    "provider_config_key": "aws.west2" 
  }
]

Solution 3

Update the configuration for a module to include the provider mappings in the JSON output. If we had this, the output might look something like this:

        "root_module": {
            "module_calls": {
                "good": {
                    "providers": {
                      "aws": "aws",
                      "aws.second": "aws.west2" 
                    },
         // ...
        }

Thoughts? We are leaning toward solution 3

[alisdair]

Hi @DaKaZ, thanks for the detailed report! This does seem like an enhancement rather than a bug, so I've marked it as such.

Solution 3 makes the most sense to me also. Due to compatibility guarantees, we can't change the meaning of any of the existing fields in the JSON output, but adding a new providers field with the structure you suggest should be acceptable.

It would be useful to know more about your use case for this, to ensure that any changes we make to the JSON are as robust to that as possible. If you can share any more details that would help. Thanks!

[DaKaZ]

Hi @alisdair - we use the JSON output against Policy as Code (similar to Sentinel). Example rules might be:

• Ensure logs are stored in a different region
• Resource only allowed in a specific region (ie: govcloud)

[DaKaZ]

Happy new year @alisdair - please see a proposed PR for this issue: #30284