Use datasources on variable validation

[somatorio]

Current Terraform Version

Terraform v0.14.8

Use-cases

As we can't use datasources at variable validation, in some cases we can only validate if that values seems like something correct, not if it IS correct...
Examples:
A)

variable "region" {
  type = string
  description = "AWS region where our cluster will be built on"
  validation {
    condition = can(regex("[a-z]{2}-[a-z]+-[0-9]",var.region))
    error_message = "Region must be one of the available regions for your account (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions)."
  }
}

Would validate if the region set looks like a valid region, not if it is an available region for this account

B)
The same case applies for the example at the docs (https://www.terraform.io/docs/language/values/variables.html#custom-validation-rules), it only validates that the variable value looks like a valid AMI Id, not if that value is indeed a valid AMI Id

variable "image_id" {
  type        = string
  description = "The id of the machine image (AMI) to use for the server."

  validation {
    # regex(...) fails if it cannot find a match
    condition     = can(regex("^ami-", var.image_id))
    error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"."
  }
}

Attempted Solutions

Using the same examples above... It would be much better if we could do something like:
A)

data "aws_regions" "regions_available" {}

variable "region" {
  type = string
  description = "AWS region where our cluster will be built on"
  validation {
    condition = can(regex(join("|",data.aws_regions.regions_available.names),var.region))
    error_message = "Region must be one of the available regions for your account (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions)."
  }
}

Then it would validate using available regions for the account
It would be even better if we could use functions and datasources at error_message, so it could be error_message = "Region must be one of the following regions: ${join(", ",data.aws_regions.regions_available.names)}." and so giving the user valid values on error.

B)

data "aws_ami_ids" "available" {
  owners = ["self", "amazon", "aws-marketplace"]
}

variable "image_id" {
  type        = string
  description = "The id of the machine image (AMI) to use for the server."

  validation {
    # regex(...) fails if it cannot find a match
    condition     = can(regex(join("|",data.aws_ami_ids.available.ids),var.image_id))
    error_message = "The image_id value must be a valid AMI id from your account, amazon or aws marketplace"
  }
}

Proposal

My proposal is to enable the use of other values than the actual var being tested, so we can test it against real values

References

[apparentlymart]

Thanks for sharing this use-case, @somatorio!

Terraform's validation phase (what terraform validate does, and also the first step of what terraform plan and terraform apply do) is an initial static validation that Terraform performs before trying to work further with the configuration, and so in the current design (where variable validation is implemented as part of this validation) it isn't possible to achieve exactly what you tried here, but we are considering other possible designs for different sorts of assertions beyond variable validation that could potentially occur in the plan or apply phases instead, and thus be able to take into account dynamic data that the validation phase cannot see.

Our current focus is on preparing the forthcoming v0.15 release (in its prerelease period at the time I'm writing this) so we aren't currently actively working on this, but I do hope we can return to this for some deeper design work in the not too far future.

[somatorio]

great to know that this is in your roadmap :D

but i just want to point out that variables input aren't validated at terraform validate, as far as i could see this is done at plan/apply

/app # cat terraform.tfvars
region = "lahksbjdfas"
#region = "us-east-1"
/app # cat variables.tf
data "aws_regions" "regions_available" {}

variable "region" {
  type = string
  description = "AWS region where our cluster will be built on"
  validation {
    condition = can(regex("[a-z]{2}-[a-z]+-[0-9]",var.region))
    error_message = "Region must be one of the available regions for your account (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions)."
  }
}
/app # terraform validate
Success! The configuration is valid.
/app # terraform plan

Error: Invalid value for variable

  on variables.tf line 3:
   3: variable "region" {

Region must be one of the available regions for your account
(https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).

This was checked by the validation rule at variables.tf:6,3-13.

[apparentlymart]

Hi @somatorio,

It's true that root module variable values can't be checked by terraform validate, because terraform validate doesn't support root module variables at all -- it's a command for verifying that the configuration is valid, not for verifying that a particular run is valid, and root module variable values are part of the arguments for a run rather than a part of the configuration.

This checking does still occur for variable values to child modules, because they are considered to be part of the configuration rather than a parameter of the operation, and Terraform does still perform this check during the "validate" step of planning, though the validate vs. plan step distinction admittedly isn't very visible in the UI for terraform plan, intentionally so.

[somatorio]

ohhh, good to know :)

[apparentlymart]

Hi all!

We've recently merged #34955 with the intention of including support for this in the forthcoming Terraform v1.9 release. For references to data resources in particular -- the focus of this issue -- there is an important caveat:

I mentioned above that the terraform validate command isn't able to read from any data sources, and so some sort of compromise would be required.

The intended compromise in this final implementation is that if an input variable validation rule refers to a data result that isn't available yet then terraform validate will effectively skip that validation rule, due to the condition expression evaluated to an unknown value -- what the plan output usually describes as "(known after apply)".

Terraform will then check the validation rule again during the planning phase once the data source result is available. Therefore the validation rule will still ultimately work, but referring to plan-time-only results automatically defers the validation check to the plan phase.

Thanks again for sharing this use-case!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.