allow variable in provider field

[dmrzzz]

I'm using multiple AWS providers to deploy resources into different regions, but I want my modules to be generic with respect to which regions.

Ideally I would write a module like this:

variable provider { default = "aws" }

resource "aws_vpc" "vpc" {
    # works:
    #provider = "aws.us-east-1"
    # does not work:
    provider = "${var.provider}"
    cidr_block = "192.168.0.0/16"
    tags { Name = "TEST - DELETE ME" }
}

and invoke it like this:

provider "aws" {
  region = "us-east-2"
}

provider "aws" {
  alias = "us-east-1"
  region = "us-east-1"
}

# do other stuff using the default AWS provider

# tell this module to use aws.us-east-1
module "module1" {
  source = "./testmodule"
  provider = "aws.us-east-1"
}

but that fails on terraform get:

dmrz@golbez:~/tmp/tftest2$ terraform get
Get: file:///home/dmrz/tmp/tftest2/testmodule
Error loading Terraform: 1 error(s) occurred:

* module module1: provider alias must be defined by the module or a parent: ${var.provider}

It works fine if I instead hardcode the provider field value within the module, but then my module isn't generic.

Note: the resource I really want to use this for at this moment is aws_cloudwatch_metric_alarm, but aws_vpc behaves the same way and makes a much simpler test case.

Thanks for all the great work on Terraform!

[kevintsai]

I also ran into this issue. Hope it can be improved. Thanks.

[apparentlymart]

This is somewhat related to #1819.

[kkarimi]

This is quite necessary in order to work effectively with modules with a multi region AWS setup

[ericfode]

This is causing sadness for us.

[nyetsche]

Also causing sadness for us :(

Here's the example of what I'm hoping to accomplish:

resource "aws_kms_key" "cluster_cmk" {
  count       = "${var.cluster_number}"
  provider = "aws.${element(values(var.regionmap), count.index)}"
}

provider "aws" {
  alias = "us-east-1"
  region = "us-east-1"
}

regionmap = {
  "clusternickname" = "us-east-1"
}

In other words, have multiple KMS keys, one per cluster, and each cluster could be in a different region. So do a map lookup.

Fails with this:

* module root: 1 error(s) occurred:

* module : provider alias must be defined by the module or a parent: aws.${element(values(var.regionmap), count.index)}

I poked around the code but couldn't figure out where a resource overrides the default provider (where the fix should go). If someone has hints, it's important enough I'd like to hack on a PR

[demonwy]

+1

[gillingw]

This is an issue for us too having to duplicate modules for each region.

[apparentlymart]

Hi all,

I'm sorry there hasn't been any movement here. This is a tricky issue to resolve because we need to understand provider usage very early on in the Terraform run, before we have enough context to do interpolations. So far a way to implement this as requested has eluded us.

A different take on the problem might be simpler to implement. I've not thought this through all the way yet so I can't promise this is how this will eventually end up, but just wanted to capture this here for future reference and feedback:

module "foo" {
  source = "./foo"

  # ...module arguments, as usual...

  providers {
    aws = "aws.use1"
  }
}

This hypothetical (not yet working!) providers block could allow customizing how providers inherit so that a provider with an alias in the parent module can be unaliased in the child module. By handing this in a first-class way we'd avoid the need to support arbitrary interpolations for it and thus be able to resolve it early in configuration processing.

I understand that this alternative approach is still not as general as the interpolation-based solution requested, but I think it would be adequate to meet the use-case of giving a child module access to an aliased provider without the child module itself needing to know about the alias.

[dmrzzz]

@apparentlymart, your idea above would solve my immediate problem.

That said, I keep thinking about how nice it would be to have some sort of limited interpolation support available earlier in the run, such that many parameters which currently allow no interpolation at all could instead allow limited interpolation (almost like a preprocessor macro replacement step, but using the same familiar interpolation syntax).

Obviously we wouldn't be able to interpolate anything at this early stage whose value depends on another resource or data source, but for my OP example code the value in question is knowable as soon as we parse the files, because it's already hardcoded elsewhere in those same files. Being able to interpolate such locally-known variable values early on would, from my perspective, cleanly solve not only this issue but also e.g. #13022 (and maybe some others too) without having to introduce new first-class syntax.

[jbarreneche]

Hi! The overriding provisioner in the module would work for our use case and would be more clean that what we are currently doing.

Just for the record, for the people who need to handle multiple regions or multiple aws accounts this works:

In the module

provider "aws" {
  alias = "module"
  region = "${var.region}" # Or some way to infer the region from the module variables
  # This is in case you want to use multiple accounts
  assume_role {
    role_arn = "arn:aws:iam::${var.account)}:role/terraform"  # Or some way to infer the account number from the module variables, we use a map for naming the accounts.
  }
}
resource "aws_..." "foo" {
  provider = "aws.module"
  ...
}

The assume_role is optional in case you use multiple accounts (and needs all accounts to have a terraform role).
Sadly you can't forget to put the provider (which is a common mistake), and you can't override the "default" or use another that's in use (As far as we tested it). But it allows you to use a module in the same tfstate with multiple accounts/regions.

module "foo" {
  account = "12312312"
  region = "us-east-1"
}

module "bar" {
  account = "32132131"
  region = "sa-east-1"
}

[gillingw]

Thank you @jbarreneche this is working for us.

[paulcollinsiii]

So @jbarreneche first up thank you for that work around. For creating resources it works perfectly.
The downside to it comes when I'm trying to tear down a module that was created with this. At least with version 0.9.5 when I tried to remove a module that included a

provider "aws" {
  region = "${var.aws_region}"
  alias = "aws.module"
}

Then when starting a plan that would remove that module terraform would ask what region it applied to and then barf during the apply. If I went about it doing a
terraform plan -out=tf.plan -target module.mymodule -destroy
applied that, and THEN removed the module "mymodule { ... } line it was okay.

Anyhow, just a note for future me when I run into this gotcha again.