0.12.0-rc1 - import fails if default provider is missing

[a13xb]

Summary

Import of an AWS resource will fail if a default (non-aliased) provider is not present, even if all resources are using a valid aliased provider.

Import of the same resource succeeds if:

1. a default provider is added, even if it is completely unrelated and unused, e.g. configured with a different region than the resource imported
2. AWS_DEFAULT_REGION environment variable is set to any region value, even if it's a different region than the resource imported

0.11 has a slightly different behaviour in that it's blocking on an interactive prompt asking for a region value. I can specify any region value and the import will proceed. I am assuming it's related, but in 0.12 it's immediate "hard" failure.

Terraform Version

Terraform v0.12.0-rc1
+ provider.aws v1.60.0-dev20190216h00-dev

Terraform Configuration Files

provider aws {
  alias = "sydney"
  region = "ap-southeast-2"
  profile = "development"
}

resource aws_vpc main {
  provider = "aws.sydney"
  cidr_block = "10.0.0.0/24"
}

resource aws_route_table main {
  provider = "aws.sydney"
  vpc_id = aws_vpc.main.id
}

Steps to Reproduce

1. Create resources outside of Terraform
2. terraform init
3. terraform import -provider=aws.sydney aws_route_table.main rtb-XXX

Expected Behavior

Import finishes successfully.

Actual Behavior

After refreshing the resource state correctly, import fails anyway.

aws_route_table.main: Importing from ID "rtb-XXX"...
aws_route_table.main: Import complete!
  Imported aws_route_table
  Imported aws_route
  Imported aws_route
  Imported aws_route
  Imported aws_route
  Imported aws_route_table_association
aws_route_table_association.main: Refreshing state... [id=rtbassoc-YYY]
aws_route.main: Refreshing state... [id=r-rtb-XXX1111111111]
aws_route.main-2: Refreshing state... [id=r-rtb-XXX222222222]
aws_route.main-3: Refreshing state... [id=r-rtb-XXX3333333333]
aws_route_table.main: Refreshing state... [id=rtb-XXX]
aws_route.main-1: Refreshing state... [id=r-rtb-XXX555555555]

Error: Missing required argument

The argument "region" is required, but was not set.

Debug Output

The only excerpt that seems to be relevant happens before the import starts:

...
BuiltinEvalContext: Initialized "aws" provider for provider.aws
<root>: eval: *terraform.EvalOpFilter
<root>: eval: *terraform.EvalSequence
<root>: eval: *terraform.EvalGetProvider
 [INFO]  plugin: configuring client automatic mTLS
...
BuiltinEvalContext: Initialized "aws" provider for provider.aws.sydney
<root>: eval: *terraform.EvalOpFilter
<root>: eval: *terraform.EvalSequence
<root>: eval: *terraform.EvalGetProvider
...
<root>: eval: *terraform.EvalSequence
<root>: eval: *terraform.EvalConfigProvider
buildProviderConfig for provider.aws: no configuration at all
GRPCProvider: GetSchema
buildProviderConfig for provider.aws.sydney: using explicit config only
GRPCProvider: GetSchema
<root>: eval: *terraform.EvalConfigProvider, non-fatal err: Missing required argument: The argument "region" is required, but was not set.
<root>: eval: *terraform.EvalSequence, err: Missing required argument: The argument "region" is required, but was not set.
<root>: eval: *terraform.EvalOpFilter, err: Missing required argument: The argument "region" is required, but was not set.
<root>: eval: *terraform.EvalSequence, err: Missing required argument: The argument "region" is required, but was not set.
[walkImport] Exiting eval tree: provider.aws
vertex "provider.aws": visit complete
dag/walk: upstream of "aws_vpc.main" errored, so skipping
dag/walk: upstream of "aws_route_table.main" errored, so skipping
dag/walk: upstream of "provider.aws (close)" errored, so skipping

And then a few seconds later after the import has finished:

[walkImport] Exiting eval tree: provider.aws.sydney (close)
vertex "provider.aws.sydney (close)": visit complete
dag/walk: upstream of "root" errored, so skipping

[a13xb]

Additionally, if I add a default provider block without region configured:

provider aws {
}

Then the error becomes more clear:

Error: Missing required argument

  on /path/to/main.tf line 1, in provider "aws":
   1: provider aws {

The argument "region" is required, but no definition was found.

But it still happens at the same stage: after the import process seems to have succeeded and all child resources have been found.

[apparentlymart]

Hi @a13xb! Thanks for reporting this.

Indeed, it looks like for some reason the import process is unnecessarily adding the default (unaliased) aws provider into the graph even though the configuration doesn't use it. Off the top of my head I can't think of why it would need to add that provider node, so I think we ought to be able to arrange for it to not be added at all and thus not need any configuration.

The import graph builder seems to be using the same TransformProviders function for its provider node management that all of the other walks do. Its intended behavior is to walk the configuration looking for provider blocks and provider meta-arguments in resources and create nodes for each of them. It also adds the default nodes for each provider in the root module, but is also intended to prune them back out once it detects that a particular one isn't used, so possibly that last step isn't functioning correctly in the import scenario for some reason.

Since this bug is pre-existing in Terraform v0.11 (albeit with some slightly different symptoms, because the prompting behavior was refactored for v0.12), it's likely that this will require some digging to fully diagnose and fix. Because we're close to v0.12.0 final now we're going to hold this for a follow-up v0.12.x patch release so we can make sure we have enough time for full debugging and testing since this will be a change to behavior that has probably been present since the -provider option was added back in Terraform 0.8.

Thanks for reporting this!

[uovobw]

this issue is still very much present in the final terraform version, i currently have:

Terraform v0.12.0
+ provider.acme v1.3.1
+ provider.aws v2.12.0
+ provider.template v2.1.2
+ provider.tls v2.0.1

and with the provider configuration

provider "acme" {
  alias      = "prod"
  server_url = "https://acme-v02.api.letsencrypt.org/directory"
}

provider "acme" {
  alias      = "staging"
  server_url = "https://acme-staging-v02.api.letsencrypt.org/directory"
}

the import of a very simple AWS EBS volume with

terraform import aws_ebs_volume.wordpress_root vol-01234567890

fails with

aws_ebs_volume.wordpress_root: Importing from ID "vol-01234567890"...
aws_ebs_volume.wordpress_root: Import complete!
  Imported aws_ebs_volume
aws_ebs_volume.wordpress_root: Refreshing state... [id=vol-01234567890]

Error: Missing required argument

The argument "server_url" is required, but was not set.

(volume id edited)

are there any workarounds?

[uovobw]

replying to my previous comment here, it appears the issue can be solved by specifying a default provider for each provider type. i have moved the acme.staging provider to be the acme one (so instead of provider="acme.prod" and provider="acme.staging" i now have provider="acme" and provider="acme.prod") and now the import process works.

[ozbillwang]

Any work around of this issue? Can't pass the validate check

$ terraform12 validate

Error: Missing required argument

The argument "region" is required, but was not set.

Got the same today when convert a module to 0.12.1

Updates

A quick fix, add below code.

variable "region" {
  description = "region"
  type        = string
  default     = "us-east-2"
}

provider "aws" {
  region = var.region
}

[WraithCadmus]

I'm having what I believe is a similar issue with a vmware provider. We leave user and password blank so they don't get committed to source. However I don't get that prompt when I try to import a vm

terraform import module.foobar.vsphere_virtual_machine.vmware_vm 1234-6578-90ab

Error: Missing required argument

  on /home/warith.cadmus/terraform/qa/provider.tf line 1, in provider "vsphere":
   1: provider "vsphere" {

The argument "password" is required, but no definition was found.

[jonathortense]

Workaround that worked for me

I have two AWS providers:

provider "aws" {
  alias      = "aws_root_account"
  version    = "~> 2.0"
  access_key = "${var.aws_access_key}"
  secret_key = "${var.aws_secret_key}"
  region     = "eu-west-1"
}

provider "aws" {
  alias      = "aws_new_subaccount"
  version    = "~> 2.0"
  access_key = "${var.aws_access_key}"
  secret_key = "${var.aws_secret_key}"
  region     = "eu-west-1"
  assume_role {
    role_arn     = "arn:aws:iam::XXXXXXXXXXX:role/OrganizationAccountAccessRole"
  }
}

On terraform 0.11.14 I can run this without problems:

terraform import -provider=aws.aws_new_subaccount ...

Now, on terraform 0.12.2 I had to add an empty AWS provider like this:

provider "aws" {
  region = "${var.region}"
}

And it worked.

[meschansky]

The same happens on terraform 0.12.3 with aliased mysql providers when no default one is present:

$ terraform import aws_ecr_repository.repo1 repo1
aws_ecr_repository.repo1: Importing from ID "repo1"...
aws_ecr_repository.repo1: Import complete!
  Imported aws_ecr_repository

Error: Missing required argument

The argument "username" is required, but was not set.


Error: Missing required argument

The argument "endpoint" is required, but was not set.```

and resources aren't imported eventually.

A workaround is to add the default mysql provider:

provider "mysql" {
  endpoint = ""
  username = ""
}

[AirCombat]

I'm having what I believe is a similar issue with a vmware provider. We leave user and password blank so they don't get committed to source. However I don't get that prompt when I try to import a vm

terraform import module.foobar.vsphere_virtual_machine.vmware_vm 1234-6578-90ab

Error: Missing required argument

  on /home/warith.cadmus/terraform/qa/provider.tf line 1, in provider "vsphere":
   1: provider "vsphere" {

The argument "password" is required, but no definition was found.

@WraithCadmus did you find a solution to this? I had an issue and lost my state and now can't import.

[pythian-ciciliani]

Also seeing this problem in 0.12.5, with AWS multiple providers. The workaround proposed by @meschansky of adding a default provider pointing to the needed region temporarily worked for me