S3 Backend doesn't pick up role from profiles

[estahn]

Terraform Version

Terraform v0.11.10
+ provider.aws v1.48.0
+ provider.github v1.3.0

Terraform Configuration Files

Non-working example:

terraform {
  backend "s3" {
    bucket         = "xxx"
    key            = "projects/xxx.tfstate"
    region         = "ap-southeast-2"
    dynamodb_table = "terraform_locks"
    kms_key_id     = "arn:aws:kms:ap-southeast-2:xxx:key/xxx"
    profile = "default"
  }
}

Working example:

terraform {
  backend "s3" {
    bucket         = "xxx"
    key            = "projects/xxx.tfstate"
    region         = "ap-southeast-2"
    dynamodb_table = "terraform_locks"
    kms_key_id     = "arn:aws:kms:ap-southeast-2:xxx:key/xxx"
    profile = "default"
    shared_credentials_file = "$HOME/.aws/credentials"
  }
}

Debug Output

Crash Output

Expected Behavior

I should be able to omit shared_credentials_file and terraform should pick up the role configuration from $HOME/.aws/config.

Actual Behavior

Authentication fails.

Steps to Reproduce

1. terraform init

Additional Context

• The documentation and actual source code seem to be unaligned. From the documentation:

  shared_credentials_file - (Optional) This is the path to the shared credentials file. If this is not set and a profile is specified, ~/.aws/credentials will be used.

  but the actual source code doesn't have that default: https://github.com/hashicorp/terraform/blob/master/backend/remote-state/s3/backend.go#L111-L115

References

• S3 Remote State and Credentials in non-default profiles #13589 – Possibly related but closed for comments.

[apparentlymart]

Hi @estahn! Sorry for this strange behavior, and thanks for reporting it.

I'm not sure what's going on here yet, but I just wanted to note for future readers that the provider-level default for that argument is intentionally unset because the default actually comes from the underlying AWS SDK, which is what is ultimately parsing this file. Terraform is just passing the values on down to there and leaving the details to the SDK.

Terraform Core is leaning on logic from the AWS provider to get this done, so it's likely that the problem is actually in the credentials-handling logic of the AWS provider, but we'll leave this here for now until further investigation is able to prove it.

[estahn]

@apparentlymart No worries, I figured it should just adopt the default behaviour of the AWS SDK. Unfortunately, it's not the case atm.

[jbruett]

@apparentlymart I have a detailed log of the non-default profile/assume role issue failing, this is the same issue as #13589 (I am the owner of this issue). I would prefer not to post it here for length and security purposes. Is there another way for me to provide? The credentials work without issue when calling direct from the CLI/PowerShell tools.

[ablackrw]

For what it's worth, I have had success using both of the following backend configurations:

terraform {
  backend "s3" {
    bucket = "xxx"
    key    = "remote/xxx"
    region = "us-east-1"
    encrypt = true 
    profile = "terraform"
  }
}

terraform {
  backend "s3" {
    bucket = "xxx"
    key    = "remote/xxx"
    region = "us-east-1"
    encrypt = true 
    profile = "saml"
    role_arn = "arn:aws:iam::xxx:role/xxx"
    session_name = "terraform"
  }
}

I will note that the later failed until recently, but works with

Terraform v0.11.11
+ provider.aws v1.55.0

(and provider.aws v1.54.0)