S3 Backend with sts_endpoint specified is using global STS endpoint instead

[nickkneafsey]

Terraform Version

0.12.20

Terraform Configuration Files

terraform {
  backend "s3" {
    sts_endpoint = "https://sts.us-west-2.amazonaws.com"
    role_arn = "<removed>"
  }
}

I have the following variables exported as well:

export AWS_DEFAULT_REGION=us-west-2
export AWS_REGION=us-west-2
export AWS_SDK_LOAD_CONFIG=1
export AWS_STS_REGIONAL_ENDPOINTS=regional

Expected Behavior

The role should have been assumed using the STS endpoint from us-west-2.

Actual Behavior

From Cloudtrail, I can see that the role was assumed using the STS global endpoint in us-east-1.

...
"eventSource": "sts.amazonaws.com",
"eventName": "AssumeRole",
"awsRegion": "us-east-1",
"userAgent": "aws-sdk-go/1.25.3 (go1.12.13; darwin; amd64)",
...

Steps to Reproduce

export AWS_DEFAULT_REGION=us-west-2
export AWS_REGION=us-west-2
export AWS_SDK_LOAD_CONFIG=1
export AWS_STS_REGIONAL_ENDPOINTS=regional

Configure an S3 Backend similar to the following (role_arn and sts_endpoint specified) and add any AWS resource (bucket in this example).

terraform {
  backend "s3" {
    sts_endpoint = "https://sts.us-west-2.amazonaws.com"
    role_arn = "<removed>"
    bucket  = "some-bucket"
    key     = "some-key"
    region  = "us-west-2"
  }
}

provider "aws" {
  version = "~> 2.0"
  region  = "us-west-2"
}

resource "aws_s3_bucket" "b" {
  bucket = "my-tf-test-bucket"
  acl    = "private"

  tags = {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

Run terraform init followed by terraform apply or terraform plan

Check CloudTrail for the region that the AssumeRole call happened. Note that it is not from us-west-2.

Additional Context

I think that this issue would be resolved if the aws-sdk-go-base (https://github.com/hashicorp/aws-sdk-go-base) dependency was using a newer version of aws-sdk-go. It is using 1.25.3 in its latest release (v0.4.0), but it looks like the regional endpoints environment variables (AWS_STS_REGIONAL_ENDPOINTS=regional) change came in aws-sdk-go/1.25.18 per aws/aws-sdk-go#2906. I am able to use STS regional endpoints for assuming a role with the AWS Provider, which is using aws-sdk-go/1.29.24.

[bflad]

The fix for this should be merged and will release with version 0.13.0-beta2 of Terraform.

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.