Terraform not allowing to pass nested values (like assume_role.role_arn) to the backend.s3 configuration using cli (-backend-config="KEY=VALUE") and config file (-backend-config=PATH)

[Humeid-Ussene-Jocordasse]

Terraform Version

1.11.4

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.33.0"
    }
  }
  
  backend "s3" {
    bucket = ""
    use_lockfile=""
    key=""
    region=""
    assume_role = {
      role_arn = ""
      external_id = ""
    }
  }
    required_version = ">= 0.14.9"
}
...terraform config...

Debug Output

not applicable

Expected Behavior

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Actual Behavior

Initializing the backend...
╷
│ Error: Invalid backend configuration argument
│ 
│ The backend configuration argument "assume_role.role_arn" given on the command line is not expected for the selected backend type.
╵
╷
│ Error: Invalid backend configuration argument
│ 
│ The backend configuration argument "assume_role.external_id" given on the command line is not expected for the selected backend type.

Steps to Reproduce

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role.role_arn=arn:aws:iam::123456789012:role/my-humble-role" \
      -backend-config="assume_role.external_id=my-humble-id" \
      -force-copy

Additional Context

I'm having the same issue when i true to use those exact same values on backend.config file, trying to pass with the -backend-config=PATH flag...

I noticed this issue after upgrading the Terraform version in our team's Continuous Integration tool from v1.8.5 to v1.11.4. Previously, we were able to provide the role_arn programmatically because it was a top-level attribute in the backend.s3 block. Now that it has to be nested under assume_role, Terraform throws an error when we try to pass it dynamically.

Edit:
I've found out this still works when using a .hcl file, passing a value like this one:

bucket = "my-humble-bucket"
key="test/my-humble-key.tfstate"
region="af-south-1"
use_lockfile="true"
assume_role= {
    role_arn = "arn:aws:iam::123456789012:role/my-humble-role"
    external_id="my-humble-id"
}

References

No response

Generative AI / LLM assisted development?

No response

[crw]

Thanks for this report!

[apparentlymart]

In case it's somehow useful, a long time ago I wrote github.com/apparentlymart/go-hcl-overlay with the goal of supporting stuff like this where a system needs to merge some command line options in with existing HCL body content.

Even if that library is not directly useful as an import, I remember there being some interesting design tradeoffs in making that work which might give some ideas on similar tradeoffs to make in solving this for Terraform. In particular, I experimented with a few different ways to deal with a situation where e.g. the CLI says assume_role.role_arn but yet there isn't already an assume_role block in the configuration for that to override into, and picked one that seemed to work relatively well for reasonable cases like this.

The example I included in the repository uses the very opinionated ExtractCLIOptions whose opinions don't match how -backend-config already works, but I think ParseCLIArgument is closer to how Terraform wants to treat -backend-config where the key=value is given as the value of an option rather than as the option itself.

Offered just in the hope that it's helpful!

[SarahFrench]

Hi @apparentlymart , thanks for sharing that project! I had a go using it directly but it looks like attributes with NestedType set cannot be handled currently (I get Invalid argument errors for assume_role.role_arn etc. from cli_args.go, due to it being an attribute where the path contains >1 identifier). I like the overlay approach though, and will keep investigating, as this is a good learning opportunity around cty and hcl.

[apparentlymart]

Ahh hmm yes I suppose I wrote this long enough ago that the idea of using the attribute syntax in a block-like way hadn't happened yet, and so my old implementation doesn't support that... it only wants to allow dotting through block types and their labels.

"Overlaying" into attributes is actually likely to be somewhat harder than what I implemented there previously because from HCL's perspective the given expression hasn't been evaluated yet and so it doesn't have any idea what type of value is likely to appear there until someone eventually calls Value on the expression.

So following a similar implementation approach as what I was using so far would probably mean adding a new wrapping implementation of hcl.Expression, similar to how it already has a wrapping implementation of hcl.Body, and then using that wrapper expression type in the returned attribute whenever the given attribute path has more than one segment, so that the wrapper expression's Value method can try to handle this as a merger of two cty.Value instead of as a merger of two hcl.Body. 😬

If it's going to ultimately end up having to do the work in the cty.Value realm anyway then probably better to try to apply the "overlays" after HCL has already done its work, directly against the result cty.Value, so that nested blocks and structural attributes can be treated in the same way, just as cty objects.

I'm sorry that it wasn't more helpful!

[jbardin]

The fact that nested structural attributes can't be declared from the CLI was a known limitation and tradeoff when the S3 backend added them to their schema, and they could only be parameterized via a config file. So in essence this is all working as intended right now.

Being able to imply the attribute's existence by using the dotted traversal notation through the object in the above proposals would be unusual, and is really only going to work for simple single-nested structural attributes. If that first level attribute is itself a set/list/map things get more complicated, and you end up still having to write complex values directly in the CLI arguments. If we want to cover all bases, then the general form would be something like

-backend-config='assume_role={role_arn="arn:aws:iam::123456789012:role/my-humble-role",external_id="my-humble-id"}'

I don't know if we want a special syntax which only works for certain schema structures, but maybe that's worth it? (this is a good question for the parameterization of pluggable state storage as well).

[Humeid-Ussene-Jocordasse]

Hi @jbardin, interesting thoughts...

1. I'd say that if we want to move to toward accepting HCL configurations in the CLI, just like the example you gave, it would make much sense to accept in one argument rather that multiple arguments, as terraforms is doing right now...

e.g.

Not this:

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config="bucket=my-humble-bucket" \
      -backend-config="key=my-humble-key.tfstate" \
      -backend-config="region=af-south-1" \
      -backend-config="use_lockfile=true" \
      -backend-config="assume_role = '{
                                                          role_arn="arn:aws:iam::123456789012:role/my-humble-role"
                                                         external_id="my-humble-id"
        }'
      -force-copy

But this:

terraform -chdir=.iac init  \
      -backend=true \
      -backend-config= '{
                 bucket="my-humble-bucket",
                 key=my-humble-key.tfstate",
                 region=af-south-1",
                 use_lockfile=true",
                 assume_role = {
                                      role_arn="arn:aws:iam::123456789012:role/my-humble-role"
                                      external_id="my-humble-id"
                 }
        }'
      -force-copy

2. One thing to remember is that these configurations are specifically for Terraform backends via CLI, which i think aren't supposed to be too much complex or deeply nested. There is some degree of complexity that we want to discourage to be introduced via CLI, because it would make much more sense to provided it via configuration files instead.

Just adding food for thoughts in the conversation...

[d47zm3]

Does anyone has idea how to workaround this? I tried to convert my current style e.g.

# backend.hcl
bucket         = "workloads-deployment-states"
key            = "services/sandbox/dev/terraform.tfstate"
dynamodb_table = "workloads-deployment-locks"
region         = "eu-west-1"
max_retries    = 3
role_arn       = "arn:aws:iam::0123456789:role/cicd/cicd-workloads-deploy"
session_name   = "sandbox-dev"

along with init

terraform init -backend-config=environment/backend.hcl -input=false

and ENV var for External ID

    TF_CLI_ARGS_init: -backend-config='external_id=***'

This one is working but reports deprecation message. Using new format e.g.

# backend.hcl
bucket         = "workloads-deployment-states"
key            = "services/sandbox/dev/terraform.tfstate"
dynamodb_table = "workloads-deployment-locks"
region         = "eu-west-1"
max_retries    = 3
assume_role = {
  role_arn     = "arn:aws:iam::0123456789:role/cicd/cicd-workloads-deploy"
  session_name = "sandbox-dev"
}

along with

terraform init -backend-config=environment/backend.hcl -input=false

and ENV

    TF_CLI_ARGS_init: -backend-config='assume_role={ external_id="***" }'

doesn't seem to work, e.g. External ID is not passed it seems (I get Access Denied).

[SarahFrench]

Hi @d47zm3 - Currently it's only possible to override the values of assume_role and its nested attributes when the -backend-config flag supplies the path to an override file. The backend.hcl contents you've shared should work for that purpose.

Overriding assume_role and its nested attributes is currently not possible when the -backend-config instead supplies a string containing a key and a value. It's expected that -backend-config='assume_role={ external_id="***" }' would not work.

So the workaround would be to pass the overrides in using a file. Does that help you?

[d47zm3]

I'm not keen on keeping ExternalID inside plaintext file (though people debate if it's secret...) but at least I got confirmation it's not currently possible so thanks @SarahFrench for your input 👍