Description
Terraform and AWS Provider Version
Terraform v1.10.5
on linux_amd64
provider registry.terraform.io/hashicorp/aws v5.92.0
Affected Resource(s) or Data Source(s)
aws_lambda_function
or the data "aws_s3_object"
Expected Behavior
When the runtime management configuration was set to "Manual," Terraform failed to update the runtime version from Python 3.12 to 3.13. If the Lambda function does not reflect the new source code, the Terraform state should not contain the updated s3_object_version, ensuring that subsequent successful apply operations trigger the source code update.
Actual Behavior
When the runtime management configuration was set to "Manual," Terraform failed to update the runtime version from Python 3.12 to 3.13. Although the s3_object_version was updated in the state file, the Lambda function did not reflect the new source code. Manually switching the runtime management to "Auto" in the AWS GUI resolved the runtime update issue, but the source code still failed to update. Terraform did not trigger another s3_object_version update because it considered the state file correct, leaving the Lambda function with outdated source code.
Relevant Error/Panic Output
Sample Terraform Configuration
Click to expand terraform configuration
module "lambda" {
source = "../../../../../placeholder-shared-terraform-modules/lambda"
environment_variables = {
}
function_name = "test"
description = ""
handler = "handler.lambda_handler"
iam_policy_arn = var.lambda_policy_arn
kms_key = var.central_kms_key
runtime = "python3.13"
security_group_ids = [var.placeholder_security_group_id]
source_code_bucket_name = "${var.system}-lambdas-${var.stage}"
source_code_file_name = "handler"
subnet_ids = data.aws_subnets.selected.ids
timeout = 360
memory_size = 512
}
resource "null_resource" "manual_runtime_management_config_lambda" {
triggers = {
always_run = timestamp()
}
provisioner "local-exec" {
command = "aws lambda put-runtime-management-config --region AWS_REGION --function-name ${module.lambda.arn} --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:${var.aws_region}::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"
}
depends_on = [
module.lambda
]
}
Click to expand terraform module configuration
data "aws_s3_object" "lambda" {
bucket = var.source_code_bucket_name
key = "${var.source_code_file_name}.zip"
}
resource "aws_lambda_function" "lambda" {
function_name = var.function_name
description = var.description
handler = var.handler
memory_size = var.memory_size
reserved_concurrent_executions = var.reserved_concurrent_executions
role = aws_iam_role.role.arn
runtime = var.runtime
s3_bucket = var.source_code_bucket_name
s3_key = "${var.source_code_file_name}.zip"
tags = var.tags
timeout = var.timeout
publish = true
s3_object_version = data.aws_s3_object.lambda.version_id
environment {
variables = var.environment_variables
}
timeouts {}
vpc_config {
security_group_ids = var.security_group_ids
subnet_ids = var.subnet_ids
}
depends_on = [aws_iam_role_policy_attachment.vpc_access, aws_iam_role_policy_attachment.basic_access]
}
resource "aws_cloudwatch_log_group" "log_group" {
name = "/aws/lambda/${aws_lambda_function.lambda.function_name}"
tags = var.tags
kms_key_id = var.kms_key
retention_in_days = 7
depends_on = [aws_lambda_function.lambda]
}
resource "aws_lambda_alias" "alias" {
name = "${var.function_name}_lambda_alias"
description = "Alias to currently used Version of ${var.function_name} Lambda"
function_name = aws_lambda_function.lambda.arn
function_version = aws_lambda_function.lambda.version
depends_on = [aws_lambda_function.lambda]
}
resource "aws_iam_role" "role" {
assume_role_policy = jsonencode(
{
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
Service = "lambda.amazonaws.com"
}
},
]
Version = "2012-10-17"
}
)
description = "Permissions for Lambda function ${var.function_name}"
force_detach_policies = false
max_session_duration = 3600
name = "${var.function_name}-lambda-role"
path = "/"
tags = var.tags
}
resource "aws_iam_role_policy_attachment" "policy_attachment" {
count = var.iam_policy_arn != "" ? 1 : 0
role = aws_iam_role.role.name
policy_arn = var.iam_policy_arn
}
resource "aws_iam_role_policy_attachment" "basic_access" {
role = aws_iam_role.role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_iam_role_policy_attachment" "vpc_access" {
role = aws_iam_role.role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
}
Steps to Reproduce
Summary of Steps Taken
The s3_key and s3_object_version in the terraform state file before any changes to the lambda function were made
"module": "module.lambda",
...
"s3_key": "handler.zip",
"s3_object_version": "HOP4InHIauGKqoxhEGk7ykxGygOoeJ9G",
The contents of the handler
file before any changes were made:
def lambda_handler(event, context):
print ("Hello")
- Update python version in the lambda module and add a new runtime management configuration:
module "lambda" {
source = "../../../../../placeholder-shared-terraform-modules/lambda"
environment_variables = {
}
function_name = "test"
description = ""
handler = "handler.lambda_handler"
iam_policy_arn = var.lambda_policy_arn
kms_key = var.central_kms_key
- runtime = "python3.12"
+ runtime = "python3.13"
security_group_ids = [var.placeholder_security_group_id]
source_code_bucket_name = "${var.system}-lambdas-${var.stage}"
source_code_file_name = "handler"
subnet_ids = data.aws_subnets.selected.ids
timeout = 360
memory_size = 512
}
resource "null_resource" "manual_runtime_management_config_lambda" {
count = var.stage == "dev" ? 1 : 0
triggers = {
always_run = timestamp()
}
provisioner "local-exec" {
- command = "aws lambda put-runtime-management-config --region AWS_REGION --function-name ${module.lambda.arn} --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:${var.aws_region}::runtime:c9875014cbcc77e3455765804516f064d18fe7b27ae7bdb2b1d84ab01ba784f5"
+ command = "aws lambda put-runtime-management-config --region AWS_REGION --function-name ${module.lambda.arn} --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:${var.aws_region}::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"
}
depends_on = [
module.lambda
]
}
The placeholder-shared-terraform-modules/lambda
module contains the following code:
data "aws_s3_object" "lambda" {
bucket = var.source_code_bucket_name
key = "${var.source_code_file_name}.zip"
}
resource "aws_lambda_function" "lambda" {
function_name = var.function_name
description = var.description
handler = var.handler
memory_size = var.memory_size
reserved_concurrent_executions = var.reserved_concurrent_executions
role = aws_iam_role.role.arn
runtime = var.runtime
s3_bucket = var.source_code_bucket_name
s3_key = "${var.source_code_file_name}.zip"
tags = var.tags
timeout = var.timeout
publish = true
s3_object_version = data.aws_s3_object.lambda.version_id
- A new source code file
handler
was pushed to the S3 bucket
def lambda_handler(event, context):
print ("Hello world")
- After running
terraform apply
, the following plan is generated:
# null_resource.manual_runtime_management_config_lambda[0] must be replaced
-/+ resource "null_resource" "manual_runtime_management_config_lambda" {
~ id = "3233780801185463294" -> (known after apply)
~ triggers = { # forces replacement
~ "always_run" = "2025-05-21T08:25:53Z" -> (known after apply)
}
}
# module.lambda.aws_lambda_alias.alias will be updated in-place
~ resource "aws_lambda_alias" "alias" {
~ function_version = "6" -> (known after apply)
id = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias"
name = "test_lambda_alias"
# (4 unchanged attributes hidden)
}
# module.lambda.aws_lambda_function.lambda will be updated in-place
~ resource "aws_lambda_function" "lambda" {
id = "test"
~ last_modified = "2025-05-21T08:25:54.000+0000" -> (known after apply)
~ qualified_arn = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:6" -> (known after apply)
~ qualified_invoke_arn = "arn:aws:apigateway:AWS_REGION:lambda:path/2015-03-31/functions/arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:6/invocations" -> (known after apply)
~ runtime = "python3.12" -> "python3.13"
~ s3_object_version = "HOP4InHIauGKqoxhEGk7ykxGygOoeJ9G" -> "0C4XpQIAXf_0j5v9jIMMuiHoL1CHUULu"
tags = {}
~ version = "6" -> (known after apply)
# (25 unchanged attributes hidden)
# (5 unchanged blocks hidden)
}
Plan: 1 to add, 2 to change, 1 to destroy.
Output from the terraform apply
command:
null_resource.manual_runtime_management_config_lambda[0]: Destroying... [id=3233780801185463294]
null_resource.manual_runtime_management_config_lambda[0]: Destruction complete after 0s
module.lambda.aws_lambda_function.lambda: Modifying... [id=test]
╷
│ Error: updating Lambda Function (test) configuration: operation error Lambda: UpdateFunctionConfiguration, https response error StatusCode: 400, RequestID: 383081de-c9c9-4c1e-8483-3906926fd486, InvalidParameterValueException: Cannot change runtime identifier when runtime management configuration is set to 'Manual'. Change the runtime management configuration to 'Auto' or 'FunctionUpdate', then retry.
│
│ with module.lambda.aws_lambda_function.lambda,
│ on ../../../../../placeholder-shared-terraform-modules/lambda/main.tf line 14, in resource "aws_lambda_function" "lambda":
│ 14: resource "aws_lambda_function" "lambda" {
│
╵
ERRO[0053] terraform invocation failed in /mnt/c/Projects/MB/dev/infrastructure/placeholder-team-infrastructure/03_cfg_import/.terragrunt-cache/tEwXzho8NYfDDW5pyDtjQ-SkWb4/XdE8TIc7nRDtDWxxBwCMHsYL3HM error=[/mnt/c/Projects/MB/dev/infrastructure/placeholder-team-infrastructure/03_cfg_import/.terragrunt-cache/tEwXzho8NYfDDW5pyDtjQ-SkWb4/XdE8TIc7nRDtDWxxBwCMHsYL3HM] exit status 1 prefix=[/mnt/c/Projects/MB/dev/infrastructure/placeholder-team-infrastructure/03_cfg_import]
ERRO[0053] 1 error occurred:
* [/mnt/c/Projects/MB/dev/infrastructure/placeholder-team-infrastructure/03_cfg_import/.terragrunt-cache/tEwXzho8NYfDDW5pyDtjQ-SkWb4/XdE8TIc7nRDtDWxxBwCMHsYL3HM] exit status 1
The state file after the terraform apply
command:
"module": "module.lambda",
...
"s3_key": "handler.zip",
"s3_object_version": "0C4XpQIAXf_0j5v9jIMMuiHoL1CHUULu",
The issue here is that the runtime management configuration was set to "Manual" and the runtime was changed from python3.12 to python3.13. This caused the error when trying to update the lambda function, as the runtime management configuration must be set to "Auto" or "FunctionUpdate" in order to change the runtime. However the s3_object_version was updated in the state file, the lambda function was not updated with the changes to the handler.zip file, which could be seen in the AWS GUI.
- Manually changing the runtime management to Auto in the GUI and then running
terraform apply
again results in the following plan:
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
~ update in-place
Terraform will perform the following actions:
# null_resource.manual_runtime_management_config_lambda[0] will be created
+ resource "null_resource" "manual_runtime_management_config_lambda" {
+ id = (known after apply)
+ triggers = {
+ "always_run" = (known after apply)
}
}
# module.lambda.aws_lambda_alias.alias will be updated in-place
~ resource "aws_lambda_alias" "alias" {
~ function_version = "6" -> (known after apply)
id = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias"
name = "test_lambda_alias"
# (4 unchanged attributes hidden)
}
# module.lambda.aws_lambda_function.lambda will be updated in-place
~ resource "aws_lambda_function" "lambda" {
id = "test"
~ qualified_arn = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:6" -> (known after apply)
~ qualified_invoke_arn = "arn:aws:apigateway:AWS_REGION:lambda:path/2015-03-31/functions/arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:6/invocations" -> (known after apply)
~ runtime = "python3.12" -> "python3.13"
tags = {}
~ version = "6" -> (known after apply)
# (27 unchanged attributes hidden)
# (5 unchanged blocks hidden)
}
Plan: 1 to add, 2 to change, 0 to destroy.
The output from the "terraform apply" command:
module.lambda.aws_lambda_function.lambda: Modifying... [id=test]
module.lambda.aws_lambda_function.lambda: Modifications complete after 7s [id=test]
module.lambda.aws_lambda_alias.alias: Modifying... [id=arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias]
module.lambda.aws_lambda_alias.alias: Modifications complete after 0s [id=arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias]
null_resource.manual_runtime_management_config_lambda[0]: Creating...
null_resource.manual_runtime_management_config_lambda[0]: Provisioning with 'local-exec'...
null_resource.manual_runtime_management_config_lambda[0] (local-exec): Executing: ["/bin/sh" "-c" "aws lambda put-runtime-management-config --region AWS_REGION --function-name arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"]
null_resource.manual_runtime_management_config_lambda[0] (local-exec): {
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "UpdateRuntimeOn": "Manual",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "FunctionArn": "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "RuntimeVersionArn": "arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"
null_resource.manual_runtime_management_config_lambda[0] (local-exec): }
null_resource.manual_runtime_management_config_lambda[0]: Creation complete after 1s [id=1699563929237199602]
Apply complete! Resources: 1 added, 2 changed, 0 destroyed.
State file after the terraform apply
command:
"module": "module.lambda",
...
"s3_key": "handler.zip",
"s3_object_version": "0C4XpQIAXf_0j5v9jIMMuiHoL1CHUULu",
The lambda function was successfully updated to python3.13 and the runtime management configuration was set to Manual.
However the new source code file handler
was still not updated in the lambda function.
- Running another
terraform apply
results in no changes:
# null_resource.manual_runtime_management_config_lambda[0] must be replaced
-/+ resource "null_resource" "manual_runtime_management_config_lambda" {
~ id = "1852426693614550083" -> (known after apply)
~ triggers = { # forces replacement
~ "always_run" = "2025-05-21T07:49:37Z" -> (known after apply)
}
}
# module.lambda.aws_lambda_alias.alias will be updated in-place
~ resource "aws_lambda_alias" "alias" {
~ function_version = "4" -> (known after apply)
id = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias"
name = "test_lambda_alias"
# (4 unchanged attributes hidden)
}
}
# module.lambda.aws_lambda_function.lambda will be updated in-place
~ resource "aws_lambda_function" "lambda" {
id = "test"
~ qualified_arn = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:4" -> (known after apply)
~ qualified_invoke_arn = "arn:aws:apigateway:AWS_REGION:lambda:path/2015-03-31/functions/arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:4/invocations" -> (known after apply)
tags = {}
~ version = "4" -> (known after apply)
# (28 unchanged attributes hidden)
# (5 unchanged blocks hidden)
}
Plan: 1 to add, 2 to change, 1 to destroy.
null_resource.manual_runtime_management_config_lambda[0]: Destroying... [id=1852426693614550083]
null_resource.manual_runtime_management_config_lambda[0]: Destruction complete after 0s
module.lambda.aws_lambda_function.lambda: Modifying... [id=test]
module.lambda.aws_lambda_function.lambda: Modifications complete after 0s [id=test]
null_resource.manual_runtime_management_config_lambda[0]: Creating...
null_resource.manual_runtime_management_config_lambda[0]: Provisioning with 'local-exec'...
null_resource.manual_runtime_management_config_lambda[0] (local-exec): Executing: ["/bin/sh" "-c" "aws lambda put-runtime-management-config --region AWS_REGION --function-name arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"]
null_resource.manual_runtime_management_config_lambda[0] (local-exec): {
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "UpdateRuntimeOn": "Manual",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "FunctionArn": "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "RuntimeVersionArn": "arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"
null_resource.manual_runtime_management_config_lambda[0] (local-exec): }
null_resource.manual_runtime_management_config_lambda[0]: Creation complete after 1s [id=6733276592831746917]
Apply complete! Resources: 1 added, 1 changed, 1 destroyed.
The state file after the terraform apply
command:
"module": "module.lambda",
...
"s3_key": "handler.zip",
"s3_object_version": "0C4XpQIAXf_0j5v9jIMMuiHoL1CHUULu",
Since the s3_object_version is already updated in the state file as per terraform's perspective, it does not trigger another update to the lambda function. The lambda function is not updated with the new source code file. And in the gui and when being invoked, the old source code is still being used.
def lambda_handler(event, context):
print ("Hello")
Updating the source code file again, uploading it to s3 and running terraform apply
results in this terraform plan:
def lambda_handler(event, context):
print ("Hello world again")
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement
Terraform will perform the following actions:
# null_resource.manual_runtime_management_config_lambda[0] must be replaced
-/+ resource "null_resource" "manual_runtime_management_config_lambda" {
~ id = "6733276592831746917" -> (known after apply)
~ triggers = { # forces replacement
~ "always_run" = "2025-05-21T07:58:14Z" -> (known after apply)
}
}
# module.lambda.aws_lambda_alias.alias will be updated in-place
~ resource "aws_lambda_alias" "alias" {
~ function_version = "4" -> (known after apply)
id = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias"
name = "test_lambda_alias"
# (4 unchanged attributes hidden)
}
# module.lambda.aws_lambda_function.lambda will be updated in-place
~ resource "aws_lambda_function" "lambda" {
id = "test"
~ last_modified = "2025-05-21T07:58:15.000+0000" -> (known after apply)
~ qualified_arn = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:4" -> (known after apply)
~ qualified_arn = "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:4" -> (known after apply)
~ qualified_invoke_arn = "arn:aws:apigateway:AWS_REGION:lambda:path/2015-03-31/functions/arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:4/invocations" -> (known after apply)
~ s3_object_version = "0C4XpQIAXf_0j5v9jIMMuiHoL1CHUULu" -> "MMoiOyRwDBTmjtPiI_kh6JDvvz7qzHZ9"
tags = {}
~ version = "4" -> (known after apply)
# (26 unchanged attributes hidden)
# (5 unchanged blocks hidden)
}
Plan: 1 to add, 2 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
null_resource.manual_runtime_management_config_lambda[0]: Destroying... [id=6733276592831746917]
null_resource.manual_runtime_management_config_lambda[0]: Destruction complete after 0s
module.lambda.aws_lambda_function.lambda: Modifying... [id=test]
module.lambda.aws_lambda_function.lambda: Modifications complete after 6s [id=test]
module.lambda.aws_lambda_alias.alias: Modifying... [id=arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias]
module.lambda.aws_lambda_alias.alias: Modifications complete after 0s [id=arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test:test_lambda_alias]
null_resource.manual_runtime_management_config_lambda[0]: Creating...
null_resource.manual_runtime_management_config_lambda[0]: Provisioning with 'local-exec'...
null_resource.manual_runtime_management_config_lambda[0] (local-exec): Executing: ["/bin/sh" "-c" "aws lambda put-runtime-management-config --region AWS_REGION --function-name arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test --update-runtime-on Manual --runtime-version-arn arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"]
null_resource.manual_runtime_management_config_lambda[0] (local-exec): {
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "UpdateRuntimeOn": "Manual",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "FunctionArn": "arn:aws:lambda:AWS_REGION:AWS_ACCOUNT_ID:function:test",
null_resource.manual_runtime_management_config_lambda[0] (local-exec): "RuntimeVersionArn": "arn:aws:lambda:AWS_REGION::runtime:f713ac0afb982fcdf9bac88eaa00c31352efae870b225c19f1603fe79159a6f1"
null_resource.manual_runtime_management_config_lambda[0] (local-exec): }
null_resource.manual_runtime_management_config_lambda[0]: Creation complete after 1s [id=1335838954038586303]
Apply complete! Resources: 1 added, 2 changed, 1 destroyed.
The state file after the terraform apply
command:
"module": "module.lambda",
...
"s3_key": "handler.zip",
"s3_object_version": "MMoiOyRwDBTmjtPiI_kh6JDvvz7qzHZ9",
The source code of the lambda is now successfully updated to the new file handler
in the S3 bucket.
Another way to force the update of the source code was to run a terraform apply -replace module.lambda.aws_lambda_function.lambda
command. This will force the lambda function to be recreated and the new source code file will be used.
Debug Logging
Click to expand log output
GenAI / LLM Assisted Development
n/a
Important Facts and References
I understand that this might also be a Terraform Core Issue, but since it happened for the Lambda resource, I decided to first open an Issue here.
Would you like to implement a fix?
No