It happened crash after upgrade the version to 1.8.0

[jinsekim]

Terraform Version

1.8.0

Terraform Configuration Files

...terraform config...

Debug Output

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

panic: no expansion has been registered for module.cert_lb
goroutine 19550 [running]:
runtime/debug.Stack()
	/Users/runner/hostedtoolcache/go/1.22.1/x64/src/runtime/debug/stack.go:24 +0x5e
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/Users/runner/work/terraform/terraform/internal/logging/panic.go:84 +0x18b
panic({0x7213d00?, 0xc00592bef0?})
	/Users/runner/hostedtoolcache/go/1.22.1/x64/src/runtime/panic.go:770 +0x132
github.com/hashicorp/terraform/internal/instances.(*Expander).ExpandAbsModuleCall(0x734ed40?, {{0x0, 0x0, 0x0}, {{}, {0xc00013c8d0, 0x7}}})
	/Users/runner/work/terraform/terraform/internal/instances/expander.go:171 +0x265
github.com/hashicorp/terraform/internal/terraform.(*evaluationStateData).GetModule(0xc00159d560, {{}, {0xc00013c8d0?, 0x4a5f547?}}, {{0xc00072a970, 0xf}, {0x3b, 0x1e, 0x892}, {0x3b, ...}})
	/Users/runner/work/terraform/terraform/internal/terraform/evaluate.go:398 +0x788
github.com/hashicorp/terraform/internal/lang.(*Scope).evalContext(0xc00159d5f0, {0xc00aa8d320, 0x5, 0x5}, {0x0, 0x0})
	/Users/runner/work/terraform/terraform/internal/lang/eval.go:381 +0x1112
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalContext(...)
	/Users/runner/work/terraform/terraform/internal/lang/eval.go:245
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalBlock(0xc00159d5f0, {0x78fb640, 0xc002ac4730}, 0xc004d29980)
	/Users/runner/work/terraform/terraform/internal/lang/eval.go:58 +0xc5
github.com/hashicorp/terraform/internal/terraform.(*BuiltinEvalContext).EvaluateBlock(0x1?, {0x78fb8d0, 0xc00123edc0}, 0xc004d29980, {0x0?, 0x0?}, {{{{0x0, 0x0}}, {0x0, 0x0}}, ...})
	/Users/runner/work/terraform/terraform/internal/terraform/eval_context_builtin.go:315 +0x145
github.com/hashicorp/terraform/internal/terraform.(*NodeAbstractResourceInstance).plan(0xc005e24248, {0x791b128, 0xc000e2ec30}, 0x0, 0x0, 0x0, {0x0, 0x0, 0x0?})
	/Users/runner/work/terraform/terraform/internal/terraform/node_resource_abstract_instance.go:808 +0xb8b
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).managedResourceExecute(0xc005dd7340, {0x791b128, 0xc000e2ec30})
	/Users/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:250 +0x1395
github.com/hashicorp/terraform/internal/terraform.(*NodePlannableResourceInstance).Execute(0x18?, {0x791b128?, 0xc000e2ec30?}, 0x0?)
	/Users/runner/work/terraform/terraform/internal/terraform/node_resource_plan_instance.go:75 +0x8a
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc003821cc0, {0x791b128, 0xc000e2ec30}, {0x554ec930, 0xc005dd7340})
	/Users/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:153 +0xbb
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x7880a60, 0xc005dd7340})
	/Users/runner/work/terraform/terraform/internal/terraform/graph.go:112 +0x62a
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc005d896e0, {0x7880a60, 0xc005dd7340}, 0xc00579f1c0)
	/Users/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2d7
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 15667
	/Users/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xff3

Expected Behavior

Nothing should have happened.

Actual Behavior

It happened crash

Steps to Reproduce

terraform plan

Additional Context

No response

References

No response

[liamcervante]

Hi @jinsekim, are you able to share your configuration or a reproduction example of this crash? Thanks!

[georgepstaylor]

We've hit the same thing this morning. 2 symptoms:

1. Plans are inconsistent between 1.7.5 and 1.8.0 - we see certain attributes changing. Now, it is actually correct, but I don't understand why terraform doesn't pick them up in 1.7.X
2. The same crash about panic: no expansion has been registered module.xxxxxx when applying - even though the plan shows no changes to the module it refers to

[liamcervante]

Hi @georgepstaylor, thanks for the info! Do you have a reproducible config for the crash? I haven't been able to reproduce it yet so can't provide any guidance or potential workarounds at the moment. Thanks!

[jinsekim]

resource "aws_route53_record" "cert_lb" {
  provider = aws.route53
  zone_id  = local.service_dns_zone_id
  name     = format("cert-%s.%s", var.aws_region, local.service_dns_name)
  type     = "A"

  alias {
    name                   = module.cert_lb["nlb"].dns
    zone_id                = module.cert_lb["nlb"].zone_id
    evaluate_target_health = false
  }
}

@liamcervante I did run to the route53 record code. The 1.8.0 version had a crash problem. However, the 1.7.5 version did not have any issues.

[liamcervante]

Thanks @jinsekim, are you able to share the config inside module.cert_lb as well? Also the module block that calls it would be helpful.

Based on the stack trace, I think Terraform is trying to access an output of the module before the module has executed. So the contents of the module and the module call itself would also be useful.

Thanks!

[georgepstaylor]

Hi @georgepstaylor, thanks for the info! Do you have a reproducible config for the crash? I haven't been able to reproduce it yet so can't provide any guidance or potential workarounds at the moment. Thanks!

Hi @liamcervante.

https://github.com/ministryofjustice/modernisation-platform-environments/tree/main/terraform/environments/delius-core
The project here causes a crash when applying.

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

panic: no expansion has been registered for module.environment_dev[0].module.oracle_db_primary
goroutine 6115 [running]:
runtime/debug.Stack()
	/opt/hostedtoolcache/go/1.22.1/x64/src/runtime/debug/stack.go:24 +0x5e
github.com/hashicorp/terraform/internal/logging.PanicHandler()
	/home/runner/work/terraform/terraform/internal/logging/panic.go:84 +0x18b
panic({0x2e00120?, 0xc00a84ce10?})
	/opt/hostedtoolcache/go/1.22.1/x64/src/runtime/panic.go:770 +0x132
github.com/hashicorp/terraform/internal/instances.(*Expander).ExpandAbsModuleCall(0x2f3a180?, {{0xc007b6c9c0, 0x1, 0x2}, {{}, {0xc0012e1f08, 0x11}}})
	/home/runner/work/terraform/terraform/internal/instances/expander.go:171 +0x265
github.com/hashicorp/terraform/internal/terraform.(*evaluationStateData).GetModule(0xc00d85e5a0, {{}, {0xc0012e1f08?, 0x7?}}, {{0xc00098cea0, 0x26}, {0x64, 0x5e, 0xc4d}, {0x64, ...}})
	/home/runner/work/terraform/terraform/internal/terraform/evaluate.go:398 +0x788
github.com/hashicorp/terraform/internal/lang.(*Scope).evalContext(0xc00d85e630, {0xc00d955488, 0x3, 0x3}, {0x0, 0x0})
	/home/runner/work/terraform/terraform/internal/lang/eval.go:381 +0x1112
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalContext(...)
	/home/runner/work/terraform/terraform/internal/lang/eval.go:245
github.com/hashicorp/terraform/internal/lang.(*Scope).EvalExpr(0xc00d85e630, {0x3d75c00, 0xc000c12ae0}, {{0x3d75ce0?, 0x5891580?}})
	/home/runner/work/terraform/terraform/internal/lang/eval.go:170 +0x9e
github.com/hashicorp/terraform/internal/terraform.(*nodeModuleVariable).evalModuleVariable(0xc009daaee0, {0x3d984a8?, 0xc0006bdd10?}, 0xd8?)
	/home/runner/work/terraform/terraform/internal/terraform/node_module_variable.go:275 +0x1a5
github.com/hashicorp/terraform/internal/terraform.(*nodeModuleVariable).Execute(0xc009daaee0, {0x3d984a8, 0xc0006bdd10}, 0x1)
	/home/runner/work/terraform/terraform/internal/terraform/node_module_variable.go:214 +0x1c8
github.com/hashicorp/terraform/internal/terraform.(*ContextGraphWalker).Execute(0xc004d688c0, {0x3d984a8, 0xc0006bdd10}, {0x7f5f296fbd00, 0xc009daaee0})
	/home/runner/work/terraform/terraform/internal/terraform/graph_walk_context.go:153 +0xbb
github.com/hashicorp/terraform/internal/terraform.(*Graph).walk.func1({0x31d97a0, 0xc009daaee0})
	/home/runner/work/terraform/terraform/internal/terraform/graph.go:112 +0x62a
github.com/hashicorp/terraform/internal/dag.(*Walker).walkVertex(0xc000f66780, {0x31d97a0, 0xc009daaee0}, 0xc009df00c0)
	/home/runner/work/terraform/terraform/internal/dag/walk.go:384 +0x2d7
created by github.com/hashicorp/terraform/internal/dag.(*Walker).Update in goroutine 3929
	/home/runner/work/terraform/terraform/internal/dag/walk.go:307 +0xff3

The module source can be found here: https://github.com/ministryofjustice/modernisation-platform-environments/tree/main/terraform/environments/delius-core/modules/components/oracle_db_instance

and here is where it is called from: https://github.com/ministryofjustice/modernisation-platform-environments/blob/4f84e9b3b88cd33b9f409da41f99001d9b0796eb/terraform/environments/delius-core/modules/delius_environment/database.tf#L28

Not sure if that's helpful?

[arkaitzj]

Observing the same issue in TerraformCloud, workspaces fixed to 1.7.5 doing ok but those at "latest" are crashing with "no expansion..." on module "terraform-aws-modules/rds-aurora/aws"

[liamcervante]

Is there anything particular about the commands you execute that cause the crash? Is it just a normal terraform plan operation, or is this during the apply stage? And, are you planning/applying this with an empty state file, or do you have partial state that is being updated?

[arkaitzj]

@liamcervante its just a PR plan using TerraformCloud, I just had a the dev workspace pinned to 1.7.5 and stage,prod where pinned latest. stage and prod started failing today when just re-running master branch, with no changes.
There was ongoing state in TFC.