`terraform test` Consistently Crashing

[novekm]

Terraform Version

Terraform v1.7.0
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.34.0
+ provider registry.terraform.io/hashicorp/random v3.6.0

Terraform Configuration Files

main.tftest.hcl

# Instructions: Create basic tests

# Configure the AWS
provider "aws" {
  region = "us-east-1"
}


# Global Testing Variables - Define variables to be used in all tests here. You can overwrite these varibles by definig an additional variables block within the 'run' block for your tests
variables {

  # - Test CodeCommit Repos -
  codecommit_repos = {
    # Test Module 1
    test_module_1 : {

      repository_name = local.tf_test_test_module_1_repository_name
      description     = "Test Module 1."
      default_branch  = "main"
      tags = {
        "ContentType" = "Test Module",
      }
    },
    # Test Module 2
    test_module_2 : {

      repository_name = local.tf_test_test_module_2_repository_name
      description     = "Test Module 2."
      default_branch  = "main"
      tags = {
        "ContentType" = "Test Module",
      }
    },
    # Test Workload 1
    test_workload_1 : {

      repository_name = local.tf_test_test_workload_1_repository_name
      description     = "Test Workload 1."
      default_branch  = "main"
      tags = {
        "ContentType" = "Test Workload",
      }
    },
    # Test Workload 2
    test_workload_2 : {

      repository_name = local.tf_test_test_workload_2_repository_name
      description     = "Test Workload 2."
      default_branch  = "main"
      tags = {
        "ContentType" = "Test Workload",
      }
    },

  }

  # - Test CodeBuild Projects -
  codebuild_projects = {
    # Test Module 1
    tf_test_test_module_1 : {
      name               = local.tf_test_test_module_1_codebuild_project_name
      description        = "Test Module 1 - Terraform Test"
      path_to_build_spec = local.tf_test_path_to_buildspec

    },
    chevkov_test_module_1 : {
      name        = local.chevkov_test_module_1_codebuild_project_name
      description = "Test Module 1 - Checkov"

      path_to_build_spec = local.checkov_path_to_buildspec

    },
    # Test Module 2
    tf_test_test_module_2 : {
      name               = local.tf_test_test_module_2_codebuild_project_name
      description        = "Test Module 2 - Terraform Test"
      path_to_build_spec = local.tf_test_path_to_buildspec

    },
    chevkov_test_module_2 : {
      name        = local.chevkov_test_module_2_codebuild_project_name
      description = "Test Module 2 - Checkov"

      path_to_build_spec = local.checkov_path_to_buildspec

    },

    # Test Workload 1
    tf_test_test_workload_1 : {
      name        = local.tf_test_test_workload_1_codebuild_project_name
      description = "Test Workload 1"

      path_to_build_spec = local.tf_test_path_to_buildspec

    },
    chevkov_test_workload_1 : {
      name        = local.chevkov_test_workload_1_codebuild_project_name
      description = "Test Workload 1"

      path_to_build_spec = local.checkov_path_to_buildspec

    },
    # Test Workload 2
    tf_test_test_workload_2 : {
      name        = local.tf_test_test_workload_2_codebuild_project_name
      description = "Test Workload 2"

      path_to_build_spec = local.tf_test_path_to_buildspec

    },
    chevkov_test_workload_2 : {
      name        = local.chevkov_test_workload_2_codebuild_project_name
      description = "Test Workload 2"

      path_to_build_spec = local.checkov_path_to_buildspec

    },
    tf_apply_test_workload_1 : {
      name        = local.tf_apply_test_workload_1_codebuild_project_name
      description = "Test Workload 1"

      path_to_build_spec = local.tf_apply_path_to_buildspec

    },
    tf_apply_test_workload_2 : {
      name        = local.tf_apply_test_workload_2_codebuild_project_name
      description = "Test Workload 2"

      path_to_build_spec = local.tf_apply_path_to_buildspec

    },

  }

  # - Test CodePipeline pipelines -
  codepipeline_pipelines = {

    # Terraform Module Validation Pipeline for 'test-module-1' Terraform Module
    tf_module_validation_test_module_1 : {
      name = local.tf_module_validation_test_module_1_codepipeline_pipeline_name
      tags = {
        "Description" = "Test Module 1.",

      }

      stages = [
        # Clone from CodeCommit, store contents in  artifacts S3 Bucket
        {
          name = "Source"
          action = [
            {
              name     = "PullFromCodeCommit"
              category = "Source"
              owner    = "AWS"
              provider = "CodeCommit"
              version  = "1"
              configuration = {
                BranchName     = "main"
                RepositoryName = local.test_module_1_repository_name
              }
              input_artifacts = []
              #  Store the output of this stage as 'source_output_artifacts' in connected the Artifacts S3 Bucket
              output_artifacts = ["source_output_artifacts"]
              run_order        = 1
            },
          ]
        },

        # Run Terraform Test Framework
        {
          name = "Build_TF_Test"
          action = [
            {
              name     = "TerraformTest"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_test_test_module_1_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_tf_test_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_tf_test_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Run Checkov
        {
          name = "Build_Checkov"
          action = [
            {
              name     = "Checkov"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.chevkov_test_module_1_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

      ]

    },

    # Terraform Module Validation Pipeline for 'test-module-2' Terraform Module
    tf_module_validation_test_module_2 : {
      name = local.tf_module_validation_test_module_2_codepipeline_pipeline_name
      tags = {
        "Description" = "Test Module 2.",

      }

      stages = [
        # Clone from CodeCommit, store contents in  artifacts S3 Bucket
        {
          name = "Source"
          action = [
            {
              name     = "PullFromCodeCommit"
              category = "Source"
              owner    = "AWS"
              provider = "CodeCommit"
              version  = "1"
              configuration = {
                BranchName     = "main"
                RepositoryName = local.test_module_2_repository_name
              }
              input_artifacts = []
              #  Store the output of this stage as 'source_output_artifacts' in connected the Artifacts S3 Bucket
              output_artifacts = ["source_output_artifacts"]
              run_order        = 1
            },
          ]
        },

        # Run Terraform Test Framework
        {
          name = "Build_TF_Test"
          action = [
            {
              name     = "TerraformTest"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_test_test_module_2_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_tf_test_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_tf_test_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Run Checkov
        {
          name = "Build_Checkov"
          action = [
            {
              name     = "Checkov"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.chevkov_test_module_2_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

      ]

    },


    # Terraform Deployment Pipeline for 'test-workload-1'
    tf_deployment_test_workload_1 : {

      name = local.tf_deployment_test_workload_1_codepipeline_pipeline_name
      tags = {
        "Description" = "Test workload 1.",
      }

      stages = [
        # Clone from CodeCommit, store contents in  artifacts S3 Bucket
        {
          name = "Source"
          action = [
            {
              name     = "PullFromCodeCommit"
              category = "Source"
              owner    = "AWS"
              provider = "CodeCommit"
              version  = "1"
              configuration = {
                BranchName     = "main"
                RepositoryName = local.test_workload_1_repository_name
              }
              input_artifacts = []
              #  Store the output of this stage as 'source_output_artifacts' in connected the Artifacts S3 Bucket
              output_artifacts = ["source_output_artifacts"]
              run_order        = 1
            },
          ]
        },

        # Run Terraform Test Framework
        {
          name = "Build_TF_Test"
          action = [
            {
              name     = "TerraformTest"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_test_test_workload_1_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_tf_test_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_tf_test_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Run Checkov
        {
          name = "Build_Checkov"
          action = [
            {
              name     = "Checkov"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.chevkov_test_workload_1_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Apply Terraform
        {
          name = "Apply"
          action = [
            {
              name     = "TerraformApply"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_apply_test_workload_1_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

      ]

    },

    # Terraform Deployment Pipeline for 'test-workload-2'
    tf_deployment_test_workload_2 : {

      name = local.tf_deployment_test_workload_2_codepipeline_pipeline_name
      tags = {
        "Description" = "Test workload 2.",
      }

      stages = [
        # Clone from CodeCommit, store contents in  artifacts S3 Bucket
        {
          name = "Source"
          action = [
            {
              name     = "PullFromCodeCommit"
              category = "Source"
              owner    = "AWS"
              provider = "CodeCommit"
              version  = "1"
              configuration = {
                BranchName     = "main"
                RepositoryName = local.test_workload_2_repository_name
              }
              input_artifacts = []
              #  Store the output of this stage as 'source_output_artifacts' in connected the Artifacts S3 Bucket
              output_artifacts = ["source_output_artifacts"]
              run_order        = 1
            },
          ]
        },

        # Run Terraform Test Framework
        {
          name = "Build_TF_Test"
          action = [
            {
              name     = "TerraformTest"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_test_test_workload_2_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_tf_test_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_tf_test_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Run Checkov
        {
          name = "Build_Checkov"
          action = [
            {
              name     = "Checkov"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.chevkov_test_workload_2_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

        # Apply Terraform
        {
          name = "Apply"
          action = [
            {
              name     = "TerraformApply"
              category = "Build"
              owner    = "AWS"
              provider = "CodeBuild"
              version  = "1"
              configuration = {
                # Reference existing CodeBuild Project
                ProjectName = local.tf_apply_test_workload_2_codebuild_project_name
              }
              # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
              input_artifacts = ["source_output_artifacts"]
              # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
              output_artifacts = ["build_checkov_output_artifacts"]

              run_order = 1
            },
          ]
        },

      ]

    },

  }

}

# - Unit Tests -
run "input_validation" {
  command = plan

  # Intentional invalid values to test functionality. These variables overwrite the above variables (helpful for testing).
  variables {
    # Intentional project_prefix that is longer than max of 40 characters (overwrite of above global variable)
    project_prefix = "this_is_a_project_prefix_and_it_is_over_40_characters_and_will_cause_a_failure"

    # CodeCommit - Intentional repository name that is longer than max of 40 characters
    codecommit_repos = {
      # Custom Terraform Module Repo
      test_module_1 : {

        repository_name = "this_is_a_repository_name_loger_than_100_characters"
        description     = "Test Module 1."
        default_branch  = "main"
        tags = {
          "ContentType" = "Test Module 1",

        },
      }
    }

    # CodeBuild - Intentional project name that is longer than max of 40 characters
    codebuild_projects = {
      # Test Module 1
      tf_test_test_module_1 : {
        name               = "this_is_a_project_name_and_it_is_longer_than_40_characters"
        description        = "Test Module 1 - Terraform Test"
        path_to_build_spec = local.tf_test_path_to_buildspec

      },


    }

    # CodePipeline - Intentional pipeline name that is longer than max of 40 characters
    codepipeline_pipelines = {

      # Terraform Module Validation Pipeline for 'test-module-1' Terraform Module
      tf_module_validation_test_module_1 : {
        name = "this_is_a_pipeline_name_and_it_is_longer_than_40_characters_and_will_fail"
        tags = {
          "Description" = "Test Module 1.",

        }

        stages = [
          # Clone from CodeCommit, store contents in  artifacts S3 Bucket
          {
            name = "Source"
            action = [
              {
                name     = "PullFromCodeCommit"
                category = "Source"
                owner    = "AWS"
                provider = "CodeCommit"
                version  = "1"
                configuration = {
                  BranchName     = "main"
                  RepositoryName = local.test_module_1_repository_name
                }
                input_artifacts = []
                #  Store the output of this stage as 'source_output_artifacts' in connected the Artifacts S3 Bucket
                output_artifacts = ["source_output_artifacts"]
                run_order        = 1
              },
            ]
          },

          # Run Terraform Test Framework
          {
            name = "Build_TF_Test"
            action = [
              {
                name     = "TerraformTest"
                category = "Build"
                owner    = "AWS"
                provider = "CodeBuild"
                version  = "1"
                configuration = {
                  # Reference existing CodeBuild Project
                  ProjectName = local.tf_test_test_module_1_codebuild_project_name
                }
                # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
                input_artifacts = ["source_output_artifacts"]
                # Store the output of this stage as 'build_tf_test_output_artifacts' in the connected Artifacts S3 Bucket
                output_artifacts = ["build_tf_test_output_artifacts"]

                run_order = 1
              },
            ]
          },

          # Run Checkov
          {
            name = "Build_Checkov"
            action = [
              {
                name     = "Checkov"
                category = "Build"
                owner    = "AWS"
                provider = "CodeBuild"
                version  = "1"
                configuration = {
                  # Reference existing CodeBuild Project
                  ProjectName = local.chevkov_test_module_1_codebuild_project_name
                }
                # Use the 'source_output_artifacts' contents from the Artifacts S3 Bucket
                input_artifacts = ["source_output_artifacts"]
                # Store the output of this stage as 'build_checkov_output_artifacts' in the connected Artifacts S3 Bucket
                output_artifacts = ["build_checkov_output_artifacts"]

                run_order = 1
              },
            ]
          },

        ]

      },


    }

  }

  # Check for intentional failure of variables defined above. We expect these to fail since we intentionally provided values that do not conform to the validation rules defined in the module's variable.tf file.
  expect_failures = [
    var.project_prefix,
    var.codecommit_repos,
    var.codebuild_projects,
    var.codepipeline_pipelines,

  ]
}

# - End-to-end Tests -
run "e2e_test" {
  command = apply

  # Using global variables defined above since additional variables block is not defined here

  # Assertions
  # CodeCommit - Ensure repositories have correct names after creation
  assert {
    condition     = aws_codecommit_repository.codecommit[0].name == "test-module-1"
    error_message = "Test CodeCommit Repo 1 name didn't match the expected value."
  }
  assert {
    condition     = aws_codecommit_repository.codecommit[1].name == "test-repo-2"
    error_message = "Test CodeCommit Repo 2 name didn't match the expected value."
  }
  assert {
    condition     = aws_codecommit_repository.codecommit[2].name == "test-workload-1"
    error_message = "Test CodeCommit Repo 2 name didn't match the expected value."
  }
  assert {
    condition     = aws_codecommit_repository.codecommit[2].name == "test-workload-2"
    error_message = "Test CodeCommit Repo 2 name didn't match the expected value."
  }

  # CodeBuild - Ensure projects have correct names after creation
  assert {
    condition     = aws_codebuild_project.codebuild[0].name == "TerraformTest-test-module-1"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[0].name}) didn't match the expected value (TerraformTest-test-module-1)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[1].name == "Checkov-test-module-1"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[1].name}) didn't match the expected value (Checkov-test-module-1)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[2].name == "TerraformTest-test-module-2"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[2].name}) didn't match the expected value (TerraformTest-test-module-2)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[3].name == "Checkov-test-module-2"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[3].name}) didn't match the expected value (Checkov-test-module-2)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[4].name == "TerraformTest-test-workload-1"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[4].name}) didn't match the expected value (TerraformTest-test-workload-1)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[5].name == "Checkov-test-workload-1"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[5].name}) didn't match the expected value (Checkov-test-workload-1)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[6].name == "TerraformTest-test-workload-2"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[6].name}) didn't match the expected value (TerraformTest-test-workload-2)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[7].name == "Checkov-test-workload-2"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[7].name}) didn't match the expected value (Checkov-test-workload-2)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[8].name == "TFApply-test-workload-1"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[8].name}) didn't match the expected value (TFApply-test-workload-1)."
  }
  assert {
    condition     = aws_codebuild_project.codebuild[9].name == "TFApply-test-workload-2"
    error_message = "The CodeBuild Project name (${aws_codebuild_project.codebuild[9].name}) didn't match the expected value (TFApply-test-workload-2)."
  }

  # CodePipeline - Ensure pipelines have correct names after creation
  assert {
    condition     = aws_codepipeline.codepipeline[0].name == "tf-module-validation-test-module-1"
    error_message = "The CodeBuild Project name (${aws_codepipeline.codepipeline[0].name}) didn't match the expected value (tf-module-validation-test-module-1)."
  }
  assert {
    condition     = aws_codepipeline.codepipeline[1].name == "tf-module-validation-test-module-2"
    error_message = "The CodeBuild Project name (${aws_codepipeline.codepipeline[1].name}) didn't match the expected value (tf-module-validation-test-module-2)."
  }
  assert {
    condition     = aws_codepipeline.codepipeline[2].name == "tf-deploy-test-workload-1"
    error_message = "The CodeBuild Project name (${aws_codepipeline.codepipeline[2].name}) didn't match the expected value (tf-deploy-test-workload-1)."
  }
  assert {
    condition     = aws_codepipeline.codepipeline[3].name == "tf-deploy-test-workload-2"
    error_message = "The CodeBuild Project name (${aws_codepipeline.codepipeline[3].name}) didn't match the expected value (tf-deploy-test-workload-2)."
  }

}

Debug Output

❯ terraform test
tests/main.tftest.hcl... in progress

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

runtime error: invalid memory address or nil pointer dereference
goroutine 25 [running]:
runtime/debug.Stack()
        /Users/runner/hostedtoolcache/go/1.21.5/x64/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
        /Users/runner/hostedtoolcache/go/1.21.5/x64/src/runtime/debug/stack.go:16 +0x1c
github.com/hashicorp/terraform/internal/logging.PanicHandler()
        /Users/runner/work/terraform/terraform/internal/logging/panic.go:58 +0x164
panic({0x1057bb600?, 0x1075d7c30?})
        /Users/runner/hostedtoolcache/go/1.21.5/x64/src/runtime/panic.go:914 +0x218
github.com/hashicorp/terraform/internal/backend/local.(*TestFileRunner).GetVariables(0x1400188bef8, 0x1400094c540, 0x14000951220, {0x0, 0x0, 0x1400050e000?})
        /Users/runner/work/terraform/terraform/internal/backend/local/test.go:1063 +0x530
github.com/hashicorp/terraform/internal/backend/local.(*TestFileRunner).run(0x1400188bef8, 0x14000951220, 0x140009512c0, 0x140009a3d70, 0x1400094c540)
        /Users/runner/work/terraform/terraform/internal/backend/local/test.go:405 +0x708
github.com/hashicorp/terraform/internal/backend/local.(*TestFileRunner).Test(0x1400188bef8, 0x140009512c0)
        /Users/runner/work/terraform/terraform/internal/backend/local/test.go:336 +0x48c
github.com/hashicorp/terraform/internal/backend/local.(*TestSuiteRunner).Test(0x14000237dd0)
        /Users/runner/work/terraform/terraform/internal/backend/local/test.go:140 +0x2bc
github.com/hashicorp/terraform/internal/command.(*TestCommand).Run.func1()
        /Users/runner/work/terraform/terraform/internal/command/test.go:230 +0x9c
created by github.com/hashicorp/terraform/internal/command.(*TestCommand).Run in goroutine 1
        /Users/runner/work/terraform/terraform/internal/command/test.go:224 +0xab4

Expected Behavior

Not crash

Actual Behavior

Crashing every single time

Steps to Reproduce

1. terraform init
2. terraform test

Additional Context

This appears to be the same issue that was identified in this GitHub Issue - #33780 that was closed.

References

• terraform test crash on multiple run blocks in file #33780

[novekm]

For additional context, as shown in the config I am using locals that are defined in a locals.tf file that's in the same tests directory as main.tftest.hcl.

Here's the tree:

├── backend.tf
├── codebuild.tf
├── codecommit.tf
├── codepipeline.tf
├── data.tf
├── iam.tf
├── locals.tf
├── s3.tf
├── tests
│   ├── README.md
│   ├── core.tftest.hcl
│   ├── locals.tf
│   └── main.tftest.hcl
└── variables.tf

[novekm]

More info - if I replace all the locals with hardcoded strings (e.g. "my-repo" instead of local.test_workload_1_repository_name, it doesn't crash. Are locals not supported, or am I referencing them incorrectly?

[liamcervante]

Hi @novekm, unfortunately you can't reference configuration locals from within the test file. The test files and their variables are evaluated outside of the Terraform context that the locals and other parts of the configuration are available within.

Locals and internal variables could rely on parts of the configuration that are only available during / after a plan or apply operation, while the test file has to make all the inputs available at the beginning of each operation, so that kind of workflow is unfortunately not possible.

However, the crash itself is definitely a bug - you should be getting an error message explaining why it's failing at the very least. I see you are running v1.7.0, if you upgrade to v1.7.1 is it still crashing. I think we fixed the crash within #34531, which was released in v1.7.1 last week.

Thanks!

[liamcervante]

locals that are defined in a locals.tf file that's in the same tests directory as main.tftest.hcl.

Terraform test would pick up variables defined within a terraform.tfvars variable file defined within the testing directory rather than a normal .tf file. Unfortunately you can't reference other variables from the global variables block yet. That is the feature request from the issue you've linked here which is currently going through prioritisation.

[novekm]

Ok gotcha, thank you! So to recap - variables are fine as long as they're in a terraform.tfvars variable file that resides in the tests directory, but locals.tf in the same directory will not work (current feature request you mentioned?). Or, do you mean the feature request is to add support to reference say variables.tf, dev.auto.tfvars, prod.auto.tfvars, from the global variables block?

I'm creating an AWS workshop about how to use this new feature of Terraform within the context of an AWS environment, so I want to make sure I have the correct messaging about what is currently supported 😃

I'll re-run the tests with v1.7.1 and keep you posted on if it crashes again. Thanks!

[liamcervante]

I'll try and summarise things a bit more verbosely here. Hopefully it's helpful!

Referencing local blocks from within variable definitions within Terraform test files will never be supported. Terraform will not load .tf files during the initialisation of a test file, instead the test files are initialised first with all their data loaded and then provided into regular Terraform files during the execution of a run block.

terraform test can already load variables defined directly in .tfvars files. It will load auto files automatically, and you can specify particular variable files with the -var-file argument. Currently, defining variables via the CLI or variable files is equivalent to defining variables directly within the testing file via the file-level variables block. All variables defined in this way must be concrete values, so they can't refer to other variables at all. This is different to variables blocks within run blocks, these can refer to earlier variable definitions.

For example:

# main.auto.tfvars

# This is variable file, it only contains static definitions for variables.

global_value = "hello"

# main.tftest.hcl

# This is our test file.

variables {
  
  # This is a file-level variables block. We can provide more static definitions for variables here.

  bad_value  = var.global_value # Wrong! You can't refer to other variables from this context.
  good_value = "hello"
}

run "test" {
  variables {

    # This is a run-level variables block. We can provide more static definitions here, and we can
    # refer to other variables here as well.

    local_value      = "hello"
    run_good_value   = var.good_value # Right! We can reference the variables defined at the file level.
    run_global_value = var.global_value # Right! We can reference the variables from the files here as well.

  }
}

That's a snapshot of the current set of available features. The linked feature request related to this line:

  bad_value  = var.global_value # Wrong! You can't refer to other variables from this context.

When we implement the linked feature request, we will allow referencing "global" variables (variables defined externally to the test file such as in variable files or on the CLI) within file-level variables blocks. Similar to how you can reference earlier variables from within run-level variables.

Hope this makes sense!

[liamcervante]

I'm going to close this issue, as I think this has been fixed in the latest versions.

[novekm]

Hi @liamcervante sorry the delay. Yes, I have tested in v1.7.2 and this is now resolved. It throws a correct error mentioning that the local cannot be used:

│ Error: Variables not allowed
│ 
│   on tests/main.tftest.hcl line 75, in variables:
│   75:       name = local.tf_module_validation_module_aws_tf_cicd_codepipeline_pipeline_name
│ 
│ Variables may not be used here.
╵
  run "e2e_test"... skip
tests/main.tftest.hcl... tearing down
tests/main.tftest.hcl... fail

Thank you so much for all of the help and the detailed explanation! The code sample helped clear up everything for me. Thanks again 🙏🏾