terraform test skipped (because of provider error?), no error output or non-zero exit code

[kimor79]

Terraform Version

Terraform v1.14.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/kubernetes v3.0.1

Terraform Configuration Files

# main.tf
resource "kubernetes_manifest" "this" {
  manifest = {
    apiVersion = "test"
    kind       = "test"
    metadata = {
      name = "test"
    }
  }
}

# tftest
provider "kubernetes" {
  api_version = "client.authentication.k8s.io/v1beta1"
}

run "test-1" {
  command = plan
}

Debug Output

2026-01-21T14:00:15.757-0800 [INFO]  Terraform version: 1.14.3
2026-01-21T14:00:15.757-0800 [DEBUG] using github.com/hashicorp/go-tfe v1.94.0
2026-01-21T14:00:15.757-0800 [DEBUG] using github.com/hashicorp/hcl/v2 v2.24.0
2026-01-21T14:00:15.757-0800 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2026-01-21T14:00:15.757-0800 [DEBUG] using github.com/zclconf/go-cty v1.16.3
2026-01-21T14:00:15.757-0800 [INFO]  Go runtime version: go1.25.5
2026-01-21T14:00:15.757-0800 [INFO]  CLI args: []string{"terraform", "test"}
2026-01-21T14:00:15.757-0800 [DEBUG] Attempting to open CLI config file: /Users/kimo/.terraformrc
2026-01-21T14:00:15.757-0800 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2026-01-21T14:00:15.758-0800 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2026-01-21T14:00:15.758-0800 [DEBUG] ignoring non-existing provider search directory /Users/kimo/.terraform.d/plugins
2026-01-21T14:00:15.758-0800 [DEBUG] ignoring non-existing provider search directory /Users/kimo/Library/Application Support/io.terraform/plugins
2026-01-21T14:00:15.758-0800 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2026-01-21T14:00:15.758-0800 [INFO]  CLI command args: []string{"test"}
2026-01-21T14:00:15.793-0800 [DEBUG] checking for provisioner in "."
2026-01-21T14:00:15.793-0800 [DEBUG] checking for provisioner in "/usr/local/bin"
2026-01-21T14:00:15.793-0800 [DEBUG] TestSuiteRunner: found 1 files with 1 run blocks
main.tftest.hcl... in progress
2026-01-21T14:00:15.793-0800 [DEBUG] created provider logger: level=debug
2026-01-21T14:00:15.793-0800 [INFO]  provider: configuring client automatic mTLS
2026-01-21T14:00:15.798-0800 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/hashicorp/kubernetes/3.0.1/darwin_arm64/terraform-provider-kubernetes_v3.0.1_x5 args=[".terraform/providers/registry.terraform.io/hashicorp/kubernetes/3.0.1/darwin_arm64/terraform-provider-kubernetes_v3.0.1_x5"]
2026-01-21T14:00:15.800-0800 [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/hashicorp/kubernetes/3.0.1/darwin_arm64/terraform-provider-kubernetes_v3.0.1_x5 pid=96729
2026-01-21T14:00:15.800-0800 [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/hashicorp/kubernetes/3.0.1/darwin_arm64/terraform-provider-kubernetes_v3.0.1_x5
2026-01-21T14:00:15.818-0800 [INFO]  provider.terraform-provider-kubernetes_v3.0.1_x5: configuring server automatic mTLS: timestamp=2026-01-21T14:00:15.818-0800
2026-01-21T14:00:15.822-0800 [DEBUG] provider.terraform-provider-kubernetes_v3.0.1_x5: plugin address: address=/var/folders/br/sjq7f63j4gb02jcjdlf8x7nc0000gn/T/plugin524025130 network=unix timestamp=2026-01-21T14:00:15.822-0800
2026-01-21T14:00:15.822-0800 [DEBUG] provider: using plugin: version=6
  run "test-1"... skip
main.tftest.hcl... tearing down
2026-01-21T14:00:15.854-0800 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2026-01-21T14:00:15.855-0800 [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/hashicorp/kubernetes/3.0.1/darwin_arm64/terraform-provider-kubernetes_v3.0.1_x5 id=96729
2026-01-21T14:00:15.855-0800 [DEBUG] provider: plugin exited
main.tftest.hcl... skip

Success! 0 passed, 0 failed, 1 skipped.

Expected Behavior

The test marked as failed, or at least some non-debug output with the reason/error and a non-zero exit status

Actual Behavior

 % terraform test
main.tftest.hcl... in progress
  run "test-1"... skip
main.tftest.hcl... tearing down
main.tftest.hcl... skip

Success! 0 passed, 0 failed, 1 skipped.
% echo $?
0

Steps to Reproduce

1. terraform init
2. terraform test

Additional Context

If I remove api_version from the provider in the test file, the test fails successfully(?):

% terraform test
main.tftest.hcl... in progress
  run "test-1"... fail
╷
│ Error: Failed to construct REST client
│ 
│   with kubernetes_manifest.this,
│   on main.tf line 14, in resource "kubernetes_manifest" "this":
│   14: resource "kubernetes_manifest" "this" {
│ 
│ cannot create REST client: no client config
╵
main.tftest.hcl... tearing down
main.tftest.hcl... fail

Failure! 0 passed, 1 failed.
% echo $?
1

References

#37801

Generative AI / LLM assisted development?

No response

[crw]

Thanks for this report!

[radeksimko]

Hi @kimor79
I might be missing some important context about K8S and/or the provider - Could you clarify why exactly do you expect the test to fail in the first case? Or what is the output from a regular plan, if it fails?

[kimor79]

While the kubernetes provider is certainly not without issue, I'm not necessarily expecting test-1 to explicitly fail. But terraform test did determine that the test should be skipped, and the reason is not exposed in normal output (or with the verbose flag).

When running locally, I can review the test output and see that a test was skipped, however I would then have to re-run the test in an uncommon fashion to see what went "wrong". This problem is apparent when running in CI/CD. Unless someone is inspecting the output of the pipeline in detail, it's very easy to assume everything is working correctly.

A -error-on-skipped or similar flag that would set a non-zero exit code if a test was implicitly skipped would at least prevent these types of false positives.

[kimor79]

With the same provider config as the test

provider "kubernetes" {
  api_version = "client.authentication.k8s.io/v1beta1"
}

resource "kubernetes_manifest" "this" {
  manifest = {
    apiVersion = "test"
    kind       = "test"
    metadata = {
      name = "test"
    }
  }
}

plan output:

│ Error: Unsupported argument
│ 
│   on main.tf line 2, in provider "kubernetes":
│    2:   api_version = "client.authentication.k8s.io/v1beta1"
│ 
│ An argument named "api_version" is not expected here.

---

With api_version removed:

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Failed to construct REST client
│ 
│   with kubernetes_manifest.this,
│   on main.tf line 4, in resource "kubernetes_manifest" "this":
│    4: resource "kubernetes_manifest" "this" {
│ 
│ cannot create REST client: no client config

[dsa0x]

Thanks for reporting this @kimor79 . I confirmed the bug. The test-defined provider config is decoded during the test runtime, and it seems the errors raised there were being omitted. I will link a fix here when available.