in-house provider not working

[Fabian1976]

Terraform Version

0.13.5

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "~> 3.0"
    }
    myphpipam = {
      source = "terraform.cmc.nl/conclusionmc/myphpipam"
      version = "1.0.0"
    }
    template = {
      source = "hashicorp/template"
      version = "2.2.0"
    }
  }
  required_version = ">= 0.13"
}

Debug Output

I won't paste complete debug output. Only relevant section.

2020/12/23 08:23:27 [DEBUG] will search for provider plugins in /Users/fvanderhoevenconclusion.nl/.terraform.d/plugins
2020/12/23 08:23:27 [TRACE] getproviders.SearchLocalDirectory: /Users/fvanderhoevenconclusion.nl/.terraform.d/plugins is a symlink to /Users/fvanderhoevenconclusion.nl/.terraform.d/plugins
2020/12/23 08:23:27 [TRACE] getproviders.SearchLocalDirectory: found terraform.cmc.nl/conclusionmc/myphpipam v1.0.0 for darwin_amd64 at /Users/fvanderhoevenconclusion.nl/.terraform.d/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64
2020/12/23 08:23:27 [DEBUG] ignoring non-existing provider search directory /Users/fvanderhoevenconclusion.nl/Library/Application Support/io.terraform/plugins
2020/12/23 08:23:27 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2020/12/23 08:23:31 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/12/23 08:23:31 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
2020/12/23 08:23:31 [TRACE] getproviders.SearchLocalDirectory: found terraform.cmc.nl/conclusionmc/myphpipam v1.0.0 for darwin_amd64 at .terraform/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64
2020/12/23 08:23:31 [TRACE] providercache.fillMetaCache: including .terraform/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64 as a candidate package for terraform.cmc.nl/conclusionmc/myphpipam 1.0.0
2020/12/23 08:23:32 [DEBUG] checking for provisioner in "/Users/fvanderhoevenconclusion.nl/.terraform.d/plugins"
2020/12/23 08:23:32 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2020/12/23 08:23:33 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/12/23 08:23:33 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
2020/12/23 08:23:33 [TRACE] getproviders.SearchLocalDirectory: found terraform.cmc.nl/conclusionmc/myphpipam v1.0.0 for darwin_amd64 at .terraform/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64
2020/12/23 08:23:33 [TRACE] providercache.fillMetaCache: including .terraform/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64 as a candidate package for terraform.cmc.nl/conclusionmc/myphpipam 1.0.0
2020/12/23 08:23:33 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
- Using previously-installed -/aws v3.22.0
2020/12/23 08:23:33 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Using previously-installed -/template v2.2.0
- Using previously-installed hashicorp/template v2.2.0
- Using previously-installed hashicorp/aws v3.22.0
- Using previously-installed terraform.cmc.nl/conclusionmc/myphpipam v1.0.0
- Finding latest version of hashicorp/myphpipam...
2020/12/23 08:23:33 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/myphpipam/versions
2020/12/23 08:23:33 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/myphpipam/versions
2020/12/23 08:23:34 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2020/12/23 08:23:34 [DEBUG] GET https://registry.terraform.io/v1/providers/-/myphpipam/versions
2020/12/23 08:23:34 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/myphpipam/versions

Failed to install provider

Error while installing hashicorp/myphpipam: provider registry
registry.terraform.io does not have a provider named
registry.terraform.io/hashicorp/myphpipam

Expected Behavior

In-house provider should be installed and working.

Actual Behavior

The logs and output seem to indicate that the in-house provider is installed, but at the final step it tries to download it from the terraform registry, even though it allready installed it from the local file-system.

Steps to Reproduce

I've placed the custom provider in the following folder ~/.terraform.d/plugins. This is the output of tree:

.
└── terraform.cmc.nl
    └── conclusionmc
        └── myphpipam
            └── 1.0.0
                └── darwin_amd64
                    └── terraform-provider-myphpipam_v1.0.0

Then run terraform init. The logs indicate that the provider is installed but at the final step it also tries to download it from the registry, which obvously fails.

I've also looked in the <PROJECT>/.terraform/plugins folder of the terraform-base, and the plugin is also installed there.
Tree output of that folder:

└── terraform.cmc.nl
    └── conclusionmc
        └── myphpipam
            └── 1.0.0
                └── darwin_amd64 -> /Users/fvanderhoevenconclusion.nl/.terraform.d/plugins/terraform.cmc.nl/conclusionmc/myphpipam/1.0.0/darwin_amd64

So all seems to install fine, until the last step. Am I missing something?

[pselle]

@Fabian1976 Apologies for the delay, the team is back after some time off. In your debug log, as you saw, it appears Terraform is looking for the wrong provider: - Finding latest version of hashicorp/myphpipam.... This makes me wonder if perhaps you have a submodule using leveraging provider? Each module needs to declare its own required_providers.

[braunsonm]

This is definitely a bug @pselle I'm not sure what your comment refers to but we're seeing the same thing. If you use modules with a local provider it will never pass the init phase even though it shows it's been installed it will still try to reach out for the latest version on the registry.
This also fails when using dev overrides.

[pselle]

Hi @braunsonm, this issue is not yet triaged (thus the new label). Could you provide a simplified config that demonstrates the issue you're talking about?

My comment is discussing provider_requirements are a per-module concept -- when people report this behavior as a bug, often it is because they do not have required_providers in the modules called by the root Terraform configuration. If you're seeing a bug, we'd be happy to investigate when there's more information to replicate it.

[braunsonm]

@pselle Hey, you're correct, it's because it needs to be in each module... Very strange we have other TF projects that definitely don't do that and it's not an issue. Only seems to be with locally provided providers.

Thank you, sorry for my misunderstanding.

[Fabian1976]

Aah, yes. I've added to other submodules and then it appears to be working. But it is strange behaviour as braunsonm pointed out.