Inconsistent Plan when using dynamic block which iterates through data source output (that is known at apply stage)

[magodo]

Terraform Version

Terraform v0.12.28

Terraform Configuration Files

provider "azurerm" {
  features {}
}

variable "prefix" {
  default = "magodoinconsist"
}

variable "location" {
  default = "West Europe"
}

resource "azurerm_resource_group" "test" {
  name = "${var.prefix}-rg"
  location = var.location
}

resource "azurerm_storage_account" "test" {
    name                      = "${var.prefix}sa"
    resource_group_name       = azurerm_resource_group.test.name
    location                  = azurerm_resource_group.test.location
    account_replication_type  = "LRS"
    account_tier              = "Standard"
}

data "azurerm_monitor_diagnostic_categories" "test" {
  resource_id = azurerm_storage_account.test.id
}

resource "azurerm_monitor_diagnostic_setting" "test" {
  name               = "${var.prefix}-ds"
  target_resource_id = azurerm_storage_account.test.id
  storage_account_id = azurerm_storage_account.test.id

  dynamic "log" {
    for_each =  data.azurerm_monitor_diagnostic_categories.test.logs

    content {
      category = log.key
    }
  }

  dynamic "metric" {
    for_each =  data.azurerm_monitor_diagnostic_categories.test.metrics

    content {
      category = metric.key
      
    }
  }
}

Debug Output

https://gist.github.com/magodo/3a2221ef7cb0af4e4d2d6e1fae5537e7

Crash Output

Error: Provider produced inconsistent final plan                                                                                                                                    
                                                                                                                                                                                    
When expanding the plan for azurerm_monitor_diagnostic_setting.test to include
new values learned so far during apply, provider
"registry.terraform.io/-/azurerm" produced an invalid new value for .log:                                                                                                           
planned set element                                                                       
cty.ObjectVal(map[string]cty.Value{"category":cty.UnknownVal(cty.String),
"enabled":cty.True,                                                                                                                                                                 
"retention_policy":cty.ListValEmpty(cty.Object(map[string]cty.Type{"days":cty.Number,
"enabled":cty.Bool}))}) does not correlate with any element in actual.                    
                                                                                                                                                                                    
This is a bug in the provider, which should be reported in the provider's own                                                                                                       
issue tracker.                               
                                                                                                                                                                                    
                                                                                                                                                                                    
Error: Provider produced inconsistent final plan
                                                                                                                                                                                    
When expanding the plan for azurerm_monitor_diagnostic_setting.test to include                                                                                                      
new values learned so far during apply, provider                                                                                                                                    
"registry.terraform.io/-/azurerm" produced an invalid new value for .metric:                                                                                                        
planned set element                                                                                                                                                                 
cty.ObjectVal(map[string]cty.Value{"category":cty.UnknownVal(cty.String),                                                                                                           
"enabled":cty.True,                          
"retention_policy":cty.ListValEmpty(cty.Object(map[string]cty.Type{"days":cty.Number,                                                                                               
"enabled":cty.Bool}))}) does not correlate with any element in actual.                                                                                                              
                                             
This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Expected Behavior

The provision shall be successful with one terraform apply.

Actual Behavior

The provision will panic at first apply. While a second apply will succeed.

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

The azurerm provider version is v2.18.0.

I understand the reason of the issue here, while I want to figure out if this is an official solution for this, or if this is some known issue to be fixed in core?

References

• azurerm_monitor_diagnostic_setting: Error: Provider produced inconsistent final plan terraform-provider-azurerm#6254

[mildwonkey]

Hi @magodo , thank you for submitting this and I'm sorry you are experiencing it.

This is a duplicate of #22409, so I am going to close this issue in favor of that one. Thanks!

[mildwonkey]

I realized this issue had the very helpful debug log and there wasn't any conversation on the other one, so I'm going to leave this open instead. Sorry for the noise!

For anyone else reading this issue: The workaround for this situation is to run apply twice. You can also get the same result without a panic by running a targeted apply to first create the resource that's being referenced in the data source (terraform apply -target azurerm_storage_account.test) and then running a normal apply afterwards.

We are looking into improving both the workflow so and the error message so we aren't sending a misleading error message in this situation.

[mildwonkey]

#4149 is one of the proposals that might resolve this kind of issue

[jbardin]

Hi @magodo

Thanks for the example!

The bug here looks is being triggered by the data source not returning any values for logs (which is the case the linked duplicate issue mentions specifically). This is a bug caused by mishandling the special case of "dynamic" set blocks, which cannot represent a number of unknown values other than 1.

If you were expecting values for data.azurerm_monitor_diagnostic_categories.test.logs, it likely is a common situation of eventual consistency in the remote cloud API. You may be able to test for this by adding a long delay in a local-exec provisioner on azurerm_storage_account.test allowing the data to propagate. If it is an eventual consistency problem, it's not something that terraform will be able to solve. Fixing the bug will allow terraform to apply correctly, however it will take a subsequent plan and apply to update the delayed data.

[magodo]

Hi @jbardin @mildwonkey

With both terraform v0.13.1 and v0.13.2, this issue still remains:

Error: Provider produced inconsistent final plan

When expanding the plan for azurerm_monitor_diagnostic_setting.test to include
new values learned so far during apply, provider
"registry.terraform.io/hashicorp/azurerm" produced an invalid new value for
.log: planned set element
cty.ObjectVal(map[string]cty.Value{"category":cty.UnknownVal(cty.String),
"enabled":cty.True,
"retention_policy":cty.ListValEmpty(cty.Object(map[string]cty.Type{"days":cty.Number,
"enabled":cty.Bool}))}) does not correlate with any element in actual.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for azurerm_monitor_diagnostic_setting.test to include
new values learned so far during apply, provider
"registry.terraform.io/hashicorp/azurerm" produced an invalid new value for
.metric: block set length changed from 1 to 2.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.