Recent update to `UseStateForUnknown` causes errors when adding to nested objects

[danafallon]

When we try to upgrade to 1.15.1, our terraform provider has a regression - it now outputs errors (Provider produced inconsistent result after apply) when users add a new entry to a nested attribute object. Each nested object has several attributes that use the UseStateForUnknown plan modifier because they're either computed fields (like uuid) or optional fields that we'll preserve existing values for if they aren't set in the terraform config.

Module version

v1.15.1

Relevant provider source code

func (r *PipelineResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
	resp.Schema = schema.Schema{
		MarkdownDescription: "Artie Pipeline resource. This represents a pipeline that syncs data from a single source (e.g., Postgres) to a single destination (e.g., Snowflake).",
		Attributes: map[string]schema.Attribute{
			"uuid":                        schema.StringAttribute{Computed: true, PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
			"name":                        schema.StringAttribute{Required: true, MarkdownDescription: "The human-readable name of the pipeline. This is used only as a label and can contain any characters."},
			"tables": schema.MapNestedAttribute{
				Required:            true,
				MarkdownDescription: "A map of tables from the source database that you want to replicate to the destination.",
				NestedObject: schema.NestedAttributeObject{
					Attributes: map[string]schema.Attribute{
						"uuid":                   schema.StringAttribute{Computed: true, PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
						"name":                   schema.StringAttribute{Required: true, MarkdownDescription: "The name of the table in the source database."},
						"schema":                 schema.StringAttribute{Optional: true, Computed: true, PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}, MarkdownDescription: "The name of the schema the table belongs to in the source database. This must be specified if your source database uses schemas (such as PostgreSQL), e.g. `public`."},
						"enable_history_mode":    schema.BoolAttribute{Optional: true, Computed: true, Default: booldefault.StaticBool(false), PlanModifiers: []planmodifier.Bool{boolplanmodifier.UseStateForUnknown()}, MarkdownDescription: "If set to true, we will create an additional table in the destination (suffixed with `__history`) to store all changes to the source table over time."},
						"is_partitioned":         schema.BoolAttribute{Optional: true, Computed: true, Default: booldefault.StaticBool(false), PlanModifiers: []planmodifier.Bool{boolplanmodifier.UseStateForUnknown()}, MarkdownDescription: "If the source table is partitioned, set this to true and we will ingest data from all of its partitions."},
						"alias":                  schema.StringAttribute{Optional: true, Computed: true, PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}, MarkdownDescription: "An optional alias for the table. If set, this will be the name of the destination table."},
						"columns_to_exclude":     schema.ListAttribute{Optional: true, Computed: true, ElementType: types.StringType, PlanModifiers: []planmodifier.List{listplanmodifier.UseStateForUnknown()}, MarkdownDescription: "An optional list of columns to exclude from syncing to the destination."},
						"columns_to_include":     schema.ListAttribute{Optional: true, Computed: true, ElementType: types.StringType, PlanModifiers: []planmodifier.List{listplanmodifier.UseStateForUnknown()}, MarkdownDescription: "An optional list of columns to include in replication."},
						"columns_to_hash":        schema.ListAttribute{Optional: true, Computed: true, ElementType: types.StringType, PlanModifiers: []planmodifier.List{listplanmodifier.UseStateForUnknown()}, MarkdownDescription: "An optional list of columns to hash in the destination."},
						"skip_deletes":           schema.BoolAttribute{Optional: true, Computed: true, PlanModifiers: []planmodifier.Bool{boolplanmodifier.UseStateForUnknown()}, MarkdownDescription: "If set to true, we will skip delete events for this table and only process insert and update events."},
...

Terraform Configuration Files

resource "artie_pipeline" "dev_postgres_to_snowflake" {
  name = "Postgres (dev db) to Snowflake"
  tables = {
    "public.account" = {
       name   = "account"
       schema = "public"
    },
    "public.company" = {
       name   = "company"
       schema = "public"
    },
  }
}

Adding a new table to the tables object is a common use case for our provider (when a customer wants to start replicating an additional database table in their data pipeline). With the above config for an existing pipeline, imagine adding another table:

"public.api_key" = {
   name   = "api_key"
   schema = "public"
}

Behavior as of v1.15.0

The plan output looked like this:

# artie_pipeline.dev_postgres_to_snowflake will be updated in-place
  ~ resource "artie_pipeline" "dev_postgres_to_snowflake" {
        name                               = "Postgres (dev db) to Snowflake"
      ~ tables                             = {
          + "public.api_key" = {
              + alias                  = (known after apply)
              + columns_to_exclude     = (known after apply)
              + columns_to_hash        = (known after apply)
              + columns_to_include     = (known after apply)
              + enable_history_mode    = false
              + is_partitioned         = false
              + name                   = "api_key"
              + schema                 = "public"
              + skip_deletes           = (known after apply)
              + uuid                   = (known after apply)
            },
            # (3 unchanged elements hidden)
        }
        # (10 unchanged attributes hidden)
    }

And the update succeeded with no errors.

Behavior as of v1.15.1

Now the plan output looks like this:

# artie_pipeline.dev_postgres_to_snowflake will be updated in-place
  ~ resource "artie_pipeline" "dev_postgres_to_snowflake" {
        name                               = "Postgres (dev db) to Snowflake"
      ~ tables                             = {
          + "public.api_key" = {
              + enable_history_mode = false
              + is_partitioned      = false
              + name                = "api_key"
              + schema              = "public"
            },
            # (4 unchanged elements hidden)
        }
        # (10 unchanged attributes hidden)
    }

And after the update is applied, these errors show up for every field that uses the UseStateForUnknown plan modifier and didn't have a value specified in the config:

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].uuid: was null, but now cty.StringVal("e84b4aae-516f-477e-8529-546ac8116ea7").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].alias: was null, but now cty.StringVal("").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].skip_deletes: was null, but now cty.False.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].columns_to_hash: was null, but now cty.ListValEmpty(cty.String).
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].columns_to_exclude: was null, but now cty.ListValEmpty(cty.String).
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to artie_pipeline.dev_postgres_to_snowflake, provider "provider[\"registry.terraform.io/artie-labs/artie\"]" produced an unexpected new value: .tables["public.api_key"].columns_to_include: was null, but now cty.ListValEmpty(cty.String).
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

References

• all: Adjust UseStateForUnknown plan modifiers to preserve known null state values #1161

This seems to be the cause of the behavior change. The note in the PR description acknowledges that there could be use cases where this would cause a regression, and it looks like this is one 😅 The change correctly handles resources being created, but in this case the resource already exists and a nested object inside of it is what's being created.

Would it make sense to add an option to the UseStateForUnknown plan modifiers so that we can configure how they should treat null values?

Thank you for reviewing!

[SBGoods]

Hi @danafallon 👋🏾,
Sorry that you're running into issues here. As you mentioned, this is an expected regression when it comes to UseStateForUnknown being used with attributes that are changing during an update. In your example, the public.api_key attribute did not exist when the resource was being initially created, so it's previous state is null and the UseStateForUnknown plan modifier is replacing all non-configured unknown values with the previous state null values.

We could consider adding a new plan modifier that does the behavior that you're describing, setting the attributes of new elements to a collection to unknown while preserving the value of existing elements, but this would be considered a separate plan modifier from UseStateForUnknown. This new plan modifier would be set at the nested attribute level (on the "tables" attribute for example) because the modifier needs to loop through all of the collection's elements to identify newly added elements to keep unknown.

In the meantime, for a more immediate fix to the errors you're seeing, you can always implement a custom plan modifier based off of the old implementation of UseStateForUnknown or you can implement a new plan modifier for the nested attribute that implements what I described above. Do any of these options sound good to you?

[danafallon]

Hi @SBGoods, thanks for the reply! I like the idea of a new plan modifier that applies to collections and allows keeping newly added elements as unknown. I'll take a swing at implementing that as a custom plan modifier, and if I can't get it working then cloning the old implementation of UseStateForUnknown is a good backup plan. Thank you!

[diya-dhan]

Hi all,

We are facing the same issue in our provider as well. Are there any plans to bring in support for a new plan modifier as discussed above? While a custom plan modifier is a useful workaround, it’s not very practical in our scenario since we have multiple parameters of varying types. It would be highly beneficial if the framework offered built-in support for this functionality.

Thanks and regards,
Diya

[lechnerc77]

@SBGoods Is there any update available on that. At the moment two of our providers developed by SAP. Cannot be upgraded due to issues with the UseStateForUnkown modifier. The workaround to implement a custom modifier is from our perspective not feasible as the issue touches boolean and string values too.