In cloudflare_ruleset removing an existing rule causes recreation of later rules

[simonblake-mp]

Confirmation

• This is a bug with an existing resource and is not a feature request or enhancement. Feature requests should be submitted with Cloudflare Support or your account team.
• I have searched the issue tracker and my issue isn't already found.
• I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version

$ terraform -v
Terraform v1.12.2
on darwin_arm64

provider: v5.8.2

Affected resource(s)

• cloudflare_ruleset

Terraform configuration files

resource "cloudflare_ruleset" "example_ruleset" {
  zone_id = cloudflare_zone.origin_marketplacer_com.id
  kind    = "zone"
  phase   = "http_ratelimit"
  name    = "default"

  rules = [
    {
      description = "rule a"
      enabled     = true
      expression  = "(http.host in {\"a.com\"})"
      action      = "block"
      ratelimit = {
        characteristics = [
          "ip.src",
          "cf.colo.id",
        ]
        mitigation_timeout  = 0
        period              = 10
        requests_per_period = 200
        requests_to_origin  = true
      }
    },
    {
      description = "rule b"
      enabled     = true
      expression  = "(http.host in {\"b.com\"})"
      action      = "block"
      ratelimit = {
        characteristics = [
          "ip.src",
          "cf.colo.id",
        ]
        mitigation_timeout  = 0
        period              = 10
        requests_per_period = 200
        requests_to_origin  = true
      }
    },
    {
      description = "rule c"
      enabled     = true
      expression  = "(http.host in {\"c.com\"})"
      action      = "block"
      ratelimit = {
        characteristics = [
          "ip.src",
          "cf.colo.id",
        ]
        mitigation_timeout  = 0
        period              = 10
        requests_per_period = 200
        requests_to_origin  = true
      }
    }
  ]
}

Link to debug output

n/a

Panic output

n/a

Expected output

if you apply the above config, it creates three rules, all good. If you then remove 'rule a', then I'd expect rule a to be removed, and 'rule b/c' to be unchanged. If at some later time I add 'rule a' back to the config, I'd expect it to be inserted without change to 'rule b/c'

Actual output

when you plan removing rule a from the config above, terraform shows rules b and c "moving up" - that is rule a is reconfigured to match rule b, rule b is reconfigured to match rule c, and then the old rule c is deleted:

          ~ {
              ~ description = "rule a" -> "rule b"
              ~ id          = "78e2c032fa934fca87b78ec51a8aa876" -> (known after apply)
              ~ ref         = "78e2c032fa934fca87b78ec51a8aa876" -> (known after apply)
                # (4 unchanged attributes hidden)
            },
          ~ {
              ~ description = "rule b" -> "rule c"
              ~ id          = "2113c87e80ca491981fd326330c413f2" -> (known after apply)
              ~ ref         = "2113c87e80ca491981fd326330c413f2" -> (known after apply)
                # (4 unchanged attributes hidden)
            },
          - {
              - action      = "block" -> null
              - description = "rule c" -> null
              - enabled     = true -> null
              - expression  = "(http.host in {\"b.com\"})" -> null
              - id          = "801316ad90c04acdba36148b31838ceb" -> null
              - ratelimit   = {
                  - characteristics     = [
                      - "ip.src",
                      - "cf.colo.id",
                    ] -> null
                  - mitigation_timeout  = 0 -> null
                  - period              = 10 -> null
                  - requests_per_period = 200 -> null
                  - requests_to_origin  = true -> null
                } -> null
              - ref         = "801316ad90c04acdba36148b31838ceb" -> null
            },

behind the scenes, though, that doesn't appear to be what happens - instead, what appears to happen is that all three rules are deleted, and then rule b and rule c are recreated as entirely new rules - so before the plan above I had:

rule a id: 78e2c032fa934fca87b78ec51a8aa876
rule b id: 2113c87e80ca491981fd326330c413f2
rule c id :801316ad90c04acdba36148b31838ceb

then after the apply I have:

rule b id: 4303ec36d8084c5b82414ddca552e3ed
rule c id: 885170548236493ea560afd88dc63117

when you add rule a back to the console, the plan shows the reverse

          ~ {
              ~ description = "rule b" -> "rule a"
              ~ id          = "4303ec36d8084c5b82414ddca552e3ed" -> (known after apply)
              ~ ref         = "4303ec36d8084c5b82414ddca552e3ed" -> (known after apply)
                # (4 unchanged attributes hidden)
            },
          ~ {
              ~ description = "rule c" -> "rule b"
              ~ id          = "885170548236493ea560afd88dc63117" -> (known after apply)
              ~ ref         = "885170548236493ea560afd88dc63117" -> (known after apply)
                # (4 unchanged attributes hidden)
            },
          + {
              + action      = "block"
              + description = "rule c"
              + enabled     = true
              + expression  = "(http.host in {\"b.com\"})"
              + id          = (known after apply)
              + ratelimit   = {
                  + characteristics     = [
                      + "ip.src",
                      + "cf.colo.id",
                    ]
                  + mitigation_timeout  = 0
                  + period              = 10
                  + requests_per_period = 200
                  + requests_to_origin  = true
                }
              + ref         = (known after apply)
            },

and when applied, all three rules are recreated

rule a id: 2dfd6e99f53e4e5a9552cbb1badf51ce
rule b id: 8494b9eaa19d41d28994e7227c918d4f
rule c id :0dd37aeeefb7415cad9e4ea9647a5d67

Steps to reproduce

1. apply the config above
2. wait some period for the rules to start accruing metrics
3. remove rule a from the config above, apply
4. observe in the plan that terraform is going to "move everything up"
5. observe in the cloudflare console that rules b and c have been recreated, and all history lost

Additional factoids

From a narrow terraform POV, this is fine - you're getting the end result you asked for . However, back in the cloudflare console you lose all the history of rules b/c - 30 days of metrics vanish - that's a pretty major violation of the principle of least surprise. The 30 days of graphs for each rule are important to our use of Cloudflare, we don't like to erase them if we can avoid it, so we've had to resort to adding and removing rules in the console.

The frustrating thing here is that the Cloudflare console handles this beautifully - inserting and deleting rules from the middle of the rule corpus doesn't cause other rules to lose their history.

This is not a new problem, the v4 provider was even worse (in that it would recreate all the rules if there was any change in any rule in the corpus). I was hoping that the move from independent rule blocks with implicit ordering to a single rules list with explicit ordering in v5 would make this issue go away, sadly, it hasn't yet.

References

This is essentially a rehashing of #3357 , but for provider v5

[simonblake-mp]

After writing the above, I wondered "hmm, i wonder if setting that ref attribute changes anything". Turns out that if you set ref = "some-value" for a rule, that makes it more resistant to adjacent change. So with the ref set for my three test rules, then deleting rule a, the plan looks like

 ~ {
              ~ description = "rule a" -> "rule b"
              ~ id          = "4cf08ddfff4648e69a31cab3f1893605" -> "380944230b454d60bf66f2cc2a696e95"
              ~ ref         = "rulea" -> "ruleb"
                # (4 unchanged attributes hidden)
            },
          ~ {
              ~ description = "rule b" -> "rule c"
              ~ id          = "380944230b454d60bf66f2cc2a696e95" -> "c683bdac08a749dea2ad827c0d0fd5d8"
              ~ ref         = "ruleb" -> "rulec"
                # (4 unchanged attributes hidden)
            },
          - {
              - action      = "block" -> null
              - description = "rule c" -> null
              - enabled     = true -> null
              - expression  = "(http.host in {\"b.com\"})" -> null
              - id          = "c683bdac08a749dea2ad827c0d0fd5d8" -> null
              - ratelimit   = {
                  - characteristics     = [
                      - "ip.src",
                      - "cf.colo.id",
                    ] -> null
                  - mitigation_timeout  = 0 -> null
                  - period              = 10 -> null
                  - requests_per_period = 200 -> null
                  - requests_to_origin  = true -> null
                } -> null
              - ref         = "rulec" -> null
            },

notice how the id and ref for a rule are being carried up to the rule above. Which functionally means that I can add/remove rule a as much as I like, and rules b/c retain the same id (and thus history) throughout the changes to rule a. Result!

I'd argue that provider feedback to the humans still isn't ideal - instead of flagging rule a with a bunch of red minus symbols, it shows change to a and b and then rule c getting deleted, so there probably is still the germ of an improvement here, so I'll leave this to the devs to decide how they want to dispose of this ticket.

[KaydeeDee]

Thank you for submitting a ticket! I've reached out to the team to investigate.

[YeongJoonHwang]

@KaydeeDee Has there been any movement on this? To add to the convo, this issue is also impactful because it creates hard-to-read plans and causes friction for developers. If I add or delete a single rule, ideally, the plan would only show that one single rule being added or deleted so that an outside reader could know exactly what change will be applied.

[zakcutner]

Thanks for opening this issue! I think there are a couple of distinct problems here:

1. Terraform showing “known after apply” for the rule ID and ref attributes.
2. Terraform not showing a nice diff when rules are inserted or deleted in the middle of the list.

It looks like you already figured out that using the ref field solves (1)—we also have some docs on this, which provide more information about this solution.

Unfortunately, (2) is due to the way that Terraform shows diffs for lists in general (it doesn't “match” elements in the state and the plan, and I don't believe there's a way for providers to customize this behavior).

I do appreciate this is not ideal, but since there's not so much we can do in the provider to improve this, I think we should probably close this issue for now.

I did open a feature request in Terraform to allow providers to better customize list diffs as I didn't see an existing one: hashicorp/terraform-plugin-framework#1213. If this feature gets implemented in the future, we can certainly update the provider to make use of this.

[KaydeeDee]

Thanks so much Zak! I'll close this issue.