Add terraform state registry

[TheFox0x7]

Adds terraform/opentofu state registry with locking

implements: #33644

---

Based mostly on: #34331 as it was in more complete state (it didn't error out on apply) and somewhat on: #33277

I also checked encrypted state, it works out of the box.

Docs PR: https://gitea.com/gitea/docs/pulls/357

[delvh]

What's keeping it a WIP PR?

[TheFox0x7]

The fact it's a messy merge of the two linked ones, the fact I haven't read the code properly and just removed failing parts and the fact I'm not sure it works properly aside from simple tests (for example if locking works as it should).

Reviews are welcome but I don't want to advertise this as ready for merge/review if I don't know it is.

[TheFox0x7]

@elbandi In case you're still around and interested in the topic, maybe you could you take a look? It's mostly based on your work which was in way better state than the other one (plus the other one is stale)
I mostly added versioning by serial and changed endpoints a bit so they match gitlab's system but you had some ideas for the feature which I could include.

unless you want to resume working on your version or take this one back.

[elbandi]

@elbandi In case you're still around and interested in the topic, maybe you could you take a look? It's mostly based on your work which was in way better state than the other one (plus the other one is stale) I mostly added versioning by serial and changed endpoints a bit so they match gitlab's system but you had some ideas for the feature which I could include.

unless you want to resume working on your version or take this one back.

This serial based versioning is not good. terraform state has only one version: the latest. Old states are obsolete.

But we want to store different states on one project: eg branch or different input variables. like in gitlab:

https://gitlab.com/api/v4/projects/<TARGET-PROJECT-ID>/terraform/state/<TARGET-STATE-NAME>

[TheFox0x7]

Makes sense but in that case why does gitlab have /projects/:id/terraform/state/:name/versions/:serial route?

[elbandi]

Makes sense but in that case why does gitlab have /projects/:id/terraform/state/:name/versions/:serial route?

Looks like, in gitlab you can only download and delete the old states. (maybe for audit?, i dont know what is it good for in normal circumstances).

But upload/lock/unlock based on state-name, UI hasn't got version control too.

[wxiaoguang]

So finally it catches 1.26 release.

[bircni]

Happy 1.26

[TheFox0x7]

Spoke too soon. Found a bug with encrypted state handling... I'll fix that and add it to tests so it's also covered. It's fairly simple, it just decodes the state wrong

[TheFox0x7]

Fixed. Dropped some validation rules which weren't that useful in the first place. I really don't expect anyone to actively push state files via curl with only two properties and even if I can reintroduce it if needed via a patch.

[bircni]

@TheFox0x7 did you create a pr for docs?

[TheFox0x7]

It's in PR description - https://gitea.com/gitea/docs/pulls/357

[elbandi]

This PR still lacks the ability to store more state files for one "package".
If you run same terraform code with different envs, you need different package names like (foorepo-devel, foorepo-stage, foo-prod).

I will try to rebase my code to this.

[TheFox0x7]

I'd argue that it makes little sense to store different states in the same package. If you can add different suffix to the name, you can have them in separate packages - why keep them in a single one? What's the usecase?

I'm not entirely against the idea of having the states user named instead of generic "tfstate", but it does complicate the UI side of locks as they now would have to list each state lock individually for the state in the package. Backend is less of an issue as you could just add a state name suffix to the tflock name of the lock record.

[elbandi]

why keep them in a single one? What's the usecase?

As i said: same terraform code with different envs. (devel, stage, prod). The difference only the input tfvars (=provider auth, setting params, like ip address, dns names, etc), but resources are same.

If we want to add separete envs for every developers, the number of state packages are grows fast (N project * M developers)

[wxiaoguang]

Maybe you can open a proposal issue with real examples (detailed config, commands, etc), then it would be very useful to help people to understand the problem in real world and discuss the design.

By the way, if we would come to a conclusion that "same terraform code with different envs" is a must and it is a breaking change (hopefully not), we can even spend more time on it to make it right before 1.26.0.

[TheFox0x7]

okay I dug for a bit and I have a good enough reason to nack this, sorry.
What you're basically talking about are workspaces, which let you split your one single config into separate states named as you'd like. I do see a use for this, sure, but the http backend does not support workspaces in the first place, so we have no way to do so.
I honestly can't see a downside with separate packages per state, especially since they all have different lineage to begin with.