Feature Request: yamldecode() function

[kinghuang]

Current Terraform Version

Terraform v0.12.0
+ provider.yaml v2.1.0-linux-amd64

Use-cases

Terraform 0.12 introduces a jsondecode() function, which interprets a given string as JSON. I often deal with data in YAML format. It would be really handy to have a yamldecode() function.

Attempted Solutions

The latest version of terraform-provider-yaml adds a new data source named yaml_to_json that is meant to be used with jsondecode in Terraform 0.12. It can be used to load YAML data in Terraform.

terraform-provider-yaml 2.1.0 - 2019-05-25

Proposal

Terraform should natively provide a yamldecode() function, similar to the jsondecode() function.

References

N/A

[apparentlymart]

Hi @kinghuang! Thanks for suggesting this.

I agree it would be a good thing to have. We didn't include it initially because there are some complexities here with the fact that YAML is quite a complex language and it can represent things that the Terraform language type system can't represent.

In particular, YAML supports cyclic data structures (a collection value with itself as an element) while the Terraform language type system intentionally does not: all data structures must be finite.

However, that doesn't mean we can't offer a yamldecode function that supports a useful subset of YAML that the Terraform language type system can represent. We just need to make sure that the subset is well-defined and that our implementation gracefully handles anything outside of that subset with a proper informative error.

It looks like the yaml_to_json data source in terraform-provider-yaml would suffer a stack overflow if it were given a cyclic structure, because it's built from Go's encoding/json and that can't handle cyclic structures. It's not the end of the world if a specialized YAML provider plugin crashes, since it will only crash itself and not any other provider or Terraform Core, but for an interpolation function we'd need to be sure that this fails in a proper way rather than crashing Terraform Core.

I have the beginnings of a cautious YAML decoder implementation for the Terraform language type system I worked on as a side-project, which I'd like to finish up and use as the basis of yamldecode. In the short term we're focused on fixing some bugs and rough edges in the initial v0.12.0 release, so we won't be able to work on this immediately, but hopefully I can finish up that YAML decoder and get it integrated in the not-too-distant future.

[kinghuang]

Thanks for the quick and detailed reply! I agree that YAML can get quite complex. I'm mostly interested in parsing Docker Compose files, Kubernetes resource files, and some simple config files. A useful subset that includes anchors and aliases would be great, though that introduces the possibility of cyclic structures.

[apparentlymart]

Indeed, I think it should be possible to support anchors and aliases in the "normal" case, but we will need to reject any input where an alias appears inside the definition of its anchor.

So, this would be allowed:

a: &foo
  - x
b: *foo

But this would be rejected:

a: &foo
  - *foo

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.