Terraform Testing Framework

[bscott]

Any Ideas around testing Terraform, my group has a requirement to tag all AWS instances, it would be nice to assert that a AWS TAG is missing. Thinking RSpec for Terraform or like Test-Kitchen.

Thoughts?

[apparentlymart]

On my team we wrote a little "linting" tool that takes as input a Terraform state and then prints out a list of complaints about how things were created. I think this isn't quite the same thing as what you're asking about, since it's only testing what Terraform thinks was created rather than what actually exists, but our purpose is the same as the example you gave: to check whether our resource tagging scheme is being respected.

We didn't release the tool publicly because it just encodes a few rules that are very specific to our environment, but it's a pretty simple thing: parse the state file as JSON, and then walk over all of the resources and apply various rules depending on resource type.

[bscott]

@apparentlymart thanks thats a good idea and a good place to start.

[bscott]

I created https://github.com/bscott/terralint to get something created and see what the community can come up with!

[radeksimko]

I like the idea of parsing tfstate files and making some conclusions based on it.

I think the challenge is to be able to test changes to an existing infrastructure. Testing fresh, new directory with no previous state IMO isn't that difficult and if you're not concerned too much about the costs & time, you could do this per pull request.

However testing incremental changes with an existing state isn't trivial, because you need to have clear "rollback" plans so that after you run such test, the infrastructure remains exactly in the same state as it was before.

[radeksimko]

btw. if you only care about AWS, there are some services you may leverage:

• https://aws.amazon.com/inspector/
• https://aws.amazon.com/config/

[apparentlymart]

@radeksimko being able to apply this sort of "linting" approach to Terraform plans was the main motivation for what I was doing over in #4610. As I noted over there, it could be used to detect and automatically apply "trivial" changes, it could be used to completely block other sorts of changes that are known to be incorrect/harmful.

Of course linting a plan can't catch all errors that would arise when applying the plan, so it doesn't quite get to what you were talking about, but there are several "common" mistakes I've seen in our infrastructure that I'd like to catch before they are live, and they're specific enough to our particular configuration that it doesn't feel right to put them in core Terraform.

[ncs-alane]

Hey all:

I'm excited to annouce kitchen-terraform. This gem is a group of Test Kitchen plugins that allows for integration testing of Terraform configuration using Inspec. Please check it out and feel free to contribute!

[stack72]

Hi @bscott

I am going to go ahead and close this one out. This has spawned some great links :) Thanks for opening it

Paul

[walterdolce]

Hi all,

I'm adding this here as I think it's a valuable tool (and the only one I could find out there) which allows people to "unit test"* terraform's AWS resources. It's young and surely needs love/attention. Still adds value when it comes to testing Terraform & AWS.

https://github.com/k1LoW/awspec

• I used quotes there because I believe that is not really unit testing. Unit testing assumes the ability to "dry run" or mock things out (something chefspec does for Chef resources). Rather, with awspec you make sure the AWS resources managed by Terraform are in the expected state.

[pradeepbhadani]

I used awspec in past and found it very useful to test aws resources.

[timurb]

I've been using plain RSpec with https://github.com/tobyclemson/ruby_terraform for running Terraform and testing against outputs -- works fine for integration-like tests.

For unit-like tests I had to write a library though for parsing output of terraform plan.
See #13738 for possible improvement around this.

[alex-harvey-z3q]

The design of Terraform is similar to the design of Puppet, and the Terraform plan is analogous to Puppet's catalog, therefore it seems to me that the Terraform equivalent of Rspec-puppet is missing from the Terraform ecosystem and should be written as soon as possible. I agree with @radeksimko that testing state changes is a harder problem, just as it is in Puppet, but it hardly changes the fact that an official Rspec-puppet equivalent would be incredibly useful.