Parse terraform files

[zunlihu]

Hi,

I am trying to parse terraform files(.tf) to extract Terraform-related information such as resources, module calls, etc.
Can you give me some instructions for that?

Thanks!

[apparentlymart]

Hi @zunlihu!

Can you please describe in a little more detail what you are trying to achieve? Terraform does not currently have a general API for external programs to parse configurations, but if you can describe your use-case we may be able to add some features to help you in a later release.

Thanks!

[zunlihu]

I want to collect all reousrce/data/modules types used in the .tf files.

For examples, given a tf file as follows, I can get that there is one resource and one data source in the file, and data type is "archive_file", resource type is "aws_lambda_permission".

data "archive_file" "lambda_zip" {
type = "zip"
source_file = "${path.module}/lambda_function.py"
output_path = "${path.module}/lambda_function.zip"
}

resource "aws_lambda_permission" "allow_bucket" {
statement_id = "AllowExecutionFromS3Bucket"
action = "lambda:InvokeFunction"
function_name = "${aws_lambda_function.function.arn}"
principal = "s3.amazonaws.com"
source_arn = "${data.aws_s3_bucket.src_bucket.arn}"
}

[apparentlymart]

Thanks for sharing the use-case, @zunlihu!

We don't currently have a mechanism in place to easily extract that sort of information, but we are planning to add new commands to Terraform to inspect configurations and plans in a machine-readable way (JSON, most likely) that I think will give you what you need.

Unfortunately I can't say at the moment when such a feature would be implemented, but we can hopefully work on a first version of it once we're finished our current project to replace the old HCL implementation with the new, as we'd discussed over in hashicorp/hcl#250.

[daluu]

Does the various 3rd party tools under #3170 help in the interim?

For me, this is also an issue of interest. At the moment, I'd like to extract out fields in a variable in *.tf file like

variable "instance_type" {
  default = "m5.xlarge"
}

getting both the variable name & the default value

[rvsingh011]

I also have a similar use case. I want to parse the resource file to get what the map out of the Resource Schema for fields. My go file should take a resource file as input and generate something like following.
for ex -

			"name": {
				Description:  "The name of the shared domain",
				Type:         schema.TypeString,
				Required:     true,
				ValidateFunc: validateDomainName,
			},
		},

Should output

{"name": String}

I am looking forward to parse using a regular expression. Is there a better way to do that. I am new to terraform and does not have a deep knowledge about the same.

[mildwonkey]

Hi @rvsingh011 ,
It sounds like you are describing something different: parsing the provider source code. You wouldn't be able to get the schema out of a terraform configuration file. I'd suggest you open a new ticket with that.

@zunlihu, if this is something you are still working with, terraform 0.12 has a new -json flag for the show command which might help you. It only displays the configuration if you are inspecting a plan, so I'm not sure if that meets your needs.

Terraform Enterprise customers also have access to Sentinel which has some features for inspecting configuration.

[apparentlymart]

In the meantime since my original reply, we open-sourced the terraform-config-inspect library, which is the component that does the shallow config parsing to generate docs for Terraform Registry. It's intentionally only exporting a shallow description of the top-level objects in the configuration, but that should be sufficient for @zunlihu's needs, at least. There's some more documentation about it in its GitHub repository.

[AKozintsov]

@apparentlymart Do you have any tool to parse terraform file that is contatins variables and their values, and convert it to Json or objects?

[mildwonkey]

@AKozintsov did you take a look at terraform-config-inspect, which @apparentlymart suggested in the comment above?

[mildwonkey]

I'm going to close this issue; terraform's json output (from both show and provider schemas) and the terraform-config-inspect tool are the proper methods for parsing terraform files. If you find that these tools are insufficient for your use case, please open a new feature request with detailed information about your use-case. Thank you!