Migrate Terraform commands from `cli.Ui` to `views.View` abstractions when rendering output. Investigate use of `Streams`, too.

[SarahFrench]

Background

Different parts of the code base render output using either the older cli.Ui abstraction, or the newer views.View abstraction.

Views were introduced in #27738 to minimise the code needed to render output in either Human or JSON formats. Since then, only commands that have the option to create JSON/machine-readable output have been migrated to using Views. Other commands are still using the old Ui approach, and it's possible for one command to use a mixture of the two, due to both being accessible in a command's Meta. Two parallel solutions for the same problem makes it harder to onboard to the codebase.

Proposal

We should ensure that we migrate all commands over to using Views eventually. This appears to have been the plan in the past, but understandably it's a low-priority piece of work:

terraform/internal/command/meta.go

Lines 635 to 636 in eee744c

	// Reconfigure the view. This is necessary for commands which use both
	// views.View and cli.Ui during the migration phase.

Commands that use the old UI approach still are:

• console
• fmt
• get
• graph
• import
• init
• login
• logout
• providers - all subcommands
• state - all subcommands
• taint
• unlock
• untaint
• version
• workspace - all subcommands

Also, an honourable mention for code in internal/command/meta_backend.go that mixes Views and Ui. That code is specific to the init command though, so could be addressed at the same time as the init command.

Getting inputs from users

Broadly this issue is specific to how we render outputs. Something probably beyond the scope of this issue and needs special consideration is how we collect input from users.

Input collection is done via the cli.Ui abstraction only. View is an alternative for how to handle output but there is no alternative for input. If we leave use of cli.Ui in the codebase for input we need to protect against it being used for output in new code after any migration to Views has occurred.

Problems

Views output warnings to stdout, whereas cli.Ui outputs warnings to stderr. If we don't handle this appropriately we're introducing breaking changes.

References

Code

The Meta allows access to both a Ui and a View:

terraform/internal/command/meta.go

Lines 75 to 79 in eee744c

	View *views.View
	Color bool // True if output should be colored
	GlobalPluginDirs []string // Additional paths to search for plugins
	Ui cli.Ui // Ui for output

The Meta struct also has some methods for handling output via Ui:

terraform/internal/command/meta.go

Line 684 in eee744c

func (m *Meta) showDiagnostics(vals ...interface{}) {

The newer alternative(s) of the method above is the Diagnostics method in the Operation interface which is implemented differently for Human or JSON output:

terraform/internal/command/views/operation.go

Lines 24 to 37 in eee744c

	type Operation interface {
	Interrupted()
	FatalInterrupt()
	Stopping()
	Cancelled(planMode plans.Mode)
	EmergencyDumpState(stateFile *statefile.File) error
	PlannedChange(change *plans.ResourceInstanceChangeSrc)
	Plan(plan *plans.Plan, schemas *terraform.Schemas)
	PlanNextStep(planPath string, genConfigPath string)
	Diagnostics(diags tfdiags.Diagnostics)
	}

Related issues

Potentially addresses #34768

[SarahFrench]

A consideration with this migration is whether HCP Terraform parses 'human' output from the unmigrated commands currently, and how confident we are to consider the output via Ui and output through a View in a OperationHuman equivalent. We probably want to work with the Core Cloud team to ensure this issue's changes doesn't cause any integration issues, if we do this work.

[SrushtiSapkale]

Hi! I’d like to work on this issue. Could you please share some pointers or guidance on how to get started? Thanks!

[crw]

@SrushtiSapkale Please see https://github.com/hashicorp/terraform/blob/main/.github/CONTRIBUTING.md.

[SarahFrench]

For anyone interested in this issue, please note that it will require the reviewer to work with other teams internal to HashiCorp before merging the changes.

A consideration with this migration is whether HCP Terraform parses 'human' output from the unmigrated commands currently, and how confident we are to consider the output via Ui and output through a View in a OperationHuman equivalent. We probably want to work with the Core Cloud team to ensure this issue's changes doesn't cause any integration issues, if we do this work.

[SarahFrench]

Another thing I've found: Some commands may use Streams to render output, such as the state show command. It's also used in test, graph and others.

I haven't investigated Streams yet, and why one may be used instead of Views (which has streams internally).

terraform/internal/cloud/test.go

Lines 98 to 101 in 808713f

	// View and Streams provide alternate ways to output raw data to the
	// user.
	View views.Test
	Streams *terminal.Streams

[apparentlymart]

FWIW I remember the original intention here was for terminal.Streams to deal with low-level problems like whether the output is a terminal at all and how wide it is if so, but that the views concept should be how commands should interact with those details, rather than accessing them directly.

IIRC a few commands did get rewritten from using c.Ui to c.Streams directly as an intermediate step because github.com/mitchellh/cli was no longer maintained and the decision hadn't yet been made to fork it as github.com/hashicorp/cli. My understanding was that the intention had always been to use views consistently in the long run though, and it was just that the status of the mitchellh/cli library created enough pressure to think about moving away from it even though there wasn't the resources to do full structuring with views at the time.

---

You're right that there's a missing piece here for accepting input. The Streams API has the lowest level part of this in its handling of stdin, but we didn't have a great answer for what "views-level" abstraction to wrap around it, because the initial priority was mainly about being able to generate JSON output and there's never any interactive input in that mode.

Note also that the interactive mode of terraform console doesn't gather input through c.Ui, and instead uses the third-party library github.com/chzyer/readline. I can't look at the Terraform code anymore but I looked at the equivalent code in another similar codebase and found that it's still using os.Stdin/etc directly instead of using the equivalents accessible through terminal.Streams, so unfortunately that seems like another entirely-separate strategy unlike how any other command is implemented. 😖

[SarahFrench]

Thanks @apparentlymart! That context about mitchellh/cli => hashicorp/cli is really useful. Hopefully we can get some movement back in the migration to Views in the near future.