Skip to content

Always return all module instances during evaluation#24697

Merged
jbardin merged 12 commits intomasterfrom
jbardin/get-module-data
Apr 22, 2020
Merged

Always return all module instances during evaluation#24697
jbardin merged 12 commits intomasterfrom
jbardin/get-module-data

Conversation

@jbardin
Copy link
Copy Markdown
Member

@jbardin jbardin commented Apr 17, 2020

In order to be able to use module values, and handle operations like possibly invalid module indexes in conditional statements; whole, expanded modules must always be returned during evaluation. This is the same methodology applied to resources in #22846.

This also requires some new methods on the Changes and State, since the evaluation is happening from within a known addrs.ModuleInstance path, but we need to retrieve all instances of the child module being referenced. We need to collect all outputs from all possible module instances, as module outputs are handled individually (as opposed to resources which are stored as objects containing all their attributes), so this allows us to compile a data structure with all the outputs for each module in order to build the cty.Object values that evaluation expects without copying all module resources as well.

jbardin added 11 commits April 10, 2020 16:52
In order to be able to use module values, and handle operations like
possibly invalid module indexes in conditional statements, whole modules
must always be returned during evaluation.
Module references, like resource references, need to always return the
and object containing all instances in order to handle modules as single
values, and to postpone index evaluation to when the expression as whole
is evaluated.
Since modules need to be evaluated as whole objects, yet the outputs are
all handled individually, we need a method to collect and return all
output changes for a module from the plan, including all known
module instances.
The evaluationStateData needs the change to the GetModule method to work
with the new evaluator. This is using a deep copy of module instances,
which we will clean up after some changes to the states package.
We need all module instance outputs to build the objects for evaluation,
but there is no need to copy all the resource instances along with that.
This allows us to only return the output states, with enough information
to connect them with their module instances.
In order to efficiently build the module objects for evaluation, we need
to collect the outputs from a set of module instances. The ModuleOutputs
method will return a copy of the state outputs, while not requiring the
unnecessary copying of each entire module.
This way we don't need the extra copy of the entire module.
There is no codepath that can use this any longer, since we need to
evaluate the modules as whole objects.

This means we're going to have to live for now with invalid module
output references returning "object" errors rather that "module".
The module error is unfortunately less specific at the moment, but
change the error text here to match.
@jbardin jbardin requested a review from a team April 17, 2020 17:30
@ghost ghost added the sdkv1 [PRs only] Marks changes that may potentially need to be ported to the plugi nSDK label Apr 17, 2020
@jbardin jbardin changed the title Always return all modules during expansion Always return all module instances during evaluation Apr 17, 2020
There is no expansion during validation, so in order for module
references to work we need to ensure that the returned values are
unknown.
@jbardin jbardin force-pushed the jbardin/get-module-data branch from 1390fec to 92837e6 Compare April 20, 2020 14:21
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 20, 2020

Codecov Report

Merging #24697 into master will increase coverage by 0.03%.
The diff coverage is 66.88%.

Impacted Files Coverage Δ
plans/changes.go 0.00% <0.00%> (ø)
plans/changes_sync.go 0.00% <0.00%> (ø)
states/sync.go 0.00% <0.00%> (ø)
terraform/evaluate.go 46.06% <75.82%> (+4.35%) ⬆️
lang/eval.go 67.02% <81.81%> (+6.03%) ⬆️
states/state.go 21.31% <84.61%> (+7.55%) ⬆️
addrs/parse_ref.go 75.89% <100.00%> (ø)
states/module.go 32.55% <100.00%> (+3.28%) ⬆️
states/state_deepcopy.go 63.36% <100.00%> (+0.36%) ⬆️
states/statefile/version4.go 55.88% <100.00%> (+0.81%) ⬆️
... and 5 more

Copy link
Copy Markdown
Contributor

@pselle pselle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General approval, one question on a comment that looks out of date

var diags tfdiags.Diagnostics

// Output results live in the module that declares them, which is one of
// the child module instances of our current module path.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment still correct? It mentions instances, and looks lime moduleAddr is an addrs.Module

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's fine here, it's just describing the existing relationship in the state, not the resulting output of the function.


parentCfg := d.Evaluator.Config.DescendentForInstance(d.ModulePath)
callConfig, ok := parentCfg.Module.ModuleCalls[addr.Name]
if !ok {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice!

@ghost
Copy link
Copy Markdown

ghost commented May 23, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators May 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

sdkv1 [PRs only] Marks changes that may potentially need to be ported to the plugi nSDK

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants