Process only relevant variables in the testing framework

[liamcervante]

This PR adds in the concept of "relevant variables" to the testing framework. Previously, each run block would process every available variable and attempt to parse even those they had no need for. This resulted in some edge case behaviour as seen in the linked issue. Now, each run block first calculates the variables it needs, and then exclusively processes the subset of required variables.

In addition, this PR does some refactoring of the variables processing. Previously, the variables were processed in their entirety twice: once for the test itself, and then once again for the assertions after the test has finished. Now, we only process the variables once at the beginning of each run and use mutator functions FilterVariablesToConfig and AddVariablesToConfig to achieve the required configurations for the test and for processing the assertions.

Also, previously we were duplicating the defaults and conversion logic while preparing the variables for assertions. I realised I can push that processing further down the stack into the terraform package, and then reuse the existing prepareFinalInputVariableValue function instead of duplicating that logic.

The refactoring here will make future PRs easier (such as #34069) where we need to pull the variables available to the whole file. We can reuse the new function added here instead of doing this refactor at that point or adding a new function.

Fixes #34070

Target Release

1.6.2

Draft CHANGELOG entry

BUG FIXES

• terraform test: Fix bug where terraform test was processing and including all available variables for every test rather than only the ones required for that particular test.

[github-actions]

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.