Conversation
Make DAG walks test-able, and add tests for more complex graph ordering. We also add breadth-first for comparison, though it's not used currently in Terraform.
A topological walk was previously only done in Terraform via the concurrent method used for walking the primary dependency graph in core. Sometime however we want a dependency ordering without the overhead of instantiating the concurrent walk with the channel-based edges. Add TopologicalOrder and ReverseTopologicalOrder to obtain a list of nodes which can be used to visit each while ensuring that all dependencies are satisfied.
The dag package did not previously provide a topological walk of a given graph. While the existing combination of a transitive reduction with a depth-first walk appeared to accomplish this, depth-first is only equivalent with a simple tree. If there are multiple paths to a node, a depth-first approach will skip dependencies from alternate paths.
jbardin
force-pushed
the
jbardin/dag-walks
branch
from
63a7751 to
e3ca4be
Compare
apparentlymart
approved these changes
Jul 22, 2022
Contributor
|
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
Contributor
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In order to traverse a graph in "dependency order", meaning all dependencies of a node have been visited before the node itself is visited, you need to first create a topological sort of the graph nodes. Terraform core has always done this, accomplishing the sort through a parallel walk of all nodes, with each node becoming unblocked as soon as dependencies are satisfied, naturally creating a topological order for the visits. There however was no simple synchronous walk function which provided the same ordering of nodes. This led the refactoring package to erroneously use the only type of synchronous walk available,
ReverseDepthFirstWalk. Since the majority of move graphs reduce into a simple tree, the depth-first walk appears to be correct at first glance, however if you take a simple diamond shaped graph like so:You can see that a depth-first walk from the top cannot work, since
4is always going to be visited before one of it's dependencies, otherwise it would not be depth-first. (Note that breadth-first appears to work in this case, but that will just fail for differently shaped graphs where a node can be found at different depths via different paths, since breadth-first is also not a topological sort.)Add
TopologicalOrderandReverseTopologicalOrdermethods todag.AcyclicGraph, returning a sorted slice which can be used to visit the nodes in an order ensuring dependencies will always be satisfied. During thedagpackage refactoring, additional breadth-first walks were also added for comparison and additional test coverage.Use
ReverseTopologicalSortin therefactoringpackage and iterate directly over each move statement. We can refactor this test too, removing all the address literals and parsing them directly from the test fixture.Fixes #31489.