Skip to content

Fix race in state dependency encoding#30958

Merged
jbardin merged 2 commits intomainfrom
jbardin/dependencies-race
Apr 28, 2022
Merged

Fix race in state dependency encoding#30958
jbardin merged 2 commits intomainfrom
jbardin/dependencies-race

Conversation

@jbardin
Copy link
Copy Markdown
Member

@jbardin jbardin commented Apr 28, 2022

Instances of the same AbsResource may share the same Dependencies, which could point to the same backing array of values. Since address values are not pointers, and not meant to be shared, we must copy the value before sorting the slice in-place. This follows the documentation for ResourceInstanceObject which states that it must not be mutated once it has been created. Because individual instances of the same resource may be encoded to state concurrently, failure to copy the slice first can result in a data race.

Dependencies in core are handled at the resource level, so when evaluating the plan, we start with a single node for a resource with the dependencies already attached. As that resource is expanded, first from a nodeExpandPlannableResource to a PlannableResoure for each module instance, then finally to a number of PlannableResourceInstance, the list of dependencies is copied into each subsequent data structure, and finally to the instance state. This list is intended to be used as a value, like the addresses it contains, and should be read-only, but that invariant was broken by the sorting which happened during instance state encoding.

Update the encoding test to trigger the race, and add the states package to the race-detector tests.

Fixes #30894
Fixes #30935

@jbardin jbardin added 1.1-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged 1.2-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged labels Apr 28, 2022
@jbardin jbardin requested a review from a team April 28, 2022 15:38
@jbardin jbardin self-assigned this Apr 28, 2022
@jbardin jbardin force-pushed the jbardin/dependencies-race branch from 7ea8fac to cb800bb Compare April 28, 2022 15:43
jbardin added 2 commits April 28, 2022 11:44
Instances of the same AbsResource may share the same Dependencies, which
could point to the same backing array of values. Since address values
are not pointers, and not meant to be shared, we must copy the value
before sorting the slice in-place. Because individual instances of the
same resource may be encoded to state concurrently, failure to copy the
slice first can result in a data race.
This runs relatively quickly, and there is at least 1 test with
concurrent operations.
@jbardin jbardin force-pushed the jbardin/dependencies-race branch from cb800bb to 2bd7291 Compare April 28, 2022 15:44
@jbardin jbardin merged commit 0bcd04a into main Apr 28, 2022
@jbardin jbardin deleted the jbardin/dependencies-race branch April 28, 2022 15:56
@github-actions
Copy link
Copy Markdown
Contributor

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

@github-actions
Copy link
Copy Markdown
Contributor

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

1.1-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged 1.2-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TF state file corruption - character \u0000 has appeared during plan: runtime error: invalid memory address or nil pointer dereference

2 participants