Excessive DeepCopy usage in remote state backend leads to performance degradation

[alexott]

Terraform Version

1.8.1

Terraform Configuration Files

Just a big number of resources defined (> 100k)

Debug Output

With DeepCopy in remote state's WriteState function, I get ~1 resource/sec on a plan with 300k object:

Tue Apr 30 13:40:21 CEST 2024
 161 databricks_notebook
Tue Apr 30 13:45:21 CEST 2024
 436 databricks_notebook
Tue Apr 30 13:50:22 CEST 2024
 709 databricks_notebook
Tue Apr 30 13:55:22 CEST 2024
 996 databricks_notebook

Without it, I'm getting close to 2 resources/second:

Tue Apr 30 15:00:27 CEST 2024
 191 databricks_notebook
Tue Apr 30 15:05:30 CEST 2024
 708 databricks_notebook
Tue Apr 30 15:10:31 CEST 2024
1218 databricks_notebook
Tue Apr 30 15:15:31 CEST 2024
1732 databricks_notebook

Expected Behavior

Performance doesn't degrade

Actual Behavior

Performance degrades with the number of resources in plan

Steps to Reproduce

just normal usage with plan/apply.

Additional Context

No response

References

No response

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.