archive/tar: write too long error when files are symlinked at least twice

[calvinbui]

Terraform Version

Terraform v1.3.1
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v3.68.0
+ provider registry.terraform.io/hashicorp/helm v2.4.1
+ provider registry.terraform.io/hashicorp/kubernetes v2.7.1

Terraform Configuration Files

...terraform config...

Debug Output

https://gist.github.com/calvinbui/044f82fb2fb6486f23750353bff6acdf

Expected Behavior

terraform apply follows the symlink to the original file, as if it was in the same directory/workspace.

Actual Behavior

Error: Failed to upload configuration files: failed copying file "/Users/calvin/common_file.tf" to archive: archive/tar: write too long

Steps to Reproduce

I symlink files between different orgs/workspaces/environments if they share a common configuration. Sometimes files are double symlinked, like in the example below.

Create a folder structure like so and run terraform init, then terraform apply to run into the error.

.
├── application
│   ├── common
│   │   ├── bucket.tf
│   │   └── common_file.tf -> ../../common_file.tf
│   ├── workspace1
│   │   ├── bucket.tf -> ../common/bucket.tf
│   │   ├── common_file.tf -> ../common/common_file.tf
│   │   └── vars.tf
│   └── workspace2
│       ├── bucket.tf -> ../common/bucket.tf
│       ├── common_file.tf -> ../common/common_file.tf
│       └── vars.tf
└── common_file.tf

Additional Context

This was working in Terraform <1.3.0.

I'm using tfenv on my Mac, but the same issue occurred on Linux without it.

References

No response

[willhughes-au]

Confirmed this is also happening for me on linux:

tf --version
Terraform v1.3.1
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.19.0
+ provider registry.terraform.io/hashicorp/helm v2.6.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.11.0

[sfroment]

same problem on macOS with v1.3.1 too

[willhughes-au]

Encountered the same issue, but with a different error message when symlink->symlink->folder.

So to modify Calvin's example for this extra test-case:

.
├── application
│   ├── common
│   │   └── extra_files/   -> ../../extra_files/
│   ├── workspace1
│   │   ├── extra_files/   -> ../common/extra_files/
│   │   └── vars.tf
│   └── workspace2
│       ├── extra_files/   -> ../common/extra_files/
│       └── vars.tf
└── extra_files
    ├── resource1.yml
    └── resource2.yml

This results in this error, during plan/apply in workspace1:
Error: Failed to upload configuration files: failed copying file "/path/application/workspace1/extra_files" to archive: read /path/application/workspace1/extra_files: is a directory

[calvinbui]

Issue still exists in 1.3.6

[willhughes-au]

Can confirm this still is happening in 1.3.8, despite hashicorp/go-tfe#623 being merged.

[willhughes-au]

And again tested on 1.3.9, still failing.

[brandonc]

hashicorp/go-tfe#623 was the prerequisite for a fix for this issue. That change will give us more options within terraform to pass additional options into the underlying config slug packer.