Improve error message around "inconsistent vendoring" for vendored apps

[kvedurmu]

There have been some recent updates made to the Go Mod Vendor Buildpack and Dep Ensure Buildpacks to fail detection if a vendor directory exists (i.e the app is vendored). This means that if I'm using the language family Go buildpack and build an app using dep or go modules that's vendored, it should detect on go-dist and go-build. While the changes that were made are reasonable, it's now possible for builds to fail with an "inconsistent vendoring" error because I might not have vendored my app with the same version of Go used to build the app, that's rather hard to make sense of.

Here's an example of this.

pack build vendored-go-modules --buildpack gcr.io/paketo-buildpacks/go -v
Pulling image gcr.io/paketo-buildpacks/builder:full-cf
full-cf: Pulling from paketo-buildpacks/builder
Digest: sha256:a644524b74ce5e6f6aa061e4ca3da49c8ac763ba2a504accafbd3846a4b49505
Status: Image is up to date for gcr.io/paketo-buildpacks/builder:full-cf
Selected run image mirror index.docker.io/paketobuildpacks/run:full-cnb-cf
Pulling image index.docker.io/paketobuildpacks/run:full-cnb-cf
full-cnb-cf: Pulling from paketobuildpacks/run
Digest: sha256:5bfb17205e2f528b91e10db7adca72c9e1689e189fad23cf5ab8793fc1b7201e
Status: Image is up to date for paketobuildpacks/run:full-cnb-cf
Pulling image gcr.io/paketo-buildpacks/go
latest: Pulling from paketo-buildpacks/go
51721ff70849: Pull complete
c4e4389e75e1: Pull complete
d66019a3339b: Pull complete
775c8ad70a39: Pull complete
ef125ae8f7da: Pull complete
94e84e81ba0b: Pull complete
Digest: sha256:ac834dc641d631ae87ee091ec917ac7b6915633b5ff7ab4bc3c4c44623a4b342
Status: Downloaded newer image for gcr.io/paketo-buildpacks/go:latest
Adding buildpack paketo-buildpacks/go version 0.0.11 to builder
Adding buildpack paketo-buildpacks/dep version 0.0.155 to builder
Adding buildpack paketo-buildpacks/dep-ensure version 0.0.12 to builder
Adding buildpack paketo-buildpacks/go-build version 0.0.15 to builder
Adding buildpack paketo-buildpacks/go-dist version 0.0.193 to builder
Adding buildpack paketo-buildpacks/go-mod-vendor version 0.0.148 to builder
Setting custom order
Creating builder with the following buildpacks:
-> paketo-buildpacks/nginx@0.0.175
-> paketo-buildpacks/httpd@0.0.149
-> paketo-buildpacks/go@0.0.10
-> paketo-buildpacks/go-build@0.0.13
-> paketo-buildpacks/go-dist@0.0.191
-> paketo-buildpacks/go-mod-vendor@0.0.145
-> paketo-buildpacks/dep@0.0.152
-> paketo-buildpacks/dep-ensure@0.0.9
-> paketo-buildpacks/java@2.0.1
-> paketo-buildpacks/jmx@1.1.10
-> paketo-buildpacks/maven@2.1.0
-> paketo-buildpacks/azure-application-insights@1.5.1
-> paketo-buildpacks/image-labels@1.1.3
-> paketo-buildpacks/encrypt-at-rest@1.2.10
-> paketo-buildpacks/procfile@1.3.10
-> paketo-buildpacks/spring-boot@2.2.1
-> paketo-buildpacks/debug@1.2.10
-> paketo-buildpacks/dist-zip@1.3.7
-> paketo-buildpacks/google-stackdriver@1.6.2
-> paketo-buildpacks/gradle@2.0.1
-> paketo-buildpacks/sbt@2.0.1
-> paketo-buildpacks/apache-tomcat@1.3.3
-> paketo-buildpacks/executable-jar@2.0.1
-> paketo-buildpacks/bellsoft-liberica@2.10.0
-> paketo-buildpacks/environment-variables@1.0.1
-> paketo-buildpacks/dotnet-core@0.0.6
-> paketo-buildpacks/dotnet-core-conf@0.0.163
-> paketo-buildpacks/dotnet-core-runtime@0.0.179
-> paketo-buildpacks/dotnet-core-sdk@0.0.178
-> paketo-buildpacks/icu@0.0.84
-> paketo-buildpacks/node-engine@0.0.240
-> paketo-buildpacks/dotnet-core-aspnet@0.0.174
-> paketo-buildpacks/dotnet-core-build@0.0.105
-> paketo-buildpacks/php@0.0.7
-> paketo-buildpacks/httpd@0.0.132
-> paketo-buildpacks/nginx@0.0.151
-> paketo-buildpacks/php-composer@0.0.90
-> paketo-buildpacks/php-dist@0.0.164
-> paketo-buildpacks/php-web@0.0.108
-> paketo-buildpacks/nodejs@0.0.4
-> paketo-buildpacks/npm@0.1.64
-> paketo-buildpacks/yarn-install@0.1.72
-> paketo-buildpacks/node-engine@0.0.240
-> paketo-buildpacks/go@0.0.11
-> paketo-buildpacks/dep@0.0.155
-> paketo-buildpacks/dep-ensure@0.0.12
-> paketo-buildpacks/go-build@0.0.15
-> paketo-buildpacks/go-dist@0.0.193
-> paketo-buildpacks/go-mod-vendor@0.0.148
Using build cache volume pack-cache-8ab2ea616a9c.build
===> DETECTING
======== Output: paketo-buildpacks/go-mod-vendor@0.0.148 ========
failed
======== Results ========
pass: paketo-buildpacks/go-dist@0.0.193
fail: paketo-buildpacks/go-mod-vendor@0.0.148
pass: paketo-buildpacks/go-build@0.0.15
======== Output: paketo-buildpacks/dep-ensure@0.0.12 ========
failed
======== Results ========
pass: paketo-buildpacks/go-dist@0.0.193
pass: paketo-buildpacks/dep@0.0.155
fail: paketo-buildpacks/dep-ensure@0.0.12
pass: paketo-buildpacks/go-build@0.0.15
======== Results ========
pass: paketo-buildpacks/go-dist@0.0.193
pass: paketo-buildpacks/go-build@0.0.15
Resolving plan... (try #1)
paketo-buildpacks/go-dist  0.0.193
paketo-buildpacks/go-build 0.0.15
===> ANALYZING
Analyzing image "ccf1ab065b680aa3c4d0ba641972213c5044dc7a8fbcc3e686b464eb88747852"
Restoring metadata for "paketo-buildpacks/go-dist:go" from cache
Writing layer metadata for "paketo-buildpacks/go-dist:go"
Restoring metadata for "paketo-buildpacks/go-build:targets" from app image
Writing layer metadata for "paketo-buildpacks/go-build:targets"
Restoring metadata for "paketo-buildpacks/go-build:gocache" from cache
Writing layer metadata for "paketo-buildpacks/go-build:gocache"
===> RESTORING
Restoring data for "paketo-buildpacks/go-dist:go" from cache
Restoring data for "paketo-buildpacks/go-build:gocache" from cache
Retrieving data for "sha256:d99c22e4554f92a43a180c4ff03cc25f55f54e084fcaa4be403ea9b1c21c01c1"
Retrieving data for "sha256:9a25daa69bcd4e66076b83759b94addc9078fdf5d90f2989e819449c3022815f"
===> BUILDING
Go Distribution Buildpack 0.0.193
  Resolving Go version
    Candidate version sources (in priority order):
      <unknown> -> ""
    Selected Go version (using <unknown>): 1.14.6
  Reusing cached layer /layers/paketo-buildpacks_go-dist/go
Go Build Buildpack 0.0.15
  Executing build process
    Running 'go build -o /layers/paketo-buildpacks_go-build/targets/bin -buildmode pie -mod vendor .'
      Failed after 22ms
        go: inconsistent vendoring in /workspace:
        	github.com/BurntSushi/toml@v0.3.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        	github.com/satori/go.uuid@v1.1.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
        run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory
failed to execute 'go build': exit status 1
ERROR: failed to build: exit status 1
ERROR: failed with status code: 7

Relevant PRs:
paketo-buildpacks/go-mod-vendor#83
paketo-buildpacks/dep-ensure#14

Proposal
Could we look into improving the error message here? Perhaps could be as simple as a message underneath like "Was your app vendored using Go Version "

cc @sophiewigmore @ForestEckhardt

[fg-j]

I was able to reproduce this issue for the go mod case with these steps:

1. Clone the current version of the go mod Paketo sample app which specifies Go version 1.15 in its go.mod
2. Running go mod vendor with Go 1.13.15
3. pack build the app with the Go buildpack 0.2.1

Output:

pack build mod-sample-vendored13 --buildpack gcr.io/paketo-buildpacks/go:0.2.1
base: Pulling from paketobuildpacks/builder
Digest: sha256:01d328f9271f51dc0e2631bb9e59fa370c783822211a6216fdadc7c6fd9a54f8
Status: Image is up to date for paketobuildpacks/builder:base
base-cnb: Pulling from paketobuildpacks/run
Digest: sha256:c8237ca7fe27a0a962ff6532cedaf06554ed963bde81ff8a4000a9fafea671ea
Status: Image is up to date for paketobuildpacks/run:base-cnb
0.2.1: Pulling from paketo-buildpacks/go
Digest: sha256:dde5cfa69ba74fa869a4435cc7138462b59ee70de34f3d8e9f83562173af7714
Status: Image is up to date for gcr.io/paketo-buildpacks/go:0.2.1
0.9.1: Pulling from buildpacksio/lifecycle
Digest: sha256:53bf0e18a734e0c4071aa39b950ed8841f82936e53fb2a0df56c6aa07f9c5023
Status: Image is up to date for buildpacksio/lifecycle:0.9.1
===> DETECTING
[detector] paketo-buildpacks/go-dist  0.2.2
[detector] paketo-buildpacks/go-build 0.0.35
===> ANALYZING
[analyzer] Previous image with name "mod-sample-vendored13" not found
===> RESTORING
===> BUILDING
[builder] Paketo Go Distribution Buildpack 0.2.2
  Resolving Go version
    Candidate version sources (in priority order):
      <unknown> -> ""
[builder]
[builder]     Selected Go version (using <unknown>): 1.15.3

  Executing build process
[builder]     Installing Go 1.15.3
[builder]       Completed in 9.334s

[builder] Paketo Go Build Buildpack 0.0.35
[builder]   Executing build process
[builder]     Running 'go build -o /layers/paketo-buildpacks_go-build/targets/bin -buildmode pie -mod vendor .'
[builder]       Failed after 33ms
[builder] failed to execute 'go build': exit status 1
[builder]         go: inconsistent vendoring in /workspace:
        	github.com/gorilla/mux@v1.7.4: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt

        run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory

I think the root cause for the go mod case is connected with this change to how go mod vendor writes its modules.txt file which was introduced in Go 1.14. If we've deprecated Go 1.13 in the buildpack but users continue to use it locally to vendor dependencies, they'll almost certainly hit this issue.

[fg-j]

Is this still an issue, now that go mod vendor doesn't fail detection when a vendor directory is present?

[ForestEckhardt]

I am going to close this issue due to inactivity. If this is still a problem that you are seeing then feel free to reopen this issue!