cmd/go: make go mod download with no arguments leave go.sum alone

[aofei]

What version of Go are you using (go version)?

$ go version
go version go1.16.2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/home/aofei/bin"
GOCACHE="/home/aofei/.cache/go-build"
GOENV="/home/aofei/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/aofei/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aofei/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1210455100=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ git clone -b v1.1.5 https://github.com/aofei/sandid.git
$ cd sandid
$ go mod download
$ git diff
diff --git a/go.sum b/go.sum
index c221f64..ec9d676 100644
--- a/go.sum
+++ b/go.sum
@@ -3,6 +3,7 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0 h1:4G4v2dO3VZwixGIRoQ5Lfboy6nUhCyYzaqnIAPPhYs4=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
$ go mod tidy
$ git status
Not currently on any branch.
nothing to commit, working tree clean

What did you expect to see?

I expect to see that go mod download doesn't change anything. Or, at least it shouldn't fight with go mod tidy.

What did you see instead?

It adds unnecessary entries to go.sum.

---

A possible solution is to modload.TrimGoSum before modload.WriteGoMod(ctx).

Or, maybe go mod download shouldn't update go.mod or go.sum at all. Maybe people just expect it to fill the local cache. So another possible solution is to remove modload.WriteGoMod(ctx).

@jayconrod @bcmills @matloob

[aofei]

@gopherbot, please add labels GoCommand, modules

[jayconrod]

This was an intentional change in 1.16, but I don't think we should close this issue unless we're sure this is the behavior we want.

Previously, go mod download update go.mod and go.sum after loading the build list, but it didn't update go.sum after downloading modules. It seems like it should do both or neither (like -mod=readonly or -mod=mod essentially, though go mod download does not accept the -mod flag).

One particular problem we were trying to solve: with the switch to -mod=readonly was that in some cases, you need the sum for a module's content, even if you don't need any packages from it. That happens when a module in the build list has a path that's a prefix of another module. When the go command loads a package from one, it needs to confirm the package is not also in the other. For example, the package example.com/a/b could be in either module example.com/a or example.com/a/b, and we need to load both to be sure.

If one of the sums is missing, we need a command more focused than go mod tidy to add the sum. go mod download example.com/a does that now.

We've gotten a fair amount of feedback that this isn't what people expect when running go mod download without arguments. go mod download doesn't load the package graph, so it doesn't know whether go mod tidy would keep a sum or not. Maybe it just shouldn't write go.sum unless a specific module is requested.

[jayconrod]

Incidentally, until this issue is resolved, I'd recommend go list -test -deps ./... instead of go mod download for populating the module cache.

[ianlancetaylor]

Marking as release blocker to force a decision for 1.17.

[toothrot]

@jayconrod Do you have any more thoughts on this? Checking in as it is labeled a release-blocker for Go 1.17.

[rsc]

@jayconrod suggests that 'go mod download' with no arguments be defined to never edit go.sum.
That sounds reasonable for Go 1.17.

[gopherbot]

Change https://golang.org/cl/318629 mentions this issue: cmd/go: don't update go.sum in 'go mod download' without args

[gopherbot]

Change https://golang.org/cl/318811 mentions this issue: all: update tests to use 'go mod download all' to populate go.sum

[jayconrod]

@gopherbot Please backport to 1.16. We plan to revert to the 1.15 behavior.

[gopherbot]

Backport issue(s) opened: #46213 (for 1.15), #46214 (for 1.16).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[dmitshur]

@jayconrod This is still open, checking what is there still left to do here? Thanks.

[jayconrod]

The fix is ready, but I'm waiting on a review.

[gopherbot]

Change https://golang.org/cl/321956 mentions this issue: cmd/go: in TestScript/mod_replace, download an explicit module path

[gopherbot]

Change https://golang.org/cl/322369 mentions this issue: cmd/go: don't let 'go mod download' save sums for inconsistent requirements