cmd/go: do not modify the 'go.mod' or 'go.sum' files if the 'go' directive indicates a newer-than-supported version?

[bcmills]

@jayconrod notes in #34217 (comment):

One problem with this: adding go 1.14 to go.mod does not prevent old versions of the Go command from adding +incompatible versions. So a module could be broken unintentionally in a mixed environment.

There is a more general version of this problem: older versions of the Go toolchain may not know how to correctly interpret a go.mod file intended for a newer version, and thus could make arbitrarily wrong edits.

Probably we should have the go command error out instead of making changes if it thinks a go.mod file generated by a newer toolchain is incomplete or inaccurate.

[mvdan]

I agree that this is generally a good idea, but - will it work well in practice? I've seen a number of issues along the lines of "a command modified my go.mod when I didn't expect it to", or "mod tidy and test add and remove deps in circles". Would those bugs not be a problem, or is the plan to fix most/all of them for 1.14 too?

[bcmills]

This change would not address most of those issues, because they occur independent of the version in the go directive. (Generally they are fixes for inconsistencies between go mod tidy and other operations.)

[bcmills]

But yes: in general the plan is to remove bugs that introduce unnecessary/erroneous edits to the go.mod file as soon as they are reported.

[mvdan]

This change would not address most of those issues

Apologies if I was unclear; I mean that those bugs could be much more annoying with this change in place. For example:

1. I'm on 1.14, go.mod is on 1.13 as it needs to support that version
2. I try to do an action (such as go test ./...) which, due to a bug, tries to add a couple of lines to go.mod
3. I get an error, which I can't solve without bumping the version in go.mod

[bcmills]

If you're on 1.14 and your go.mod file says go 1.13, you would not get an error anyway: the Go 1.14 toolchain will still know how to interpret go.mod files using go 1.13 semantics, so there is no need to flag the mismatch as a potential error.

[mvdan]

The error I mean is that my version would try to make changes to go.mod due to a bug (such as adding unnecessary dependency lines), and so would refuse to make those changes and error.

I'll stop trying to explain this hypothetical scenario if I'm still not making sense :)

[bcmills]

The proposal is to produce an error only for versions of the go.mod file that the go command in use does not know how to interpret.

So the problem you describe could only occur for a go.mod file starting at go 1.15, and only for cases in which the file would be correctly interpreted by the go 1.14 toolchain save for some bug that causes an erroneous edit.

Ideally, we we have all of those bugs fixed before go 1.14 is released anyway, so I'm not worried about that actually happening. 🙂

[bcmills]

CC @ianlancetaylor

[gopherbot]

Change https://golang.org/cl/196519 mentions this issue: cmd/go: do not modify an existing go.mod file if its 'go' version is unsupported

[jayconrod]

We should be careful here -- I want to think about this a bit more before moving ahead.

We already assume there may be incompatible changes to the go.mod language. If we add a new kind of statement, an old version of the go command will reject it in the main module and ignore it in other modules. The assumption now is that module, require, and go statements won't change and may be interpreted safely by old versions of the go command.

This change makes that stricter: old versions of the go command can't modify a main go.mod file with a greater go version, even if the file appears syntactically correct. Maybe that's okay: a greater go version indicates newer Go language features may be used, and we shouldn't expect older toolchains to build code marked that way.

But I think a few questions need to be resolved:

1. Should the go command be even stricter and error out if it sees a go statement with a newer version in the main module regardless of whether it needs to make changes? This is probably the right thing to do if we need to change the interpretation of an existing statement. However, this should rarely be necessary, and we should avoid doing it if we can.
2. Is this necessary for proposal: cmd/go: ignore +incompatible versions as of Go 1.14 #34217? That issues suggests that a go 1.14 statement would make +incompatible versions non-canonical, which is a change to the meaning of an existing statement. I'd like to understand whether we can solve the problems described in that issue without such a change (but let's discuss over there).
3. Do we need this change outside of proposal: cmd/go: ignore +incompatible versions as of Go 1.14 #34217? Before I checked on how we handle go.mod syntax compatibility, I was thinking yes, but now I'm not sure. Maybe the go command can just print better error messages when it sees new go.mod syntax it doesn't recognize together with a go statement with a newer version.

[bcmills]

Should the go command be even stricter and error out if it sees a go statement with a newer version in the main module regardless of whether it needs to make changes?

No: it won't do that for the compile step, so it shouldn't do it for loading either.

Is this necessary for #34217?

Rendered moot by withdrawing #34217.

Do we need this change outside of #34217?

On CL 198319, you suggested that we avoid writing vendor/modules.txt if the go version in go.mod is unsupported, because it might not include information needed by higher Go versions.

@rsc has also brought up the possibility of adding redundant information to go.mod files in order to allow the module loader to prune out parts of the dependency graph in some situations. If that happens, the situation will be similar: older go tools may try to remove redundant information that newer versions would retain, even though both tools interpret that information in the same way.