cmd/go: Proposal: Modules without releases have the newest code fetched when go get -u is ran.

[integrii]

Currently modules that have no releases do not fetch the latest code, even when go get -u is run. Essentially there is a user-hostile edge-case in go get workflow:

If the dependency has releases, then use go get -u, else use go get module/path@master

The code that was present on the first go get of the module is still used until a new release is cut and go get -u is run. It is possible to manually update the module without a release by doing go get module/path@master, but I think it would make sense to see this behavior be the default for modules without releases.

Steps to reproduce

• Create a repository and init a module for it
• Do not cut a release of this repository
• Use unreleased module and repository as a dependency in another program
• Update the unreleased module's code
• Try to go get -u the unreleased module from your other program
• Notice that the build does not contain the newest code from the unreleased module

[integrii]

tag this with modules and Proposal maybe?

[jayconrod]

If I understand correctly, this is already the intended behavior. If you run go get example.com/mod@latest, the Go command will retrieve a list of tagged versions. If there are no tagged versions (or if all tagged versions are excluded), it will find the commit that HEAD points to, and it will encode that as a pseudo-version. If HEAD points to something different than before, you will get a new pseudo-version.

If you're using a proxy (for example, if you're using go1.13beta1, GOPROXY is set to https://proxy.golang.org,direct by default), you may not see new commits right away. The <module>/@latest endpoint is cached (I think the TTL is roughly an hour). So temporarily setting GOPROXY=direct or running go get example.com/mod@<commit> is the right thing to do there. #32870 discusses this.

Note that go get -u is changing behavior in go1.13, so if you're proposing a change in behavior in go get -u specifically, not go get <module>@latest, please explain the difference. In go1.13, go get -u will upgrade modules providing packages transitively imported by the package in the current directory.

[integrii]

Thanks for your reply jayconrod. I am not suggesting a change to go get <module>@latest. That functionality works as intended.

I think the existing behavior is expected and works well in all circumstances where module releases exist, but not when the module maintainer hasn't cut a release yet (effectively been running the module as version 0.0.0). In that case, the consumer of the module will unexpectedly not get the newest code in the repository through go get -u. Instead, they will get the exact same code they already have without a warning of any kind. They need to somehow know that the upstream module hasn't tagged any releases, so they need to run a different command entirely (go get example.com/mod@latest).

I am proposing that, in the case of there not being any releases for a module, go get -u example.com/mod will always effectively run go get example.com/mod@latest.

If the maintainer has not specified a stable release of their module (0.0.0), we should assume that things are still changing rapidly and without any guaranteed stable release. Therefore, it should be sensible to always fetch the newest code in the repository when go get -u example.com/mod is ran.

If we do not adopt this behavior, then developers have to know the hidden logic I described before:

If the dependency has releases, then use go get -u, else use go get module/path@master

[jayconrod]

I am proposing that, in the case of there not being any releases for a module, go get -u example.com/mod will always effectively run go get example.com/mod@latest

This is what go get -u does though. go get -u pkg upgrades modules that provide packages transitively imported by the package pkg. For these modules, it is as if you ran go get example.com/dependency@upgrade. The @upgrade suffix is like @latest except if the current version is newer than the "latest" version (for example, if the current version is a pre-release or a chronologically newer pseudo-version), the module won't be upgraded.

[integrii]

I just simulated this using a couple repos. I made a repo with module and a repo with an app that uses it.

I did the following with these two repos:

• Push a version of the module that says First commit
• Build my test-app binary for the first time
• Note that the following go mod was created:

module github.com/integrii/test-app

go 1.13

require github.com/integrii/go-module-test v0.0.0-20190820213515-828ef694dc70

• When using the app, it of course prints First commit
• I then changed the master branch print statement to say Second commit
• I then ran go get -u in my test-app repository
• The go.mod file for test-app remained the same
• The test-app binary still produced First commit when rebuilt

In order to actually get the Second commit string to build into test-app, I must run go get -u github.com/integrii/go-module-test@update (or @master or @latest).

Then, I found out today that by waiting for an hour or two, then re-running go get -u, I got more messaging about downloading and extracting files where previously I only had finding.

go: finding github.com...
go: downloading github.com...
go: extracting github.com...

Perhaps the behavior I am seeing (that you say already exists) only exists after some time has passed. My organization is not running any kind of module proxy and I have no environment variables configured to change the default proxy settings.

So my ask remains the same, but with the additional qualifier that reproducing this requires you to act relatively quickly between making updates to an un-tagged module and running go get -u.

[jayconrod]

Thanks for posting this example. From your description, I believe this is the same as #32870, so I'll close this issue.

As mentioned above, go1.13 uses a module proxy by default. If you have no other setting, you should see that go env GOPROXY prints https://proxy.golang.org,direct, which means that the Go command will try to fetch the module from the Go module mirror first, and if that fails, it will attempt to access the origin repository directly.

You can work around this issue by setting GOPROXY=direct when running go get commands, or, more narrowly, by setting GONOPROXY=github.com/integrii/test-app.

[integrii]

Thanks @jayconrod ! Having this issue de-duped is valuable and I appreciate your careful consideration.