cmd/go: go get module/pkg@master doesn't seem to work sometimes

[mvdan]

Here is a standalone reproducer:

#!/bin/bash
  
docker run -i golang:1.14rc1 <<-SCRIPT

        mkdir foo
        cd foo
        go mod init test.tld/foo

        go get -d github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0
        go list -m -f {{.Version}} github.com/grpc-ecosystem/go-grpc-prometheus

        go get -d github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@master

SCRIPT

It's minified from a real situation at work. You can see that https://github.com/grpc-ecosystem/go-grpc-prometheus/tree/master/packages/grpcstatus exists at master, but it didn't exist when 1.2.0 was tagged, which is the latest release.

The output is:

go get github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@master: module github.com/grpc-ecosystem/go-grpc-prometheus@latest found (v1.2.0), but does not contain package github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus

This message doesn't make sense to me. If I asked for @master, why is it settling for @latest?

If I replace the commands for go get module@master; go build module/pkg, it works:

go get -d github.com/grpc-ecosystem/go-grpc-prometheus@master
go build github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus

This seems like a bug. I'm not sure what exactly is triggering it. CC @bcmills @jayconrod

[bcmills]

I can confirm that something is strange here.

go get manages to find the module, but then (for some reason) tries to re-resolve the package without that module.

foo$ go version
go version devel +450d0b2f Tue Feb 25 08:36:15 2020 +0000 linux/amd64

foo$ go mod init test.tld/foo
go: creating new go.mod: module test.tld/foo

foo$ go get -d github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0
…

foo$ go list -m -f {{.Version}} github.com/grpc-ecosystem/go-grpc-prometheus
v1.2.0

foo$ go get -d github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@master
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.1-0.20191002090509-6af20e3a5340
go: found github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus in github.com/grpc-ecosystem/go-grpc-prometheus v1.2.1-0.20191002090509-6af20e3a5340
go: finding module for package github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus
go get github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@master: module github.com/grpc-ecosystem/go-grpc-prometheus@latest found (v1.2.0), but does not contain package github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus

CC @matloob

[mvdan]

Precisely. Almost like it forgets that I specified @master halfway through its work. This might be an edge case, such as @master doesn't work when go-getting a package path within a module when the module is already at the latest version in go.mod.

[bcmills]

I can reproduce the same behavior with an explicit commit hash. Having the module already present at v1.2.0 seems to be essential.

foo$ go get -d github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: finding module for package github.com/prometheus/client_golang/prometheus
go: finding module for package golang.org/x/net/context
go: finding module for package google.golang.org/grpc
go: finding module for package google.golang.org/grpc/status
go: finding module for package google.golang.org/grpc/codes
go: downloading github.com/prometheus/client_golang v1.4.1
go: downloading google.golang.org/grpc v1.27.1
go: downloading golang.org/x/net v0.0.0-20200222125558-5a598a2470a0
go: found github.com/prometheus/client_golang/prometheus in github.com/prometheus/client_golang v1.4.1
go: found golang.org/x/net/context in golang.org/x/net v0.0.0-20200222125558-5a598a2470a0
go: found google.golang.org/grpc in google.golang.org/grpc v1.27.1
go: found google.golang.org/grpc/codes in google.golang.org/grpc v1.27.1
go: found google.golang.org/grpc/status in google.golang.org/grpc v1.27.1
go: downloading github.com/golang/protobuf v1.3.2
go: downloading golang.org/x/sys v0.0.0-20200122134326-e047566fdf82
go: downloading github.com/prometheus/common v0.9.1
go: downloading github.com/cespare/xxhash/v2 v2.1.1
go: downloading github.com/beorn7/perks v1.0.1
go: downloading golang.org/x/text v0.3.0
go: downloading github.com/prometheus/procfs v0.0.8
go: downloading github.com/prometheus/client_model v0.2.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1

foo$ go get -d github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@6af20e3a5340
go: found github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus in github.com/grpc-ecosystem/go-grpc-prometheus v1.2.1-0.20191002090509-6af20e3a5340
go: finding module for package github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus
go get github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus@6af20e3a5340: module github.com/grpc-ecosystem/go-grpc-prometheus@latest found (v1.2.0), but does not contain package github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus

[ianlancetaylor]

I guess this didn't get fixed for 1.15. Moving milestone to 1.16.

[bcmills]

Almost like it forgets that I specified @master halfway through its work.

I don't think it's quite that. Rather, it is trying to classify the arguments as either packages or modules before it computes the upgrades, and at that point in time github.com/grpc-ecosystem/go-grpc-prometheus/packages/grpcstatus is neither. (It is not yet a package because we haven't upgraded to the version containing that package, and it is certainly not a module either.)

[gopherbot]

Change https://golang.org/cl/254820 mentions this issue: cmd/go/internal/modget: factor out functions for argument resolution

[gopherbot]

Change https://golang.org/cl/255054 mentions this issue: cmd/go: test the behavior of 'go get' in module mode with package vs. module arguments