x/build: catch problems in x/ repos gated by module go directive version sooner

[dmitshur]

Issue #67783 is an example of a test failure that happened only at tip (future Go 1.23 to be) and when the go directive in x/net is increased to Go language version 1.23. Bumping x/net go directive to 1.23 will not happen on its own soon since Go 1.22 and 1.21 are still supported now, but it would be good to have at least one builder that could catch such problems sooner anyway. This is a tracking issue for that.

Edit: Discussion here shows another case where having this implemented would help, by letting vet changes be rolled out more gradually without delay in signal.

More examples of issues that this could help with:

• x/tools: tests fails when run from a module (or with a go.work file) that uses go1.23+ #70082

CC @golang/release.

[findleyr]

This is a great idea. I suspect x/tools will pick up some problems when its go.mod directive is updated.

[dmitshur]

A sketch of how this can be implemented:

• add an input property to golangbuild, something like upgrade_go_directive
  • when set, builders run go get go@{target go directive} in each module before their usual go test ./...
• add a builder runmod that uses the new property, something like "tiplang", and have one linux/amd64 builder for each golang.org/x repository at tip (main Go repo doesn't need it)

Using x/build as an example, it would have a new post-submit builder named:

x_build-gotip-linux-amd64-tiplang

And the top-level module test sequence would have this one extra command:

 [...] # discover modules to test, download module dependencies in advance, etc.
+go get go@1.24
 go test ./...

I think we can target doing this this at tip, one builder per x/ repo, and afterwards can consider if there's a benefit to doing it for release branches and/or more than just linux/amd64. We'll also need to find a good place where the 1.24 value is determined (config vs golangbuild), aiming to make it easier to maintain the value over time. This can be a starting point.

[mknyszek]

We'll also need to find a good place where the 1.24 value is determined (config vs golangbuild), aiming to make it easier to maintain the value over time. This can be a starting point.

golangbuild already has logic to fetch the latest release version from go.dev for perf builders, so we can reuse that and not have any configuration at all.

Is that sufficient? Or is the problem that it's always one more than whatever that is? (We can always increment it in golangbuild from the latest one.)

[dmitshur]

Yeah, determining it automatically without configuration, like for perf builders, sounds good. (If we later on realize we want more flexibility for some reason, we can add that flexibility afterwards; no need to do it pre-emptively.) Note that the most canonical source of the "latest tip version" is probably the internal/goversion package itself, and we also have code that reads that (e.g., here). golangbuild might be able to just read that file from disk after fetching the toolchain. (But going through the go.dev/dl/?mode=json API is probably also fine for this.)

One correction to my previous comment – I realized now that go get go@1.24 can only be used after a released version is out (or at minimum a pre-release version, e.g., go get go@1.24rc1), so we'll likely want to edit the go.mod file with the lower-level go mod edit (e.g., go mod edit -go=1.24) instead of the higher-level go get.

[gopherbot]

Change https://go.dev/cl/643316 mentions this issue: main.star: add tiplang builders

[mknyszek]

The build-side work for this is all done now, but I haven't closed the issue because I want to confirm that this build which ran in the new mode (ignore the builder name) has a real failure (it should be showing up soon in postsubmit as well).

[dmitshur]

Looks real; it reproduces consistently with go1.24rc2 darwin/arm64. Glad to see it paying off already!

tools $ go test -count=1 ./cmd/signature-fuzzer/internal/fuzz-generator 
ok  	golang.org/x/tools/cmd/signature-fuzzer/internal/fuzz-generator	6.828s
tools $ go mod edit -go=1.24
tools $ go test -count=1 ./cmd/signature-fuzzer/internal/fuzz-generator
wraprand consistency check failed:
 checker: {f32:3 f64:12 i:76}
 caller: {f32:3 f64:12 i:77}
--- FAIL: TestIsBuildable (0.13s)
panic: bad [recovered]
	panic: bad
[…]
tools $ go mod edit -go=1.22.0                                         
tools $ go test -count=1 ./cmd/signature-fuzzer/internal/fuzz-generator
ok  	golang.org/x/tools/cmd/signature-fuzzer/internal/fuzz-generator	7.012s

[gopherbot]

Change https://go.dev/cl/647776 mentions this issue: main.star: set known issues for -tiplang builder in x/crypto and x/tools

[dmitshur]

As far as I can tell everything's working as expected. I filed #71613 and #71612 for the current findings.

Is there anything more to do here @mknyszek or ready to call this done?

[mknyszek]

I think this is done! Closing.