proposal: cmd/go: allow GOPRIVATE, GONOSUMDB, GONOPROXY to be defined in go.mod

[heyvito]

Background

It is not uncommon for companies to host Go libraries as private repositories, and even provide vanity URLs that redirect to specific directories in a private repository; Go already provides ways for dealing with those private repositories by using the GOPRIVATE, GONOSUMDB, and GONOPROXY environments, which requires users to export those environments and/or configure their tooling appropriately to handle those exceptions.

I have seen an array of different solutions for those cases, which ranges from repositories providing wrappers for go mod and go build by exporting the aforementioned environment variables before invoking the command, to requiring plugins for handling .env files containing the necessary set of environments for handling projects with private dependencies.

Solution

I believe Go can provide a far better developer experience by leveraging go.mod, which could (a) allow private repositories to be provided in a private, nosum, and noproxy blocks, or (b) allow private/nosum/noproxy dependencies to be marked as so through a comment, just like indirect does.

Option A: Augmenting go.mod with blocks

Just like require, one would be able to provide URLs that will be interpreted just like when passed to their related environment variables:

module my-application-using-private-mods

go 1.17

private (
	go.myvanityurl.com
	github.com/organization-with-private-repositories
)

require (
	// ...
)

Extra blocks such as nosum and noproxy could also be allowed, and all blocks provide the same behaviour as when setting environment variables:

list of glob patterns (in the syntax of Go's path.Match) of module path prefixes that should always be fetched directly or that should not be compared against the checksum database.

Option B: Leveraging comments like indirect

The same is already done with the indirect comment, but instead of implementing a new block type, a single comment may be placed on every private (nosumdb + noproxy) dependency:

module my-application-using-private-mods

go 1.17

require (
	github.com/organization-with-private-repositories/foo v1.0.0 // private
	github.com/google/uuid v1.1.2
	// ...
)

[ianlancetaylor]

CC @bcmills @jayconrod

[seankhliao]

Related #33985

[jayconrod]

This seems like a duplicate of #42343. #39005 and #43288 are also pretty close.

I think we should close this unless there's new information about why this is different. In particular, @rsc's comment is relevant:

In general go.mod is very narrowly scoped to information about dependency versions. It's not a catch-all for all possible configuration, in contrast to similar files in other systems, like package.json.

I would suggest that you check in a script with environment settings to be sourced, if you want your developers to be on the same page.

A couple other thoughts:

• go.mod may not be the right place, but go.work (cmd/go: add a workspace mode #45713) may be a better fit. cc @matloob
• We need very detailed semantics for a change here in order to understand the security implications. It must not be possible for a module not developed by the user (or their organization) to downgrade the user's security in any way. A module must not be able to opt itself out of sumdb checking or proxying.

[heyvito]

Hi there @jayconrod! Thank you so much for taking your time to review this!
@rsc's comment is extremely on point! I agree that go.mod should not be a catch-all; you may have noticed my proposal is specific for dependency management.

We need very detailed semantics for a change here in order to understand the security implications.

I see. That's a very specific point, and I'm afraid I don't have enough property to discuss this at all; it's for sure best for other contributors to express their opinion on this matter, just like you did.
Perhaps this configuration should be disregarded for dependencies, and only read for the application being built/tested?

It would be awesome if this could be covered by #45713 or any other proposal, if that's the case feel free to close this. <3

[rsc]

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group