go/types: var declaration changes representation of its type (alias related)

[adonovan]

This is a minimized repro of another gotypesalias=1 problem affecting the x/tools gcimporter.
The root cause is clearly in go/types (at b24ec88).

package types // "go/types"

func TestAliasProblem(t *testing.T) {
	t.Setenv("GODEBUG", "gotypesalias=1")
	const src = `package a

var x T[B] // <-- this causes the problem! deleting it causes type of B to be ok

type T[_ any] struct{}
type A T[B]
type B = T[A]
`
	fset := token.NewFileSet()
	f, err := parser.ParseFile(fset, "a.go", src, 0)
	if err != nil {
		t.Fatal(err)
	}
	pkg, err := new(Config).Check("a", fset, []*ast.File{f}, nil)
	if err != nil {
		t.Fatal(err)
	}

	B := pkg.Scope().Lookup("B")
	t.Error(Unalias(B.Type())) // "invalid type" if var x is present, "a.T[a.A]" otherwise
}

[findleyr]

FWIW, putting this in go/types/testdata/manual.go and running go test -v -run=Manual go/types , I see that when GODEBUG=gotypesalias=1, there are only 3 delayed calls to validType. With gotypesalias=0, there are 6.

[adonovan]

Unalias is sticky. If called too soon on a type that's part of a cycle of declarations, it latches on to the invalid type which is destined to be updated, but too late. I guess that the var declaration calls Unalias on T[B].

[adonovan]

Not memoizing Unalias if the actual type is Invalid makes the test pass. Not sure whether that's a fix or a hack.

[findleyr]

Hmm, this is a tale as old as time, and gives me flashbacks to Named.Underlying and Named.Method, with generics. It took a lot of careful work to ensure that we don't touch the underlying or methods of instances until the type is fully set up. (but this type of problem didn't start with generics; I believe we had similar problems with method set completeness).

I suspect this may be very nontrivial to fix. Absent a formal partial ordering of type checking operations, the work tends to be whack-a-mole, delaying operations.

Not memoizing Unalias if the actual type is Invalid makes the test pass. Not sure whether that's a fix or a hack.

Sigh, we can change Alias.cleanup to say "really Unalias though".

[findleyr]

Not memoizing Unalias if the actual type is Invalid makes the test pass. Not sure whether that's a fix or a hack.

That may actually be a reasonable solution, for now. Perhaps pre-existing code used typ == Typ[Invalid] to detect whether the type was still being set up. When that code was transposed to call Unalias, it may have inadvertently picked up this bug.

CC @griesemer

[gopherbot]

Change https://go.dev/cl/576975 mentions this issue: go/types: fix bug in premature Unalias of alias cycle

[gopherbot]

Change https://go.dev/cl/579015 mentions this issue: cmd/compile/internal/types2: port CL 576975 to types2

[gopherbot]

Change https://go.dev/cl/579075 mentions this issue: go/types, types2: simplify TestUnaliasTooSoonInCycle (cleanup)