types2, go/types: missing errors for invalid parameterized type declarations

[griesemer]

This is a new version of frozen #39938. The following code

package p

type T0[P any] struct {
        f P
}

type T1 struct { // <<< should get a cycle error here
        _ T0[T1]
}

is invalid because T1 would expand indefinitely. The previous fix (https://golang.org/cl/240519) could lead to infinite sequences of instantiations (see #48951).

Correct solution of this problem requires some form of type flow analysis and is related to (and may be addressed with) #48098.

[gopherbot]

Change https://golang.org/cl/355732 mentions this issue: cmd/compile/internal/types2: avoid infinite expansion for invalid recursive generic types

[griesemer]

cc: @findleyr

[griesemer]

The above program currently crashes the compiler with a simple nil pointer panic. @danscales is there an easy way to address this?

More generally, could cases like these be detected in the type checker through some variation of the nomono checker? cc: @mdempsky

[gopherbot]

Change https://golang.org/cl/361398 mentions this issue: cmd/compile: fix TypeDefn to deal with node with no Ntype set

[danscales]

OK, the compiler will no longer crash, and will report a type loop, though the position information is off. So no longer a release blocker. We will either fix things to make the position info more accurate, or Matthew may be able to update the nomono pass to catch this case earlier.

[mdempsky]

@griesemer @findleyr Looking into this now. Happy to extend the nomono check to handle this, but I'm wondering why this isn't caught by whatever logic is already responsible for rejecting type T struct{ _ T }?

[griesemer]

@mdempsky The logic (Checker.validType) that detects type T struct{ _ T } can (and used to) detect cases like this, but for this to work it will need to instantiate the types fully. This will lead to an infinite sequence of instantiations. I'm not sure there's an easy extension of the validType function to handle this case and avoid the infinite recursion.