types2, go/types: missing errors for invalid parameterized type declarations

[griesemer]

This is a new version of frozen #39938. The following code

package p

type T0[P any] struct {
        f P
}

type T1 struct { // <<< should get a cycle error here
        _ T0[T1]
}

is invalid because T1 would expand indefinitely. The previous fix (https://golang.org/cl/240519) could lead to infinite sequences of instantiations (see #48951).

Correct solution of this problem requires some form of type flow analysis and is related to (and may be addressed with) #48098.

[gopherbot]

Change https://golang.org/cl/355732 mentions this issue: cmd/compile/internal/types2: avoid infinite expansion for invalid recursive generic types

[griesemer]

cc: @findleyr

[griesemer]

The above program currently crashes the compiler with a simple nil pointer panic. @danscales is there an easy way to address this?

More generally, could cases like these be detected in the type checker through some variation of the nomono checker? cc: @mdempsky

[gopherbot]

Change https://golang.org/cl/361398 mentions this issue: cmd/compile: fix TypeDefn to deal with node with no Ntype set

[danscales]

OK, the compiler will no longer crash, and will report a type loop, though the position information is off. So no longer a release blocker. We will either fix things to make the position info more accurate, or Matthew may be able to update the nomono pass to catch this case earlier.

[mdempsky]

@griesemer @findleyr Looking into this now. Happy to extend the nomono check to handle this, but I'm wondering why this isn't caught by whatever logic is already responsible for rejecting type T struct{ _ T }?

[griesemer]

@mdempsky The logic (Checker.validType) that detects type T struct{ _ T } can (and used to) detect cases like this, but for this to work it will need to instantiate the types fully. This will lead to an infinite sequence of instantiations. I'm not sure there's an easy extension of the validType function to handle this case and avoid the infinite recursion.

[griesemer]

@mdempsky see also the description on CL 355732.

[danscales]

@mdempsky Assigning this issue to you, so you can comment as to whether you think it is feasible to change nomono to catch this recursive type in the beta time-frame (i.e. in the next couple weeks). You can assign it back to me, if you don't think it makes sense or is feasible to do for Go 1.18.

[gopherbot]

Change https://golang.org/cl/378176 mentions this issue: cmd/compile/internal/types2: use a map instead of a field for marking in validType

[gopherbot]

Change https://golang.org/cl/378675 mentions this issue: cmd/compile/internal/types2: remove special case for external types in validType

[gopherbot]

Change https://golang.org/cl/379414 mentions this issue: cmd/compile/internal/types2: consider type parameters for cycle detection

[gopherbot]

Change https://golang.org/cl/379434 mentions this issue: cmd/compile/internal/types2: validType argument must be *Named type

[gopherbot]

Change https://golang.org/cl/378674 mentions this issue: go/types, types2: move validType code into its own file

[gopherbot]

Change https://golang.org/cl/379694 mentions this issue: go/types, types2: in SetUnderlying, set Named.fromRHS if not set yet