proposal: Go 2: allow pointer types to be receiver base types in method declarations

[leighmcculloch]

What version of Go are you using (go version)?

$ go version
go version go1.17.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="/Users/leighmcculloch/.local/bin"
GOCACHE="/Users/leighmcculloch/Library/Caches/go-build"
GOENV="/Users/leighmcculloch/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/leighmcculloch/.local/gopath/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/leighmcculloch/.local/gopath"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/leighmcculloch/.local/bin/go/latest"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/leighmcculloch/.local/bin/go/latest/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="0"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/q5/xb4dl4bs3cs32khlg0ryy0vw0000gp/T/go-build1564949535=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Define a type that is a pointer to another type, and define methods for that type, so that type can be marshaled differently to include information about the optional quality of the pointer.

type S struct {
	// ...
}

func (s S) MarshalBinary() ([]byte, error) {
  // ...
}

func (s *S) UnmarshalBinary(inp []byte) error {
  // ...
}

type P *S

func (p P) MarshalBinary() ([]byte, error) {
  // ...
}

func (p *P) UnmarshalBinary(inp []byte) error {
  // ...
}

Example: https://github.com/stellar/go/blob/e24fae34b9166cc61bb9e6c4645eaabff844768b/xdr/xdr_generated.go#L1620

What did you expect to see?

I expected go build to suceed at compiling.

What did you see instead?

invalid receiver type P (P is a pointer type)

Discussion

This behavior appears to be consistent with the specification that states:

A receiver base type cannot be a pointer...

There is also prior discussion on this topic on the golang-nuts mailing list that points out why this behavior is this way.

Why is it not allowed to define pointer types as named types with methods?

Because in a case like this:

type I int
type P *I
func (i I) Get() int { return int(i) }
func (p P) Get() int { return int(*p) }
var v I
var x = (&v).Get()

it would be unclear whether the Get method in the last line would be
I.Get or P.Get. We could define a rule for it, but that would become
another thing that people would have to know.

Ref: https://groups.google.com/g/golang-nuts/c/qf76N-uDcHA

However, after viewing code running in Go 1.17.1 today the reason that was given for why these types cannot be receivers is less clear. The prior conversation suggests we don't have a rule for what type &v is, however &v is possible today and so we can write code that tells us what that rule is.

For example, the output of the following code tells us what the rule is:

type I int

type P *I

func main() {
	i := I(0)
	fmt.Printf("i = %T\n", i)
	fmt.Printf("&i = %T\n", &i)
	p := P(&i)
	fmt.Printf("p = %T\n", p)
	fmt.Printf("&p = %T\n", &p)
	fmt.Printf("*p = %T\n", *p)
}

i = main.I
&i = *main.I
p = main.P
&p = *main.P
*p = main.I

Ref: https://play.golang.org/p/YdYlHDub0eg

The output of the above code is pretty clear that &i is always *I and never P, therefore in the example on the mailing list &v is always *I, therefore (&v).Get() should also always call I.Get since according to the specification:

The method set of the corresponding pointer type *T is the set of all methods declared with receiver *T or T (that is, it also contains the method set of T).

Being able to define types that are pointer types would make defining parsers for nested optional values simpler, because it would remove the need to wrap those nested types in a struct.

The proposal is support for pointer types to be a receiver base type in method declarations.

The specification would be changed from:

A receiver base type cannot be a pointer or interface type and it must be defined in the same package as the method.

To:

A receiver base type cannot an interface type and it must be defined in the same package as the method.

Proposal template

• Would you consider yourself a novice, intermediate, or experienced Go programmer?
  Experienced

• What other languages do you have experience with?
  Java, Ruby, C#, C, JavaScript

• Would this change make Go easier or harder to learn, and why?
  Neither.

• Has this idea, or one like it, been proposed before? If so, how does this proposal differ?
  See discussion on the mailing list above.

• Who does this proposal help, and why?
  People building mashalers with nested optional types.

• What is the proposed change?
  See above.

  • Please describe as precisely as possible the change to the language.
    See above.

  • What would change in the language spec?
    The definition

  • Please also describe the change informally, as in a class teaching Go.
    See above.

• Is this change backward compatible?
  Yes

  • Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.

• Show example code before and after the change.
  See above.

• What is the cost of this proposal? (Every language change has a cost).

  • How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?
    Likely none.
  • What is the compile time cost?
    Insignificant.
  • What is the run time cost?
    Insignificant.

• Can you describe a possible implementation?
  See above.

  • Do you have a prototype? (This is not required.)
    No. But I can take a stab at contributing it.

• How would the language spec change?
  See above.

• Orthogonality: how does this change interact or overlap with existing features?
  Not sure.

• Is the goal of this change a performance improvement?
  No.

  • If so, what quantifiable improvement should we expect?
  • How would we measure it?

• Does this affect error handling?
  No.

  • If so, how does this differ from previous error handling proposals?

• Is this about generics?
  No.

  • If so, how does this differ from the the current design draft and the previous generics proposals?

[leighmcculloch]

I found some more discussion that is relevant to this proposal.

The Go Wiki the Rationals page has a section on pointer receivers that provides a similar justification for why pointer receivers are not supported:

Suppose you have:

type T blah
type P *T

func (t *T) String() string { ... }
func (p P) String() string { ... }

var p P

Then the meaning of the expression (*p).String() is ambiguous, because it can refer to both (*T).String and P.String.

Ref: https://github.com/golang/go/wiki/Rationales/c2d2e158f292621916801012908bc02819421be3#why-method-receivers-base-type-cannot-be-pointer-or-interface

I think this justification has the same problem as the similar but different justification that I included in this issues first comment/description. (*p) is always of type T, not *T or P.

This is demonstrated with the below code. Because (*p) is always of type T, the Go spec is clear that the String method called is T.String().

type T int
type P *T

var t T = 0
var p P = &t

fmt.Printf("%T\n", p)
fmt.Printf("%T\n", *p)

main.P
main.T

Ref: https://play.golang.org/p/G6TIyJQVB2g

Other variations are also deterministic, such as &p which is always of type *P, which according to the Go spec is unambiguous that it calls P.String() since the method set of pointer type *P is the set of all methods declared with receiver *P or P:

fmt.Printf("%T\n", &p)

*main.P

Ref: https://play.golang.org/p/I8eaEWAOK7C

[ianlancetaylor]

I agree that it is possible to define rules to make this work without ambiguity. But I think the result will be subtle and difficult to understand. It will require a clear grasp of the exact type of different expressions, and a clear understanding of the exact rules.

Given that, and the fact that we have lived with the current rules for some time, what is the compelling reason for making this change? It seems possible to write the initial example using a struct with a pointer field and adding methods to the struct. This may require a bit more code in some places, but the advantage of that code is that it makes clear what is happening, and does not require the reader to understand the types and the rules. Having different marshaling for a type and a pointer to a type is confusing; introducing a different type to clarify matters seems like a good idea even if it requires writing a little more code.

[leighmcculloch]

But I think the result will be subtle and difficult to understand. It will require a clear grasp of the exact type of different expressions, and a clear understanding of the exact rules.

The counterintuitive quality of pointer types for me is that you can dereference a pointer type P and get a non-pointer type I. Even though it makes sense, it is counterintuitive because dereferences usually result in the same type but with less asterisks, and since pointers to types inherit method sets, usually method sets of a dereferenced value are only a subset for a single dereference.

I don't think this proposal defines new rules that are counterintuitive, because it is valid to create a pointer type P today, and valid to dereference it and get I. I mean to say that the counterintuitive rule exists today.

I do think this proposal will create new instances where this counterintuitive quality shows up, which could be a problem. There are two instances I can think of:

1. There is no function today where the receiver when dereferenced can change from P to I. The receiver can only change from *I to I. This could mean dereferencing the receiver, then calling functions on it would be confusing. A reader could see the function calls assuming they are calling P.Func, when they are calling I.Func. The fact they call I.Func is correct, just surprising.

2. This counterintuitive quality can already show up for any other parameter of a method. Dereferencing a parameter of type P will become I. So it's possible to misunderstand what is happening inside a function already. However, the chance of calling a wrong function is nil since today P has no functions.

Both of these situations seem like they affect readability too greatly.

what is the compelling reason for making this change? It seems possible to write the initial example using a struct with a pointer field and adding methods to the struct.

The compelling reason was to be able to write parsers that map directly onto types defined in other forms, such as XDR, where types can be pointer types. Using a struct as you say is definitely valid alternative.

The specific library this proposal would benefit supports marshaling/unmarshaling both by using reflection and by using generated code. The reflection marshaler can easily support the pointer type because it needs no functions on the type, it only needs to be able to inspect the type, and its fields in the case of a struct or its underlying type. The code generator on the other hand needs to add functions to the pointer type. My dilemma is that while introducing a struct with fields in place of the pointer type will work in the code generated case, it will break the reflection based marshaler because the reflection based marshaler can no longer identify that the type is a pointer type, since the type is now a struct that wraps a pointer. I realize this isn't an insurmountable problem, I'm just detailing it here for the full picture.

@ianlancetaylor Thanks. I agree with your comments. 👍🏻

[ianlancetaylor]

Based on the discussion above, this is a likely decline. Leaving open for four weeks for final comments.

[ianlancetaylor]

No further comments.