proposal: spec: represent interfaces by their definition and not by package and name

[metakeule]

An interface is a collection of method names and signatures.

Say, we have:

package a

type Writer interface {
    Write([]byte) (int, error)
}


type WriterTo interface {
     WriteTo(Writer) (int64, error)
}

type WriterTo2 interface {
    WriteTo(w interface{Write([]byte) (int, error)} (int64, error)
}


From the perspective of the one writing an implementation, the a.Writer
and io.Writer are the same: As long as the Write method has the right
signature they are both fulfilled. 

However a.WriterTo and io.WriterTo are not, simply because they refer
to Writers of different packages although they have the same definition. Even WriterTo2
would only be compatible with io.WriterTo if
they later had an anonymous interface too.

see http://play.golang.org/p/kOH11TOlsE

Since interfaces are all about reusable code this behaviour must be 
considered an artifact of the way, interfaces are represented and treated during
compilation.

Since interfaces have no methods and no values, their only value is to help the
programmers via the type constraints. 

From the implementation perspective a.WriterTo, io.WriterTo and a.WriterTo2 must be
treated as being the same, a contraint to have a method WriteTo(w
interface{Write([]byte) (int, error)} (int64, error).

The names (and packages) of the interfaces should not be relevant for the compiler.
Instead the definition should be the representation and the names just shortcuts to
refer to them.

[ianlancetaylor]

Comment 1:

Labels changed: added go2, languagechange, repo-main, release-none.

Status changed to LongTerm.

[griesemer]

Comment 2:

See also issue #8691.

Status changed to Thinking.

[gopherbot]

Comment 3 by [email protected]:

While admittedly this issue comes up rarely in regular code, as a data point, I run into
this issue quite a bit.  I've built an RPC stub compiler that takes an API specified in
my own language, compiles it, and then generates RPC stubs in different languages.  Go
is the core generated language.
As a simplified example, let's say the user tells me to generate stubs for a RPC
function "Print" that streams int32 from client to server, and streams strings from the
server back to the client.  Here's the Go API that I generate for the user on the client
side.
type PrintClientStream interface {
    // SendStream returns the send side of the client stream.
    SendStream() interface {
        Send(item int32) error // Places an item on the output stream
        Close() error // Indicates no more items will be sent
    }
    // RecvStream returns the receiver side of the client stream.
    RecvStream() interface {
        Advance() bool // Stages an item to be retrieved via Value
        Value() string // Returns the staged item
        Err() error // Returns errors
    }
}
Obviously the choice of "int32" and "string" are up to the user, as is the RPC function
name.  The reason SendStream() and RecvStream() return anonymous interfaces is because
of this issue.  A client may need to use two different services that define different
APIs, but result in the same streaming types; client sends int32 and receives string. 
I'd like them to be able to write a single function that operates over the two different
PrintClientStream interfaces.  But if I named the return types from SendStream and
RecvStream, they'd lose this ability.
I also have a straw man proposal.  
I understand the concepts of type identity and assignability as defined in the language
spec:
http://golang.org/ref/spec#Properties_of_types_and_values
I also appreciate the consistency of the current type identity rules wrt named types: a
named and an unnamed type are always different.  Whenever you see a "type" declaration
in a source file, you know that a distinct type is being defined.
Let's say we changed the language spec to say that type identity for interfaces ignores
the name.  I.e. interface names are simply a convenient shorthand, but for type identity
purposes, it's as if every interface were anonymous.  The rules for comparing these
interfaces remains the same; it's an order-agnostic comparison of method names and their
function types, and unexported methods in different packages are always different.
I think this is a nice and consistent way to specify this feature.  The main downside I
see is that it's yet another special-case in the language spec.  But interfaces are
already special; we already have the special-case in our value assignability rules.
<wishful thinking>
If you haven't tuned out so far, here's a thought.  Why wait until Go2 to incorporate
this feature?  My rationale: it seems unlikely that anyone was actually relying on the
existing semantics.  After all the compiler has ensured that all static type checks obey
the existing rules, and we're simply relaxing those rules so that more cases are allowed.
But for full disclosure, I do recognize that if we change this in Go1, existing code can
notice a change in behavior.  After all we provide type assertions in the language, as
well as the reflect package.  E.g.
http://play.golang.org/p/OscKQCdDOL
That's where my wishful thinking comes in.  I can't think of a reasonable way someone
would be using the existing semantics; my assertion is that if they're relying on
something this subtle, their code deserves to be broken (and I'm only half-kidding). 
But I can see that if we strictly follow the Go 1 compatibility rules, we simply cannot
change this feature, regardless of how much the code that relies on this deserves to be
broken.
Thoughts?

[maxnordlund]

This reminds me of the strawman to add typed objects to JavaScript, which considers two struct types equivalent iff they have the same fields (name and type) in the same order.

[mattn]

I'm thinking main.Writer should be treated as different as io.Writer. Because WriterTo have different signature of argument. I prefer to use embed interface of io.Writer in this case.

http://play.golang.org/p/IklkWaIvWG

[metakeule]

@mattn I want to be able to write a function that can receive any of the WriterTo interfaces.

[evanphx]

A quick note on this issue for anyone that is still watching it: Go "does the right thing" if the 2 inner interfaces are declared anonymous. That means the compiler and runtime already support the ability to do the right thing, it's just being overly strict if the interface has a name. Seems that a simple change, to ignore any given name when type checking 2 interfaces, would easily solve this issue.

[evanphx]

Here shows that if the interfaces are declared anonymous Go "does the right thing": http://play.golang.org/p/jpbd2325sj

[griesemer]

Removing Go2 label in light of use cases pointed out by #16209. Perhaps this (or some other solution) needs consideration before Go2.

[extemporalgenome]

Interfaces are already special enough that I think this would be warranted (one could argue the same for function types, so that first-order functions aren't pinned to type name either).

This may change/break existing programs, though. It's possible that some type assertions which fail now would succeed after this change.

[puellanivis]

I'm not sure this won't break existing programs, though. It's possible that some type assertions which fail now would succeed after this change.

Not sure how. If the two interfaces have identical methods, then all type assertions between the two should be knowably and provably valid…

I discussed this on the other bug #16209, in all, the worst thing possible is that two unnamed struct types might become assignable where they were not before, i.e.

var a struct { ctx context.Context } = struct { ctx x/net/context.Context }{ ctx: myctx }

But this seems like REALLY weird code…

Otherwise, you have unnamed function types, which would accept either interface the same way when called, but then that's kind of exactly the desired behavior for context.Context moving from x/net/ into the mainline library.

Finally, unnamed interfaces assign between identical interface definitions already…

[puellanivis]

https://play.golang.org/p/NPkhXi0VuD

Type assertions already go through fine:
if _, ok := a.(A).(B).(A); ok {

and the type switch tests for “implements interface” not “is actually typed this interface”

As such, a in the example will type switch to A or B depending on which case is presented first. (A in the first type switch, B in the second.)

[mdempsky]

@puellanivis Under the proposal, this code would change behavior:

type U1 interface{}
type U2 interface{}

var u interface{} = (*U1)(nil)
_ = u.(*U2)

In Go 1, this panics because *U1 and *U2 are different types.

If the proposal is accepted, the code would succeed because *U1 and *U2 are now identical types.

[puellanivis]

In Go 1, this panics because *U1 and *U2 are different types.

A) don't use pointers to interfaces… it's pretty much redundant, and almost certainly not what you want to do.

B) your code panics even if you attempt: _ = u.(U1) instead.

https://play.golang.org/p/alO15c93b-

And it STILL fails even if you change the var decl to var u U1 instead of var u interface{}

And then change the nil to 0, and both assertions succeed no panics.

[mdempsky]

A) don't use pointers to interfaces… it's pretty much redundant, and almost certainly not what you want to do.

That may be, but they're still part of Go 1 and they do have valid uses.

B) your code panics even if you attempt: _ = u.(U1) instead.

That's because u is the nil interface value, which always causes the type assertion to fail. See Type assertions [emphasis added]:

For an expression x of interface type and a type T, the primary expression

x.(T)

asserts that x is not nil and that the value stored in x is of type T.

[puellanivis]

Quoting the Go1 guarantee:

It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification.

As a subset of all code accepted by Go1 any code that compiles and runs without panic will, unchanged continue to compile and run without panic if this proposal were implemented.

Next, according to specs:

In other words, even though the dynamic type of x is known only at run time, the type of x.(T) is known to be T in a correct program.

Note that Go1 only guarantees for programs that “run correctly.” And by spec, having a type assertion that will knowably at compile-time panic a type-assertion is, by definition, not a correct program.

Thus, the only case we're REALLY dealing with here is:
if u2, ok := u.(*U2); ok { /* this code will begin to start running */ } else { /* instead of this code */ }

First, let's set aside that this code is wrong wrong wrong, and the wrongiest wrong that ever wronged.

u2 will with this proposal be a valid pointer to an interface U2, which is actually—by design—identical to a pointer to an interface of U1. That is, to get at anything other than methods of U1 or U2, you would STILL have to type assert the pointed-to value into something usable.

Thus, the code will actually start working in a way that is guaranteed to work correctly. Because whether u points to an interface of type U1, or U2, then (_u).(U2), and (_u).(U1) would work if the pointer were not nil (otherwise nil-pointer deref) or the value of the pointed-to interface is not nil.

Namely, there is no code where treating type *U1 and *U2 as identical could possibly cause “broken” behavior. After all, they were already “meta-semantically” identical…

So, as noted, this actually makes MORE code correct, and is a strict super-set of all currently correct code.

var u interface{} = (*U1)(nil)

I just have to say, breaking this out into what it is actually saying; this is making a variable u, which is an untyped empty interface, which is set with the concrete type to an empty interface, and a value of concrete value of nil.

This code kinda made me throw up a little in my mouth…

[mdempsky]

Note that Go1 only guarantees for programs that “run correctly.” And by spec, having a type assertion that will knowably at compile-time panic a type-assertion is, by definition, not a correct program.

Your interpretation of "correct" here is stricter than intended: correct Go programs are allowed to panic.

Happy to discuss further if you disagree, but let's move it to golang-dev. It's tangential to this proposal.

[metakeule]

I want to add that if this feature was implemented, it would allow a whole new class of packages/libraries to require no imports.

Let's say you follow the paradigm of receiving an interface and returning an implementation when designing your functions and methods.

Now you are writing package z which should be compatible to the interface A in package x and B in package y. ATM you would need to reference x.A and y.B directly and therefor need to import packages x and y. Your package becomes dependent of them.

With the proposed change you could simply copy the definitions of the interfaces A and B to your library z and use them as parameter types or embed them to structs without having to import x or y. This way your package z has no dependencies and the users of your package z might combine it with x and y and at a later point swap x or y out and replace them with compatible versions without breaking any dependency of z.

All in all this could lead to a better and more decoupled library ecosystem based on compatible interfaces. This is the most important argument for the change IMHO.

[puellanivis]

In other words, even though the dynamic type of x is known only at run time, the type of x.(T) is known to be T in a correct program.

This isn't a matter of “correct programs aren't allowed to panic” it's that a type assertion that is KNOWN to always panic, is not a correct program. But as well…

The exact error values that represent distinct run-time error conditions are unspecified.

Thus, there is no possible correct code that can depend upon a type assert and only a type assert throwing a panic. Any deferred function recovering run-time panics must by necessity handle all possible run-time panics. (like instead of let's say nil-pointer dereference)

Otherwise, they are—by necessity—depending upon unspecified behavior.

[griesemer]

@neild There may possibly be minor technical differences (e.g., how an interface is printed, either by name or by its literal, depending on how this would be implemented), but my understanding of this proposal is that if we would write every interface type definition as a alias type declaration, we'd get the same effect. The problem is of course that you can't do that with pre-existing code that you don't control but may depend on.

Interfaces already behave essentially like the proposal suggests for operations such as assignments, type assertions, equality operations, etc. because there we have special rules in place.

But as @ianlancetaylor pointed out, this proposal is really affecting the definition of type identity for interfaces, which comes into play when we have other (non-interface) types composed of interfaces: A function type func(x Stringer) currently is different from func(x interface{String() string}). The proposal would make those identical.

[jimmyfrasche]

@griesemer wouldn't this also have to extend to interfaces such as

type I interface {
  RecursivelyReferences(I)
}

which cannot be expressed with type aliases?

[griesemer]

@jimmyfrasche I'm not sure I understand the question. Do you mean something like

type T = interface{
   f(T)
}

except that we wouldn't have to write the = ? (I guess in this case, we're talking about the non-interface type of f(T)).

[bcmills]

This compiles today (https://play.golang.org/p/cnLlSHpwRz):

type I interface {
	SetFrom(I)
}

As an alias, it fails to compile with invalid recursive type alias I (https://play.golang.org/p/CoesgI8bCs):

type I = Interface {
	SetFrom(I)
}

So that's one of the “minor technical differences”: interfaces can be recursive, but aliases (at the moment) cannot.

[bcmills]

Upon further reflection, the fact that aliases cannot be recursive, combined with the fact that mutually-assignable interface types are not identical, is probably going to be a significant headache for me.

I'm trying to write a code generator that can wrap C++ APIs as Go APIs, and I had planned to use aliases of interface types to represent template instantiations (which will need to be defined in every Go package that wraps a C++ API that instantiates the template).

I have to use type aliases rather than defined interface types so that the instantiated interfaces are mutually-assignable with the same instantiations in other wrapper packages, even if they include methods that refer to other template instantiations. (For example, consider std::vector<std::unique_ptr<T>> for some non-copyable, non-movable type T.)

Unfortunately, the prohibition on recursive aliases means that I can't use that approach to wrap any C++ member function that refers to its own type. Notably, that includes all assignment operators.

(On the other hand, my problem could be addressed by allowing recursive aliases, which would be strictly compatible with Go 1 but perhaps more difficult to implement in practice.)

[griesemer]

@bcmills Let's not digress from the proposal at hand. Yes, the above example is currently not accepted, but it's not clear if that's "just" and implementation issue. At least I don't see why it couldn't be accepted.

[bcmills]

Right, I don't mean to digress. My point is that there are several possible solutions to the problem I have, and this proposal is one of them.

(Accepting mutually-recursive aliases would be another. Allowing aliases of struct types to define methods would be a third, although that seems less likely to happen.)

[metakeule]

When it comes to recursive interface definitions, I would propose, that

type A interface {
   SomeThing(A)
}

and

type B interface {
   SomeThing(B)
}

would result in the same (nameless) internal representation.

[cznic]

Let's go exponential?

[griesemer]

@metakeule The internal representation doesn't have to be the same, or nameless; the implementation may choose whatever representation is suitable. The point of this issue is that the rules for identity for interfaces would have to be changed in the spec.

[metakeule]

@griesemer Yes, you are right.

I chose "internal representation" as a helper to illustrate the point, that when checking for identity there would be some representation that is not part of the language but has some meaning or checks for "self-ness" if that is a word; an argument that is the surrounding interface itself would not be considered the interface name, but this special placeholder "self".

[evanphx]

I'd still love for this to be fixed for Go 2!

[DeedleFake]

This proposal seems to be missing the Proposal label. Is that intentional?

[bvk]

I came across this problem recently when I am trying to implement marshaling/unmarshaling objects to/from key-value stores. I think solution to the original problem in comment 1 would be good to have in Go 2. Below is my interface design for marshaling objects into key-value stores. Any other solutions to my problem are also much appreciated. Thanks.

The Problem - Cyclic Dependencies between interfaces wouldn't compile.

type Encoder interface {
    Encode(name string, object KVMarshaler) error
}
type KVMarshaler interface {
    MarshalKV(enc Encoder) (key string, value []byte, status error)
}

My Solution 1 - Break the cycle using expanded anonymous interface.

type Encoder interface {
  Encode(name string, object interface{MarshalKV(Encoder)(string,[]byte,error)}) error
}
type KVMarshaler interface {
  MarshalKV(enc Encoder) (key string, value []byte, status error)
}

Problem 2 with Solution 1 - Following TreeEncoder doesn't compile because Encode signature doesn't satisfy the Encoder interface with the anonymous interface.

type TreeEncoder struct {
  ...
}

func (te *TreeEncoder) Encode(name string, object Marshaler) error {
   ...
}

Solution for Problem 2 - This works. Since number of encoder types would be far fewer than number of object types that must implement marshaling methods, I am going with this for now.

type TreeEncoder struct {
  ...
}
func (te *TreeEncoder) Encode(name string, object interface{MarshalKV(string,Encoder)(string,[]byte,error)}) error {
...
}