errors: Is and As make it impossible to maintain error hygiene

[rogpeppe]

$ go version
go version devel +e883d000f4 Wed May 29 13:27:00 2019 +0000 linux/amd64

It's typical that Go code has many lines of code that return errors as a consequence of other errors. In large programs, it can be hard to refactor this code with confidence because by changing the code, we might be changing the set of possible errors that can be returned, and some unknown caller might be broken by the refactor because they're relying on a specific error that's no longer returned.

The nature of such breakage is that it often goes unnoticed for long periods because errors are usually infrequent.

For this reason, I believe it's good to maintain what I call error hygiene: to restrict the set of returned errors to those that are explicitly part of a function's contract.

One good example of this is fmt.Errorf("message: %v", err) which hides the type and value of err from any subsequent Is or As call.

Unfortunately, I don't believe it's possible to preserve error hygiene in general with the scheme used for Is and As.

Suppose that we want to define the following function that we'll use to restrict error values to a particular type. This is a bare-minimum requirement for error hygiene.

// KeepType keeps only the given targetType as a possible target
// type for the As function.
//
// For any error e and types T and U:
//
//	As(KeepType(e, new(T)), new(U))
//
// will return true if and only if As(e, new(T)) is true and U is
// identical to T. In other words, it hides all types other than T from
// future inspections with As.
func KeepType(err error, targetType interface{}) error

How can we implement this function? For errors that don't fit the designated target type, the implementation is easy: just hide the whole error chain. However, if there's an error that does fit the target type, there's a problem: we can't preserve the target type without also preserving the error chain that's below it, because in general the chain is held in a field within the error value that we might not have access to. We can't make a new error value because then it wouldn't be the required target type.

I think that As and Is are thus potentially problematic and we should consider whether it's a good idea to keep them as is. (so to speak :) )

[rogpeppe]

@mpvl @jba

[bcmills]

The generic KeepType function seems like a bit of a red herring: if you're maintaining the sort of error hygiene you describe, you know the specific set of types T in question, and you can use functions or methods specific to those types to construct opaque instances.

You could even imagine adding a new well-known method (perhaps Flatten() error?) that allows the type to construct such an instance itself.

[rogpeppe]

I don't believe it's entirely a red-herring.

The type you're wanting to preserve might not be defined in the package that wants to preserve it. It's not uncommon for modules to use a common errors package shared by many packages, for example. Another example: I might want to preserve a database-specific error type. Pq isn't a great example because all the fields are exported, but that might not be the case in future or with other drivers.

Also, AFAICS the flatten operation can need O(num-errors-in-chain) allocations if you want to preserve stack information, which isn't ideal. I guess there might be ways around that though.

[rogpeppe]

FWIW the same issue applies to Is, although it's likely to be less of a common problem.
Consider:

var MyEOF = fmt.Errorf("my EOF: %w", io.EOF)

This satisfies both errors.Is(MyEOF) and errors.Is(io.EOF) but there's no way to preserve the former without also preserving the latter.

[bcmills]

This satisfies both errors.Is(MyEOF) and errors.Is(io.EOF) but there's no way to preserve the former without also preserving the latter.

That's true, but I think that only reinforces my point: it is up to the author of the error (or error type) — not the user of the error — to decide how much information that type or value is capable of masking.

[bcmills]

AFAICS the flatten operation can need O(num-errors-in-chain) allocations if you want to preserve stack information, which isn't ideal. I guess there might be ways around that though.

If you want to preserve the first k errors in a chain of N links, you only need to allocate k new links in general: the remainder can opaquely wrap the tail of the chain (using %v instead of %w).

[earthboundkid]

I think the problem is the errors.As/Is function will check for As/Is methods on the first error, but if it returns false, they will keep unwrapping the chain and checking until they find something that returns true. They should instead short-circuit the first time an error in the chain implements an As/Is method.

[JavierZunzunegui]

var MyEOF = fmt.Errorf("my EOF: %w", io.EOF)

This satisfies both errors.Is(MyEOF) and errors.Is(io.EOF) but there's no way to preserve the former without also preserving the latter.

If I understand your issue correctly, the more general form of what you ask is

given err = ErrorA /*wraps*/ ErrorB /*wraps*/ ErrorC
can there be a way to convert it to
ErrorA /*wraps*/ ErrorC

I don't think there is a way of doing that in the errors package as it is currently proposed.

It can be done trivially easy if doing error wrapping using a linked list of errors, but that won't be supported under the new changes.

An example: JavierZunzunegui@4489292#diff-653dcb659c69a84ac6ce23e6e48509a3R6

[jba]

I believe this is an implementation of KeepType:

func KeepType(err error, targetType interface{}) error {
    return &keepError{err, reflect.PtrTo(reflect.TypeOf(targetType))}
}

type keepError struct {
    err        error
    targetType reflect.Type
}

func (k *keepError) Error() string {
    return k.err.Error()
}

func (k *keepError) As(target interface{}) bool {
    if reflect.TypeOf(target) != k.targetType {
        return false
    }
    return errors.As(k.err, target)
}

Proof: KeepType should "return true if and only if As(e, new(T)) is true and U is identical to T. "

If direction: if U is identical to T, we skip past the initial if statement, execute As(e, new(T)) (since U is identical to T) and return its value, so if U is identical to T and As(e, new(T)) returns true, so does KeepType.

Only if direction: we need to show that if U is not identical to T or if As(e, new(T)) is false, then KeepType returns false. The initial if handles the first disjunct, and the subsequent call to errors.As handles the second.

[jba]

They should instead short-circuit the first time an error in the chain implements an As/Is method.

We considered that seriously for a while. Ultimately we rejected it. It does give the As/Is methods more power, but the cost is that every such method must follow the chain to get the right answer. We opted for the less powerful but easier to implement semantics.

[jba]

In my comment above I left out an important part of the proof: the keepError type has no Unwrap method, so the return value from its As method is the return value from errors.As.

I also mixed up the wording: I wrote about whatKeepType returns but I meant what errors.As(KeepType(...), ...) returns.

[earthboundkid]

Another way to think about this problem is that there are multiple uses for error chains:

• you may want to know about all errors in a chain, for logging
• you may only want to know about some errors in a chain, for handling
• you may want some ability to format error output or keep stack frames (dropped from 1.13)

The current Is/As functions tie the first two kinds of errors chains together, when conceptually they are different.

[earthboundkid]

every such method must follow the chain to get the right answer.

Just call the function on your wrapped error if you want the default semantics. That’s not hard to implement. Or just don’t have a method at all and get default semantics for free. I think you only want the method at all because you want to do something unusual.