Alternative proposal: property read/write trapping decorators

[mweststrate]

Trapping decorators

Motivation

This proposal is a proposal to introduce decorators that have fewer concepts, and are hopefully easier to implement by JVM authors.

The fundamental difference in comparison is that this proposal does not attempt to process and modify property descriptors on a target. Rather, it recognises that modifying the property descriptors has always been a means to an end for 95% of the decorators: Namely a way to trap reads and writes. To transform values.

The motivation behind this proposal is that we want to trap reads and writes to properties. Very similar to the functionality that Proxies provide so successfully. The main difference is that decorators trap all interactions with the decorated instance, and that an entire 'type' is trapped, rather than specific instances. Things that cannot be done by Proxies.

The rest of the proposal is a proposal to achieve the "property trapping". Probably any other proposal would work for me personally as well. If engine or library authors want to adjust this proposal for whatever benefit: I think the wiggle room is there, as long as it can achieve the goal of trapping property reads and writes per instance, but being able to declare such traps for an entire type.

A big benefit of traps is that they often avoid the need of introducing additional properties. E.g. if there is a getter / setter to normalize input or make some assertions, currently it is needed to introduce an additional property to store the getted / setted value. With traps. this can collapsed into one property, avoiding the need of patterns like _name = "x"; get name() { return this._name }} etc.

The proposal

Property traps

A property trap is an object that provides two functions, named get and set. The goal of these traps is to be able to trap any read and write to a property. Traps can be used to achieve two effects

1. Transform values on read / writes
2. Introduce side effects on reads / writes

The signature of a PropertyTrap object is:

{
    get(target, instance, property, storedValue) => value
    set(target, instance, property, newValue) => valueToStore
}

Field Decorators

A decorator syntactically is a prefix of a class or object member.
Syntactically: @<DECORATOR1> <...DECORATORN> <MEMBERDECLARATION>

Everyone of those expressions is evaluated and should resolve a function that returns a PropertyTrap. The functions get's passed in the target that will receive the property definition, and the property name. In other words, the signature of a field decorator is:

(target: object, propName: string | Symbol) => PropertyTrap | undefined

If a field decorator does not return a trap, the decorator does not install additonal traps. However, the decorator could still be valuable as it can have side effects, such as storing meta data. (example: @serializable, @deprecated etc)

A first example

function logged() {
  return  {
     get(target, instance, property, value) {
        console.log(`GET`, target, instance, property, value)
        return value
    },
    set(target, instance, property, value) {
        console.log(`SET`, target, instance, property, value)
        return value
    }
  }
}

class C {
  @logged x = 3
}
// (1) SET C.propotype undefined 'x' 3

const c = new C()
//

c.x
// (2) GET C.prototype c 'x' 3
c.x = 4
// (3) SET c c 'x' 4
c.c
// (4) GET c c 'x' 4

Semantic notes:

When construction the initial property descriptor, the traps are already applied, so that means that the @logged traps are applied during the construction of member x on class C above, and it is the transformed value that ends up in the initial property descriptor. Which is the reason we can observe side effect (1) above.

In the traps target represents the (to be) owner of the property descriptor that is being created. instance represents the this context that is used during a property access. When constructing C.x above, there is a target (C.prototype) but not a this context, hence no instance argument.

On the first read, the read is handled (or intercepted upon) the C.prototype, and the context is c, which we can observe in the output of (2).

When writing to c.x, this will result in a new property descriptor on the instance c. This is again represented in the arguments passed to the traps. Note that again the traps are applied during the computation of the new descriptor for c (the traps are inherited, similar to other properties of the original property descriptor, such as enumerable and writeable).

Note that the original decorator expression @logger is not re-evaluated again! Rather, the traps in which this originally resulted are reused and inherited by the new property descriptor (see below).

If we read from c.x again, this read is now handled by the prop descriptor on the instance, which is reflected in the arguments passed to the traps: the property owner equals the this now.

Storing traps

Traps are stored as the traps property on a PropertyDescriptor. When copying a property descriptor from one object to another, those traps are preserved. So in the above example the following would hold:

Object.getOwnPropertyDescriptor(C.prototype, "x")
// Results in
{
    writable: true,
    configurable: true,
    enumerable: true,
    value: 3,
    traps: [logger]
}

(Note, could also be stored somewhere else, as long as semantics are similar to here, and traps can be requested through reflection)

Reflection

As shown above traps are detectable trough reflection (this is different from Proxies).

Also, traps can be by-passed at any time, for example:

Object.getOwnPropertyDescriptor(c, "x").value // prints 3, without the side effects of logging

This is intentional: it makes sure that developer tools such as the debugger don't accidentally trigger side effects, makes it easy to inspect the underlying data structures, and provides an escape hatch from the traps when needed (either by libraries or users). it is very well conceivable that traps themselves use this trick to bypass themselves (e.g. a set trap might use this to get the original value, but not trigger the side effect of it's corresponding get trap)

Getters and setters

Traps are just pass-through things, so they don't necessarily operate on just value based property descriptors, but work for get / set based descriptors as well. Except that the last category will not hit the traps when the property is originally declared (since there is no value to be initialized)

Trap initializer

Decorators can be parameterized by creating functions that return decorators. For example:

function logged(enabled) {
    return () => {
        get(target, instance, prop, value) {
            return value
        },
        set(target, instance, prop, value) {
            if (enabled)
                console.log(prop, value)
            return value
        }
    }
}

const obj = {
    @logged(true) x: 4    
}

Note again that in the property descriptor for obj.x the resolved value of the decorator expression is stored, in this case a logger trap object where enabled=true is trapped in its closure.

Chaining

Decorator can be chained, which means that all the traps are applied from inside out (or, right to left):

const obj = {
    @private @logged x: 5
}

In the traps property of the descriptor this results in an array with the order of the traps as they will be applied:

Object.getOwnPropertyDescriptor(obj, "x").traps
// [trapFromLogged, trapFromPrivate]

Classes and function decorators

Class and function decorators can be large kept as they are currently implemented in babel and typescript

For example @a class X is desugared to const X = a(class X) just as currently implemented already by babel-plugin-legacy-decorators and typescript's experimentalDecorators flag.

Two open questions raised by this, on which I don't have a strong opinion:

1. is a class decorator a side effect, or can it potentially return something different? I prefer the second; it is a little more flexible, and it makes clear that you ought to to be doing export @something class (see next question)
2. allow decorate-before-export (What would it mean to "decorate an export"? #135)?
3. what is the impact on hoisting? Will decorating a class / function implicitly kill the hoisting? Should class / function decorators be only side effect full to prevent that?

(if somebody could point me to a more accurate write down of the currently implemented semantics, that would be great)

Engine impact (and summary of the proposal)

This proposal tries to keep the impact on JS engines very limited:

1. Something something parsing
2. Whenever a property descriptor is being created for an object literal or class member, the following happens:
   1. the decorator expressions are evaluated. This should result in a function
   2. the function is called with the object that will receive the property, and the property name. The return of that function should be a trap or nothing
   3. (if applicable) the set handlers of the traps this results in are applied to the value the property descriptor would be initialized with
   4. (if applicable) the value this results in is stored in the value slot of the descriptor
   5. the set of traps is stored in the traps slot of the descriptor (only if there were any)
3. Whenever we write to an object: if there are traps stored in the descriptor that receives the write, the set handlers of those traps are applied first, and the resulting value is stored in .value of the descriptor / passed to the set handler of the (potentially new) descriptor
4. When reading from an object: if there are traps stored in the descriptor that receives the read, the value that is stored in .value of the descriptor (or: the value that is returned from the get handler of the descriptor) is passed trough all get traps of the traps stored in the descriptor
5. Profit

Reference examples

Not all examples of the current readme are here:

• @defineElement decorating entire classes is not in this proposal (see below).
• @metaData didn't work it out, but should be doable
• @frozen, @callable applies to a class, so skipped
• @set cannot be done in this proposal. So field initializers in subclasses would need to receive the same decorators as the superclass if needed.

@defineElement

function defineElement(name, options) {
  return klass => {
    customElements.define(name, klass, options)
    return klass
  }
}

@defineElement('my-class')
class MyClass extends HTMLElement { }

@metadata

const allClasses = new Map() // string -> constructor
const allMembers = new Map() // klass -> propName[]

function metaData(target, propName?) {
   if (arguments.length === 1) { // used on class
      allClasses.set(target.constructor.name, target)
      return target
   } else {
      allMembers.set(target, [...(allMember.get(target) ?? []), propName])
   }
}

@metaData class X {
   @metaData member1
   @metaData function() {
   }
}

@logger

const logged () => ({
    set(t, i, p, f) {
        const name = f.name;
        function wrapped(...args) {
            console.log(`starting ${name} with arguments ${args.join(", ")}`);
            f.call(this, ...args);
            console.log(`ending ${name}`);
        }
        return wrapped
    }
})

class C {
    @logged method(arg) {
        this.#x = arg;
    }
}

@Frozen

function frozen(klass) {
    Object.freeze(klass);
    for (const key of Reflect.ownKeys(klass)) {
      Object.freeze(klass[key]);
    }
    for (const key of Reflect.ownKeys(klass.prototype)) {
      Object.freeze(klass.prototype[key]);
    }
    return klass
}

@frozen class ICannotBeChangedAnymore {

}

@bound

function bound() { 
  return {
    get(target, instance, property, fn) {
        // happens when calling Foo.method for example in example below
        if (!instance)
            return fn
        // remember that traps get inherited? 
        // if we bound before, this trap doesn't have to do anything anymore 
        // (we could skip storing the bound methods below, but better cache those bound methods)
        if (target === instance)
            return fn
        // target is not the instance, we still need to bind
        const bound = fn.bind(instance)
        instance[property] = bound
        return bound 
    }
  }
}

class Foo {
  x = 1;

  @bound method() { console.log(this.x); }

  queueMethod() { setTimeout(this.method, 1000); }
}

new Foo().queueMethod();  // will log 1, rather than undefined

@Tracked

function tracked() {
  return {
    set(t, i, p, v) {
        // Note that we can't render synchronously, 
        // as the new value would otherwise not be visible yet!
        setImmediate(() => this.render())
        
        // In practice, what would happen in MobX etc is that they 
        // would mark this object as 'dirty', and run render
        // at the end of the current event handler, rather than awaiting a next tick

        // alternatively, it would be possible to write this value
        // to _another_ property / backing Map, like done in the Readme, so that the new value is externally visible before this chain of traps ends, as done below
        return v
    }
  }
}

class Element {
  @tracked counter = 0;

  increment() { this.counter++; }

  render() { console.log(counter); }
}

const e = new Element();
e.increment();
e.increment();  
// logs 2
// logs 2

@syncTracked

function syncTracked() {
  return {
    set(t, instance, prop, value) {
        instance["_" + prop] = value // or use a Map
        this.render()
        return undefined // booyah what is stored in *this* prop
    }
    get(t, instance, prop, value) {
        return instance["_" + prop]
    }
  }
}

class Element {
  @syncTracked counter = 0;

  increment() { this.counter++; }

  render() { console.log(counter); }
}

const e = new Element();
e.increment();
e.increment();  
// logs 2
// logs 2

Non goal: modify signature

In previous decorator proposal, it is possible to significantly modify the property descriptor, or even the entire shape of the thing under construction. That is not the case with this proposal; decorators can not influence where the descriptor ends up, it's enumerability, configurability, it's type etc.

The only thing that the decorator can influence is

1. The initial value of the property descriptor (except for getters / setters)
2. As side effect it could introduce other members on the target, however, this is considered bad practice and is typically best postponed until we trap a read / write with a known instance.

Non goal: run code upon instance initialization

This proposal doesn't offer a way to run code guaranteed when construction a new instance, although code might run when a trap is hit during a read or write in the constructor

TypeScript

Traps don't modify the type signature at this moment. Technically, they can be used to divert the type of x.a from Object.getOwnProperty(x, "a").value, but that doesn't seem to be an interesting case to analyse statically.

More interestingly, traps can be used to normalize arguments, which means that the type being read could be narrow than the type to which is written. For example:

const asInt = () => ({
    set(t, i, p, value: any): number {
        if (typeof value === "number")
            return value
        return parseInt(value)
    }
})

const person = {
    @asInt age = "20"
}
typeof person.age /// number

In the above example, the "write type" of age property would be any, while the read type would be number. However, TS can at the moment not distinguish between the readable and writeable property, and the same limitation already applies to normal getters and setters.

Edge cases

It is possible to omit set or get from a trap. The absence of such trap represents a verbatim write-trough (just like Proxies). See also the asInt example above

Instead of storing the decorators on the property descriptor, it probably suffices to be at least able to detect them trough reflection, e.g.: Reflect.getOwnDecorators(target, property): PropertyTrap[]

Optimization notes

Like proxy handers, property traps are shared as much as possible, so they are not bound to a specific instance, type or property, but rather receive those as arguments.

An assumption in this proposal is that when writing to a property that is defined in the prototype, the original descriptor is copied onto the instance, and we could just as well copy the traps with it. However, if this assumption is wrong, this proposal doesn't strictly require this copy-traps mechanism; as the trap on the prototype could take care itself of the copying the traps in cases where this is need (for example, in @tracked it is, but in @bound it isn't).

The reason that both the decorator and the traps themselve receive the target is that this makes it much easier to reuse traps among different properties and classes. Probably this will be much better optimizable, just like proxy traps are a lot more efficient as soon as the handlers are reused. So for example, it would be adviseable to hoist traps from the decorator definitions whenever possible:

const loggerTraps = {
     get(target, instance, property, value) {
        console.log(`GET`, target, instance, property, value)
        return value
    },
    set(target, instance, property, value) {
        console.log(`SET`, target, instance, property, value)
        return value
    }
  }

function logger() { 
  return loggerTraps 
}

class X {
   @logger methodA() {

    }
    // both methods now use the same logging trap
    @logger methodB() {

    }
}

Note that this optimization notice applies to most examples above

---

update 27-nov-2019:

• clarified / linked to the old class decoration proposal
• changed the signature to of decorators to be a function, to make it easier to do meta-data-only decorators
• added examples for @frozen, @defineElement, @metaData
• added note about trap hoisting optimization

[littledan]

Thanks for this detailed writeup. It'll take me some time to review this properly and catch up with the rest of the activity in this repo, but @pzuraq @fkleuver @wycats @syg may have thoughts.

[justinfagnani]

I think this is where we need a central list of use cases to judge variants against. I don't see this proposal solving several of them.

[pabloalmunia]

Thanks @mweststrate for this detailed proposal.

You start with an important assumption:

The fundamental difference in comparison is that this proposal does not attempt to process and modify property descriptors on a target. Rather, it recognises that modifying the property descriptors has always been a means to an end for 95% of the decorators: Namely a way to trap reads and writes. To transform values.

In my humble opinion class decorators are important and solve many problems and user cases that are not contemplated in your proposal, which is exclusively focused on properties.

[littledan]

Yeah, at a glance, it seems like this would only make sense for certain field decorators, and not for method or class decorators, or decorators used just to attach metadata.

I'm curious, for example, how useful this proposal would be for someone like Angular @mgechev , if they ever wanted to migrate off of a custom transform.

I think such a capability to intercept reads and writes could fit into this more complex decorator proposal, as I suggested for a @tracked decorator, or @accessor.

How should we assess whether this proposal would meet needs for most use cases?

(Edit: I minimized this comment as resolved because examples showing otherwise were added above.)

[littledan]

@justinfagnani I tried to solicit a list of use cases in #231 . If someone wants to help put that all together to find themes, it'd be really helpful. What I tried to do was list some canonical use cases in the explainer, derived from what I'd generally heard about what was useful.

[mgechev]

In Angular we primarily use decorators for attaching metadata to classes, properties, and constructor parameters. The semantics described here should be good enough for properties, although we really don't need to "trap" them.

Regarding classes, we don't have complex runtime requirements, so:

Out of scope: Decorating classes and functions

Decorating classes and functions is in principle out of scope of this proposal. However, using decorators as purely syntactic sugar would suffice imho.

E.g.: @a class X -> const X = a(class X) and similar for functions worked fine for us in the legacy proposal.

should be good enough on our side even for JIT.

I'm curious, for example, how useful this proposal would be for someone like Angular @mgechev , if they ever wanted to migrate off of a custom transform.

I don't think we'll ever move from custom build & runtime time transforms. We compile the templates declared in the component metadata to a format similar to incremental DOM, so we can speed up rendering and change detection.

[fkleuver]

In Angular we primarily use decorators for attaching metadata to classes, properties, and constructor parameters.

In Aurelia 2 we also use them for this purpose, but I see that as the opposite of "they cover our use cases".
It's a compatibility / safeguard mechanism. More along the lines of "they are far too limited, so we'll just do it our own way".

Do we really want to force every feature-rich framework down that path? Using decorators purely for static metadata to do runtime rendering (or build time codegen) means you won't get any native performance benefits. You're still relying on the browser's JIT for all the behavior produced by the metadata. What's the point then?

So let's talk about a use case that stands out to me the most: @tracked. We need some configurability here, for example the ability to specify a callback method on an individual property basis, because forcing everything through a single render() method is going to be a performance killer for sure.

Here's what we do in Aurelia with @bindable:

export class NameTag {
  // short-hand for: @bindable({ attribute: 'first-name', callback: 'firstNameChanged', mode: BindingMode.toView })
  @bindable firstName;
  @bindable lastName;

  firstNameChanged(newValue, oldValue) {
    console.log(`firstName changed from ${oldValue} to ${newValue}`);
  }
  lastNameChanged(newValue, oldValue) {
    console.log(`lastNamechanged from ${oldValue} to ${newValue}`);
  }
}

What this translates to at runtime, looks something like this:

export class NameTag {
  set firstName(value) { this.$observers.firstName.setValue(value); }
  get firstName() { return this.$observers.firstName.getValue(); }

  set lastName(value) { this.$observers.lastName.setValue(value); }
  get lastName() { return this.$observers.lastName.getValue(); }

  firstNameChanged(newValue, oldValue) {
    console.log(`firstName changed from ${oldValue} to ${newValue}`);
  }
  lastNameChanged(newValue, oldValue) {
    console.log(`lastNamechanged from ${oldValue} to ${newValue}`);
  }
}

In summary, the runtime renderer wires things up in such a way that incoming changes are sent directly (and only) to listeners that need those changes. In the case of a custom element, that would be for example a text node (which is wrapped by a light-weight accessor object to avoid unnecessary DOM reads).

This is what incremental dom mentioned by @mgechev more or less boils down to and from a performance point of view this is vastly superior to any VDom approach.

There are more useful configuration properties for tweaking performance and control flow but this is the main thing I wanted to touch base on first. How do other people feel about a per-property callback granularity? Or in general, configurability via object literals?

[mweststrate]

Thanks for the feedback guys! I updated the original proposal above to clarify more stuff.

1. I confusingly mentioned class decorators to be out of scope, with that I meant out of scope of this issue, not of the entire proposal. So clarified that, but I think in essence we can largely follow the semantics babel legacy / typescript experimental decorators already supports.
2. There are quite a bunch of cases where a decorator are just used to collect meta data. Although that was not impossible in the original proposal, it wasn't very clean either. To handle that case better I updated the proposal from "a decorator expression should result in a trap" to "a decorator expression should result in a function, that receives the target and propname, and might result in a trap"
3. I added more examples from the original readme. I think I now covered all except for @set and @callable. I think the @set one is the only one we can't implement in this proposal

@pabloalmunia @justinfagnani let me know if this satisfies all (important) use cases for you. Sorry for the initial confusion, I didn't intend to limit decorators to properties, but just this issue ;-). As my impression the biggest problem with the previous proposal is the possibility to establish properties in to much unpredicatable ways (from VM perspective)

@littledan @mgechev with the above changes, attaching meta data only should no be a first-class supported pattern

@fkleuver I agree that tracked is the most interesting case. With this proposal the runtime trap would be a s simple as the following, without any further compile step:

const bindableTraps = {
   get(_target, instance, prop, _value) {
    return instance.$observers[prop].getValue()
   },
   set(_, instance, prop, value) {
     instance.$observers[prop].setValue(value)
     return undefined
   }
}

const bindable = () => bindableTraps

[fkleuver]

@mweststrate Thanks for the example.

So if I understand correctly, interacting with those properties would give you the power and flexibility as if you were talking to a proxy, but without actually talking to a proxy.

Is it also possible to let users pass in arbitrary config into the decorators with your idea? I'm sorry if that's already detailed in your proposal or elsewhere, but I couldn't find it:

e.g.

bindable.js

// Module-scoped lookup
const lookups = new WeakMap();

function getObserver(instance, prop, config) {
  if (!lookups.has(instance)) {
    lookups.set(instance, {});
  }
  const lookup = lookups.get(instance);
  return lookup[prop] || (lookup[prop] = new Observer(instance, prop, config));
}

// The decorator
export function bindable(config) {
  return function () {
    return {
      get(_target, instance, prop, _value) {
        return getObserver(instance, prop, config).getValue();
      },
      set(_, instance, prop, value) {
        getObserver(instance, prop, config).setValue(value);
      }
    };
  };
}

date-picker.js

export class DatePicker {
  @bindable({ mode: BindingMode.twoWay, callback: 'dateChanged' }) day;
  @bindable({ mode: BindingMode.twoWay, callback: 'dateChanged' }) month;
  @bindable({ mode: BindingMode.twoWay, callback: 'dateChanged' }) year;
  

  dateChanged() {
    // ...
  }
}

If the traps idea is this flexible, then from a property decorator standpoint I'm pretty confident this would address all our needs.

EDIT I missed the @logged(true) example you added, so it seems the above would be possible. Correct me if I'm wrong though.

[mweststrate]

You're correct :).

[fkleuver]

Then this has my vote :)

@littledan From a framework author's point of view (one that does a lot of fancy stuff with decorators), this proposal looks perfect as far as property decorators is concerned.

[benjamingr]

This looks like a really nice proposal and I would love to play with it if there is a babel/typescript plugin.

[blikblum]

I tried to port some decorators i use currently to this proposal and found is not clear what parameters are expected to be received in the decorator. In metadata example target, propName are received as parameters but other examples are not explicit.

Below are some of my cases that (to be used in custom elements):

state decorator tracks the changes to it, rendering the element. But also needs to set some metadata to be used in connect/disconnectedCallback lifecycle methods.

// usage:
class MyView extends MyBaseElement {
  @state({copy: true}) model
}

This is how i would implement with this proposal where registerStateProperty sets some meta data

function state(options) {
  return (target, name) => {
    registerStateProperty(target.constructor, name, options)
    return {
      set(t, i, p, v) {
        // if (!initializingField) -> possible to do that check?
          instance.queueUpdate()        
        return v
      }
    }
  }
}

Is this right?
One drawback over static proposal is that i don't know if the value is being set because of a class property initialization (to not call instance.queueUpdate())

Static implementation:

decorator @state(options) {
  @initialize((instance, name, value) => instance[`__internal_${name}`] = value)
  @register((target, name) => {
    registerStateProperty(target.constructor, name, `__internal_${name}`, options);
  })
}

[blikblum]

Other use case i have with decorators is delegating some events, done at class constructor.

In static proposal i can do:

decorator @event(eventName, selector) {
  @initialize((instance, name, value) => delegate(selector ? this.renderRoot || this : this, eventName, selector, value, this);)
}

Seems is explicitly not possible with this:

This proposal doesn't offer a way to run code guaranteed when construction a new instance

I could implement by setting some metadata and in constructor of a BaseElement loop over the events to be delegated (just like i do in the decorator version implemented by Babel)

[mweststrate]

During field installation instance will be undefined.

---

I could implement by setting some metadata and in constructor of a BaseElement loop over the events to be delegated (just like i do in the decorator version implemented by Babel)

Correct, this would require the same workaround as in Babel. Unless you use a class decorator, because than you could do something like to wire up logic after the constructor. This approach might or might not be too obtrusive

function classDecorator(klass) { 
  return class extends klass { 
    constructor() { 
       super.apply(this, arguments); 
       fireTheEvents() 
    } 
  }
}

@classDecorator class someClass {

}