Scoped Custom Element Registries

[justinfagnani]

Since #488 is closed, I thought I'd open up a new issue to discuss a relatively specific proposal I have for Scoped Custom Element Registries.

Scoped Custom Element Definitions

Overview

Scoped Custom Element definitions is an oft-requested feature of Web Components. The global registry is a possible source of name collisions that may arise from coincidence, or from an app trying to define multiple versions of the same element, or from more advanced scenarios like registering mocks during tests, or a component explicitly replacing an element definition for its scope.

Since the key DOM creation APIs are global, scoping definitions is tricky because we'd need a mechanism to determine which scope to use. But if we offer scoped versions of these APIs the problem is tractable. This requires that DOM creation code is upgraded to use the new scoped APIs, something that hopefully could be done in template libraries and frameworks.

This proposal adds the ability to construct CustomElementRegistrys and chain them in order to inherit custom element definitions. It uses ShadowRoot as a scope for definitions. ShadowRoot can be associated with a CustomElementRegistry when created and gains element creation methods, like createElement. When new elements are created within a ShadowRoot, that ShadowRoot's registry is used to Custom Element upgrades.

API Changes

CustomElementRegistry

• CustomElementRegistry(parent?: CustomElementRegistry)

  CustomElementRegistry is constructible, and able to inherit from a parent registry.

  New definitions added to a registry are not visible to the parent, and mask any registrations with the same name defined in the parent so that definitions can be overridden.

• CustomElementRegistry.prototype.get(name: string)

  get() now returns the closest constructor defined for a tag name in a chain of registries.

• CustomElementRegistry.prototype.getRegistry(name: string)

  Returns the closest registry in which a tag name is defined.

ShadowRoot

ShadowRoots are already the scoping boundary for DOM and CSS, so it's natural to be the scope for custom elements. ShadowRoot needs a CustomElementRegistry and the DOM creation APIs that current exist on document.

• customElements: CustomElementRegistry

  The CustomElementRegistry the ShadowRoot uses, set on attachShadowRoot().

• createElement(), createElementNS()
  These methods create new elements using the CustomElementRegistry of the ShadowRoot.

• importNode()
  Imports a node into the document that owns the ShadowRoot, using the CustomElementRegistry of the ShadowRoot.

  This enables cloning a template into multiple scopes to use different custom element definitions.

Element

New properties:

• Element.prototype.scope: Document | ShadowRoot
  Elements have DOM creation APIs, like innerHTML, so they need a reference to their scope. Elements expose this with a scope property. One difference between this and getRootNode() is that the scope for an element can never change.

• Element.prototype.attachShadow(init: ShadowRootInit)

  ShadowRootInit adds a new property, customElements, in its options argument which is a CustomElementRegistry.

With a scope, DOM creation APIs like innerHTML and insertAdjacentHTML will use the element's scope's registry to construct new custom elements. Appending or inserting an existing element doesn't use the scope, nor does it change the scope of the appended element. Scopes are completely defined when an element is created.

Example

// x-foo.js is an existing custom element module that registers a class
// as 'x-foo' in the global registry.
import {XFoo} from './x-foo.js';

// Create a new registry that inherits from the global registry
const myRegistry = new CustomElementRegistry(window.customElements);

// Define a trivial subclass of XFoo so that we can register it ourselves
class MyFoo extends XFoo {}

// Register it as `my-foo` locally.
myRegistry.define('my-foo', MyFoo);

class MyElement extends HTMLElement {
  constructor() {
    super();
    // Use the local registry when creating the ShadowRoot
    this.attachShadow({mode: 'open', customElements: myRegistry});

    // Use the scoped element creation APIs to create elements:
    const myFoo = this.shadowRoot.createElement('my-foo');
    this.shadowRoot.appendChild(myFoo);

    // myFoo is now associated with the scope of `this.shadowRoot`, and registy
    // of `myRegistry`. When it creates new DOM, is uses `myRegistry`:
    myFoo.innerHTML = `<my-bar></my-bar>`;
  }
}

Open Issues

• Interaction with declarative shadow DOM #914
• Moving elements with shadow roots between documents #907
• Element upgrade ordering #923
• [scoped-registries] How does customElements.upgrade() work? #1001

Questions

This section is not current. See the open issues list

• What happens to existing upgraded elements when an overriding definition is added to a child registry?

  The simplest answer is that elements are only ever upgraded once, and adding a new definition that's visible in an element's scope will not cause a re-upgrade or prototype change.

• Should classes only be allow to be defined once, across all registries?

  This would preserve the 1-1 relationship between a class and a tag name and the ability to do new MyElement() even if a class is not registered in the global registry.

  It's easy to define a trivial subclass if there's a need to register the same class in different registries or with different names.

• Should registries inherit down the tree-of-trees by default, or only via the parent chain of CustomElementRegistry?

  Inheriting down the DOM tree leads to dynamic-like scoping where definitions can change depending on your position in the tree. Restricting to inheriting in CustomElementRegistry means there's a fixed lookup path.

• Should the registry of a ShadowRoot be final?

• Is Element.prototype.scope neccessary?

  It requires all elements to remember where they were created, possibly increasing their memory footprint. Scopes could be dynamically looked up during new DOM creation via the getRootNode() process instead, but this might slow down operations like innerHTML.

• How does this interact with the Template Instantiation proposal?

  With Template Instantiation document.importNode() isn't used to create template instances, but HTMLTemplateElement.prototype.createInstance(). How will that know which scope to use? Should it take a registry or ShadowRoot?

/cc @domenic @rniwa @hayatoito @TakayoshiKochi

[caridy]

Questions:

• when you talk about "inheritance", do you mean "hierarchy" instead?
• is parent exposed somehow in a custom element registry? or is it just an internal slot of some sort?
• can you clarify the use case for exposing CustomElementRegistry.prototype.getRegistry(name: string)
• is there a way to know when a name is being defined in a particular registry?
• when a new entry is registered in the global registry, and 10 other registers are depending on it (waiting to be upgraded), what's the process to upgrade them all? Is there a booking process somewhere?

Missing features

• Today, with the global registry, there is no way to intercept the usage of a particular tag, which forces application to load them all during the booting process, or do some sort of book keeping on each template to load what they needed as dependencies. I wonder if we can have some sort of hook at the registry level to tell you when an tagName is being used, so you can decide what to do, go fetch it and register it, register it from another registry, etc.
• Having to have a lineal chain of registries might be insufficient, and harder to use (this about the namespacing use-case where components in the same namespace can see/use each other, while some namespaces will have some hierarchical organization).

Recommendation

Based on those two possible missing features, and extensibility point of view, it is probably easier to find an API that delegates part of the resolution to user-land, and let users to implement the hierarchical/resolution algo. e.g.:

class MyFoo extends HTMLElement {}

// lookup must return a registry
function lookup(registry, name) {
    if (name === 'my-bar') {
        return customElements; // delegate the lookup to another known registry (in this case the global registry)
    }
    if (name === 'my-baz') {
        registry.define('my-baz', class extends HTMLElement {}); // define inline
    } else {
        // import `name` component, and define it in `register` when it is ready...
    }
    return registry;
}
const myRegistry = new CustomElementRegistry(lookup);
myRegistry.define('my-foo', MyFoo); // you can still prime the registry

This will require effectible a new constructor with a single argument, nothing else. Or could potentially be fold into ShadowRoot constructor as well.

Wish List

• fully composable registry graph where the resolution of a name can be delegated to any registry where the logic can be customized in user-land.
• the ability to introspect into the resolution mechanism to support mocking, lazy fetching and registration, custom resolution rules.
• preserve the semantics of the current registry to lower the friction for implementers (being realistic here).

[caridy]

hey @justinfagnani, this is the comment that I've mentioned today, let's chat about this tomorrow, I can explain more.

[trusktr]

What happens in my-foo from the above example is removed then appended into another tree unrelated to the scope? Does it just continue to work? Or is an error thrown?

fully composable registry graph where the resolution of a name can be delegated to any registry where the logic can be customized in user-land.

This seems like it could be useful for some sort of framework, but would definitely be nice to have a default (that just looks in the parent scope) so that the end user can just do something perhaps as easy as

const myRegistry = new CustomElementRegistry()
myRegistry.define('my-foo', MyFoo);
this.root = this.attachShadow({mode: 'open', registry: myRegistry})

which causes lookup to look in the parent scope (parent shadow root) when the element name is not found in the current registry.

Just tossing in a syntax idea:

this.root = this.attachShadow({mode: 'open', registry: true})
this.root.define('my-foo', MyFoo);

or maybe just simply:

this.root = this.attachShadow({mode: 'open'})
this.root.define('my-foo', MyFoo); // creates a registry internally on first use

And for advanced use (f.e. defining lookup):

this.root = this.attachShadow({mode: 'open'})

console.log(this.root.registry) // null

this.root.define('my-foo', MyFoo); // creates a registry on first use

console.log(this.root.registry) // CustomElementsRegistry

// ... and for advanced users:
this.root.registry.defineLookup(function() { ... })

// also use the registry directly:
this.root.registry.define('other-foo', OtherFoo)

This way it's easier, yet still configurable for advanced cases.

[trusktr]

Oh! This is also a great opportunity to not require hyphens in element names of scoped registries! Maybe it's possible?

[Jamesernator]

@trusktr Although I really like the idea of hyphen-less names what would happen if you happen to upgrade an existing name?

e.g. What on earth would happen in this situation:

<link rel="stylesheet" src=".." />

<script>
    class MyLink extends HTMLElement {
        constructor() {
            // Would this still be a proper link element?

            // If so this clearly wouldn't work as HTMLLinkElement
            // doesn't support attachShadow 
            this.attachShadow({ mode: 'open' })
        }
    }

    window.customElements.define('link', MyLink)
</script>

Now I think this could actually be resolvable by having a special element that must be included in head (similar to <base>/<meta>) that declares all names that will be overridden so that the browser knows ahead of time not to assign any builtin behavior to those elements.

For example it might look something like this:

<head>
    <override element="link">
    <override element="input">
    <!-- Or maybe <override elements="link input">
</head>
<body>
    <!-- link no longer works as a link tag but is just a plain html tag -->
    <link rel="stylesheet" />

    <!-- Neither does input -->
    <input type="date">

    <script>
        ...
        customElements.define('input', MyCustomInput)
    </script>
</body> 

Of course this doesn't explain what'd happen if override itself is overriden (not allowed? namespaces (<html:override element="link">)?) or any other tag like meta. Perhaps metadata tags would need to be strictly reserved by html (which would prevent future metadata tags being added but maybe the existing metadata tags (particularly <meta>) are already sufficiently flexible for all such purposes?).

I think it's simpler to keep this and #658 separate given that I don't think it's worth blocking scoped registries on a topic that I personally think is much more complicated than scoped registries.

[trusktr]

What on earth would happen in this situation:

Just like with variable shadowing in practically any programming language, then in that case that <link> is no longer the sort of <link> from the outer scope, and it will not pull in the stylesheet, unless the Custom Element implements that.

const foo = "foo"

~function() {
  const foo = "bar"

  console.log(foo)
  // is "foo" here the same "foo" as outside? Nope, not any more, we're in a new scope!
}()

Same thing for elements! If you override a "variable" (in this case an element name) in the inner scope, then it is no longer what it was in the outer scope.

<override element="input">

But, that's in global scope. Overriding should only be possible in non-global scope. Maybe, <override element="XXX"> is something that could work, as it can signal the parser not to do certain things if xxx is tr, for example.

But then again, I don't like redundancy, I like things DRY.

Imagine if this was required in JavaScript:

const foo = "foo"

~function() {
  override foo;
  const foo = "bar"
  console.log(foo)
}()

I would not prefer a similar thing for HTML name scope. But, what if an HTML/CSS no-JS person looks at the markup? They might get confused? True! I would probably not want to do that, just like I don't override globals in JS. What it would really be useful for is, for example, coming up with new element names that don't already exist (like <node3d> or <magic>), and then if the browser by chance introduces one of those names, oh well, then the component will just work, and everyone can be happy. If a component wasn't using a yet-to-exist <magic> element before, but rather a custom element called <magic>, then, who cares if the browser introduces a new <magic> element later, as long as that component continues to work. Some other component can decide to use the builtin <magic> element by not overriding it.

I find myself in situation where I'm forced to think of another word to add to a single-word component, just to appease the hyphen rule. Sometimes I do something dumb like <stu-pid> just to get around the limitation and keep the single <wo-rd> element, which is awkward.

So my specific argument isn't leaning towards overriding certain builtins, though I can imagine that if someone wanted to implement a "super <link>" that worked the same, plus did much more, while making it easy to adopt by simple override, then why not?

Personally, I just want to use single words when I want to.

[trusktr]

I don't think it's worth blocking scoped registries on a topic that I personally think is much more complicated than scoped registries.

Good point. That'd be great regardless of hyphen-or-not!

[TakayoshiKochi]

Scopes are completely defined when an element is created.

This sounds the most important principle in the proposal, to understand the behaviors. It's a new ownerDocument, so to say.

comments

• We already have a global registry on window, not on document. Do we also want a registry which is associated to document such that any descendant shadow roots is not affected?

• Related, for Element.prototype.scope, if the element is from the global registry, why document but not window?

• Why CustomElementRegistry inherits from another CustomElementRegistry? I'd guess users may want to use elements registered to its parent scope, but rarely want to use sibling or descendant ones. Why not this inheritance (custom elements name lookup chain) be specified via ShadowRoot creation? (e.g. ShadowRootInit has inheritCustomElementRegistry: true?)

• For mix-ins, CustomElementRegistry.prototype.import(registry: CustomElementRegistry) to import already defined elements? And for not overridden definitions, a newly created element will be given its definition's original scope.

• If global registry contains a definition for <my-element>, and if a shadow root contains <my-element> as well, it will be upgraded before the definition for scope-local <my-element> is given, but as the element is already upgraded, it never gets upgraded to its scoped version. Later if <my-element> is appended after the scoped definition, it becomes a scoped <my-element>. This behavior is understandable as well as confusing - will this be better if we have explicit customElements.upgrade (Add customElements.upgrade #710)?

• Shall we introduce ShadowRoot.prototype.adoptNode in addition to importNode?