Introducing the new state, "ResolveDependencies"

[Constellation]

I think that the current draft spec incurs infinite recursion when there is the circular dependencies.
The situation is the following,

There are several modules,

A: This is the entry point.

import "B"

B

import "C"

C

import "B"

1. At that time, before linking the modules, we call RequestInstantiateAll(A).
2. In RequestInstantiateAll(A), step 1-b-i-1, we extract the B dependency.
3. So, step 1-b-i-3, we perform RequestInstantiateAll(B).
4. Later, it will extract the C dependency. So, step 1-b-i-3 in RequestInstantiateAll(B), we call RequestInstantiateAll(C).
5. Since C has the B dependency, in RequestInstantiateAll(C) step 1-b-i-3, we again call RequestInstantiateAll(B).
6. Here, now, since once RequestInstantiate(B) is called inside RequestInstantiateAll(B), it immediately returns the promise B.[[Instantiate]]. But, Now, we stored the status of instantiation in [[Instantiate]], but we didn't store the status of collecting the dependencies.
7. So, in RequestInstantiateAll(B), RequestInstantiate(B) returns the stored promise, but it performs the dependency resolution again in 1-a or later. So it will call RequestInstantiateAll(C) at step 1-b-i-3 again.
8. RequestInstantiateAll(B) and RequestInstantiateAll(C) are called repeatedly.

This problem comes from that the current algorithm does not store the status of "resolving the dependencies".

To solve this, I would like to propose introducing the new state ResolveDependencies. That means, "Now, the instantiation is done, so the entry is ready to resolve the dependencies. And if there's entry.[[ResolveDependencies]] promise and the state is ResolveDependencies, we're resolving the dependency of the module just now".

The algorithm becomes the following,

CommitInstantiated(loader, key)

1. Let instance be Instantiation(loader, optionalInstance, source).
2. ReturnIfAbrupt(instance).
3. // TODO: edge case: what if instance is a thenable function?
4. Fulfill entry.[[Instantiate]] with instance.
5. Let deps be a new empty List.
6. If instance is a Module Record, then:
   1. Assert: instance is a Source Text Module Record.
   2. Set instance.[[RegistryEntry]] to entry.
   3. For each dep in instance.[[RequestedModules]], do:
      1. Append the record { [[key]]: dep, [[value]]: undefined } to deps.
7. Set entry.[[Dependencies]] to deps.
8. Set entry.[[Module]] to instance.
9. SetStateToMax(entry, "resolveDependencies").

RequestResolveDependencies(loader, key)

1. Let entry be EnsureRegistered(loader, key).
2. Let linkStateValue be GetStateValue("link").
3. If stateValue is greater than linkStateValue, return a new error promise.
4. If entry.[[ResolveDependencies]] is not undefined, return entry.[[ResolveDependencies]].
5. Return the result of transforming RequestInstantiate(loader, key) with a fulfillment handler that, when called with argument entry, runs the following steps:
   1. Let depLoads be a new empty List.
   2. For each pair in entry.[[Dependencies]], do:
      1. Let p be the result of transforming Resolve(loader, pair.[[key]], key) with a fulfillment handler that, when called with value depKey, runs the following steps:
         1. Let depEntry be EnsureRegistered(loader, depKey).
         2. If depEntry.[[ResolveDependencies]] is not undefined
            1. Return the result of transforming depEntry.[[Instantiate]] with a fulfillment handler that, when called with value depEntry, runs the following steps:
               1. Let dep be depEntry.[[Module]].
               2. Set pair.[[value]] to dep.
               3. Return depEntry.
         3. Return the result of transforming RequestCollect(loader, depKey) with a fulfillment handler that, when called with value depEntry, runs the following steps:
         4. Let dep be depEntry.[[Module]].
         5. Set pair.[[value]] to dep.
         6. Return depEntry.
      2. Append p to depLoads.
   3. Let p be the result of waiting for all depLoads.
   4. Return the result of transforming p with a fulfillment handler that, when called, runs the following steps:
      1. SetStateToMax(entry, "link").
      2. Return entry.

[caridy]

hey @Constellation, happy to see you guys digging around here ;).

I will review this asap.

[Constellation]

@caridy Thank you. We're now prototyping the loader implementation in WebKit. So I think we can help your spec. (And your spec already helps us a lot :D)

[Constellation]

After implementing this part in the prototype in WebKit, we slightly modified the proposal and its name.
I've reflected it.

[caridy]

most likely related to #79

[guybedford]

I do think a new state is needed here, it's about distinguishing between instantiate (just this module) and instantiate all (this module and all its dependencies) as two separate states before link.

[guybedford]

Note this resolves the last issue in #40.

[caridy]

we were unable to get to this issue yesterday, we were stuck on the promise issues, but we will try to think through this issue early next week. stay tuned.

[Constellation]

How about this? @caridy @domenic

[caridy]

sorry for the delay on this, will try to get dave and yehuda to discuss it next week. Btw, @dherman already spent some time on this few weeks ago, we haven't get to conclusion yet.

[dherman]

@Constellation — I just want to thank you for your contribution here. I've gone ahead and specified essentially the same algorithm (but with different names) in 261543c.

[Constellation]

@dherman: :-)