pipeline state bugs

[dherman]

A few bugs in the management of internal state in the pipeline:

• We don't want to save any intermediate results of the pipeline once we move on to later states. So the conditional in the beginning of each Request<<X>> operation should check if it's in any later pipeline stage, not just link-or-later.
• Instead of an error promise, if the data is gone it should succeed with a tombstone value of undefined. This indicates that the stage is complete but we no longer have the data available. One way of looking at this is the .load method enables prefetching and signals completion, and that completion provides the intermediate value only if it's still available. Prefetching logic needs to be prepared to handle this case. (Note that the high-level .import method has none of this subtlety.) Another way of looking at it is if you want to be guaranteed to get the data as it's available, you need to be setting the hooks, not using .load.
• In order to avoid race conditions, each Request<<X>> stage should have a three-armed conditional:
  • If we're already in exactly the stage X, we have the promise for its result already, so just return that.
  • If we're in a later stage (> X), return a tombstone promise.
  • If we're in an earlier stage, call the preceding Request<<X-1>> stage and .then its result. Note that this guarantees we aren't in a later stage than X-1 so we are guaranteed not to get a tombstone.
• The .provide method should be async -- it should Promise.resolve its argument and then update the state. (The result of .provide should be a Promise<undefined> instead of undefined, so that the caller can be notified when the update has happened.) This resolves several issues:
  • It ensures that the user argument gets promise-assimilated (i.e. in case it's thenable).
  • It pushes race conditions to between event turns.
  • It provides a guarantee that the only synchronous question you should be able to ask of the pipeline state -- namely, what pipeline stage are we in -- should be stable throughout the current event turn. (Iteration is also synchronously sensitive to the set of keys in the registry, but the values it provides should be promises.)

[dherman]

Note that the last bullet point will resolve the TODO question in CommitInstantiated: we have an invariant (should be an assert towards the beginning of the abstract operation's steps) that the value is fully assimilated. It can never be a thenable.

[caridy]

related to #96

[caridy]

solved by PR #97