@custom and repeated sections

[alexcrichton]

This may not be the best repository for this issue, but proposals like module-linking and conditional sections are both proposing changes to the binary format which complicate the @custom annotation. Both the proposals allow the binary format to contain multiple seconds of the same type (e.g. multiple type sections), but the @custom directive doesn't currently have syntax/affordance to figure out what to do in this case.

For example in the module-linking proposal there are three new sections (alias/instance/module) which can appear interleaved at the front of a module (or at least that's what's proposed). It's not entirely clear to me how you'd write an annotation to select which alias section, for example, you'd place the custom section around. Furthermore if there aren't any alias sections emitted and you asked to be emitted after the alias section, it's not clear where you'd actually get emitted.

I'm not sure of a great way to solve this myself, all I can think of is to specify an index of some form:

(@custom "x" (section 3) "...")

where that means the custom section would be the third section in the module. (or something like that).

I'm also not sure where's the best place to ask about this since this issue is predicated on this proposal and also other future proposals which may change. I figured this would be a good place to start though!

[tlively]

Huh, this seems complicated. Even without annotations, has anyone suggested a mechanism to specify in the text format how many sections there should be? It seems like this would be difficult for something like the type section, where types can be lazily added from multiple places in the text format.

Stepping back, is it necessary to support interleaved custom sections? What if they just always went at the end?

[alexcrichton]

I figured that one of the goal of the custom section annotations was to be able to faithfully convert a binary to text and back. The currently specified support I believe is enough to implement this, but this property is harder to implement once sections can appear multiple times.

I agree as well yeah that you can't really have predictable section indices given all the inference a text parser does with injecting types.

[rossberg]

Ouch, yeah, getting way more ugly.

FWIW, the proposal currently is careful to avoid talking about absolute section positions. Everything is expressed relatively to other sections, to minimise interference or backwards-compat issues with the introduction of future sections. However, we don't have a story for how to address repeated sections. Tbh, I don't have a good idea.

[rossberg]

Thinking about this some more, the simple way forward will be to extend this (once we have repeated sections) to be able to denote the n-th occurrence of a certain section.

For example, (after func 2) would mean "after the second function section". Plain (after func) continues to work and means "after the last function section". Similarly for before, except that (before func) inversely defaults to "before the first function section".

Does that make sense?

[lars-t-hansen]

Doesn't that (in a sense) reintroduce "absolute section positions" even though these are pieces-of-sections? Might it instead work to introduce named labels for these module positions?

(Alex's question about what (after S) means if there is no S is probably covered by the spec language that says that if S precedes T then (after S) precedes (before T). It would probably be useful though if the example in the spec draft showed this case.)

[rossberg]

Doesn't that (in a sense) reintroduce "absolute section positions" even though these are pieces-of-sections?

Sort of, but I'd think that's much less problematic, since it doesn't interfere with other, unrelated sections (especially ones that are optional, like other custom sections).

In practice, I would assume that one almost never cares about N, but would place a custom section before or after all of the respective section segments. So N wouldn't show up or matter in most cases, which is quite different from truly absolute section numbering.

FWIW, thinking again, @tlively made the important point that the more fundamental problem to solve is how to represent multiple sections in the text format in the first place. Perhaps if we have that solved, perhaps custom section placement for these falls out for free?

Might it instead work to introduce named labels for these module positions?

Hm, how so?

(Alex's question about what (after S) means if there is no S is probably covered by the spec language that says that if S precedes T then (after S) precedes (before T). It would probably be useful though if the example in the spec draft showed this case.)

Ah, interesting point. I had indeed assumed that, i.e., before/after is interpreted relative to the position where a section would be, even if absent. Though the spec does not say that clearly atm.

However, this interpretation is no longer meaningful in the presence of repeated and arbitrarily ordered sections. So I'd rather stay conservative and say that the placement is actually ill-formed (whatever that means for a tool interpreting it) if the referenced section does not exist. WDYT?

[rossberg]

@lars-t-hansen, see #16 for the latter point.

[lars-t-hansen]

Might it instead work to introduce named labels for these module positions?

Hm, how so?

A half-baked idea that hasn't improved with the additional baking time :-/ Please disregard.

(Alex's question about what (after S) means if there is no S is probably covered by the spec language that says that if S precedes T then (after S) precedes (before T). It would probably be useful though if the example in the spec draft showed this case.)

Ah, interesting point. I had indeed assumed that, i.e., before/after is interpreted relative to the position where a section would be, even if absent. Though the spec does not say that clearly atm.

However, this interpretation is no longer meaningful in the presence of repeated and arbitrarily ordered sections. So I'd rather stay conservative and say that the placement is actually ill-formed (whatever that means for a tool interpreting it) if the referenced section does not exist. WDYT?

That seems reasonable, at least for an MVP. Though it's really a question for the tools people to consider.

[alexcrichton]

Lots of time has passed since I first opened this and I'm going to close this as probably no longer applicable. The original motivation for this issue was the module-linking proposal and the conditional sections proposal, and neither still motivate this issue.

Module linking has evolved into the component model as a layer on top of core wasm. In components I've implemented support for @custom but there's no "place" unlike core wasm (e.g. no (after data) or similar). The text format generates custom sections where they are in the text format, e.g. positionally between other items. This reflects sections otherwise already work in the component model where there aren't the same restrictions as core wasm sections. Component sections can appear in any order and any number of times, and by controlling the location of @custom you can control the output.

The conditional sections proposal was decided against (IIRC, I may also be misremembering) for other unrelated reasons. This means that core wasm no longer has the problem of possibly having multiple copies of a single section within a module. That means that the current "place" syntax (e.g. (after data)) works for specifying all the various locations custom sections can go.

Given that I'm going to go ahead and close this. This may be a problem in the future if a future proposal adds reapeated custom sections, but I don't think that it causes issues for this proposal as-is.