Toolchain support in wasm-tools

[alexcrichton]

This isn't a bug report or an issue per se but more of a 👋 about the toolchain support for this proposal in the wasm-tools repository. I wanted to open this as a status if it's helpful for phase 4, so feel free to just take note and then close!

A summary of what's implemented is:

• Annotations are parsed by the text parser and ignored by default.
• While parsing annotations that are "known", for example @custom, they must be valid in the sense that they must be only placed in allowed locations and have valid syntax. For example (module (func (@custom))) is a parse error.
• Support for @name as specified in this proposal is implemented.
• Support for @custom as specified in this proposal is implemented.
• Support for @metadata.code.branch_hint is implemented. (more relevant for https://github.com/WebAssembly/branch-hinting but wanted to note)
• Support for @producers is implemented. This corresponds the producers section and uses invented syntax (not specified anywhere). The purpose is to make reading this easier on the eyes.
• Support for @dylink.0 is implemented. This corresponds to the dynamic linking tooling conventions and also uses invented syntax. This is intended to assist in writing text-format test cases for some component-related things.

The binary-to-text printing done by wasm-tools print will additionally print all the annotations above. For example:

$ echo 'fn main() {}' > foo.rs
$ rustc foo.rs --target wasm32-wasi --emit obj
$ wasm-tools print foo.o
(module
...
  (@custom "linking" (after data) "\02\08\c9\85\80\80...rodata..L__unnamed_1\02\00")
  (@custom "reloc.CODE" (after data) "\05\19\00\06\01\07...\02")
  (@custom "reloc.DATA" (after data) "\06\04\02\06\0a...\1a\06")
  (@producers
    (processed-by "rustc" "1.78.0 (9b00956e5 2024-04-29)")
  )
  (@custom "target_features" (after data) "\02+\0fmutable-globals+\08sign-ext")
)

Round-tripping doesn't work in general for binaries like this since custom sections refer to binary offsets and the binary->text->binary transformation can change binary offsets, but it's been very useful to at least even see the presence of custom sections in the text format (whereas before they weren't printed so there wasn't an indication something was being dropped)

[tlively]

Amazing, thanks @alexcrichton! Would you be willing to add the invented annotation syntax to the documentation of relevant custom sections in the tool-conventions repo? It would be great if tools could be interoperable even for those quasi-standard custom sections.

[alexcrichton]

Sure! I was semi-waiting for this to reach phase 4 and/or get merged into the spec, but now feels like a good time as well. I'd love to do the same for reloc/linking sections as well but I think those are much trickier as they deal with binary offsets (either that or I'm exposing my lack of knowledge of those sections)

[yuri91]

for code metadata we deal with binary offsets by placing the annotation next to the item that the offsets points to (instructions in this case), so that the offset is implicit and can be reconstructed (possibly with a different value) when roundtripping.

I am not too familiar with these linking sections, but at a glance something similar seems possible

[alexcrichton]

I've opened WebAssembly/tool-conventions#228 for the @producers and @dylink.0 bits.

@yuri91 indeed! I was referring to how generic printing of @custom sections is not suitable for sections with binary offsets, such as in the example above the printing of reloc.CODE and reloc.DATA. I agree that the solution would involve annotating individual instructions themselves.

To me this is a bit of a tension/tradeoff in being able to print aribtrary custom sections but the output not being round-trippable. The consequence is that any custom section with binary offsets will require a dedicated annotation syntax (such as @metadata.code.branch_hint). This, subjectively to me, feels like an acceptable tradeoff insofar as I can't think of anything else as an alternative.