Extend sample format support by integrating AudioSampleFormat from WebCodecs

[bendmorris]

(This is a specific proposal to address an issue which has been raised elsewhere, e.g. #2458)

The WebCodecs standard supports a range of buffer formats, while the Web Audio public API is currently limited to planar 32-bit floats.

This use case has come up when interfacing between native decoders compiled to WebAssembly and Web Audio. Interleaved formats like "s16" are a common output format of decoders. Piping this data into Web Audio requires format conversion in JavaScript (example implementation from Emscripten) which can be error-prone and may be less efficient than doing it within the implementation, where SIMD is available, results can be cached, etc.

AudioBuffer could be extended to provide APIs to read and write data in any of these supported formats. A hypothetical API could look like this:

AudioBuffer.readBufferData(format: 'u8'): Uint8Buffer;
AudioBuffer.readBufferData(format: 's16'): Int16Buffer;
// ...

AudioBuffer.writeBufferData(format: 'u8', data: Uint8Buffer, offset?: number, frames?: number);

Implementations would still be free to use whatever underlying format they choose, and decide when and how to perform the conversion.