Unclear timing relationship between multiple media element nodes and web Audio

[SteveALee]

Describe the issue

Apologies if I missed something but I cannot find a clear specification of how multiple media element nodes should synchronise between themselves and with web audio streams/buffers. I also find behaviour significantly varies between browsers. So I wonder if different implementation choices have been made due a gap in the web audio spec?

An approach to making a simple audio mixer is to use HTML audio media elements as inputs and merging the outputs to a mixer node (or just the audioContext.destination). The high level Audio element API can then be scripted to manage the audio loading or streaming (URL fetch or via a blob, say) and wired into the low level web audio graph with createMediaElementSource(). Then. audio element methods like play/pause and loop provide easy 'transport' control`.

However, I find with this simple approach that playing and mixing two identical audio waveforms gives a range of results across browsers:

• Chrome / Edge on Windows - perfect - always sounds like a single waveform
• Firefox on Windows - light flanging / slip, often made worse after multiple pause / play cycles
• Safari on MacOs, Safari or Chrome on iPadOS - very bad sync slip of many mS making an audable mess

Here's an example script where the wave forms are simple metronome click tracks to highlight the issue

    let audioContext
    let audio1, audio2

    function playAudio() {
      if (!audioContext) {
        audioContext = new AudioContext()

        audio1 = new Audio('./click.wav')
        const a1 = audioContext.createMediaElementSource(audio1)
        audio2 = new Audio('./click2.wav') // clone of click
        const a2 = audioContext.createMediaElementSource(audio2)

        a1.connect(audioContext.destination)
        a2.connect(audioContext.destination)
      }

      function togglePlay(audio) {
        audio[audio.paused ? 'play' : 'pause']()
      }

      togglePlay(audio1)
      togglePlay(audio2)
    }

Where Is It

Missing info?

Additional Information

Example hosted at https://mixer.vision-score.com/test-sync/

I raised an issue for webkit - https://bugs.webkit.org/show_bug.cgi?id=301824