Skip to content

Audio: decode 24-bit PCM and WAVE_FORMAT_EXTENSIBLE float WAV uploads correctly - #5556

Open
singhharsh1708 wants to merge 1 commit into
tenstorrent:mainfrom
singhharsh1708:fix/wav-decode-24bit-extensible-float
Open

singhharsh1708 wants to merge 1 commit into
tenstorrent:mainfrom
singhharsh1708:fix/wav-decode-24bit-extensible-float

Conversation

@singhharsh1708

Copy link
Copy Markdown

/v1/audio/transcriptions decodes WAV uploads in AudioManager._decode_wav_file (tt-media-server/utils/audio_manager.py). Two sample formats come out wrong:

24-bit PCM pads each 3-byte little-endian sample with a trailing zero byte and shifts the int32 right by 8, which computes floor(unsigned value / 256) before dividing by 2^23: positive samples land about 256 times too quiet and negative samples wrap to positive values. The decoder now pads with a leading zero byte and shifts, vectorised, so the sign survives.

32-bit samples are read as float only when the format tag is 3. Float WAVs with a WAVE_FORMAT_EXTENSIBLE header (tag 0xFFFE, which ffmpeg and sox write for float audio above 48 kHz or with more than two channels, and what soundfile's WAVEX writes) were read as int32 PCM. The decoder now takes the format tag from the SubFormat GUID when the header is extensible; 16-bit and 24-bit extensible PCM were already decoded by bit depth and are unchanged.

Measured on CPU: openai/whisper-large-v3 on the 73 LibriSpeech clean clips of hf-internal-testing/librispeech_asr_dummy, saved as 16 kHz WAV and decoded with a line-for-line copy of this function (24-bit) or the function itself (extensible float), gives WER 37.2% for 24-bit PCM and 29.5% for extensible float, against 2.65% when the same bytes are decoded with soundfile. The 24-bit waveform has SNR -0.3 dB against the source and a peak of 0.008 where the source peaks at 0.526.

Tests added to tests/test_audio_manager.py: a 24-bit WAV written with the wave module and an extensible float WAV with a hand-built header, both checked against the source samples to one 24-bit step, plus a round trip against soundfile for both formats (importorskip, so it is skipped where soundfile is absent). On main all four fail (max abs error 1.0 and 0.59 on the hand-built files); with this change tests/test_audio_manager.py has 47 passed and the same one pre-existing failure as main on this machine (test_cpu_seconds_measures_a_real_subprocess, which reads /proc). Not run on device.

@singhharsh1708
singhharsh1708 requested a review from a team as a code owner November 9, 2026 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant