[parthash0804]

Summary

PR #890 ("[ROCm][Bench] Add audio input support to CustomMMDataset") was the fork's interim way to drive omnimodal models with audio through vllm bench serve, by adding audio_files support to CustomMMDataset plus a process_audio() helper and unit tests.

That functionality has since been superseded by upstream. The merge cdd11a6ab ("Merge upstream vLLM code into gfx11 (#983)") pulled in upstream PR vllm-project#41576 ("Implement custom dataset class for ASR benchmarking"), which:

• replaced CustomMMDataset with CustomImageDataset + CustomAudioDataset,
• introduced the official --dataset-name custom_audio path, and
• rewrote process_audio() to the upstream implementation.

As a result, PR #890's datasets.py changes no longer exist on gfx11, but its test file tests/benchmarks/test_custom_mm_dataset_audio.py was left behind. That test imports CustomMMDataset (now removed), so it is broken/orphaned and would fail at import.

Changes

• Remove the orphaned test tests/benchmarks/test_custom_mm_dataset_audio.py.
• No change to datasets.py: reverting PR [Bench] Add audio input support to CustomMMDataset #890's code commit (927bfcd33) is a no-op because the upstream merge already removed it.

Impact

• Audio benchmarking is unaffected — it now flows through the upstream --dataset-name custom_audio (CustomAudioDataset) path, which does not depend on PR [Bench] Add audio input support to CustomMMDataset #890.

How to migrate

Old (PR #890): --dataset-name custom_mm with audio_files in the JSONL.
New (upstream): --dataset-name custom_audio with {"prompt": ..., "audio": "..."} in the JSONL.

Test plan

Verified that removing PR #890 does not break audio benchmarking, by running the upstream custom_audio path against a live server on this branch.

Model: Qwen/Qwen2.5-Omni-7B (ROCm, gfx1151)

Command:
vllm bench serve
--backend openai-chat --endpoint /v1/chat/completions
--model Qwen/Qwen2.5-Omni-7B
--dataset-name custom_audio
--dataset-path audio.jsonl
--enable-multimodal-chat --num-prompts 8

Result: 8/8 requests succeeded, 0 failed.
============ Serving Benchmark Result ============
Successful requests:                     8
Failed requests:                         0
Benchmark duration (s):                  16.25
Total input tokens:                      874
Total generated tokens:                  143
Request throughput (req/s):              0.49
Output token throughput (tok/s):         8.80
Peak output token throughput (tok/s):    17.00
Peak concurrent requests:                8.00
Total token throughput (tok/s):          62.58
---------------Time to First Token----------------
Mean TTFT (ms):                          12125.90
Median TTFT (ms):                        12864.34
P99 TTFT (ms):                           15433.12
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          58.62
Median TPOT (ms):                        58.55
P99 TPOT (ms):                           59.61
---------------Inter-token Latency----------------
Mean ITL (ms):                           55.48
Median ITL (ms):                         59.05
P99 ITL (ms):                            80.32
==================================================

Conclusion: the custom_audio benchmark runs end-to-end with PR #890 removed, confirming audio benchmarking depends only on the upstream CustomAudioDataset.

[mgehre-amd]

Just to make sure, this PR removes all relevant diffs between gfx11 and upstream/main that were introduced by #890, right?

[parthash0804]

Just to make sure, this PR removes all relevant diffs between gfx11 and upstream/main that were introduced by #890, right?

Yes, This PR removes the test file added by PR #890 and the changes in datasets.py are already overwritten by upstream merge.