Add Axera backend support for Supertonic TTS - #3634
Abandon-ht wants to merge 1 commit into
Conversation
dosubot
Bot
added
the
size:XXL
📝 WalkthroughWalkthroughThis PR adds complete Axera-backed SuperTonic offline TTS support: a model wrapper that loads and runs four Axera sub-model engines (duration predictor, text encoder, vector estimator, vocoder), an implementation that orchestrates text-to-audio generation through voice-style parsing, input padding, iterative latent denoising, and vocoder inference, plus build and factory integration. ChangesAxera SuperTonic Offline TTS
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the Axera-specific implementation for the Supertonic offline text-to-speech (TTS) model. It adds the implementation class OfflineTtsSupertonicImplAxera, the model wrapper OfflineTtsSupertonicModelAxera, and integrates them into the main offline TTS creation factory. The review feedback highlights several critical and high-severity issues, including a potential buffer overflow when audio duration is long, lack of validation for silence_duration and max_len (which can lead to massive memory allocations or bypassed checks due to negative-to-unsigned casts), unsafe handling of NaN values for speed and duration, and missing safety checks for JSON types and null pointers in the model initialization.
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; |
There was a problem hiding this comment.
The variable out.actual_latent_len can exceed kFixedLatentLen (300) if the audio duration is long (e.g., > 9.6 seconds at 16kHz with chunk size 512). This causes out-of-bounds writes in out.xt_flat and out.latent_mask_flat, leading to a critical buffer overflow. Clamp out.actual_latent_len to kFixedLatentLen to prevent this.
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
if (out.actual_latent_len > kFixedLatentLen) {
out.actual_latent_len = kFixedLatentLen;
}| return {}; | ||
| } | ||
|
|
||
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); |
There was a problem hiding this comment.
silence_duration is retrieved from the configuration but not validated. If a negative value is provided, casting the resulting negative float to size_t for silence_len will result in an extremely large value, causing a massive memory allocation attempt and crashing the application. Validate that silence_duration is non-negative.
float silence_duration = config.GetExtraFloat("silence_duration", 0.3f);
if (!(silence_duration >= 0)) {
SHERPA_ONNX_LOGE("silence_duration must be >= 0. Given: %f", silence_duration);
return {};
}| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
If config.GetExtraInt("max_len", ...) returns a negative value, casting it directly to size_t will result in a very large positive number, bypassing the max_len == 0 check. Retrieve max_len as an integer first, validate that it is strictly positive, and then cast it to size_t.
int32_t max_len_int =
(lang == "ko" || lang == "ja")
? config.GetExtraInt("max_len", 120)
: config.GetExtraInt("max_len", 300);
if (max_len_int <= 0) {
SHERPA_ONNX_LOGE("Max length must be > 0. Given: %d", max_len_int);
return {};
}
size_t max_len = static_cast<size_t>(max_len_int);| config.GetExtraFloat("speed", config.speed > 0 ? config.speed : 1.05f); | ||
| int32_t num_steps = config.GetExtraInt( | ||
| "num_steps", config.num_steps > 0 ? config.num_steps : 5); | ||
| if (speed <= 0) { |
There was a problem hiding this comment.
If speed is NaN, the check speed <= 0 will evaluate to false, allowing NaN to propagate and cause undefined behavior or NaN durations. Using !(speed > 0) correctly handles both negative/zero values and NaN.
if (!(speed > 0)) {| if (duration < kMinDuration) { | ||
| duration = kMinDuration; | ||
| } |
There was a problem hiding this comment.
If the duration predictor outputs NaN, the check duration < kMinDuration will evaluate to false, leaving duration as NaN. This can lead to undefined behavior or crashes during subsequent calculations. Guard against NaN values using std::isnan.
if (std::isnan(duration) || duration < kMinDuration) {
duration = kMinDuration;
}| const auto &ae = j["ae"]; | ||
| const auto &ttl = j["ttl"]; |
There was a problem hiding this comment.
If the config file is parsed successfully but the "ae" or "ttl" keys are not JSON objects (e.g., they are primitives or arrays), calling .find() or indexing them will throw an unhandled nlohmann::json::type_error exception, crashing the application. Add a check to ensure both are JSON objects.
const auto &ae = j["ae"];
const auto &ttl = j["ttl"];
if (!ae.is_object() || !ttl.is_object()) {
SHERPA_ONNX_LOGE("Invalid config file: 'ae' or 'ttl' is not a JSON object");
SHERPA_ONNX_EXIT(-1);
}| const std::vector<size_t> &input_sizes) const { | ||
| std::lock_guard<std::mutex> lock(model->mutex); | ||
|
|
||
| if (model->io_info->nInputSize != inputs.size()) { |
There was a problem hiding this comment.
If InitInputOutputAttrs fails or is unable to retrieve input/output attributes, model->io_info could be nullptr. Dereferencing it without a check will cause a segmentation fault. Add a defensive null check.
if (!model->io_info) {
SHERPA_ONNX_LOGE("%s: Model IO info is null", name);
SHERPA_ONNX_EXIT(-1);
}
if (model->io_info->nInputSize != inputs.size()) {![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
sherpa-onnx/csrc/CMakeLists.txt (1)
213-219:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGate the Axera SuperTonic sources on TTS as well.
These translation units depend on TTS-only code, but they are currently added whenever Axera is enabled.
SHERPA_ONNX_ENABLE_AXERA=ONwithSHERPA_ONNX_ENABLE_TTS=OFFwill pull them into the build without the rest of the TTS implementation.Proposed fix
-if(SHERPA_ONNX_ENABLE_AXERA) +if(SHERPA_ONNX_ENABLE_AXERA AND SHERPA_ONNX_ENABLE_TTS) list(APPEND sources ./axera/ax-engine-guard.cc ./axera/offline-sense-voice-model-axera.cc ./axera/offline-tts-supertonic-model-axera.cc ./axera/offline-tts-supertonic-impl-axera.cc ./axera/utils.cc ) endif()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sherpa-onnx/csrc/CMakeLists.txt` around lines 213 - 219, The Axera SuperTonic source files are being added whenever SHERPA_ONNX_ENABLE_AXERA is true even if TTS is disabled; wrap the list(APPEND sources ...) that adds ./axera/ax-engine-guard.cc, ./axera/offline-sense-voice-model-axera.cc, ./axera/offline-tts-supertonic-model-axera.cc, ./axera/offline-tts-supertonic-impl-axera.cc, and ./axera/utils.cc in a condition that requires both SHERPA_ONNX_ENABLE_AXERA and SHERPA_ONNX_ENABLE_TTS to be ON (i.e., gate these translation units on TTS as well) so they are only appended when both features are enabled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc`:
- Around line 380-384: The code multiplies cfg.ttl.latent_dim by
cfg.ttl.chunk_compress_factor when computing latent_dim before calling
SampleNoisyLatentFixed, which incorrectly changes the latent channel count and
causes RunVectorEstimator()/RunVocoder() fixed-size checks to fail; change
latent_dim to use cfg.ttl.latent_dim directly (remove the multiplication by
cfg.ttl.chunk_compress_factor) so SampleNoisyLatentFixed() is called with the
correct channel count and the downstream RunVectorEstimator/RunVocoder tensor
shapes match expectations.
- Around line 182-205: After computing out.actual_latent_len, immediately check
if it exceeds kFixedLatentLen (300) and fail fast: log an error and return/throw
before calling gen.Fill or any mask loops to avoid buffer overruns; ensure the
check references out.actual_latent_len, kFixedLatentLen, gen.Fill, out.xt_flat
and out.latent_mask_flat and prevents further execution if actual_latent_len >
kFixedLatentLen so no writes occur past the fixed-size buffers.
- Around line 302-314: The code currently reads silence_duration via
config.GetExtraFloat and passes it to ProcessChunksAndConcatenate, allowing
negative values that later get cast to size_t and wrap; before using or
forwarding silence_duration, validate it (from the value returned by
config.GetExtraFloat) and reject or clamp negative inputs: if silence_duration <
0 log an error with the problematic value (similar to the max_len check) and
return {} (or set silence_duration = 0.0f) so ProcessChunksAndConcatenate and
the downstream conversion to size_t cannot wrap; update the handling around the
silence_duration variable and any early-return path to prevent negative values
reaching ProcessChunksAndConcatenate.
- Around line 155-167: The PadTextInputs function currently copies text_ids_raw
and text_mask_raw into fixed-size buffers without bounds checks; add explicit
validation against kFixedTextLen and the expected batch size: if
text_ids_raw.size() > kFixedTextLen or actual_len > kFixedTextLen (or
text_mask_raw.size() < actual_len) then fail early (return an error/throw or set
a failure flag) or clamp/truncate inputs to kFixedTextLen before performing the
std::copy; update PadTextInputs to validate text_ids_raw.size(),
text_mask_raw.size(), and actual_len, and only copy up to kFixedTextLen (use
std::min on lengths) to prevent overruns when filling out.text_ids and
out.text_mask.
- Around line 451-455: The callback loop currently ignores the
GeneratedAudioCallback return value, so modify the code around the call to
callback (the invocation that passes chunk_result.samples.data(),
chunk_result.samples.size(), progress) to capture its integer return and stop
further synthesis when it returns 0; i.e., call the callback, store its result
in a local int (or auto) variable, and if that value equals 0, break out of the
chunk-processing loop (or otherwise abort generation) to honor the API contract
and cancel remaining chunks.
---
Outside diff comments:
In `@sherpa-onnx/csrc/CMakeLists.txt`:
- Around line 213-219: The Axera SuperTonic source files are being added
whenever SHERPA_ONNX_ENABLE_AXERA is true even if TTS is disabled; wrap the
list(APPEND sources ...) that adds ./axera/ax-engine-guard.cc,
./axera/offline-sense-voice-model-axera.cc,
./axera/offline-tts-supertonic-model-axera.cc,
./axera/offline-tts-supertonic-impl-axera.cc, and ./axera/utils.cc in a
condition that requires both SHERPA_ONNX_ENABLE_AXERA and SHERPA_ONNX_ENABLE_TTS
to be ON (i.e., gate these translation units on TTS as well) so they are only
appended when both features are enabled.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 69489a77-6c49-4859-b63f-035d2ee7296f
📒 Files selected for processing (6)
sherpa-onnx/csrc/CMakeLists.txtsherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.ccsherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.hsherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.ccsherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.hsherpa-onnx/csrc/offline-tts-impl.cc
| PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw, | ||
| const std::vector<float> &text_mask_raw, | ||
| int64_t actual_len) { | ||
| PaddedTextInputs out; | ||
| out.actual_len = actual_len; | ||
| out.text_ids.assign(kFixedBatch * kFixedTextLen, 0); | ||
| std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin()); | ||
| out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f); | ||
| std::copy(text_mask_raw.begin(), | ||
| text_mask_raw.begin() + | ||
| std::min(static_cast<size_t>(actual_len), text_mask_raw.size()), | ||
| out.text_mask.begin()); | ||
| return out; |
There was a problem hiding this comment.
Reject tokenized text longer than the fixed 320-token window.
These std::copy calls assume the tokenizer output always fits kFixedTextLen, but nothing enforces that before the copy. Any chunk that expands past 320 IDs will overrun both vectors and corrupt memory before inference starts.
Proposed fix
PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw,
const std::vector<float> &text_mask_raw,
int64_t actual_len) {
PaddedTextInputs out;
+ if (actual_len > kFixedTextLen ||
+ text_ids_raw.size() > static_cast<size_t>(kFixedTextLen) ||
+ text_mask_raw.size() > static_cast<size_t>(kFixedTextLen)) {
+ SHERPA_ONNX_LOGE("Tokenized text length exceeds fixed Axera limit %d",
+ kFixedTextLen);
+ SHERPA_ONNX_EXIT(-1);
+ }
out.actual_len = actual_len;
out.text_ids.assign(kFixedBatch * kFixedTextLen, 0);
std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin());
out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 155
- 167, The PadTextInputs function currently copies text_ids_raw and
text_mask_raw into fixed-size buffers without bounds checks; add explicit
validation against kFixedTextLen and the expected batch size: if
text_ids_raw.size() > kFixedTextLen or actual_len > kFixedTextLen (or
text_mask_raw.size() < actual_len) then fail early (return an error/throw or set
a failure flag) or clamp/truncate inputs to kFixedTextLen before performing the
std::copy; update PadTextInputs to validate text_ids_raw.size(),
text_mask_raw.size(), and actual_len, and only copy up to kFixedTextLen (use
std::min on lengths) to prevent overruns when filling out.text_ids and
out.text_mask.
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | ||
|
|
||
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | ||
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | ||
| out.actual_latent_len; | ||
| gen.Fill(out.xt_flat.data(), actual_noise_size); | ||
|
|
||
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | ||
| for (int i = 0; i < out.actual_latent_len; ++i) { | ||
| out.latent_mask_flat[i] = 1.0f; | ||
| } | ||
|
|
||
| // Apply mask | ||
| for (int b = 0; b < kFixedBatch; ++b) { | ||
| for (int d = 0; d < latent_dim; ++d) { | ||
| for (int t = 0; t < kFixedLatentLen; ++t) { | ||
| size_t idx = static_cast<size_t>(b) * latent_dim * kFixedLatentLen + | ||
| d * kFixedLatentLen + t; | ||
| out.xt_flat[idx] *= out.latent_mask_flat[b * kFixedLatentLen + t]; | ||
| } | ||
| } |
There was a problem hiding this comment.
Fail fast when the predicted latent length exceeds 300 frames.
xt_flat and latent_mask_flat are sized for kFixedLatentLen, but actual_latent_len is unbounded here. Once it grows past 300, gen.Fill() and the mask loop both write past the end of their buffers.
Proposed fix
int32_t chunk_size = base_chunk_size * chunk_compress_factor;
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
+ if (out.actual_latent_len > kFixedLatentLen) {
+ SHERPA_ONNX_LOGE(
+ "Predicted latent length %d exceeds fixed Axera limit %d",
+ out.actual_latent_len, kFixedLatentLen);
+ SHERPA_ONNX_EXIT(-1);
+ }
out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } | |
| // Apply mask | |
| for (int b = 0; b < kFixedBatch; ++b) { | |
| for (int d = 0; d < latent_dim; ++d) { | |
| for (int t = 0; t < kFixedLatentLen; ++t) { | |
| size_t idx = static_cast<size_t>(b) * latent_dim * kFixedLatentLen + | |
| d * kFixedLatentLen + t; | |
| out.xt_flat[idx] *= out.latent_mask_flat[b * kFixedLatentLen + t]; | |
| } | |
| } | |
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| if (out.actual_latent_len > kFixedLatentLen) { | |
| SHERPA_ONNX_LOGE( | |
| "Predicted latent length %d exceeds fixed Axera limit %d", | |
| out.actual_latent_len, kFixedLatentLen); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } | |
| // Apply mask | |
| for (int b = 0; b < kFixedBatch; ++b) { | |
| for (int d = 0; d < latent_dim; ++d) { | |
| for (int t = 0; t < kFixedLatentLen; ++t) { | |
| size_t idx = static_cast<size_t>(b) * latent_dim * kFixedLatentLen + | |
| d * kFixedLatentLen + t; | |
| out.xt_flat[idx] *= out.latent_mask_flat[b * kFixedLatentLen + t]; | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 182
- 205, After computing out.actual_latent_len, immediately check if it exceeds
kFixedLatentLen (300) and fail fast: log an error and return/throw before
calling gen.Fill or any mask loops to avoid buffer overruns; ensure the check
references out.actual_latent_len, kFixedLatentLen, gen.Fill, out.xt_flat and
out.latent_mask_flat and prevents further execution if actual_latent_len >
kFixedLatentLen so no writes occur past the fixed-size buffers.
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | ||
| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } | ||
| auto text_chunks = ChunkText(text_single, max_len); | ||
| return ProcessChunksAndConcatenate(text_chunks, lang, sid, num_steps, | ||
| speed, silence_duration, seed, | ||
| callback); |
There was a problem hiding this comment.
Reject negative silence_duration before it is cast to size_t.
A negative value reaches ProcessChunksAndConcatenate(), where it is converted to size_t silence_len. That wraps into a huge insertion count and can trigger massive allocations.
Proposed fix
float silence_duration = config.GetExtraFloat("silence_duration", 0.3f);
+ if (silence_duration < 0) {
+ SHERPA_ONNX_LOGE("silence_duration must be >= 0. Given: %f",
+ silence_duration);
+ return {};
+ }
size_t max_len =🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 302
- 314, The code currently reads silence_duration via config.GetExtraFloat and
passes it to ProcessChunksAndConcatenate, allowing negative values that later
get cast to size_t and wrap; before using or forwarding silence_duration,
validate it (from the value returned by config.GetExtraFloat) and reject or
clamp negative inputs: if silence_duration < 0 log an error with the problematic
value (similar to the max_len check) and return {} (or set silence_duration =
0.0f) so ProcessChunksAndConcatenate and the downstream conversion to size_t
cannot wrap; update the handling around the silence_duration variable and any
early-return path to prevent negative values reaching
ProcessChunksAndConcatenate.
| int32_t latent_dim = cfg.ttl.latent_dim * cfg.ttl.chunk_compress_factor; | ||
| auto latent_result = | ||
| SampleNoisyLatentFixed(duration, cfg.ae.sample_rate, | ||
| cfg.ae.base_chunk_size, | ||
| cfg.ttl.chunk_compress_factor, latent_dim, gen); |
There was a problem hiding this comment.
Use cfg.ttl.latent_dim directly for the latent channel count.
chunk_compress_factor is already consumed when converting waveform duration into latent length. Multiplying it into the channel dimension here changes the tensor shape and can make RunVectorEstimator() / RunVocoder() fail their fixed-size input checks.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 380
- 384, The code multiplies cfg.ttl.latent_dim by cfg.ttl.chunk_compress_factor
when computing latent_dim before calling SampleNoisyLatentFixed, which
incorrectly changes the latent channel count and causes
RunVectorEstimator()/RunVocoder() fixed-size checks to fail; change latent_dim
to use cfg.ttl.latent_dim directly (remove the multiplication by
cfg.ttl.chunk_compress_factor) so SampleNoisyLatentFixed() is called with the
correct channel count and the downstream RunVectorEstimator/RunVocoder tensor
shapes match expectations.
| if (callback) { | ||
| float progress = | ||
| static_cast<float>(i + 1) / static_cast<float>(num_chunks); | ||
| callback(chunk_result.samples.data(), chunk_result.samples.size(), | ||
| progress); |
There was a problem hiding this comment.
Honor the callback’s stop signal.
GeneratedAudioCallback is documented to stop generation when it returns 0, but this implementation always keeps synthesizing the remaining chunks. That breaks the API contract and defeats cancellation.
Proposed fix
if (callback) {
float progress =
static_cast<float>(i + 1) / static_cast<float>(num_chunks);
- callback(chunk_result.samples.data(), chunk_result.samples.size(),
- progress);
+ if (callback(chunk_result.samples.data(), chunk_result.samples.size(),
+ progress) == 0) {
+ chunk_samples.push_back(std::move(chunk_result.samples));
+ break;
+ }
}
chunk_samples.push_back(std::move(chunk_result.samples));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 451
- 455, The callback loop currently ignores the GeneratedAudioCallback return
value, so modify the code around the call to callback (the invocation that
passes chunk_result.samples.data(), chunk_result.samples.size(), progress) to
capture its integer return and stop further synthesis when it returns 0; i.e.,
call the callback, store its result in a local int (or auto) variable, and if
that value equals 0, break out of the chunk-processing loop (or otherwise abort
generation) to honor the API contract and cancel remaining chunks.
1 participant
model: https://huggingface.co/Abandon-ht/supertonic-3-axmodel
Summary by CodeRabbit