Add Axcl backend support for Supertonic TTS - #3637
Open
Abandon-ht wants to merge 2 commits into
Open
Abandon-ht wants to merge 2 commits into
Abandon-ht wants to merge 2 commits into
Conversation
dosubot
Bot
added
the
size:XXL
📝 WalkthroughWalkthroughThis PR adds offline TTS support for the Supertonic model targeting AXERA and AXCL hardware backends. It includes model loading/inference pipelines for both backends, voice-style parsing, multi-step generation (duration prediction → text encoding → iterative denoising → vocoding), text chunking with speaker selection, and factory wiring to automatically instantiate the correct implementation by provider. ChangesSupertonic TTS Backend Integration
Sequence Diagram(s)sequenceDiagram
participant Client as TTS Client
participant Factory as OfflineTtsImpl::Create
participant AxclModel as AxclSupertonicModel
participant AxclImpl as AxclSupertonicImpl
participant Config as Voice Style
Client->>Factory: Create(config, "axcl" provider)
Factory->>AxclImpl: new OfflineTtsSupertonicImplAxcl(config)
AxclImpl->>Config: InitVoiceStyle(binary buffer)
Config-->>AxclImpl: Parse TTL/DP tensors, num_speakers
Client->>AxclImpl: Generate(text, GenerationConfig)
AxclImpl->>AxclImpl: Chunk text by language max_len
loop for each chunk
AxclImpl->>AxclModel: TextProcessor.Process(text)
AxclModel-->>AxclImpl: text_ids, text_mask
AxclImpl->>AxclModel: RunDurationPredictor(text_ids, style_dp, text_mask)
AxclModel-->>AxclImpl: duration values
AxclImpl->>AxclModel: RunTextEncoder(text_ids, style_ttl, text_mask)
AxclModel-->>AxclImpl: text_embedding
AxclImpl->>AxclImpl: SampleNoisyLatent(duration × sample_rate)
loop num_steps iterations
AxclImpl->>AxclModel: RunVectorEstimator(noisy_latent, current_step, ...)
AxclModel-->>AxclImpl: denoised_latent
AxclImpl->>AxclImpl: Update latent in-place
end
AxclImpl->>AxclModel: RunVocoder(latent)
AxclModel-->>AxclImpl: waveform samples
AxclImpl->>AxclImpl: Trim to duration, callback progress
end
AxclImpl->>AxclImpl: Concatenate chunks with silence
AxclImpl-->>Client: GeneratedAudio(samples, sample_rate)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for the Supertonic TTS model on Axera and AXCL hardware accelerators, adding the corresponding model and implementation files and updating the build configuration and TTS factory. The code review identified several critical and medium-severity issues across both implementations. These include potential buffer overflows in PadTextInputs and SampleNoisyLatentFixed due to missing bounds checks on fixed-size buffers, undefined behavior when casting negative silence_duration values, potential crashes in JSON parsing if the input is not validated as an object, and potential out-of-bounds access in RunModel if the model has no outputs.
Comment on lines
+155
to
+168
| PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw, | ||
| const std::vector<float> &text_mask_raw, | ||
| int64_t actual_len) { | ||
| PaddedTextInputs out; | ||
| out.actual_len = actual_len; | ||
| out.text_ids.assign(kFixedBatch * kFixedTextLen, 0); | ||
| std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin()); | ||
| out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f); | ||
| std::copy(text_mask_raw.begin(), | ||
| text_mask_raw.begin() + | ||
| std::min(static_cast<size_t>(actual_len), text_mask_raw.size()), | ||
| out.text_mask.begin()); | ||
| return out; | ||
| } |
There was a problem hiding this comment.
In PadTextInputs, if text_ids_raw.size() or text_mask_raw.size() exceeds kFixedTextLen (320), std::copy will write out of bounds of out.text_ids or out.text_mask, leading to a critical buffer overflow. We must clamp the copy range to kFixedTextLen.
PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw,
const std::vector<float> &text_mask_raw,
int64_t actual_len) {
PaddedTextInputs out;
out.actual_len = actual_len;
out.text_ids.assign(kFixedBatch * kFixedTextLen, 0);
std::copy(text_ids_raw.begin(),
text_ids_raw.begin() + std::min(text_ids_raw.size(), static_cast<size_t>(kFixedTextLen)),
out.text_ids.begin());
out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f);
std::copy(text_mask_raw.begin(),
text_mask_raw.begin() +
std::min({static_cast<size_t>(actual_len), text_mask_raw.size(), static_cast<size_t>(kFixedTextLen)}),
out.text_mask.begin());
return out;
}
Comment on lines
+155
to
+168
| PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw, | ||
| const std::vector<float> &text_mask_raw, | ||
| int64_t actual_len) { | ||
| PaddedTextInputs out; | ||
| out.actual_len = actual_len; | ||
| out.text_ids.assign(kFixedBatch * kFixedTextLen, 0); | ||
| std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin()); | ||
| out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f); | ||
| std::copy(text_mask_raw.begin(), | ||
| text_mask_raw.begin() + | ||
| std::min(static_cast<size_t>(actual_len), text_mask_raw.size()), | ||
| out.text_mask.begin()); | ||
| return out; | ||
| } |
There was a problem hiding this comment.
In PadTextInputs, if text_ids_raw.size() or text_mask_raw.size() exceeds kFixedTextLen (320), std::copy will write out of bounds of out.text_ids or out.text_mask, leading to a critical buffer overflow. We must clamp the copy range to kFixedTextLen.
PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw,
const std::vector<float> &text_mask_raw,
int64_t actual_len) {
PaddedTextInputs out;
out.actual_len = actual_len;
out.text_ids.assign(kFixedBatch * kFixedTextLen, 0);
std::copy(text_ids_raw.begin(),
text_ids_raw.begin() + std::min(text_ids_raw.size(), static_cast<size_t>(kFixedTextLen)),
out.text_ids.begin());
out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f);
std::copy(text_mask_raw.begin(),
text_mask_raw.begin() +
std::min({static_cast<size_t>(actual_len), text_mask_raw.size(), static_cast<size_t>(kFixedTextLen)}),
out.text_mask.begin());
return out;
}
Comment on lines
+181
to
+195
| NoisyLatentResult out; | ||
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | ||
|
|
||
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | ||
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | ||
| out.actual_latent_len; | ||
| gen.Fill(out.xt_flat.data(), actual_noise_size); | ||
|
|
||
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | ||
| for (int i = 0; i < out.actual_latent_len; ++i) { | ||
| out.latent_mask_flat[i] = 1.0f; | ||
| } |
There was a problem hiding this comment.
In SampleNoisyLatentFixed, if out.actual_latent_len exceeds kFixedLatentLen (300), actual_noise_size will be larger than out.xt_flat.size(), causing a buffer overflow in gen.Fill. Additionally, the loop initializing out.latent_mask_flat will write out of bounds. We must clamp out.actual_latent_len to kFixedLatentLen.
NoisyLatentResult out;
int32_t wav_len = static_cast<int32_t>(duration * sample_rate);
if (wav_len < 1) wav_len = 1;
int32_t chunk_size = base_chunk_size * chunk_compress_factor;
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
if (out.actual_latent_len > kFixedLatentLen) {
out.actual_latent_len = kFixedLatentLen;
}
out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f);
size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim *
out.actual_latent_len;
gen.Fill(out.xt_flat.data(), actual_noise_size);
out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f);
for (int i = 0; i < out.actual_latent_len; ++i) {
out.latent_mask_flat[i] = 1.0f;
}
Comment on lines
+181
to
+195
| NoisyLatentResult out; | ||
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | ||
|
|
||
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | ||
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | ||
| out.actual_latent_len; | ||
| gen.Fill(out.xt_flat.data(), actual_noise_size); | ||
|
|
||
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | ||
| for (int i = 0; i < out.actual_latent_len; ++i) { | ||
| out.latent_mask_flat[i] = 1.0f; | ||
| } |
There was a problem hiding this comment.
In SampleNoisyLatentFixed, if out.actual_latent_len exceeds kFixedLatentLen (300), actual_noise_size will be larger than out.xt_flat.size(), causing a buffer overflow in gen.Fill. Additionally, the loop initializing out.latent_mask_flat will write out of bounds. We must clamp out.actual_latent_len to kFixedLatentLen.
NoisyLatentResult out;
int32_t wav_len = static_cast<int32_t>(duration * sample_rate);
if (wav_len < 1) wav_len = 1;
int32_t chunk_size = base_chunk_size * chunk_compress_factor;
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
if (out.actual_latent_len > kFixedLatentLen) {
out.actual_latent_len = kFixedLatentLen;
}
out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f);
size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim *
out.actual_latent_len;
gen.Fill(out.xt_flat.data(), actual_noise_size);
out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f);
for (int i = 0; i < out.actual_latent_len; ++i) {
out.latent_mask_flat[i] = 1.0f;
}
Comment on lines
+302
to
+310
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | ||
| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
If silence_duration is negative, casting it to size_t in ProcessChunksAndConcatenate is undefined behavior and can result in a huge memory allocation. We should validate that silence_duration is non-negative. Additionally, we should clamp max_len to kFixedTextLen to ensure that chunk sizes do not exceed the model's fixed input capacity.
float silence_duration = config.GetExtraFloat(
Comment on lines
+302
to
+310
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | ||
| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
If silence_duration is negative, casting it to size_t in ProcessChunksAndConcatenate is undefined behavior and can result in a huge memory allocation. We should validate that silence_duration is non-negative. Additionally, we should clamp max_len to kFixedTextLen to ensure that chunk sizes do not exceed the model's fixed input capacity.
float silence_duration = config.GetExtraFloat(
Comment on lines
+169
to
+173
| void ParseConfig(const json &j) { | ||
| if (j.find("ae") == j.end() || j.find("ttl") == j.end()) { | ||
| SHERPA_ONNX_LOGE("Invalid config file: missing 'ae' or 'ttl' section"); | ||
| SHERPA_ONNX_EXIT(-1); | ||
| } |
There was a problem hiding this comment.
In ParseConfig, we should validate that the root JSON j is an object, and that ae and ttl are objects before calling find or accessing keys. If they are not objects, calling find will throw a nlohmann::detail::type_error exception and crash the application.
void ParseConfig(const json &j) {
if (!j.is_object()) {
SHERPA_ONNX_LOGE(
Comment on lines
+177
to
+181
| void ParseConfig(const json &j) { | ||
| if (j.find("ae") == j.end() || j.find("ttl") == j.end()) { | ||
| SHERPA_ONNX_LOGE("Invalid config file: missing 'ae' or 'ttl' section"); | ||
| SHERPA_ONNX_EXIT(-1); | ||
| } |
There was a problem hiding this comment.
In ParseConfig, we should validate that the root JSON j is an object, and that ae and ttl are objects before calling find or accessing keys. If they are not objects, calling find will throw a nlohmann::detail::type_error exception and crash the application.
void ParseConfig(const json &j) {
if (!j.is_object()) {
SHERPA_ONNX_LOGE(
Comment on lines
+168
to
+169
|
|
||
| const auto &out_meta = model->io_info->pOutputs[0]; |
There was a problem hiding this comment.
In RunModel, we should check if model->io_info->nOutputSize > 0 before accessing pOutputs[0] and pOutputs[0]. If the model has no outputs, accessing index 0 will result in an out-of-bounds access and undefined behavior.
if (model->io_info->nOutputSize == 0) {
SHERPA_ONNX_LOGE("%s: Model has no outputs", name);
SHERPA_ONNX_EXIT(-1);
}
const auto &out_meta = model->io_info->pOutputs[0];
auto &out_buf = model->io_data.pOutputs[0];![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.cc`:
- Around line 302-314: The code reads silence_duration via config.GetExtraFloat
but doesn't reject negative values, which later get cast to size_t and cause
huge allocations; add a validation after obtaining silence_duration (the
silence_duration local variable) to check if it is < 0, log an error via
SHERPA_ONNX_LOGE (including the invalid value) and return an empty result (same
style as the max_len check) before calling ProcessChunksAndConcatenate so
downstream conversion to size_t/silence_len is prevented.
- Around line 182-195: The computed out.actual_latent_len (from duration) can
exceed the fixed buffer kFixedLatentLen causing gen.Fill() and the mask loop to
write out of bounds; add a guard after computing out.actual_latent_len to check
if it > kFixedLatentLen and either reject/return an error or clamp it to
kFixedLatentLen, then use that bounded value when computing actual_noise_size
for gen.Fill() and when iterating to set out.latent_mask_flat (and ensure
xt_flat/latent_mask_flat sizing uses kFixedLatentLen), referencing
actual_latent_len, kFixedLatentLen, gen.Fill, out.xt_flat and
out.latent_mask_flat to locate the changes.
- Around line 493-513: Add strict per-speaker-dimension checks after the
existing rank and speaker-count validation: verify that style.ttl_shape equals
[N,50,256] and style.dp_shape equals [N,8,16] (where N is style.ttl_shape[0]),
and if not call SHERPA_ONNX_LOGE with the actual shapes and
SHERPA_ONNX_EXIT(-1). Update the block that currently reads style.ttl_shape,
style.dp_shape, num_speakers and num_speakers_ so it validates exact dims for
ttl_shape[1]==50 && ttl_shape[2]==256 and dp_shape[1]==8 && dp_shape[2]==16
before assigning num_speakers_ and moving full_style_.
- Around line 340-349: PadTextInputs currently assumes inputs fit into fixed
[1,320] buffers but text_ids_raw/text_mask_raw can be longer; before calling
PadTextInputs (near the check comparing text_seq_len and text_mask_len) add a
guard that ensures text_seq_len <= 320 (or re-chunk/reject the input) and return
an error (or split into smaller chunks) if it exceeds 320 to avoid buffer
overruns; reference the symbols text_ids_raw, text_mask_raw, text_seq_len, and
the PadTextInputs call when implementing the early-fail or re-chunking logic.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc`:
- Around line 182-195: Clamp out.actual_latent_len to not exceed kFixedLatentLen
before using it to size/fill buffers: compute wav_len and chunk_size as before,
set out.actual_latent_len = min(calculated_value, kFixedLatentLen) (and keep the
existing minimum-of-1 guard), then use that clamped value when computing
actual_noise_size for gen.Fill(out.xt_flat.data(), actual_noise_size) and when
iterating to set out.latent_mask_flat; this prevents gen.Fill and the mask loop
from overrunning xt_flat and latent_mask_flat which are sized by
kFixedLatentLen.
- Around line 302-314: Validate the signed generation extras before casting to
unsigned types: read max_len and silence_duration using
config.GetExtraInt/GetExtraFloat into signed temporaries, check they are within
acceptable ranges (e.g., max_len > 0 and within a sane upper bound,
silence_duration >= 0 and <= some max), log and return on invalid values, then
cast to size_t only after validation and use those validated values in ChunkText
and ProcessChunksAndConcatenate; update references to max_len and
silence_duration in this block to use the validated/capped values.
- Around line 155-166: In PadTextInputs, guard against tokenized inputs longer
than the model window (kFixedTextLen): before copying, check text_ids_raw.size()
and actual_len (and text_mask_raw length) against kFixedTextLen and either
truncate/cap the copy lengths to kFixedTextLen or return an error/indicator when
inputs exceed kFixedTextLen; ensure the std::copy calls that write into
out.text_ids and out.text_mask only copy up to kFixedTextLen elements (or set
out.actual_len to the capped value) so no writes can exceed the buffers sized
for kFixedTextLen.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.cc`:
- Around line 169-173: Validate the output metadata before copying: ensure
model->io_info and model->io_info->pOutputs exist and contain at least one
element, confirm out_meta.nSize is non‑zero and divisible by sizeof(float) (so
out_elems = out_meta.nSize / sizeof(float) is integral and >0), and verify
out_buf.pVirAddr is non‑null and that any available buffer length matches nSize;
if any check fails (e.g., in the block using out_meta, out_buf, out_elems, and
memcpy), return an error or abort gracefully instead of performing the memcpy to
avoid out‑of‑bounds writes.
In `@sherpa-onnx/csrc/offline-tts-impl.cc`:
- Around line 65-74: When config.model.provider is explicitly set to "axera" or
"axcl" but the corresponding build flag (SHERPA_ONNX_ENABLE_AXERA /
SHERPA_ONNX_ENABLE_AXCL) is disabled, add a warning log in the Create factory
notifying the user that the requested provider is unavailable and the code is
falling back to the default; update the blocks around the checks that currently
return OfflineTtsSupertonicImplAxera and OfflineTtsSupertonicImplAxcl so that if
the provider string matches but the feature flag branch is not compiled in you
emit a clear warning mentioning config.model.provider and the missing flag
before proceeding to the fallback, and apply the same change to the second
templated Create factory (the other Create implementation at lines ~100-109).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 75cf2686-438b-4606-8ff7-652e94989a00
📒 Files selected for processing (10)
sherpa-onnx/csrc/CMakeLists.txtsherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.ccsherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.hsherpa-onnx/csrc/axcl/offline-tts-supertonic-model-axcl.ccsherpa-onnx/csrc/axcl/offline-tts-supertonic-model-axcl.hsherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.ccsherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.hsherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.ccsherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.hsherpa-onnx/csrc/offline-tts-impl.cc
Comment on lines
+182
to
+195
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | ||
|
|
||
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | ||
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | ||
| out.actual_latent_len; | ||
| gen.Fill(out.xt_flat.data(), actual_noise_size); | ||
|
|
||
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | ||
| for (int i = 0; i < out.actual_latent_len; ++i) { | ||
| out.latent_mask_flat[i] = 1.0f; | ||
| } |
There was a problem hiding this comment.
Reject durations that exceed the 300-step latent buffer.
actual_latent_len comes from predicted duration and can exceed kFixedLatentLen. When that happens, gen.Fill() and the mask loop both write past xt_flat / latent_mask_flat. Long chunks or very slow speed can trigger this.
Suggested fix
int32_t chunk_size = base_chunk_size * chunk_compress_factor;
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
+ if (out.actual_latent_len > kFixedLatentLen) {
+ SHERPA_ONNX_LOGE("Latent length %d exceeds fixed limit %d",
+ out.actual_latent_len, kFixedLatentLen);
+ SHERPA_ONNX_EXIT(-1);
+ }
out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } | |
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| if (out.actual_latent_len > kFixedLatentLen) { | |
| SHERPA_ONNX_LOGE("Latent length %d exceeds fixed limit %d", | |
| out.actual_latent_len, kFixedLatentLen); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.cc` around lines 182 -
195, The computed out.actual_latent_len (from duration) can exceed the fixed
buffer kFixedLatentLen causing gen.Fill() and the mask loop to write out of
bounds; add a guard after computing out.actual_latent_len to check if it >
kFixedLatentLen and either reject/return an error or clamp it to
kFixedLatentLen, then use that bounded value when computing actual_noise_size
for gen.Fill() and when iterating to set out.latent_mask_flat (and ensure
xt_flat/latent_mask_flat sizing uses kFixedLatentLen), referencing
actual_latent_len, kFixedLatentLen, gen.Fill, out.xt_flat and
out.latent_mask_flat to locate the changes.
Comment on lines
+302
to
+314
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | ||
| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } | ||
| auto text_chunks = ChunkText(text_single, max_len); | ||
| return ProcessChunksAndConcatenate(text_chunks, lang, sid, num_steps, | ||
| speed, silence_duration, seed, | ||
| callback); |
There was a problem hiding this comment.
Validate silence_duration before converting it to size_t.
Negative values are accepted here and later cast to size_t for silence_len, which turns into a huge allocation request during concatenation. Reject < 0 up front.
Suggested fix
float silence_duration = config.GetExtraFloat("silence_duration", 0.3f);
+ if (silence_duration < 0) {
+ SHERPA_ONNX_LOGE("silence_duration must be >= 0. Given: %f",
+ silence_duration);
+ return {};
+ }
size_t max_len =📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | |
| size_t max_len = | |
| (lang == "ko" || lang == "ja") | |
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | |
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | |
| if (max_len == 0) { | |
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | |
| return {}; | |
| } | |
| auto text_chunks = ChunkText(text_single, max_len); | |
| return ProcessChunksAndConcatenate(text_chunks, lang, sid, num_steps, | |
| speed, silence_duration, seed, | |
| callback); | |
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | |
| if (silence_duration < 0) { | |
| SHERPA_ONNX_LOGE("silence_duration must be >= 0. Given: %f", | |
| silence_duration); | |
| return {}; | |
| } | |
| size_t max_len = | |
| (lang == "ko" || lang == "ja") | |
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | |
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | |
| if (max_len == 0) { | |
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | |
| return {}; | |
| } | |
| auto text_chunks = ChunkText(text_single, max_len); | |
| return ProcessChunksAndConcatenate(text_chunks, lang, sid, num_steps, | |
| speed, silence_duration, seed, | |
| callback); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.cc` around lines 302 -
314, The code reads silence_duration via config.GetExtraFloat but doesn't reject
negative values, which later get cast to size_t and cause huge allocations; add
a validation after obtaining silence_duration (the silence_duration local
variable) to check if it is < 0, log an error via SHERPA_ONNX_LOGE (including
the invalid value) and return an empty result (same style as the max_len check)
before calling ProcessChunksAndConcatenate so downstream conversion to
size_t/silence_len is prevented.
Comment on lines
+340
to
+349
| int64_t text_seq_len = static_cast<int64_t>(text_ids_raw.size()); | ||
| int64_t text_mask_len = text_mask_shape[2]; | ||
| if (text_seq_len != text_mask_len) { | ||
| SHERPA_ONNX_LOGE("Text sequence length mismatch: text_ids=%" PRId64 | ||
| ", text_mask=%" PRId64 ". Text: \"%s\"", | ||
| text_seq_len, text_mask_len, text.c_str()); | ||
| return {}; | ||
| } | ||
|
|
||
| auto padded = PadTextInputs(text_ids_raw, text_mask_raw, text_seq_len); |
There was a problem hiding this comment.
Guard tokenized chunks against the fixed 320-token limit.
PadTextInputs() writes into fixed [1,320] buffers, but nothing here rejects text_ids_raw / text_mask_raw longer than 320. A chunk that expands past that after normalization/tokenization will overrun both destinations and corrupt memory. Fail early or re-chunk on token count before padding.
Suggested fix
int64_t text_seq_len = static_cast<int64_t>(text_ids_raw.size());
int64_t text_mask_len = text_mask_shape[2];
+ if (text_seq_len > kFixedTextLen || text_mask_len > kFixedTextLen ||
+ text_mask_raw.size() > static_cast<size_t>(kFixedTextLen)) {
+ SHERPA_ONNX_LOGE(
+ "Tokenized text exceeds fixed length %d: text_ids=%" PRId64
+ ", text_mask=%" PRId64 ". Text: \"%s\"",
+ kFixedTextLen, text_seq_len, text_mask_len, text.c_str());
+ return {};
+ }
if (text_seq_len != text_mask_len) {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| int64_t text_seq_len = static_cast<int64_t>(text_ids_raw.size()); | |
| int64_t text_mask_len = text_mask_shape[2]; | |
| if (text_seq_len != text_mask_len) { | |
| SHERPA_ONNX_LOGE("Text sequence length mismatch: text_ids=%" PRId64 | |
| ", text_mask=%" PRId64 ". Text: \"%s\"", | |
| text_seq_len, text_mask_len, text.c_str()); | |
| return {}; | |
| } | |
| auto padded = PadTextInputs(text_ids_raw, text_mask_raw, text_seq_len); | |
| int64_t text_seq_len = static_cast<int64_t>(text_ids_raw.size()); | |
| int64_t text_mask_len = text_mask_shape[2]; | |
| if (text_seq_len > kFixedTextLen || text_mask_len > kFixedTextLen || | |
| text_mask_raw.size() > static_cast<size_t>(kFixedTextLen)) { | |
| SHERPA_ONNX_LOGE( | |
| "Tokenized text exceeds fixed length %d: text_ids=%" PRId64 | |
| ", text_mask=%" PRId64 ". Text: \"%s\"", | |
| kFixedTextLen, text_seq_len, text_mask_len, text.c_str()); | |
| return {}; | |
| } | |
| if (text_seq_len != text_mask_len) { | |
| SHERPA_ONNX_LOGE("Text sequence length mismatch: text_ids=%" PRId64 | |
| ", text_mask=%" PRId64 ". Text: \"%s\"", | |
| text_seq_len, text_mask_len, text.c_str()); | |
| return {}; | |
| } | |
| auto padded = PadTextInputs(text_ids_raw, text_mask_raw, text_seq_len); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.cc` around lines 340 -
349, PadTextInputs currently assumes inputs fit into fixed [1,320] buffers but
text_ids_raw/text_mask_raw can be longer; before calling PadTextInputs (near the
check comparing text_seq_len and text_mask_len) add a guard that ensures
text_seq_len <= 320 (or re-chunk/reject the input) and return an error (or split
into smaller chunks) if it exceeds 320 to avoid buffer overruns; reference the
symbols text_ids_raw, text_mask_raw, text_seq_len, and the PadTextInputs call
when implementing the early-fail or re-chunking logic.
Comment on lines
+493
to
+513
| if (style.ttl_shape.size() != 3 || style.dp_shape.size() != 3) { | ||
| SHERPA_ONNX_LOGE( | ||
| "Invalid voice style: ttl_shape or dp_shape must have 3 dimensions"); | ||
| SHERPA_ONNX_EXIT(-1); | ||
| } | ||
| int32_t num_speakers = static_cast<int32_t>(style.ttl_shape[0]); | ||
| if (num_speakers <= 0) { | ||
| SHERPA_ONNX_LOGE( | ||
| "Invalid voice style: num_speakers must be >= 1. Given: %d", | ||
| num_speakers); | ||
| SHERPA_ONNX_EXIT(-1); | ||
| } | ||
| if (style.ttl_shape[0] != style.dp_shape[0]) { | ||
| SHERPA_ONNX_LOGE( | ||
| "Invalid voice style: ttl_shape[0] != dp_shape[0]. Given: %d != %d", | ||
| static_cast<int32_t>(style.ttl_shape[0]), | ||
| static_cast<int32_t>(style.dp_shape[0])); | ||
| SHERPA_ONNX_EXIT(-1); | ||
| } | ||
| num_speakers_ = num_speakers; | ||
| full_style_ = std::move(style); |
There was a problem hiding this comment.
Validate per-speaker style tensor shapes at load time.
This only checks rank and speaker count. If the voice-style blob has unexpected per-speaker dims, initialization succeeds and the first inference later aborts on tensor-size mismatch. Check for the exact [N,50,256] TTL and [N,8,16] DP shapes here so bad assets fail fast.
Suggested fix
if (style.ttl_shape[0] != style.dp_shape[0]) {
SHERPA_ONNX_LOGE(
"Invalid voice style: ttl_shape[0] != dp_shape[0]. Given: %d != %d",
static_cast<int32_t>(style.ttl_shape[0]),
static_cast<int32_t>(style.dp_shape[0]));
SHERPA_ONNX_EXIT(-1);
}
+ if (style.ttl_shape[1] != 50 || style.ttl_shape[2] != 256 ||
+ style.dp_shape[1] != 8 || style.dp_shape[2] != 16) {
+ SHERPA_ONNX_LOGE(
+ "Invalid voice style: expected ttl [N,50,256] and dp [N,8,16], got "
+ "[%" PRId64 ",%" PRId64 ",%" PRId64 "] and "
+ "[%" PRId64 ",%" PRId64 ",%" PRId64 "]",
+ style.ttl_shape[0], style.ttl_shape[1], style.ttl_shape[2],
+ style.dp_shape[0], style.dp_shape[1], style.dp_shape[2]);
+ SHERPA_ONNX_EXIT(-1);
+ }
num_speakers_ = num_speakers;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (style.ttl_shape.size() != 3 || style.dp_shape.size() != 3) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: ttl_shape or dp_shape must have 3 dimensions"); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| int32_t num_speakers = static_cast<int32_t>(style.ttl_shape[0]); | |
| if (num_speakers <= 0) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: num_speakers must be >= 1. Given: %d", | |
| num_speakers); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| if (style.ttl_shape[0] != style.dp_shape[0]) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: ttl_shape[0] != dp_shape[0]. Given: %d != %d", | |
| static_cast<int32_t>(style.ttl_shape[0]), | |
| static_cast<int32_t>(style.dp_shape[0])); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| num_speakers_ = num_speakers; | |
| full_style_ = std::move(style); | |
| if (style.ttl_shape.size() != 3 || style.dp_shape.size() != 3) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: ttl_shape or dp_shape must have 3 dimensions"); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| int32_t num_speakers = static_cast<int32_t>(style.ttl_shape[0]); | |
| if (num_speakers <= 0) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: num_speakers must be >= 1. Given: %d", | |
| num_speakers); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| if (style.ttl_shape[0] != style.dp_shape[0]) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: ttl_shape[0] != dp_shape[0]. Given: %d != %d", | |
| static_cast<int32_t>(style.ttl_shape[0]), | |
| static_cast<int32_t>(style.dp_shape[0])); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| if (style.ttl_shape[1] != 50 || style.ttl_shape[2] != 256 || | |
| style.dp_shape[1] != 8 || style.dp_shape[2] != 16) { | |
| SHERPA_ONNX_LOGE( | |
| "Invalid voice style: expected ttl [N,50,256] and dp [N,8,16], got " | |
| "[%" PRId64 ",%" PRId64 ",%" PRId64 "] and " | |
| "[%" PRId64 ",%" PRId64 ",%" PRId64 "]", | |
| style.ttl_shape[0], style.ttl_shape[1], style.ttl_shape[2], | |
| style.dp_shape[0], style.dp_shape[1], style.dp_shape[2]); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| num_speakers_ = num_speakers; | |
| full_style_ = std::move(style); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axcl/offline-tts-supertonic-impl-axcl.cc` around lines 493 -
513, Add strict per-speaker-dimension checks after the existing rank and
speaker-count validation: verify that style.ttl_shape equals [N,50,256] and
style.dp_shape equals [N,8,16] (where N is style.ttl_shape[0]), and if not call
SHERPA_ONNX_LOGE with the actual shapes and SHERPA_ONNX_EXIT(-1). Update the
block that currently reads style.ttl_shape, style.dp_shape, num_speakers and
num_speakers_ so it validates exact dims for ttl_shape[1]==50 &&
ttl_shape[2]==256 and dp_shape[1]==8 && dp_shape[2]==16 before assigning
num_speakers_ and moving full_style_.
Comment on lines
+155
to
+166
| PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw, | ||
| const std::vector<float> &text_mask_raw, | ||
| int64_t actual_len) { | ||
| PaddedTextInputs out; | ||
| out.actual_len = actual_len; | ||
| out.text_ids.assign(kFixedBatch * kFixedTextLen, 0); | ||
| std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin()); | ||
| out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f); | ||
| std::copy(text_mask_raw.begin(), | ||
| text_mask_raw.begin() + | ||
| std::min(static_cast<size_t>(actual_len), text_mask_raw.size()), | ||
| out.text_mask.begin()); |
There was a problem hiding this comment.
Reject tokenized inputs that exceed the fixed 320-token model window.
out.text_ids and out.text_mask are always sized for kFixedTextLen, but these std::copy calls use the processed lengths directly. If unicode processing expands a chunk past 320 tokens, this writes past both buffers.
🛡️ Suggested guard
PaddedTextInputs PadTextInputs(const std::vector<int64_t> &text_ids_raw,
const std::vector<float> &text_mask_raw,
int64_t actual_len) {
+ if (actual_len > kFixedTextLen ||
+ text_ids_raw.size() > static_cast<size_t>(kFixedTextLen) ||
+ text_mask_raw.size() > static_cast<size_t>(kFixedTextLen)) {
+ SHERPA_ONNX_LOGE(
+ "Tokenized text length exceeds fixed model input: actual_len=%" PRId64
+ ", max=%d",
+ actual_len, kFixedTextLen);
+ SHERPA_ONNX_EXIT(-1);
+ }
+
PaddedTextInputs out;
out.actual_len = actual_len;
out.text_ids.assign(kFixedBatch * kFixedTextLen, 0);
std::copy(text_ids_raw.begin(), text_ids_raw.end(), out.text_ids.begin());
out.text_mask.assign(kFixedBatch * 1 * kFixedTextLen, 0.0f);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 155
- 166, In PadTextInputs, guard against tokenized inputs longer than the model
window (kFixedTextLen): before copying, check text_ids_raw.size() and actual_len
(and text_mask_raw length) against kFixedTextLen and either truncate/cap the
copy lengths to kFixedTextLen or return an error/indicator when inputs exceed
kFixedTextLen; ensure the std::copy calls that write into out.text_ids and
out.text_mask only copy up to kFixedTextLen elements (or set out.actual_len to
the capped value) so no writes can exceed the buffers sized for kFixedTextLen.
Comment on lines
+182
to
+195
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | ||
| if (wav_len < 1) wav_len = 1; | ||
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | ||
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | ||
|
|
||
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | ||
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | ||
| out.actual_latent_len; | ||
| gen.Fill(out.xt_flat.data(), actual_noise_size); | ||
|
|
||
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | ||
| for (int i = 0; i < out.actual_latent_len; ++i) { | ||
| out.latent_mask_flat[i] = 1.0f; | ||
| } |
There was a problem hiding this comment.
Bound actual_latent_len to the fixed 300-step buffer before filling it.
actual_latent_len is derived from predicted duration, but xt_flat and latent_mask_flat stay fixed at kFixedLatentLen. Once the duration maps past 300 steps, both gen.Fill() and the mask loop overrun those buffers.
🛡️ Suggested guard
int32_t chunk_size = base_chunk_size * chunk_compress_factor;
out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size;
+ if (out.actual_latent_len > kFixedLatentLen) {
+ SHERPA_ONNX_LOGE(
+ "Predicted latent length %d exceeds fixed model limit %d",
+ out.actual_latent_len, kFixedLatentLen);
+ SHERPA_ONNX_EXIT(-1);
+ }
out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } | |
| int32_t wav_len = static_cast<int32_t>(duration * sample_rate); | |
| if (wav_len < 1) wav_len = 1; | |
| int32_t chunk_size = base_chunk_size * chunk_compress_factor; | |
| out.actual_latent_len = (wav_len + chunk_size - 1) / chunk_size; | |
| if (out.actual_latent_len > kFixedLatentLen) { | |
| SHERPA_ONNX_LOGE( | |
| "Predicted latent length %d exceeds fixed model limit %d", | |
| out.actual_latent_len, kFixedLatentLen); | |
| SHERPA_ONNX_EXIT(-1); | |
| } | |
| out.xt_flat.assign(kFixedBatch * latent_dim * kFixedLatentLen, 0.0f); | |
| size_t actual_noise_size = static_cast<size_t>(kFixedBatch) * latent_dim * | |
| out.actual_latent_len; | |
| gen.Fill(out.xt_flat.data(), actual_noise_size); | |
| out.latent_mask_flat.assign(kFixedBatch * 1 * kFixedLatentLen, 0.0f); | |
| for (int i = 0; i < out.actual_latent_len; ++i) { | |
| out.latent_mask_flat[i] = 1.0f; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 182
- 195, Clamp out.actual_latent_len to not exceed kFixedLatentLen before using it
to size/fill buffers: compute wav_len and chunk_size as before, set
out.actual_latent_len = min(calculated_value, kFixedLatentLen) (and keep the
existing minimum-of-1 guard), then use that clamped value when computing
actual_noise_size for gen.Fill(out.xt_flat.data(), actual_noise_size) and when
iterating to set out.latent_mask_flat; this prevents gen.Fill and the mask loop
from overrunning xt_flat and latent_mask_flat which are sized by
kFixedLatentLen.
Comment on lines
+302
to
+314
| float silence_duration = config.GetExtraFloat("silence_duration", 0.3f); | ||
| size_t max_len = | ||
| (lang == "ko" || lang == "ja") | ||
| ? static_cast<size_t>(config.GetExtraInt("max_len", 120)) | ||
| : static_cast<size_t>(config.GetExtraInt("max_len", 300)); | ||
| if (max_len == 0) { | ||
| SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len); | ||
| return {}; | ||
| } | ||
| auto text_chunks = ChunkText(text_single, max_len); | ||
| return ProcessChunksAndConcatenate(text_chunks, lang, sid, num_steps, | ||
| speed, silence_duration, seed, | ||
| callback); |
There was a problem hiding this comment.
Validate signed generation extras before converting them to sizes.
max_len is cast to size_t before validation, so a negative override wraps to a huge value and effectively disables chunking. silence_duration is also unchecked, and a negative value later becomes a huge size_t silence allocation.
🧩 Suggested fix
float silence_duration = config.GetExtraFloat("silence_duration", 0.3f);
- size_t max_len =
- (lang == "ko" || lang == "ja")
- ? static_cast<size_t>(config.GetExtraInt("max_len", 120))
- : static_cast<size_t>(config.GetExtraInt("max_len", 300));
- if (max_len == 0) {
- SHERPA_ONNX_LOGE("Max length must be > 0. Given: %zu", max_len);
+ if (silence_duration < 0) {
+ SHERPA_ONNX_LOGE("silence_duration must be >= 0. Given: %f",
+ silence_duration);
return {};
}
+ int32_t max_len_i =
+ (lang == "ko" || lang == "ja") ? config.GetExtraInt("max_len", 120)
+ : config.GetExtraInt("max_len", 300);
+ if (max_len_i <= 0) {
+ SHERPA_ONNX_LOGE("Max length must be > 0. Given: %d", max_len_i);
+ return {};
+ }
+ size_t max_len = static_cast<size_t>(max_len_i);
auto text_chunks = ChunkText(text_single, max_len);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-impl-axera.cc` around lines 302
- 314, Validate the signed generation extras before casting to unsigned types:
read max_len and silence_duration using config.GetExtraInt/GetExtraFloat into
signed temporaries, check they are within acceptable ranges (e.g., max_len > 0
and within a sane upper bound, silence_duration >= 0 and <= some max), log and
return on invalid values, then cast to size_t only after validation and use
those validated values in ChunkText and ProcessChunksAndConcatenate; update
references to max_len and silence_duration in this block to use the
validated/capped values.
Comment on lines
+169
to
+173
| const auto &out_meta = model->io_info->pOutputs[0]; | ||
| auto &out_buf = model->io_data.pOutputs[0]; | ||
| size_t out_elems = out_meta.nSize / sizeof(float); | ||
| std::vector<float> out(out_elems); | ||
| std::memcpy(out.data(), out_buf.pVirAddr, out_meta.nSize); |
There was a problem hiding this comment.
Validate output metadata before copying the first tensor.
This trusts pOutputs[0] and nSize blindly. A malformed or incompatible model can report zero outputs or a byte count that's not divisible by sizeof(float), which turns the memcpy here into an out-of-bounds write.
🛡️ Suggested guard
- const auto &out_meta = model->io_info->pOutputs[0];
- auto &out_buf = model->io_data.pOutputs[0];
- size_t out_elems = out_meta.nSize / sizeof(float);
+ if (model->io_info->nOutputSize < 1) {
+ SHERPA_ONNX_LOGE("%s: Model has no outputs", name);
+ SHERPA_ONNX_EXIT(-1);
+ }
+
+ const auto &out_meta = model->io_info->pOutputs[0];
+ auto &out_buf = model->io_data.pOutputs[0];
+ if (out_meta.nSize % sizeof(float) != 0) {
+ SHERPA_ONNX_LOGE("%s: Output 0 size %u is not float-aligned", name,
+ out_meta.nSize);
+ SHERPA_ONNX_EXIT(-1);
+ }
+ size_t out_elems = out_meta.nSize / sizeof(float);
std::vector<float> out(out_elems);
std::memcpy(out.data(), out_buf.pVirAddr, out_meta.nSize);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/axera/offline-tts-supertonic-model-axera.cc` around lines
169 - 173, Validate the output metadata before copying: ensure model->io_info
and model->io_info->pOutputs exist and contain at least one element, confirm
out_meta.nSize is non‑zero and divisible by sizeof(float) (so out_elems =
out_meta.nSize / sizeof(float) is integral and >0), and verify out_buf.pVirAddr
is non‑null and that any available buffer length matches nSize; if any check
fails (e.g., in the block using out_meta, out_buf, out_elems, and memcpy),
return an error or abort gracefully instead of performing the memcpy to avoid
out‑of‑bounds writes.
Comment on lines
+65
to
+74
| #if SHERPA_ONNX_ENABLE_AXERA | ||
| if (config.model.provider == "axera") { | ||
| return std::make_unique<OfflineTtsSupertonicImplAxera>(config); | ||
| } | ||
| #endif | ||
| #if SHERPA_ONNX_ENABLE_AXCL | ||
| if (config.model.provider == "axcl") { | ||
| return std::make_unique<OfflineTtsSupertonicImplAxcl>(config); | ||
| } | ||
| #endif |
There was a problem hiding this comment.
Add diagnostic logging when requested provider is unavailable.
When a user explicitly sets config.model.provider to "axera" or "axcl" but the corresponding feature flag is disabled at build time, the code silently falls back to the default implementation. This makes it difficult for users to understand why their requested backend isn't being used.
Consider adding a warning log before the fallback to help users diagnose configuration issues.
📋 Suggested enhancement
For the first Create factory (around line 64):
} else if (!config.model.supertonic.tts_json.empty()) {
`#if` SHERPA_ONNX_ENABLE_AXERA
if (config.model.provider == "axera") {
return std::make_unique<OfflineTtsSupertonicImplAxera>(config);
}
+#else
+ if (config.model.provider == "axera") {
+ SHERPA_ONNX_LOGE("Provider 'axera' requested but SHERPA_ONNX_ENABLE_AXERA not enabled. Falling back to default implementation.");
+ }
`#endif`
`#if` SHERPA_ONNX_ENABLE_AXCL
if (config.model.provider == "axcl") {
return std::make_unique<OfflineTtsSupertonicImplAxcl>(config);
}
+#else
+ if (config.model.provider == "axcl") {
+ SHERPA_ONNX_LOGE("Provider 'axcl' requested but SHERPA_ONNX_ENABLE_AXCL not enabled. Falling back to default implementation.");
+ }
`#endif`
return std::make_unique<OfflineTtsSupertonicImpl>(config);Apply the same pattern to the second templated Create factory at lines 100-109.
Also applies to: 100-109
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/csrc/offline-tts-impl.cc` around lines 65 - 74, When
config.model.provider is explicitly set to "axera" or "axcl" but the
corresponding build flag (SHERPA_ONNX_ENABLE_AXERA / SHERPA_ONNX_ENABLE_AXCL) is
disabled, add a warning log in the Create factory notifying the user that the
requested provider is unavailable and the code is falling back to the default;
update the blocks around the checks that currently return
OfflineTtsSupertonicImplAxera and OfflineTtsSupertonicImplAxcl so that if the
provider string matches but the feature flag branch is not compiled in you emit
a clear warning mentioning config.model.provider and the missing flag before
proceeding to the fallback, and apply the same change to the second templated
Create factory (the other Create implementation at lines ~100-109).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit