Conversation
The offline transducer greedy decoder already populates OfflineRecognitionResult::ys_log_probs (see csrc/offline-transducer-greedy-search-decoder.cc, lines 68-76), but the JNI binding dropped the field on the floor, so downstream Kotlin / Java callers can't compute a real per-token confidence on-device. This patch mirrors PR k2-fsa#2736 (which surfaced ysProbs on OnlineRecognizerResult for streaming) for the offline path: - jni/offline-recognizer.cc: extend the GetMethodID signature with an extra '[F', allocate NewFloatArray + SetFloatArrayRegion for result.ys_log_probs, pass into the Kotlin data-class constructor, DeleteLocalRef afterwards. - kotlin-api/OfflineRecognizer.kt: append val ysLogProbs: FloatArray to OfflineRecognizerResult. The field is empty for non-greedy / non-transducer decoders until the corresponding C++ paths are extended; the empty array is the documented fallback. Tested with Parakeet TDT 0.6B-v2 on an Android arm64-v8a Versity 9740 (Qualcomm QCS6490). Mean exp(log_prob) tracks engine confidence and reveals weak tokens that mean-only metrics hide (observed: "Call the House Supervisor" mis-heard as "Call Health Supervisor" with mean 0.91 but per-token min 0.35).
dosubot
Bot
added
the
size:S
📝 WalkthroughWalkthroughThe pull request exposes per-token log-probabilities from the offline recognizer to Java/Kotlin clients. A new ChangesPer-token log-probabilities in offline recognizer result
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the JNI and Kotlin APIs for the offline recognizer to expose per-token log-probabilities (ys_log_probs / ysLogProbs). This allows callers to implement confidence-based policies. A review comment points out a potential issue where calling SetFloatArrayRegion with a nullptr data pointer (when ys_log_probs is empty) can trigger JNI warnings or undefined behavior, suggesting a guard check to ensure the vector is not empty before copying.
| jfloatArray jys_log_probs = env->NewFloatArray(result.ys_log_probs.size()); | ||
| env->SetFloatArrayRegion(jys_log_probs, 0, result.ys_log_probs.size(), | ||
| result.ys_log_probs.data()); |
There was a problem hiding this comment.
When result.ys_log_probs is empty (which is the default for non-transducer or non-greedy decoders), result.ys_log_probs.data() may return nullptr. Calling SetFloatArrayRegion with a nullptr buffer, even with a size of 0, can trigger JNI warnings (especially with -Xcheck:jni enabled) or lead to undefined behavior/crashes on some JVM implementations.
It is safer to guard the SetFloatArrayRegion call with a check to ensure the vector is not empty.
jfloatArray jys_log_probs = env->NewFloatArray(result.ys_log_probs.size());
if (!result.ys_log_probs.empty()) {
env->SetFloatArrayRegion(jys_log_probs, 0, result.ys_log_probs.size(),
result.ys_log_probs.data());
}![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
sherpa-onnx/kotlin-api/OfflineRecognizer.kt (1)
16-19: ⚡ Quick winDefault
ysLogProbsto an empty array.This is a public data class, so making the new field mandatory forces Kotlin call sites that construct
OfflineRecognizerResultmanually to change even though the documented fallback is already “empty”. A default here keeps those source call sites working without changing the JNI path.Proposed change
- val ysLogProbs: FloatArray, + val ysLogProbs: FloatArray = floatArrayOf(),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sherpa-onnx/kotlin-api/OfflineRecognizer.kt` around lines 16 - 19, Make the public data class OfflineRecognizerResult's new field ysLogProbs optional by giving it a default empty FloatArray so existing Kotlin call sites don't break; specifically update the declaration of val ysLogProbs in OfflineRecognizerResult to have a default (e.g., FloatArray(0) or floatArrayOf()) so callers can omit the parameter while JNI consumers still receive an empty array when not populated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@sherpa-onnx/jni/offline-recognizer.cc`:
- Around line 673-682: Check that result.ys_log_probs has the same length as the
token sequence before exposing it to Kotlin: if result.ys_log_probs.size() !=
result.tokens.size() (or is zero) then create an empty jfloatArray (length 0)
and use that instead of blindly creating jys_log_probs from result.ys_log_probs;
otherwise create jys_log_probs with env->NewFloatArray and
env->SetFloatArrayRegion as currently done and pass it into NewObject (jresult)
alongside jtext, jtokens, jtimestamps, jlang, jemotion, jevent, jdurations so
the JNI path mirrors the C API guard.
---
Nitpick comments:
In `@sherpa-onnx/kotlin-api/OfflineRecognizer.kt`:
- Around line 16-19: Make the public data class OfflineRecognizerResult's new
field ysLogProbs optional by giving it a default empty FloatArray so existing
Kotlin call sites don't break; specifically update the declaration of val
ysLogProbs in OfflineRecognizerResult to have a default (e.g., FloatArray(0) or
floatArrayOf()) so callers can omit the parameter while JNI consumers still
receive an empty array when not populated.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 798d29aa-54fa-4964-b7c8-49249f277134
📒 Files selected for processing (2)
sherpa-onnx/jni/offline-recognizer.ccsherpa-onnx/kotlin-api/OfflineRecognizer.kt
| // Per-token log-probabilities. The C++ greedy decoder populates | ||
| // result.ys_log_probs; surface them to Kotlin so callers can apply | ||
| // a real confidence-based policy instead of a heuristic proxy. | ||
| jfloatArray jys_log_probs = env->NewFloatArray(result.ys_log_probs.size()); | ||
| env->SetFloatArrayRegion(jys_log_probs, 0, result.ys_log_probs.size(), | ||
| result.ys_log_probs.data()); | ||
|
|
||
| jobject jresult = env->NewObject(cls, ctor, jtext, jtokens, jtimestamps, | ||
| jlang, jemotion, jevent, jdurations); | ||
| jlang, jemotion, jevent, jdurations, | ||
| jys_log_probs); |
There was a problem hiding this comment.
Only expose ys_log_probs when it matches the token count.
The C API already guards this field behind a length check, but this JNI path forwards any non-empty result.ys_log_probs. If native code ever returns a partial/misaligned vector, Kotlin callers will see “per-token” confidences that no longer line up with tokens. Please mirror the existing guard here and fall back to an empty array on mismatch.
Proposed change
- jfloatArray jys_log_probs = env->NewFloatArray(result.ys_log_probs.size());
- env->SetFloatArrayRegion(jys_log_probs, 0, result.ys_log_probs.size(),
- result.ys_log_probs.data());
+ const jsize ys_log_probs_size =
+ result.ys_log_probs.size() == result.tokens.size()
+ ? static_cast<jsize>(result.ys_log_probs.size())
+ : 0;
+ jfloatArray jys_log_probs = env->NewFloatArray(ys_log_probs_size);
+ if (ys_log_probs_size > 0) {
+ env->SetFloatArrayRegion(jys_log_probs, 0, ys_log_probs_size,
+ result.ys_log_probs.data());
+ }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@sherpa-onnx/jni/offline-recognizer.cc` around lines 673 - 682, Check that
result.ys_log_probs has the same length as the token sequence before exposing it
to Kotlin: if result.ys_log_probs.size() != result.tokens.size() (or is zero)
then create an empty jfloatArray (length 0) and use that instead of blindly
creating jys_log_probs from result.ys_log_probs; otherwise create jys_log_probs
with env->NewFloatArray and env->SetFloatArrayRegion as currently done and pass
it into NewObject (jresult) alongside jtext, jtokens, jtimestamps, jlang,
jemotion, jevent, jdurations so the JNI path mirrors the C API guard.
1 participant
Summary
OfflineRecognitionResult::ys_log_probs) through the offline JNI to the KotlinOfflineRecognizerResultdata class asysLogProbs: FloatArray.OnlineRecognizerResult(streaming). The offline path is the missing half.csrc/offline-transducer-greedy-search-decoder.cc, lines 68-76); the binding was simply dropping it.Why
With the field exposed, downstream callers can compute a real per-token confidence on-device:
geo_mean = exp(mean(log_probs))— joint probability proxymin = exp(min(log_probs))— weakest-link signalWithout it, applications fall back to heuristic proxies (tokens-per-second, timestamp variance) that saturate near 1.0 even on mishearings.
Concrete example from a smoke test with Parakeet TDT 0.6B-v2 on a Qualcomm QCS6490 device (Android arm64-v8a):
Call James Smith(clean)Call the on-call anesthesiologist(clean)Call Health Supervisor(House mis-heard)Col Cath Lad(catastrophic mis-hear)The geo-mean alone misses single-token uncertainty; the per-token array unlocks a min-based gate. That's only possible if the values reach Kotlin.
What changed
sherpa-onnx/jni/offline-recognizer.ccGetMethodIDsignature extended with one extra[F.NewFloatArray+SetFloatArrayRegionforresult.ys_log_probs.DeleteLocalRefon cleanup.sherpa-onnx/kotlin-api/OfflineRecognizer.ktval ysLogProbs: FloatArrayappended toOfflineRecognizerResult.The field is empty (0-length
jfloatArray) for non-transducer / non-greedy decoders until the corresponding C++ paths are extended. Documented in the Kotlin comment.Compatibility
result.ys_log_probswas already being computed; only the JNI pass-through is new.text,tokens,timestamps, etc.); anyone constructingOfflineRecognizerResultmanually (rare — usually constructed by the JNI) will need to add the field. Symmetric to PR Expose ys probs to JNI, Kotlin and Java API #2736's change toOnlineRecognizerResult.Test plan
build-android-arm64-v8a.shon Android NDK 27.ysLogProbs.size == tokens.size, values are negative log-probs as expected,exp(values)lies in (0, 1].Related
Summary by CodeRabbit