feat(speech): expose DashScope voice-enrollment (voice cloning) API - #9
Merged
Merged
Conversation
Adds POST /api/v1/services/audio/tts/customization as a fork-only, native DashScope endpoint so voice-cloning management (create_voice, list_voice, query_voice, update_voice, delete_voice) flows through the gateway with the usual auth / logging / RBAC surface attached. Design: this is a pass-through, not a translation. There is no OpenAI-shaped counterpart to voice-cloning; the caller speaks DashScope's native schema and the gateway forwards the body unchanged. Only body.model is parsed (to populate x-ai-eg-model=voice-enrollment for route matching) and body.input.action is exposed on the request struct so operators can tag metrics per action. Layout mirrors the existing GeminiCachedContentsEndpointSpec / cachedContents translator pair: - internal/apischema/dashscope: VoiceEnrollmentRequest + VoiceEnrollmentInput minimal parsing types (Model + Action). Rest of the payload stays as raw JSON because it varies per action and modelling it here would just track upstream drift. - internal/translator/voice_enrollment_dashscope.go: pass-through translator that rewrites `:path` to /api/v1/services/audio/tts/customization and only replays the body when Envoy's upstream stage forces a mutation (retry / streaming). No response reshaping — DashScope's action-specific envelope is the caller's contract. - internal/endpointspec/endpointspec.go: DashScopeVoiceEnrollmentEndpointSpec with ParseBody / GetTranslator / RedactSensitiveInfoFromRequest. Only AlibabaDashScope schema is supported. - cmd/extproc/mainlib/main.go: server.Register for the native path with a noop tracer (no response object to record) and speech metrics factory. Not covered by this PR (defer): - gitops backend + route rule (workspace endpoint + AIGatewayRoute) — separate follow-up so the gateway image can roll independently of the route change. - OpenAI-shaped surface (/v1/audio/voices etc.). If demand shows up, layer it on top; DashScope's action-set doesn't map cleanly to any OpenAI shape today. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
📝 WalkthroughWalkthroughDashScope 음성 등록 요청 스키마와 전용 엔드포인트 스펙이 추가되었습니다. 번역기는 네이티브 경로로 요청을 전달하고 응답·오류를 패스스루하며, 외부 프로세서에 새 라우팅 경로가 등록되었습니다. ChangesDashScope 음성 등록
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant ExtProc
participant EndpointSpec
participant Translator
participant DashScope
Client->>ExtProc: POST 음성 등록 요청
ExtProc->>EndpointSpec: 요청 파싱 및 백엔드 확인
EndpointSpec->>Translator: 번역기 선택
Translator->>DashScope: 네이티브 경로와 요청 본문 전달
DashScope-->>Translator: 응답 또는 오류 반환
Translator-->>Client: 원본 응답 또는 오류 전달
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/endpointspec/endpointspec.go`:
- Around line 617-624: Validate req.Model in the voice-enrollment request
handling before returning from this path: require the exact value
"voice-enrollment", reject empty or mismatched values with
internalapi.ErrMalformedRequest, and return the fixed model value for routing
instead of arbitrary req.Model.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 375a61da-79eb-4f3a-b56b-ef824dcdd6c6
📒 Files selected for processing (5)
cmd/extproc/mainlib/main.gointernal/apischema/dashscope/dashscope.gointernal/endpointspec/endpointspec.gointernal/translator/voice_enrollment_dashscope.gointernal/translator/voice_enrollment_dashscope_test.go
Comment on lines
+617
to
+624
| var req dashscopeschema.VoiceEnrollmentRequest | ||
| if len(body) > 0 { | ||
| if err := json.Unmarshal(body, &req); err != nil { | ||
| return "", nil, false, nil, fmt.Errorf("%w: failed to parse JSON for voice enrollment: %w", internalapi.ErrMalformedRequest, err) | ||
| } | ||
| } | ||
| // voice-enrollment is never streamed. | ||
| return req.Model, &req, false, nil, nil |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
model을 voice-enrollment으로 검증해야 합니다.
주석과 테스트는 이 엔드포인트의 모델이 항상 voice-enrollment이라고 정의하지만, 현재 구현은 임의의 req.Model을 그대로 라우팅에 반환합니다. 이 값이 x-ai-eg-model 및 모델별 라우팅/RBAC 선택에 사용되므로, 불일치하거나 빈 model은 ErrMalformedRequest로 거부하고 고정값만 허용해야 합니다.
제안 수정
if len(body) > 0 {
if err := json.Unmarshal(body, &req); err != nil {
return "", nil, false, nil, fmt.Errorf("%w: failed to parse JSON for voice enrollment: %w", internalapi.ErrMalformedRequest, err)
}
}
+ if req.Model != "voice-enrollment" {
+ return "", nil, false, nil, fmt.Errorf("%w: model must be voice-enrollment", internalapi.ErrMalformedRequest)
+ }
// voice-enrollment is never streamed.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| var req dashscopeschema.VoiceEnrollmentRequest | |
| if len(body) > 0 { | |
| if err := json.Unmarshal(body, &req); err != nil { | |
| return "", nil, false, nil, fmt.Errorf("%w: failed to parse JSON for voice enrollment: %w", internalapi.ErrMalformedRequest, err) | |
| } | |
| } | |
| // voice-enrollment is never streamed. | |
| return req.Model, &req, false, nil, nil | |
| var req dashscopeschema.VoiceEnrollmentRequest | |
| if len(body) > 0 { | |
| if err := json.Unmarshal(body, &req); err != nil { | |
| return "", nil, false, nil, fmt.Errorf("%w: failed to parse JSON for voice enrollment: %w", internalapi.ErrMalformedRequest, err) | |
| } | |
| } | |
| if req.Model != "voice-enrollment" { | |
| return "", nil, false, nil, fmt.Errorf("%w: model must be voice-enrollment", internalapi.ErrMalformedRequest) | |
| } | |
| // voice-enrollment is never streamed. | |
| return req.Model, &req, false, nil, nil |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/endpointspec/endpointspec.go` around lines 617 - 624, Validate
req.Model in the voice-enrollment request handling before returning from this
path: require the exact value "voice-enrollment", reject empty or mismatched
values with internalapi.ErrMalformedRequest, and return the fixed model value
for routing instead of arbitrary req.Model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
POST /api/v1/services/audio/tts/customizationas a fork-only, native DashScope endpoint so voice-cloning management (create_voice,list_voice,query_voice,update_voice,delete_voice) flows through the gateway with the usual auth / logging / RBAC surface attached. Depends on the AlibabaDashScope schema shipped in #6.Design — pass-through, not translation
DashScope's voice-cloning API has no OpenAI counterpart — the caller must speak DashScope's native schema (
{"model":"voice-enrollment","input":{"action":"create_voice",...}}). Every action shares the same URL and differs only byinput.action. We would gain nothing by modelling each action's fields on our side; the schema would just chase upstream drift.So the gateway:
body.model(→x-ai-eg-model=voice-enrollmentfor route matching) andbody.input.action(available for metrics tagging).:pathto/api/v1/services/audio/tts/customization.Files
internal/apischema/dashscope/dashscope.goVoiceEnrollmentRequest+VoiceEnrollmentInputminimal parsing typesinternal/translator/voice_enrollment_dashscope.gointernal/translator/voice_enrollment_dashscope_test.gointernal/endpointspec/endpointspec.goDashScopeVoiceEnrollmentEndpointSpecwithParseBody/GetTranslator/redactioncmd/extproc/mainlib/main.goserver.Registerfor the native path (speech metrics + noop tracer)Not in this PR
x-ai-eg-model=voice-enrollment→ new AIServiceBackend pointing at the workspace endpoint (ws-…-maas.aliyuncs.com) withschema.name: AlibabaDashScope./v1/audio/voicesetc.). DashScope's action-set doesn't map cleanly to any OpenAI shape today.Verified
go build ./... && go vet ./...✓golangci-lint run→ 0 issuesinternal/translator,internal/endpointspec,internal/extproc🤖 Generated with Claude Code
Summary by CodeRabbit
새 기능
테스트