Skip to content

feat(ai-proxy): add responses support for doubao #2509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 28, 2025
Merged

feat(ai-proxy): add responses support for doubao #2509

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions plugins/wasm-go/extensions/ai-proxy/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,6 +406,9 @@ func getApiName(path string) provider.ApiName {
if util.RegDeleteFineTuningCheckpointPermissionPath.MatchString(path) {
return provider.ApiNameDeleteFineTuningCheckpointPermission
}
if strings.HasSuffix(path, "/v1/responses") {
return provider.ApiNameResponses
}
Comment on lines +409 to +411
Copy link

@lingma-agents lingma-agents bot Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

路径中的版本号与常量定义不一致，导致API名称无法正确识别

🟠 Critical | 🐞 Bugs

📋 问题详情

在getApiName函数中新增的条件检查路径是否以“/v1/responses”结尾，而doubao.go中定义的doubaoResponsesPath是“/api/v3/responses”。这两个路径的版本号（v1 vs v3）和结构不一致，导致条件判断失败，无法正确返回ApiNameResponses。需统一路径版本和结构。

💡 解决方案

将路径检查中的“/v1/responses”改为与doubao.go中定义的doubaoResponsesPath一致的“/api/v3/responses”：

-if strings.HasSuffix(path, "/v1/responses") {
+if strings.HasSuffix(path, "/api/v3/responses") {

您的反馈对我们很重要！(建议右键在新标签页中打开以下链接)

有用意见👍 ｜ 无用意见👎 ｜ 错误意见❌

// cohere style
if strings.HasSuffix(path, "/v1/rerank") {
return provider.ApiNameCohereV1Rerank
Expand Down
34 changes: 34 additions & 0 deletions plugins/wasm-go/extensions/ai-proxy/provider/doubao.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@ import (
"strings"

"github.com/alibaba/higress/plugins/wasm-go/extensions/ai-proxy/util"
"github.com/alibaba/higress/plugins/wasm-go/pkg/log"
"github.com/alibaba/higress/plugins/wasm-go/pkg/wrapper"
"github.com/higress-group/proxy-wasm-go-sdk/proxywasm/types"
"github.com/tidwall/gjson"
"github.com/tidwall/sjson"
)

const (
doubaoDomain = "ark.cn-beijing.volces.com"
doubaoChatCompletionPath = "/api/v3/chat/completions"
doubaoEmbeddingsPath = "/api/v3/embeddings"
doubaoImageGenerationPath = "/api/v3/images/generations"
doubaoResponsesPath = "/api/v3/responses"
)

type doubaoProviderInitializer struct{}
Expand All @@ -31,6 +35,7 @@ func (m *doubaoProviderInitializer) DefaultCapabilities() map[string]string {
string(ApiNameChatCompletion): doubaoChatCompletionPath,
string(ApiNameEmbeddings): doubaoEmbeddingsPath,
string(ApiNameImageGeneration): doubaoImageGenerationPath,
string(ApiNameResponses): doubaoResponsesPath,
}
}

Expand Down Expand Up @@ -70,6 +75,32 @@ func (m *doubaoProvider) TransformRequestHeaders(ctx wrapper.HttpContext, apiNam
headers.Del("Content-Length")
}

func (m *doubaoProvider) TransformRequestBody(ctx wrapper.HttpContext, apiName ApiName, body []byte) ([]byte, error) {
var err error
switch apiName {
case ApiNameResponses:
// 移除火山 responses 接口暂时不支持的参数
// 参考: https://www.volcengine.com/docs/82379/1569618
// TODO: 这里应该用 DTO 处理
for _, param := range []string{"parallel_tool_calls", "tool_choice", "stream_options"} {
body, err = sjson.DeleteBytes(body, param)
if err != nil {
log.Warnf("[doubao] failed to delete %s in request body, err: %v", param, err)
}
}
Comment on lines +85 to +90
Copy link

@lingma-agents lingma-agents bot Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参数删除错误处理不充分，导致错误被忽略

🟠 Critical | 🐞 Bugs

📋 问题详情

在TransformRequestBody函数中，当sjson.DeleteBytes发生错误时仅记录警告并继续处理，但未返回错误可能导致后续逻辑异常。建议在首次错误时立即返回错误。

💡 解决方案

在参数删除失败时立即返回错误：

@@ -88,6 +88,7 @@
     log.Warnf("[doubao] failed to delete %s in request body, err: %v", param, err)
+    return nil, err
     }
 }

您的反馈对我们很重要！(建议右键在新标签页中打开以下链接)

有用意见👍 ｜ 无用意见👎 ｜ 错误意见❌

case ApiNameImageGeneration:
// 火山生图接口默认会带上水印,但 OpenAI 接口不支持此参数
// 参考: https://www.volcengine.com/docs/82379/1541523
if res := gjson.GetBytes(body, "watermark"); !res.Exists() {
body, err = sjson.SetBytes(body, "watermark", false)
if err != nil {
log.Warnf("[doubao] failed to set watermark in request body, err: %v", err)
}
}
}
return m.config.defaultTransformRequestBody(ctx, apiName, body)
}

func (m *doubaoProvider) GetApiName(path string) ApiName {
if strings.Contains(path, doubaoChatCompletionPath) {
return ApiNameChatCompletion
Expand All @@ -80,5 +111,8 @@ func (m *doubaoProvider) GetApiName(path string) ApiName {
if strings.Contains(path, doubaoImageGenerationPath) {
return ApiNameImageGeneration
}
if strings.Contains(path, doubaoResponsesPath) {
return ApiNameResponses
}
Comment on lines +114 to +116
Copy link

@lingma-agents lingma-agents bot Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

使用Contains检查路径可能导致误匹配，应改为更精确的检查方式

🟡 Major | 🐞 Bugs

📋 问题详情

在GetApiName函数中，使用strings.Contains检查路径是否包含doubaoResponsesPath（“/api/v3/responses”）可能导致误匹配，例如路径“/api/v3/responsesomething”也会触发该条件。建议改为使用strings.HasSuffix确保路径结尾精确匹配。

💡 解决方案

将Contains改为HasSuffix确保路径精确匹配：

-if strings.Contains(path, doubaoResponsesPath) {
+if strings.HasSuffix(path, doubaoResponsesPath) {

您的反馈对我们很重要！(建议右键在新标签页中打开以下链接)

有用意见👍 ｜ 无用意见👎 ｜ 错误意见❌

return ""
}
Loading