how to use aws bedrock?

[VinceCui]

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

• I have searched the issues of this repository and believe that this is not a duplicate.

Ⅰ. Issue Description

I have try AI-proxy aws bedrock, my request body is:

{
"model": "arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxxx",
"messages": [{
"role": "user",
"content": "今天是2024年07月29日。用json输出最近三天的日期，包含字段： today, next_day, day_after_tomorrow."
}]
}'

the Resp code 200, and body is:
{"id":"d6afdf1c-bc70-4a21-9d02-448f9bcef24a","choices":[{"index":0,"message":{"content":""}}],"created":1747205050,"model":"arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxxx","object":"chat.completion","usage":{}}

we can see there is no content reply.

Ps: my higress is deploy at Singapore, and when use boto3(bedrock SDK) is working with the same ak/sk 。

Ⅱ. Describe what happened

If there is an exception, please attach the exception trace:

Just paste your stack trace here!

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

1. xxx
2. xxx
3. xxx

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

• Higress version:
• OS :
• Others:

[johnlanni]

cc @HecarimV

[daixijun]

@VinceCui 我这边没有问题吖， 你调用的是哪个模型？

[VinceCui]

@daixijun 调用的是 Claude Sonnet 3.7

---

@daixijun is calling Claude Sonnet 3.7

[HecarimV]

anthropic.claude-3-7-sonnet-20250219-v1:0 是这个吗

---

anthropic.claude-3-7-sonnet-20250219-v1:0 Is this true

[VinceCui]

@HecarimV
是这个：arn:aws:bedrock:us-west-2:62246xxxxxxxx9:application-inference-profile/u5cfjxxxxxx4t。是基于claude-3-7-sonnet开了一个独立项目

---

@HecarimV
It's this: arn:aws:bedrock:us-west-2:62246xxxxxxx9:application-inference-profile/u5cfjxxxxx4t. It is based on claude-3-7-sonnet to open an independent project

[HecarimV]

@VinceCui https://docs.aws.amazon.com/zh_cn/bedrock/latest/userguide/conversation-inference-supported-models-features.html
这边是调用的Converse API去做处理的，他的modelId的取法
https://docs.aws.amazon.com/zh_cn/bedrock/latest/APIReference/API_runtime_Converse.html#API_runtime_Converse_RequestSyntax

[HecarimV]

@HecarimV 是这个：arn:aws:bedrock:us-west-2:62246xxxxxxxx9:application-inference-profile/u5cfjxxxxxx4t。是基于claude-3-7-sonnet开了一个独立项目

higress-gateway 日志你有看吗，有没啥有用信息

---

@HecarimV is this: arn:aws:bedrock:us-west-2:62246xxxxxxxx9:application-inference-profile/u5cfjxxxxx4t. It was based on claude-3-7-sonnet to open an independent project

Have you read the hgress-gateway log? Is there any useful information?

[VinceCui]

@HecarimV 我用的是higress-gateway：2.1.2，您用的什么版本呢？日志没有报错，因为请求返回是200

另外，我刚刚试了一下arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0，也不行。

[HecarimV]

另外，我刚刚试了一下arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0，也不行。

是不是IAM策略没有授予相关API权限啊

[VinceCui]

@HecarimV 我用boto3的python SDK可以通，是不是意味着IAM是OK的？难道还有其他需要开的权限？

---

@HecarimV I can use the python SDK of boto3. Does it mean that IAM is OK? Are there any other permissions that need to be opened?

[HecarimV]

@HecarimV 我用boto3的python SDK可以通，是不是意味着IAM是OK的？难道还有其他需要开的权限？

      provider:
        awsAccessKey: xxxxx
        awsRegion: xxx
        awsSecretKey: xxx

好的，plugin参数的话，bedrock必填就是这三个，访问路由到bedrock-runtime.{awsRegion}.amazonaws.com，其他没啥配置了

[VinceCui]

@HecarimV @daixijun
两位大佬，我打开了debug日志，输出如下，辛苦帮忙看一下

[johnlanni]

cc @rinfx

[VinceCui]

大家好，我在aws技术支持的帮助下解决了这个问题，方法是，在传入model时，先对"arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxxx"进行URL Encode，会得到形如“arn%3Aaws%3Abedrock%3Aus-west-2%3Axxxxxxxxxxxxxxxxxx”的字符串；然后，还需要在bedrock.go中的urlEncode函数中，补充将'%'转换为'%25'的逻辑。很神奇，这样就成功了。

但是，我不清楚为什么你们可以直接传入原始的model名称即可..

---

Hello everyone, I solved this problem with the help of aws technical support. The method is to first perform URL Encode on "arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxx" when passing in the model, and you will get a string like "arn%3Aaws%3Abedrock%3Aus-west-2%3Axxxxxxxxxxxxxxxxx"; then, you also need to add the logic of converting '%' to '%25' in the urlEncode function in bedrock.go. It's amazing, so it's successful.

However, I don't know why you can just pass in the original model name directly...

[CH3CHO]

在传入model时，先对"arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxxx"进行URL Encode

AWS Bedrock 的 model 是放到 URL 里的吧。URL 里的“:”按照 RFC 3986 规定，作为保留字符，是必须要进行转义的。

https://datatracker.ietf.org/doc/html/rfc3986#section-2.4

---

When passing in the model, first URL "arn:aws:bedrock:us-west-2:xxxxxxxxxxxxxxxxxx"

AWS Bedrock's model is placed in the URL. According to RFC 3986, ":" in the URL must be escaped as a reserved character.

https://datatracker.ietf.org/doc/html/rfc3986#section-2.4