Question about ollama + qwen

[zzllkk2003]

Can Spring AI support the Qwen large language model?
And can spring-ai-ollama-spring-boot-starter support function calling?

[qinfengge]

For the first one, it supports Qwen. For the second one, it does not yet support ollama's function call.

[zzllkk2003]

For the first one, it supports Qwen. For the second one, it does not yet support ollama's function call.

Which Maven package can I use?

[codespearhead]

1. Does Spring AI support the Qwen large language model?

Spring AI supports Ollama, so you just have to use the Ollama starter package.

2. Does spring-ai-ollama-spring-boot-starter support function calling?

Spring AI does [1][2][3], but Ollama doesn't officially support function calling yet [4][5], so naturally spring-ai-ollama-spring-boot-starter doesn't have it baked-in. By the way, neither vLLM nor SGLang do either [6].

You might want to take a look at LangChain4j [8][9].

---

[1] #193
[2] https://www.devturtleblog.com/sring-ai-function-calling-tutorial/
[3] https://spring.io/blog/2024/03/06/function-calling-in-java-and-spring-ai-using-the-latest-mistral-ai-api

[4] QwenLM/Qwen3#378 (QwenLM/Qwen3#378 (comment))
[5] ollama/ollama#3137

[6] https://github.com/QwenLM/Qwen2/blob/d439e04e84ebe3734dea4e9769e17f380a8c72b8/README.md?plain=1#L192
[7] https://docs.langchain4j.dev/tutorials/tools#:~:text=following%20models%20have%20tool%20support

[8] https://docs.langchain4j.dev/tutorials/tools/#:~:text=following%20models%20have%20tool%20support
[9] https://docs.langchain4j.dev/tutorials/spring-boot-integration/#langchain4j-spring-boot-starter

[jidaojiuyou]

1. spring cloud alibaba ai for qwen
2. 目前无法支持。因为 ollama 的 api 不支持 tools 和 tools_choice 字段。但是可以参考以下文档自己实现。（不推荐但是可用，以下两种方式都可以实现ollama的function call。）
   langchain ollama function call
   qwen agent funciton call

[tzolov]

@zzllkk2003, Ollama added (partial) function calling support last week and Spring AI just provided support for it: https://docs.spring.io/spring-ai/reference/api/chat/ollama-chat.html#_function_calling

But of it to work the underling models must support function calling. I'm not sure qwen2 supports it? Can you point me to a model that does?
Just tried with qwen2 and got:

{"error":"qwen2 does not support tools"}

For this to work we need a qwen/qwen2 model that has been trained for function calling?

[SpiritCloude]

Llama 3.1-8B、70B、405B is a new state-of-the-art model and support Tools.but Spring AI does support function calling ，It would be great if the next step could support function calling

[zzllkk2003]

For this to work we need a qwen/qwen2 model that has been trained for function calling?

@tzolov tzolov please see at here
https://qwen.readthedocs.io/en/latest/framework/function_call.html

[markpollack]

@SpiritCloude you can call the llama models with spring ai and function calling.