Pass context information to Function Call Thread

[dolukhanov]

Expected Behavior

When using Function Calling with OpenAI and other models - the function gets invoked in a new thread.

The function being called in my application requires context, so it can persist the response data to the correct user / session etc.

My application uses Spring Security - however, I anticipate there could be other requirements to pass context to this thread.

It would be ideal to be able to store the context before the thread is called and then retrieve the context e.g. using:

SecurityContext context = SecurityContextHolder.getContext();

and

SecurityContextHolder.setContext( context );

Perhaps you could register listeners / functions that can be called to pass this information.

Current Behavior

The calling thread does not have any context about the session / user that invoked the call to the AI model.

Context

The only workaround is to request that the model pass an identified to the function via the prompt.

[miltonhit]

I've opened a similar issue 3 weeks ago:
#864

A possible workahound for it is passing a function "instance" in each interaction.
Then you can pass some user/session in constructor.

[tzolov]

Thanks you highlighting this use case @dolukhanov, @miltonhit

Can you please elaborate on why the function information should be kept in Thread context rather, for example, being returned with the chat response metadata as suggested here?

[markpollack]

Hi! Can you see if #1458 fixes this issue for you. Here are some minimal docs - https://docs.spring.io/spring-ai/reference/api/chat/functions/openai-chat-functions.html#_tool_context_support see the test here as well

spring-ai/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackWithPlainFunctionBeanIT.java

Line 60 in eb2deba

void functionCallTest2() {

please reopen if you think something is not addressed.