flows doesn't unregister_function() functions that are not in the new node

[trycatchal]

Hello friends.

When going from node A with functions F to node B with functions G, we should unregister_function() all functions in F-G.

I thought it's harmless to leave them in there, but these days we're seeing our LLM do a lot of hallucinated function calls that aren't in the tools list, and in those cases, we want the unregistered function handler catch-all handler to handle those instead of the should-not-be-available-functionhandler.

Thanks!

EDITED: Semantics

[markbackman]

Function calls have two parts required:

1. Registering the function which makes it available to the LLM instance in the Python code
2. Adding the tool to the context at inference time

1 wires up your Python code, 2 makes the tool available to the LLM to call.

Flows uses LLMSetToolsFrames to manage 2. This ensures that only tools specified in the node are callable. There is no need to unregister the function.

[markbackman]

Closing out this item as that hopefully clarifies things.

For reference, this was discussed in more detail recently here:

pipecat-ai/pipecat#3306

[trycatchal]

That's what I thought too - that's why I wrote the "I thought it's harmless..." part... I hope you'll reconsider...

The problem is that if you only do "2", you have no way of protecting yourself when an LLM hallucinates and tries to call a function that's not in "1".

I'm seeing this happen more - especially when in the messages array there's previous calls to a function that's not supposed to be available.

[markbackman]

I can confirm that they're harmless.

Here's how it works:

• Upon node transition an LLMSetToolsFrame is pushed.
• The context aggregators handle the frame (user, assistant), both calling set_tools().
• set_tools() updates the context by setting tools in the context used for the completion.

To make sure this is working, I enabled LangFuse tracing on Flows to confirm that the tools are swapped out after every node.

Here's a short video demonstrating that the code works as expected:
https://share.cleanshot.com/6b0md8dl

[trycatchal]

Thanks again for the response Mark.

I can see where the misunderstanding is coming from and I'm sure this message will clear up what I'm saying (and hopefully reopen this issue).

I understand what you're saying - the LLM is being called with ONLY the new node's (+ global) tools. Yes I see that too 100%. Sorry in advance for the long message, but I can see I wasn't able to clearly get my point across with shorter messages.

What I'm having a hard time getting across is this: The LLM doesn't always respect the tools provided and sometimes tries to call a function that's not in the list (especially if it sees a function was successfully called in previous messages).

I thought it was impossible too, but it isn't. Here's some langfuse screenshots to show it (thanks for the idea)...

Here's an earlier part of the messages array when in a previous node save_name_to_lead is successfully called - remember that name for later...

We are now at a different node, and here are the tools that flows provides the LLM in the payload just before the problematic inference

Note that just as you said, save_name_to_lead is NOT in the list. So far we're 100% on the same page.

Here's where things go sideways and defy my previous expectations (and yours)...

Even though it's not in the list, the dumb LLM STILL requests it:

When the LLM misbehaves like this, we expect to be able to catch it and handle it. But if the residual function is still registered WITH PIPECAT, pipecat won't treat it like an unregistered function, but a VALID function, and will happily call its handler.

What I want is for these cases to either call a registered catch-all handler or otherwise throw an error so we can report the stupidity back to the LLM and pray it responds differently. Instead, the handler for the invalid tool is being happily called 😢.

I hope the screenshots help this time, and sorry again for the long message. And FYI I've seen this behavior across 3+ models (kimi-k2-instruct-0905, one of the llamas, and chatgpt-oss) and across 3 providers (groq, moonshot, together).

Thanks again,
Al

[markbackman]

Thanks for the details. That's helpful.

It sounds like your LLMs are being influenced by the tools previously called in the context. Can you confirm that?

I guess you're saying that, for models that will call non-existent tools, you want Flows to not call the handler. I have not personally seen this, but I've also tested with models from the top labs, which likely have better tool calling reliability. I'm not surprised (especially with Llama) that tool calling reliability is subpar.

I'll reopen this issue, but we might not get a chance to look at it for a bit.

[trycatchal]

Hooray! Thanks Mark!

Yes, it seems like previously called tools are influencing it because we've seen other cases of this happening where the function has more complex parameters, and it's calling it very similarly to a previous call (enough to where the handler executes without error).

I really appreciate you reopening the issue and understand you're busy.

Thanks,
Al

[trycatchal]

Just wanted to post a quick update ... Apparently the imaginary-tools-calling epidemic has spread to google too (here's gemini 2.5 flash lite)

...
{'parts': [{'text': 'I just wanna hear more about your product.'}], 'role': 'user'},
{'parts': [{'function_call': {'id': 'd1332d04-7e5b-45d6-b69c-8cb9548b2767', 'args': {'product_name': '(my-product)'}, 'name': 'get_product_info'}}], 'role': 'model'},
{'parts': [{'function_response': {'id': 'd1332d04-7e5b-45d6-b69c-8cb9548b2767', 'name': 'get_product_info', 'response': {'status': 'error', 'error': "CRITICAL ERROR: 'get_product_info' does not exist. This function was NEVER defined and will NEVER be available. STRICT INSTRUCTION: Do NOT call 'get_product_info' again under any circumstances. Use ONLY the functions explicitly provided in your current tool list.", 'is_error': True}}}], 'role': 'user'},
{'parts': [{'text': '(my-product) is an AI-powered phone receptionist that helps businesses never miss important calls. Our AI answers calls naturally, just like a real person, and is available around the clock, every day of the year. No sick days, no vacations, no breaks. Would you like to schedule a discovery call to learn more?'}], 'role': 'model'}

We've worked around this from the outside by intercepting LLMSetToolsFrames and unregistering ourselves (which is why you can see it handled with the global handler), but in this case get_product_info isn't even something in history - gemini is just yolo'ing a function name and parameter and crossing its silly little digital fingers :D

[markbackman]

We've been discussing this internally and we don't have a great solution just yet. This is a weakness of LLMs, in general; they will call tools from the context, regardless of what's in the API call.

@kompfner has been thinking about it, but it's unclear what the outcome will be. If/when there is a solution, it will be in the core Pipecat code.

[markbackman]

Actually, @kompfner has this PR open which addresses this issue to the best of our ability. This is still a weakness for LLMs and there are features starting to emerge that may help an LLM understand which tools are active or not—that's for the future though...