homeassistant

So you're getting into function/tool calling with llms. If you don't know, they are structured things that you give to your LLM (at the API/model level, not you directly) that says "I have coded a function called get weather, if the user asks about the weather, let me know by returning a tool call response asking for the weather and I will get it for you" (in layman's terms).

This is how HA works too, if you ask the LLM to do something it is essentially running a tool call to HA for you.

Now, for your question, I'm betting there are ways to add in more, if HA supports this, idk. However, that's what you're looking for. It's also kinda named Agents, like you would create a web agent which takes in what the LLM asks for, gets the top 10 responses, collates them and gives them back to the LLM for summarization.

I'm guessing others have done this, but now you know what you're looking for!

Would need more information on what you've done here, but I assume you just used the default add-on, which does not handle asynchronous requests like that out of the box. From the very limited docs, it does mention building 'Sentence Triggers'. You'd need to probably do a few things:

• Tweak the timeout for responses to allow time for it get a formed payload and send it back
• Look at other plugins as examples to form custom sentence triggers
• Set some sort of boundaries to decide when it expects a response from the remote Ollama instance versus using internal data as a response

Just looking at the code here, none of the currently configured interfaces would be expecting any sort of response back from Ollama such as a web search. It's only using it for local inference about entities and performed sentences within HA itself.