Ollama now supports tool calling with popular models in local LLM
1–10 of 27 posts
Re: Ollama now supports tool calling with popular models in local LLM
#2[deleted]
Re: Ollama now supports tool calling with popular models in local LLM
#3How does this compare to Agent Zero (frdel/agent-zero on GitHub)? Seems that provides similar functionality and uses docker for running the scripts / code generated.
Re: Ollama now supports tool calling with popular models in local LLM
#4Where is `get_current_weather` implemented?
> Tool responses can be provided via messages with the `tool` role.
Re: Ollama now supports tool calling with popular models in local LLM
#5Where is `get_current_weather` implemented? > Tool responses can be provided via messages with the `tool` role.
Not on the Ollama side.
This sample code shows how a sample implementation of a tool like `get_current_weather` might look like in Python:
https://github.com/ollama/ollama-python/blob/main/examples/t...
Re: Ollama now supports tool calling with popular models in local LLM
#6Where is `get_current_weather` implemented? > Tool responses can be provided via messages with the `tool` role.
Not on the Ollama side. This sample code shows how a sample implementation of a tool like `get_current_weather` might look like in Python: https://github.com/ollama/ollama-python/blob/main/examples/t...
Ah, I see. The model returns the name of an appropriate tool, then the client takes arbitrary action, and appends the `tool` message to the chat context, and finally a second call to the model minges these together.
Part of me was hoping for some magic plugin space where I could drop named functions, but I couldn't imagine how.
Re: Ollama now supports tool calling with popular models in local LLM
#7My guess since programmer blog post writing (plus autism?) assumes “Everyone already knows everything about my project because I do!”…
Is this to the effect of running a local LLM, that reads your prompt and then decides which correct/specialized LLM to hand it off to? If that is the case, isn’t it going to be a lot of latency to switch models back and forth as most people usually run the single largest model that will fit on their GPU?
Re: Ollama now supports tool calling with popular models in local LLM
#8Earlier quoted context omitted.
Not on the Ollama side. This sample code shows how a sample implementation of a tool like `get_current_weather` might look like in Python: https://github.com/ollama/ollama-python/blob/main/examples/t...
Ah, I see. The model returns the name of an appropriate tool, then the client takes arbitrary action, and appends the `tool` message to the chat context, and finally a second call to the model minges these together. Part of me was hoping for some magic plugin space where I could drop named functions, but I couldn't imagine how.
You specify the API of your functions (inputs/description).
The LLM will decide which functions to call and with what values.
You perform the actual function execution.
Re: Ollama now supports tool calling with popular models in local LLM
#9My guess since programmer blog post writing (plus autism?) assumes “Everyone already knows everything about my project because I do!” … Is this to the effect of running a local LLM, that reads your prompt and then decides which correct/specialized LLM to hand it off to? If that is the case, isn’t it going to be a lot of latency to switch models back and forth as most people usually run the single largest model that w…
[dead]
Re: Ollama now supports tool calling with popular models in local LLM
#10My guess since programmer blog post writing (plus autism?) assumes “Everyone already knows everything about my project because I do!” … Is this to the effect of running a local LLM, that reads your prompt and then decides which correct/specialized LLM to hand it off to? If that is the case, isn’t it going to be a lot of latency to switch models back and forth as most people usually run the single largest model that w…
No, this is a bit different. When GPT 4.o came out OpenAI also added new features that allow the models to perform actions. This allows you to do that, but locally.
The reason this is cool is because it allows you to integrate with things like Home Assistant, so you can ask your chat bot or whatever to actual take actions. "Hey bot, turn on the lights in the basement" as an example.