The first I think of when anyone mentions agent-like “tool use” is: - Is the environment that the tools are run from sandboxed? I’m unclear on when/how/why you’d want an LLM executing code on your machine or in a non-sandboxed environment. Anyone care to enlighten?
It is up to the person who implements the tool to sandbox it as appropriate.
> I’m unclear on when/how/why you’d want an LLM executing code on your machine or in a non-sandboxed environment.
The LLM does not execute code on your computer. It returns the fact that the LLM would like to execute a tool with certain parameters. You should trust those parameters as much as you trust the prompt and the LLM itself. Which in practice probably ends up being "not much".
Good news is that in your tool implementation you can (and should) apply all the appropriate checks using regular coding practices. This is nothing new. We do this all the time with web requests. You can check if the prompt originates from an authenticated user, if they have the necessary permissions to do the action they are about to do. You can throttle the requests, you can check that the inputs are appropriate etc etc.
If the tool is side-effect free and there are no access restrictions you can just run it easy. For example imagine an LLM which can turn the household name of a plant to its latin name. You would have a "look_up_latin_name" tool which searches in a local database. You have to make sure to follow best practices to avoid an sql injection attack, but otherwise this should be easy.
Now imagine a more sensitive situation. A tool with difficult to undo side-effects, and strict access controls. For example launching an ICBM attack. You would create a "launch_nukes" tool, but the tool wouldn't just launch willy nilly. First of all it would check that the prompt arrived from directly the president. (how you do that is best discussed with your NSA rep in person) Then it would check that the parameter is one of the valid targets. But that is not enough yet. You want to make sure it is not the LLM hallucinating the action. So you would pop up a prompt directly on the UI to confirm the action. Something like "Looks like you want to destroy . Do you want to proceed? " And would only launch when the president clicks the yes.