Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: My LLM CLI tool can run tools now, from Python code or plugins

simonwillison.net

Post reply on HN
Print view View original

151–160 of 178 posts

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#151

Post by anthk »

sillysaurusx wrote: post #148
xk_id wrote: post #102

Earlier quoted context omitted.

The transition from assembly to C was to a different layer of abstraction within the same context of deterministic computation. The transition from programming to LLM prompting is to a qualitatively different context, because the process is no longer deterministic, nor debuggable. So your analogy fails to apply in a meaningful way to this situation.

Ultimately it’s about being able to create features within a certain period of time, not just to write code. And in that context the shift from assembly to C and the shift from deterministic programming to LLM prompting seem to have similar magnitudes of impact.

We tried with Common Lisp, Prolog... it didn't work for all cases, and it won't work again.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#152

Post by rcarmo »

kristopolous wrote: post #84
kristopolous wrote: post #45

It's worth noting the streaming markdown renderer I wrote just for this tool: https://github.com/day50-dev/Streamdown More background: https://github.com/simonw/llm/issues/12 (Also check out https://github.com/day50-dev/llmehelp which features a tmux tool I built on top of Simon's llm. I use it every day. Really. It's become indispensable)

Also I forgot to one other built on llm. This one is a ZSH plugin that uses zle to translate your English to shell commands with a keystroke. https://github.com/day50-dev/Zummoner It's been life changing for me. Here's one I wrote today: $ git find out if abcdefg is a descendent of hijklmnop In fact I used it in one of these comments $ for i in $(seq 1 6); do printf "%${i}sh${i}\n\n-----\n" | tr " " "#"; done | pv -b…

Okay, this is very cool.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#153

Post by howmayiannoyyou »

icarito wrote: post #104

For all of you using `llm` - perhaps take a look at [Gtk-llm-chat]( https://github.com/icarito/gtk-llm-chat ). I put a lot of effort into it - it integrates with `llm` command line tool and with your desktop, via a tray icon and nice chat window. I recently released 3.0.0 with packages for all three major desktop operating systems.

This is pretty great.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#154

Post by yangikan »

simonw wrote: post #34
ttul wrote: post #32

GPT-4.1 is a capable model, especially for structured outputs and tool calling. I’ve been using LLMs for my day to day grunt work for two years now and this is my goto as a great combination of cheap and capable.

I'm honestly really impressed with GPT-4.1 mini. It is my default from messing around by their API because it is unbelievably inexpensive and genuinely capable at most of the things I throw at it. I'll switch to o4-mini when I'm writing code, but otherwise 4.1-mini usually does a great job. Fun example from earlier today: llm -f https://raw.githubusercontent.com/BenjaminAster/CSS-Minecraft/refs/heads/main/main.css \…

Thanks for this. I am planning to cancel my ChatGPT plus subscription and use something like the llm tool with the API key. For regular interactions, how do you handle context? For example, the UI allows me to ask a question, and then a followup and the context is kind of automatically handled.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#155

Post by yangikan »

yangikan wrote: post #154
simonw wrote: post #34

Earlier quoted context omitted.

I'm honestly really impressed with GPT-4.1 mini. It is my default from messing around by their API because it is unbelievably inexpensive and genuinely capable at most of the things I throw at it. I'll switch to o4-mini when I'm writing code, but otherwise 4.1-mini usually does a great job. Fun example from earlier today: llm -f https://raw.githubusercontent.com/BenjaminAster/CSS-Minecraft/refs/heads/main/main.css \…

Thanks for this. I am planning to cancel my ChatGPT plus subscription and use something like the llm tool with the API key. For regular interactions, how do you handle context? For example, the UI allows me to ask a question, and then a followup and the context is kind of automatically handled.

I should have RTFM https://llm.datasette.io/en/stable/usage.html#starting-an-in...

Are you aware of any user interfaces that expose some limited ChatGPT functionality using a UI, that internally uses llm. This is for my non-techie wife.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#156

Post by losvedir »

Very cool!

I've wondered how exactly, say, Claude Code knows about and uses tools. Obviously, an LLM can be "told" about tools and how to use them, and the harness can kind of manage that. But I assumed Claude Code has a very specific expectation around the tool call "API" that the harness uses, probably reinforced very heavily by some post-training / fine tuning.

Do you think your 3rd party tool-calling framework using Claude is at any disadvantage to Anthropic's own framework because of this?

Separately, on that other HN post about the GitHub MCP "attack", I made the point that LLMs can be tricked into using up to the full potential of the credential. GitHub has fine-grained auth credentials, and my own company does as well. I would love for someone to take a stab at a credential protocol that the harness can use to generate fine-grained credentials to hand to the LLM. I'm envisioning something where the application (e.g. your `llm` CLI tool) is given a more powerful credential, and the underlying LLM is taught how to "ask for permission" for certain actions/resources, which the user can grant. When that happens the framework gets the scoped credential from the service, which the LLM can then use in tool calls.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#157

Post by simonw »

mihau wrote: post #147
simonw wrote: post #14

Earlier quoted context omitted.

Yeah I had a bit of an experiment with MCP this morning, to see if I could get a quick plugin demo out for it. It's a bit tricky! The official mcp Python library really wants you to run asyncio and connect to the server and introspect the available tools.

Hi Simon! I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP. I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem ) - https://github.com/Virtuslab/llm-tools-mcp . Very early stage, so please make sure to use with --ta option (Manually approve every tool execution). The code is still messy and there…

OK this looks like a very promising start!

You're using function-based tools at the moment, hence why you have to register each one individually.

The alternative to doing that is to use what I call a "toolbox", described here: https://llm.datasette.io/en/stable/python-api.html#python-ap...

Those get you two things you need:

1. A single class can have multiple tool methods in it, you just have to specify it once 2. Toolboxes can take configuration

With a Toolbox, your plugin could work like this:

  llm -T 'MCP("path/to/mcp.json")' ...
You might even be able to design it such that you don't need a mcp.json at all, and everything gets passed to that constructor.

There's one catch: currently you would have to dynamically create the class with methods for each tool, which is possible in Python but a bit messy. I have an open issue to make that better here: https://github.com/simonw/llm/issues/1111

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#158

Post by simonw »

yangikan wrote: post #155
yangikan wrote: post #154

Earlier quoted context omitted.

Thanks for this. I am planning to cancel my ChatGPT plus subscription and use something like the llm tool with the API key. For regular interactions, how do you handle context? For example, the UI allows me to ask a question, and then a followup and the context is kind of automatically handled.

I should have RTFM https://llm.datasette.io/en/stable/usage.html#starting-an-in... Are you aware of any user interfaces that expose some limited ChatGPT functionality using a UI, that internally uses llm. This is for my non-techie wife.

Here's one: https://github.com/icarito/gtk-llm-chat

I've been meaning to put together a web UI for ages, I think that's the next big project now that tools is out.

It's not using LLM, but right now one of the best UI options out there is https://openwebui.com/ - it works really well with Ollama (and any other OpenAI-compatible endpoint).

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#159

Post by simonw »

losvedir wrote: post #156

Very cool! I've wondered how exactly, say, Claude Code knows about and uses tools. Obviously, an LLM can be "told" about tools and how to use them, and the harness can kind of manage that. But I assumed Claude Code has a very specific expectation around the tool call "API" that the harness uses, probably reinforced very heavily by some post-training / fine tuning. Do you think your 3rd party tool-calling framework us…

That credentials trick is possible right now using LLM's tool support. You'd have to write a pretty elaborate tool setup which exposes the "ask for more credentials" tool and then prompts the user when that's called. The tool should keep the credentials and never pass the actual tokens back to the LLM, but it can pass e.g. a symbol "creds1" and tell the LLM to request to make calls with "creds1" in future requests.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#160

Post by theptip »

isaacremuant wrote: post #55
kbelder wrote: post #50

Earlier quoted context omitted.

If you hook an llm up to your brokerage account, someone is being stupid, but it ain't the bot.

You think "senior leadership/boards of directors" aren't thinking of going all in with AI to "save money" and "grow faster and cheaper"? This is absolutely going to happen at a large scale and then we'll have "cautionary tales" and a lot of "compliance" rules.

Evolution in action. This is what the free market is good for.
Post reply on HN