Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: My LLM CLI tool can run tools now, from Python code or plugins

simonwillison.net

Post reply on HN
Print view View original

71–80 of 178 posts

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#71

Post by th0ma5 »

simonw wrote: post #52
xrd wrote: post #49

Earlier quoted context omitted.

Is no one else bothered by that way of using tools? Tools feel like a way to get deterministic behavior from a very hallucinatory process. But unless you put a very lengthy and comprehensive non-deterministic English statement, you can't effectively use tools. As we all know, the more code, the more bugs. These long and often hidden prompts seem like the wrong way to go. And, this is why I'm very excited about this a…

As an experienced software engineer I'm bothered about pretty much everything about how we develop things on top of LLMs! I can't even figure out how to write automated tests for them. See also my multi-year obsession with prompt injection and LLM security, which still isn't close to being a solved problem: https://simonwillison.net/tags/prompt-injection/ Yet somehow I can't tear myself away from them. The fact that…

> The fact that we can use computers to mostly understand human language

I agree that'd be amazing if they do that, but they most certainly do not. I think this is the core my disagreement here that you believe this and let this guide you. They don't understand anything and are matching and synthesizing patterns. I can see how that's enthralling like watching a rube goldberg machine go through its paces, but there is no there there. The idea that there is an emergent something there is at best an unproven theory, is documented as being an illusion, and at worst has become an unfounded messianic belief.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#72

Post by th0ma5 »

simonw wrote: post #43
bgwalter wrote: post #42

[flagged]

Are you an alt for th0ma5?

They are not. I think it is a good criticism though. Many people seem to be touting productivity that is only in the context of productivity towards more LLM inference operations and not productive in the sense of solving real world computing problems. There is a lot of material that suggests positive results are a kind of wish casting and people are not aware of the agency they are bringing to the interaction. The fact that you can hack things that do cool stuff is more of a reflection that you do those things, and that these models are not capable of it. That's why I recommend you work with others and you'll see your concepts that you feel are generalizable are not, and any learnings or insights are not like learning math or how to read, but more like learning a specific video game's rules. This is also why it is enthralling to you because you actually have only the illusion of controlling it.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#73

Post by roxolotl »

simonw wrote: post #64
roxolotl wrote: post #63

Simon thank you so much for this tool! I use it daily now since charmbraclet’s Mods[0] doesn’t support anthropics models. And now with tool calling it’ll be even more useful. I am curious though if there’s any appetite for improving performance? It’s noticeably slow to even just print the help on all of my machines(M1 32gb/M2 21g/ryzen 7700 64gb). 0: https://github.com/charmbracelet/mods

How many plugins do you have installed? We've seen problems in the past where plugins with expensive imports (like torch) slow everything down a lot: https://github.com/simonw/llm/issues/949 I'm interested in tracking down the worst offenders and encouraging them to move to lazy imports instead.

I’ve only got the Anthropic and Gemini plugins installed. I’d be happy to do a bit more digging. I’m away for a bit but would be happy to file an issue with more context when I get a chance.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#74

Post by hanatanaka1984 »

kristopolous wrote: post #66
hanatanaka1984 wrote: post #65

Earlier quoted context omitted.

Interesting, I will be sure to check into this. I have been using llm and bat with syntax highlighting.

Do you just do | bat --language=markdown --force-colorization ?

| bat -p -l md

simple and works well.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#75

Post by th0ma5 »

davely wrote: post #21
th0ma5 wrote: post #9

Earlier quoted context omitted.

[flagged]

I work for a tech company you’ve definitely heard of. I use the “llm” tool every single day. You may not know it, and that’s okay, but Simon’s tools are an immense help to tons of developers using LLMs. I know it’s fun and trendy to hate on LLMs, but if you’re not productive with them at this point, you’re either: 1. Working on a novel problem or in some obscure language, or 2. Have a skill issue related to how to ut…

They are described as being useful for novel problems, but no matter the vendor or if it is an agentic system I set up, or I watch a reasoning model prattle on, I understand those things as filters that are limiting the conceptual search space, and regardless, it is very easy to bump up against the limits. I understand the use as a rubber duck, that's fine, but this cult like belief that we can't criticize is out of control. My skill issue is that I keep trying to use all of these skills and yet I don't have this default pro-LLM belief which seems to be the requirement. Just today I got multiple models to invent QEMU configuration items that don't exist while trying to solve my problem, which I guess I have to say now is novel by your list here, but it was also something pretty easily found in the documentation I later found out... and even knowing that I wasn't able to get the models to understand that even when I explicitly gave them that information. I've had other experiences like trying to squeeze a watermelon seed. At this point, there is just too much risk of anything they produce giving me a wild goose to hunt down. It is absolutely maddening, but also the people telling me I need to pray about it aren't helpful. These things have not fundamentally improved since they did an impression of D&D games, but I can totally see why people would think that. They approach a database with a natural language query interface but that implies that it knows the context of your language and that they have the data, and when they don't, they make it very difficult to find their errors because they are so adjacent to correct.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#76

Post by hanatanaka1984 »

kristopolous wrote: post #66
hanatanaka1984 wrote: post #65

Earlier quoted context omitted.

Interesting, I will be sure to check into this. I have been using llm and bat with syntax highlighting.

Do you just do | bat --language=markdown --force-colorization ?

A simple bash script provides quick command line access to the tool. Output is paged syntax highlighted markdown.

  echo "$@" | llm "Provide a brief response to the question, if the question is related to command provide the command and short description" | bat --plain -l md
Lauch as:

  llmquick "why is the sky blue?"

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#77

Post by th0ma5 »

simonw wrote: post #18
th0ma5 wrote: post #16

Earlier quoted context omitted.

"LLM was used to find" is not what they did > had I used o3 to find and fix the original vulnerability I would have, in theory [...] they ran a scenario that they thought could have lead to finding it, which is pretty much not what you said. We don't know how much their foreshadowing crept into their LLM context, and even the article says it was also sort of chance. Please be more precise and don't give into these fa…

I said "LLM was also used recently in that..." which is entirely true. They used my LLM CLI tool as part of the work they described in that post.

Very fair, I expect others to confuse what you mean productivity of your tool called LLM vs. the doubt that many have on the actually productivity of LLM the large language model concept.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#78

Post by minimaxir »

th0ma5 wrote: post #72
simonw wrote: post #43

Earlier quoted context omitted.

Are you an alt for th0ma5?

They are not. I think it is a good criticism though. Many people seem to be touting productivity that is only in the context of productivity towards more LLM inference operations and not productive in the sense of solving real world computing problems. There is a lot of material that suggests positive results are a kind of wish casting and people are not aware of the agency they are bringing to the interaction. The f…

No one is happy about the need for prompt engineering and other LLM hacks, but part of being a professional in the real world is doing what works.

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#79

Post by simonw »

roxolotl wrote: post #73
simonw wrote: post #64

Earlier quoted context omitted.

How many plugins do you have installed? We've seen problems in the past where plugins with expensive imports (like torch) slow everything down a lot: https://github.com/simonw/llm/issues/949 I'm interested in tracking down the worst offenders and encouraging them to move to lazy imports instead.

I’ve only got the Anthropic and Gemini plugins installed. I’d be happy to do a bit more digging. I’m away for a bit but would be happy to file an issue with more context when I get a chance.

Try running this and see if anything interesting comes out of it:

  sudo uvx py-spy record -o /tmp/profile.svg -- llm --help

Re: Show HN: My LLM CLI tool can run tools now, from Python code or plugins

#80

Post by kristopolous »

hanatanaka1984 wrote: post #76
kristopolous wrote: post #66

Earlier quoted context omitted.

Do you just do | bat --language=markdown --force-colorization ?

A simple bash script provides quick command line access to the tool. Output is paged syntax highlighted markdown. echo "$@" | llm "Provide a brief response to the question, if the question is related to command provide the command and short description" | bat --plain -l md Lauch as: llmquick "why is the sky blue?"

I've got a nice tool as well

https://github.com/day50-dev/llmehelp/blob/main/Snoopers/wtf

I've thought about redoing it because my needs are things like

   $ ls | wtf which endpoints do these things talk to, give me a map and line numbers.
What this will eventually be is "ai-grep" built transparently on https://ast-grep.github.io/ where the llm writes the complicated query (these coding agents all seem to use ripgrep but this works better)

Conceptual grep is what I've wanted my while life

Semantic routing, which I alluded to above, could get this to work progressively so you quickly get adequate results which then pareto their way up as the token count increases.

Really you'd like some tampering, like a coreutils timeout(1) but for simplex optimization.

Post reply on HN