Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models on the command line

simonwillison.net

Post reply on HN
Print view View original

21–30 of 46 posts

Re: Language models on the command line

#21

Post by evmar »

simonw wrote: post #18
evmar wrote: post #17

Earlier quoted context omitted.

I am using a hacky one I wrote myself. I looked at llm but it doesn't appear to have a mechanism for multi-shot prompting, where you provide both prompts and responses within your query. (Ref https://platform.openai.com/docs/guides/prompt-engineering/t... .) Maybe take this as a feature request? It feels like the 'template' system in llm might be able to encompass this but the docs don't appear to provide a reference…

Yeah, I've been thinking a bit about the multi shot thing. I've had great results from Claude by "faking" the previous conversation to include example question/answer pairs. With LLM you can do that using the undocumented Python Conversation API, but it's undocumented for a reason (I don't think it's good enough yet). You could also fake a previous conversation through the CLI tool but that is VERY undocumented - you…

In case it helps any, here are some more details about what I used it for. (Summary: providing examples of a specific translation task, to make it use a particular API in the output.) https://inuh.net/@evmar/112001414385042731

And what I expected of llm is for the template file to (optionally) contain an array of prompt/response pairs. You could even imagine the --save flag constructing one from an ongoing conversation of llm -c maybe.

Re: Language models on the command line

#22

Post by simonw »

evmar wrote: post #21
simonw wrote: post #18

Earlier quoted context omitted.

Yeah, I've been thinking a bit about the multi shot thing. I've had great results from Claude by "faking" the previous conversation to include example question/answer pairs. With LLM you can do that using the undocumented Python Conversation API, but it's undocumented for a reason (I don't think it's good enough yet). You could also fake a previous conversation through the CLI tool but that is VERY undocumented - you…

In case it helps any, here are some more details about what I used it for. (Summary: providing examples of a specific translation task, to make it use a particular API in the output.) https://inuh.net/@evmar/112001414385042731 And what I expected of llm is for the template file to (optionally) contain an array of prompt/response pairs. You could even imagine the --save flag constructing one from an ongoing conversati…

Yeah, doing this with templates is a great idea.

Re: Language models on the command line

#23

Post by jimmcslim »

simonw wrote: post #4
behnamoh wrote: post #3

I wish llm were more stable, but unfortunately things just kept breaking out of the blue without me touching any settings of the program. I often had to reinstall the package. but finally I gave up and implemented my own.

Which plugins are you using? Did you install via pipx or Homebrew or something else?

I followed along with the blog post, but got unstuck with llm-cmd not working on Mac OS. Looks like this PR would fix it https://github.com/simonw/llm-cmd/pull/12

Re: Language models on the command line

#24

Post by kelvie »

throwup238 wrote: post #8
simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

> What other CLI tools are people using to work with LLMs in the terminal? I use aichat: https://github.com/sigoden/aichat I especially like the terminal integration where I can type a natural language request at the terminal and press Alt+E to have it converted to a command to run.

I also use aichat. For some reason (well, I know the reason, it's shell quoting) I don't like passing prompts in via command line, so having a simple text-based TUI like a readline prompt is nice.

Re: Language models on the command line

#25

Post by nyellin »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

Most of the cli tools just wrap an LLM, but don't give it access to the data it needs to be useful. Aider is an exception of course - it gives great results because it feeds the LLM your source files.

We built http://github.com/robusta-dev/holmesgpt/ to investigate Prometheus/Jira/PagerDuty issues. We're able to get pretty good results (we benchmark extensively) because we use function-calling to give the LLM read acess to relevant data. I think we're the only open source AIOps tool, and the only AIOps tool period that does something more complex than RAG + summarization.

Re: Language models on the command line

#26

Post by snthpy »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I haven't tried it yet but this appeared a few days ago and I'm a big fan of Textual.

https://github.com/paulrobello/parllama

Re: Language models on the command line

#28

Post by godelski »

bagels wrote: post #16
QuiDortDine wrote: post #14

Earlier quoted context omitted.

Do you have a followup command to execute the suggested command?

No, I want to review or tweak them, just in case it's trying to do something bad. It's usually pretty good though.

If you use zsh and are willing to source your scripts you can do

  print -z $command
And the command will appear on your cli as if you had written it.

I don't think you can do this in bash. Interestingly, this is something that seems quite difficult to both google and ask GPT for help. Both get confused and are thinking different questions are being asked. Probably because there are similar more common questions but the subtleties of possible wordings makes it difficult to differentiate.

Re: Language models on the command line

#29

Post by FergusArgyll »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I made my own python wrapper around the Gemini API, I wanted one feature: there is a default system prompt but -s overrides that with a temporary system prompt. -p is for the prompt, if it is left out, the cli is interactive (one long chat until I exit) but with -p I can use regular linux utils

for example: I wrote a short bash script which uses yt-dlp and ffmpeg to download a song, convert it to my preferred format and then uses gemini to add metadata.

  artist=$(gemini -s "Please respond with the name of the Artist based on 
  the songs title. do not use any other words, just the artist name.
  example:
  'Bruce Springsteen - Old Dan Tucker [S-GHbDFrwlU].opus'
  Bruce Springsteen" -p "$opus_file" | tr -d '\n')

Re: Language models on the command line

#30

Post by jillesvangurp »

> We have implemented basic RAG—Retrieval Augmented Generation, where search results are used to answer a question—using a terminal script that scrapes search results from Google and pipes them into an LLM.

I love this. Simple and effective. RAG is just search leveled up with LLMs. Such an obvious thing to do. We know how to do search and can use it to unlock vast amounts of knowledge. Instead of letting LLMs dream up facts by compressing all knowledge into them, a better use of them is letting them summarize and reason about the facts it finds. IMHO the art is actually going to be in letting them come up with the right query as well. Or queries. It could be a lot more exhaustive in its searches than we could be.

Post reply on HN