Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Language models on the command line

simonwillison.net

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Language models on the command line

#31

Post by Jaydenaus »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I've been using this one: https://github.com/egoist/shell-ask

very similar, although the ability to continue a conversation like you can in yours is a killer feature I wish it had.

Re: Language models on the command line

#32

Post by RansomStark »

jillesvangurp wrote: post #30

> We have implemented basic RAG—Retrieval Augmented Generation, where search results are used to answer a question—using a terminal script that scrapes search results from Google and pipes them into an LLM. I love this. Simple and effective. RAG is just search leveled up with LLMs. Such an obvious thing to do. We know how to do search and can use it to unlock vast amounts of knowledge. Instead of letting LLMs dream u…

While I agree with the sentiment in general, I've came to the conclusion that what I really want is the flexibility of the natural language interface that LLM's provide, and the return of the correct document. No 'reasoning', no summarizing, just better search [0].

The issue with the current generation of models is that they can't reason, they may do very well at pretending to reason, but they can't [1]. Reasoning requires the ability to identify and reuse patterns, and while there has been some advancement in this area [2] with getting models to learn the underlying pattern and rule, it doesn't generalize. This results in models that will happily tell you that a statement is both true and false, and be unable to identify the logical problem with that.

Even creating summaries is difficult, and LLM's are more than happy to hallucinate even when summarizing documents, providing incorrect, or entirely made up facts [3]. The general workaround is multiple runs with the same work and averaging the response, but that's a lot of work, and energy.

[0] https://win-vector.com/2024/05/21/i-want-flexible-queries-no...

[1] https://medium.com/@konstantine_45825/gpt-4-cant-reason-2eab...

[2] https://arxiv.org/abs/2405.15071

[3] https://community.openai.com/t/gpt-4o-hallucinating-at-temp-...

Re: Language models on the command line

#33

Post by lynx23 »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I am experimenting with my own tool[1] which is built using asnycio and prompt-toolkit. Made it particularily easy to define function_tools via a decorator which can automagically transform async defs into pydantic models.

[1] https://github.com/mlang/ass

Re: Language models on the command line

#35

Post by librasteve »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

The raku LLM modules are excellent CLI and Jupyter notebook tools. Suggest you start here https://raku.land/?q=LLM%3A%3APrompts and review the comprehensive videos and examples from here https://www.reddit.com/r/rakulang/s/W1UqivfFA9

Re: Language models on the command line

#36

Post by jgalt212 »

Every time I see an LLM demo, I'm blown away. Every time I use one for myself, I feel like a fool.

I say this because the scraper demo bit looks very neat, but I've been down this path before and I don't want to waste my time getting bad or deceptively incorrect results.

Re: Language models on the command line

#37

Post by mark_l_watson »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

My workspace for experimenting with LLMs is chaotic: currently about a 100 Python scripts that each have a single purpose. I sometimes use Ollama or Golamma from the command line. Since I am an old Lisp hacker, I also have a large collection of short Racket and Common Lisp LLM experiments.

Chaos.

Re: Language models on the command line

#38

Post by mark_l_watson »

snthpy wrote: post #26
simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I haven't tried it yet but this appeared a few days ago and I'm a big fan of Textual. https://github.com/paulrobello/parllama

Thanks for sharing that, I also like Textual. I will try parllama and compare it to Gollama.

Re: Language models on the command line

#39

Post by mark_l_watson »

RansomStark wrote: post #32
jillesvangurp wrote: post #30

> We have implemented basic RAG—Retrieval Augmented Generation, where search results are used to answer a question—using a terminal script that scrapes search results from Google and pipes them into an LLM. I love this. Simple and effective. RAG is just search leveled up with LLMs. Such an obvious thing to do. We know how to do search and can use it to unlock vast amounts of knowledge. Instead of letting LLMs dream u…

While I agree with the sentiment in general, I've came to the conclusion that what I really want is the flexibility of the natural language interface that LLM's provide, and the return of the correct document. No 'reasoning', no summarizing, just better search [0]. The issue with the current generation of models is that they can't reason, they may do very well at pretending to reason, but they can't [1]. Reasoning re…

I have an experiment that you can reproduce: use a search API (e.g., Brave or Duckduckgo) and for each result, ask a local LLM to rate it as useful or not useful, then I fetch the entire web pages of ‘useful’ results and ask for summaries made considering the original search query. I like to look at these summaries, and I take one more pass asking for the concatenated summaries to be summarized as a group, asking for concise and de-duplicated final summary.

Anyway, I enjoy playing with this and because I am using my own little Python scripts, I can switch models and hack in it easily.

Re: Language models on the command line

#40

Post by deathmonger5000 »

simonw wrote: post #5

This was a workshop I gave in my https://llm.datasette.io/ CLI tool. What other CLI tools are people using to work with LLMs in the terminal? There one comment here about https://github.com/paul-gauthier/aider and Ollama is probably the most widely used CLI tool at the moment: https://github.com/ollama/ollama/blob/main/README.md#quickst...

I created a CLI tool called Promptr which is an open source developer tool that allows the user to modify their codebase using plain language. The tool sends the user’s query as well as the relevant source code to an LLM. The changes from the LLM are applied directly to the user’s filesystem eliminating the need for copy pasting. Promptr is implemented in Javascript, and it incorporates liquidjs templating so users can build a library of reusable prompt templates for common tasks and contexts.

You can find out more here: https://github.com/ferrislucas/promptr

Post reply on HN