Live data from Hacker News

Scaffolded LLMs as natural language computers

beren.io

21–26 of 26 posts

Re: Scaffolded LLMs as natural language computers

#22
post #9

I think the issue is that the sentence "Many tasks cannot be specified easily and precisely in computer code but can be described in a sentence or two of natural language" is not, in fact, true. Natural language is a pretty fantastically bad interface for specifying unambiguous, repeatable, and reliable tasks, which is why most technical advancement has involved the introduction of expressive notations that clarify a…

It's pretty often that people tell a programmer what they want in not-that-much natural language and then the programmer makes it happen. You do have to verify the programmer's work and often provide some corrections that may have been the result of unrecognized ambiguity, but it still ends up being vastly less natural language sent to the programmer than the amount of precise language the programmer ultimately specifies.

Re: Scaffolded LLMs as natural language computers

#23
post #9

I think the issue is that the sentence "Many tasks cannot be specified easily and precisely in computer code but can be described in a sentence or two of natural language" is not, in fact, true. Natural language is a pretty fantastically bad interface for specifying unambiguous, repeatable, and reliable tasks, which is why most technical advancement has involved the introduction of expressive notations that clarify a…

Yes, the problem is "easily and precisely" should just be "easily". Natural language tends to be easy but not precise, while computer code is precise but not easy. Obviously something like idiomatic Python or other similar examples are relatively easy, but a very advanced SQL query can be expressed with a relatively concise syntax but unless you are an expert is not at all easy to produce on your own compared to expressing what you want in natural language.

Re: Scaffolded LLMs as natural language computers

#24
post #3

I am interested to see how the concept longterm memory is developed with LLMs. It seems very slow to use fine tuning for this process. On the topic of summarizing and determinism, I wonder if an intermediate bytecode loke language or structured subset of the English language could improve the outcome across models.

I want to see huge context where fine-tuning could happen as something like a digest at the arrival of the end of the context window. so amortized cost is low.

Re: Scaffolded LLMs as natural language computers

#25
> Error correction itself is not new to hardware – huge amounts of research has been expended in creating error correcting codes to repair bit-flips. We will likely need similar ‘semantic’ error correcting codes for LLM outputs to be able to stitch together extended sequences of NLOPs in a highly coherent and consistent way.

Can we dive into the idea of "semantic error correcting codes" in this thread please?

Post reply on HN