Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

131–140 of 385 posts

Re: What can LLMs never do?

#131

Post by jncfhnb »

dragonwriter wrote: post #120
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

Re: What can LLMs never do?

#132

Post by CamperBob2 »

ccppurcell wrote: post #128
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

I would argue that the G in AGI means it can't require better prompting.

We should probably draw a distinction between a human-equivalent G, which certainly can require better prompting (why else did you go to school?!) and god-equivalent G, which never requires better prompting.

Just using the term 'General' doesn't seem to communicate anything useful about the nature of intelligence.

Re: What can LLMs never do?

#133

Post by zer00eyz »

_wire_ wrote: post #119
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

Yes:

An LLM isnt a model of human thinking.

An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI.

That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the potential to make our day to day lives easier (think digital personal assistant that has current knowledge).

Re: What can LLMs never do?

#134

Post by HarHarVeryFunny »

_wire_ wrote: post #119
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

I'd guess because the Transformer architecture is (I assume) fairly close to the way that our brain learns and produces language - similar hierarchical approach and perhaps similar type of inter-embedding attention-based copying?

Similar to how CNNs are so successful at image recognition, because they also roughly follow the way we do it too.

Other seq-2-seq language approaches work too, but not as good as Transformers, which I'd guess is due to transformers better matching our own inductive biases, maybe due to the specific form of attention.

Re: What can LLMs never do?

#135

Post by dragonwriter »

ccppurcell wrote: post #128
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

I would argue that the G in AGI means it can't require better prompting.

That would like saying that because humans’ output can be better or worse based on better or worse past experience (~prompting, in that it is the source of the equivalent of “in-context learning”), humans lack general intelligence.

Re: What can LLMs never do?

#136

Post by SomeCallMeTim »

_wire_ wrote: post #119
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

LLMs are good at tasks that don't require actual understanding of the topic.

They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data.

What they're doing doesn't really match any definition of "understanding." An LLM (and any current AI) doesn't "understand" anything; it's effectively no more than a really big, really complicated spreadsheet. And no matter how complicated a spreadsheet gets, it's never going to understand anything.

Not until we find the secret to actual learning. And increasingly it looks like actual learning probably relies on some of the quantum phenomena that are known to be present in the brain.

We may not even have the science yet to understand how the brain learns. But I have become convinced that we're not going to find a way for digital-logic-based computers to bridge that gap.

Re: What can LLMs never do?

#137

Post by choeger »

zer00eyz wrote: post #133
_wire_ wrote: post #119

Earlier quoted context omitted.

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI. That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the pot…

Great comment. Just one thought: Language, unlike weather, is meta-circular. All we know about specific words or sentences is again encoded in words and sentences. So the embedding encodes a subset of human knowledge.

Hence, a LLM is predicting not only language but language with some sort of meaning.

Re: What can LLMs never do?

#138

Post by moffkalast »

keskival wrote: post #129

All points described are simple artifacts of tokenization.

> If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse.

This is not a tokenization artefact. And furthermore it's a problem for human brains as well.

Let's say you get a name, idk, Tom Cruise. You immediately know what his face looks like. Now let's say you get a random face. How quickly would you be able to tell me what that person is named? Likely a lot of "uhhs" and "ermms" will follow. It's super hard for us to generalize this reversal automatically in lots of cases. Associations tend to be one directional.

Re: What can LLMs never do?

#139

Post by cs702 »

esafak wrote: post #109
cs702 wrote: post #60

Earlier quoted context omitted.

As an aside, at one point I experimented a little with transformers that had access to external memory searchable via KNN lookups https://github.com/lucidrains/memorizing-transformers-pytorc... ( great work by lucidrains) or via routed queries with https://github.com/glassroom/heinsen_routing (don't fully understand it; apparently related to attention). Both approaches seemed to work, but I had to put that work on ho…

> transformers that had access to external memory searchable via KNN lookups This is common, and commonly called retrieval augmented generation , or RAG. edit: I did not pay attention to the link. It is about Wu et al's "Memorizing Transformers", which contain an internal memory.

No. RAG is about finding relevant documents/paragraphs (via KNN lookups of their embeddings) and then inserting those documents/paragraphs into the input context, as sequences of input tokens. What I'm talking about is different: https://arxiv.org/abs/2203.08913

Re: What can LLMs never do?

#140

Post by kromem »

LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't being proven false in under a day, but somehow doesn't think maybe their own can't examples are on similarly thin ice?

In particular, the line about "what models can't do tells us what they don't know" is infuriating.

No, that's not the case at all. At least in a number of instances, what they can't do is because of what they do know.

As an example, one of thecan'ts I got from HN a year ago for GPT-4 was a variation of a classic logic puzzle. And indeed, the model can't solve it - nor can most major models since.

But it's not because the model can't solve the logic - it's because the token similarity to the standard form biases the output towards the standard solution. A hack as simple as changing the nouns to emojis can allow the model to get the correct answer and work through the logic successfully every attempt because it breaks that similarity bias.

People are way too confident around a topic where what's 'known' is more mercurial than maybe any field since 1930s particle physics.

I'd strongly recommend deleting 'never' or 'can't' from one's vocabularies on the subject unless one enjoys ending up with egg on their faces.

Post reply on HN