Oh, I thought this was going to be a semi-rigorous discussion on computability. It's actually just another "GPT-4 still fails at these prompts" essay.
What can LLMs never do?
41–50 of 385 posts
Re: What can LLMs never do?
#42I had found that GPT4 couldn’t play wordle about a year ago [1]. At the time, I thought it must be because it wasn’t in the training data but now it seems to point to something larger.
I might just get nerd sniped trying to teach it GoL now…
Re: What can LLMs never do?
#43Reason.
LLMs mimic human language which is separate from reasoning. Tech bros are remarkably ignorant of the field of linguistics and don’t appreciate this distinction. They thus mistake the output of LLMs for reason.
Re: What can LLMs never do?
#44> This ‘goal drift’ means that agents, or tasks done in a sequence with iteration, get less reliable. It ‘forgets’ where to focus, because its attention is not selective nor dynamic.
I don't know if I agree with this. The attention module is specifically designed to be selective and dynamic, otherwise it would not be much different than a word embedding (look up "soft" weights vs "hard" weights [1]). I think deep learning should not be confused with deep RL. LLMs are autoregressive models which means that they are trained to predict the next token and that is all they do. The next token is not necessarily the most reasonable (this is why datasets are super important for better performance). Deep RL models on the other hand, seem to be excellent at agency and decision making (although in restricted environment), because they are trained to do so.
[1] https://en.wikipedia.org/wiki/Attention_(machine_learning)
Re: What can LLMs never do?
#45The article should be titled " What can LLM never do, yet". By definition, Large Language Models would keep growing larger and larger, to be trained on faster and more advanced hardware, and certain points like "completing complex chains of logical reasoning" tasks, would be just a time hurdle. Only time will tell.
I think that the article is correct. There are indeed things that LLMs will never be able to do, at least not consistently, however much the hardware improves or on how much more material they are trained.
How come? Note my emphasis on the 2nd 'L'. I'm not saying that there are things that AI models will never be able to do, I'm saying that there are things that Large Language Models will be unable to do.
Training LLMs is often argued to be analogous to human learning, most often as a defence against claims of copyright infringement by arguing that human creativity is also based on training from copyrighted materials. However, that is a red herring.
The responses from ever more powerful LLMs are indeed impressive, and beyond what an overwhelming majority of us believed possible just 5 years ago. They are nearing and sometimes surpassing the performance of educated humans in certain areas, so how come I can argue they are limited? Consider it from the other side: how come an educated human can create something as good as an LLM can when said human's brain has been "trained" on an infinitesimal fraction of the material which was used to train even the 1st release of ChatGPT?
That is because LLMs do not learn nor reason like humans: they do not have opinions, do not have intentions, do not have doubts, do not have curiosity, do not have values, do not have a model of mind — they have tokens and probabilities.
For an AI model to be able to do certain things that humans can do it needs to have many of those human characteristics that allow us to do impressive mental feats having absorbed barely any training material (compared to LLMs) and being virtually unable to even remember most of it, let alone verbatim. Such an AI model is surely possible, but it needs a completely different paradigm from straightforward LLMs. That's not to say however that a Language Model will almost certainly be an necessary module of such an AI, but it will not be sufficient.
Re: What can LLMs never do?
#46Earlier quoted context omitted.
> ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text I can’t fault llms for not knowing what to do here because I, a human, have no idea what on earth this means.
“Ten” is a word, “10” are digits. I’m not a native English speaker, how would you write it? FWIW the LLMs get it right many times, but fail other times.
If your not a native English speaker, why are you even expecting the LLM to understand even 80% of the time?
Just ask it in your own native language.
Re: What can LLMs never do?
#47Earlier quoted context omitted.
> ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text I can’t fault llms for not knowing what to do here because I, a human, have no idea what on earth this means.
“Ten” is a word, “10” are digits. I’m not a native English speaker, how would you write it? FWIW the LLMs get it right many times, but fail other times.
I couldn't understand the original wording either, but after reading one of the sibling comments that explains it, it suddenly made sense.
I think you left out a few words that most English writers would include. So instead of:
> "ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text",
something like the following might be better for most people:
> "ensure that the numbers from one to ten are written as words, and the numbers greater ten are written using numerical digits in the given text"
There are multiple ways to write this, so other people may have better versions.
I'm not an English grammar expert, so I cannot explain to you why the addition of those extra words helps with the clarity of that sentence.
Re: What can LLMs never do?
#48It is interesting that all the examples I noticed in this article have a geometric aspect (even wordle - I model it as a grid with geometric rules when playing it). I think that the "first grader" comment is actually somewhat illuminating - it takes several years of learning how to navigate in a spatial world before this stuff becomes trivially easy.
The underlying point this article might be that LLMs don't understand the non-textual aspects of a grid. Which is a fair point, they model language, not space. I wouldn't expect text corpuses to explain space either, since possibly literally everyone who can read and write already knows a lot about spatial layouts.
Re: What can LLMs never do?
#49> This ‘goal drift’ means that agents, or tasks done in a sequence with iteration, get less reliable. It ‘forgets’ where to focus, because its attention is not selective nor dynamic. I don't know if I agree with this. The attention module is specifically designed to be selective and dynamic, otherwise it would not be much different than a word embedding (look up "soft" weights vs "hard" weights [1]). I think deep lea…
LLMs are fine-tuned with RL too. They are NOT simply next token predictors. RLHF uses whole answers at once to generate gradients, so it is looking further into the future. This might not be perfect but it is clearly more than focusing just 1 token ahead.
In the future the RL part of LLM training will increase a lot. Why am I saying this? There are two sources for learning - the past and the present. Training on human text is using past data, that is off-policy. But training on interactive data is on-policy. There is nothing we know that doesn't come from the environment. What is not written in any books must be learned from outside.
That is why I think supervised pre-training from human text is just half the story and RL based agent learning, interactivity in other words, is the next step. The two feed on which intelligence stands are language (past experience) and environment (present experience). We can't get ahead without both of them.
AlphaZero showed what an agent can learn from an environment alone, and LLMs show what they can learn from humans. But the world is big, there are plenty of environments that can provide learning signal, in other words feedback to LLMs.
Re: What can LLMs never do?
#50Earlier quoted context omitted.
I really hate this reductive, facile, "um akshually" take. If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. That's like saying "humans aren't supposed to reason, they're supposed to make sounds with their mouths".
> If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. I don't know... you're still describing a talking parrot here, if you'd ask me.
I feel the use of the word "parrot" is unintentionally apt, given that parrots were long thought to be mere mimics but were ultimately shown to have (at least the capacity for) real linguistic understanding.