A simpler question that seems to stump GPT4, llama3 (8b and 70b) so far - https://twitter.com/srikumarks/status/1784214593146868071
What can LLMs never do?
221–230 of 385 posts
Re: What can LLMs never do?
#222"The models, in other words, do not well generalise to understand the relationships between people." Curiously, the need to do this well - handling the quadratic complexity of a shifting set of human relationships, grudges, and alliances - is thought to be one of the things that led is to higher levels of intelligence.
Just to be clear, these models can answer questions about relationships between people if you mean family relationships.
Answering questions about what you're describing sounds really interesting. What would a training set be like that describes a bunch of complex human relationships and then asks questions about them with objective answers?
Re: What can LLMs never do?
#223Earlier quoted context omitted.
I couldn't understand the original wording either, but after reading one of the sibling comments that explains it, it suddenly made sense. I think you left out a few words that most English writers would include. So instead of: > "ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text", something like the following might be better for most people: > "ensure that th…
Much better, but still missing "than" after "greater", which seems kind of critical. "Using" is important as a number greater than ten can't be written as a digit, but can be written using digits ("with" would be just as good). Repeating "written" makes it clearer that there are two instructions.
It's funny, I didn't notice the missing "than" until much later. After I learned the intended meaning of the original sentence, my mind just seemed to insert the missing "than" automatically.
Re: What can LLMs never do?
#224My biggest concern with LLMs in programming, a complete loss of context. Unless the model is regularly trained on the latest codebase, code will always be generated in isolation. No real architectural decisions made with regards to reuse or testability, and no consideration for how the code will be used in 6 months or why the existing code is the way it is.
To anyone using LLMs for meaningful code, I wish you luck maintaining the code long term and hope you really do enjoy doing code reviews.
Re: What can LLMs never do?
#225A simpler question that seems to stump GPT4, llama3 (8b and 70b) so far - https://twitter.com/srikumarks/status/1784214593146868071
An analogy I made to friend - language models capture the constraints in the arrangement of tokens in streams of communication. LLMs that model the constraints placed by human intelligence on token streams can no more be said to have attained (human)intelligence than physicists who decode the constraints placed by a god-like intelligence on the universe can be said to have attained god-like intelligence themselves. (Using comments by theist physicists to the tune of "deciphering the mind of God")
Re: What can LLMs never do?
#226I love when people propose concrete claims like this: if they're wrong, they're disprovable. If they're right, you get unique and interesting insights from the attempts to disprove them.
I suspect these are all tokenization artifacts, but I'll probably take some time to try out the Conway's Game of Life problem by finetuning a model. A few issues I've noticed from the problems proposed in the article:
1. Wordle. This one TBH is a clear tokenization problem, not a proof of the reasoning capabilities of LLMs or lack thereof. LLMs are trained on multi-character tokens, and consume words as multi-character tokens: they don't "see" characters. Wordle is primarily a game based around splitting words into discrete characters, and LLMs can't see the characters they're supposed to operate on if you give them words — and depending on how you structure your answers, they also might not be able to see your answers! By breaking the words and answers into character-by-character sequences with spaces in between the characters (forcing the tokenizer into breaking each character into a separate token visible to the LLM), I successfully got GPT-4 to guess the word "BLAME" on my first attempt at playing Wordle with it: https://chat.openai.com/share/cc1569c4-44c3-4024-a0c2-eeb498...
2. Conway's Game of Life. Once again, the input sequences are given as a single, long string with no spacing, which will probably result in it being tokenized and thus partially invisible to the LLM. This one seems somewhat annoying to prompt, so I haven't tried yet, but I suspect a combination of better prompting and maybe finetuning would result in the LLM learning to solve the problem.
Similarly, complaints about finetuned models not being able to generalize well on input sequences of lengths longer than they were trained on are most likely token-related. Each token an LLM sees (both during training and inference) is encoded alongside its absolute position in the input sequence; while you as a human being see 1 and 1 1 and 1 1 1 as repeated series of 1s, an LLM would see those characters as being at least somewhat distinct. Given a synthetic dataset of a specific size, it can start to generalize over problems within the space that it sees, but if you give it new data outside of that context space, the new data will not be visible to the LLM as being necessarily related to what it was trained on. There are architectural tricks to get around it (e.g. RoPE scaling), but in general I wouldn't make generalizations about what models can or can't "reason" about based on using context window sizes the model didn't see during training: that's more about token-related blindspots and not about whether the model can be intelligent — at least, intelligent within the context window it's trained on.
One thing the author repeats several times throughout the article is that the mistakes LLMs make are far more instructive than their successes. However, I think in general this is not the case: if they can succeed sometimes, anyone who's spent much time finetuning knows that you can typically train them to succeed more reliably. And the mistakes here don't necessarily seem instructive at all: they're tokenization artifacts, and rewriting the problem to work around specific types of blindness (at least in Wordle's case) seems to allow the LLMs to succeed.
FWIW, the author brings up Victor Taelin's famous A::B problem; I believe I was the first to solve it [1] (albeit via finetuning, so ineligible for the $10k prize; although I did it before the prize was announced, just for the pleasure of playing around with an interesting problem). While I think that it's generally a useful insight to think of training as giving more intuition than intelligence, I do think the A::B problem getting solved eventually even by pure prompting shows that there's actually intelligence in there, too — it's not just intuition, or stochastic parroting of information from its training set. However, tokenization issues can easily get in the way of these kinds of problems if you're not aware of them (even in the winning Clause 3 Opus prompt slightly rephrased the problem to get it to work with the tokenizer), so the models actually can appear dumber than they really are.
1. https://twitter.com/reissbaker/status/1776531331562033453
Re: What can LLMs never do?
#227Earlier quoted context omitted.
LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "und…
> the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?
No.
We know how current deep learning neural networks are trained.
We know definitively that this is not how brains learn.
Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically.
This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.
Re: What can LLMs never do?
#228A simpler question that seems to stump GPT4, llama3 (8b and 70b) so far - https://twitter.com/srikumarks/status/1784214593146868071
An analogy I made to friend - language models capture the constraints in the arrangement of tokens in streams of communication. LLMs that model the constraints placed by human intelligence on token streams can no more be said to have attained (human)intelligence than physicists who decode the constraints placed by a god-like intelligence on the universe can be said to have attained god-like intelligence themselves. (…
> language models capture the constraints in the arrangement of tokens in streams of communication
Yes but ultimately that includes all of math, logic, science, physics, etc. which as far as we can tell are fundamental truths of the universe. And if there's a large enough LLM that can capture enough constraints, functionally what's the difference between its intelligence and ours?
Re: What can LLMs never do?
#229Earlier quoted context omitted.
> the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?
No. We know how current deep learning neural networks are trained. We know definitively that this is not how brains learn. Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically. This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.
> We know definitively that this is not how brains learn.
Ok then, I guess the case is closed.
> an entity needs to be able to form new memories dynamically.
LLMs can form new memories dynamically. Just pop some new data into the context.
Re: What can LLMs never do?
#230“Never” is a long time. I wouldn’t bet on that. It pays to remember that even the oldest SOTA model, GPT-4 is only just over a year old.