It's so mind-boggling to think that our everyday reality can be encoded as weights and biases in a giant matrix. Maybe we are just weights and biases.
Why are LLMs general learners?
41–50 of 64 posts
Re: Why are LLMs general learners?
#42Why do people keep saying its good at math when we have no clue about the training data, and all they do is insert some examples in an unscientific way in a program we have no clue what's behind it or whether its one system or even multiple.
Re: Why are LLMs general learners?
#43Earlier quoted context omitted.
Thanks for responding to my comment. I'm not an expert in the field, but, there are lots of previous algorithms for predicting the next token in a series (Markov chains, autocomplete). None of them felt so much pressure to make an accurate prediction that they had no alternative but to teach themselves arithmetic! It seems what is different about LLMs (as far as the post goes) is that we can anthropomorphize them. Mo…
> what problems an LLM can solve is a function of what it can model. Well said. The model that LLM has is very simple: If text X precedes current conversation then the most likely continuation of discussion is, according to the model held by LLM, Y. Right? So the point is LLM does not create models. It has only a single model based on probabilities of text-sequences, created by its programmers. So it can (mostly?) on…
What would be an example of a difficult problem?
Re: Why are LLMs general learners?
#44LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus, though, because (contrary to the thesis of the article) they don’t magically acquire whatever ability would “simplify” predicting the next token. I don’t feel like the points made here align with any insight about the workings of LLMs. The fact that, as a human, I “wouldn’t know where to start” when asked to add two numbers witho…
[deleted]
Re: Why are LLMs general learners?
#45Earlier quoted context omitted.
Thanks for responding to my comment. I'm not an expert in the field, but, there are lots of previous algorithms for predicting the next token in a series (Markov chains, autocomplete). None of them felt so much pressure to make an accurate prediction that they had no alternative but to teach themselves arithmetic! It seems what is different about LLMs (as far as the post goes) is that we can anthropomorphize them. Mo…
> what problems an LLM can solve is a function of what it can model. Well said. The model that LLM has is very simple: If text X precedes current conversation then the most likely continuation of discussion is, according to the model held by LLM, Y. Right? So the point is LLM does not create models. It has only a single model based on probabilities of text-sequences, created by its programmers. So it can (mostly?) on…
A lot of things that LLMs can already do reliably don't fall into the category of "having a great chat" either. Examples include retrieving data from external sources using commands (known as "plugins" in ChatGPT / Langchain) or writing working code to calculate information needed for answers or to create artifacts, such as charts.
Yes, all of this stems from the task of continuing text. However, more and more, this is veering into the category of behavior. I don't mean "conscious behavior," but "behavior" nevertheless. It's surprising, but it is also the reality in which we currently live.
Re: Why are LLMs general learners?
#46LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus, though, because (contrary to the thesis of the article) they don’t magically acquire whatever ability would “simplify” predicting the next token. I don’t feel like the points made here align with any insight about the workings of LLMs. The fact that, as a human, I “wouldn’t know where to start” when asked to add two numbers witho…
Transformers don’t predict next tokens, right? They predict sequences based on their self-attention to some preceding token sequence?
No, what they do is predict a single token that follows the preceding token sequence (which was indeed analyzed using self-attention). Longer output sequences are created by repeating this simple task multiple times, where the previously output tokens become part of the preceding token sequence.
Re: Why are LLMs general learners?
#47I'm not sure I'm personally convinced LLMs are bad at arithmetic, I think they might just approach it differently to us. Something you'll find if you ever train a neural network to learn a mathematical function is that it will only ever approximate that function. It won't try to guess what the function is exactly like a human might do. For example consider, f(1) = 2, f(2) = 4, f(3) = 6, f(4) = 8, f(5) = 10. As a huma…
LLMs are bad at math because they don't actually understand the rules of math. They can write code to do math, but without code they can only estimate how likely a series of numbers are to be seen together. They're very likely to get things like 2+2=4 correct because that's probably unique and common in their training data. They're unlikely to get two random numbers correct because it doesn't actually know what those…
I think the statement that LLMs don't understand the rules of maths is far too strong. And this notion that LLMs are not able to answer a random arithmetic question "correctly" only holds if you assume "correctness" exists as binary and not a scalar.
I'd propose that your claim that LLMs don't understand at maths is very similar to the claim that Neuton didn't understand the Laws of Motion.
Yes – Neuton's laws are wrong, but they're also practically correct for 99.999% of applications. If correctness is viewed as a binary, Neuron is 100% wrong, but as a scalar Neuron is basically right.
Neural networks are inherently bad at finding exact rules, but they're excellent at approximating them to an accuracy that is acceptably good, this is bit that people miss when they say LLMs can't do maths.
When you claim they don't understand the rules of maths, I agree that they don't understand the explicit rules, but with the caveat that they probably understand something that allows them to approximate those rules "well enough".
This is why if you ask ChatGPT a question like 23435234 + 3243423 it's not going to say -33.1. It might not give the right answer, but it will almost always give you something that's close and very plausible. So while it might not understand the exact rules, it basically understands what happens when you add two numbers and 99% of the time will give you an answer that is basically correct.
The larger point I was trying to make here is that I think we humans are kinda biased when it comes to maths because we understand character precision which is the bias I think you're basing your reasoning on here. We humans believe precision is extremely important in the context of maths unlike other textual content. But an LLM isn't operating with that bias. It's just trying to approximate maths in a way that is correct enough in a similar way that it's trying to approximate the likely next character (or more correctly token) of other text content.
I don't think approximations are 100% wrong and perhaps us humans being bothered about LLMs giving answers to maths questions that are 0.1% wrong actually says more about our values and how we view maths than it says about an LLMs mathematical abilities.
Re: Why are LLMs general learners?
#48Earlier quoted context omitted.
Could it also be that syllables are intrinsically mechanical? They are strongly related to how our mouths work. While it may be possible to extract syllables from written text - following the consonants and vowels - I'm not sure that many humans could easily count syllables without using their mouths.
Many humans are also often really bad at doing speech related things when writing. I've known many native English speakers who write things like "an healthy" (because they learned to write "an" before words starting with "h") and write poems that don't rhyme because the words end with the same letters (e.g. "most" and "cost").
Yeah, I find it weird how LLMs make a lot of the kind of mistakes that people do, but somehow this is held up as being a reason why LLMs don’t work similarly to brains.
Since discovering LLMs I’ve become convinced that my brain works like them. I really don’t know the next word I’m going to say until it’s nearly out. And since learning about how LLMs work, I really can’t argue it away.
It’s a reasonably disturbing feeling.
Re: Why are LLMs general learners?
#49Earlier quoted context omitted.
Current LLM’s are also piss poor general learners, they are however really good at learning specific things which people value highly.
Some 15 years ago, textbooks taught that multi level perceptrons (fully connected feed forward network) with one hidden layer were sufficient because they were universal approximators. That thought kinda held back the field for a long time. Going against that dogma was so revolutionary that new paradigm was given its own name: deep learning. Just because you can find some gotcha counterexample LLM's struggle with doe…
I think that was largely a misunderstanding. 20+ years ago I took an AI class that mentioned using multiple levels was useful for training neural networks. It also mentioned a 2 layer network was only a universal approximator given arbitrarily large numbers of nodes which again seems to be forgotten about.
Though the teacher worked in industry for a while which may have been relevant as we didn’t focus that much on theory.
PS: Deep learning was also more about improving computational power than some major theoretical advancement.
Re: Why are LLMs general learners?
#50> Yet, they demonstrate a crucial point: a deeper understanding of reality simplifies next-token prediction tasks. I'm not sure LLMs are trained to simplify anything. They have billions of parameters after all.
They "simplify" the training data, which they are vastly smaller than. LLMs are like compression algorithms. You could imagine feeding the training data back in, letting it guess the next token, and entropy coding the residual - this would result in an excellent compression ratio. This compression performance is a direct consequence of abstract features of the dataset that it has managed to encode - knowing that the capital of France is Paris allows you to make predictions about many sentences, not just "The capital of France is...".