Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are LLMs general learners?

intuitiveai.substack.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Why are LLMs general learners?

#51

Post by Tainnor »

kypro wrote: post #11

I'm not sure I'm personally convinced LLMs are bad at arithmetic, I think they might just approach it differently to us. Something you'll find if you ever train a neural network to learn a mathematical function is that it will only ever approximate that function. It won't try to guess what the function is exactly like a human might do. For example consider, f(1) = 2, f(2) = 4, f(3) = 6, f(4) = 8, f(5) = 10. As a huma…

The exactness matters, though. Unless you'd like things like encryption to stop working.

Re: Why are LLMs general learners?

#52

Post by Tainnor »

kypro wrote: post #47
SkyPuncher wrote: post #17

Earlier quoted context omitted.

LLMs are bad at math because they don't actually understand the rules of math. They can write code to do math, but without code they can only estimate how likely a series of numbers are to be seen together. They're very likely to get things like 2+2=4 correct because that's probably unique and common in their training data. They're unlikely to get two random numbers correct because it doesn't actually know what those…

I think the statement that LLMs don't understand the rules of maths is far too strong. And this notion that LLMs are not able to answer a random arithmetic question "correctly" only holds if you assume "correctness" exists as binary and not a scalar. I'd propose that your claim that LLMs don't understand at maths is very similar to the claim that Neuton didn't understand the Laws of Motion. Yes – Neuton's laws are wr…

You're just trying to redefine "mathematics" in order to be able to say that ChatGPT is good at it. But mathematics is about precision.

Re: Why are LLMs general learners?

#53

Post by Tainnor »

famouswaffles wrote: post #28
wrs wrote: post #23

Earlier quoted context omitted.

It’s actually better at math than it is at arithmetic, and I think this discussion has been about arithmetic. I could make up something about how math is more like language than arithmetic is. I suspect the hypothesis that math tests tend to have a lot of stereotypical problem structures from a shared curriculum is also relevant. But who knows at this point? Anyway, to convince me it’s good at arithmetic is not compl…

>That is do it correctly, every time, for any size number. Then no human is good at arithmetic.

Okay, but we have since invented machines that can do arithmetic correctly, every time. When we try to do maths via an LLM, we're just throwing all of that away.

Re: Why are LLMs general learners?

#54

Post by pgspaintbrush »

dgreensp wrote: post #27
pgspaintbrush wrote: post #16

Earlier quoted context omitted.

Author here. First off, thank you for reading and for your thoughts. I provided examples that I thought would be intuitive for humans to help folks understand that an understanding of the underlying phenomena is useful for next token prediction (I've added this as a note). Could you share what part of the article came across as suggesting that LLMs "magically" acquire whatever ability helps them to predict? I'd like…

Thanks for responding to my comment. I'm not an expert in the field, but, there are lots of previous algorithms for predicting the next token in a series (Markov chains, autocomplete). None of them felt so much pressure to make an accurate prediction that they had no alternative but to teach themselves arithmetic! It seems what is different about LLMs (as far as the post goes) is that we can anthropomorphize them. Mo…

Hmm, yea, I agree with you on several points. For one, we don't fully understand the internal mechanisms of LLMs. I'm also with you on Markov chains and autocomplete tools not having an understanding of the underlying concepts. They merely use statistical patterns in the data.

Based on what you've said, it sounds like your take is that unless we can specify the exact mechanism by which LLMs understand, we have no business saying that they understand. In a lot of cases, this is a reasonable approach. In many areas, if someone tells you X, and you ask for a mechanism of action, and they can't produce one, you have solid grounds for thinking they're bullshitting.

But this case isn't quite the same. We know that LLMs learn to represent their inputs in a high-dimensional vector space (embeddings) and learn the relationships between those vectors. We also see them effectively solve problems in a variety of domains using this representation. I think these two ingredients: having a semantic representation and being able to effectively solve problems amount to something like "understanding." The lack of both properties is why I'd say Markov chains and autocomplete tools don't "understand" -- they haven't learned an effective representation of the underlying phenomena. (I'd also argue this is similar to us as humans. We don't have a good understanding of the human brain or precise mechanisms of action underlying thought. All we know is we as humans have semantic representations and can effectively solve problems.)

small note on your chess point: it now looks like chat gpt 3.5 can achieve draws against stockfish 8: https://marginalrevolution.com/marginalrevolution/2023/06/th...

bigger note on your chess point: this example illustrates that LLMs are "semi-decidable." We thought they were bad at chess, but we just hadn't discovered the right way to prompt. More generally, we can confirm when an LLM is good at X because we feed it a prompt that produces performance in X, but given the size of the input space we're dealing with here, we can't confirm that LLMs are bad at X just because we haven't seen them do well at it. Maybe we just haven't discovered the right prompt. (These input spaces are massive, by the way. ChatGPT-3.5, for example, has a context window of 4,096 tokens, so if we were considering only the English alphabet, we're looking at more than 26^{4,096} possible inputs.)

Re: Why are LLMs general learners?

#55

Post by iliane5 »

rcme wrote: post #9
iliane5 wrote: post #5

Earlier quoted context omitted.

> LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus I suspect most of this is due to tokenization making it difficult to generalize these concepts. There are some weird edge cases though, for example GPT-4 will almost always be able to add two 40 digits number but it is also almost always wrong when adding a 40 digit and 35 digit number.

It doesn't have anything to do with tokenization. You can define binary addition using symbols, e.g. a and b , and provide properly tokenized strings to GPT-4. GPT-4 appears to solve the arithmetic puzzles for a few bits, but quickly falls apart on larger examples.

What I was saying is that because you need to go out of your way to make sure it's tokenized properly, I wouldn't be surprised if there are enough non properly tokenized examples in the dataset.

If that was the case, it would make it difficult to generalize these concepts.

Re: Why are LLMs general learners?

#56

Post by Terr_ »

dgreensp wrote: post #4

LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus, though, because (contrary to the thesis of the article) they don’t magically acquire whatever ability would “simplify” predicting the next token. I don’t feel like the points made here align with any insight about the workings of LLMs. The fact that, as a human, I “wouldn’t know where to start” when asked to add two numbers witho…

> LLMs are not particularly good at arithmetic

I'm reminded of "Benny's Rules", where someone sat down with a "self-directed" 6th grader of high IQ who had been doing okay in math classes... but their success so far was actually based on painstakingly constructing somewhat-lexical rules about "math", mumbo-jumbo that had been just good enough to carry them through a lot of graded tests.

> Benny believed that the fraction 5/10 = 1.5 and 400/400 = 8.00, because he believed the rule was to add the numerator and denominator and then divide by the number represented by the highest place value. Benny was consistent and confident with this rule and it led him to believe things like 4/11 = 11/4 = 1.5.

> Benny converted decimals to fractions with the inverse of his fraction-to-decimal rule. If he needed to write 0.5 as a fraction, "it will be like this ... 3/2 or 2/3 or anything as long as it comes out with the answer 5, because you're adding them" (Erlwanger, 1973, p. 50).

[0] https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conce...

Re: Why are LLMs general learners?

#57

Post by famouswaffles »

Tainnor wrote: post #53
famouswaffles wrote: post #28

Earlier quoted context omitted.

>That is do it correctly, every time, for any size number. Then no human is good at arithmetic.

Okay, but we have since invented machines that can do arithmetic correctly, every time. When we try to do maths via an LLM, we're just throwing all of that away.

So ? I didn't tell you to use GPT-4 for arithmetic over a calculator. I simply pointed out that the only standard where GPT-4 is not good at arithmetic is a standard humans wouldn't fit the bill either. Especially since zero shot "mental" arithmetic is not even close to GPT-4 at its most accurate.

Re: Why are LLMs general learners?

#58

Post by mxkopy »

dTal wrote: post #50
mxkopy wrote: post #6

> Yet, they demonstrate a crucial point: a deeper understanding of reality simplifies next-token prediction tasks. I'm not sure LLMs are trained to simplify anything. They have billions of parameters after all.

They "simplify" the training data, which they are vastly smaller than. LLMs are like compression algorithms. You could imagine feeding the training data back in, letting it guess the next token, and entropy coding the residual - this would result in an excellent compression ratio. This compression performance is a direct consequence of abstract features of the dataset that it has managed to encode - knowing that the…

True, but I still think there's some fallacy here. Are we sure that models of the world (i.e. understanding) are the only way to achieve compression?

Re: Why are LLMs general learners?

#59

Post by Tainnor »

famouswaffles wrote: post #57
Tainnor wrote: post #53

Earlier quoted context omitted.

Okay, but we have since invented machines that can do arithmetic correctly, every time. When we try to do maths via an LLM, we're just throwing all of that away.

So ? I didn't tell you to use GPT-4 for arithmetic over a calculator. I simply pointed out that the only standard where GPT-4 is not good at arithmetic is a standard humans wouldn't fit the bill either. Especially since zero shot "mental" arithmetic is not even close to GPT-4 at its most accurate.

The discussion started "what would it take to convince people that [insert favourite LLM] is good at maths", and the response to that IMHO is that we have much better tools to do arithmetic (I don't even want to say maths), even if humans themselves are also poor at arithmetic.

What's the point of building a system to be equally bad as humans at something that we know humans are bad at? LLMs have their uses but (at least at the current stage) performing arithmetic calculations is not one of them (to say nothing of more advanced mathematics).

Re: Why are LLMs general learners?

#60

Post by clarge1120 »

courseofaction wrote: post #7

Intuitively, I think this also hints at why LLMs get more prone to confusion when trained to be "safe" - the underlying representations for applying human morality in context are much more complex to learn than simpler but potentially psychopathic logic.

This sounds correct. Humans are highly fickle and contradictory when it comes to morality. Even the Golden Rule is hotly contested. LLMs lose touch with reality as they try to navigate humanity’s moral landscape. Our current solution is to align an LLM to a worldview.

The good news is that this will pit one LLM against others, and virtually eliminate any potential for a single powerful AI to emerge and do something harmful.

Post reply on HN