Why are LLMs general learners?

I'm not sure I'm personally convinced LLMs are bad at arithmetic, I think they might just approach it differently to us.

Something you'll find if you ever train a neural network to learn a mathematical function is that it will only ever approximate that function. It won't try to guess what the function is exactly like a human might do.

For example consider, f(1) = 2, f(2) = 4, f(3) = 6, f(4) = 8, f(5) = 10.

As a human you know how important precision is in maths and you know generally humans like round numbers so you naturally assume that, f(x) = x2

Neural networks don't have these biases by default. They'll look for a function that gets close enough maybe something like, f(x) = x1.993929910302942223

From a neural network's perspective the loss between this answer and the actual answer is almost so trivial that it's basically irrelevant.

Then a human who likes round numbers comes along and asks the network, what's f(1,000)? To which the neural network replies, 19939.3

Then the human then goes away convinced the AI doesn't know maths, when in reality the AI basically does know maths, it just doesn't care as much about aromatic precession as the human does. Because again, to the AI 19939.3 is a perfectly acceptable answer.

So now for fun let me ask ChatGPT some arithmetic questions...

> ME

> what's 2343423 + 9988733?

> ChatGPT

> The sum of 2343423 and 9988733 is 12392156.

WRONG! It's actually 12332156. That's an entire digit out and almost 0.5% larger than the actual answer!

> ME

> what is 8379270 + 387299177?

> ChatGPT

> The sum of 8379270 and 387299177 is 395678447.

Er, okay, that was right. Bad example, let me try again.

> ME

> what is 2233322223333 + 387299177?

> ChatGPT

> The sum of 2233322223333 and 387299177 is 2233322610510.

WRONG! It's actually 2233709522510. That's 6 digits out and almost 0.02% smaller than the actual answer!

If you take a more open minded view I think it's fair to say ChatGPT basically does know arithmetic, but its reward function probably didn't prioritise arithmetic precision in the same way a decade of schooling does for us humans. For ChatGPT having a few digits wrong in an arithmetic problem is probably less important that its reply containing that sum being slightly improperly worded.

I guess what I'm saying is that I'm not sure I quite agree with the author that LLMs don't do arithmetic at all. It's not that they're trying to guess the next word without arithmetic, but more that they're not doing arithmetic the same as we humans do it. Which is may have been the point the author was making... I'm not really sure.

> LLMs are not particularly good at arithmetic, counting syllables, or recognizing haikus, though, because (contrary to the thesis of the article) they don’t magically acquire whatever ability would “simplify” predicting the next token.

LLMs understand it to a certain extent. It's more then "predicting" the next token. When people ascribe "predicting the next token" it's a niave and unintelligent description to cover up what they don't understand.

I mean you can describe a human brain as simply wetware, a jumble of signals and chemical reactions that twitch muscles and react to pressure waves in the air and light. But obviously there is a higher level description of the human brain that is missing from that description.

The same thing could be said about LLMs. I can tell you this, researchers completely understand token prediction that much can be said. What we don't currently understand is the high level description. Perhaps it's not something we can understand as we've never been able to understand human consciousness at a high level either.

That's the thing with people. Nobody actually understands the high level description of a fully trained LLM. People are lambasting others because they "think" they understand when they only actually understand the low level primitives. We understand assembly, but you don't understand the Operating system written in assembly.

Take this for example:

     Me: 4320598340958340958340953095809348509348503480958340958304985038530495830 + 1
     chatGPT: 4320598340958340958340953095809348509348503480958340958304985038530495830 + 1 equals 4320598340958340958340953095809348509348503480958340958304985038530495831.

The chances of chatGPT memorizing or even predicting the next tokens here are in a probability too low to even consider. There are so many possible numbers here even numbers that aren't true but have a "higher probability" of being close to the truth from a token/edit-distance standpoint. It's safe to say, from a scientific standpoint, chatGPT in this scenario understands what it means to add 1.

Realize that this calculation results in an overflow. chatGPT needs symbolic understanding to perform the feat it did above.

But there are, of course, things it gets wrong. But again we don't truly understand what's going on here. Is it lying to us? Perhaps it can't differentiate between just a generated statistical token or a actual math equation. It's hard to say. But from the example above, by probability, we know that an aspect of true understanding and ability exists.

I do think that LLMs have emergent properties that do some interesting things, however I would like to point out simple next token prediction would work on your example quite well.

0 + 1 -> 1

Even simple attention mechanisms would handle that quite well with enough examples of

LLMs are bad at math because they don't actually understand the rules of math.

They can write code to do math, but without code they can only estimate how likely a series of numbers are to be seen together.

They're very likely to get things like 2+2=4 correct because that's probably unique and common in their training data. They're unlikely to get two random numbers correct because it doesn't actually know what those numbers mean.

This number is tokenized as a list: 43 20 59 83 409 58 340 9 58 340 95 30 95 809 34 850 9 34 850 34 809 58 340 9 58 30 49 850 385 30 49 58 30. If GPT recognizes the context "+ 1 equals" through the attention mechanism, it can predict that the next number in the sequence should be 31: ... 58 30 -> ... 58 31