Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why can't transformers learn multiplication?

arxiv.org

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Why can't transformers learn multiplication?

#51

Post by zer00eyz »

nico wrote: post #10

Would love to see an architecture that learned more like humans. Start with just imitating one letter, then a few more, than some syllables, then full words, then sentences, etc. Progressively adding on top of previous knowledge Also, it’s interesting that one of the big goals/measures of models is their capacity to “generalize”, but the training methods optimize for loss/accuracy, and only after training test for ge…

"Would love to see an architecture that learned"

Would be a far more accurate statement. Training != Learning.

Re: Why can't transformers learn multiplication?

#52

Post by lossolo »

kovek wrote: post #44
lossolo wrote: post #42

Earlier quoted context omitted.

It doesn't work like mapping CPU caches/registers into an LLM context. Transformers have no mutable registers, they attend over past tokens and can't update prior state. RAG isn't RAM. Even with huge context, you still can't step CPU style instructions without an external, read/write memory/tooling. And temperature 0 makes outputs deterministic, not magically correct.

Well, the LLM may re-infer the whole state fully on every instruction. Temperature 0 is deterministic and that's what we are looking for. If the model is trained properly on how the CPU state + instructions should be handled, then it should be able to produce the next state.

With temp = 0 if the model is off by one bit at step k, all subsequent steps are deterministically wrong.

Your previous example shows the best case, which is a model can sometimes follow a textual recipe for long multiplication on short inputs. That's not the same as learning a length generalizing bit exact algorithm.

Basically what you shown is the model can describe the algorithm. It doesn't show it can execute it at scale. Without writable state and bit exact ops, errors grow with length and "focus more" only slows that failure, it doesn’t eliminate it.

Re: Why can't transformers learn multiplication?

#53

Post by CaveTech »

hodgehog11 wrote: post #46
CaveTech wrote: post #36

Earlier quoted context omitted.

The amount of paths in the wrong direction are infinitely more than then number in the right direction. You'll quickly realize this doesn't actually scale.

I'm a bit confused by this; are you referring to vanishing/exploding gradients during training or iteration at inference? If the former, this is only true if you take too many steps. If the latter, we already know this works and scales well.

The latter, and I would disagree that “this works and scales well” in the general sense. It clearly has very finite bounds by the fact we haven’t achieved agi by running an llm in a loop..

The approach of “try a few more things before stopping” is a great strategy akin to taking a few more stabs at RNG. It’s not the same as saying keep trying until you get there - you won’t.

Re: Why can't transformers learn multiplication?

#54

Post by rokobobo »

zer00eyz wrote: post #51
nico wrote: post #10

Would love to see an architecture that learned more like humans. Start with just imitating one letter, then a few more, than some syllables, then full words, then sentences, etc. Progressively adding on top of previous knowledge Also, it’s interesting that one of the big goals/measures of models is their capacity to “generalize”, but the training methods optimize for loss/accuracy, and only after training test for ge…

"Would love to see an architecture that learned" Would be a far more accurate statement. Training != Learning.

Do you have an example of an algorithm that learns, rather than is trained/trains itself? I don’t really see the boundary between the two concepts.

Re: Why can't transformers learn multiplication?

#55

Post by hodgehog11 »

CaveTech wrote: post #53
hodgehog11 wrote: post #46

Earlier quoted context omitted.

I'm a bit confused by this; are you referring to vanishing/exploding gradients during training or iteration at inference? If the former, this is only true if you take too many steps. If the latter, we already know this works and scales well.

The latter, and I would disagree that “this works and scales well” in the general sense. It clearly has very finite bounds by the fact we haven’t achieved agi by running an llm in a loop.. The approach of “try a few more things before stopping” is a great strategy akin to taking a few more stabs at RNG. It’s not the same as saying keep trying until you get there - you won’t.

> It clearly has very finite bounds by the fact we haven’t achieved agi by running an llm in a loop..

That's one hell of a criterion. Test-time inference undergoes a similar scaling law to pretraining, and has resulted in dramatically improved performance on many complex tasks. Law of diminishing returns kicks in of course, but this doesn't mean it's ineffective.

> akin to taking a few more stabs at RNG

Assuming I understand you correctly, I disagree. Scaling laws cannot appear with glassy optimisation procedures (essentially iid trials until you succeed, the mental model you seem to be implying here). They only appear if the underlying optimisation is globally connected and roughly convex. It's no different than gradient descent in this regard.

Re: Why can't transformers learn multiplication?

#56

Post by zer00eyz »

rokobobo wrote: post #54
zer00eyz wrote: post #51

Earlier quoted context omitted.

"Would love to see an architecture that learned" Would be a far more accurate statement. Training != Learning.

Do you have an example of an algorithm that learns, rather than is trained/trains itself? I don’t really see the boundary between the two concepts.

If we make some massive physics breakthrough tommrow is an LLM going to be able to fully integrate that into its current data set?

Or will we need to produce a host of documents and (re)train a new one in order for the concept to be deeply integrated.

This distinction is subtle but lost on many who think that our current path will get us to AGI...

That isn't to say we haven't created a meaningful tool but the sooner we get candid and realistic about what it is and how it works the sooner we can get down to the business of building practical applications with it. (And as an aside scaling it, something we arent doing well with now).

Re: Why can't transformers learn multiplication?

#57

Post by jakefromstatecs »

Nifty3929 wrote: post #45

Numbers aren't language, or even sequences of tokens, or vectors. There is an inherent numeric-ness and logic to math that I don't think we can represent well using LLMs and transformers. 3 isn't about the word "three" - it is a quantity or a measurement. And 3x4 is a specific numerical operation that is not really contained in that sequence of symbols.

Math is just symbol manipulation with a set of rules, no?

Re: Why can't transformers learn multiplication?

#59

Post by ares623 »

onlyrealcuzzo wrote: post #49
ares623 wrote: post #48

Earlier quoted context omitted.

Isn’t that what all the hundreds of billions are banking on? “General” intelligence.

You don't need general intelligence to make good memes to keep people scrolling through Instagram. You don't need general intelligence to make a decent coding tool like Cursor. You don't need general intelligence to improve SERPs. You don't need general intelligence to sell a subscription for a decent AI assistant. There's tons of value already added without anything general.

Yes but $500B and counting for memes wasn’t what was sold

Re: Why can't transformers learn multiplication?

#60

Post by reassess_blind »

onlyrealcuzzo wrote: post #49
ares623 wrote: post #48

Earlier quoted context omitted.

Isn’t that what all the hundreds of billions are banking on? “General” intelligence.

You don't need general intelligence to make good memes to keep people scrolling through Instagram. You don't need general intelligence to make a decent coding tool like Cursor. You don't need general intelligence to improve SERPs. You don't need general intelligence to sell a subscription for a decent AI assistant. There's tons of value already added without anything general.

The question is whether, if the models plateau, and "AGI" as it was claimed in the beginning never arrives, if it's enough to justify these ongoing multi-hundred billion dollar deals.

I mean, probably, LLMs as they are today are already changing the world. But I do think a lot of the ongoing investment is propped up on the promise of another breakthrough that is looking less likely.

Post reply on HN