Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why can't transformers learn multiplication?

arxiv.org

Post reply on HN
Print view View original

101–110 of 111 posts

Re: Why can't transformers learn multiplication?

#101

Post by razodactyl »

arbot360 wrote: post #18
serced wrote: post #14

Earlier quoted context omitted.

Yes, I also wonder about this! Progress from children books to scientific papers etc. Could it learn e.g. language structure faster in a pre-training stage? Also somehow one needs to define a proxy to generalization to compute a loss and do backpropagation.

This field of study is known as "Curriculum Learning" for your Googling pleasure (or I guess ChatGPT Deep Research now).

Yeah. This comment is profound to me. The internet works differently with these tools.

I haven't used the deep research features much but their ability to hash out concepts and build knowledge or even provide an amplified search experience is something...

Re: Why can't transformers learn multiplication?

#102

Post by razodactyl »

fooker wrote: post #62
nico wrote: post #10

Would love to see an architecture that learned more like humans. Start with just imitating one letter, then a few more, than some syllables, then full words, then sentences, etc. Progressively adding on top of previous knowledge Also, it’s interesting that one of the big goals/measures of models is their capacity to “generalize”, but the training methods optimize for loss/accuracy, and only after training test for ge…

There's an interesting question here. Would a single human/entity learn more in ..say.. three million years or would short lived ones evolving over three million years and then ~20 years of education learn more? The current AI tech cycle is focusing on the first, but we don't really know if there are benefits of both. There's no obvious way to combine these yet.

Opinion: a lot can change over such a span of time and knowledge goes in and out of relevance - I think the natural progression of models shrinking in parameter count goes to show it's better to know how to use knowledge than to attempt to remember everything.

That said, optimising for capability of maximal learning seems to be a natural occurrence in nature.

I think the non-obvious emergent effects are something to look into.

Culling bad models in favour of the A/B version and check pointing is a kind of combination of the two and the feedback loop of models trained on new snapshots of Internet data that are written with humans and AI.

There's an unintended long-form training loop which I think is going to get weirder as time goes on.

The wave of models being able to manipulate Cursor / Windsurf etc., being trained to be smarter and more efficient at this and then being retrained for other purposes, even though the model is deleted, the pattern of data can be saved and trained into more advanced models over time.

Re: Why can't transformers learn multiplication?

#103

Post by razodactyl »

nico wrote: post #37
exit wrote: post #34

Earlier quoted context omitted.

"an architecture that learned more like humans" i.e. enduring countless generations of evolutionary selection and cross breeding, then fine-tuning a bit? although it could be interesting, i don't think training on progressively complex strings entirely recapitulates this.

That’s a very interesting take. I hadn’t really considered evolution I guess if you really wanted to start from scratch, you could figure out how to evolve the whole system from a single cell or something like that. In some ways neural networks have kind of evolved in that way, assisted by humans. They started with a single perceptron, and have gone all the way to deep learning and convolutional networks I also remem…

It's this take on the situation which I think needs more emphasis.

Whether anyone likes it or not, these systems have co-evolved with us.

Hundreds of researchers contributing and just like English for example, it's ever-changing and evolving.

Given this trend, it's highly unlikely we won't achieve ASI.

It's not like hardware engineers stop innovating or venture capital stops wanting more. There might be a massive dip or even another AI winter but like the last one, eventually it picks up momentum again because there's clearly utility in these systems.

I've been coding for 25+ years and only a couple of days ago did it hit me that my profession has changed in a very dramatic way - I'm very critical of AI output, but I can read and comprehend code much quicker than I can write it relative to these systems.

Of course, that creates a barrier to holding a system in your head so going slow is something that should be pushed for when appropriate.

Re: Why can't transformers learn multiplication?

#104

Post by razodactyl »

bjt wrote: post #73
llbbdd wrote: post #72

Earlier quoted context omitted.

What is the subtle distinction? I'm "many" and it's not clear at all here. If we had some massive physics breakthrough, the LLM needs to be tought about it, but so do people. Teaching people about it would involve producing a host of documents in some format but that's also true of teaching people. Training and learning here seem to be opposite ends of the same verb no matter the medium, but I'm open to being enlight…

Not sure exactly what the parent comment intended, but it does seem to me that it's harder for an LLM to undergo a paradigm shift than for humans. If some new scientific result disproves something that's been stated in a whole bunch of papers, how does the model know that all those old papers are wrong? Do we withhold all those old papers in the next training run, or apply a super heavy weight somehow to the new one,…

You approach it from a data-science perspective and ensure more signal in the direction of the new discovery. Eg saturating / fine-tuning with biased data in the new direction.

The "thinking" paradigm might also be a way of combatting this issue, ensuring the model is primed to say "wait a minute" - but this to me is cheating in a way, it's likely that it works because real thought is full of backtracking and recalling or "gut feelings" that something isn't entirely correct.

The models don't "know". They're just more likely to say one thing over another which is closer to recall of information.

These "databases" that talk back are an interesting illusion but the inconsistency is what you seem to be trying to nail here.

They have all the information encoded inside but don't layer that information logically and instead surface it based on "vibes".

Re: Why can't transformers learn multiplication?

#105

Post by razodactyl »

ddingus wrote: post #31

A while back I saw a post where people ran a model over and over to accomplish a code base port from one language to another. In their prompt, they told it to leave itself a note and to accomplish something each time. Then they put the model in a loop and it worked. In one instance, a model removed itself from the loop by editing a file or some other basic means. To me, iterative tasks like like multiply and long div…

The feedback from compilation tools / linters fed into the training loops is an example of this.

What we end up with however is a model good at coding for example but bad at something else. And without enough general coding, good at one language over another.

And we're back to square one. The problem of being able to achieve true intelligence by distilling the essence of it not just knowing the answers to specific problems.

Given enough time, we'll plug the gaps and maybe get good enough but it's not true intelligence until it can learn in a way that excels at all fields in a cross-disciplinary way - much better than the side-effect way it's doing now where some other knowledge does actually contribute to achieving goals in other domains.

Re: Why can't transformers learn multiplication?

#106

Post by razodactyl »

hodgehog11 wrote: post #55
CaveTech wrote: post #53

Earlier quoted context omitted.

The latter, and I would disagree that “this works and scales well” in the general sense. It clearly has very finite bounds by the fact we haven’t achieved agi by running an llm in a loop.. The approach of “try a few more things before stopping” is a great strategy akin to taking a few more stabs at RNG. It’s not the same as saying keep trying until you get there - you won’t.

> It clearly has very finite bounds by the fact we haven’t achieved agi by running an llm in a loop.. That's one hell of a criterion. Test-time inference undergoes a similar scaling law to pretraining, and has resulted in dramatically improved performance on many complex tasks. Law of diminishing returns kicks in of course, but this doesn't mean it's ineffective. > akin to taking a few more stabs at RNG Assuming I un…

But test-time inference leads to better data to train better models that can generate better test-time inference data.

There's an obvious trend going on here, of course we're still just growing these systems and going with whatever works.

It's worked well so far, even if it's more convoluted than elegant...

What puts my mind at ease is that the current state of these AI systems isn't going to go backwards because of the data they generate which contributes to the pool of possible knowledge for more advanced systems.

Re: Why can't transformers learn multiplication?

#107

Post by razodactyl »

lossolo wrote: post #42
kovek wrote: post #40

Earlier quoted context omitted.

Well, if the model can reliably keep in context CPU cache plus CPU registers plus CPU instructions and is able to do operations based on those, then we pretty much solved computation using LLMs, right? It could use RAG to operate on RAM and SSD. Here we can see the amount of data a high end traditional non-SOC CPU holds: > For a recent high-end non-SoC desktop CPU: > Cache: ~40-100 MB total (L1 + L2 + shared L3) > Re…

It doesn't work like mapping CPU caches/registers into an LLM context. Transformers have no mutable registers, they attend over past tokens and can't update prior state. RAG isn't RAM. Even with huge context, you still can't step CPU style instructions without an external, read/write memory/tooling. And temperature 0 makes outputs deterministic, not magically correct.

Well mostly but they can generate more state that can push old state out of context.

If an LLM were sufficiently trained to be able to roll-forward and correctly set the current state of some registers written into the conversation..? I wouldn't trust it though, leaves too much to chance.

I too make mistakes trying to keep track of things, I end up using tools too.

Re: Why can't transformers learn multiplication?

#108

Post by razodactyl »

kovek wrote: post #86
lossolo wrote: post #52

Earlier quoted context omitted.

With temp = 0 if the model is off by one bit at step k, all subsequent steps are deterministically wrong. Your previous example shows the best case, which is a model can sometimes follow a textual recipe for long multiplication on short inputs. That's not the same as learning a length generalizing bit exact algorithm. Basically what you shown is the model can describe the algorithm. It doesn't show it can execute it…

> It doesn't show it can execute it at scale. Without writable state and bit exact ops, Well, modern LLM coding agent products (eg. Claude Code) are able to store state in files in the current repository. So, you could have the model keep the "CPU State", and the files in the repository be the "RAM". Also, could this https://arxiv.org/html/2402.17764v1 possibly reduce errors when doing inference? There is no floating…

It seems to be the conclusion that we come to though, we ourselves use tools.

The focus here is the LLM being able to do it unaided.

The space of all combinations of steps is so large for many problems that require precision and usually one incorrect step breaks everything. "I forgot to carry the 1".

Even then, while brilliant, Claude does screw up sometimes - we're not there yet but it doesn't prevent it from being adequately useful.

Re: Why can't transformers learn multiplication?

#109

Post by razodactyl »

carodgers wrote: post #11

Because they produce output probabilistically, when multiplication is deterministic. Why is this so hard for everyone?

Bad take. It's not that it's hard for everyone - there's critical pushback because we don't know for certain if LLM technology can or cannot do the task in question. Which is the reason there's a paper being discussed.

If we were to take the stance of "ok, that happened so it must be the case" we wouldn't be better off in many cases, we would still be accusing people of being witches most likely.

Science is about coming up with a theory and trying to poke holes into it until you can't and in which case, after careful peer-review to ensure you're not just tricking yourself into seeing something which isn't there a consensus is approached in which we can continue to build more truth and knowledge.

Re: Why can't transformers learn multiplication?

#110

Post by jerf »

lacy_tinpot wrote: post #15
jerf wrote: post #5

This is a gut impression and I don't deny it, but LLMs are Large Language Models, and in my own brain, my Language Model isn't doing large-scale multiplication. I have a language-based intuition for the sigle-digit multiplication table and a touch beyond (and based on my observations that's already above average for a human Language Model, at least in my age peer group), but it's not my Language Model doing 283 times…

A lot of savants that are able to do really cool calculations, or even people that have synesthesia seeing numbers as colors, don't actually do "real" calculations. I think most humans that do math aren't actually literally computing things as some kind of logic machine. We can produce logic, and follow the steps of using that logic, but it doesn't seem to me that our cognition is some kind of logic machine itself.

I'm not sure if you think you're agreeing with me or not, but that is my point. Compared to the nominal account of computational power our brains have, we are staggeringly bad at logical manipulation. We extremely expensively and laboriously simulate them.
Post reply on HN