Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers Can Do Arithmetic with the Right Embeddings

arxiv.org

Post reply on HN
Print view View original

41–50 of 215 posts

Re: Transformers Can Do Arithmetic with the Right Embeddings

#41

Post by golol »

YeGoblynQueenne wrote: post #7

What is the point of this work? 99% on 100-digit arithmetic means there's a 0% chance anyone will ever use a Transformer as an ALU or anything of the kind. We already know how to hard-code a (literally) infinitely more accurate addition machine. And not only addition: all four arithmetic operations. The technique proposed in the article -imposing a strong inductive bias for addition- kiind of works for multiplication…

Ok you want the general answer? Consider a discrete time Markov process with memory length N on a finite state space. Train a transformer with context length N on sample trajectories with SGD. Can you expect the transformer to become a good approximation for the dynamics of the Markov process? More specifically, suppose your Markov process is generated by some algorithm/Turing machine couple with some random data. Then, can you expect the transformer to learn to emulate the behavior of the underlying Turing machine, even when run on data which was notnin the initial distribution?

Another way to phrase it: Given a physical process that generates discrete time series trajectories, can our current transformer + SGD method learn to emulate the underlying physical processes by observing sample trajectories?

This question can be somewhat mathematically stated but it is quite difficult because there are still some words in there where I used common sense. For example mathematically there will always exist weird counterexamples, so you would have to quantify things very carefully. That's very difficult, so experiments are the best we can do right now.

Hence any instance where transformers fail to learn a Marko process are very interesting. Example: Addition of random numbers.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#42

Post by isaacfung »

Terr_ wrote: post #9
infogulch wrote: post #2

The other day I was wondering if LLMs are bad at at maths because they don't have readily apparent access to the concept of "columns". Apparently the answer is yes. Vertical alignment across lines is pretty important for humans to learn operations on digits, but the way we encode lines with a \n separator doesn't really help. In a recent codebullet video gpt really struggled with any kind of vertical alignment task.…

Isn't it more that they don't have ready access to the much-more-fundamental concept of decimal numbers? My understanding was that they tokenized them into chunks and tried to learn associations between the chunks, the same as if one was breaking apart English words. So "2+2=4" isn't being treated that differently from "all's well that ends well." This might lead to a kind of Benny's Rules [0] situation, where suffic…

The current gen llms tokenize numbers digit by digit unlike earlier llms.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#44

Post by psychoslave »

short_sells_poo wrote: post #37
sshine wrote: post #19

Earlier quoted context omitted.

> Problem is the current systems can’t reason about things Sounds like the AGI argument trap: They're not able to reason, but we can't succintly define what it is. I don't come with a reasoning chip. Whatever I call reasoning happens as a byproduct of my neural process. I do think that the combination of a transformer network and calls to customized reasoning chips (systems that search and deduce answers, like Wolfra…

I suppose it's a question whether what we call "reasoning" is an emergent phenomenon from having enough connections in a graph, or whether it's some other special sauce which we simply don't have in our current models yet. E.g. humans follow a deductive process to answer questions which they haven't encountered yet. Do we gain this ability purely from a denser/larger graph of knowledge, or from a completely different…

> E.g. humans follow a deductive process to answer questions which they haven't encountered yet.

Rarely, actually.

More generally humans use all kind of inferences where problem at hand is intertwined with all other attention points that is occupying the mental load of the person. Giving a topic full mental attention and finding a path through pure deduction about a circumscribed subject is a rarity, even if you consider only those situations that require any conscious attention at all to perform some action before moving on.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#45

Post by itchyjunk »

psychoslave wrote: post #38
msoad wrote: post #4

It seems like a hack to be honest. Problem at hand is not to make transformers do addition of 100 digit numbers. Problem is the current systems can’t reason about things, math included. Optimizing for a certain use case is not gonna take us where we wanna be. We want to have a system that can learn to reason.

I didn’t test with all LLM out there, but all of thus I tested failed with something as basic as "What is the number of words in the sentence coming before the next one? Please answer."

How many humans have you tested this with?

Re: Transformers Can Do Arithmetic with the Right Embeddings

#46

Post by psychoslave »

itchyjunk wrote: post #45
psychoslave wrote: post #38

Earlier quoted context omitted.

I didn’t test with all LLM out there, but all of thus I tested failed with something as basic as "What is the number of words in the sentence coming before the next one? Please answer."

How many humans have you tested this with?

Interesting point. Would you please answer the question I was mentioning? :)

Re: Transformers Can Do Arithmetic with the Right Embeddings

#47

Post by Last5Digits »

isaacfung wrote: post #42
Terr_ wrote: post #9

Earlier quoted context omitted.

Isn't it more that they don't have ready access to the much-more-fundamental concept of decimal numbers? My understanding was that they tokenized them into chunks and tried to learn associations between the chunks, the same as if one was breaking apart English words. So "2+2=4" isn't being treated that differently from "all's well that ends well." This might lead to a kind of Benny's Rules [0] situation, where suffic…

The current gen llms tokenize numbers digit by digit unlike earlier llms.

They don't. Which you can easily check with any of the dozen web apps currently implementing the GPT-4o tokenizer.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#48

Post by Shrezzing »

simiones wrote: post #31
Shrezzing wrote: post #24

Since models are very good at writing very short computer programs, and computer programs are very good at mathematical calculations, would it not be considerably more efficient to train them to recognise a "what is x + y" type problem, and respond with the answer to "write and execute a small javascript program to calculate x + y, then share the result"?

But that's not all that relevant to the question "can LLMs do math". People don't really need ChatGPT to replace a calculator. They are interested in whether the LLM has learned higher reasoning skills from it's training on language (especially since we know it has "read" more math books than any human could in a lifetime). Responding with a program that reuses the + primitive in JS proves no such thing. Even respond…

That's a fair summary of why the research is happening. Thanks.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#49

Post by logicallee »

sshine wrote: post #19
msoad wrote: post #4

It seems like a hack to be honest. Problem at hand is not to make transformers do addition of 100 digit numbers. Problem is the current systems can’t reason about things, math included. Optimizing for a certain use case is not gonna take us where we wanna be. We want to have a system that can learn to reason.

> Problem is the current systems can’t reason about things Sounds like the AGI argument trap: They're not able to reason, but we can't succintly define what it is. I don't come with a reasoning chip. Whatever I call reasoning happens as a byproduct of my neural process. I do think that the combination of a transformer network and calls to customized reasoning chips (systems that search and deduce answers, like Wolfra…

>They're not able to reason, but we can't [succinctly] define what it is.

People also routinely fail to reason, even programmers often write "obvious" logic bugs they don't notice until it gives an unexpected result at which point it's obvious to them. So both humans and AI don't always reason. But humans reason much better.

I myself have observed ChatGPT 4 solving novel problems I invented to my personal satisfaction well enough to say that it seems to have a rudimentary ability to sometimes show abilities we would typically call reasoning, but only at the level of a child. The issue isn't that it is supposed to reason perfectly or that humans reason perfectly, the issue is that it doesn't reason well enough to succeed at completing many kinds of tasks we would like it to succeed at. Please note that nobody expects it to reason perfectly. "Prove Fermat's last theorem in a rigorous way. Produce a proof that can be checked by Coq, Isabelle, Mizar, or HOL in a format supported directly by any of them" is arguably a request that includes nothing but reasoning and writing code. But we would not expect even Wiles to be able to complete it, and Wiles has actually proved Fermat's last theorem.

So we have an idea of reasoning as completing certain types of tasks successfully, and today humans can do it and AI can't.

Today, it fails badly at tasks that require reasoning. A simple example: https://chatgpt.com/share/da95843e-218a-4d69-a161-6aa2d7a3c9...

The issue is that humans can see its answer is wrong and its "reasoning" is wrong.

The issue isn't that it never reasons correctly. It's that it doesn't do so often enough or well enough, and it doesn't complete tasks we expect humans to complete, and it doesn't always notice when it is printing something outrageously wrong and illogical.

It notices sometimes, it engages in elementary rudimentary guesswork sometimes, but just not often enough or well enough.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#50

Post by Last5Digits »

msoad wrote: post #8
josehackernews wrote: post #5

Earlier quoted context omitted.

how do you argue that these models are not able to reason? deductive reasoning is just drawing specific conclusion from general patterns. something I would argue this models can do (of course not always and are still pretty bad in most cases) the point i’m trying to make is that sometimes reasoning is overrated and put on the top of the cognitive ladder, sometimes I have seen it compared to self-awareness or stuff li…

> how do you argue that these models are not able to reason? I don't make this argument. Benchmarks like CLUTRR[1] show how poorly LLMs do in reasoning. [1] https://github.com/facebookresearch/clutrr

There is a difference between poor reasoning and no reasoning. SOTA LLMs correctly answer a significant number of these questions correctly. The likelihood of doing so without reasoning is astronomically small.

Reasoning in general is not a binary or global property. You aren't surprised when high-schoolers don't, after having learned how to draw 2D shapes, immediately go on to draw 200D hypercubes.

Post reply on HN