Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transformers Can Do Arithmetic with the Right Embeddings

arxiv.org

Post reply on HN
Print view View original

191–200 of 215 posts

Re: Transformers Can Do Arithmetic with the Right Embeddings

#191

Post by int_19h »

jiggawatts wrote: post #14

Something I've been thinking about is how the Minds -- the super-human AI hyper-computers that fly the ships in the Culture series of novels are described. The image built up in my head[1] is that they're hybrids blending neural networks and regular compute substrates. They can calculate, simulate, and reason in combination. There have been crude attempts at this already, hooking in Mathematica and Python into ChatGP…

The problem, if you embed an ALU like that, is how to train it to use them properly. And then it's not clear if they actually need to be able to do that in the middle of a pass that, at the end, is going to produce a single token anyway.

Controlling that stuff via output tokens actually kinda makes sense by analogy, since that is how we use calculators etc. But I do agree that specialized tokens that are used specifically to activate tools like that might be a better idea than just using plain text to signal in-band. And production of such specialized tokens can be easily trained.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#192

Post by jhanschoo »

lupire wrote: post #158
jhanschoo wrote: post #128

Earlier quoted context omitted.

> They're not able to reason, but we can't succintly define what it is. For transformer-based LLMs, and most LLMs there's an obvious class of problems that they cannot solve. LLMs generally perform bounded computation per token, so they cannot reason about computational problems that are more than linearly complex, for a sufficiently large input instance. If you have a back-and-forth (many shot) your LLM can possibly…

Humans have the same limitation and use same solution: showing your work and taking notes. There's no blocker here.

There is a distinction. Humans with the use of an unbounded scratchpad can emulate a general-purpose Turing machine and perform general computation given unbounded time. A LLM is still restricted to its context window which is a comparatively extreme limitation of memory. In comparison, our general-purpose computers have so much memory this isn't something we care about for most practical instances of hard problems that we solve with a classical CS algorithm. You can obviously modify LLMs to perform unbounded computation per token (and furnish it with a scratchpad) but afaict commercial LLMs today don't offer that.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#193

Post by jhanschoo »

phkahler wrote: post #169
EVa5I7bHFq9mnYK wrote: post #165

Earlier quoted context omitted.

Yes, because it's feed forward. It must have loops to be a Turing machine.

It does. The output is fed back in.

It indeed does, but it must generate a token per loop, and can thereby solve some linearly complex problems, but it cannot solve harder problems.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#194

Post by klabb3 »

jhanschoo wrote: post #128
sshine wrote: post #19

Earlier quoted context omitted.

> Problem is the current systems can’t reason about things Sounds like the AGI argument trap: They're not able to reason, but we can't succintly define what it is. I don't come with a reasoning chip. Whatever I call reasoning happens as a byproduct of my neural process. I do think that the combination of a transformer network and calls to customized reasoning chips (systems that search and deduce answers, like Wolfra…

> They're not able to reason, but we can't succintly define what it is. For transformer-based LLMs, and most LLMs there's an obvious class of problems that they cannot solve. LLMs generally perform bounded computation per token, so they cannot reason about computational problems that are more than linearly complex, for a sufficiently large input instance. If you have a back-and-forth (many shot) your LLM can possibly…

> LLMs generally perform bounded computation per token, so they cannot reason about computational problems that are more than linearly complex, for a sufficiently large input instance.

I can’t judge if this is true, because I don’t know transformers well, but if it is, it unravels an intuitive thought I’ve never been able to articulate about not only LLMs, but possibly all pattern matching and the human analog of System 1 thinking.

Another fuzzy way of saying this is there’s something irreducible about complexity that can’t be pattern matched by any bounded heuristic – that it’s wishful thinking to assume historical data contains hidden higher-level patterns that unlock magical shortcuts to novel problems.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#195

Post by CamperBob2 »

andoando wrote: post #175
CamperBob2 wrote: post #166

Earlier quoted context omitted.

Just our visual object recognition is immensely powerful and far beyond and current AI. That's a point you'll likely have to revisit pretty soon. Radiology, for instance, probably won't exist as a profession 20-30 years from now. Captchas are already pretty much done for.

Well 1. Radiology is an insanely niche subject not indiciative of general intelligence, and 2. AI being at good radiology isn't about object recognition or spatial reasoning, its data analysis connecting features to outcomes. Lastly, check out the ARC challenge or any other spatial reasoning tests for AI. Humans get ~80% on these challenges whereas the best AI is still at 25%

Can you point me towards a citation for the 25% figure? I'm seeing numbers like 96% ( https://paperswithcode.com/sota/common-sense-reasoning-on-ar... ) but I'm guessing that's just for a subset of the larger class of questions.

Also, are you familiar with this study? What are your thoughts on it? https://www.esmo.org/newsroom/press-and-media-hub/esmo-media... Seems like a valid case where AI is competitive with skilled humans at object/image recognition.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#196

Post by imtringued »

uoaei wrote: post #101
vessenes wrote: post #57

Wow, a lot of grumpiness in here. If it's true that adding like 20 or so tokens to encode column location / decimal spot triples math performance in out of band tasks, that's a big deal. It's a simple fix, it improves performance A LOT, and they even indicate it's not just a party trick, in that the LLM can use the information to do better on related tasks like sorting and list making. This is basically free to add,…

It's also obvious and it's hacky. Frankly I'm stunned this hasn't been tried yet. The people thinking this is a stepping stone to More Intelligence are missing the forest for the trees. Deep learning is always and only ever about representing data abstractly . The more abstractions you can make irrelevant (why would you have to learn how to do math when the base-10 perspective on ASCII-digits is already provided for…

You seemingly missed the part where the next model could learn how to generate its own hierarchical position embeddings. The problem here is obviously that you want the model to look at position i in object a and object b where the position i was chosen by a previous layer. If anything, the answer is probably to just have a dynamic position input from the model into the RoPE embedding, then it can learn the ideal position encoding on its own.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#197

Post by YeGoblynQueenne »

famouswaffles wrote: post #179
YeGoblynQueenne wrote: post #150

Earlier quoted context omitted.

Yes, but humans invented arithmetic. And then we invented computers that are much better than us at arithmetic calculations. That's a pattern we can observe all over the place: we're pretty damn good at inventing rich models of complex environments and processes but we're not very good at calculating the results of such models when that requires a lot of computation. E.g., take chess. Modelling a game of chess as a g…

>Yes, but humans invented arithmetic. I think most would argue Mathematics is a discipline that is discovered more than invented. That said, this isn't really the point I think. A few humans invented/discovered arithmetic. Most humans will be born, live and die inventing absolutely nothing, even those with the opportunity and resources to do so. It doesn't make sense to me that a bar most humans can't reach is the ba…

>> Most humans will be born, live and die inventing absolutely nothing, even those with the opportunity and resources to do so.

I don't think that's right at all. I like to visit museums. You really get hit in the face with the unending creativity of the human mind and the variety of all that human hands have crafted over thousands of years across hundreds of cultures. I would go as far as to say that the natural state of the human mind is to create new things all the time. And mathematics itself was not created (invented or discovered) by one person, but by many thousands.

In any case, it doesn't matter if one instance of the class of human minds hasn't invented anything, in the same way that it doesn't matter if one car can't do 80mph. It's indisputable that we have the capacity for some novelty, and generality, in our thinking. Maybe not every member of the species will achieve the same things, but the fact is that the species, as a species, has the ability to come up with never-before seen things: art, maths, tech, bad poetry, you name it.

>> Lecun may disagree but some others like Hinton, Ilya and Norvig don't.

I'm with LeCun and Bengio. There's a fair amount of confusion about what a "model" is in that sense: a theory of the world. There's no reason why LLMs should have that. Maybe a transformer architecture could develop a model of the world- but it would have to be trained on, well, the world, first. Sutskever's bet is that a model can be learned from text generated by entities that already have a world model, i.e. us, but LeCun is right in pointing out that a lot of what we know about the world is never transmitted by text or language.

I can see that in my work: I work with planning, right now, where the standard thing is to create a model in some mathematical logic notation, that is at once as powerful as human language and much more precise, and then let a planning agent make decisions according to that model. It's obvious that despite having rich and powerful notations available there is information about the world that we simply don't know how to encode. That information will not be found in text, either.

Sutskever again seems to think that, that kind of information, can somehow be guessed from the text, but that seems like a very tall order, and Transformers don't look like the right architecture. You need something that can learn hidden (latent) variables. Transformers can't do that.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#198

Post by PeterisP »

uoaei wrote: post #101
vessenes wrote: post #57

Wow, a lot of grumpiness in here. If it's true that adding like 20 or so tokens to encode column location / decimal spot triples math performance in out of band tasks, that's a big deal. It's a simple fix, it improves performance A LOT, and they even indicate it's not just a party trick, in that the LLM can use the information to do better on related tasks like sorting and list making. This is basically free to add,…

It's also obvious and it's hacky. Frankly I'm stunned this hasn't been tried yet. The people thinking this is a stepping stone to More Intelligence are missing the forest for the trees. Deep learning is always and only ever about representing data abstractly . The more abstractions you can make irrelevant (why would you have to learn how to do math when the base-10 perspective on ASCII-digits is already provided for…

ASCII digits do not always imply base-10 numbers, they can also be identifiers (e.g. phone numbers), parts of words (IPv6, Log4j), and used in various 'written slang' such as g2g, 4ever, m8 for mate, etc, etc.

And, crucially, I'd argue that for in "chatbot" tasks those other uses are more common than arithmetic, so arbitrary focus to specifically optimize arithmetic doesn't really make sense - the bitter lesson is that we don't want to bias our architecture according to our understanding of a specific problem space but rather enable the models to learn the problem space directly from data.

Re: Transformers Can Do Arithmetic with the Right Embeddings

#199

Post by PeterisP »

neves wrote: post #109
wrsh07 wrote: post #105

Earlier quoted context omitted.

This is cool, but special casing digits is unsatisfying. It makes me think that the authors have correctly identified an issue (positional embeddings) but don't propose a general solution. I'm not sure if such a thing is possible, but if it is, it would feel more complete. (Fwiw, positional embeddings have had issues for a long time! So a general solution to this would benefit more than just arithmetic. Helpfully, we…

but it makes sense to have a different encoding. Mathematics is a completely different language. Maybe we should have more than one class of encodings.

For arbitrary documents and queries, how do we reliably segment the text between those two different languages? And if we can do that, why can't the model do it implicitly?

Re: Transformers Can Do Arithmetic with the Right Embeddings

#200

Post by mike_hearn »

HarHarVeryFunny wrote: post #70
mike_hearn wrote: post #52

Earlier quoted context omitted.

Generalizable techniques is mostly the point of papers like this one yes. What they show here is that apparently fundamental problems with transformer reasoning can be fixed by encoding data in a more sophisticated manner. This is exciting. I've been thinking for a long time that the tokenization schemes are a low hanging fruit for improving coding LLM performance, this isn't exactly the same thing but it's in the sa…

It seems sub-word tokenization vs using character inputs is just a trade off to gain computational efficiency, and obviously isn't how our brain works. We're not born with a fixed visual tokenization scheme - we learn to create our own groupings and object representations. However, transformers seem to struggle a bit with accurately manipulating sequences, so going to character inputs and hoping for those to be aggre…

Yeah a tiny vocab of characters doesn't work that well, it was tried very early on and creating large vocabs of tokens was a big improvement. Which makes sense. A lot of tokens are full words and so the token->embedding phase can quickly look up an embedding in vector space that contains a lot of meaning, whereas an embedding of 'z' or whatever is going to be meaningless.
Post reply on HN