Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Smallest transformer that can add two 10-digit numbers

github.com

Post reply on HN
Print view View original

61–70 of 105 posts

Re: Smallest transformer that can add two 10-digit numbers

#61

Post by wtallis »

reerdna wrote: post #53

I couldn't help but laugh out loud at the notion of a "held-out test set" for addition of 10-digit numbers.

I don't think we have good tools for formally proving that a transformer's output will match a more traditionally-defined function. But the leading transformers are small enough that formal verification may be possible.

Without any formal verification: The input space of two 10-digit numbers is a bit bigger than 64-bits, so exhaustively verifying all possible inputs doesn't sound practical. Using the same subset of the input space for verifying each submission seems like the easiest way to be fair, and not disclosing that subset to the competitors is obviously necessary.

Re: Smallest transformer that can add two 10-digit numbers

#62

Post by gaigalas »

akoboldfrying wrote: post #60
computersuck wrote: post #36

this is the dumbest fking thing to do math with

Yes, but it's interesting that you can teach it to do arithmetic, don't you think? Most things can't be taught to do arithmetic, making this "transformer" thing slightly magical. And so then it seems interesting to investigate exactly how much magic is needed to achieve this.

In theory, there is an infinite number of systems with simple emergent rules that can eventually be taught arithmetic.

Re: Smallest transformer that can add two 10-digit numbers

#63

Post by alexlitz »

ks2048 wrote: post #48
alexlitz wrote: post #17

I made a blogpost on my submission (currently the top handwritten one at 36 parameters) https://alexlitzenberger.com/blog/building_a_minimal_transfo...

I didn't look at all the details, but wanted to see how you did the initial embedding and see you do have a 14x5 matrix there. I guess when you are setting things by-hand (rather than learning), the definition of counting "parameters" is a bit unclear. One could say all those are parameters! even if setting in a straight-forward way.

Yeah basically it is an implementation detail but most of them are zero, there is an equivalent 14 parameter sparse matrix for that.

Re: Smallest transformer that can add two 10-digit numbers

#64

Post by benob »

i000 wrote: post #15

Would it make sense to embed such single-purpose network with fixed weights within a LLM before pre-training?

I had that in mind too. What if you handcraft a subnetwork with (some subset of) Turing machine capability? Do those kinds of circuits emerge naturally during training? Can reasoning use them for complex computation?

Re: Smallest transformer that can add two 10-digit numbers

#65

Post by Lerc »

qsera wrote: post #38
Lerc wrote: post #35

Earlier quoted context omitted.

>what exactly is this specific challenge of adding numbers with a transformer model demonstrating/advancing? Well for starters, it puts the lie to the argument that a transformer can only output examples it has seen before. Performing the calculation on examples that haven't been seen demonstrates generalisation of the principles and not regurgitation. While this misconception persists in a large number of people, co…

>it puts the lie to the argument But it does not, right? You can either show it something, or modify the parameters in a way that resemble the result of showing it something. You can claim that the model didn't see the thing, but that would mean nothing, because you are making the same effect with parameter tweaks indirectly.

That's a counterargument to a different thing.

Iteratively measuring loss is a way to reconstruct values. That's trivial to show for a single value If 5 gives you a loss of 2 and 9 gives you a loss of 2 then you know the missing value is 7.

A model with enough parameters can memorise the training set in a similar manner. Technically the model hasn't seen that data by direct input either, but the mechanism provides the means to determine the what the data was. In that respect it is reasonable to say the model has seen the data.

Performing well on examples not in the training set is doing something else.

Any attempt to characterise that as having been seen before negates any distinction between taking in data and reasoning about that data.

Re: Smallest transformer that can add two 10-digit numbers

#66

Post by medi8r »

Filligree wrote: post #30
medi8r wrote: post #11

Earlier quoted context omitted.

A transformer tokenizes input, does a bunch of matmul and relu set up in a certain way. It doesn't get to see the raw number (just like you don't when you look at 1+1 you need visual cortex etc. first.)

So the question is, why do we tokenise it in such a way that it makes everything harder?

There is no encoding that makes everything easier. You trade off maths for general intelligence. Now we are at a point where the LLM can just choose to use a normal calculator anyway!

Re: Smallest transformer that can add two 10-digit numbers

#67

Post by rao-v »

ACCount37 wrote: post #52
i000 wrote: post #15

Would it make sense to embed such single-purpose network with fixed weights within a LLM before pre-training?

Good question. It might work, I considered running a test like this. But it does demand certain things. The subnetwork has to be either crafted as "gradient resistant" or remain frozen. Not all discovered or handcrafted circuits would survive gradient pressure as is. Especially the kind of gradients that fly in early pre-training. It has to be able to interface with native representations that would form in a real LL…

+1 I’ve always had the feeling that training from randomly initialized weights without seeding some substructure is unnecessarily slowing LLM training.

Similarly I’m always surprised that we don’t start by training a small set of layers, stack them and then continue.

Re: Smallest transformer that can add two 10-digit numbers

#68

Post by qsera »

Lerc wrote: post #65
qsera wrote: post #38

Earlier quoted context omitted.

>it puts the lie to the argument But it does not, right? You can either show it something, or modify the parameters in a way that resemble the result of showing it something. You can claim that the model didn't see the thing, but that would mean nothing, because you are making the same effect with parameter tweaks indirectly.

That's a counterargument to a different thing. Iteratively measuring loss is a way to reconstruct values. That's trivial to show for a single value If 5 gives you a loss of 2 and 9 gives you a loss of 2 then you know the missing value is 7. A model with enough parameters can memorise the training set in a similar manner. Technically the model hasn't seen that data by direct input either, but the mechanism provides th…

Yea, because "seeing" is also tweaking the parameters. Which this example is doing manually.

So I don't understand how any one can make the claim that the model as not seen it. Because the internal transformation is similar.

Re: Smallest transformer that can add two 10-digit numbers

#69

Post by ACCount37 »

rao-v wrote: post #67
ACCount37 wrote: post #52

Earlier quoted context omitted.

Good question. It might work, I considered running a test like this. But it does demand certain things. The subnetwork has to be either crafted as "gradient resistant" or remain frozen. Not all discovered or handcrafted circuits would survive gradient pressure as is. Especially the kind of gradients that fly in early pre-training. It has to be able to interface with native representations that would form in a real LL…

+1 I’ve always had the feeling that training from randomly initialized weights without seeding some substructure is unnecessarily slowing LLM training. Similarly I’m always surprised that we don’t start by training a small set of layers, stack them and then continue.

Better-than-random initialization is underexplored, but there are some works in that direction.

One of the main issues is: we don't know how to generate useful computational structure for LLMs - or how to transfer existing structure neatly across architectural variations.

What you describe sounds more like a "progressive growing" approach, which isn't the same, but draws from some similar ideas.

Post reply on HN