Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differential Transformer

arxiv.org

Post reply on HN
Print view View original

41–50 of 188 posts

Re: Differential Transformer

#41

Post by HarHarVeryFunny »

msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

I don't understand either. It seems the general idea is that they calculate attention twice, which due to random initialization might be expected to give two slightly different results. I'd have thought that what these two attention maps would have in common would be the signal, and where they would differ would be noise, so rather than subtracting them (resulting in all noise?!) what you really want is to add (so the common signal gets reinforced) and normalize.

Re: Differential Transformer

#42

Post by om8 »

causal wrote: post #35
digdugdirk wrote: post #33

Is there any way to replicate this with existing models, or are we going to need to wait for models to be trained in this style? I'm imagining a smaller model examining the output tokens of a larger model and metaphorically slapping it on the wrist with a ruler if the output tokens start drifting off topic. Not quite the same, but an entertaining thought nonetheless.

It's a different attention mechanism with a different map setup, so fundamentally a different type of model

Looks like it is a drop in replacement for attention, but models will need to be retrained for this one, yes.

Re: Differential Transformer

#43

Post by blackbear_ »

msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper).

To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax allows the model to predict a weight of exactly zero for some of the values, while keeping a reasonable gradient flowing through.

So the model already knows what is noise, but a single softmax makes it harder to exclude it.

Moreover, with a single softmax the output of all heads is forced to stay in the convex hull of the value vectors, whereas with this variant each head can choose its own lambda, thus shifting the "range" of the outputs outside the convex hull pre-determined by the values. This makes the model as a whole more expressive.

Re: Differential Transformer

#44

Post by spencerchubb »

happypumpkin wrote: post #37
spencerchubb wrote: post #27

Earlier quoted context omitted.

it's not "just" model error during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess large transformer models are really good at approximating their dataset. there is no data on the internet about what LLMs know. and even if there were such data, it would probably become obsolete soon that being said, maybe…

> it would probably become obsolete soon Suppose there are many times more posts about something one generation of LLMs can't do (arithmetic, tic-tac-toe, whatever), than posts about how the next generation of models can do that task successfully. I think this is probably the case. While I doubt it will happen, it would be somewhat funny if training on that text caused a future model to claim it can't do something th…

also presumes that the LLM knows it is an LLM

Re: Differential Transformer

#45

Post by pxdm »

What's the comparison with conventional attention using a more aggressive (lower temperature) softmax? I can imagine that for the multi-needle retrieval test this may also give a performance boost, although at some cost other more creative tasks.

Re: Differential Transformer

#46

Post by Carlseymanh »

HarHarVeryFunny wrote: post #41
msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

I don't understand either. It seems the general idea is that they calculate attention twice, which due to random initialization might be expected to give two slightly different results. I'd have thought that what these two attention maps would have in common would be the signal, and where they would differ would be noise, so rather than subtracting them (resulting in all noise?!) what you really want is to add (so th…

I think there might be some communalities with system engineering, where you subtract the output from the input in order to get a control signal that steers the plant to the target values. I too fail to see how that would be supposed to work in practice.

Re: Differential Transformer

#47

Post by magicalhippo »

patcon wrote: post #34

I wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between

> Surely there's a trade-off...

For one, speed and memory. They have twice as many Q and K weights in the attention blocks, leading to a ~10% reduction in throughput on their H100 (table 7 in appendix A).

Re: Differential Transformer

#48

Post by bionhoward »

digdugdirk wrote: post #33

Is there any way to replicate this with existing models, or are we going to need to wait for models to be trained in this style? I'm imagining a smaller model examining the output tokens of a larger model and metaphorically slapping it on the wrist with a ruler if the output tokens start drifting off topic. Not quite the same, but an entertaining thought nonetheless.

Yes, I believe this is possible, you could clone weights of one or more existing models and fine tune them in groups with different random seeds for noise/drop to produce reasonable outputs under a differential transformer decoding scheme whereby tokens with disagreement receive more attention (surprisal analysis)

Re: Differential Transformer

#49

Post by freeqaz »

blackbear_ wrote: post #43
msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

I'm able to follow most of what you're saying. It's unclear to me what "convex hull" means though.

Also, where is each softmax happening here? For each attention head?

Re: Differential Transformer

#50

Post by timcobb »

viraptor wrote: post #40
ToValueFunfetti wrote: post #36

Earlier quoted context omitted.

>Whether that sequence matches any kind of objective truth is orthogonal to how models work. Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.

It matches the training data. Whether the training data matches truth (and whether it's correctly understood - sarcasm included) is a completely separate thing. > The training does not perfectly align the model with truth, but 'orthogonal' Nitpicky, but the more dimensions you have, the easier it is for almost everything to be orthogonal. ( https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dime... ) That's why…

Isn't this the same thing that happens when you train a human on truths vs falsehoods?
Post reply on HN