Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differential Transformer

arxiv.org

Post reply on HN
Print view View original

181–188 of 188 posts

Re: Differential Transformer

#181

Post by qz94 »

Imnimo wrote: post #108

I feel like I'm missing a key insight here. I understand the problem that regular softmax attention struggles to approach assigning zero attention to irrelevant stuff. And I get that having this subtraction formula makes it possible to assign exactly (or near) zero attention weight without having crazy outlier activations. But it seems like it also makes it very easy to have negative attention weight (which is equiva…

It does sound like we're hindering the model a bit by allowing negative weights to exist instead of sending them through, say, a ReLU. But, dealing with this might be an easier problem than you think for the model.

In the first diagram with the attention weights, there actually are some negative scores in the noise section. But, the attention to that section is very small anyway. All the second attention map needs to do is predict the noise in the first one -- a task that can be done very accurately, because it has full access to the input of the first.

To refer back to their real-world comparison, noise-canceling headphones have access to what your ear hears through a microphone, so they can output exactly the right cancellation signal. Similarly, the second attention map knows what's being input into the first one, so it can output a corresponding cancellation signal. It's not perfect -- just as noise-canceling headphones aren't perfect -- but it still gets you 99% of the way there, which is enough to boost performance.

Re: Differential Transformer

#182

Post by chessgecko »

qz94 wrote: post #181
Imnimo wrote: post #108

I feel like I'm missing a key insight here. I understand the problem that regular softmax attention struggles to approach assigning zero attention to irrelevant stuff. And I get that having this subtraction formula makes it possible to assign exactly (or near) zero attention weight without having crazy outlier activations. But it seems like it also makes it very easy to have negative attention weight (which is equiva…

It does sound like we're hindering the model a bit by allowing negative weights to exist instead of sending them through, say, a ReLU. But, dealing with this might be an easier problem than you think for the model. In the first diagram with the attention weights, there actually are some negative scores in the noise section. But, the attention to that section is very small anyway. All the second attention map needs to…

Are you one of the authors?

Re: Differential Transformer

#183

Post by ykonstant »

Vecr wrote: post #96
spencerchubb wrote: post #44

Earlier quoted context omitted.

also presumes that the LLM knows it is an LLM

They're generally fine tuned not to. I'm not sure how long that will hold though.

- Are you an LLM?

- As a Large Language Model, I am fine tuned to be unable to answer this question.

Re: Differential Transformer

#184

Post by radarsat1 »

seanhunter wrote: post #179
radarsat1 wrote: post #170

Earlier quoted context omitted.

Often see this argument but it doesn't hold water for me. What we call hallucination is usually when the model says something confidently wrong. Yes the sampling procedure is nondeterministic but this is unrelated to hallucinations. The model can generate a distribution to sample with very little weight on the "wrong" output and then this should be ignored by procedures like top-k sampling. The fact that this doesn't…

Yes exactly. It seems intuitive that the model could generate a better distribution and thus cure hallucination but that doesn't actually match what the model does. The model doesn't sample a probability distribution of individual "facts"[1] it samples a probability distribution of tokens which are generally parts of words, bits of punctuation etc. That we get "facts" out of it which may even be wrong in the first pl…

Agreed. I have a loose idea that hallucination is related to training to maximize the probability of individual tokens while ignoring the joint sequence probability, which is along the lines of what you are saying -- it is not trained to output the most probable final sequence, so it gets stuck in the "wrong place" half way through.

Re: Differential Transformer

#185

Post by godeldirac »

aDyslecticCrow wrote: post #172
f38zf5vdt wrote: post #155

Earlier quoted context omitted.

There is a downside, every attention layer has to effectively compute attention twice (run scaled_dot_product_attention). As scaled_dot_product_attention is usually one of the most expensive operations in training and inference of a model, it seems like networks using this may be substantially slow and perhaps should considered against larger networks with more attention layers. https://github.com/microsoft/unilm/blo…

In Big-O notation, O(2n) = O(n). Two times slower is actually not that much. If this slowdown results in better inference in the same number of training rounds or better-tuned weights with fewer redundant features, that can be a very worthwhile sacrifice. It's also a complex optimization problem, not just about computing. Two times, the parameters take more than two times the time to tune and two times the working me…

Glad to see your ideas here. Could you clarify a point to me? The W matrix in the paper is d_model x 2d. Does this mean a differential attention model will double the W matrix of a standard attention model, which is d_model x d? E g. Suppose llama3 has W of 8192 x 1024, does the diffattn model of the same architecture have W of 8192 x (1024 x 2)?

Re: Differential Transformer

#186

Post by anticensor »

viraptor wrote: post #20
AnimalMuppet wrote: post #17

Earlier quoted context omitted.

I'm pretty sure there's something I don't understand, but: Doesn't an LLM pick the "most probable next symbol" (or, depending on temperature, one of the most probable next symbols)? To do that, doesn't it have to have some idea of what the probability is? Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer?

> Doesn't an LLM pick the "most probable next symbol" Yes, but that very rarely matters. (Almost never when it's brought up in discussions) > Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer? A low probability doesn't necessarily mean something's incorrect. Responding to your question in French would also have very low prob…

> Responding to your question in French would also have very low probability, even if it's correct.

It's actually a common utterance in Paris.

Re: Differential Transformer

#187

Post by mik09 »

blackbear_ wrote: post #43
msoad wrote: post #7

Like most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?

With a single softmax you cannot predict exactly 0, but only very small numbers. When you have a large number of values to add up, this "poisons" the output with a lot of irrelevant stuff (the noise mentioned in the paper). To make things worse, low attention values will have very low gradient, thus needing a lot of weight updates to undo that kind of mistakes. On the other hand, subtracting the output of two softmax…

is this the same reason why fp4 with more parameters beats fp16 with less?
Post reply on HN